Java rocketmq — Generation of messages (ordinary messages)

Time:2019-7-27

Preface

Several lines of code closely related to message sending:

1. DefaultMQProducer producer = new DefaultMQProducer(“ProducerGroupName”);

2. producer.start();

3. Message msg = new Message(…)

4. SendResult sendResult = producer.send(msg);

5. producer.shutdown();

So what’s behind these lines of code when they’re executed?

First, DefaultMQ Producer.start


@Override
public void start() throws MQClientException {
this.defaultMQProducerImpl.start();
}

Called the implementation class of the default generated message — DefaultMQProducerImpl

Calling defaultMQProducerImpl. start () method, DefaultMQProducerImpl. start () initializes the MQClientInstance instance object, and MQClientInstance instance object calls its own start method to start some services, such as pulling the message service PullMessageService. Start (), and starting the load balancing service RebalanceService.S. Tart (), such as network communication service MQClientAPIImpl. Start ()

In addition, information related to production messages, such as registering produceGroup, new TopicPublishInfo object and using default TopicKey as key value, will be executed to form a key-value pair and stored in TopicPublishInfoTable of DefaultMQ Producer Impl.

After efaultMQProducerImpl. start (), the acquired MQClientInstance instance object will call the sendHeartbeat ToAllBroker () method and continuously send heartbeat packets to the broker. Yin’b can use the following figure to roughly describe the DefaultMQProducerImpl. start () process:

The three parts in the figure above cover:

1.1 Initialization of MQClientInstance

A client can only produce one instance object of MQClientInstance, using factory mode and singleton mode. The MQClientInstance. start () method starts some services with the following source code:


public void start() throws MQClientException {
synchronized (this) {
switch (this.serviceState) {
case CREATE_JUST:
this.serviceState = ServiceState.START_FAILED;
// If not specified,looking address from name server
if (null == this.clientConfig.getNamesrvAddr()) {
this.mQClientAPIImpl.fetchNameServerAddr();
}
// Start request-response channel
this.mQClientAPIImpl.start();
// Start various schedule tasks
this.startScheduledTask();
// Start pull service
this.pullMessageService.start();
// Start rebalance service
this.rebalanceService.start();
// Start push service
this.defaultMQProducer.getDefaultMQProducerImpl().start(false);
log.info("the client factory [{}] start OK", this.clientId);
this.serviceState = ServiceState.RUNNING;
break;
case RUNNING:
break;
case SHUTDOWN_ALREADY:
break;
case START_FAILED:
throw new MQClientException("The Factory object[" + this.getClientId() + "] has been created before, and failed.", null);
default:
break;
}
}
}

1.2 Register producer

This process registers the current producer object into the producerTable of the MQClientInstance instance object. There can only be one instance of a producer group in a JVM (a client). MQClientInstance operates producer table in the following ways:

  • — selectProducer
  • — updateTopicRouteInfoFromNameServer
  • — prepareHeartbeatData
  • — isNeedUpdateTopicRouteInfo
  • — shutdown

Note:

According to different clientId, MQClientManager will give different MQClientInstance.

According to different groups, MQClientInstance will give different MQProducers and MQConsumers

1.3 Adding routing to routing information table

Topic Publish InfoTable definition:


public class DefaultMQProducerImpl implements MQProducerInner {
private final Logger log = ClientLogger.getLog();
private final Random random = new Random();
private final DefaultMQProducer defaultMQProducer;
private final ConcurrentMap<String/* topic */, TopicPublishInfo> topicPublishInfoTable = new ConcurrentHashMap<String, TopicPublishInfo>();

It is a Map data structure with topic as key. DefaultMQProducerImpl. start () creates a TopicPublishInfo with key = MixAll. DEFAULT_TOPIC by default and stores it in TopicPublishInfoTable.

1.4 Send Heart Packet

When MQClientInstance sends heartbeat packets to brokers, it calls sendHeartbeat ToAllBroker (), and gets all broker addresses from the broker AddrTable of the MQClientInstance instance object to send heartbeat packets to these brokers.

SendHeartbeat ToAllBroker involves the prepareHeartbeat Data () method, which generates heartbeat Data data, which acts as the body of a heartbeat packet when sending a heartbeat packet. Part of the code related to producer is as follows:


// Producer
for (Map.Entry<String/* group */, MQProducerInner> entry : this.producerTable.entrySet()) {
MQProducerInner impl = entry.getValue();
if (impl != null) {
ProducerData producerData = new ProducerData();
producerData.setGroupName(entry.getKey());
heartbeatData.getProducerDataSet().add(producerData);
}

SendResult sendResult = producer. send (msg)

DefaultMQProducer. send (msg) is called first, and sendDefaultImpl is then called:


public SendResult send(Message msg,
long timeout) throws MQClientException, RemotingException, MQBrokerException, InterruptedException {
return this.sendDefaultImpl(msg, CommunicationMode.SYNC, null, timeout);
}

What did sendDefaultImpl do?

2.1. Get topicPublishInfo

According to the topic of msg, the corresponding topicPublishInfo is obtained from the topicPublishInfoTable. If not, the routing information is updated and the latest routing information is pulled from the name server. The latest routing information pulled from the name server side is roughly as follows:

First get TopicRouteInfoFromName Server, then topicRouteData2 TopicPublishInfo.

2.2 Select the queue for sending messages

Normal message: By default, select One MessageQueue selects a queue from the message Queue List in Topic Publish Info to send messages. By default, select queue by long polling.

Its mechanism is as follows: normally, queue is selected sequentially for sending; if a node has a timeout, the next time queue is selected, the same broker is skipped. Different queue selection strategies form several production message modes, such as sequential message and transaction message.

Sequential message: Send a group of messages that need ordered consumption to the same queue of the same broker to achieve ordered message. Assuming that the same order number is paid and the refund needs to be placed in the same queue, then you can implement MessageQueue Selector on send and select queue according to the parameter Arg field.


private SendResult sendSelectImpl(
Message msg,
MessageQueueSelector selector,
Object arg,
final CommunicationMode communicationMode,
final SendCallback sendCallback, final long timeout
) throws MQClientException, RemotingException, MQBrokerException, InterruptedException { 。。。}

Transaction message: only when the message is sent successfully and the local operation is executed successfully, can the submission transaction message be sent, the transaction submission is done, the message transmission fails, the rollback message is sent directly, and the rollback is carried out. How to realize this will be analyzed separately in the future.

2.3 Encapsulating Message Body Communication Packet and Sending Packet

First, according to getBrokerName in the acquired MessageQueue, call findBrokerAddressInPublish to get the corresponding broker address for the message. If not, retrieve the address with the new routing information:

brokerAddrTable.get(brokerName).get(MixAll.MASTER_ID)

It is known that all the brokers obtained are master (id = 0)

Then, the message-related information is packaged into a RemotingCommand packet, whose RequestCode. SEND_MESSAGE

According to the acquired broke address, the packet is sent to the corresponding broker, and the default time-out is 3s.

Package headers encapsulating message request packages:


SendMessageRequestHeader requestHeader = new SendMessageRequestHeader();
requestHeader.setProducerGroup(this.defaultMQProducer.getProducerGroup());
requestHeader.setTopic(msg.getTopic());
requestHeader.setDefaultTopic(this.defaultMQProducer.getCreateTopicKey());
requestHeader.setDefaultTopicQueueNums(this.defaultMQProducer.getDefaultTopicQueueNums());
requestHeader.setQueueId(mq.getQueueId());
requestHeader.setSysFlag(sysFlag);
requestHeader.setBornTimestamp(System.currentTimeMillis());
requestHeader.setFlag(msg.getFlag());
requestHeader.setProperties(MessageDecoder.messageProperties2String(msg.getProperties()));
requestHeader.setReconsumeTimes(0);
requestHeader.setUnitMode(this.isUnitMode());
requestHeader.setBatch(msg instanceof MessageBatch);

Send message packages (normal messages default to synchronization):


SendResult sendResult = null;
switch (communicationMode) {
   case SYNC:
  sendResult = this.mQClientFactory.getMQClientAPIImpl().sendMessage(
  brokerAddr,
  mq.getBrokerName(),
   msg,
  requestHeader,
   timeout,
  communicationMode,
  context,
  this);
break;

Processing response packets from broker end:


private SendResult sendMessageSync(
final String addr,
final String brokerName,
final Message msg,
final long timeoutMillis,
final RemotingCommand request
) throws RemotingException, MQBrokerException, InterruptedException {
RemotingCommand response = this.remotingClient.invokeSync(addr, request, timeoutMillis);
assert response != null;
return this.processSendResponse(brokerName, msg, response);
}

After processing the request packet, the broker will store the message to commitLog, and the specific process will be analyzed later.

The above is the whole content of this article. I hope it will be helpful to everyone’s study, and I hope you will support developpaer more.

Recommended Today

SQL exercise 20 – Modeling & Reporting

This blog is used to review and sort out the common topic modeling architecture, analysis oriented architecture and integration topic reports in data warehouse. I have uploaded these reports to GitHub. If you are interested, you can have a lookAddress:https://github.com/nino-laiqiu/TiTanI recorded a relatively complete development process in my hexo blog deployed on GitHub. You can […]