Rocketmq master-slave read-write separation mechanism

Time:2020-1-24

The WeChat public is “back-end”, focusing on the sharing of back-end technologies: Java, Golang, WEB framework, distributed middleware, service governance, and so on.

Generally speaking, in the architecture of selecting master-slave backup to achieve high availability, there will be a read-write separation mechanism, such as MySQL read-write separation. The client can read data from the master-slave server, but the client can only write data through the master server.

Rocketmq’s read-write separation mechanism is not quite consistent with the above description. Rocketmq has its own set of read-write separation logic, which determines the message accumulation of the main server to determine whether consumers pull messages from the slave server for consumption.

The value that determines whether the consumer pulls message consumption from the server exists in the getmessageresult class:

org.apache.rocketmq.store.GetMessageResult:

private boolean suggestPullingFromSlave = false;

The default value is false, that is, the default consumer will not consume the slave server, which can be changed by the following logic:

org.apache.rocketmq.store.DefaultMessageStore#getMessage:

long diff = maxOffsetPy - maxPhyOffsetPulling;
long memory = (long) (StoreUtil.TOTAL_PHYSICAL_MEMORY_SIZE
    * (this.messageStoreConfig.getAccessMessageInMemoryMaxRatio() / 100.0));
getResult.setSuggestPullingFromSlave(diff > memory);

Where maxoffsetpy is the current maximum physical offset, maxphyoffsetpulling is the current maximum physical offset of the message pull, their difference can represent the message stack, total﹣ physical﹣ memory﹣ size is the current system physical memory, the default value of accessmessageinmemorymaxratio is 40, and the above logic can calculate whether the current message stack is greater than 40% of the physical memory , set suggestpullingfromslave to true if greater than.

Next, the parameter value will work in the message pull logic:

org.apache.rocketmq.broker.processor.PullMessageProcessor#processRequest:

if (getMessageResult.isSuggestPullingFromSlave()) {
  responseHeader.setSuggestWhichBrokerId(subscriptionGroupConfig.getWhichBrokerWhenConsumeSlowly());
} else {
  responseHeader.setSuggestWhichBrokerId(MixAll.MASTER_ID);
}

switch (this.brokerController.getMessageStoreConfig().getBrokerRole()) {
  case ASYNC_MASTER:
  case SYNC_MASTER:
    break;
  case SLAVE:
    if (!this.brokerController.getBrokerConfig().isSlaveReadEnable()) {
      response.setCode(ResponseCode.PULL_RETRY_IMMEDIATELY);
      responseHeader.setSuggestWhichBrokerId(MixAll.MASTER_ID);
    }
    break;
}

if (this.brokerController.getBrokerConfig().isSlaveReadEnable()) {
  // consume too slow ,redirect to another machine
  if (getMessageResult.isSuggestPullingFromSlave()) {
    responseHeader.setSuggestWhichBrokerId(subscriptionGroupConfig.getWhichBrokerWhenConsumeSlowly());
  }
  // consume ok
  else {
    responseHeader.setSuggestWhichBrokerId(subscriptionGroupConfig.getBrokerId());
  }
} else {
  responseHeader.setSuggestWhichBrokerId(MixAll.MASTER_ID);
}

If it is found that the message heap of the primary server exceeds 40% of the physical memory, suggestwhichbrokerid is set to the slave server broker ID.

There is also a slavereadenable value to determine whether messages can be pulled from the server:

  1. If slavereadenable = true and the heap has exceeded 40% of the physical memory, it is recommended to pull messages from the slave server, otherwise, pull messages from the master server;
  2. If slavedeadenable = false, the message can only be pulled from the primary server.

org.apache.rocketmq.client.impl.consumer.PullAPIWrapper#updatePullFromWhichNode:

public void updatePullFromWhichNode(final MessageQueue mq, final long brokerId) {
    AtomicLong suggest = this.pullFromWhichNodeTable.get(mq);
    if (null == suggest) {
        this.pullFromWhichNodeTable.put(mq, new AtomicLong(brokerId));
    } else {
        suggest.set(brokerId);
    }
}

When the consumer receives the data from the pull response, it will cache the broker ID of the next proposed pull. The next time the pull message is pulled, the pull broker ID will be taken from the pullfromwhichnodetable.

Rocketmq master-slave read-write separation mechanism

Pay attention to public key reply key word “backend” free collection of backend development gift!

Recommended Today

Spring cloud Alibaba architecture practice

Making Java can’t bypass spring. Using java for microservice development can’t bypass spring cloud. With the restart of Dubbo and its maintenance by the Apache open source community, the Dubbo ecosystem has become more and more perfect. Although it’s not appropriate to compare spring cloud with Dubbo, many friends will struggle to choose Dubbo or […]