Reliable messages are ultimately consistent (asynchronous guaranteed)



Consistency design is an important problem in distributed systems. If a system uses multiple data systems to store and read data at the same time, it is necessary to design a consistent definition to meet the functional requirements. If the results of different data subsystems are inconsistent, not only users may be confused, but also more serious data problems or system errors may be caused. There are many levels of consistency that can be applied to different business scenarios. For industries such as finance, which require higher data consistency, traditional transactions can provide higher consistency assurance. It is also acceptable to sacrifice certain strong consistency for better user experience in scenarios with high performance and availability requirements such as distributed systems.

I wrote a note on TCC transactions before, and also understood the causes of distributed transactions and some solutions. This time, I would like to summarize the relevant contents of the design ideas of the final consistency scheme;


The concept of message sending consistency: it refers to the consistency between the business action that generates the message and the message sending.

In other words, if the business operation is successful, the message generated by the business operation must be delivered successfully, otherwise the message will be lost

The final consistency can be used in the following functional scenarios:

  • Accounting asynchronous bookkeeping of corresponding payment system
  • Ordinary points account add points service

In other words, data systems with final consistency usually do not require a rollback when data operations fail. The user or system log will know that the operation failed, but data inconsistencies will not be automatically fixed until another successful operation.

PS: there will be some doubts from my partners. When I execute the program, the code error is reported, which leads to the failure of the final consistent solution. What should I do??? Do you have a lot of children???

If it is a code error, it indicates that there is a problem with your business code, rather than the pot of the final consistency scheme.

Implementation process

Finally, the consistency can be realized by means of message middleware, message queue and other tools;

We will introduce a reliable message service system based on rabbitmq to complete the transaction execution. The specific flow chart is as follows:
Reliable messages are ultimately consistent (asynchronous guaranteed)

  1. The active application sends the message to the message middleware, and the message status is marked as “to be confirmed”;
  2. After receiving the message, the message middleware persisted the message to the message store, but did not post the message to the passive application;
  3. Message middleware returns message persistence results (success / failure), and the application of the active party judges how to process business operations based on the returned results

    • Failure: give up the business operation processing and end (return the failure result to the upper layer if necessary);
    • Success: execute business operation processing;
  4. After the business operation is completed, the business operation result (success / failure) is sent to the message middleware;
  5. After receiving the business operation results, the message middleware processes them according to the business results;

    • Failed: delete the message in the message store, end;
    • Success: update the message status in the message store to “to be sent (can be sent)”, followed by message delivery;
  6. The passive application monitors and receives messages in the “to be sent” status and executes business processing;
  7. After business processing, send ack to message middleware to confirm that the message has been received (message) middleware will delete the message from the queue)

In addition to the above processes, the message system should also provide ackmsg message confirmation service and message status query service.

Exception handling process

Reliable messages are ultimately consistent (asynchronous guaranteed)

Angle of active party

Reliable messages are ultimately consistent (asynchronous guaranteed)

Reliable messages are ultimately consistent (asynchronous guaranteed)

From the perspective of Middleware

Reliable messages are ultimately consistent (asynchronous guaranteed)
Reliable messages are ultimately consistent (asynchronous guaranteed)

Summary and treatment of abnormal conditions

Reliable messages are ultimately consistent (asynchronous guaranteed)

Scheme implementation

Reliable messages are ultimately consistent (asynchronous guaranteed)

Composition of message system

  1. Message service subsystem:

Is the most important subsystem, it receives and stores the pre sent message, and provides further confirmation function. Generally, the following interface services need to be implemented.

  • Store pre sent message (active party application system)
  • Confirm and send message (active party application system)
  • Query status confirmation timeout message (message status confirmation subsystem)
  • Confirmation message has been successfully consumed (passive application system)
  • Message recovery timeout for message recovery subsystem
  1. Message management subsystem:

It provides a visual management interface to query and manage the data in reliable message service system. For example, you can view the dead message and resend it manually through the interface

  1. Message status confirmation subsystem:

Provides the handling of abnormal conditions. When the message service subsystem receives and saves the pre sent message, but it does not receive the confirmation sending message due to abnormal conditions, the message cannot be kept in the database all the time. In this case, it is necessary for the message status confirmation subsystem to retrieve the timeout data to be confirmed regularly and call the business query interface in the active application system for verification and confirmation. According to the check result, it is decided whether to send the message or delete the data.

  1. Message recovery subsystem:

If the message data has received the business confirmation, the message must be sent to MQ, and be consumed by the consumer successfully. It must not be lost. The message recovery subsystem periodically fetches the timeout messages whose status is “sending” but has not been confirmed by consumption, and then resends them.

  1. Real time message service subsystem (MQ)

The consumer listener program receives the MQ message. After successful processing, it calls the interface of the message service subsystem and confirms that the message has been successfully consumed and can be deleted.

Overall process

  1. When the user places an order, the active party sends the message to the message service subsystem in advance.
  2. The message service subsystem stores pre sent messages.
  3. Returns the result of storing the pre sent message.
  4. If the result returned in step 3 is successful, the business operation is executed; otherwise, it is not executed.
  5. After the business operation is successful, the message service subsystem is called to confirm and send the message.
  6. Sends a pre sent message stored in the message service library and updates the status of the message to sent (but not consumed).
  7. Message middleware sends messages to consumer applications.
  8. The consumer application calls the passive application service.
  9. The passive application returns the result to the consumer application.
  10. The consumer application acks the message to the message middleware and confirms to the message service subsystem that the message is successfully consumed,
  11. Let the message service subsystem delete the message or set the status to successfully consumed.
  12. If a message has been sent successfully by the subsystem, it is necessary to check whether the message has been sent by the application side
  13. If the business data is processed successfully, the confirmation is called again and the message is sent, which is step 6.
  14. If the business data fails to be processed, the message service subsystem is called to delete the message data.

Thanks for watching

This article is the note arrangement in the learning process. If there is something wrong, please contact me to correct it in time, so as to avoid misleading the children’s shoes. Thank you for your patience. I hope this article will help you plan to write a demo in hyperf later. If you are interested, please pay attention to my GitHub.

Recommended Today

Queue chain storage

Linked list realizes queue Create three files: queuelinked. H, queuelinked. C, queuelinkedtest. C queueLinked.h #ifndef QUEUE_LINKED_H_ #define QUEUE_LINKED_H_ #ifdef __GNUC__ #define DEPRECATED __attribute__( (deprecated) ) #elif defined(_MSC_VER) #define DEPRECATED __declspec( deprecated ) #else #define DEPRECATED #endif #ifndef PTOI #define PTOI( p ) ((int32_t)(int64_t)(p)) #endif #ifndef ITOP #define ITOP( i ) ((void *)(int64_t)(i)) #endif #define ADT […]