Introduction to message middleware

Time:2021-4-21

preface

This article does not involve the code, just standing on the theoretical point of view to think, sort out, a clearer understanding of the message queue.

What is message middleware

There is no standard definition. Generally speaking, message middleware belongs to a subsystem of distributed system, which focuses on the sending and receiving of data, and integrates other subsystems of distributed system by using efficient and reliable asynchronous message passing mechanism.

What are his application scenarios

  1. Asynchronous:
    For example, there is an e-commerce platform to place an order: after the user generates an order, click the payment button to make payment. The logic of this operation for the back end should be as follows:
    Request user service for account deduction —-> Call the order service to modify the order status of the order table —-> Call inventory service to deduct inventory —-> Call points service to increase points for users —-> Back to front end “Payment successful” prompt message.
    There is no problem with this logic, but when you think about it carefully, how long will it take for such a process to run down when the server is full of requests? If you are a user, can you bear it? Now let’s optimize the process. First of all, the user’s request is directly to the user service. When the payment interface is successfully called, it means that the user’s operation is OK and the payment is also successful. In fact, for subsequent operations such as modifying order status, reducing inventory and increasing points, we can execute them asynchronously. There is no need to queue up one by one, because it triggers One point of these operations is whether the payment is successful.
  2. Decoupling:
    In the same example above, when the logic has a problem in the step of adding points, or the interface of the points service has changed in the iteration of the new version, there will be an exception when calling the points service. Although the user’s payment is successful, the front-end response is still “payment failure”. In fact, this is unreasonable. Points can be added later, but it’s not reasonable We must ensure that the current payment business of the user is normal, so we can separate these unrelated businesses, do not make mandatory calls to the interface, decouple but also connect them. The communication between services carries out message transmission. When the user pays successfully, a message of successful payment is sent to the queue, and other services carry out subsequent business processing after listening to the message This ensures a small impact on payment services.
  3. Flow peak clipping:
    In fact, this scenario is still very good. Suppose there is a second kill function now. What we need to do on the server is:
    1. Ensure that the service of high-speed parallel distribution is still available;
    2. Ensure that the goods will not oversold;
    A careful analysis: the service of second kill is instantaneous request. When the second kill is not started, the request is also normal. When it is on time, tens of thousands or even hundreds of billions of requests suddenly come in, but only a few hundred or thousands of goods can be seized in the end. Then why can’t we turn the instantaneous request into a serial request according to the first come first served principle? The fast network speed Fast first into the queue to determine the inventory of goods for payment, behind the hand slow on the second kill failed.

Of course, there are many application scenarios of message queuing, such as distributed transaction, delayed order processing and other advanced usage, but the most commonly used scenarios are the above three.

Since the message queue is so good, we’d better use it; In fact, no one is perfect. There are both advantages and disadvantages. Let’s summarize his advantages and disadvantages

  1. As the technical complexity of the system becomes higher, the reliability consumption of messages should be considered to solve the problems of repeated consumption and message loss.
  2. Can not achieve real-time, if the system for real-time data requirements are very high, then the message queue is obviously a little redundant.

What is rabbitmq

What is rabbitmq? Learn from Baidu:

Rabbitmq is an open source message broker software that implements advanced message queuing protocol (AMQP). Rabbitmq server is written in Erlang language, while cluster and fail over are built on the framework of open telecommunication platform. All major programming languages have client libraries that communicate with agent interfaces.

So what is AMQP? For better understanding, I will not apply Baidu, simply speaking, it is an open standard of ampq application layer protocol , It formulates a standard model of message server, which can be implemented by different middleware. It is designed for message oriented middleware. Based on this protocol, the client and the message middleware can deliver messages without being affected by the client / Middleware is limited by different products and development languages. The goal is to achieve a standard message middleware technology widely used in the whole industry, so as to reduce the cost of enterprise and system integration, and provide industrial level integration services to the public. The main implementation is rabbitmq.

In a simple way, we can explain rabbitmq: it is an intermediate platform for logical interaction between systems in a decoupled and asynchronous wayErlangLanguage implementation, there is no strong dependence between systems, just need to ensure that the intermediate platform can successfully deliver messages.

Of course, message middleware is not only rabbitmq, but also others, such as rocketmq, Kafka, ActiveMQ and so on…..
So what is rabbitmq’s position in comparison? Look at the picture below
消息中间件对比图

Technology selection

In practical work, I think that the selection of MQ technologies is as follows:

  • The amount of user access is within the range of ActiveMQ, and it is really mainly based on decoupling and asynchronous. ActiveMQ can be considered, and it is also close to Java
    Engineers are used to it, but ActiveMQ has stopped maintenance now, and ActiveMQ concurrency is not high, so we can consider using it when the business volume is certain.

  • As a pure pedigree of message middleware, rabbitmq has a perfect combination of advanced message protocol AMQP. It plays an irreplaceable role in message middleware, but Erlang
    Language prevents us from further research and control. For the company, the underlying technology can’t be controlled, but it’s really open source, with relatively stable support and high activity.

  • You can use rocketmq if you have absolute confidence in your company’s technical strength. However, rocketmq was born relatively late and the update iteration is very fast, which means that you may encounter many pitfalls in the process of using it. Therefore, if your company’s Java technology is not very strong, it is not recommended to use it.

  • For small and medium-sized companies, the technical strength is relatively general, and the technical challenges are not particularly high. ActiveMQ and rabbitmq are good choices. For large companies, with strong infrastructure R & D strength, rocketmq is a good choice

  • For real-time computing, log collection and other scenarios in the field of big data, Kafka is the industry standard, absolutely no problem. The community activity is very high, which is almost the world’s de facto specification in this field.

  • From the performance point of view, the message middleware using file system (Kafka, rokcetmq) has the best performance, so the message middleware based on file system storage is the development trend. (in terms of storage mode and efficiency, file system > kV storage > relational database)

There is no work for nothing, no gain for nothing. If you are more greedy than others, please be more attentive than others.