Chapter 3 interprocess communication in microservice architecture




This is a book about microservice architecture design. This is my study note. First, some symbols are explained:

() for supplement, it is generally the content in the book;
[] is the author’s note;

Microservice architecture builds an application into a set of services that must often cooperate to handle various external requests. Service instances are usually processes running on multiple machines, so they must interact using interprocess communication.

At present, there are many inter process communication mechanisms, and the more popular one is rest (using JSON). Choosing the appropriate interprocess communication mechanism is an important architectural decision, which affects the availability of applications.

1. Overview of interprocess communication in microservice architecture

Interprocess communication technologies include: synchronous request / response based, asynchronous message based communication mechanism and so on.

1.1 two dimensions of interaction mode

  • First dimension
    • one-on-one: each client request is processed by a service instance;
    • One to many: each client request is processed by multiple service instances;
  • Second dimension
    • Synchronous mode: the client request requires the server to respond in real time, and the client may be blocked when waiting for the response;
    • Asynchronous mode: the client request will not block the process, and the server response can be non real time;


1.2 types of interaction modes

  • One to one interaction
    • Request / response: a client sends a request to the server and waits for a response; The client expects the server to send a response soon. In a thread based application, the waiting process may cause thread blocking. This approach will lead to tight coupling of services;
    • Asynchronous request / response: the client sends a request to the server, and the server responds to the request asynchronously. The client will not block the thread while waiting, because the server response will not return immediately;
    • One way notification: the request of the client is sent to the server, but the server is not expected to make any response;
  • One to many interaction
    • Publish / subscribe mode: the client publishes notification messages and is subscribed by zero or more interested services;
    • Publish / asynchronous response mode: the client publishes a request message and waits for a response from the service of interest;

1.3 evolution of API

  • Semantic version control: specifies how to use the version number and increment the version in the correct way. It consists of three parts:
    • MAJOR: incompatible modification of API;
    • MINOR: when the API is enhanced for backward compatibility;
    • PATCH: when performing backward compatible error repair;
    • standard:MAJOR.MINOR.PATCH
  • Make minor and backward compatible changes: additional modification, replacement or function enhancement of ADP. It includes:
    • Add optional attributes
    • Add properties to response
    • Add new action
  • Major and backward incompatible versions: when the service needs to support both old and new versions of API for a period of time;

1.4 message format

Message format will affect the efficiency of inter process communication, API availability and evolvability. The use of cross language message formats is particularly important;

  • Text based message format
    • Examples: JSON, XML;
    • Advantages: high readability, self description and good backward compatibility [message recipients only need to select the values they are interested in and ignore others];
    • Disadvantages: the information is lengthy, and parsing the text requires additional performance and efficiency overhead;
  • Binary message format
    • Examples: tars, protocol buffers, Avro;
    • Benefits: IDL (interface description file) with strong type definition is provided to define messages; The compiler generates serialization and deserialization code according to these formats;
    • Disadvantages: we have to adopt the API first method for service design

2. Communication based on synchronous remote procedure call mode

2.1 remote procedure call RPI

It refers to that the client uses synchronous remote procedure call protocol (such as rest) to call services.


Illustration: client business logic callProxy interface, this interface is provided byRemote procedure call proxyAdapter class implementation.Remote procedure call proxySend a request to the server byRemote procedure call serverThe adapter class calls the business logic of the service through the interface. It then sends the recovery backRemote procedure call proxy, the agent returns the result to the business logic of the client.

  • Proxy interface: it usually encapsulates the underlying communication protocol, such as rest and grpc described below.

2.2 characteristics, advantages and disadvantages of rest communication protocol

Rest is an inter process communication mechanism that (always) uses the HTTP protocol.


  • Rest uses HTTP verbs to operate resources, and uses URLs to refer to these resources;
  • Resources are usually in the form of XML documents or JSON objects, or other formats (binary, etc.);
  • The mature models of rest are: there are 4 levels (P71);
  • REST API: the most popular rest IDL is the open API specification, which is developed from the swagger open source project;
  • Challenges of rest API
    • The challenge of getting multiple resources in one request: refers to how to retrieve multiple related objects in a single request;
    • The challenge of mapping operations to HTTP verbs: an HTTP verb may correspond to a variety of methods, such as put request to update the order, including canceling the order, modifying the order, etc;


  • Very simple, you are familiar with it;
  • You can use browser extensions (such as postman plug-in) or command lines such as curl to test HTTP APIs;
  • Directly support the communication of request / response mode;
  • HTTP is firewall friendly;
  • No intermediate agent is needed to simplify the system architecture;


  • Only request / response communication is supported;
  • May result in reduced availability. Since clients and services communicate directly without agents to buffer messages, they must remain online during rest API calls;
  • The client must know the location (URL) of the service instance. The client must use the so-calledService discovery mechanismTo locate the service instance;
  • Acquiring multiple resources in a single request is challenging;
  • Sometimes it is difficult to map multiple update operations to HTTP verbs;

2.3 characteristics, advantages and disadvantages of grpc communication protocol

Grpc is a framework for writing cross language client and server. It is a binary protocol.


  • Grpc API consists of one or more service and request / response message definitions;
  • The service definition is similar to the java interface and is a collection of strongly typed methods;
  • Using protocol buffers as the message format is an efficient and compact binary format and a tag format;
    • Therefore, grpc enables the API to change while maintaining backward compatibility;


  • Designing an API with complex update operations is very simple;
  • It has an efficient and compact inter process communication mechanism, especially when exchanging a large number of messages;
  • Support the use of two-way flow message mode in remote procedure call and message transmission;
  • The interoperability between the client and the server written in various languages is realized;


  • Compared with rest / JSON based API mechanism, JavaScript client needs to do more work to use grpc based API;
  • The old firewall may not support HTTP / 2;

2.4 local fault risk under synchronous communication

The client and server are independent processes, and the server may not be able to respond to the client’s request in a limited time.

Illustration: when the order service does not respond, the orderserviceproxy will block indefinitely and wait for a response. It will consume time, waste threads and other resources. Finally, the API gateway will consume resources, unable to process requests, and the entire API is unavailable.

The solution is

  • The remote procedure call proxy (such as orderserviceproxy) must have the ability to correctly handle unresponsive services;
  • You need to decide how to recover from a failed remote service;

2.5 ideas and methods for solving local faults

  • Develop reliable remote procedure call proxy: the method described by Netflix can include a combination of the following mechanisms;
    • Network Timeout : when waiting for the response to the request, do not make unlimited blocking, but set a timeout to ensure that resources will not be wasted on unresponsive requests all the time;
    • Limit the number of client requests to the server: set an upper limit for the requests that the client can initiate to a specific service. If the request reaches the upper limit, the request will fail immediately;
    • Circuit breaker mode: monitor the number of successes and failures of requests sent by the client. If the failure ratio exceeds a certain threshold, start the circuit breaker to make subsequent calls fail immediately. If a large number of requests fail, the called service is unavailable. After a certain time, the client continues to try. If the call is successful, remove the circuit breaker;
  • Recovery from service failure
    • The service can simply return an error to its client;
    • Return alternate values (such as default values or cached responses);

2.6 application layer service discovery mode

The service and its customers interact directly with the service registry;


  • The service instance uses the service registry to register its network location. The client first calls the service by querying the service registry to obtain the list of service instances, and then it sends a request to one of the instances;
  • This service discovery is a combination of the following two modes:
    • Self registration mode: the service instance registers itself with the service registry;
      • You can provide an operation status check URL (“heartbeat” function. The service registry calls the endpoint regularly to verify whether the service instance is normal and can be used to process requests);
    • Client discovery mode: the client retrieves the list of available service instances from the service registry and performs load balancing between them;
      • In order to improve performance, the client may cache service instances;
  • The industry has Eureka component developed by Netflix and a highly available service registry; Spring cloud developed by pivot
    Make the use of relevant components very simple;

2.7 platform layer service discovery mode

Handle service discovery by deploying infrastructure;


  • The deployment platform includes a service registry for tracking the IP address of deployed services;
  • The deployment platform provides DNS name, virtual IP (VIP) address and DNS name resolved to VIP address for each service;
  • This service discovery is a combination of the following two modes:
    • Third party registration mode: a third party (called the registration server) is responsible for processing the registration, instead of the service itself, registering itself in the service registry first;
    • Server discovery mode: the client sends a request to the DNS name, the request for the DNS name is resolved to the router, and the router queries the service registry and load balances the request;
  • There are docker and kubernetes in the industry, both of which have built-in service registry and service discovery mechanism;

3. Communication based on asynchronous message mode

When using message mechanism, the communication between services is completed by asynchronous message exchange.

Applications based on message mechanism usually adopt message broker; Another option is to use an agent-free architecture.

3.1 about messages

The message consists of a message header and a message body;

  • Message header
    • title: name and value pair;
    • Message ID: unique ID of the messaging base;
    • Return address: specify the message channel for sending reply;
  • Message body: data sent in text or binary format;
    • file: generic message containing data. The recipient decides how to interpret it. Replying to a command message is an application scenario of a document message;
    • command: a message equivalent to an RPC request. It specifies the operation to be called and its parameters;
    • event: indicates that an important event has occurred on the sender side. Events are usually domain events, indicating the state change of domain objects;

3.2 about message channel

There are two types of message channels:

  • Point to point channel
    • Delivering a message to a consumer reading from the channel;
    • For example, command messages are usually sent through point-to-point channels;
  • Publish subscribe channel
    • Send a message to all subscribers;
    • For example, event messages are usually sent through the publish subscribe channel;

3.3 use message mechanism to realize interaction mode

The message mechanisms of the following four interaction modes are introduced:

  • Implement one-way notification
    • The client sends a message (usually a command message) to the point-to-point channel owned by the service;
    • The service subscribes to the channel and processes the message, but the service will not send back a reply;
  • Implement publish / subscribe
    • The client publishes the message to the publish / subscribe channel read by multiple receivers;
    • The service that publishes domain events has its own publish / subscribe channel, and the channel name is often derived from the domain class;
    • For example, the order service publishes the order event to the order channel; The delivery service publishes delivery events to the delivery channel;
  • Implement publish / asynchronous response
    • A more advanced interaction mode, which combines the elements of publish / subscribe and request / response;
    • The client publishes a message and specifies the reply channel in the message header. This channel is also a publish subscribe channel;
    • The consumer writes the reply message containing the correlation ID into the reply channel;
    • The client collects the response by using the correlation ID, so as to match the reply message with the request;
  • Implement request / response and asynchronous request / response
    • When the client sends a request, the service will send back a reply;
    • The client must tell the service where to send the reply message, and must match the reply message with the request;
      • That is, the client sends a command message with a reply channel header. The server writes a reply message to the reply channel, which contains the same correlation ID as the message identifier. The client uses the correlation ID to match the reply message with the request;
    • Because the client and server communicate using message mechanism, the interaction is asynchronous in nature;
    • The working principle diagram is as follows:


3.4 create API Specification for service API based on message mechanism

The asynchronous API specification of the service must specify the name of the message channel, the message type exchanged through each channel and its format.


  • The asynchronous API of the service includes operations called by the client and events published by the service;
  • (record asynchronous operations) you can use one of the following two different interaction methods to invoke the operation of the service:
    • Request / asynchronous responsive API: including the command message channel of the server, the specific type and format of the command message accepted by the service, and the type and format of the reply message sent by the service;
    • One way notification API: including the command message channel of the service and the specific type and format of the command message accepted by the service;
  • The (log event Publishing) service can also usePublish / subscribe modeExternal release events;
    • The API style and other specifications include the event channel and the type and format of event messages published by the service to the channel;

3.5 pros and cons of agentless messages

In an agent-free architecture, services can exchange information directly.


  • Allow lighter network traffic and lower latency because there is no intermediate agent process;
  • It eliminates the possibility that the message broker may become a performance bottleneck or a single point of failure;
  • It has low operational complexity because there is no need to set up and maintain the message agent;


  • Services need to know each other’s location, so service discovery mechanism must be used;
  • Reduce availability, because when exchanging messages, the receiver and sender of information must be online at the same time;
  • It is more challenging to implement complex functions such as ensuring that messages can be successfully delivered;

give an example

  • Zeromq: a popular agentless messaging technology;


3.6 pros and cons of proxy message based

Message broker is the mediation node of all messages; The sender writes the message to the message agent, which sends the message to the receiver.


  • Loose coupling;
  • Message caching: the message broker can always cache messages before they are processed;
  • Flexible communication: the message broker supports all the interaction modes mentioned above;
  • Explicit interprocess communication


  • Potential performance bottleneck: solution – scale horizontally;
  • Potential single point of failure: solution – most modern message brokers are highly available;
  • Additional operational complexity: the message system must be a system component installed, configured and operated independently;

give an example

  • Popular open source message brokers: Apache ActiveMQ (JMS), rabbit MQ (AMQP), Apache Kafka;
  • Cloud based messaging services: AWS kinesis, AWS SQS;
  • In addition to AWS SQS, the above support point-to-point and publish subscribe channels; AWS SQS only supports point-to-point channels;


3.7 factors to be considered in selecting message broker

  • Supported programming languages
  • Supported information standards
  • message sequencing: whether the message agent can keep the sorting of messages;
  • Delivery guarantee: what message delivery guarantee does the message agent provide;
  • persistence
  • Durability: if the receiver reconnects to the message broker, whether it will receive the message sent when disconnecting;
  • Scalability
  • delay
  • Competitive (concurrent) receiver: whether the message agent supports competitive receivers;

3.8 processing concurrency and message order

Problem Description: when multiple instances of message receivers are expanded horizontally, the order of messages may be misplaced.

resolvent: use the fragmented message channel to extend the receiver;


  • The slice channel is composed of two or more slices, and the behavior of each slice is similar to that of one channel;
  • The sender specifies the fragment key in the message header, usually any string or byte sequence. The message agent uses the fragment key to assign the message to a specific fragment;
    • For example, select fragmentation by calculating the hash of fragmentation key;
  • The message agent combines multiple instances of the receiver and treats them as the same logical receiver;
    • For example, Apache Kafka uses the term consumer group; The message agent assigns each fragment to a single receiver; It redistributes shards when the receiver starts and shuts down;

3.9 handling duplicate messages

Problem Description: the failure of the client, network or message agent may cause the message to be delivered multiple times.

There are two solutions:

  • Write idempotent message processor
    • Idempotent operation characteristics: the impact of any multiple execution is the same as that of one execution;
  • Track messages and discard duplicate messages
    • Register the message handler into the application table (NoSQL) [Chapter 7 introduction];
    • Use message ID to track messages and discard duplicate messages, as shown in the following figure:

使用message id跟踪消息并丢弃重复消息

3.10 transactional messages

  • Use database tables as message queues
    • Transactional outbox: publish events or messages as part of database transactions by saving them in the database outbox table;
  • Publish events through polling mode
    • Polling publish data: issue messages by polling the Outbox in the database;
    • It runs well in a small scale. The disadvantage is that frequent polling of the database will cause large overhead;
  • Publish events using transaction log trailing mode
    • Transaction log tailing: publish the changes made to the database through the trailing data log;
    • Some industry cases: debezium, linkedln databus, dynamodb streams, eventuate tram;
    • The following diagram: each update submitted by the application to the database corresponds to an entry in the database transaction log; The transaction log miner can read the transaction log and send each record related to the message to the message agent;


3.11 message related class libraries and frameworks

Services need to use libraries to send and receive messages.

There are two methods:

  • Client library using message broker, the problems are:
    • The client library couples the business logic of publishing messages to the message broker API;
    • The client library usually only provides the basic mechanism for sending and receiving messages, and does not support higher-level interaction;
    • The client library of message broker is usually very low-level and requires multiple lines of code to send / receive messages;
  • Use higher-level libraries or frameworks to hide the underlying details and directly support higher-level interaction
  • Such as eventuate tram framework;

4. Use asynchronous messages to improve availability

Using synchronous communication mechanism to process requests will affect the availability of the system. Therefore, asynchronous communication mechanism should be selected as much as possible to handle calls between services.

4.1 synchronization messages reduce availability


4.2 methods to eliminate synchronous interaction

  • Use asynchronous interaction mode

    • The following diagram: the customer creates an order by sending a request message and exchanging messages through the order service; The service then communicates with other services by asynchronously exchanging messages to complete the creation of orders;
    • Disadvantages: in many cases, synchronous communication protocol APIs such as rest are used, which cannot be replaced by asynchronous;
  • Copy data

    • The following figure shows: consumer service and restaurant service publish events when their data changes; The order service subscribes to these events and updates its own data copies accordingly;
    • Disadvantages: low efficiency when the amount of data is huge;


  • Return the response before completing the processing
    • The following diagram: the order service creates an order in pending status, and then communicates directly with other services through asynchronous interaction to complete verification;
    • Disadvantages: make the client more complex.


5. Summary of this chapter

  • Microservice architecture is a distributed architecture, so interprocess communication plays a key role;
  • Careful management of the evolution of service APIs is critical. Backward compatible changes are easiest to make because they do not affect the client. If major changes are made to the API of the service, it is usually necessary to support both the old version and the new version until the client is upgraded;
  • There are many interprocess communication technologies, each of which has different advantages and disadvantages. A key design decision is to choose synchronous remote procedure call mode or asynchronous message mode. Protocols based on synchronous remote procedure calls, such as rest, are the easiest to use. However, ideally, services should communicate using asynchronous messages to improve availability;
  • In order to prevent faults from spreading through the system layer by layer, the client using the synchronization protocol service must be designed to handle local faults, which occur when the invoked service is down or shows high delay. In particular, it must use timeout when sending requests, limit the number of outstanding requests, and use circuit breaker mode to avoid calling failed services;
  • The architecture using the synchronization protocol must include a service discovery mechanism so that the client can determine the network location of the service instance. The simplest method is to use the service discovery mechanism implemented by the deployment platform: server-side discovery and third-party registration mode. But another approach is to implement service discovery at the application level: customer discovery and self registration mode. It requires more work, but it can handle scenarios where services run on multiple deployment platforms;
  • A good way to design a message based architecture is to use the message and channel model, which abstracts the details of the underlying message system. Then, you can map the design to a specific message infrastructure, which is usually based on message broker;
  • A key challenge of using message mechanism is to update database and publish messages simultaneously in an atomic way. A good solution is to use the transactional outbox pattern and first write messages to the database as part of the database transaction. A separate process then retrieves information from the database using polling publisher mode or transaction log trailing mode and publishes it to the message broker.


Newcomer production, if there are mistakes, welcome to point out, thank you very much!Welcome to the official account and share some more everyday things.If you need to reprint, please mark the source!
Chapter 3 interprocess communication in microservice architecture