Summary of microservice and RPC ideas


Microservice is a distributed way. Business can be split through microservices, so that business responsibilities can be simplified and business can be decoupled. Microservices are usually deployed in clusters, and the communication between services needs to be completed through RPC. Clusters need to be managed through service governance, which mainly manages the mapping relationship between interface methods and services, load balancing, health detection, service renewal, service discovery, disaster tolerance, etc.

1、 RPC

RPC is mainly completed through TCP transmission protocol and efficient serialization and deserialization. High performance TCP transmission manual is mainly through IO multiplexing and zero copy. The most typical framework is netty. The main methods of IO multiplexing are select, poll and epoll. Epoll is the main method of liunx’s IO multiplexing model, and it is also the highest performance. SPI mechanism of plug-in extension.

It can be seen that a set of RPC processes is to transmit serialized binary data to a specific address through TCP, and then restore the data through deserialization.RPC is responsible for helping us solve the communication problems after the system is split, and enabling us to call remote methods like local methods. We don’t need to write a lot of code unrelated to business because this method is called remotely. RPC helps us shield the details of network programming.However, a series of detailed operations such as shielding network programming and serialization are generally performed by usingDynamic proxyTo do it.

Dynamic proxy

RPC will automatically generate a proxy class for the interface. When we inject the interface into the project, the proxy class generated by the interface is actually bound during the operation. In this way, when the interface method is called, it is actually intercepted by the generated proxy class, so we can add remote call logic to the generated proxy class. Common dynamic proxy technologies include JDK dynamic proxy, javassist, byte buddy, cglib, etc

IO multiplexing:

The TCP framework used by Dubbo is netty, which is an IO multiplexing model, and its HTTP performance is better than that of springcloud. IO multiplexing models mainly include the following:
select: in the user mode, the connected file descriptors are placed in an array each time. Each time select is called, the entire array needs to be copied to the kernel, so that the kernel can poll the file descriptors. After that, the kernel will mark the file descriptors with data in the array, and then return the number of readable file descriptors. The user should traverse to find the marked file descriptors.
poll: the principle is the same as that of select, except that poll maintains that there is no upper limit on the number of linked list file descriptors, while select can only have 1024 at most
epoll: the performance of IO multiplexing model is improved in three ways

    1. A set of file descriptors is saved in the kernel. The set is a red black tree structure. Users do not need to re-enter it every time. They only need to tell the kernel what to modify, reducing the consumption of copying file descriptors back and forth between select and poll.
    1. The kernel no longer finds the ready file descriptor through polling, but wakes up through asynchronous IO events.
    1. The kernel only returns file descriptors with IO events to the user, and the user does not need to traverse the entire set of file descriptors.

Zero copy (picture transferred from wechat official account: Xiaolin coding):

The two main methods of zero copy are MMAP and sendfile. Sendfile is the most commonly used method with the best performance. The traditional copy needs to go through such a process: when the user process wants to send data in the network.
1. First, you need to read the data from the disk. You need to send a read request. You need to switch from user mode to kernel mode to complete this command
2. Then, in the kernel state, DMA technology is used to copy the disk data to the kernel state cache, and then the kernel state cache data is copied to the user state cache
3. Then you need to copy these data from the user state cache to the kernel state cache. At this time, the user state is switched to the kernel state

4. Then, these data cached in the kernel state must be copied to the network card cache through DMA technology, then sent, and switched back to the user state.

Summary of microservice and RPC ideas

There are four user mode and kernel mode switches and four data copies. In fact, there is no need to copy data back and forth in user mode and kernel mode during the sending process. There are two zero copy ways to solve these problems

  • Mmap: the system call function will directly map the data in the kernel buffer to the user space. This cached data becomes an area that is too shared between the user mode and the kernel, so that the operating system kernel and the user space do not need to copy any data. However, this is not the ideal zero copy, because it still needs to copy the data from the kernel buffer to the socket buffer through the CPU, and it still needs four context switches

    Summary of microservice and RPC ideas

  • Sendfile:sendfile is more violent and efficient than MMAP. First, it copies the data on the disk to the kernel buffer through DMA, and then directly copies the data in the kernel buffer to the buffer of the network card. This process does not need to copy the data from the operating system kernel buffer to the socket buffer, thus reducing one data copy. This process only requires two kernel mode and user mode switches and two data copies

    Summary of microservice and RPC ideas


RPC data transmission relies on TCP. The data transmitted by TCP is binary data. At this time, it is necessary to serialize and deserialize the objects transmitted in the method. The serialization mainly considers several points: security, compatibility, performance, and volume after serialization.
At present, the well-known ones include Jason, Hessian, protobuf and kryo. For example, the default serialization method of Dubbo is Hessian, the feign of springcloud is JSON, and grpc is protobuf.


Extension plug-in function of RPC. In Java, JDK has its own SPI service discovery mechanism, which can dynamically find service implementation for an interface. To use the SPI mechanism, you need to create a file named after the service interface in the meta-inf/services directory under the classpath. The contents of this file are the specific implementation classes of this interface. In fact, the SPI function of JDK is rarely used. The reasons are as follows: 1. The spi of JDK standard will load all implementations of instantiated extension points at one time. When the JDK is started, all the implementation classes in the meta-inf/service file will be loaded at one time. If some extension point implementations are time-consuming to initialize, or if some implementation classes are not used, it will be a waste of resources. 2. If the extension point fails to load, the caller will report an error, which is difficult to locate.

Dubbo’s SPI mechanism is well-known in the market. Dubbo’s SPI mechanism can expand plug-ins on demand, and almost all functional components are implemented based on SPI. By default, it provides many extension points that can be used directly, such as serialization method, transmission protocol, etc.
Of course, springcloud also has its own SPI mechanism, which is only implemented through injection, mainly through condition on-demand injection

2、 Service governance

Service governance mainly includes: service discovery, service renewal, health detection, load balancing, disaster tolerance, etc. These functions are mainly realized by registering the service in the registry. Famous registration centers include Eureka, Nacos, zookeeper, etc

Service discovery

Service discovery is mainly used to maintain the mapping between the calling interface and the service provider address. It can help us locate the calling service provider’s IP address and maintain the dynamic change of the cluster’s IP address. With service discovery, we can get the IP set of the corresponding interface from the cluster and select the IP address to call through load balancing, which is also the service discovery mechanism of PRC. Service discovery is realized in the following two ways:
1. Service registration: when the service provider starts, it registers its own node messages (IP address, port, service interface, etc.) to the registry.
2. Service subscription: when the service caller starts, go to the registry to find and subscribe to the IP address of the service provider, and then
Stored locally and used for subsequent remote calls.
3. Service renewal: regularly pull the information of other nodes from the registry and update the local cache.

Currently, the popular service discovery frameworks include zookeeper, Eureka and Nacos. A distributed system cannot satisfy C (consistency), a (availability) and P (partition fault tolerance) simultaneously, where zookeeper is CP and Eureka is AP.
When Eureka’s self-protection mode can lose too many clients in a short time (network failure may have been sent), the node will enter the self-protection mode and no longer log off any micro services. After the network fault is recovered, the node will automatically exit the self-protection mode. In the large-scale cluster service discovery system, the strong consistency should be abandoned and the robustness of the system should be considered more. Finally, consistency is the more common strategy in the design of distributed systems

Health testing

When PRC invokes the service for load balancing, it usually needs to know which IP address nodes are available and which nodes are unavailable. This information requires the registry to perform health detection on the registered service nodes. The most common method is regular heartbeat detection, which sends Ping like requests through the established TCP connection. According to the replies of the service node, it can be divided into the following three states:
1. Health status: the connection is established successfully, and the heartbeat detection is always successful.
2. Sub health status: the connection is established successfully, but the heartbeat request fails continuously.
3. Dead status: failed to establish connection.

Different frameworks have different heartbeat detection frequencies. For example, Eureka has a heartbeat every 30 seconds. If the contract is not renewed within 90 seconds, the service will be rejected

load balancing

Both the microservice framework and RPC framework have the function of load balancing, and they are also the necessary capabilities of the cluster. Generally, the service provider information is pulled from the registry and cached locally. When calling the corresponding interface service, a node that can be called will be selected from the set of service provider nodes of the corresponding interface. The specific node to call will have different selection strategies, such as polling, random, weighted polling, minimum active calls, consistency hash, etc.

For example, the springcloud ribbon is the core component of springcloud service consumption and load balancing. The ribbon is a load balancing component based on HTTP and TCP. Eureka uses the ribbon to realize service consumption. The ribbon performs polling access through the configured service list. However, when integrated with Eureka, the service list will be rewritten by Eureka’s service list and expanded to obtain the service list from the registry (Eureka’s service list is the information data of service nodes pulled from the registry). At the same time, replace its own iping function with Eureka’s niwsdiscoveryping function, and Eureka will confirm which services are available. Therefore, ribbon uses the mechanism of Eureka registry service discovery, service registration, service renewal and health detection to discover the node address and health information of available services, and then uses its own load policy to consume services.

Fuse current limiting degradation

Fuse current limiting degradation is a necessary function of micro service distributed system. It mainly deals with many problems caused by excessive access traffic, high node load and long service time. Through fusing and degradation, the cascading fault problem of too many interface service timeouts caused by too long service processing time can be protected, and the current limiting protection system can avoid the problems of too large traffic access and too high node load.
The most classic in the industry for fuse current limiting degradation is springcloud’s hystrix, which implements a complete set of fuse current limiting degradation ideas.

Next, let’s introduce the hystrix:
Hystrix has two isolation strategies for requests: thread isolation and semaphore. There are also current limiting means: isolation, fusing and degradation
Thread isolation:
The hystrix will open the thread pool to execute requests. Just like ordinary thread pools, this thread pool has the number of core threads, the maximum number of threads, task queues, thread idle time, etc. requests will be thrown into the thread pool for execution. When the number of core threads exceeds, threads will be expanded, and when the number of maximum threads exceeds, they will be thrown into the queue. The mechanism is the same as that of an ordinary thread pool. By default, the request traffic of all methods of a class share a thread pool.
Semaphore isolation: current limiting
Unlike thread isolation, semaphore isolation can limit the maximum concurrent requests of a corresponding service interface call. Controlling the maximum concurrent number according to the semaphore can achieve the purpose of current limiting
Fuse degradation:
The timeout function of calling the service interface can be enabled by hystrix. By default, it is 1000ms. You can set whether to interrupt when the timeout occurs. This setting is only valid for the thread isolation policy. By default, the execution is interrupted when the timeout occurs. Hystrix also determines whether to enable the fusing and the subsequent half fusing according to the success rate of calling the interface within a fixed time interval. The default fusing and half fusing rules are as follows:
1. Fusing:
When the number of failed requests for back-end services within the time window exceeds a certain proportion (the proportion can be set by circuitbreaker.errorthresholdpercentage at 50% by default), the circuit breaker will switch to the fusing state (open). It should be noted that the number of failed requests within the time window must exceed a certain value before statistics are made (20 by default), If only 19 requests fail by default, the fuse will not be turned on (the number of requests can be set through circuitbreaker.requestvolumthreshold). After the circuit breaker is opened, all requests will directly fail to trigger the corresponding degradation method.
2. Half fuse:
After the circuit breaker is kept in the open state for a period of time (5 seconds by default, and the fusing sleep time can be set through circuitbreaker.sleepwindowinmilliseconds), it will automatically switch to the half-open state. At this time, a request will be missed to set the state according to the return condition. If the request is successful, the circuit breaker will close the fused state (closed), otherwise it will switch to the fused state (open). When hystrix finds that the end service is unavailable according to the rules, it will directly cut off the request chain to avoid sending a large number of invalid requests affecting the system throughput, and the circuit breaker has the ability to self detect and recover.

During the fusing process, all requests will be triggered with corresponding degradation methods.

Distributed link tracking:

RPC calls in distributed systems may involve calls to multiple remote services. Suppose that the distributed application system is composed of four sub services, and the dependencies of the four services are a->b->c->d. if an exception occurs in the calling service, printing the log is often unable to quickly locate the problem. In this case, you need to use distributed link tracking. The well-known distributed link tracking components are sleuth and zipking.
Distributed link tracking has the concepts of trace and span. Trace is a service call link. Each distributed link will generate a trace. Each trace has its unique identifier, traceid. In the distributed link tracking system, each trace is distinguished by traceid.
A span represents a link in a call link, that is, a trace is composed of multiple spans. Under a trace, each span also has its unique identifier spanid, and the span has a parent-child relationship.
For example, in the case of a->b->c->d, three spans will be generated in the whole call chain, namely span1 (a->b), Span2 (b->c) and span3 (c->d). At this time, span3
The parent span of Span2 is Span2, and the parent span of Span2 is span1

Another example:
Such logs are generated in Sleuth: [order service, 96f95a0dd81fe3ab, 852ef4cfcdecabf3, false]
1. The first value is spring application. Value of name
2. The second value, 96f95a0dd81fe3ab, is an ID generated by sleuth, that is, trace ID, which is used to identify a request link. A request link contains a trace ID and multiple span IDs.
3. The third value, 852ef4cfcdecabf3, is spanid.
4. The fourth value: false, whether to output this information to the Zipkin service for collection and display.

Routing gateway and configuration center:

The role of the routing gateway is similar to nginx haproxy, etc. before all requests reach the cluster, the gateway will first perform URL filtering, permission verification, load balancing, etc. the routing gateway also pulls all node IP information from the registry to the local through the service discovery function, and then filters URLs and performs load balancing through configuration. Zuul and gateway are well-known in the industry. The NiO network programming technology used by gateway is better than zuul in performance.
The configuration center centralizes the configuration of the system, such as spring YML, the change of gateway configuration, the change of gateway flow restriction policy, and other configurations. Then each node pulls down the corresponding configuration of the configuration center through the long TCP connection to achieve the effect of hot loading and unified management. Apollo is popular in the market