Recently, we are often brainwashed by the fuse. The turbulence of the stock market makes the circuit breaker appear in front of us again. The fusing in microservice means that the service provider will stop the access of the service to prevent avalanche effect if it returns abnormally or responds slowly due to the reasons such as too much access pressure or dependence on exceptions within a certain period of time. Lightboat gateway is implemented based on envoy, which is extended on the idea of community fusing. This paper starts with the fusing of istio, and discusses the fuse of lightboat gateway.
Starting from fusing
Recently, we are often brainwashed by the circuit breaker. The turbulence in the stock market has witnessed four circuit breakers in the US stock market in two weeks. What exactly is circuit breaker? In the stock sector, circuit breaker is actually automatic suspension of trading. After the stock market falls to a certain extent, stock trading is suspended. Compared with the stock market, the circuit breaker in microservices means that the service provider returns abnormally or responds slowly due to the pressure of access or dependence on the exception within a certain period of time. Fusing means stopping the call of the service to prevent “avalanche effect”. From the perspective of service gateway, fusing is to prevent the call of service provider on the gateway side. From the perspective of service gateway, this paper explores how to fuse and protect back-end services in istio.
Istio is blown
Istio provides rich means of fusing, which can protect the back-end through abnormal point detection and connection pool configuration. Outlier detection dynamically determines the health status of the upstream cluster. If there are continuous access exceptions, the abnormal back-end will be expelled from the load balancing instance, and no traffic will be sent to it for a period of time. After a period of time, join the load balancing cluster and continue to provide services. If it is still abnormal, increase the isolation time. Istio provides active and passive health check. Anomaly detection can be considered as passive health check. Data plane envoy provides active health check. Health check interface is configured on the upstream cluster. Active and passive health check are used together. As a whole health check scheme, the upstream cluster is highly available.
Outlier detection is configured in the control plane in the desitionrule CRD, corresponding to the configuration in the cluster in the data plane Evoy. The main configuration items include:
1. Consecutive errors: the number of consecutive errors. For HTTP services, 502, 503 and 504 are considered as exceptions. For TPC services, connection timeout means exceptions 2. Intervals: the interval between evictions. The default is 10 seconds 3. Baseobjectiontime: minimum expulsion time. The expulsion time increases with the number of errors. The minimum expulsion time is the number of errors 4. Maxobjectionpercent: the maximum proportion of instances in the load balancing pool that can be evicted. In order to avoid the instantaneous unavailability of an interface, too many instances will be expelled, and the whole service will be unavailable. `
At the same time, istio also provides connection pool configuration to restrict client access through the configuration of four layer TCP and seven layer HTTP connections. The specific configuration includes: TCP connection pool configuration and HTTP connection pool configuration. Configuration items mainly include:
HTTP connection pool configuration works with TCP connection settings. Corresponding to the data plane envoy, it is divided according to the business attributes, and part of it is divided into cluster.circuit_ The other part belongs to the cluster configuration.
Istio fusing and hystrix fusing**
Hystrix is a mature fuse framework in the field of microservices, and is an open source fuse framework of Netflix. However, since the end of 18, it has not been actively maintained. Hystrix encapsulates the request, the related logic runs in independent threads, and achieves the effect of resource isolation through thread pool. Hystrix provides fusing capability, with automatic detection and recovery. The circuit breaker switches automatically between open, half open and closed states. At the same time, fallback method is provided to facilitate users to degrade in case of back-end service fusing.
Any architecture has different usage scenarios, just as there is no perfect architecture, only the right architecture. Flexible configuration and scalability make hystrix integrate into the spring cloud system to provide fusing function for spring cloud microservice framework. However, in the process of using hystrix, hystrix dependency package must be introduced. Hystrix can be regarded as a white box, and developers can customize the business, including the preparation of degradation methods. Although the business and related fuse governance can be decoupled from the code through the SDK mode, the business and SDK still need to be compiled together. At the same time, hystrix is only suitable for the Java language.
Istio’s fusing is completely transparent to the business and does not need to have any dependence on the business; in the sidecar mode, it can be seamlessly connected with the business cluster. However, istio’s fusing is more based on the cluster configuration, and the fusing strength is still weak compared with some business scenarios.
Examples of fusing
The lightboat gateway provides a wealth of fuse configuration pages, and the connection pool configuration can be configured through the console of the light boat gateway. The specific configuration is as follows: the maximum number of TCP connections is 1, and the maximum number of HTTP pending requests is 1.
Gateway fuse diagram 005.png
Through the configuration of the console, the specific configuration can be transformed into the configuration of trafficpolicy of virtualservice in istio. The details are as follows
Serial request service interface for five times, and check the status of the data plane envoy stat, you can see that all five requests return 200
The client receives three 503 and two 200 status returns. After viewing the envoy stat of the data plane, it can be seen that the corresponding 2XX is twice and 5xx is three times. The cause of 5xx is upstream_ Rq_ pending_ overflow。 Because of max_ If the pending number is set to 1, the request is blown.
Light boat gateway is broken
Based on istio’s fusing, lightboat gateway realizes service level fuse current limiting. It provides active health check, passive health check and connection pool configuration of service dimension. At the same time, based on the service, the gateway extends the route level fusing. Combined with the thinking of hystrix, it uses sliding window to count the error rate and RT. The fuse plug-in is implemented, which acts on Routing and virtualhost level. The specific process is as follows:
At the same time, the light boat gateway provides a wealth of current limiting policies to protect the back-end services, which is implemented based on rate limit server. It provides flow control based on percentage request and frequency control based on condition, including current limiting of header and other dimensions. The specific configuration page is as follows: This is the simplest configuration. It is configured as the request header with IP: 127.0.0.1 requests 100 times per minute. If more than 100 times trigger current limiting, return 429.
At present, there are 10 kinds of plug-ins in the gateway to extend the function of envoy. Including cache, CORS, jsonp, current limiting, dynamic degradation, static degradation, fusing, authentication and other plug-ins. Interested students can learn about the light boat gateway.