Do you understand the timeout delivery of microservices?


Why do I need timeout control?

A common problem in many cascading failure scenarios is that the server is consuming a lot of resources to process requests that have already exceeded the deadline of the client. As a result, the server consumes a lot of resources and does not do any valuable work. It is meaningless to reply to requests that have timed out.

Timeout control can be said to be an important defense line to ensure service stability. Its essence is fail fast. A good timeout control strategy can empty high latency requests as soon as possible, release resources as soon as possible and avoid the accumulation of requests.

Inter service timeout delivery

If a request has multiple stages, such as a series of RPC calls, our service should check the deadline before the beginning of each stage to avoid useless work, that is, to check whether there is enough time left to process the request.

A common error implementation method is to set a fixed timeout time for each RPC service. We should pass the timeout time between each service. The timeout time can be set at the top of the service call, and the same absolute deadline will be set for the whole RPC tree triggered by the initial request. For example, set the timeout at the top layer of the service request as 3S, service a requests service B, and the execution time of service B is 1s. When service B requests service C again, the timeout remains 2S, and the execution time of Service C is 1s. At this time, service C requests service D again, and the execution time of service d is 500ms, and so on. Ideally, the same timeout delivery mechanism is adopted in the whole call chain.

If the timeout delivery mechanism is not adopted, the following situations will occur:

  1. Service a sends a request to service B, and the set timeout is 3S
  2. Service B takes 2s to process the request and continues to request service C
  3. If timeout delivery is used, the timeout time of Service C should be 1s, but timeout delivery is not used here, so the timeout time is 3S written dead in the configuration
  4. Service C takes 2s to continue execution. In fact, the timeout set at the top layer has expired. The following request is meaningless
  5. Continue requesting service d

If service B adopts the timeout delivery mechanism, service C should immediately give up the request, because the deadline has expired and the client may have reported an error. When setting timeout delivery, we usually reduce the delivery deadline, such as 100ms, so as to take into account the network transmission time and the processing time after the client receives the reply.

In process timeout delivery

Timeout delivery is not only required between services, but also within processes. For example, mysql, redis and service B are called serially in a process. The total request time is set to 3S. It takes 1s to request MySQL and then re request redis. At this time, the timeout time is 2S. Redis takes 500ms to execute and re request service B. at this time, the timeout time is 1.5s, Because each middleware or service will set a fixed timeout in the configuration file, we need to take the minimum of the remaining time and the set time.

Context implements timeout delivery

The principle of context is very simple, but its function is very powerful. Go’s standard library has also realized the support for context, and various open source frameworks have also realized the support for context. Context has become the standard, and timeout delivery also depends on context.

We usually pass the timeout control at the top layer of the service by setting the initial context. For example, set the timeout time to 3S

ctx, cancel := context.WithTimeout(context.Background(), time.Second*3)
defer cancel()

When context transfer is performed, such as requesting redis in the above figure, the remaining time is obtained in the following way, and then the smaller time is taken compared with the timeout set by redis

dl, ok := ctx.Deadline()
timeout := time.Now().Add(time.Second * 3)
if ok := dl.Before(timeout); ok {
    timeout = dl

Timeout delivery between services mainly refers to the timeout delivery during RPC calls. We do not need to do additional processing for grpc. Grpc itself supports timeout delivery. The principle is similar to the above. It is delivered through metadata and will eventually be converted into grpc timeout value, as shown in the following codegrpc-go/internal/transport/handler_server.go:79

if v := r.Header.Get("grpc-timeout"); v != "" {
        to, err := decodeTimeout(v)
        if err != nil {
            return nil, status.Errorf(codes.Internal, "malformed time-out: %v", err)
        st.timeoutSet = true
        st.timeout = to

Timeout delivery is an important defense line to ensure service stability. The principle and implementation are very simple. Has timeout delivery been implemented in your framework? If not, move quickly.

Timeout delivery in go zero

In go zero, you can use theTimeoutto configureapi gatewayandrpcThe timeout of the service and will be automatically passed between services.

previousHow to realize go timeout controlIt explains how to use timeout control.

reference resources

Sre: Google operation and maintenance decryption

Project address

Welcomego-zeroandstar/forkSupport us!

Wechat communication group

Focus onMicroservice practiceOfficial account and clickCommunication groupGet community QR code.

This work adoptsCC agreement, reprint must indicate the author and the link to this article

Recommended Today

Example of realizing canvas shadow effect with HTML5

Realize canvas shadow effect in HTML5 Copy code The code is as follows: <!DOCTYPE html><html><head><meta http-equiv=”X-UA-Compatible” content=”chrome=IE8″><meta http-equiv=”Content-type” content=”text/html;charset=UTF-8″><title>Canvas Clip Demo</title><link href=”default.css” rel=”stylesheet” /> <script> var ctx = null; // global variable 2d context var imageTexture = null; window.onload = function() { var canvas = document.getElementById(“text_canvas”); console.log(canvas.parentNode.clientWidth); canvas.width = canvas.parentNode.clientWidth; canvas.height = canvas.parentNode.clientHeight; if (!canvas.getContext) […]