Understanding the Microsoft Architecture of Java Architecture – Avalanche Effect



Micro-service product line, each service concentrates on its own business logic, and provides corresponding interfaces to the outside world, it seems very clear, in fact, there are many things to consider, such as: automatic expansion of services, fuse and current limitation, etc., with the expansion of business, the number of services will increase, logic will increase. More complex, a service’s logic needs to rely on multiple other services to complete.

Once a dependency fails to provide a service, it is likely to have an avalanche effect, resulting in inaccessibility of the entire service.

When RPC or HTTP calls are made between micro-services, we usually set up mechanisms such as call timeout and failure retry to ensure the successful execution of services. It looks very beautiful. If we do not consider service fusing and current limiting, it is the source of avalanche.

Suppose we have two services A and B with large visits, which depend on C and D respectively, and both C and D depend on E services.

A and B constantly call C and D to process customer requests and return needed data. C and D timeouts and retries are performed when E services are not available.

Because of the continuous generation of new calls, it will lead to a large backlog of C and D calls to E services, resulting in a large number of call waiting and retry calls, which will slowly exhaust C and D resources such as memory or CPU, and then download.

A and B services duplicate C and D operations, exhaust resources, then download, and ultimately the entire service is inaccessible.

The common causes of avalanches are the following:

  • Program bugs cause services to be unavailable or slow to run
  • Cache breakdown, which results in all calls to a service and downs
  • A sudden surge in visits.
  • Hardware problems, this feeling can only be said to be a bit behind the scenes _____________.

Although there are tens of millions of avalanche effects, it is our unshirkable responsibility to ensure that the service does not hang up and run smoothly. There are still many protection schemes for avalanche effects.

Horizontal Expansion of Services

Now we can use a lot of tools to ensure that services do not hang up, and then when traffic is high, we can expand services horizontally to ensure smooth business. For example, we use k8s most often, which can guarantee the running state of the service, and also can make the service expand horizontally automatically. This is a good way to deal with the surge of user visits, but the horizontal expansion has an end. If the response time of E service is too long in a certain environment, it may still lead to avalanche effect.

Current limiting

Limiting client calls to reach current limits is common. For example, we limit the maximum number of requests per second to 200, and more than one number of requests are rejected directly. Common algorithms such as token bucket algorithm

Place tokens in buckets at a certain speed. When clients request services, they need to get tokens from buckets before they can be processed. If the tokens in buckets run out, they refuse access.


When the client controls access to the dependency, if the dependency of the call is unavailable, it will not be called again, return the error directly, or degrade the processing. Open source libraries, such as hystrix-go, are also a library for source code analysis that I’m going to write next. The function of fusing and degrading is well realized. His main idea is to set some thresholds, such as maximum concurrency, percentage of errors, fuse attempt recovery time and so on. These thresholds can be used to change the state of the fuse:

  • Close state to allow invocation dependencies
  • Open state, do not allow calling dependencies, return errors directly, or call fallback
  • Half-open state, which is opened according to the recovery time of fuse attempt, allows call dependencies, and continues to open if the call fails to close if the call succeeds.

The above is the whole content of this article. I hope it will be helpful to everyone’s study, and I hope you will support developpaer more.

Recommended Today

Swift advanced 08: closure & capture principle

closure closurecanCapture and storageOf any constants and variables defined in their contextquote, this is the so-calledClose and wrap those constants and variablesTherefore, it is called“closure”Swift can handle everything for youCaptured memory managementOperation of. Three forms of closure [global function is a special closure]: a global function is a closure that has a name but does […]