Why is it easy to break the wall by importing microservice architecture?


Guest Author: Fred Chien, brobridge

Understand various technical issues and pitfalls of microservice architecture at one time

Why is it easy to break the wall by importing microservice architecture?

The microservice architecture is like a fog. No matter what you do, it seems to be wrong. You accidentally hit the wall hidden in the fog. With a little knowledge, some people choose to import container platforms (such as kubernetes) step by step, while others choose to wait and see or first understand what needs to be “deployed in advance”. But no matter how you play this game, no matter how many steps you take, you can never see through the fog, and you don’t know where the end of this game is.

In particular, some customers who have already spent money on infrastructure are more flustered than others. When they come to us, they hope that we can help point out a feasible way. Even if you point it out, it’s still a hard way to go, but it’s better than groping in the dark, which makes customers feel more at ease.

This paper will try to sort out all kinds of key technical issues and neglected issues in the whole microservice architecture, hoping that enterprises and organizations who want to import microservice architecture can understand the challenges that will be faced along the way.

What have most companies done?

It can be said that many enterprises already have the infrastructure to implement microservice architecture (such as kubernetes, CI / CD, etc.). Apart from issues such as development specifications and organizational policy transformation, there are no problems in deployment and maintenance. Next, we have to face issues such as application development, service transformation and implementation architecture.

In application development, the first step is to choose a lightweight application framework to meet the deployment requirements of micro services. Under this consideration, enterprises have different choices according to their original language habits. For example, enterprises accustomed to Java usually choose spring boot.

On the other hand, from the perspective of architecture design, the service splitting of micro service thinking is an obvious problem to be solved. Therefore, many enterprises have begun to take Domain Driven Design (DDD) as the only means to sort out existing businesses and split existing services.

In all respects, it seems that everything is complete.

What are you missing?

Why is it easy to break the wall by importing microservice architecture?

There are still many issues to be solved under the split of service logic and above the application framework (only part in the figure)

At first glance, most of the enterprises that want to invest in the microservice architecture seem to have application framework under them and Domain Driven Design Methods on top of them in terms of technology selection. Theoretically, they should go smoothly on the road of microservice. However, in the market, there are still voices that some enterprises fail to import or can not go on in the end.

In fact, there are still many things missing between “application framework” and “service logic splitting”, as shown in the red block in the figure. The missing part is the most complex and trap in the design and implementation of microservice architecture.

There should be cognition. Various designs under the decentralized architecture are different from the past thinking. In the past, the functions that we could implement in a single database and the same memory would be moved out to different machines and nodes, and then cross network collaborative operation. For many developers, it’s a new field, a new design pattern, and even a useless place for old experience.

Looking at the microservice architecture from past experience, it’s easy to find that after a long time of working around, we just built a bunch of decentralized single architectures, and didn’t get any benefits from the microservice, or even attracted many difficult problems and fell into the trap of turning over.

After ignoring various issues, it is almost impossible to construct the microservice architecture correctly.

Data decoupling is the starting point in the fog

Why is it easy to break the wall by importing microservice architecture?

Microservice architecture design topics include logic splitting and data decoupling

Despite the introduction of many design patterns or the use of Domain Driven Design (DDD) to decouple services, most of the focus is still on the separation of application logic, especially if the lack of relevant technical knowledge, most people will choose to ignore or give up the decoupled work of the data layer. This is also why many people find consultants to import DDD. Once they enter the technology implementation, they either fail to achieve good results or fail to implement it.

You should know that data is the most “heavy” part of an application. Only data decoupling can truly realize the service splitting of a microservice architecture. If we do the DDD or service splitting work, we will not be able to correctly implement the microservice architecture and eventually lead to failure.

As for data decoupling, is it necessary to do high-risk table and database splitting? unwanted. If you want to understand the real purpose of data decoupling and the data management method after decoupling, you can refer to the old paper:The magic trick of cqrs: Micro service data decoupling

In some DDD books, technical design patterns such as cqrs are often mentioned to assist in data exchange or aggregation at the domain boundary of DDD. However, many people only treat DDD as a pure theoretical method, and ignore the technical issues mentioned in these books, resulting in a mere formality and lack of practical treatment.

Data consistency is the core of everything

Why is it easy to break the wall by importing microservice architecture?

Issues surrounding data consistency

Basically, microservice architecture is the design of distributed system, which means that service status and data are not centralized. In addition to the consistency of service status, data synchronization and sharing should also be considered. But in the face of different applications, the requirements of data consistency are also different. Some applications need absolute consistency, some applications can accept strong consistency, and some application scenarios only need to achieve the final consistency.

In the face of different data consistency requirements, solutions are quite diverse. The main topics are distributed data management, state and object lifecycle management, and distributed transaction or stateful services.

Only by properly designing these management mechanisms or integrating them can we realize the microservice architecture in different applications. As long as too much emphasis on specific design patterns, or in inappropriate situations using the wrong method, will lead to many sequelae, or even failure.

Wrong design may lead to uncoupled database, single architecture that is difficult to expand, and even performance impact and maintenance problems.

Figure out how to meet low latency requirements

In theory, the problem that distributed architecture can’t solve is delay. Due to the dispersion of program logic and data, most tasks need to communicate, synchronize and exchange state across services. This means that delay will always happen. It’s just how much and whether the target application can accept the delay.

Although the performance and speed of computers and networks are very fast, these delays are not obvious to users, and are acceptable for most applications. But if your application can’t accept these delays and needs more extreme immediate needs, then you need to rely on the design of stateful services.

However, stateful services are not good at scalability and fault tolerance, and can easily become performance bottlenecks and stability gaps. The accompanying single point of failure and decentralized monomer architecture problems are traps. Therefore, how to decouple and optimize stateful service and minimize its operational risk is the key. At this time, it is necessary to know how to design state management mechanism and event scheduling mechanism to achieve low latency requirements with scalability and high throughput.

Painful developers don’t have three heads and six arms

Why is it easy to break the wall by importing microservice architecture?

Various micro service issues to be implemented

Many developers originally expected the application framework to have a complete solution, but the reality is that there is still a long way to go, which needs to be solved by developers themselves, and there is no ultimate and complete solution.

In addition, although there are some open source projects that can partially solve or cover some issues, because of their different development backgrounds, their starting point and focus are not the same. Many of them may not meet the needs of various enterprises, and even need customization and self-maintenance. More still depends on the experience and ability of developers, in order to achieve the required micro service mechanism.

However, each of these issues should not be underestimated. If application developers in general enterprises want to invest in any one issue, they will have to spend a lot of energy and manpower, and they may not have time to take care of application development. In particular, the stability and scalability of these mechanisms are the key to the smooth operation of the microservice architecture. If not well designed and maintained, the final result can be imagined.

It has not been mentioned how high the risk will be for developers who are new to microservice architecture to try to complete these key mechanisms with little knowledge. It is also a very painful thing for developers who are originally focused on application logic.

How to solve these problems and avoid pitfalls?

Why is it easy to break the wall by importing microservice architecture?

Based on the basic mechanism, it solves the decentralized data and state management to meet the demand of service decoupling

As the issue of data consistency is the core of microservices, the mechanism to support the consistency of all kinds of data is the top priority, which is the best place to start. Then, after solving most of the data issues, we further expand the technical issues of service splitting.

In this way, the introduction of cqrs, distributed transaction mechanism, object state and life cycle management mechanism and event scheduling mechanism is the key core of technology implementation. The issue of event driven and communication mechanism under this is not to worry too much. It can be met through proper education and training or traditional solutions.

Finally, it is strongly recommended that

If you are still not familiar with the various topics and design patterns mentioned in this article, you have never heard of them. It is recommended that you stop and understand the meaning and solutions behind these issues before you have a bigger problem, and then continue your journey of micro service.

The sources of this paper are all fromBrobridgeIf you want to know more about microservice architecture design, please contact us. If you are interested in specific microservice components or microservice component packages, you are also welcome to contact us.

Click to read the original text of the website

CNCF (cloud native Computing Foundation) was established in December 2015 and belongs to Linux   Foundation, a non-profit organization.
CNCF (cloud native Computing Foundation) is committed to cultivating and maintaining a vendor neutral open source ecosystem to promote cloud native technology. We democratize cutting-edge models to make these innovations available to the public. Scanning two-dimensional code concerns CNCF WeChat official account.
Why is it easy to break the wall by importing microservice architecture?