Discussion on the relationship between service mesh and API gateway



As for the relationship between service mesh and API gateway, this question has been frequently asked in the past two years, and many articles and materials in the community have also given answers. Among them, there are many Internet celebrities such as Christian Posta who have given in-depth introduction. I am here to do a data collation and summary, combined with personal understanding to give some views. In addition, at the end of this paper, we will introduce some innovative practices and explorations of ant financial services in the latest field of service mesh and API gateway integration, hoping to give you a more intuitive cognition.

Note 1: in order to save space, we will go straight to the topic, assuming that readers have a basic understanding of service mesh and API gateway.
Note 2: This article focuses more on combing the whole context, and the content will not be expanded in detail, especially the parts that have been elaborated in other articles. If you want to know more about the details after browsing this article, please continue to read the references and recommended readings at the end of the article.

Originally clear boundaries: positioning and responsibilities

First of all, there are very clear boundaries between service mesh and API gateway in terms of function orientation and responsibilities

  • Service mesh: the network communication infrastructure of microservices, which is responsible for the communication between services (within the system);
  • API gateway: it is responsible for exposing services in the form of API (to the outside of the system) to realize business functions;

As shown in the figure above:

Discussion on the relationship between service mesh and API gateway

In terms of functions and responsibilities:

  • At the bottom is the split atomic microservices, which provide various capabilities in the form of services;
  • In atomic microservices, they are (optional) composite services. In some scenarios, the capabilities of several microservices need to be combined to form new services;
  • Atomic microservices and composite services are deployed onInside the systemIn the case of using service mesh, service mesh provides the ability of communication between services;
  • API gateway is used to expose these services within the system toSystem externalTo accept external requests in the form of API;

In terms of deployment:

  • Service mesh is deployed inside the system: atomic microservices and composite services are usually not directly exposed to external systems;
  • API gateway is deployed at the edge of the system: on the one hand, it is exposed outside the system, providing external API for external system access; on the other hand, it is deployed inside the system to access various internal services.

Two widely used terms are introduced here:

Discussion on the relationship between service mesh and API gateway

  • East WestCommunication: refers to the mutual access between services. The communication flow flows between services, and the traffic is located in the system;
  • North SouthCommunication: it refers to the external access provided by the service, usually through the API provided by the API gateway to the external Paul, and its communication flow is from the outside of the system to the internal system;

Explain the origin of “East, West, north, South”: as shown in the figure above, we usually follow the principle of “up north, down south, left East and right West” on the map.

Conclusion: service mesh and API gateway have clear division of functions and responsibilities and clear boundaries. But if this is the end of the matter, there will be no discussion on the relationship between service mesh and API gateway, and there will be no article.

Where is the root of the problem?

Highly recommended reading: in the appendix, Christian Posta’s article “do I need an API gateway if I use a service mesh?” has in-depth analysis and explanation.

Philosophical question: is gateway access to internal services East-West or north-south?

As shown in the figure below, the yellow line in the figure indicates that API gateway accesses internal services

Discussion on the relationship between service mesh and API gateway

Here comes the problem. From the perspective of traffic trend, this is that after the external traffic enters the system, it begins to access the services exposed to the outside. It should belong to the “North-South” communication, which is typical of the drawing in the figure above. However, from another perspective, if we divide the API gateway into two parts logically, ignoring the exposed part first and only looking at the part of API gateway accessing internal services, we can consider API gateway as an ordinary client service, and its communication with internal services is more like “East-West” communication

Discussion on the relationship between service mesh and API gateway

Therefore, when the API gateway is used as a client to access internal services, whether it is north-south or East-West has become a philosophical issue: it depends entirely on how we view the API gateway as a whole or logically divided into internal and external parts.

This philosophical problem is not groundless. In various products of API gateway, there are two schools on how to realize “API gateway as a client accessing internal services”

  1. There are two independent communication mechanisms between gateway and gateway;
  2. Compatibility and integration: regarding API gateway as a common internal service client, reusing its internal service communication mechanism;

And the final decision is usually related to the positioning of the product: if you want to maintain the independent product positioning of API gateway and hope that it can be used under different inter service communication schemes, the former is usually selected, such as Kong; if it has a very deep relationship with the inter service communication scheme, the latter is usually selected, such as zuul and spring cloud under the spring cloud ecology Gateway

However, no matter which genre you choose, you can’t change the fact that “API gateway as a client accesses internal services” It does not have essential differences with an ordinary internal service as a client to access other services: service discovery, load balancing, traffic routing, fusing, current limiting, service degradation, fault injection, logging, monitoring, link tracking, access control, encryption, identity authentication When we list the functions of gateway accessing internal services one by one, we find that almost all of these functions are called repeatedly between services.

This also leads to a common phenomenon: if there is a mature inter service communication framework, and then consider the implementation of API gateway, it will become a natural choice to reuse these repetitive capabilities. For example, zuul under the spring cloud ecology mentioned above and the spring cloud gateway developed later realize the reuse of these capabilities in the way of reusing class libraries.

Here is a similar philosophical problem: when “API gateway is used as a client to access internal services”, it realizes code level capability reuse by reusing class libraries, which is equivalent to implementing a client side which is exactly the same as the communication scheme between ordinary services. Then, is the traffic from the “client” calculated as East-West or north-south?

The answer doesn’t matter.

Sidecar: the real coincidence point

After entering the era of service mesh, the relationship between service mesh and API gateway begins as follows:

  1. Clear division of functions and responsibilities;
  2. The function of client accessing service is highly overlapped;

At this time, the relationship between the two is very clear, and since service mesh and API gateway are different products at that time, the coincidence point of the two is only in function.

As time goes on, when service mesh products and API gateway products begin to penetrate each other, the relationship between them becomes ambiguous.

After the emergence of service mesh, how to choose an appropriate API gateway scheme for services based on service mesh is gradually put on the agenda, and the ability to reuse service mesh naturally becomes an exploration direction, and new API gateway products appear gradually. The idea is very direct

How to integrate east-west and North-South communication schemes?

One of the methods is to implement API gateway based on sidecar of service mesh, so as to introduce service mesh into north-south communication. We don’t go into details here. I quote a picture (thanks to Zhao Huabing) to explain the idea of this scheme

Discussion on the relationship between service mesh and API gateway

At this time, the relationship between service mesh and API gateway becomes interesting. Because of the introduction of sidecar in service mesh, there is a new solution to the above “philosophical problem”: API gateway can be really divided into two independent deployed physical entities, rather than two logical parts

  1. API gateway Ontology: realize the functions of API gateway besides accessing internal services;
  2. Sidecar: according to the standard practice of service mesh, we regard API gateway as a common service deployed in service mesh, and deploy sidecar for this service 1:1;

Discussion on the relationship between service mesh and API gateway

In this scheme, sidecar, which was originally used for service mesh, is used in API gateway instead of the original client access functions in API gateway. This scheme simplifies the implementation of API gateway, and realizes the reuse and integration of East-West and North-South communication capabilities. API gateway can focus more on the core functions of “API management”.

At this time, the relationship between service mesh and API gateway changes from “completely different” to “compatible”.

Companies that adopt this solution usually have service mesh products first, and then plan (or re plan) API gateway solutions based on service mesh products. For example, ant financial’s sofa gateway products are based on mosn, while community open-source products, Ambassador and GLOO, are based on Evoy.

The advantages of the above solutions are that API gateway and sidecar are deployed independently, with clear responsibilities and clear architecture. However, just as service mesh’s use of sidecar is questioned, one more hop will cause performance overhead and affect efficiency, so does API gateway’s use of sidecar: one more hop

The solution to the problem of “one more jump” is simple and crude. Based on sidecar, the function of API gateway is added. In this way, the API gateway ontology and sidecar are integrated into one again

Discussion on the relationship between service mesh and API gateway

What is the relationship between service mesh and API gateway after this step: is this the integration of service mesh / sidecar with API gateway or API gateway with service mesh / sidecar? The question is just like whether a zebra has black stripes on a white background or white stripes on a black background.

BFF: carry out integration to the end

The introduction of BFF (back end for front) will bring service mesh and API gateway to a more intimate level.

Let’s take a look at the regular BFF game

Discussion on the relationship between service mesh and API gateway

Here, an additional BFF layer is added between API gateway and internal services (including composite services and atomic microservices). Note that the working mode of BFF is very similar to that of composite services, in which multiple services are combined. But the difference is that:

  1. Composite service also belongs to the category of service, but it is composed of multiple services in the implementation mechanism, which still exposes a complete and standardized service;
  2. BFF is different. BFF, as the name suggests, is for the front end. One of the core goals is to simplify the access of the front end;
  3. For today’s topic, the most critical point is: BFF completely stops the traffic from outside, while composite services do not. API gateway can directly access atomic microservices;

After the integration of API gateway and sidecar, this will become very imaginative. Let’s first look at the scenario after the integration of API gateway and sidecar with BFF

Discussion on the relationship between service mesh and API gateway

Zoom in a little bit and look at API gateway and BFF separately

Discussion on the relationship between service mesh and API gateway

Notice that the traffic is received by API gateway and enters BFF. In this process, there are two sidecars in the request path:

  1. The common sidecar without API gateway function is deployed with BFF;
  2. After the integration of API gateway and sidecar, this is a “big sidecar with API gateway function” (or “special API gateway with sidecar function”): although it plays the role of API gateway, it still contains a complete sidecar in essence, which is equivalent to the sidecar of BFF;

So, the question comes: why put two sidecars in the process, and how about reducing it to one? We try to combine the two sidecars into one, remove the sidecar from BFF, and directly use the sidecar playing API gateway to BFF

Discussion on the relationship between service mesh and API gateway

Here’s the scenario:

  1. Traffic directly hits BFF (other network components may be hung in front of BFF to provide load balancing and other functions);
  2. The sidecar of BFF receives the traffic, completes the function of API gateway, and then transfers the traffic to BFF;
  3. BFF calls the internal service through sidecar (the same as when it is not merged);

Discussion on the relationship between service mesh and API gateway

Note that there is a key point here, which was specially noted before:“BFF fully closed external flow”。 This is a prerequisite because the original API gateway cluster no longer exists. If BFF fails to collect all the traffic, the API gateway will not be found. Of course, if you are willing to be a little bit more troublesome, it is also feasible to clearly define the services that need to be exposed to the outside world during deployment. It is also feasible to directly deploy sidecar with API gateway function on these services, but the management will be more complex than BFF mode.

In addition, in terms of deployment, according to the above scheme, we will find that: API gateway “disappears” – there is no longer an API gateway cluster with clear physical deployment. The conventional centralized gateway is integrated into each BFF instance in this scheme, so as to realize another important feature: decentralization.

The above integration scheme of service mesh and API gateway is not on paper.

In ant financial, we have carried out pioneering practice and exploration based on the idea of service mesh and API gateway integration + decentralization. Take Alipay mobile gateway as an example, in the past ten years, the gateway has gone from monomer to micro service, from centralization to centralization, from sharing. gateway.jar From packet to gateway mesh / sidecar based on mosn, the scheme evolved into such a scheme

Discussion on the relationship between service mesh and API gateway

Highly recommended reading: in the appendix, my colleague Jia Dao’s article “thinking and practice of ant financial API gateway mesh” has in-depth introduction and detailed description.


This paper summarizes the relationship between service mesh and API gateway. On the whole, the positioning and responsibilities of the two are quite different. However, in the specific implementation, there is a trend of integration: the early traditional way is code reuse at the class library level, and the latest trend is the combination of API gateway and sidecar.

The development of the latter is just in its infancy, including in ant financial. However, we believe that in the next two years, more similar product forms may appear in the community.

A supplementary introduction to “mosn” mentioned many times below:

Mosn is the abbreviation of modular open smart network. It is a network agent software developed by go language. It is open-source by ant financial and has been verified by hundreds of thousands of containers. As a cloud native network data plane, mosn aims to provide multi protocol, modular, intelligent and secure agent capabilities for services. Mosn can be integrated with any service mesh that supports the XDS API. It can also be used as an independent four or seven layer load balancing, API gateway, cloud native progress, etc.

  • GitHub:https://github.com/mosn/mosn
  • Official website: https://mosn.io

Appendix: references and recommended readings

For those who still have more ideas, please continue to read the following contents.

By time of publication:

  • The difference between API gateways and service mesh: 2020-02, guide architects to determine when to use API gateway and when to use service grid, author Marco Palladino, from Kong.
  • Do I need an API gateway if I use a service mesh?: 2020-01, author Christian Posta, for Chinese translation, please refer to Ma Ruofei. Do I need an API gateway after using service mesh. By comparing service mesh technology with API gateway, this paper focuses on the functional overlap and divergence of the two technologies, providing guidance for technology selection and implementation.
  • Ant Gateway API Gateway Mesh thinking and practice: 2019-12, the author, Jia Dao, introduces the development of ant gateway Alipay gateway and the origin of API Gateway Mesh. It strongly recommends reading. This article gives a very clear introduction to the practice of ant gold suit in Service Mesh and API Gateway fusion.
  • Identity crisis of API gateway: May 2019, author Christian Posta and translator Zhou Yuqing, describe the basic concepts of API gateway, such as the definition of API, the meaning of API management, the mode of API gateway, and the relationship between service grid and API gateway.
  • A long way to go: Practical Exploration of ant financial services service mesh: in October 2018, in my speech at qcon, I shared the exploration of ant financial services in the scope of inter service communication at that time, and proposed to reuse the ability of service grid in east-west communication to North-South communication. At that time, sofa gateway product based on sidecar was just developed.
  • API gateway vs service mesh: September 2018, written by Richard Li, CEO of datawire, is developing Ambassador API gateway. Ambassador is an open source API gateway product based on Evoy. This paper describes the views, differences and integration of service grid and API gateway.
  • Dreammesh (9) – API gateway: March 2018, this article was also written by me. After in-depth discussion with some friends in the service mesh community in early 2018, I recorded the scheme conceived at that time in dreammesh series of blog articles, especially the detailed discussion on the difference between API gateway and sidecar. At that time, the idea was not mature enough, but the general direction had already taken shape. Thanks to the students who participated in the discussion at that time!
  • Service mesh vs API gateway: October 2017, the original author kasun indrasiri and Zhao Huabing translated the Chinese version. The article is not long. It mainly compares the product functions of service grid and API gateway, and puts forward a way to integrate the two: calling downstream services through service grid in API gateway.
  • Application network functions with ESBs, API management, and now.. service mesh?: August 2017, written by Christian Posta, describes the relationship between service grid and ESB, message broker and API management. The content is very good and it is highly recommended to read (I have to vomit: the picture is too hot for eyes).

Recommended Today

Explain idea git branch backoff specified historical version

scene When I submitted this modification to the local and remote branches, I found that there were still some changes missing in this submission, or this modification was totally wrong, but I also pushed it to the remote repository. How to go back? problem How can the content that has been submitted to the repository […]