Take the last mile of the data center, why do you say that the data service API is the standard configuration of the data center?


Original link:Take the last mile of the data center, the data service API is the standard configuration of the data center

Video review:Click here

Courseware acquisition:Click here

1. Background of data service API construction

In the era of digital transformation, with the massive growth of new demands, the continuous iteration of new technologies, and the deepening of the process of "Internetization and digitization", more and more businesses have been migrated to the Internet, resulting in a large number of business interactions and external The demand for services and the demand for API interfaces are increasing day by day. How to quickly improve the ability of enterprises to open and share data is a key proposition for enterprises facing digital transformation.


Traditional methods, such as back-end developers writing in languages ​​such as Java or Python, generate API interfaces. The development cycle is too long and the operation and maintenance costs are too high, which can no longer meet the needs of enterprises. Enterprises often face many challenges in the process of digital transformation:


In order to solve these problems more, we need to determine the following goals in the process of enterprise opening and sharing data:

  • Build APIs quickly

  • System stability and data security

  • Easy to integrate and use

  • Authorized Delivery

  • Low cost operation and maintenance


2. Methodology of Data Service Platform Construction

Before sharing the data service platform construction methodology, let's take a look at the common data middle-end application architecture:


The data service layer is in the middle position in the overall application architecture of the data middle platform, and the results of the data calculation layer are shared with the data application layer in the form of data API. The data service layer has three main functions:

1. After the data has been integrated and calculated, it needs to be provided to products and applications for data consumption;

2. In order to have better performance and experience, build a data service layer and provide external data services through interface services;

3. Meet various complex data service requirements of applications (simple data query service, complex data query service, real-time data push)

In the process of providing services to the outside world by the data service layer, it has experienced“DWSOA”arrive“OneService”evolution process.


From the perspective of the "OneService" data service itself, it mainly solves the four problems of heterogeneous data sources, repeated construction, difficulty in audit operation and maintenance, and difficulty in understanding. Through the "OneService" service, thematic data services, unified and diversified data can be realized. Service, service target of cross-origin data service.

Therefore, if you want to build a complete data service platform, you need to have the following six elements:

  • Convenient development, with low-code development capabilities

  • Easy to manage, API management operations Visual query API

  • Easy to use, with normalized document description information

  • Security and stability, service call tracking monitoring, service usage auditing, authentication, etc.

  • Easy operation and maintenance, testing, correction, problem rule configuration

  • performance, load balancing, high concurrency

3. Building a data system based on OneService

After understanding the "OneService" theory, I will share with you how to build a data system based on OneService, mainly following the following steps:


● Step 1: API Definition

The definition of API includes: quick configuration parameters, selection and sorting fields, API type diversity, data preview, copy fields, etc.


The type of API includes four aspects: generating API, registering API, service grouping and service orchestration.



● Step 2: API release

The release of the API includes testing, submitting to the API gateway, publishing to the API market, and version management.


● Step 3: API call

API calls include data preview, API application, approval, downloading interface documents, and formal calls.


● Step 4: call monitoring

Business: In-depth analysis of the statistical data of API calls to obtain key information;

Technically: Through the analysis of the statistical chart of API calls, it can be found which APIs are the most popular; and which ones are almost ignored and should be eliminated;

Security: Monitor the calling IP and the number of calls, and trace the origin of the caller.


● Step 5: Data Security

Data security includes: unified authentication and authentication, transmission encryption, security group, role assignment, row-level permissions, call approval, etc.


The construction process of the above data service API is actually the implementation process of the data stack data service EasyAPI product developed by Kangaroo Cloud.

Data service (EasyAPI), an efficient enterprise-level data service product, generates and registers APIs through dual-mode visual configuration, quickly builds OneService data sharing services, forms an enterprise-level API market and API service management platform, and improves the efficiency of data opening and sharing.


At the same time, the product has the following characteristics:

  • Build quickly

Configuration is development, support 0 code, low code to quickly build API

  • High security

User authentication, monitoring, transmission encryption, API-level security policy, row-level permissions, role assignment, call application approval, limit on the number of call cycles, black and white lists

  • high flexibility

"Service Orchestration" can combine different APIs, support Python integration for data processing, support "condition judgment" nodes, and select eligible branches

  • Flexible configuration

Horizontal expansion of API gateways and caches

  • Low cost operation and maintenance

Adopt serverless architecture, only need to focus on the business logic of the API itself, and rarely consider the infrastructure such as the operating environment

Four, API implementation landing case

Next, we will share three actual cases of using customers to introduce how EasyAPI can help customers solve problems.

● Finance: A securities company applies data services


● School: a university application data service


● Retail: A network company applies data services


Kangaroo Cloud open source framework Dingding Technology Exchange qun (30537511), welcome students who are interested in big data open source projects to join in to exchange the latest technical information, open source project library address:https://github.com/DTStack