All architects have an architecture blueprint in their mind, which is that they design the system architecture from the overall perspective, and then reasonably apply and arrange corresponding resources for implementation, so as to finally reduce costs and increase efficiency for the enterprise.
In the initial planning, what needs to be done, at what time, at what stage, at what time, and what system needs to be done, build the infrastructure from 0 to 1 to provide basic services and components for subsequent businesses. In general, it is more in line with the practical application of small and medium-sized companies.
1. Development framework
Most students are in contact with business development and pay most attention to the development framework. From the perspective of infrastructure department, we need to provide corresponding development framework for business development students, such as supporting their RPC (remote call).
We will simply layer the business architecture, including gateway layer, business logic layer and data access layer. All kinds of micro services and combinations of business are done according to the stratification of roles. Later, we only need to understand the original intention of the layered architecture.
As the only external entrance of the system, the gateway is the middle layer between the client and the server. It mainly provides the functions of routing request, authentication, monitoring, caching, current limiting and so on. Just like the concert security check, like Zhang Xueyou’s previous concerts, several fugitives can be caught in each concert, that is, he can be identified at the gateway (security check).
- Unified access: intelligent routing, AB test, gray scale test, load balancing, disaster recovery processing, log embedding point;
- Intelligent routing is actually to request the corresponding service according to the user’s request path;
- AB test, collect data through comparative test, and finally analyze and evaluate to determine which is the better option according to the actual effect;
- Gray scale test: designate some resources to test, and gradually expand the number of probationers, so as to find and correct the problems in time;
- Load balancing refers to balancing the load (work task) and allocating it to multiple operation units for operation to alleviate the access pressure;
- Disaster recovery processing, relieve the nodes with too much pressure and distribute them to the node services with less pressure;
- Log embedding point to obtain daily traffic and service call time.
- Flow monitoring: current limiting treatment, service degradation, etc;
- For current limiting processing, I’m worried that the traffic is too large, resulting in too much pressure on the service layer. Sometimes an activity is too popular. At the beginning, the number of interface visits increases sharply; Some malicious users or robots brush the interface, so they have to limit the current in many ways, or they will be accessed and exploded;
- Service degradation mainly refers to that when the server pressure surges, some services and pages are handled strategically or in other simplified ways according to the actual business situation, so as to release server resources to ensure the smooth and efficient operation of core functions.
- Security protection: authentication processing, monitoring, etc., machine network isolation;
- Authentication processing refers to verifying whether the user has the right to access the system. Generally, there are four methods: http basic authentication, session cookie, token authentication and OAuth (open authorization);
- Monitoring: the monitoring of API gateway supports viewing the monitoring charts of region, grouping and API. The indicators of the monitoring chart mainly include the number of requests, traffic, delay and httpstatuscode;
- Machine network isolation mainly refers to that the gateway interacts with the service through the intranet and interacts with the user requesting the IP of the external network.
1.2 business logic layer
The business logic layer is the part of the system architecture that embodies the core value. Its focus is mainly on the system design related to business requirements, such as the formulation of business rules and the implementation of business processes. That is to say, it is related to the domain logic that the system responds to. In many cases, the business logic layer is also called the domain layer.
- Connecting the preceding and the following: the business logic layer plays a key role in the architecture. It is located between the data access layer and the presentation layer, and plays a connecting role in data exchange;
- Extractable and replaceable: because the layer is a weak coupling structure, the dependence between layers is downward, the bottom layer is “ignorant” to the upper layer, and changing the design of the upper layer has no impact on the bottom layer. If the idea of interface oriented design is followed in layered design, this downward dependency should also be a weak dependency. Therefore, without changing the interface definition, the ideal layered architecture should be a “drawer” architecture that supports extractable and replaceable;
- Extensible support: because of this, the design of business logic layer is particularly critical for an architecture that supports extensibility, because it plays two different roles. For the data access layer, it is the caller; For the presentation layer, it is the callee. The relationship between dependency and dependency is tangled in the business logic layer. How to realize the decoupling of dependency is the task left to the designer in addition to realizing the business logic.
1.3 data access layer
Simply put, it is to realize the operations of select, insert, update and delete on the data table. If you want to add the elements of ORM, it will include the mapping between objects and data tables and the persistence of object entities.
- Dal: data access layer, in short, refers to SQL statements and other operations on the database through DAL;
- Main responsibilities: read data and transfer data.
It mainly provides more functions, more infrastructure and more basic capabilities for business development and rapid business iteration.
2.1 service management platform manager
The service management platform is to access, manage and schedule the services based on service technology, component-based development and independent operation in a unified way, so as to realize the integration and management between heterogeneous services, and finally realize the orderly, normal and continuous operation of all running services.
- Quick deployment: classify the service as a response independent functional service according to function and behavior. Each service can be deployed and run independently, and then provide a unified service API interface. Flexible service configuration can quickly meet and deliver the needs of users;
- Service management support: according to different product modules, it can realize the functions of service registration, service dependency management, service resource directory, server routing, etc. at the same time, it can also realize the service measurement function and achieve a more comprehensive unified service management function;
- Consistent service call: it can make unified management on the operation of the accessed service, and realize the scheduling functions such as degradation and fuse in the process of service operation according to the service call relationship, so as to finally achieve the orderly, efficient and normal operation of the service.
2.2 request tracking platform
The link tracking component restores a distributed request into a call link by means of data representation, embedding, transmission, collection, storage, display and other technical means in the way of code intrusion or non intrusion in the microservice application, and displays the call situation of a distributed request, such as the time consumption on each service node, the specific machine on which the request arrives, the request status of each service node and so on.
2.3 log query platform
- Keep the site and investigate relevant problems;
- Estimate the problems that may occur in the near future according to the existing state;
- Commercial operation of data;
2.4 configuration center config
Centralized control of configuration files. It is very troublesome to maintain the configuration files of our modules by ourselves. A platform level component is provided to support the maintenance of configuration items, such as adding, modifying and deleting, so there is no need to log in to the physical machine.
2.5 three dimensional monitoring platform monitor
Microservices are split. Our modules have changed from one to many to hundreds of thousands. The governance, monitoring and control of services should be very strict, or we don’t know if there is a problem. Therefore, a three-dimensional monitoring platform is needed, that is, the monitoring has many dimensions and full perspectives. Various situations of the whole system can be monitored from multiple dimensions and perspectives. It can not only monitor the process, but also monitor servers, machines, clusters, calls, etc.
2.6 message center MQ
When calling, RPC alone is not enough. Sometimes pure asynchronous interaction is required.
2.7 task scheduling platform job
Task scheduling system is the core component of the data platform. In daily data processing, it is important to run some businesses regularly, such as regularly importing new data from the database to the data platform, and exporting the data processed by the data platform to the database or file system. The task scheduling system is responsible for when a task runs, when it ends, and correctly handle the dependencies between tasks. The correctness of the process is the core. There is also a resource scheduling system, which pays more attention to the allocation of system physical resources, such as the use of CPU and memory. The core is to maximize the utilization of system resources.
2.8 Registration Center
When RPC calls are more and more, more flexible, better and more elegant service discovery and service registration mechanisms are needed.
3. Basic components
3.1 distributed transaction middleware transaction
Mainly the solution of “final consistency”.
- The business is vertically split into multiple modules. Each module uses different databases (possibly different database types), and a business spans multiple business modules. For example, in e-commerce business, payment when placing an order, inventory deduction and point distribution services, as well as multiple databases of transactions, payments and accounts involved in payment.
- After the horizontal splitting of the database, the business transaction involves multiple data fragments. For example, in the transfer business from a to B, the accounting data of a and B are in different sub databases.
3.2 database middleware JDBC
The main function of database middleware is to shield application developers from the challenges of read-write separation and database and table separation, and hide the underlying implementation details, so that developers can operate data like single database and single table.
4. Storage layer
4.1 curing kV storage kV
Redis is a key value storage system, which is similar to memcached. It supports relatively more types of stored values, including string, list, set, Zset and hash. These data types support push / pop, add / remove, intersection, union, difference and richer operations, and these operations are atomic. On this basis, redis supports sorting in different ways. Like memcached, data is cached in memory to ensure efficiency. The difference is that redis will periodically write the updated data to the disk or write the modification operation to the additional record file, and realize master-slave synchronization on this basis.
4.2 curing relational storage MySQL
MySQL is the most popular relational database management system. In terms of web application, MySQL is one of the best RDBMS (relational database management system) application software.
4.3 newsql database tidb
Tidb is an open-source distributed relational database independently designed and developed by pingcap company. It is an integrated distributed database product that supports both online transaction processing and online analytical processing (HTAP). It has important features such as horizontal expansion or reduction, financial level high availability, real-time HTAP, cloud native distributed database, compatibility with MySQL 5.7 protocol and MySQL ecology. The goal is to provide users with one-stop OLTP (online transactional processing), OLAP (online analytical processing) and HTAP solutions. Tidb is suitable for various application scenarios such as high availability, strong consistency, high requirements and large data scale.
4.4 cache system CODIS
CODIS is a distributed redis solution. For the upper application, there is no obvious difference between connecting to the CODIS proxy and connecting to the native redis server (unsupported command list). The upper application can be used as a stand-alone redis. The lower layer of CODIS will handle request forwarding, non-stop data migration and other work. Everything behind is transparent to the front client, You can simply think that what is connected behind is a redis service with unlimited memory.
4.5 object storage
Object storage, also known as “Object-Oriented storage”, is a general term used to describe the methods of solving and processing discrete units, which are called objects. English is object based storage. Now many cloud manufacturers also directly call it “cloud storage”. For example, Alibaba cloud calls its own object storage service OSS, Huawei cloud OBS, Tencent cloud cos, qiniu kodo, baidu BOS and Netease Nos.
5. Basic services
5.1 short domain name service tinyurl
Three major problems have been solved. First of all, the link becomes shorter, and there will be more editable text when posting on the platform with limited content length. Secondly, we often need to turn the link into a QR code to share with others. If it is a long chain, the QR code is dense and difficult to identify, so this problem does not exist in the short chain. Finally, links are too long to be automatically recognized as hyperlinks on some platforms.
5.2 instant messaging service im
Instant messaging is the most popular way of communication at present, and a variety of instant messaging software are emerging in endlessly; Service providers also provide more and more rich communication service functions.
5.3 push service
The basic idea of push technology is to change the browser to actively query information into the server to actively send information. The server sends a batch of data, and the browser displays these data while ensuring the connection with the server. When the server needs to send a batch of data again, the browser displays the data and remains connected. Later, the server can still send batch data, and the browser continues to display the data, and so on.
6. Operation and maintenance support
6.1 asset management platform CMDB
Generally, one object and one code technology (bar code, two-dimensional code, etc.) is used to conduct all-round and accurate supervision on the physical assets from purchase, collection, cleaning, inventory, borrowing, return, maintenance to scrapping, and truly realize the consistency between accounts and materials in combination with reports such as asset classification statistics.
6.2 publishing system publish
The information release system is composed of server, network, player and display device. The information of the server is sent to the player through the network (WAN / LAN / private network are applicable, including wireless network), and then the player combines audio and video, pictures, text and other information (including playing position and playing content) and transmits it to the audio and video input acceptable to the terminal display device to form the playback of audio and video files, This forms a set that can send all server information to the terminal through the network.
6.3 work order system
The work order system can be divided into two categories: one is the system to convey the work tasks of the internal departments of the enterprise; The second is the system specially used for after-sales installation and maintenance. This is the software that sends work orders to field personnel for door-to-door maintenance. It is used to record, process and track the completion of a work.
6.4 virtualization platform docker
In computer, Virtualization (English: virtualization) is a resource management technology, which abstracts and transforms various physical resources of computer, such as server, network, memory and storage, so as to break the inseparable obstacles between physical structures, so that users can apply these resources in a better way than the original configuration. The new virtual part of these resources is not limited by the erection mode, region or physical configuration of existing resources. Generally speaking, virtualization resources include computing power and data storage.
In the actual production environment, virtualization technology is mainly used to solve the overcapacity of high-performance physical hardware and the reorganization and reuse of old and old hardware with low capacity, and make transparent the underlying physical hardware, so as to maximize the full use of physical hardware and physical resources. There are many kinds of virtualization technologies, such as software virtualization, hardware virtualization, memory virtualization, network virtualization, desktop virtualization, service virtualization Virtual machines, etc.
7. MIS support
7.1 authority management system auth
- In terms of control strength, authority management can be divided into two categories:
- Function level authority management;
- Data level authority management.
- From the perspective of control direction, permission management can also be divided into two categories:
- Obtain data from the system, such as querying orders and customer data;
- Submit data to the system, such as deleting orders and modifying customer data.
7.2 user authentication system SSO
Each application system provides users with different services based on their own rights and interests. However, each application system provides users with different services based on their own rights and interests.