Full cycle data control to escort “express big data +”


With the vigorous development of e-commerce and online retail, China’s postal express industry has developed very rapidly and has become one of the country’s important strategic infrastructure and social organization systems. “Receiving express” is known as the first of the “seven new things”. In the first half of 2020, the business volume of national express service enterprises totaled 33.88 billion, a year-on-year increase of 22.1%.

The rapid growth of business volume means that data risk also increases. In the process of building a digital economy with data as the key element, how to ensure data security has become a problem faced by the postal express industry. A provincial postal center puts forward several core demands for its current data management difficulties.

Core appeal

Change database access mode

At present, personnel access to the database is realized through direct connection to the database, which has a high risk of data leakage. The postal center hopes to change the original direct database connection to indirect query, so that the cloudquery platform becomes an isolation zone between users and databases, and all database operations of users reach the platform first.

User rights management

The types of personnel in contact with the database in the postal center are complex. The situation that several people share one account makes the personnel do not match the authority, which is easy to lead to authority abuse. Once a problem occurs, it is impossible to accurately trace the source. Therefore, the postal center needs a perfect authority and audit system to classify and classify users and data in advance, so as to realize one-to-one correspondence between users and accounts. At the same time, improve the granularity of user authority control, strengthen the audit of user account authorization, expiration and cancellation, and improve the level of authority control while meeting the normal use of users.

Sensitive data management

In order to prevent the leakage and dissemination of data information, it is necessary to desensitize sensitive data. According to the user’s query requirements, the desensitization strategy is flexibly formulated to ensure no data leakage while not affecting the user’s use.

Operation process control

Open the authorization raising process for low authority users. When a user needs to operate without permission, he can apply for permission. After approval, he can open the corresponding permission for the user. When the permission expires, the system will automatically recover it, so as to reduce the workload of managers and avoid time and information deviation caused by manual operation.

High availability

In order to ensure that the service functions of the platform are not affected, it is necessary to support high availability configuration. The cloudquery platform supports two high availability modes: HA and cluster, so as to increase the concurrency of stand-alone services as much as possible on the basis of maintaining the stability of the server itself.


According to the needs of a provincial postal center, we put forward the idea of using cloudquery mainly as a data operation center and personnel control center, strengthen the current database security access control mechanism, and solve the problems caused by the direct database, such as the inconsistency between people and accounts, the difficulty in controlling permissions, and the difficulty in locating the responsible person in case of problems.

Data operation center

Database access control

Cloudquery supports ordinary users to access the database through access connection configuration, and changes the original direct connection to indirect query, so as to strengthen the database security access control mechanism. All users’ database operations will first arrive at the platform, pass the platform system permissions, and then be sent by the platform to the database side for execution. After the database operation is completed, the returned results will be directly pushed to the platform and finally displayed to the user.

Database operation support

As an isolation zone between users and databases, cloudquery undertakes all users’ operations on the database. Cloudquery supports multiple types of data sources. The supported database object types include tables, views, synonyms, stored procedures, packages, sequences, triggers, indexes, dblinks, jobs, etc. At the same time, DML, DDL and other database operation types are supported.

Abnormal operation management

For abnormal operations in the platform, cloudquery can display the database operation error log in detail, distinguish between unauthorized operations and SQL syntax errors, and clearly locate the cause of the error through the error log.

Personnel control center

User authority control

According to the needs and permissions of different types of personnel, cloudquery divides user management permissions into three-level system roles: system administrator, connection administrator and ordinary user. The system administrator can manage system level resources, such as menu, organizational structure, position, etc., and specify the connection administrator. The connection administrator manages data resources, can allocate user permissions under the connection, and can configure sensitive resources and high-risk resources. Ordinary users can only connect and access, and ordinary users can be divided into auditor, monitor and other roles for authorization.

At present, cloudquery database operation permission types are mainly divided into operation permission, time permission and restricted resource permission. Operation permissions include data source operation permissions, data export permissions, filtering permissions, row filtering permissions, execution times permissions and execution rows permissions. Time permission can limit the user’s execution time. Ordinary users cannot access the unauthorized time period. Restricted resource permission means that some resources can be set as high-risk resources.

Through permission division, users’ illegal access and unauthorized operations can be blocked. For operations without permission, users can only apply for authorization, and normal operations can be carried out after approval.

Operation process control

The operation process mainly includes data revision process, connection access authorization, database desensitization authorization, result set export authorization, database operation authorization, high-risk resource authorization and UI authorization.

1. Data revision process

When batch data changes are required in data operation, the user can initiate the data correction process, attach the script or script file to be used for data correction, or choose to attach the rollback script in case of error. However, the data correction does not have the ability to rollback. You need to click the rollback button in the executed process to generate a new data correction process.

2. Connection access rights

When a new user starts to use the platform, there are no available connections. When he needs the access permission of a connection, he needs to select the connection access permission. After the permission is successfully raised, he can have the access permission of the connection. However, by default, he only has the query permission, not the permission to modify data and table structure.

3. Database desensitization right
When a user joins a new connection, he / she is spontaneously restricted by sensitive resources. When a user needs to view sensitive resources temporarily, he / she can apply for desensitization and authorization. After the authorization is successfully raised, he / she can access sensitive resources.

4. Result set export and weighting
By default, cloudquery users do not have export permission when joining the connection. If they need export permission, they need to initiate an export authorization work order. After being approved by the connection administrator, they can log in again to export the corresponding database elements.

5. Database operation authorization
By default, the platform user only has query permission. When other permissions to edit data are required, you can select database operation authorization.

6. High risk resource empowerment
When you need to edit a high-risk resource, you need to apply for the authorization of high-risk operation first, and then you can edit the resource.

7. Right raising
The audit, process, monitoring and other menus on the page are UI permissions, but when you need to view audit data, you can select UI audit authorization, and the user can log in to the system again to see the corresponding menu.

Sensitive data management

Cloudquery can desensitize the query results. The platform desensitization function only involves data query and does not involve data change. The way of data not landing in the process of technical implementation not only ensures the data security of the database, but also does not affect the normal operation of the business system.

In the desensitization module, cloudquery provides different desensitization methods for different types of data objects, and can customize desensitization rules. Different desensitization policies can also be enabled for different users, or applied to users with the same needs through the same desensitization policy template.

High availability module

Cloudquery supports the active and standby mode composed of two physical servers. When one of the physical servers goes down, the platform functions will not be affected. At the same time, it supports high availability failover. When one server or application service is abnormal, it can switch to another service without affecting the platform service function.

So far, the problems of personnel management and control, database operation management and sensitive data management faced by a provincial postal center have been solved. From the perspective of the overall organizational structure, cloudquery provides a full link interference tracking and protection mechanism according to the internal data flow, controls data and personnel throughout the entire data life cycle, and solves the problem of difficult data control from a global perspective.

Official website address:https://www.bintools.cn/cloud…

Full cycle data control to escort

Recommended Today

Swift advanced (XV) extension

The extension in swift is somewhat similar to the category in OC Extension can beenumeration、structural morphology、class、agreementAdd new features□ you can add methods, calculation attributes, subscripts, (convenient) initializers, nested types, protocols, etc What extensions can’t do:□ original functions cannot be overwritten□ you cannot add storage attributes or add attribute observers to existing attributes□ cannot add parent […]