and gdpr of EU1Time is getting closer. From May 25, 2018, any organization that fails to meet the new regulations will face a fine of up to 4% of global revenue, or EUR 20 million – regardless of the fine – and any further data processing activities will be subject to a potential call off risk. Therefore, whether you are a member of the EU or not, as long as you are processing the data of EU citizens in any way, you must comply with the gdpr treaty.
In other words, some regulations should not be regarded as imposing. Instead, it offers an opportunity for more active organizations to change their relationships with customers in the digital economy.
In the next blog series, we’ll learn more about this rule and what it means to us:
- Part 1 will provide an introduction to gdpr – this will cover the basic principles and key measures specified.
- Part 2 explores what gdpr means for our data platform.
- In part 3 today, we’ll discuss how mongodb’s products and services will support our business.
- The fourth part discusses how gdpr will help customers to implement gdpr, and provides several cases for study.
If you can’t wait for all four parts and want to know all of them now, you can download the full gdpr: impact to your data management landscape white paper.
How mongodb helps meet gdpr requirements
Although gdpr, HIPAA, PCI-DSS and other data protection regulations specify the unique requirements of specific regions, industries or applications, all directives have basic requirements, including:
- Restrict data access through predefined permissions and roles
- Loss, accidental disclosure or malicious damage of personal data
- Separation of responsibilities is required when accessing and processing data
- Use a database to record user, administrator, and application activities
These requirements inform mongodb’s security architecture department to provide best practices for implementing a secure and compatible data management platform.
Using the advanced security features provided by mongodb enterprise advanced and mongodb atlas cloud database services, mongodb has a wide range of functions to implement the data discovery, defense and detection requirements required by gdpr.
Identification of personal data
There are many ways to check the contents of a database. The most common approach is to query the database and extract all records to identify the tables and rows that contain user data (called collections and documents in mongodb). However, this approach also requires the2A lot of manual analysis is done to track which data is stored while imposing processing overhead on the database itself.
Mongodb provides a simple method on compass (mongodb’s GUI). Compass enables users to browse their data intuitively. By extracting a group of documents from the collection, compass provides a graphical view of its mongodb architecture, thus minimizing database overhead and presenting the results to users immediately.
Using mongodb compass’s schema visualization enables users to quickly browse their schema to understand the frequency, type, and range of fields in each dataset. Users don’t need to be familiar with mongodb query language — powerful professional queries can be built by clicking on the interface, opening the discovery and data loss prevention process beyond developers and database administrators to data protection supervisors and other business users.
Based on compass, mongodb’s query language and rich auxiliary indexes enable users to query and analyze data in a variety of ways. Data can be accessed through a single key, range, text search, graphics, and geospatial queries, returning responses in milliseconds. Data can dynamically enrich elements such as user identity, location, and last access time to add context to personal identity information (PII), providing behavioral insights and actionable customer intelligence. The complex query itself is executed in the database without the use of additional analysis frameworks or tools, and avoids the delay of ETL processes necessary to move data between operations in traditional enterprise architectures and analysis systems.
Retention of personal data
By using a specific TTL (time to live) index, administrators can automatically expire EU citizen data in the database. By configuring the required retention period based on the date field in the document (that is, the date the user data was collected or last accessed), mongodb will use an automatic background process to delete documents after the expiration date. For example, the database every 60 seconds.
Compared with the application level implementation expiration code, the database must be scanned periodically to find records that need to be deleted,Mongodb TTL indexObviously, it simplifies the implementation process of data expiration policy and reduces the cost of database.
The access control of database can be divided into two different stages
- Authentication, to confirm the identity of the client accessing the database.
- Authorization, management, the operations that customers have the right to perform after they have access to the database, such as reading data, writing data, performing management and maintenance connection activities, etc.
Mongodb provides a variety of authentication methods, allowing the most appropriate method to meet the requirements of different environments. Authentication can be managed from the database itself or through integration with external authentication mechanisms.
MongoDB AtlasadoptSCRAM IETF RFC 5802The standard enforces in database authentication. becauseMongoDB AtlasThe service runs on a public cloud platform, and it also implements additional security controls to reduce the risk of unauthorized access. By default, Atlas clusters will not be allowed direct access from the Internet. Each atlas cluster is deployed in a virtual private environment (for example,AWSorGCP Virtual Private Cloud，Azure Virtual Network）The private environment is configured to not allow inbound access by default. In addition, IP whitelists can be used to restrict network access to the database (i.e., unless its IP address has been added to the applicableMongoDB AtlasThe atlas AWS VPC peer-to-peer option allows peer-to-peer organizations’ atlas networks to connect to their ownAWS VPCNetwork to ensure that network traffic does not traverse the public Internet, but uses internal private networks.
MongoDB Enterprise AdvancedIt is also allowed to useSCRAMAuthentication, andLDAP、Kerberosorx.509 PKIOther integration options for certificates.
It is widely used by many organizationsLDAPTo standardize and simplify the management of large numbers of users through internal systems and applications. In many cases,LDAPIt is also used as a centralized authority for user access control to ensure that internal security policies comply with corporate and regulatory guidelines. adoptLDAPintegrate,MongoDB Enterprise AdvancedUsers can be authenticated and authorized directly against the existing LDAP infrastructure to take advantage of a centralized access control architecture.
MongoDB Enterprise AdvancedIt also supports the use ofKerberosService for authentication. adoptLDAPandKerberos，MongoDB Enterprise AdvancedProvide for the use ofMicrosoft Active DirectoryAuthentication support.Active DirectoryDomain controller verification inWindowsMongodb users and servers running in the network make use of centralized access control again.
Through supportx. 509 certificateMongodb can also be integrated with certification authority (CA) to support encryption and authentication between users and nodes, so as to reduce the risk in password or key file.
Check the documentation in the authentication section to learn aboutMongoDB Enterprise AdvancedMore information on the different mechanisms available.
Role based access control (RBAC) support is provided for more than ten predefined roles for normal users and database administrator permissions. useMongoDB Enterprise AdvancedThese functions can be further customized through user-defined roles, so that administrators can assign fine-grained permissions to clients according to their corresponding data access and processing requirements. To simplify account configuration and maintenance, roles can be delegated across teams to ensure consistent policies are implemented in specific data processing functions within the organization.
In addition to the above certification,MongoDB Enterprise AdvancedIt also supported the adoption ofLDAPAuthorization. This will cause theLDAPExisting user permissions in the server are mapped to mongodb roles without re creating users in mongodb itself. This integration strengthens and simplifies access control by implementing centralized processes.
Check the authorization section of the document to learn more about role-based access control in mongodb.
Pseudonym and encryption
As described in part 2, pseudonyms and encryption of data are designed to prevent the identification of any particular person in the event that an unauthorized party accesses the data.
Mongodb provides multiple levels of pseudonyms. With read only views, mongodb can automatically filter specific fields, such as including citizens when querying the databasePIIField of. Instead of querying the collection directly, clients can only be granted access to a specific predefined view of the data. The permissions granted to the view are specified separately from those granted to the underlying collection, so clients with different access rights can be granted different data views.
Read only views allows you to include or exclude fields, mask field values, filter, pattern conversion, group, sort, restrict, and connect data from multiple collections. Read only views are transparent to the application accessing the data, and do not modify the underlying raw data in any way.
MongoDB Enterprise AdvancedYou can also configure the log modification feature to prevent potentially sensitive information, such as personal identifiers, from being written to the database’s diagnostic logs. Developers and DBAs who may need access to logs for database performance optimization or maintenance tasks can still view metadata, such as error or operation code, line numbers, and source file names, but cannot view any personal data related to database events.
Encryption protects transmitted and unused data and allows only authorized access. If an unauthorized user has access to a network, server, file system, or database, the encryption key can still be used to protect the data.
Support transport layer security (TLS) allows clients to connect to mongodb through encrypted network channels, thus protecting the data in transit. In addition, mongodb encrypts static data in persistent storage and backup.
useMongoDB AtlasManaged database service. TLS is the default value and cannot be disabled. Traffic from the client to atlas and between the atlas cluster nodes is authenticated and encrypted. For customers using disk and volume encryption services from public cloud providers,Encryption-at-restIs an available cost free option.
MongoDB Enterprise AdvancedIt also provides encryption storage engine to protect static data as an integral part of the database. By encrypting the database files on the disk locally, administrators can reduce the management and performance overhead of external encryption options, while providing an additional level of defense. Only employees with appropriate database credentials can access encrypted personal data. Accessing database files on the server does not expose any stored personal information.
The storage engine encrypts each database with a separate key. Mongodb recommends that the encryption key be rotated and replaced periodically, and that the key can be rotated without database downtime by performing a rolling restart of the replica set. useKey Management Interoperability Protocol(kmip) service, the database file itself does not need to be re encrypted, thus avoiding the performance cost of key rotation.
Refer to the documentation for more information about mongodb encryption. For flexibility and disaster recovery, protecting service availability and recovering from events that cause data corruption or loss, mongodb provides fault tolerance for system failures, as well as backup and recovery tools for disaster recovery.
Using local replication, mongodb maintains multiple copies of data in the replica set. A replica set is a fully self-healing cluster distributed over multiple nodes to eliminate a single point of failure. In the event of a node failure, replica failover is fully automated, and no administrator intervention is required to restore database availability.
The number of replicas in the mongodb replica set is configurable: a larger number will provide increased data availability and prevent database downtime (for example, in the case of multiple machine failures, rack failures, data center failures, or network partitions). Replica sets also provide operational flexibility, providing a way to upgrade hardware and software without taking the database offline. Replica set members can be deployed within and across physical data centers and cloud regions, providing resilience to regional failures.
Data can be affected by many unforeseen events: database or its infrastructure failure, user error, malicious attack or application error. Through the backup and recovery strategy, administrators can restore business operations by rapidly recovering data, thus enabling organizations to meet regulatory and compliance obligations.
MongoDB Enterprise AdvancedandMongoDB AtlasThe operational tools provided as part of the management database service enable us to maintain database backup continuously. If mongodb fails, the most recent backup is only a small part of the operating system to minimize data loss. The tool provides point in time recovery of partitioned cluster and cluster wide snapshot of distributed cluster. These operations can be performed without interrupting the database service. The administrator can restore the database to the required time, fast and safe time. Automated driven recovery allows a fully configured cluster to be redeployed directly from a database snapshot with just a few clicks, speeding up service recovery time.
We can access the Ops Manager documentation and [mongodb atlas documentation] https://docs.atlas.mongodb.co… Learn more aboutMongoDB Enterprise AdvancedFor more information on backup and restore.
Data Sovereignty: data transmission outside the EU
To support data sovereignty requirements,MongoDB ZonesIt can precisely control where personal data is physically stored in the cluster. Zones can be configured to automatically “partition” (partition) data based on user location – enabling administrators to isolate EU citizen data to physical facilities only identified as gdpr compliant. If there is a change in the EU’s policy of storing data in a specific region, updating the range of the shard key enables the database to automatically move personal data to an alternative area.
In addition to geographically specific applications, regions can accommodate a range of deployment scenarios, such as supporting a tiered storage deployment model for data lifecycle management, or segmenting data through application functions or customers.
We can learn more about mongodb zone sharding from the documentation.
Please note that,MongoDB AtlasThe service of can be configured to any one of multiple supported cloud service provider regions, but it must run within that region. A single cluster cannot currently span multiple regions.
Active monitoring of all components in an application platform is always a best practice. System performance and availability depend on finding and resolving potential problems in a timely manner before posing problems to users. In terms of memory and CPU utilization, the sudden unexpected peak may be attacked, among other factors. If the administrator alerts in real time, the attack can be mitigated.
MongoDB Enterprise AdvancedandMongoDB AtlasThe operation tools provided by management database services provide deep operational visibility for database operations. Mongodb’s operational tools include charts, custom dashboards and automatic alerts. It tracks more than 100 key database and system health indicators, including Operation counters, memory, CPU, storage consumption, replication, node status, open connections, queues, and so on. These indicators are safely reported to the management interface, processed, summarized, reminded and visualized in the browser, so that administrators can easily track the health status of mongodb. Network metrics can also be pushed to application performance management platforms such asAppDynamicsandNew Relic）To support centralized visibility in the global IT domain.
Custom alerts can be generated when key metrics are out of range. These alerts can be sent via SMS and email, or integrated into existing event management and collaboration systems, such asPagerDuty，Slack，HipChatTo proactively alert potential problems and help prevent interruptions or violations.
Operational tools also enable administrators to push upgrades and patches to the database without downtime. useMongoDB AtlasDatabase services, which automatically apply patches to eliminate the overhead of manual operator intervention.
By maintaining audit trails, you can capture personal data and database configuration changes for each client accessing the database, and provide compliance and court analysis logs for data controllers and regulators.
MongoDB Enterprise AdvancedThe audit framework records all access and operations performed on the database, including:
- Management operations, such as adding, modifying, deleting database users, mode operation and backup.
- Authentication and authorization activities, including failed attempts to access personal data.
- Read and write the database.
Administrators can build and filterMongoDB Enterprise AdvancedAudit trail of any operations. They can capture all activities, or just a small number of operations, according to requirements set by data controllers and auditors. For example, you can record and audit the identity of the user accessing a specific document, as well as any changes made to the database during the session. Learn more from mongodb enterprise advanced audit documentation.
MongoDB AtlasThe managed database service provides audit logs of administrative operations, such as deployment and expansion of clusters, and adding or removing users from Atlas groups. Database log access is also provided, which can be used by the controller to track the user’s connection to the database.
Services help teams create a secure database environment
The training requirements of Article 47 (article 2n) of the “binding company rules” are clearly specified in the GDP text,
“Appropriate data protection training for personnel who access personal data on a continuous or regular basis”.
Mongodb provides a wide range of training and consulting services to help customers implement the best security practices:
- Mongodb security course is a free 3-week online training course at mongodb University.
- Mongodb University also provides a range of public and private training for developers and operations teams covering best practices in using and managing mongodb.
- Mongodb global consulting services offers a range of packages including health checks, production readiness assessments and visits to professional consulting engineers. Mongodb consulting engineers work directly with our team to guide development and operations to ensure that skills are passed on to employees.
The second part is the summary
This is the third part of the blog series. In the fourth part, we will study how gdpr helps customer experience and provide several case studies.
Remember to download the full gdpr: impact to your data management landscape white paper today.
For a comprehensive description of the provisions, roles and responsibilities of GDP, readers are advised to refer to the text of the official journal of the European Union (EU) 2016 / 679 (International Labour Organization (EU) 2016 / 679) and how the rules of legal interpretation apply to their organizations. In addition, in order to effectively implement the functions described in this blog series, it is essential to ensure that the database is implemented according to the instructions and instructions detailed in the mongodb security documentation. Readers should consider engaging mongodb global consulting services to assist in the implementation.
This article is translated from: https://www.mongodb.com/blog/post/gdpr-impact-to-your-data-management-landscape-part-2
I am Shanghai xiaopang [miracle young], focusing on the Devops of mongodb, mysql, redis and other open source databases, embracing open source and accepting fees.
Original address of Shanghai xiaopang https://segmentfault.com/u/shanghaixiaopang/articles
You are welcome to comment.
Every Friday, please look forward to Shanghai xiaopang.
If Xia Yuhe is still waiting for me by Daming Lake, I will not change.
- GDPR: General Data Protection Regulation. ↩
- Schema: schema, which refers to the schema of mongodb. Compared with the data structure in relational database, model. A kind of