Mongodb test

Time:2020-5-28

Mongodb common deployment architecture

  • Stand alone: development and testing
  • Replica sets: highly available
  • Sharding cluster: scale out

Why use sharding cluster?

  • Data capacity is increasing, access performance is decreasing, how to break?
  • How to support more concurrent users?
  • A single database has 10TB of data, and recovery takes 1-2 days. How to accelerate?
  • Geographic distribution data

How to solve fragmentation? Here is an example:

The situation is as follows:

  • 1 billion data in bank transaction list, overload operation
  • Transaction No. 0 – 1000000000

Solution 1

  • Divide the data into two parts and put them into two library physics.

    • Transaction No.: 0 – 500000000
    • Transaction No.: 500000000 – 1000000000

Solution 2

  • Divide the data into four parts and put them into four physical databases.

    • Transaction No.: 0 – 250000000
    • Transaction No.: 250000000 – 500000000
    • Transaction No.: 500000000 – 750000000
    • Transaction No.: 750000000 – 1000000000

Slice and cluster anatomy: routing node mongos

graph TD
A[Application] --> B(Driver)
    B --> C{mongos<br>123}
    C -- > | shared1 / config1| Z [replica set 1]
    B --> D{mongos}
    D -- > | shared2 / config2| y [replica set 2]
    B --> E{mongos}
    E -- > | shared3 / config3| x [replica set 3]
Routing node
  • Provide a single entrance to the cluster
  • Forward application request
  • Select the appropriate data node for reading and writing
  • Merge the return of multiple data nodes
  • Stateless
  • Mongos recommends at least 2
Configure (catalog) node
  • Provide cluster metadata storage
  • Mapping of partition data distribution
  • Normal replica set schema
Data node
  • Scale out by copy set
  • Maximum 1024 pieces
  • Data is not repeated between segments
  • All pieces work together

The characteristics of mongodb cluster

  • Transparent application without special treatment
  • Data auto equalization
  • Dynamic expansion without offline
  • Three ways to slice

The characteristics of mongodb cluster

  • Transparent application without special treatment
  • Data auto equalization
  • Dynamic expansion without offline
  • Three ways to slice

Data distribution mode of fragmentation and cluster

  • Based on scope
  • Based on Hash
  • Based on zone / tag

Data distribution mode of fragmentation Cluster – based on range (most commonly used)

Pros Cons
Good query performance of chip key range Data distribution may be uneven
Optimized reading Easy to get hot spots

Shard cluster data distribution mode – based on Hash

Pros Cons
Uniform data distribution and write optimization Low range query efficiency
  • Application: high concurrency scenarios such as log, Internet of things, etc

Partition cluster data distribution mode – Custom zone

Summary

  • Fragmentation clustering can effectively solve the performance bottleneck and system expansion problems
  • There is a lot of extra consumption and complex management in fragmentation. Try not to fragment
  • If you really want to use it, please study the next lecture carefully

Recommended Today

The way of nonlinear optimization

Mathematical knowledge 1、 Nonlinear functionLinear function is another name of a function of first degree, then nonlinear function means that the function image is not a function of a straight line.Nonlinear functions include exponential function, power function, logarithmic function, polynomial function and so on. 2、 Taylor expansion1. Taylor formula:Taylor’s formula is to add a_ The […]