Mongodb sharding + replica set


1. Introduction

Mongodb is basically a big data storage tool of NoSQL. Internal data is stored in bson (binary JSON) in the concept of document.
All data storage concepts are inseparable from “fragmentation” (horizontal expansion of data to improve the performance of data storage and retrieval by means of separate storage), “replication” (high availability, including redundant backup, fault self-test, automatic recovery, read-write separation and other functions).
This article will not introduce the basic and distribution concepts of mongodb in detail. Readers can consult and learn by themselves.

2. Actual combat

2.1 version information

2.1.1 system version

# uname -a
Linux 1ddc8b08470a 4.9.49-moby #1 SMP Thu Sep 21 05:50:41 UTC 2017 x86_64 GNU/Linux

2.1.2 mongodb version

# mongo -version
MongoDB shell version v3.4.10
git version: 078f28920cb24de0dd479b5ea6c66c644f6326e9
OpenSSL version: OpenSSL 1.0.1t  3 May 2016
allocator: tcmalloc
modules: none
build environment:
    distmod: debian81
    distarch: x86_64
    target_arch: x86_64

2.2 creating sharding replica set rs0

# mkdir /data/log
# mkdir /data/db1
# nohup mongod --port 27020 --dbpath=/data/db1 --logpath=/data/log/rs0-1.log --logappend --fork --shardsvr --replSet=rs0 &

# mkdir /data/db2
# nohup mongod --port 27021 --dbpath=/data/db2 --logpath=/data/log/rs0-2.log --logappend --fork --shardsvr --replSet=rs0 &

2.3 replica set rs0 configuration

# mongo localhost:27020
> rs.initiate({_id: 'rs0', members: [{_id: 0, host: 'localhost:27020'}, {_id: 1, host: 'localhost:27021'}]})
>Rs.ismaster() # view master-slave relationship

2.4 creating sharding replica set RS1

# mkdir /data/db3
# nohup mongod --port 27030 --dbpath=/data/db3 --logpath=/data/log/rs1-1.log --logappend --fork --shardsvr --replSet=rs1 &
# mkdir /data/db4
# nohup mongod --port 27031 --dbpath=/data/db4 --logpath=/data/log/rs1-2.log --logappend --fork --shardsvr --replSet=rs1 &

2.5 replica set RS1 configuration

# mongo localhost:27030
> rs.initiate({_id: 'rs1', members: [{_id: 0, host: 'localhost:27030'}, {_id: 1, host: 'localhost:27031'}]})
>Rs.ismaster() # view master-slave relationship

2.6 create config replication set conf

# mkdir /data/conf1
# nohup mongod --port 27100 --dbpath=/data/conf1 --logpath=/data/log/conf-1.log --logappend --fork --configsvr --replSet=conf &
# mkdir /data/conf2
# nohup mongod --port 27101 --dbpath=/data/conf2 --logpath=/data/log/conf-2.log --logappend --fork --configsvr --replSet=conf &

2.7 replica set conf configuration

# mongo localhost:27100
> rs.initiate({_id: 'conf', members: [{_id: 0, host: 'localhost:27100'}, {_id: 1, host: 'localhost:27101'}]})
>Rs.ismaster() # view master-slave relationship

2.8 create route

# nohup mongos --port 40000 --configdb conf/localhost:27100,localhost:27101 --fork --logpath=/data/log/route.log --logappend & 

2.9 setting up slices

# mongo localhost:40000
> use admin
> db.runCommand({ addshard: 'rs0/localhost:27020,localhost:27021'})
> db.runCommand({ addshard: 'rs1/localhost:27030,localhost:27031'})
> db.runCommand({ enablesharding: 'test'})
> db.runCommand({ shardcollection: 'test.user', key: {name: 1}})

Through the above steps, you can directly connect the 40000 port mongdos to enjoy the slicing and replication functions.

3. Summary

The products born from the distributed background of basic big data are still very simple and convenient in the distributed cluster configuration. This is just a basic operation demonstration, and readers need to understand more functions.

  1. First, two sharding are configured and given the ability to copy.
  2. Then, configdb configuration management is configured, and the replication set function must also be provided.
  3. Finally, configure the route, specify configdb during route startup, and add sharding configured previously.
  4. The guest room only needs to connect route to enjoy sharding and replication transparently, which brings high performance and high availability. Note: if sharding is connected separately, the data generated will not be reflected in the route.

4. Reference: