Redis clustering and simple use tutorial

Time:2019-10-23

preface

In fact, the purpose of Redis cluster building is also the purpose of cluster building. All clusters are mainly for solving one problem, horizontal expansion.

Before the concept of clustering, we used to scale up our hardware resources vertically, but scaling up quickly reached a point where the Cpu speed, memory size, and hard disk size of a single machine could not keep up with the demand, and the cost of scaling up machines vertically was quite high. The emergence of clustering is to enable multiple machines to work like one machine and realize the horizontal expansion of resources.

Redis is an in-memory database. When the data we want to store reaches a certain level, the memory of a single machine cannot meet our needs. Therefore, clustering is a good solution.

Describes the installation environment and version

Simulate 6 nodes with two virtual machines and 3 nodes with one machine to create 3 master and 3 salve environments.

Redis USES redis-3.2.4.

Both VMS are CentOS, one CentOS6.5 (IP:192.168.31.245) and one CentOS7 (IP:192.168.31.210).

The installation process

1. Download and unzip


cd /root/software
wget http://download.redis.io/releases/redis-3.2.4.tar.gz
tar -zxvf redis-3.2.4.tar.gz 

2. Compile and install


cd redis-3.2.4
make && make install

3. Copy redis-trib.rb to /usr/local/bin


cd src
cp redis-trib.rb /usr/local/bin/  

4. Create Redis node

First, create the redis_cluster directory on the 192.168.31.245 machine under the directory /root/software/redis-3.2.4;


mkdir redis_cluster  

In the redis_cluster directory, create directories named 7000, 7001, and 7002 and copy redis.conf to all three directories


mkdir 7000 7001 7002<br>cp redis.conf redis_cluster/7000
cp redis.conf redis_cluster/7001
cp redis.conf redis_cluster/7002  

Modify each of the three profiles as follows

Port 7000 // port 7000,7002,7003  
Bind native IP // the default IP is 127.0.0.1 and needs to be changed to an IP accessible by other node machines. Otherwise, the corresponding port cannot be accessed when creating the cluster and the cluster cannot be created
Daemonize yes //redis runs in the background
Pidfile /var/run/redis_70.pid //pidfile corresponds to 7000,7001,7002
Cluster enabled yes // cluster enabled yes // cluster enabled
Conf // cluster configuration file nodes_7000.conf // cluster configuration file is generated automatically on the first startup of 7000,7001,7002
Cluster -node-timeout 15000 // request timeout default 15 seconds, can be set
Appendonly yes //aof log on if necessary, then log every write

Then on another machine (192.168.31.210), the operation repeat the above three steps, just change the directory 7003, 7004, 7005, the corresponding configuration file can also be modified according to this rule

5. Start each node

Execute on the first machine
redis-server redis_cluster/7000/redis.conf
redis-server redis_cluster/7001/redis.conf
redis-server redis_cluster/7002/redis.conf
 
Execute on another machine
redis-server redis_cluster/7003/redis.conf
redis-server redis_cluster/7004/redis.conf
redis-server redis_cluster/7005/redis.conf

6. Check redis startup

A machine <br> ps-ef | grep redis
root  61020  1 0 02:14 ?  00:00:01 redis-server 127.0.0.1:7000 [cluster] 
root  61024  1 0 02:14 ?  00:00:01 redis-server 127.0.0.1:7001 [cluster] 
root  61029  1 0 02:14 ?  00:00:01 redis-server 127.0.0.1:7002 [cluster] 
 
netstat -tnlp | grep redis
tcp  0  0 127.0.0.1:17000    0.0.0.0:*     LISTEN  61020/redis-server 
tcp  0  0 127.0.0.1:17001    0.0.0.0:*     LISTEN  61024/redis-server 
tcp  0  0 127.0.0.1:17002    0.0.0.0:*     LISTEN  61029/redis-server 
tcp  0  0 127.0.0.1:7000    0.0.0.0:*     LISTEN  61020/redis-server 
tcp  0  0 127.0.0.1:7001    0.0.0.0:*     LISTEN  61024/redis-server 
tcp  0  0 127.0.0.1:7002    0.0.0.0:*     LISTEN  61029/redis-server
1
2
3
4
5
6
7
8
9
10
11
12
13
 
## another machine
ps -ef | grep redis
root  9957  1 0 02:32 ?  00:00:01 redis-server 127.0.0.1:7003 [cluster]
root  9964  1 0 02:32 ?  00:00:01 redis-server 127.0.0.1:7004 [cluster]
root  9971  1 0 02:32 ?  00:00:01 redis-server 127.0.0.1:7005 [cluster]
root  10065 4744 0 02:38 pts/0 00:00:00 grep --color=auto redis
netstat -tlnp | grep redis
tcp  0  0 127.0.0.1:17003   0.0.0.0:*    LISTEN  9957/redis-server 1
tcp  0  0 127.0.0.1:17004   0.0.0.0:*    LISTEN  9964/redis-server 1
tcp  0  0 127.0.0.1:17005   0.0.0.0:*    LISTEN  9971/redis-server 1
tcp  0  0 127.0.0.1:7003   0.0.0.0:*    LISTEN  9957/redis-server 1
tcp  0  0 127.0.0.1:7004   0.0.0.0:*    LISTEN  9964/redis-server 1
tcp  0  0 127.0.0.1:7005   0.0.0.0:*    LISTEN  9971/redis-server 1

7. Create the cluster

Redis officially provides the redis-trib.rb tool, which is in the SRC directory of the unzip directory. In the third step, it has been copied to the /usr/local/bin directory and can be used directly from the command line. Use the following command to complete the installation.


redis-trib.rb create --replicas 1 192.168.31.245:7000 192.168.31.245:7001 192.168.31.245:7002 192.168.31.210:7003 192.168.31.210:7004 192.168.31.210:7005

Among them, the first three IP: ports are the nodes of the first machine, and the remaining three are the second machine.

Wait, there’s a mistake. This tool is implemented in ruby, so you need to install ruby. The installation command is as follows:


yum -y install ruby ruby-devel rubygems rpm-build
gem install redis

Then run the redis-trib.rb command and the following prompt will appear:

Enter yes, and the following appears to indicate that the installation was successful.

  

8. Cluster validation

Connect the node of port 7002 of the cluster on the first machine and the node of port 7005 on the other machine redis-cli -h 192.168.31.245 -c -p 7002, add the parameter -c to connect to the cluster, because the above redis.conf changed bind to the IP address, so the -h parameter cannot be omitted.

Execute the command on node 7005set hello world , the execution result is as follows:

 

And then on another 7002 port, check out what key is hello,get hello  , the execution result is as follows:

Indicates that the cluster is functioning normally.

So just a little bit about how it works

In the design of redis cluster, decentralization and middleware are considered. That is to say, each node in the cluster is equal and equal, and each node keeps its own data and the state of the whole cluster. Each node is connected to all other nodes, and these connections remain active, which ensures that we only need to connect to any node in the cluster to get data from other nodes.

Instead of using traditional consistent hashes to allocate data, Redis clusters do so using another method called hash slots. The redis cluster is allocated 16384 slots by default. When we set a key, we will use CRC16 algorithm to extract modules to get the slot to which we belong, and then divide the key into the nodes of the hash slot interval. The specific algorithm is: CRC16(key) % 16384. So when we saw set and get during the test, we jumped directly to the node of port 7000.

The Redis cluster stores the data in a master node and synchronizes the data between the master and its salve. When the data is read, the data is also obtained from the corresponding master node according to the consistent hash algorithm. Only when a master dies does a corresponding salve node start to act as the master.

Note that there must be three or more primary nodes, otherwise the cluster will fail to be created, and the entire cluster will be unable to provide services when the number of surviving primary nodes is less than half of the total.

conclusion

The above is the whole content of this article, hope that the content of this article can bring some help to your study or work, if you have any questions you can leave a message to exchange.