Sentinel, a highly available component of redis service

Time:2021-1-27

Previously, we learned about the common data types of redis, and the use and description of related commands. For review, please refer tohttps://www.cnblogs.com/qiuhom-1874/p/13419690.htmlToday, let’s talk about sentinel, a highly available component of redis. First of all, let’s review the master-slave synchronization of redis. The main function of master-slave synchronization is to make the master’s data have real-time copies on other servers, which has the effect of backup. For the read-write of redis, the master-slave architecture can disperse the read requests to multiple slave servers, thus reducing the read request cost of a single redis It also improves the concurrency of redis read requests. Usually, for the sake of data consistency, once the slave server becomes a redis slave, the previous data on the server will be cleared, and then the data sent by the master will be applied to the memory, so as to achieve consistency with the master data. In addition, the slave is usually read-only, that is, SLA Ve can only perform read operation, and write operation will be rejected, so the write request is always completed by the master. Then the problem comes. In this master-slave replication architecture environment, if the Master goes down, the master will not be able to write data to redis. Obviously, we should solve this problem in time. How can we solve this problem? Is there such a component to help us monitor the master in real time? Once the master is down, a slave will be promoted and the new master will be elected. If there are other slaves in the original master, the other slaves will be subordinate to the new master. In addition, it should also enable the system to trigger an alarm when switching the master, so that the administrator can repair the broken master as soon as possible; yes, sentine L has the above functions. It can monitor the master node in the master-slave synchronous cluster, automatically fail over when the Master goes down, promote a slave as a new master, and then notify the administrator;

Sentinel is a distributed system. We can run multiple sentinels in one architecture. These sentinel processes use gossip protocols to receive information about whether the master is offline or not, and use agreement protocols to decide whether to perform automatic fault migration and which slave to choose as the new master. Each sentinel process will send messages to other sentinel processes, master and slave regularly to ensure that the other party is “alive”. If it is found that the other party has not received a response within the specified configuration time (configurable), it is temporarily considered that the other party has been offline, which is the so-called “subjective down”. If there is subjective downtime, there must be objective downtime. When the majority of sentinel processes in multiple sentinel processes make sdown judgment on the master, and communicate with each other through sentinel is master down by addr command, the master server offline judgment is obtained. This way is called “objective down”. The English name is objective down, referred to as odown. Through a certain voite algorithm, one of the remaining Slave Slave Slave server nodes is promoted to master server node, and then the relevant configuration is automatically modified, and the fail over is started.

Configure using sentinel

Environmental description

role IP address port
master 192.168.0.41 6379
slave01 192.168.0.42 6379
slave02 192.168.0.43 6379
sentinel01 192.168.0.41 26379
sentinel02 192.168.0.42 26379
sentinel03 192.168.0.43 26379

 

 

 

 

 

 

 

 

 

Architecture diagram

Tip: from the above architecture diagram, we can see that first we must have a master-slave architecture cluster, and then deploy sentinel to monitor the master-slave synchronous cluster;

Construction of redis master slave replication cluster

1. To install redis on 192.168.0.41/42/43, you can use Yum or compile. Please refer tohttps://www.cnblogs.com/qiuhom-1874/p/13378138.html

2. Configure the redis on 192.168.0.41/42/43 to listen on non local 127.0.0.1, and configure the redis on 42 / 43 to be subordinate to 192.168.0.41

  master

  slave01

  slave02

Tips: redis supports online configuration modification and saves the configuration to the configuration file; slaveof instruction is used to specify the IP address and port of redismaster, which indicates that redis is configured to the slave role of the corresponding master; config rewrite is used to save our configuration to the configuration file;

Check whether there are two slave nodes connected to the master on the master

Verification: write data on the master to see if it can be synchronized to two slaves in time?

Tip: you can see that the data is written on the master database, and the data on the master database can be synchronized in time from the slave database;

Configure sentinel to monitor master

Tip: the configuration of the three sentinels is the same. Here you need to specify the IP address and port of the master monitoring the master-slave synchronous cluster, as well as the number of valid legal votes. The number of valid legal votes refers to at least how many sentinels think the master is Generally, in this kind of rumor protocol, it is more than half of the cluster. If there are three sentinels, at least two subjectively think that the master is down, then the election of a new master will be triggered. If there are five, at least three will be triggered. If the master is configured with an authentication password, we need to specify the authentication password in sentinel;

Sentinel profile description

Bind: this instruction is used in the same way as bind in redis configuration file to specify the listening address of sentinel; it is not specified by default and monitors all available addresses of the local machine;

Protected mode: Specifies whether to turn on the protection mode;

Port: used to specify the listening port of sentinel; the default is 26379

Daemonize: used to specify whether sentinel is running as a daemons, yes means running as a background daemons, no means not running as a daemons, directly running in the foreground;

Pidfile: Specifies the PID file path;

Logfile: specify the log file path;

Dir: Specifies the working path of sentinel;

  sentinel monitor : used to specify the IP address and port of the monitoring master node and the number of valid legal votes; whereIt is a name given to the monitoring master, which can be written at will and plays the role of identification;It represents the quorum mechanism of sentinel cluster, that is, if there are at least quorum sentinel nodes judging the failure of the primary node at the same time, it will be considered as the true failure;

  sentinel auth-pass : specify the master authentication password. Usually, you need to set the password, and the password of master and slave should be the same;

  sentinel down-after-milliseconds : the configuration monitors how long the abnormal state of the master node of the specified cluster lasts before it is marked as “failure”;

  sentinel parallel-syncs : refers to the number of slave nodes that can be configured in parallel by sentinel in the process of failure;

  sentinel failover-timeout : sentinel must complete the fail over operation within the specified time, otherwise, it will be regarded as a failure of the fail over operation;

  sentinel notification-script : notification script, which is automatically passed multiple parameters;

After understanding the configuration file of sentinel, we start all three sentinels

  master

  slave01

  slave02

Tip: from the above information, we can see that the three sentinels monitor the IP address and port of the master. In fact, the configuration files of the three sentinels are the same;

View sentinel log

Tip: from the above log information, we can see that the master monitored by sentinel is 192.168.0.41:6379, and there are two slaves, 192.168.0.42:6379 and 192.168.0.43:6379;

View sentinel status

Tip: it prompts us to turn on the protection mode;

Turn off protection mode

Restart sentinel and check the sentinel status again

[[email protected] ~]# systemctl restart redis-sentinel.service 
[[email protected] ~]# ss -tnl
State      Recv-Q Send-Q        Local Address:Port                       Peer Address:Port              
LISTEN     0      511                       *:26379                                 *:*                  
LISTEN     0      511                       *:6379                                  *:*                  
LISTEN     0      128                       *:22                                    *:*                  
LISTEN     0      100               127.0.0.1:25                                    *:*                  
LISTEN     0      511                      :::26379                                :::*                  
LISTEN     0      128                      :::22                                   :::*                  
LISTEN     0      100                     ::1:25                                   :::*                  
[[email protected] ~]# redis-cli -h 192.168.0.41 -p 26379       
192.168.0.41:26379> info sentinel
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
sentinel_simulate_failure_flags:0
master0:name=mymaster,status=ok,address=192.168.0.41:6379,slaves=2,sentinels=3
192.168.0.41:26379> info clients
# Clients
connected_clients:3
client_longest_output_list:0
client_biggest_input_buf:0
blocked_clients:0
192.168.0.41:26379> CLIENT LIST
id=2 addr=192.168.0.42:59048 fd=14 name=sentinel-f60b324b-cmd age=38 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=ping
id=3 addr=192.168.0.43:37480 fd=15 name=sentinel-eada229c-cmd age=38 idle=1 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=publish
id=4 addr=192.168.0.41:36706 fd=16 name= age=32 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=32768 obl=0 oll=0 omem=0 events=r cmd=client
192.168.0.41:26379> 

Tip: from the above status information, we can see that the master monitored by sentinel is in the normal OK state, with two slave and three sentinel; for 192.168.0.41:26379, there are three client connections, two sentinel and one local machine; the three sentinel construction is completed;

Verification: shut down the master and see if sentinel will elect one of the two slave nodes as a new master? Do you want to point another slave to the new master again?

Viewing master slave synchronization information on slave01

Tip: the first view only tells us that the master is down, and the second view tells us that the current node is the master and has a slave node, which means that the fail over has been completed and slave01 has been promoted to a new master;

Check the master-slave information on 192.168.0.43 to see if it points to the new master?

Tip: if you look at the master-slave synchronization information on slave02, you can see that slave02 is subordinate to the new master;

View the sentinel log during a failover

Tip: from the above log information, after sdown to odown, the vote algorithm will be triggered to select the leader; then the original master will be demoted to slave, and the original salve attribute of the selected leader will be removed (slave of no one); then the new master will be promoted, and then the remaining slave will be reconfigured to the new master; finally, the master will be switched to start the new monitoring;

View the redis configuration file after the failure transfer

Tip: after the fail over redis.conf The master IP of the slaveof line in will be modified, sentinel.conf The sentinel monitor IP in will be modified. At the end of the sentinel configuration file, there will be information such as adding known slave and known sentinel;

Repair the old master and bring it back online

Tip: after the original master is started, it will automatically become the slave of the new master. This is mainly because sentinel changes the slave of its configuration file to the new master address when it fails over;

View master slave synchronization information on new master

Tip: if you do not restore the original master, you can only see one save when you view the master-slave synchronization information on the new master. After you start the original master, you can see that two slaves are online;

Recommended Today

JS function

1. Ordinary function Grammar: Function function name (){ Statement block } 2. Functions with parameters Grammar: Function function name (parameter list){ Statement block } 3. Function with return value Grammar: Function function name (parameter list){ Statement block; Return value; } Allow a variable to accept the return value after calling the function Var variable name […]