MOOCS 2021-01-29 redis6 live notes – Part 2 (multi thread / hash slot / cluster)

Time:2021-3-6

<div style=”margin: 0px auto; color: rgb(85, 85, 85); max-width: 23cm; padding: 1em;”><div style=”margin: 0px; padding: 0px;”>
<div style=” padding:8px; “> < H3 style =” font family: & quot; Pt sans narrow & quot;, sans serif; font weight: 700; line height: 1.1em; color: rgb (133, 153, 0); font size: 1.4em; “> IO multithreading</h3>

<p>In the past, redis was a single thread model, which is actually a multiplexing mechanism. We know that the first wave of multiplexing is 6. We have talked about it in the course of architects. Then, netty also has it, and those who have read the relevant courses of teachers should know it. I won’t say much about it here. </p>

<p>Redis6 began to have IO read-write multithreading, except that the execution of user commands is the same as that of earlier versions, which is single threaded, so thread safety. </p>

<p>Let’s take a look at the old version of single threading

MOOCS 2021-01-29 redis6 live notes - Part 2 (multi thread / hash slot / cluster)</p>

<p>First, read the client’s commands, then execute them, and then write them back to the client. This is the execution of a group of commands. Because of single thread safety, they will execute them in groups. </p>

<p>Their read and write commands and execution commands are executed in one thread, which is called the main thread in redis6. </p>

<p>Here we can review the thread model of netty’s reactor, which is multiplexing.

MOOCS 2021-01-29 redis6 live notes - Part 2 (multi thread / hash slot / cluster)

You can think of this single thread as a person. He is the receptionist of the bar and club. After receiving them at the door, he will receive them one by one. </p>

<p>We can open it redis.conf Configuration file, take a look:

MOOCS 2021-01-29 redis6 live notes - Part 2 (multi thread / hash slot / cluster)

By default, it’s the same as the old version. If you want to use multithreading, you can turn it onio-threadsIt is to set the number of multithreads. After multithreading is turned on, the overall performance is higher than that of single thread. </p>

<p>io-threadsThe maximum number of settings is not recommended to exceed 8, and there is little room for improvement. In addition, the number of threads is also related to the hardware configuration of the server. For example, if 4-core 8g is used, it is recommended to set 2 or 3. If 8-core 8g is used, it is recommended to set 6 or 7, because it must be reserved to cover everything and guarantee everything. This is the same as the configuration of nginx. </p>

<p>Suppose nowio-threadsSet it to 4, then his model is as follows:

MOOCS 2021-01-29 redis6 live notes - Part 2 (multi thread / hash slot / cluster)</p>

<p>So here, the read user’s command and the execution command are executed in one thread, and then the write operation is executed in multiple threads. Here, if it is compared to a bank, thenRead operationIt’s like the security guard at the door, taking your temperature,implementIt’s the lobby manager in the store, who will take you to get the number, manyWrite operationIt’s the window clerk. </p>

<p>You can refer to the following figure:

MOOCS 2021-01-29 redis6 live notes - Part 2 (multi thread / hash slot / cluster)

Just now, for example, there is only one person in the reception. At this time, there is only one person in the reception. He only brings the guests in at the door. There will be a number of special receptionists to handle the requests of these guests. </p>

Can I set multithreading for read operation? </h3>

<p>MOOCS 2021-01-29 redis6 live notes - Part 2 (multi thread / hash slot / cluster)

If you want the read operation to be multithreaded, thenio-threads-do-readsIt can be turned on and set to yes. At this time, his model is the following picture:

MOOCS 2021-01-29 redis6 live notes - Part 2 (multi thread / hash slot / cluster)</p>

<p>But the official note, this multi-threaded read open is not significant, will not help a lot.

MOOCS 2021-01-29 redis6 live notes - Part 2 (multi thread / hash slot / cluster)

At this time, we will introduce the example just now. When there are more guests, business is good, and the number of receptionists at the door is increased, then the reception and reception on both sides are handled by multiple employees. </p>

<p>Let’s make a point here. Multithreading is only for data reading and writing and protocol parsing. The real client command execution is single thread, so it is thread safe. </p>

< H3 style = “font family: & quot; Pt sans narrow & quot;, sans serif; font weight: 700; line height: 1.1em; color: rgb (133, 153, 0); font size: 1.4em;” > redis cluster principle</h3>

<p>MOOCS 2021-01-29 redis6 live notes - Part 2 (multi thread / hash slot / cluster)</p>

<p>In fact, as long as middleware is involved, there must be the concept of cluster. On the one hand, for high availability and on the other hand, for horizontal expansion, let’s talk about the redis cluster. </p>

<p>In fact, a long time ago, the early version of redis did not have the concept of cluster. You need to rely on some middleware to implement cluster, such as CODIS and tweakroxy. The concept of redis cluster was introduced in 3.0. It is a self-contained distributed storage solution. It is a decentralized cluster, called redis cluster. It is a mode that multiple master-slave nodes co-exist. Generally, the classic mode is 3 master and 3 slave. Of course, multi master and multi slave can also be used. The master is responsible for reading and writing requests and the maintenance of the whole cluster information, while the slave only copies the data and status information of the corresponding master node. </p>

<h3 style=”font-family: &quot;PT Sans Narrow&quot;, sans-serif; font-weight: 700; line-height: 1.1em; color: rgb(133, 153, 0); font-size: 1.4em;”>Redis Cluster</h3>

<p>There are two things to note about clustering</p>

<ul>
<li>Previously, we talked about the master-slave mode, which can do read-write separation. Although the redis cluster has three masters and three slaves, it also has the concept of master-slave, but we will not do read-write separation. The read-write is all handled by the master, and the data will be synchronized to slave. If the cluster does read-write separation, on the one hand, it is meaningless, on the other hand, slave has no fault tolerance mechanism, which needs to be noted. </li>
<li>In addition, a single node redis has 16 dB by default, but in the cluster mode, these dB are integrated. There is no concept of DB library, it is a vast ocean. </li>
</ul>

<p>In terms of single node, master-slave and sentinel, redis can always be said to be a single database. There is an upper limit on the amount of data storage. How much memory your server node has, then this is the upper limit of storage. Once the storage limit is reached, redis will implement the self elimination mechanism of cache key. Obviously, this is not a good way to deal with massive data. Even if you buy one t memory, it will reach the storage bottleneck one day. Therefore, we have to consider a horizontal expansion mechanism for any distributed system, and this redis cluster can do it. While expanding the capacity, it also meets the high availability mechanism. </p>

<p>At the same time, after using the cluster, the data is actually partitioned, and multiple master nodes can provide read-write services, so the response ability of the whole cluster is better than that of the original single node. Moreover, in the cluster mode, when any node fails, the cluster can continue to provide external services, and the primary node has the function of fail over. </p>

< H3 style = “font family: & quot; Pt sans narrow & quot;, sans serif; font weight: 700; line height: 1.1em; color: rgb (133, 153, 0); font size: 1.4em;” > traditional horizontal expansion</h3>

<p>In fact, the traditional early redis expansion scheme is based on the business layer, in which redis can be divided to expand capacity and cluster isolation.

We have talked about the master-slave of redis before. Take this as an example. Of course, we can also use sentry. We can manually expand the capacity of redis. Let’s take a look at the following figure:

MOOCS 2021-01-29 redis6 live notes - Part 2 (multi thread / hash slot / cluster)</p>

<p>The three master-slave redis in the figure above is actually composed of three master-slave redis. When users set or get, they first need to hash the key. The hash value is modulo to the number of nodes. The modulo value is that the data is routed to a master-slave database, and then set and get operations are performed on the master-slave database. The hash principle is actually the same as the IP of nginx_ Hash is the same, and so is the database branch. The purpose is to find the final storage location to store the value or value according to the result of key hash. This is actuallyHash modulus。</p>

<p>The hashing and addressing process needs to be encapsulated in the business layer, but the essential purpose is to achieve horizontal expansion. Assuming that each master-slave capacity is 8g, the current total capacity is 24g. How to expand the capacity, just add the machine directly.

MOOCS 2021-01-29 redis6 live notes - Part 2 (multi thread / hash slot / cluster)</p>

<p>In this case, the expansion will bring a disadvantage, that is, hash modulus. Once a node goes down or increases nodes, it is necessary to re hash and modulus. This will affect all the cached data. At this time, all the data will be rerouted, which will have a great impact. How can we solve this problem? You can use a consistent hash algorithm, that is, a small part of the data is affected, which we mentioned in the architecture class. </p>

< H3 style = “font family: & quot; Pt sans narrow & quot;, sans serif; font weight: 700; line height: 1.1em; color: rgb (133, 153, 0); font size: 1.4em;” > consistent hash</h3>

<p>MOOCS 2021-01-29 redis6 live notes - Part 2 (multi thread / hash slot / cluster)

First of all, it will have a domain, which is very long, with the 32nd power of 2 minus 1. The hash is not only for the key, but also for the server node. Here we are redis. If it is a server cluster, the same is true.

MOOCS 2021-01-29 redis6 live notes - Part 2 (multi thread / hash slot / cluster)</p>

< H3 style = “font family: & quot; Pt sans narrow & quot;, sans serif; font weight: 700; line height: 1.1em; color: rgb (133, 153, 0); font size: 1.4em;” > redis cluster hash slot algorithm</h3>

<p>The traditional way above is very old. I’ve seen it once. Let’s get to know it. Now we all use redis Cluster. In this mode, redis can have multiple master nodes, and each master node can correspond to 1 ~ n slave nodes. In this way, a multi master and multi slave cluster mode will be formed. In this mode, it is no longer necessary to process in the business layer as mentioned above. Redis itself helps us to complete the access process, and we can do it without paying too much attention to the business process Direct use and application. It is based on Hash slot algorithm to store value, no matter which algorithm, in fact, the essence is to solve the problem of data distribution. </p>

<p>The redis cluster actually splits the data. Because it consists of multiple masters, it splits the data and has a hash Slot algorithm, also known as hash slot, stores some data in each slot. The slot range is 0 ~ 16384. Redis has a CRC16 algorithm to access the key. The obtained value is modeled at 16384 to determine which slot the key should be accessed in. The essence is the same as that of the server node, except redis The slot algorithm of cluster is more sophisticated and cheaper. </p>

<p>Take three masters and three slaves as an example. At this time, the total number of slots will be equally divided, so that there will be slots in each master node. At present, if there are three, that is {0.. 5461} {5462.. 10922} {10923.. 16383}. This value range can be seen after the cluster is built. </p>

<p>If at this time, the cluster environment increases or decreases the number of nodes, and the slot can be re migrated or merged, then the cache key in the slot actually exists. In this way, node downtime or new addition will not cause cache loss. This is equivalent to the demolition of my house. When I get to my new home, I will carry my personal belongings, furniture and electrical appliances together. Things follow people. Things are cache keys, people are slots, and server nodes are redis instances, which are more flexible. It’s not that when your house is gone, your things are gone.

MOOCS 2021-01-29 redis6 live notes - Part 2 (multi thread / hash slot / cluster)

Question: in the process of moving, can people live in the new house?

Of course, you can’t move right away. It’s time-consuming and you need to wait, so your furniture and appliances are not available during the waiting period. The same is true for redis slots. When the system is down, the slots that need to be migrated are not available. There will be a short migration process. Only after the migration is completed can read and write services be provided to users. </p>

<p>Hash slot diagram:

MOOCS 2021-01-29 redis6 live notes - Part 2 (multi thread / hash slot / cluster)</p>

<p>As shown in the following figure: if the master in the middle is down, the slot will be migrated to the slave

MOOCS 2021-01-29 redis6 live notes - Part 2 (multi thread / hash slot / cluster)</p>

<p>If a new master node is added and reallocated, some slots will be migrated, and the cache data will remain unchanged. Although slots will be changed to other master nodes, the data will still be hashed with fixed slots. </p>

< H3 style = “font family: & quot; Pt sans narrow & quot;, sans serif; font weight: 700; line height: 1.1em; color: rgb (133, 153, 0); font size: 1.4em;” > lattice shop</h3>

<p>If you still can’t understand it, take another example: chestnut, grid shop. Each grid belongs to a seller. The seller entrusts each grid to some shops and runs it on their behalf. If the shops close down and the grid still exists, he can take each grid with him. The goods in each grid are the data in the redis cluster. It won’t change, but the shell has changed It’s just a shop. </p>

< H3 style = “font family: & quot; Pt sans narrow & quot;, sans serif; font weight: 700; line height: 1.1em; color: rgb (133, 153, 0); font size: 1.4em;” > redis cluster construction</h3>

<p>Time relationship, we only talk about the principle here, the operation part, we provide the MOOCS network Notes document, very simple, you can go to have a look on the line.

Link address: https://www.imooc.com/article… </p>

<p>However, there are some points for attention in the cluster, which are summarized as follows:</p>

<ol>
<li><p>Reading and writing all join the cluster in master and slave, and data synchronization will be carried out. When connecting any master or slave node in the cluster to read and write data, it will be routed to a master node according to the key hash. Slave does not provide read-write service, it only synchronizes data. </p></li>
<li><p>Closing any master node will lead to some write failures, because the slave node can not perform write operations, and there may be a small number of failures during the upgrade from slave to master. </p></li>
<li><p>Shutting down the slave has no effect on the entire cluster</p></li>
<li><p>When a master node and all the slave nodes under his command all hang up, our cluster will enter the fail state and will not be available. Because the slot is incomplete. </p></li>
<li><p>If more than half of the masters in the cluster fail, no matter whether they have a corresponding slave or not, the cluster will enter the fail state because it is unable to vote. </p></li>
<li><p>If any master in the cluster is down and there is no slave in this master. The cluster is not available. (same as 3)</p></li>
<li><p>All the masters in the cluster participate in the voting process. If more than half of the master nodes communicate with the master node timeout, the current master node will be considered dead. </p></li>
<li><p>The election is only for all slave elections in a master, not for all full slave elections. </p></li>
<li><p>After the original master restores the connection, it will become the slave server of the new master. Due to master-slave synchronization, the client’s write command may be lost. Redis is not a strong consistency, because of the master-slave characteristics, so the last part of the data will be lost. This is also in line with the cap theory. </p></li>
<li><p>The cluster only realizes the primary node’s fail over; the secondary node will only be offline when it fails, and will not fail over. Therefore, when cluster is used, read-write separation technology is generally not used, because the failure of slave node will lead to the unavailability of read service and the poor availability. So don’t do read-write separation in the cluster. </p></li>
</ol>

<p>It should be noted that in order to ensure the integrity of the cluster, the cluster can only be launched when all 16384 slots have been allocated. Moreover, if the primary node fails over and is in the process of failing over, the slot slot of the original primary node will not be in any node, and the cluster will be offline, and the client cannot call it. Cap principle. </p>

< H3 style = “font family: & quot; Pt sans narrow & quot;, sans serif; font weight: 700; line height: 1.1em; color: rgb (133, 153, 0); font size: 1.4em;” > redis cluster data migration</h3>

< H3 style = “font family: & quot; Pt sans narrow & quot;, sans serif; font weight: 700; line height: 1.1em; color: rgb (133, 153, 0); font size: 1.4em;” > how to expand a single node redis with existing data into a cluster</h3>

<p>When building a cluster, the package clears the existing AOF and RDB files in the node. If it is a single node, how can it be expanded to a cluster? And to ensure that the data is not lost. In fact, the principle is to save the RDB or AOF file, import it, and allocate it through slot. The specific steps are as follows:

MOOCS 2021-01-29 redis6 live notes - Part 2 (multi thread / hash slot / cluster)

As shown in the figure above, the principle is slot migration. The whole process is blocked, and the website service does not provide external services. If you use a self built cluster, then the operation and maintenance costs are very high. Generally, it is better to use cloud redis. Even Alibaba cloud will have network interruption when the configuration changes.

MOOCS 2021-01-29 redis6 live notes - Part 2 (multi thread / hash slot / cluster)</p>
</div>
</div>

< / div > < IO multithreading

In the past, redis was a single thread model, which is actually a multiplexing mechanism. We know that the first wave of multiplexing is 6. We have talked about it in the course of architects. Then, netty also has it, and those who have read the relevant courses of teachers should know it. I won’t say much about it here.

Redis6 began to have IO read-write multithreading, except that the execution of user commands is the same as that of earlier versions, which is single threaded, so thread safety.

Let’s take a look at the old version of single threading
MOOCS 2021-01-29 redis6 live notes - Part 2 (multi thread / hash slot / cluster)

First, read the client’s commands, then execute them, and then write them back to the client. This is the execution of a group of commands. Because of single thread safety, they will execute them in groups.

Their read and write commands and execution commands are executed in one thread, which is called the main thread in redis6.

Here we can review the thread model of netty’s reactor, which is multiplexing.
MOOCS 2021-01-29 redis6 live notes - Part 2 (multi thread / hash slot / cluster)
You can think of this single thread as a person. He is the receptionist of the bar and club. After receiving them at the door, he will receive them one by one.

We can open it redis.conf Configuration file, take a look:
MOOCS 2021-01-29 redis6 live notes - Part 2 (multi thread / hash slot / cluster)
By default, it’s the same as the old version. If you want to use multithreading, you can turn it onio-threadsIt is to set the number of multithreads. After multithreading is turned on, the overall performance is higher than that of single thread.

io-threadsThe maximum number of settings is not recommended to exceed 8, and there is little room for improvement. In addition, the number of threads is also related to the hardware configuration of the server. For example, if 4-core 8g is used, it is recommended to set 2 or 3. If 8-core 8g is used, it is recommended to set 6 or 7, because it must be reserved to cover everything and guarantee everything. This is the same as the configuration of nginx.

Suppose nowio-threadsSet it to 4, then his model is as follows:
MOOCS 2021-01-29 redis6 live notes - Part 2 (multi thread / hash slot / cluster)

So here, the read user’s command and the execution command are executed in one thread, and then the write operation is executed in multiple threads. Here, if it is compared to a bank, thenRead operationIt’s like the security guard at the door, taking your temperature,implementIt’s the lobby manager in the store, who will take you to get the number, manyWrite operationIt’s the window clerk.

You can refer to the following figure:
MOOCS 2021-01-29 redis6 live notes - Part 2 (multi thread / hash slot / cluster)
Just now, for example, there is only one person in the reception. At this time, there is only one person in the reception. He only brings the guests in at the door. There will be a number of special receptionists to handle the requests of these guests.

Can I set multithreading for read operation?

MOOCS 2021-01-29 redis6 live notes - Part 2 (multi thread / hash slot / cluster)
If you want the read operation to be multithreaded, thenio-threads-do-readsIt can be turned on and set to yes. At this time, his model is the following picture:
MOOCS 2021-01-29 redis6 live notes - Part 2 (multi thread / hash slot / cluster)

But the official note, this multi-threaded read open is not significant, will not help a lot.
MOOCS 2021-01-29 redis6 live notes - Part 2 (multi thread / hash slot / cluster)
At this time, we will introduce the example just now. When there are more guests, business is good, and the number of receptionists at the door is increased, then the reception and reception on both sides are handled by multiple employees.

Let’s make a point here. Multithreading is only for data reading and writing and protocol parsing. The real client command execution is single thread, so it is thread safe.

Redis cluster principle

MOOCS 2021-01-29 redis6 live notes - Part 2 (multi thread / hash slot / cluster)

In fact, as long as middleware is involved, there must be the concept of cluster. On the one hand, for high availability and on the other hand, for horizontal expansion, let’s talk about the redis cluster.

In fact, a long time ago, the early version of redis did not have the concept of cluster. You need to rely on some middleware to implement cluster, such as CODIS and tweakroxy. The concept of redis cluster was introduced in 3.0. It is a self-contained distributed storage solution. It is a decentralized cluster, called redis cluster. It is a mode that multiple master-slave nodes co-exist. Generally, the classic mode is 3 master and 3 slave. Of course, multi master and multi slave can also be used. The master is responsible for reading and writing requests and the maintenance of the whole cluster information, while the slave only copies the data and status information of the corresponding master node.

Redis Cluster

There are two things to note about clustering

  • Previously, we talked about the master-slave mode, which can do read-write separation. Although the redis cluster has three masters and three slaves, it also has the concept of master-slave, but we will not do read-write separation. The read-write is all handled by the master, and the data will be synchronized to slave. If the cluster does read-write separation, on the one hand, it is meaningless, on the other hand, slave has no fault tolerance mechanism, which needs to be noted.
  • In addition, a single node redis has 16 dB by default, but in the cluster mode, these dB are integrated. There is no concept of DB library, it is a vast ocean.

In terms of single node, master-slave and sentinel, redis can always be said to be a single database. There is an upper limit on the amount of data storage. How much memory your server node has, then this is the upper limit of storage. Once the storage limit is reached, redis will implement the self elimination mechanism of cache key. Obviously, this is not a good way to deal with massive data. Even if you buy one t memory, it will reach the storage bottleneck one day. Therefore, we have to consider a horizontal expansion mechanism for any distributed system, and this redis cluster can do it. While expanding the capacity, it also meets the high availability mechanism.

At the same time, after using the cluster, the data is actually partitioned, and multiple master nodes can provide read-write services, so the response ability of the whole cluster is better than that of the original single node. Moreover, in the cluster mode, when any node fails, the cluster can continue to provide external services, and the primary node has the function of fail over.

Traditional horizontal expansion

In fact, the traditional early redis expansion scheme is based on the business layer, in which redis can be divided to expand capacity and cluster isolation.
We have talked about the master-slave of redis before. Take this as an example. Of course, we can also use sentry. We can manually expand the capacity of redis. Let’s take a look at the following figure:
MOOCS 2021-01-29 redis6 live notes - Part 2 (multi thread / hash slot / cluster)

The three master-slave redis in the figure above is actually composed of three master-slave redis. When users set or get, they first need to hash the key. The hash value is modulo to the number of nodes. The modulo value is that the data is routed to a master-slave database, and then set and get operations are performed on the master-slave database. The hash principle is actually the same as the IP of nginx_ Hash is the same, and so is the database branch. The purpose is to find the final storage location to store the value or value according to the result of key hash. This is actuallyHash modulus

The hashing and addressing process needs to be encapsulated in the business layer, but the essential purpose is to achieve horizontal expansion. Assuming that each master-slave capacity is 8g, the current total capacity is 24g. How to expand the capacity, just add the machine directly.
MOOCS 2021-01-29 redis6 live notes - Part 2 (multi thread / hash slot / cluster)

In this case, the expansion will bring a disadvantage, that is, hash modulus. Once a node goes down or increases nodes, it is necessary to re hash and modulus. This will affect all the cached data. At this time, all the data will be rerouted, which will have a great impact. How can we solve this problem? You can use a consistent hash algorithm, that is, a small part of the data is affected, which we mentioned in the architecture class.

Consistent Hashing

MOOCS 2021-01-29 redis6 live notes - Part 2 (multi thread / hash slot / cluster)
First of all, it will have a domain, which is very long, with the 32nd power of 2 minus 1. The hash is not only for the key, but also for the server node. Here we are redis. If it is a server cluster, the same is true.
MOOCS 2021-01-29 redis6 live notes - Part 2 (multi thread / hash slot / cluster)

Redis cluster hash slot algorithm

The traditional way above is very old. I’ve seen it once. Let’s get to know it. Now we all use redis Cluster. In this mode, redis can have multiple master nodes, and each master node can correspond to 1 ~ n slave nodes. In this way, a multi master and multi slave cluster mode will be formed. In this mode, it is no longer necessary to process in the business layer as mentioned above. Redis itself helps us to complete the access process, and we can do it without paying too much attention to the business process Direct use and application. It is based on Hash slot algorithm to store value, no matter which algorithm, in fact, the essence is to solve the problem of data distribution.

The redis cluster actually splits the data. Because it consists of multiple masters, it splits the data and has a hash Slot algorithm, also known as hash slot, stores some data in each slot. The slot range is 0 ~ 16384. Redis has a CRC16 algorithm to access the key. The obtained value is modeled at 16384 to determine which slot the key should be accessed in. The essence is the same as that of the server node, except redis The slot algorithm of cluster is more sophisticated and cheaper.

Take three masters and three slaves as an example. At this time, the total number of slots will be equally divided, so that there will be slots in each master node. At present, if there are three, that is {0.. 5461} {5462.. 10922} {10923.. 16383}. This value range can be seen after the cluster is built.

If at this time, the cluster environment increases or decreases the number of nodes, and the slot can be re migrated or merged, then the cache key in the slot actually exists. In this way, node downtime or new addition will not cause cache loss. This is equivalent to the demolition of my house. When I get to my new home, I will carry my personal belongings, furniture and electrical appliances together. Things follow people. Things are cache keys, people are slots, and server nodes are redis instances, which are more flexible. It’s not that when your house is gone, your things are gone.
MOOCS 2021-01-29 redis6 live notes - Part 2 (multi thread / hash slot / cluster)
Question: in the process of moving, can people live in the new house?
Of course, you can’t move right away. It’s time-consuming and you need to wait, so your furniture and appliances are not available during the waiting period. The same is true for redis slots. When the system is down, the slots that need to be migrated are not available. There will be a short migration process. Only after the migration is completed can read and write services be provided to users.

Hash slot diagram:
MOOCS 2021-01-29 redis6 live notes - Part 2 (multi thread / hash slot / cluster)

As shown in the following figure: if the master in the middle is down, the slot will be migrated to the slave
MOOCS 2021-01-29 redis6 live notes - Part 2 (multi thread / hash slot / cluster)

If a new master node is added and reallocated, some slots will be migrated, and the cache data will remain unchanged. Although slots will be changed to other master nodes, the data will still be hashed with fixed slots.

Plaid shop

If you still can’t understand it, take another example: chestnut, lattice shop. Each lattice belongs to a seller. The seller entrusts each lattice to some shops and runs it on their behalf. If the shops close down and the lattice still exists, he can take each lattice with him. The commodities in each lattice are the data in the redis cluster. It won’t change, but the shell has changed a shop nothing more.

Construction of redis cluster

Time relationship, we only talk about the principle here, the operation part, we provide the MOOCS network Notes document, very simple, you can go to have a look on the line.
Link address: https://www.imooc.com/article/313301

However, there are some points for attention in the cluster, which are summarized as follows:

  1. Reading and writing all join the cluster in master and slave, and data synchronization will be carried out. When connecting any master or slave node in the cluster to read and write data, it will be routed to a master node according to the key hash. Slave does not provide read-write service, it only synchronizes data.
  2. Closing any master node will lead to some write failures, because the slave node can not perform write operations, and there may be a small number of failures during the upgrade from slave to master.
  3. Shutting down the slave has no effect on the entire cluster
  4. When a master node and all the slave nodes under his command all hang up, our cluster will enter the fail state and will not be available. Because the slot is incomplete.
  5. If more than half of the masters in the cluster fail, no matter whether they have a corresponding slave or not, the cluster will enter the fail state because it is unable to vote.
  6. If any master in the cluster is down and there is no slave in this master. The cluster is not available. (same as 3)
  7. All the masters in the cluster participate in the voting process. If more than half of the master nodes communicate with the master node timeout, the current master node will be considered dead.
  8. The election is only for all slave elections in a master, not for all full slave elections.
  9. After the original master restores the connection, it will become the slave server of the new master. Due to master-slave synchronization, the client’s write command may be lost. Redis is not a strong consistency, because of the master-slave characteristics, so the last part of the data will be lost. This is also in line with the cap theory.
  10. The cluster only realizes the primary node’s fail over; the secondary node will only be offline when it fails, and will not fail over. Therefore, when cluster is used, read-write separation technology is generally not used, because the failure of slave node will lead to the unavailability of read service and the poor availability. So don’t do read-write separation in the cluster.

It should be noted that in order to ensure the integrity of the cluster, the cluster can only be launched when all 16384 slots have been allocated. Moreover, if the primary node fails over and is in the process of failing over, the slot slot of the original primary node will not be in any node, and the cluster will be offline, and the client cannot call it. Cap principle.

Redis cluster data migration

How can a single node redis with existing data be extended to a cluster

When building a cluster, the package clears the existing AOF and RDB files in the node. If it is a single node, how can it be expanded to a cluster? And to ensure that the data is not lost. In fact, the principle is to save the RDB or AOF file, import it, and allocate it through slot. The specific steps are as follows:
MOOCS 2021-01-29 redis6 live notes - Part 2 (multi thread / hash slot / cluster)
As shown in the figure above, the principle is slot migration. The whole process is blocked, and the website service does not provide external services. If you use a self built cluster, then the operation and maintenance costs are very high. Generally, it is better to use cloud redis. Even Alibaba cloud will have network interruption when the configuration changes.
MOOCS 2021-01-29 redis6 live notes - Part 2 (multi thread / hash slot / cluster)