Distributed cache (with code)

Time:2021-3-5

Distributed cache (with code)

It’s another company that didn’t start work with a red envelope!!!

problem analysis

Through the above dialogue, can you guess the reason for all cache penetration? Before answering, let’s take a look at the specific code of cache policy

Cache server IP = hash (key)% number of servers

One more thing to say here is that the value of key can be designed according to the specific business. For example, I want to do load balancing, the key can be the server IP of the caller; get user information, the key can be the user ID; and so on.

With the same number of servers, there is no problem with the above design. But you know, the real world of programmers is miserable, the only constant is that the business has been changing. I had no choice but to rely on technology to change this situation.

If the number of servers is 10, when we request key 6, the result is 4. Now we add a server, and the number of servers becomes 11. When we request a server with key 6 again, the result is 5. It’s not hard to find that not only the request with key 6, but most of the request results have changed. This is the problem we need to solve, This is also the main problem we need to pay attention to when we design distributed caching and other similar scenarios.

Our ultimate design goal is: in the case of changes in the number of servers

  1. Maximize cache hit rate (transfer the least data)
  2. Cache data should be allocated as evenly as possible

Solution

Through the above analysis, we understand that the root cause of a large number of cache failures is the change of the denominator of the formula. If we keep the denominator unchanged, we can basically reduce the movement of a large number of data

Denominator invariant scheme

If we keep the denominator unchanged based on the formula: cache server IP = hash (key)% number of servers, we can basically improve the existing situation. Our strategy for choosing a cache server will change to:

Cache server IP = hash (key)% n (n is constant)
The value of N can be selected according to the specific business. For example, we can be sure that the number of servers will not exceed 100 in the future, so n can be set to 100. What about the problems?

In the current situation, it can be considered that the server number is continuous, and any request will hit a server. Or as an example, whether the number of servers is 10 or increased to 11, the request with key 6 can always get the information of a server, but now the denominator of our policy formula is 100. If the number of servers is 11, the result of the request with key 20 is 20 , server number 20 does not exist.

The above is the problem caused by the simple hash strategy (the simple redundancy hash strategy can be abstracted as a continuous array element, which can be accessed according to the subscript)

In order to solve the above problems, the industry already has a solution, that isConsistent Hashing

Consistent hash algorithm was proposed in 1997 by Karger and others of MIT in solving distributed cache. The design goal is to solve the hot spot problem in the Internet, and the original intention is very similar to that of carp. Consistent hashing corrects the problems caused by the simple hashing algorithm used by carp, and makes DHT really be applied in P2P environment.

Consistency hash specific features, please Baidu, here is not in detail. As for the way to solve the problem, I would like to emphasize the following:

  1. First, the hash value of the server (node) is calculated and configured to the ring, which has 2 ^ 32 nodes.
  2. Using the same method, the hash value of the key to store data is calculated and mapped to the same circle.
  3. Then start the clockwise search from where the data is mapped, and save the data to the first server found. If the server cannot be found after 2 ^ 32, it will be saved to the first server

Distributed cache (with code)

What happens when new servers are added?
Distributed cache (with code)

From the picture above, we can see that only the yellow part shows the change. It is similar to deleting a server.

Through the above introduction, consistent hashing is a solution to our current problems. There are tens of thousands of solutions. It is better to solve the problem.

Optimization scheme

So far, the plan seems perfect, but the reality is cruel. Although the above scheme is good, there are still some defects. If we have three servers, the ideal allocation of servers in the hash ring is as follows:

Distributed cache (with code)
But the reality is often like this:

Distributed cache (with code)

This is called hash ring skew. Uneven distribution will crush servers in some scenarios, so we must pay attention to this problem in the actual production environment. In order to solve this problem, virtual nodes emerge as the times require.

Distributed cache (with code)

As shown in the figure above, the hash ring is no longer the actual server information, but the mapping information of the server information. For example, servera-1 and servera-2 are mapped to server a, which is a replica of server a on the ring. This solution is to use the quantity to achieve the purpose of uniform distribution, and then the required memory may be a little larger, which can be regarded as a scheme of space for design.

Extended reading

  • Since it is a hash, there will be hash conflicts. What should I do if the hash values of multiple server nodes are the same? We can use the hash table addressing scheme: from the current position clockwise to find an empty position, until we find an empty position. If not, Caicai thinks that your hash ring should be expanded, or your denominator parameter is too small.
  • In the actual business, the operation of adding or reducing servers is much less than that of finding servers, so the search speed of the data structure storing the hash ring must be fast. Specifically speaking, the essence is: from a certain value of the hash ring, we can quickly find the first element that is not empty.
  • If you go through it, you will find that the number of virtual hash ring nodes on the Internet is 2 ^ 32 (the 32nd power of 2), which is the same. Can’t it be done except for this number? In Cai Cai’s opinion, this number is absolutely necessary, as long as it meets our business needs and business data.
  • The hash function used in consistent hashing should not only guarantee relatively high performance, but also keep the average distribution of hash values as far as possible, which is also the requirement of an industrial level hash function. The hash function of the code example is not the best, and students who are interested can optimize it.
  • The GetHashCode () method of some languages is problematic when applied to consistent hashing, such as C #. The hash value of the same string changes after the program is restarted. All need a more stable string to int hash algorithm.

The essence of consistent hashing is that the same key can correctly route to the same target through the same hash function. For example, we usually use database table splitting strategy, database splitting strategy, load balancing, data fragmentation and so on, which can be solved by consistent hashing.

Combining theory with practice is the essence (NETCORE code)

After a little modification, the following code can be directly applied to the production environment of small and medium-sized projects

//Real node information
    public abstract class NodeInfo
    {
        public abstract string NodeName { get; }
    }

Node information used by the test program:

    class Server : NodeInfo
        {
            public string IP { get; set; }
            public override string NodeName
            {
                get => IP;
            }
        }

The following is the core code of consistent hashing:

/// <summary>
    ///1. Adopt virtual node mode 2. The total number of nodes can be customized 3. The number of virtual nodes of each physical node can be customized
    /// </summary>
    public class ConsistentHash
    {
        //Virtual node information of hash ring
        public class VirtualNode
        {
            public string VirtualNodeName { get; set; }
            public NodeInfo Node { get; set; }
        }

        //Add element and delete element lock to ensure thread safety, or use read-write lock
        private readonly object objLock = new object();

        //The total number of virtual ring nodes, which is 100 by default
        int ringNodeCount;
        //The number of virtual nodes corresponding to each physical node
        int virtualNodeNumber;
        //Hash ring, where array is used to store
        public VirtualNode[] nodes = null;
        public ConsistentHash(int _ringNodeCount = 100, int _virtualNodeNumber = 3)
        {
            if (_ringNodeCount <= 0 || _virtualNodeNumber <= 0)
            {
                throw new Exception("_ Ringnodecount and_ Virtualnodenumber must be greater than 0 ');
            }
            this.ringNodeCount = _ringNodeCount;
            this.virtualNodeNumber = _virtualNodeNumber;
            nodes = new VirtualNode[_ringNodeCount];
        }
        //The node information is obtained according to the consistent hash key, and the business party is required to handle the timeout problem for the lookup operation, because in the multi-threaded environment, all nodes of the ring may be cleared
        public NodeInfo GetNode(string key)
        {
            var ringStartIndex = Math.Abs(GetKeyHashCode(key) % ringNodeCount);
            var vNode = FindNodeFromIndex(ringStartIndex);
            return vNode == null ? null : vNode.Node;
        }
        //Virtual ring adds a physical node
        public void AddNode(NodeInfo newNode)
        {
            var nodeName = newNode.NodeName;
            int virtualNodeIndex = 0;
            lock (objLock)
            {
                //Transform physical node into virtual node
                while (virtualNodeIndex < virtualNodeNumber)
                {
                    var vNodeName = $"{nodeName}#{virtualNodeIndex}";
                    var findStartIndex = Math.Abs(GetKeyHashCode(vNodeName) % ringNodeCount);
                    var emptyIndex = FindEmptyNodeFromIndex(findStartIndex);
                    if (emptyIndex < 0)
                    {
                        //The maximum number of nodes set has been exceeded
                        break;
                    }
                    nodes[emptyIndex] = new VirtualNode() { VirtualNodeName = vNodeName, Node = newNode };
                    virtualNodeIndex++;
                   
                }
            }
        }
        //Delete a virtual node
        public void RemoveNode(NodeInfo node)
        {
            var nodeName = node.NodeName;
            int virtualNodeIndex = 0;
            List<string> lstRemoveNodeName = new List<string>();
            while (virtualNodeIndex < virtualNodeNumber)
            {
                lstRemoveNodeName.Add($"{nodeName}#{virtualNodeIndex}");
                virtualNodeIndex++;
            }
            //Cycle through the position with index 0 to delete all virtual nodes
            int startFindIndex = 0;
            lock (objLock)
            {
                while (startFindIndex < nodes.Length)
                {
                    if (nodes[startFindIndex] != null && lstRemoveNodeName.Contains(nodes[startFindIndex].VirtualNodeName))
                    {
                        nodes[startFindIndex] = null;
                    }
                    startFindIndex++;
                }
            }

        }


        //Because of the GetHashCode provided by the system, the method of getting hash value by hash ring will change when the service is restarted
        protected virtual int GetKeyHashCode(string key)
        {
            var sh = new SHA1Managed();
            byte[] data = sh.ComputeHash(Encoding.Unicode.GetBytes(key));
            return BitConverter.ToInt32(data, 0);

        }

        #Region private method
        //Find the first node from a location in the virtual ring
        private VirtualNode FindNodeFromIndex(int startIndex)
        {
            if (nodes == null || nodes.Length <= 0)
            {
                return null;
            }
            VirtualNode node = null;
            while (node == null)
            {
                startIndex = GetNextIndex(startIndex);
                node = nodes[startIndex];
            }
            return node;
        }
        //Starting from a position in the virtual ring, find the empty position
        private int FindEmptyNodeFromIndex(int startIndex)
        {

            while (true)
            {
                if (nodes[startIndex] == null)
                {
                    return startIndex;
                }
                var nextIndex = GetNextIndex(startIndex);
                //If the index returns to the original place, it means that the virtual ring node is full and will not be added
                if (nextIndex == startIndex)
                {
                    return -1;
                }
                startIndex = nextIndex;
            }
        }
        //Gets the next location index of a location
        private int GetNextIndex(int preIndex)
        {
            int nextIndex = 0;
            //If the search position reaches the end of the ring, the search starts at position 0
            if (preIndex != nodes.Length - 1)
            {
                nextIndex = preIndex + 1;
            }
            return nextIndex;
        }
        #endregion
    }

Test generated nodes

            ConsistentHash h = new ConsistentHash(200, 5);
            h.AddNode(new Server() { IP = "192.168.1.1" });
            h.AddNode(new Server() { IP = "192.168.1.2" });
            h.AddNode(new Server() { IP = "192.168.1.3" });
            h.AddNode(new Server() { IP = "192.168.1.4" });
            h.AddNode(new Server() { IP = "192.168.1.5" });

            for (int i = 0; i < h.nodes.Length; i++)
            {
                if (h.nodes[i] != null)
                {
                    Console.WriteLine($"{i}===={h.nodes[i].VirtualNodeName}");
                }
            }

The output result is fairly uniform

2====192.168.1.3#4
10====192.168.1.1#0
15====192.168.1.3#3
24====192.168.1.2#2
29====192.168.1.3#2
33====192.168.1.4#4
64====192.168.1.5#1
73====192.168.1.4#3
75====192.168.1.2#0
77====192.168.1.1#3
85====192.168.1.1#4
88====192.168.1.5#4
117====192.168.1.4#1
118====192.168.1.2#4
137====192.168.1.1#1
152====192.168.1.2#1
157====192.168.1.5#2
158====192.168.1.2#3
159====192.168.1.3#0
162====192.168.1.5#0
165====192.168.1.1#2
166====192.168.1.3#1
177====192.168.1.5#3
185====192.168.1.4#0
196====192.168.1.4#2

Test the performance

            Stopwatch w = new Stopwatch();
            w.Start();
            for (int i = 0; i < 100000; i++)
            {
                var aaa = h.GetNode("test1");
            }
            w.Stop();
            Console.WriteLine(w.ElapsedMilliseconds);

Output result (100000 calls, 657 Ms.):

657

Write at the end

The above code has room for optimization

  1. hash function
  2. A lot of temporary variables for the for loop

Students interested in optimization can leave a message!!

More wonderful articles

Distributed cache (with code)