Load balancing algorithm implemented in Java — polling and weighted polling

Time:2022-6-9

1. general polling algorithm

Round robin (RR) allocates the user’s access requests to the web service nodes in a circular order, starting from 1 to the end of the last server node, and then starts a new round of the cycle. This algorithm is simple, but does not take into account the specific performance of each node server, and the request distribution is often uneven.

Code implementation:

/**
 *General polling algorithm
 */public class RoundRobin {
    private static Integer index = 0;
    private static List nodes = new ArrayList<>();
    //Prepare analog data
    static {
        nodes.add("192.168.1.101");
        nodes.add("192.168.1.103");
        nodes.add("192.168.1.102");
        System. out. Println ("all nodes of normal polling algorithm:" +nodes)// Print all nodes
    }
    
    //Key code
    public String selectNode(){
        String ip = null;
        synchronized (index){
            //Subscript reset
            if(index>=nodes.size()) index = 0;
            ip = nodes.get(index);
            index++;
        }
        return ip;
    }

    //Concurrency test: two threads cycle to obtain nodes
    public static void main(String[] args) {
        new Thread(() -> {
            RoundRobin roundRobin1 = new RoundRobin();
            for (int i=1;i<=5;i++){
                String serverIp = roundRobin1.selectNode();
                System. out. println(Thread.currentThread(). Getname() + "= = the" +i+ "second acquisition node:" +serverip);
            }
        }).start();

        RoundRobin roundRobin2 = new RoundRobin();
        for (int i=1;i<=nodes.size();i++){
            String serverIp = roundRobin2.selectNode();
            System. out. println(Thread.currentThread(). Getname() + "= = the" +i+ "second acquisition node:" +serverip);
        }
    }
}

Execution result: when accessed by different threads, the result is still cyclic allocation of nodes in sequence

All nodes of common polling algorithm: [192.168.1.101, 192.168.1.103, 192.168.1.102]

Main== get node for the first time: 192.168.1.101

Thread-0== get node for the first time: 192.168.1.103

Thread-0== get node for the second time: 192.168.1.102

Thread-0== get node for the third time: 192.168.1.101

Thread-0== get node for the fourth time: 192.168.1.103

Thread-0== get node for the 5th time: 192.168.1.102

Main== get node for the second time: 192.168.1.101

Main== get node for the third time: 192.168.1.103

2. weighted polling algorithm

Weighted round robin (WRR) allocates access requests according to the set weight value. The larger the weight value, the more requests will be allocated. Generally, the weight is allocated according to the specific performance of each node server.

2.1. Implementation mode I

All nodes that will need to be polledBy weightGenerate a list set circularly, and then, just like the ordinary polling algorithm, assign one, and carry 1 bit.

For example:

All node information: {{“192.168.1.100”, 5}, {“192.168.1.101”, 1}, {“192.168.1.102”, 3}}

The generated list set is:

{“192.168.1.100“,

“192.168.1.100“,

“192.168.1.100“,

“192.168.1.100“,

“192.168.1.100“,

“192.168.1.101“,

“192.168.1.102“,

“192.168.1.102“,

“192.168.1.102“}

The following is the logic of ordinary polling algorithm

Code implementation:

It is similar to reducing the dimension of a two-dimensional array to a one-dimensional array, and then using normal polling

/**
 *Weighted polling for simple version
 */public class WeightedRoundRobinSimple {
    private static Integer index = 0;
    private static Map mapNodes = new HashMap<>();

    //Prepare analog data
    static {
        mapNodes.put("192.168.1.101",1);
        mapNodes.put("192.168.1.102",3);
        mapNodes.put("192.168.1.103",2);
        /*-- the following code is only for the convenience of viewing all nodes. Deletion does not affect -- S*/
        List nodes = new ArrayList<>();
        Iterator> iterator = mapNodes.entrySet().iterator();
        while (iterator.hasNext()){
            Map.Entry entry = iterator.next();
            String key = entry.getKey();
            for (int i=0;i nodes = new ArrayList<>();
        Iterator> iterator = mapNodes.entrySet().iterator();
        while (iterator.hasNext()){
            Map.Entry entry = iterator.next();
            String key = entry.getKey();
            for (int i=0;i=nodes.size()) index = 0;
            ip = nodes.get(index);
            index++;
        }
        return ip;
    }

    //Concurrency test: two threads cycle to obtain nodes
    public static void main(String[] args) {
        new Thread(() -> {
            WeightedRoundRobinSimple roundRobin1 = new WeightedRoundRobinSimple();
            for (int i=1;i<=6;i++){
                String serverIp = roundRobin1.selectNode();
                System. out. println(Thread.currentThread(). Getname() + "= = the" +i+ "second acquisition node:" +serverip);
            }
        }).start();

        WeightedRoundRobinSimple roundRobin2 = new WeightedRoundRobinSimple();
        for (int i=1;i<=6;i++){
            String serverIp = roundRobin2.selectNode();
            System. out. println(Thread.currentThread(). Getname() + "= = the" +i+ "second acquisition node:" +serverip);
        }
    }
}

Execution result: the output results of the two threads will be alternately allocated to different IPs, but the final effect is to allocate them one by one in order, similar to the ordinary polling algorithm.

Weighted polling in simple version: [192.168.1.103, 192.168.1.103, 192.168.1.101, 192.168.1.102, 192.168.1.102, 192.168.1.102]

Main== get node for the first time: 192.168.1.103

Main== get node for the second time: 192.168.1.103

Main== get node for the third time: 192.168.1.101

Main== get node for the fourth time: 192.168.1.102

Main== get node for the 5th time: 192.168.1.102

Thread-0== get node for the first time: 192.168.1.102

Thread-0== get node for the second time: 192.168.1.103

Main== get node for the 6th time: 192.168.1.103

Thread-0== get node for the third time: 192.168.1.101

Thread-0== get node for the fourth time: 192.168.1.102

Thread-0== get node for the 5th time: 192.168.1.102

Thread-0== get node for the 6th time: 192.168.1.102

2.2. Implementation mode II (key and difficult points)

The key points and difficulties of this paper.

In the algorithm of implementation mode 1, it is obvious that the IP with the same weight will be allocated continuously, that is, the same IP will receive different requests in a short time. After this continuous point, it will not be allocated until the next round, and the nodes are not evenly allocated.

Implementation mode II willDistribute each node as evenly as possible, and the node allocation is no longer continuousBut the final weight ratio is the same as the previous method. This weighted polling is also called smooth weighted polling.

Understand several key parameters and algorithm logic to facilitate the understanding of code implementation.

2.2.1. summary

key parameter

  • IP: load IP
  • Weight: weight to save the configured weight
  • Effectiveweight: effective weight. The process weight of polling may change
  • Currentweight: current weight. Compare the value to obtain the node

Note the following:

Weight, which will not be modified in the whole process, is only used to save the weight parameter values during configuration. If you directly calculate the weight without saving the original weight parameters of the configuration, you will lose the weight parameters of the most critical user configuration.

The effective weight of effectiveweight may change throughout the process. The initial value is equal to weight. It is mainly used to reduce the weight value when the node fails to allocate, and to increase the weight value when the node succeeds (but not greater than the weight value). In this case, this function is not added to simplify the algorithm, so the effectiveweight in this case is always equal to weight.

Currentweight is the current weight. The node with the largest weight is allocated by cycling all nodes to compare the value. The initial value is equal to weight.

Changes of three weight parameters

For this case only, in order to simplify the algorithm, this case does not include the function of [reducing the weight value when the node fails to allocate, and increasing the weight value (but not greater than the weight value) when the node succeeds], so the effective weight effectiveweight will not change.

  • For the first weighted polling: currentweight = weight = effectiveweight;
  • During each subsequent weighted polling: the currentweight value will change continuously, and the weight and effectiveweight values will remain unchanged;
  • Currentweight of the assigned node = currentweight – sum of weights
  • Currentweight of all nodes = currentweight + effectiveweight

2.2.2. Take an example to understand the algorithm

There are three bottles a, B and C in front of you, which contain 1L, 3L and 2L of water respectively.

Distribution in the first round: there are many B bottles. Therefore, 1L of 3L water in bottle B is divided into a and 2L into C (according to the weight). After the division, a, B and C are 2L, 0l and 4L respectively

The second round of distribution: there are many C bottles, so 1L of 4L water in C bottle is divided into a and 3L to B (according to the weight). After the division, a, B and C are 3l, 3L and 0l respectively

The third round of distribution: there are as many as a and B, so who will score?It’s the same with everyone(a is written in the algorithmgreater thanOnly a is selected for B, which is now equal to, so a is not selected). Therefore, 1L of 3L water in bottle B is divided into a and 2L into C (according to the weight). After the division, a, B and C are respectively 4L, 0l and 2L

And keep going

Two key steps to simplify into mathematical logic (code implementation)

  • Currentweight of the assigned node = currentweight – sum of weights
  • Currentweight of all nodes = currentweight + effectiveweight

Read the code below to understand

2.2.3. code implementation

Node object

/**
 *String IP: load IP
 *Final integer weight: weight to save the configured weight
 *Integer effectiveweight: effective weight. The process weight of polling may change
 *Integer currentweight: current weight. Compare the value to obtain the node
 *At the first weighted poll: currentweight = weight = effectiveweight
 *During each subsequent weighted polling: the value of currentweight will change continuously, and other weights will remain unchanged
 */public class Node implements Comparable{
    private String ip;
    private final Integer weight;
    private Integer effectiveWeight;
    private Integer currentWeight;

    public Node(String ip,Integer weight){
        this.ip = ip;
        this.weight = weight;
        this.effectiveWeight = weight;
        this.currentWeight = weight;
    }

    public Node(String ip, Integer weight, Integer effectiveWeight, Integer currentWeight) {
        this.ip = ip;
        this.weight = weight;
        this.effectiveWeight = effectiveWeight;
        this.currentWeight = currentWeight;
    }

    public String getIp() {
        return ip;
    }

    public void setIp(String ip) {
        this.ip = ip;
    }

    public Integer getWeight() {
        return weight;
    }

    public Integer getEffectiveWeight() {
        return effectiveWeight;
    }

    public void setEffectiveWeight(Integer effectiveWeight) {
        this.effectiveWeight = effectiveWeight;
    }

    public Integer getCurrentWeight() {
        return currentWeight;
    }

    public void setCurrentWeight(Integer currentWeight) {
        this.currentWeight = currentWeight;
    }

    @Override
    public int compareTo(Node node) {
        return currentWeight > node.currentWeight ? 1 : (currentWeight.equals(node.currentWeight) ? 0 : -1);
    }

    @Override
    public String toString() {
        return "{ip='" + ip + "', weight=" + weight + ", effectiveWeight=" + effectiveWeight + ", currentWeight=" + currentWeight + "}";
    }
}

Weighted Round Robin

/**
 *Weighted polling algorithm
 */public class WeightedRoundRobin {

    private static List nodes = new ArrayList<>();
    //Sum of weights
    private static Integer totalWeight = 0;
    //Prepare analog data
    static {
        nodes.add(new Node("192.168.1.101",1));
        nodes.add(new Node("192.168.1.102",3));
        nodes.add(new Node("192.168.1.103",2));
        nodes.forEach(node -> totalWeight += node.getEffectiveWeight());
    }

    /**
     *Obtain IP according to the maximum value of currentweight
     * @return Node
     */
    public Node selectNode(){
        if (nodes ==null || nodes.size()<=0) return null;
        if (nodes.size() == 1)  return nodes.get(0);

        Node nodeOfMaxWeight = null; //  Save the node information selected by polling
        synchronized (nodes){
            //Print information object: avoid too messy information printed during concurrency, which is not conducive to viewing results
            StringBuffer sb = new StringBuffer();
            sb. append(Thread.currentThread(). Getname() + "= = weighted polling -- change of [current weight] value:" +printcurrentweight (nodes));

            //Select the node with the largest weight at present
            Node tempNodeOfMaxWeight = null;
            for (Node node : nodes) {
                if (tempNodeOfMaxWeight == null)
                    tempNodeOfMaxWeight = node;
                else
                    tempNodeOfMaxWeight = tempNodeOfMaxWeight.compareTo(node) > 0 ? tempNodeOfMaxWeight : node;
            }
            //New node instances must be used to save the information, otherwise the reference points to the same heap instance, and the subsequent set operation will modify the node information
            nodeOfMaxWeight = new Node(tempNodeOfMaxWeight.getIp(),tempNodeOfMaxWeight.getWeight(),tempNodeOfMaxWeight.getEffectiveWeight(),tempNodeOfMaxWeight.getCurrentWeight());

            //Adjust the current weight ratio: adjust according to the proportion of the effectiveweight to ensure that the request distribution is reasonable.
            tempNodeOfMaxWeight.setCurrentWeight(tempNodeOfMaxWeight.getCurrentWeight() - totalWeight);
            sb.append(" -> "+printCurrentWeight(nodes));

            nodes.forEach(node -> node.setCurrentWeight(node.getCurrentWeight()+node.getEffectiveWeight()));

            sb.append(" -> "+printCurrentWeight(nodes));
            System. out. println(sb); // Print weight change process
        }
        return nodeOfMaxWeight;
    }

    //Format print information
    private String printCurrentWeight(List nodes){
        StringBuffer stringBuffer = new StringBuffer("[");
        nodes.forEach(node -> stringBuffer.append(node.getCurrentWeight()+",") );
        return stringBuffer.substring(0, stringBuffer.length() - 1) + "]";
    }

    //Concurrency test: two threads cycle to obtain nodes
    public static void main(String[] args){
        Thread thread = new Thread(() -> {
            WeightedRoundRobin weightedRoundRobin1 = new WeightedRoundRobin();
            for(int i=1;i<=totalWeight;i++){
                Node node = weightedRoundRobin1.selectNode();
                System. out. println(Thread.currentThread(). Getname() + "= = the" +i+ "th poll selects the node with [maximum current weight]: + node +" \n ");
            }
        });
        thread.start();
        //
        WeightedRoundRobin weightedRoundRobin2 = new WeightedRoundRobin();
        for(int i=1;i<=totalWeight;i++){
            Node node = weightedRoundRobin2.selectNode();
            System. out. println(Thread.currentThread(). Getname() + "= = the" +i+ "th poll selects the node with [maximum current weight]: + node +" \n ");
        }

    }
}

Execution result:

Main== weighted polling — change of [current weight] value: [1,3,2] – > [1, -3,2] – > [2,0,4]

Thread-0== weighted polling — change of [current weight] value: [2,0,4] – > [2,0, -2] – > [3,3,0]

Main== weighted polling — change of [current weight] value: [3,3,0] – > [3, -3,0] – > [4,0,2]

Main== weighted polling — change of [current weight] value: [4,0,2] – > [-2,0,2] – > [-1,3,4]

Thread-0== weighted polling — change of [current weight] value: [-1,3,4] – > [-1,3, -2] – > [0,6,0]

Main== weighted polling — change of [current weight] value: [0,6,0] – > [0,0,0] – > [1,3,2]

Thread-0== weighted polling — change of [current weight] value: [1,3,2] – > [1, -3,2] – > [2,0,4]

Main== weighted polling — change of [current weight] value: [2,0,4] – > [2,0, -2] – > [3,3,0]

Thread-0== weighted polling — change of [current weight] value: [3,3,0] – > [3, -3,0] – > [4,0,2]

Main== weighted polling — change of [current weight] value: [4,0,2] – > [-2,0,2] – > [-1,3,4]

Thread-0== weighted polling — change of [current weight] value: [-1,3,4] – > [-1,3, -2] – > [0,6,0]

Thread-0== weighted polling — change of [current weight] value: [0,6,0] – > [0,0,0] – > [1,3,2]

To facilitate analysis, simplify the results after two threads are executed

Change of [current weight] value: [1,3,2] – > [1, -3,2] – > [2,0,4]

Change of [current weight] value: [2,0,4] – > [2,0, -2] – > [3,3,0]

Change of [current weight] value: [3,3,0] – > [3, -3,0] – > [4,0,2]

Change of [current weight] value: [4,0,2] – > [-2,0,2] – > [-1,3,4]

Change of [current weight] value: [-1,3,4] – > [-1,3, -2] – > [0,6,0]

Change of [current weight] value: [0,6,0] – > [0,0,0] – > [1,3,2]

Change of [current weight] value: [1,3,2] – > [1, -3,2] – > [2,0,4]

Change of [current weight] value: [2,0,4] – > [2,0, -2] – > [3,3,0]

Change of [current weight] value: [3,3,0] – > [3, -3,0] – > [4,0,2]

Change of [current weight] value: [4,0,2] – > [-2,0,2] – > [-1,3,4]

Change of [current weight] value: [-1,3,4] – > [-1,3, -2] – > [0,6,0]

Change of [current weight] value: [0,6,0] – > [0,0,0] – > [1,3,2]

Because only the current weight changes in the whole process, analyzing it clearly will understand the whole process.

Conclusion:

The current weight changes after allocation, butThe sum of permissions is still equal to the initial value

Every 6 rounds (1+3+2 weights) all the weights are 0, so a new cycle will occur. 6 is exactly the sum of the weights, and the weight ratio is equal to 1/6: 3/6: 2/6;

A= weight 1, b= weight 3, c= weight 2, then in the 6 (a+b+c) times of weight change,Distribution: B C B a C B, it is obvious that each node is evenly distributed by weight, and the node allocation is no longer continuous. This is also the most important conclusion, which is the key point to be realized in the implementation mode 2 mentioned at the beginning of this paper.

When the weight ratio of the algorithm varies greatly, for example, a=1, b=5, the result of the algorithm is no different from that of mode 1, and the allocation result becomes: {a, B, B, B, B, b}. Since there is no difference, mode 1 is certainly better according to the complexity of the algorithm, so mode 1 and mode 2 can complement each other, and different algorithms can be selected according to the weight ratio.

Leave suspense

The first point: the function of reducing the effective weight value in case of node allocation failure and increasing the effective weight value (but not greater than the weight value) in case of success. After understanding mode 2, you can easily understand it by adding this function later;

The second point: what mathematical theory is used to prove the mathematics behind the implementation of the algorithm?

image

Previous Java articles

Java full stack learning route, learning resources and interview questions

What is a good architect in my heart?

Download classic programming books for free

image

Recommended Today

CommunityToolkit.Mvvm-IOC

CommunityToolkit.Mvvm does not have IOC built in, you can use Microsoft.Extensions.DependencyInjection. Register ViewModel and other services in App public partial class App : Application { public App() { Services = ConfigureServices(); this.InitializeComponent(); } public new static App Current => (App)Application.Current; public IServiceProvider Services { get; } private IServiceProvider ConfigureServices() { var sc = new ServiceCollection(); […]