Distributed election algorithm

Time:2021-10-13

1. Distributed basic theory

  • Cap Theory:
     
    Consistency: consistency
     
    Availability: availability
     
    Partition tolerance: partition tolerance
     
    Distributed services can also provide external services in case of system failure. Basically, all distributed systems can meet this requirement. There is a contradiction between C and A. if you pursue consistency, you need to abandon availability. If you pursue availability, data consistency will not be met.
     
  • Base theory:
     
    Basic available: basic available
     
    Softstate: soft state
     
    Eventually: final consistency
     
    The system can be in the intermediate consistency state, but it is available. The system can reach the final consistency state. Its implementation method can be realized by using message middleware + scheduled task reissue message.
     

2. Distributed consistency algorithm

 
     In cap theory, if strong data consistency is ensured, all replicas need to synchronize all the data of the master node. In the process of data synchronization, it is not available externally. In all distributed consistency algorithms, the idea to solve this problem is that after most replicas synchronize (generally more than half) the data of the master node, the master node can use it to synchronize the client accordingly.
 

  • Basic Paxos algorithm
     
    Basic Paxos algorithm has three stages: prepare, accept and learn, as well as proponents, decision makers and learners.
     
    Prepare phase: the proposer sends a request to more than half of the decision makers. The request carries the generated globally unique incremental ID, which can be timestamp + machine ID.
     
    Accept phase: after receiving the ID information of the proposer, the decision-maker compares the saved ID with the requested ID. if the requested ID is greater than the saved ID, it will send the saved maximum ID and the corresponding proposal value to the proposer as a response. After that, the decision maker will reject all proposal requests whose ID is less than the current ID and only accept requests whose ID is greater than the current ID. When the proposer receives more than half of the response from the decision-maker, he will obtain the proposal value with the maximum ID in all the responses as the proposal value and send it to more than half of the decision-makers. This more than half of the decision-makers can be different from the proposal decision-makers. After receiving the request, the decision-maker will submit the request.
     
    Learn phase: all learners synchronize the data of decision makers. The specific method can be that each learner requests to accept, or all learners can select a master learner. The master learner synchronizes the accepted data, and other learners synchronize the master learner.

     
    The above Paxos algorithm is a basic Paxos algorithm. If there are multiple proponents, it will lead to the livelock problem. In order to solve this problem, it is proposed that there is only one main proposer. Among multiple proponents, select one main proposer first, and all proposals will be submitted by the main proposer.
     

  • Raft algorithm
     
    Leader election
     
    Log synchronization
     
  • Zab algorithm
     

    • Start election leader
      Each machine sends (myid, zxid) to other machines. Other machines check whether it is in the election process, then compare zxid and vote for zxid. If zxid is the same, vote for myid. When more than half of the votes are received by a machine, it becomes a leader

 

  • Running election
    The logic is roughly the same as the election leader at startup, but it is replaced by Sid and zxid, with more than half of the voting mechanism. Then synchronize the data.

 

  • Kafka election
     
    ISR list
     
    High water level HW
     
    More than half mechanism

Recommended Today

The selector returned by ngrx store createselector performs one-step debugging of fetching logic

Test source code: import { Component } from ‘@angular/core’; import { createSelector } from ‘@ngrx/store’; export interface State { counter1: number; counter2: number; } export const selectCounter1 = (state: State) => state.counter1; export const selectCounter2 = (state: State) => state.counter2; export const selectTotal = createSelector( selectCounter1, selectCounter2, (counter1, counter2) => counter1 + counter2 ); // […]