1. Distributed basic theory
- Cap Theory:
Partition tolerance: partition tolerance
Distributed services can also provide external services in case of system failure. Basically, all distributed systems can meet this requirement. There is a contradiction between C and A. if you pursue consistency, you need to abandon availability. If you pursue availability, data consistency will not be met.
- Base theory:
Basic available: basic available
Softstate: soft state
Eventually: final consistency
The system can be in the intermediate consistency state, but it is available. The system can reach the final consistency state. Its implementation method can be realized by using message middleware + scheduled task reissue message.
2. Distributed consistency algorithm
In cap theory, if strong data consistency is ensured, all replicas need to synchronize all the data of the master node. In the process of data synchronization, it is not available externally. In all distributed consistency algorithms, the idea to solve this problem is that after most replicas synchronize (generally more than half) the data of the master node, the master node can use it to synchronize the client accordingly.
- Basic Paxos algorithm
Basic Paxos algorithm has three stages: prepare, accept and learn, as well as proponents, decision makers and learners.
Prepare phase: the proposer sends a request to more than half of the decision makers. The request carries the generated globally unique incremental ID, which can be timestamp + machine ID.
Accept phase: after receiving the ID information of the proposer, the decision-maker compares the saved ID with the requested ID. if the requested ID is greater than the saved ID, it will send the saved maximum ID and the corresponding proposal value to the proposer as a response. After that, the decision maker will reject all proposal requests whose ID is less than the current ID and only accept requests whose ID is greater than the current ID. When the proposer receives more than half of the response from the decision-maker, he will obtain the proposal value with the maximum ID in all the responses as the proposal value and send it to more than half of the decision-makers. This more than half of the decision-makers can be different from the proposal decision-makers. After receiving the request, the decision-maker will submit the request.
Learn phase: all learners synchronize the data of decision makers. The specific method can be that each learner requests to accept, or all learners can select a master learner. The master learner synchronizes the accepted data, and other learners synchronize the master learner.
The above Paxos algorithm is a basic Paxos algorithm. If there are multiple proponents, it will lead to the livelock problem. In order to solve this problem, it is proposed that there is only one main proposer. Among multiple proponents, select one main proposer first, and all proposals will be submitted by the main proposer.
- Raft algorithm
- Start election leader
Each machine sends (myid, zxid) to other machines. Other machines check whether it is in the election process, then compare zxid and vote for zxid. If zxid is the same, vote for myid. When more than half of the votes are received by a machine, it becomes a leader
- Start election leader
- Running election
The logic is roughly the same as the election leader at startup, but it is replaced by Sid and zxid, with more than half of the voting mechanism. Then synchronize the data.
- Kafka election
High water level HW
More than half mechanism