10000 word long article: Interpretation of blockchain 7 consensus algorithms

Time:2021-11-16

abstract: This paper will introduce seven common consensus algorithms in blockchain, hoping to help readers explore blockchain.

Blockchain technology originated from bitcoin and was originally an underlying technology of bitcoin and other digital currencies. Blockchain integrates cryptography, networking technology, consensus algorithm, smart contract and other technologies. With the gradual maturity of blockchain technology, it has gradually attracted the attention of scientific research institutions, governments, financial institutions and science and technology enterprises. Blockchain has the characteristics of anonymity, tamper proof, traceability and decentralization.

Traditional transactions require a trusted third party as the transaction intermediary. In contrast, blockchain technology can realize the decentralization of transactions and ensure the consistency of data throughout the network, making point-to-point transactions possible. This requires the design of transaction confirmation rules, which is the consensus algorithm to be introduced in this section. As the core of blockchain technology, consensus algorithm plays a decisive role in blockchain security and efficiency.

In the application of blockchain, consensus algorithm needs to solve two problems: double flower problem [1,2] and Byzantine general problem [3]. The double flower problem refers to the problem of repeated use of money in the process of use. The traditional currency has entity uniqueness, which can prevent the double flower problem by means of anti-counterfeiting. The current electronic transaction can also solve the double flower problem through the central trust organization. Blockchain solves the double flower problem through distributed nodes to jointly verify transactions. In the blockchain, a transaction needs to be verified by a sufficient number of consensus nodes. After confirmation, the transaction is recorded and synchronized to all consensus nodes in the network. The completion of the “double flower” attack in the blockchain needs to pay a sufficient price. By selecting the consensus algorithm, this price can be extended to be large enough or exceed the benefits obtained by the double flower attack.

This paper will introduce seven common consensus algorithms in blockchain, hoping to help readers explore blockchain.

1. Proof of workload (POW)

Bitcoin proposed by Nakamoto in 2009 is the earliest application of blockchain technology. It uses POW as the consensus algorithm. Its core idea is that nodes obtain accounting rights and bitcoin rewards through the competition of hash computing power. In pow, different nodes compete to calculate the solution of a mathematical problem according to specific information. This mathematical problem is difficult to solve, but it is easy to verify the results. The node that solves this mathematical problem first can create the next block and obtain a certain amount of monetary reward. Nakamoto uses hashcash [4] mechanism to design this mathematical problem in bitcoin. This section will take the pow algorithm adopted by bitcoin as an example. The consensus steps of pow are as follows:

  • The node collects the transactions to be confirmed in the whole network after the generation of the last block, records the qualified transactions in the transaction memory pool, then updates and calculates the Merkle root value of the transactions in the memory pool, and writes it into the block header;

Fill in the block version number, hash value of the previous block, timestamp, current target hash value, random number and other information in the block header as shown in Table 1.1;

Table 1.1 block header information

10000 word long article: Interpretation of blockchain 7 consensus algorithms

  • The random number nonce is taken from 0 to 232 to hash the block header information. When the hash value is less than or equal to the target value, the block is packaged and broadcast, and bookkeeping is completed after verification by other nodes;
  • If the hash value that meets the requirements cannot be calculated within a certain time, repeat step 2. If other nodes have completed the calculation during the calculation, restart from step 1.

The average time for bitcoin to generate blocks is 10 minutes. To maintain this speed, the target value (difficulty) needs to be adjusted according to the current computing power of the whole network [5]. Difficulty is a description of the degree of difficulty in calculating blocks that meet the requirements. When calculating blocks of the same height, the difficulty of all nodes is the same, which also ensures the fairness of mining. The relationship between difficulty and target value is:

Difficulty value = maximum target value / current target value (1.1)

The maximum target value and the current target value are 256 bits in length, and the maximum target value is the target value when the difficulty is 1, i.e. 2224. Assuming that the current difficulty is, the calculation force is, the current target value is, and the average calculation time of the new block is, then

10000 word long article: Interpretation of blockchain 7 consensus algorithms

According to the design of bitcoin, the system will adjust the current target value every 2016 blocks (about 2 weeks). The node calculates the adjusted difficulty value from formula (1.4) according to the actual production time of the first 2016 blocks. If the actual production time is less than 2 weeks, increase the difficulty value; If the actual production time is greater than 2 weeks, reduce the difficulty value. According to the longest chain principle, all nodes will get the same difficulty value after a certain time without node synchronization difficulty information.

In the blockchain using pow, due to network delay and other reasons, bifurcation may occur when the generation time of two blocks at the same height is close. That is, different miners have calculated a block of a certain height that meets the requirements and obtained the confirmation of its adjacent nodes. The nodes of the whole network will continue mining on the basis of the block received first according to the time of receiving the block. In this case, the subsequent blocks of which block appear first will become longer, and this block will be included in the main chain. The nodes mining on the non main chain will switch to the main chain to continue mining.

POW consensus algorithm takes computing power as the basis for competing for bookkeeping rights and workload as the guarantee of safety. All miners follow the principle of the longest chain. The newly generated block contains the hash value of the previous block. All existing blocks form a chain. The length of the chain is proportional to the workload. All nodes trust the longest blockchain. If an organization has enough computing power, it can launch an attack against bitcoin network. When the attacker has enough computing power, he can calculate the latest block first, so as to master the longest chain. At this time, most of the blocks on the bitcoin main chain are generated by it. He can deliberately refuse the confirmation of some transactions and carry out double flower attack, which will affect the credibility of the bitcoin network, but this behavior will also bring losses to the attacker. By solving the one-dimensional random walk problem, the relationship between the success probability of malicious node attack and computing power can be obtained:

10000 word long article: Interpretation of blockchain 7 consensus algorithms

10000 word long article: Interpretation of blockchain 7 consensus algorithms

10000 word long article: Interpretation of blockchain 7 consensus algorithms

Figure 1.1 attacker’s computing power and attack success probability

2. Proof of interest (POS)

With more and more people participating in bitcoin mining, many problems of pow gradually appear. For example, with the rapid intensification of computing power competition, the energy required to obtain tokens increases greatly, and the bookkeeping right is gradually concentrated in the “mining pool” with a large amount of computing power [6-9]. Therefore, researchers try to use a new mechanism to replace workload proof. The concept of POS was mentioned in the earliest bitcoin projects, but it was not used due to robustness and other reasons. The earliest application of POS is ppcoin. POS puts forward the concept of coin age. Coin age is the accumulation of the product of tokens held and holding time. The calculation is shown in formula (1.4). The competition of currency age is used to replace the competition of computing power, so that the proof of blockchain no longer depends only on workload, and the problem of resource waste of pow is effectively solved.

10000 word long article: Interpretation of blockchain 7 consensus algorithms

The holding time is the time when a currency was last traded on the network. The longer the currency age held by each node, the more rights and interests it has in the network. At the same time, the currency holder will obtain certain benefits according to the currency age. In the design of counting coins, it is not completely separated from the workload proof. The acquisition of accounting right of POS mechanism also requires simple hash calculation:

10000 word long article: Interpretation of blockchain 7 consensus algorithms

Proofhash is the hash value obtained by the fuzzy sum of weight factor, unused output value and current time. At the same time, the computing power of each node is limited. It can be seen that the currency age is inversely proportional to the difficulty of calculation. In POS, the security of the blockchain increases with the increase of the value of the blockchain. An attack on the blockchain requires the attacker to accumulate a large number of currency ages, that is, he needs to hold a large number of digital currencies for a long enough time, which also greatly increases the difficulty of the attack. Compared with pow, blockchain systems using POS may face long range attack and nothing at stake attack.

In addition to counting coins, many coins also use POS, but there are different methods for the distribution of bookkeeping rights. For example, NXT and blackcion combine the interests owned by the node and use a random algorithm to allocate bookkeeping rights. Ethereum is also gradually adopting POS instead of pow.

3. Certificate of entrusted interests (dpos)

At the beginning of bitcoin design, it is hoped that all mining participants use CPU for calculation, and the computing power matches the nodes. Each node has enough opportunities to participate in the decision-making of blockchain. With the development of technology, a large number of mining machines using GPU, FPGA, ASIC and other technologies appear, and the computing power is concentrated in the hands of participants with a large number of mining machines, while the opportunity for ordinary miners to participate is greatly reduced.

In the blockchain with dpos, each node can vote and select representatives according to its share rights and interests. The N nodes participating in the election and obtaining the most votes in the whole network obtain the bookkeeping right, produce blocks in sequence according to the predetermined order, and therefore obtain certain incentives. A successful representative node needs to pay a certain amount of security deposit, and must ensure the online time. If the node that should produce a block fails to perform its duties at a certain time, he will be disqualified as a representative, and the system will continue to vote to elect a new representative to replace him.

All nodes in dpos can independently select the voting objects, and the elected representatives are recorded in order, which saves computing resources compared with POW and pos. moreover, there are only a limited number of consensus nodes, and the efficiency is improved. Moreover, each participating node has the right to vote. When there are enough nodes in the network, the security and decentralization of dpos are also guaranteed.

4. Practical Byzantine fault tolerance algorithm (pbft)

In pbft algorithm, all nodes run under the same configuration, and there is a primary node and other nodes act as backup nodes. The master node is responsible for sorting the client requests and sending them to the backup node in order. There is the concept of view. In each view, all nodes process messages normally. However, when the backup node detects that the primary node is abnormal, it will trigger the view change mechanism to replace the next numbered node as the primary node and enter a new view. The main process from the client sending a request to receiving a reply in pbft is shown in Figure 4.1 [10] [11]. The servers exchange information three times. The whole process includes the following five stages:

10000 word long article: Interpretation of blockchain 7 consensus algorithms

Figure 4.1 pbft execution process

10000 word long article: Interpretation of blockchain 7 consensus algorithms

10000 word long article: Interpretation of blockchain 7 consensus algorithms

At present, the Byzantine fault-tolerant algorithm represented by pbft is used by many blockchain projects. In the alliance chain, pbft algorithm was first adopted by hyper ledger fabric project. In version 0.6, hyperledger fabric adopts pbft consensus algorithm, and the functions of authorization and endorsement are integrated into consensus nodes. All nodes are consensus nodes. This design leads to too heavy burden on nodes and has a great impact on TPS and scalability. In versions after 1.0, the functions of the node are separated. The node is divided into three endorser nodes, orderer nodes and committee nodes. The functions of the node are separated, which improves the efficiency of consensus to a certain extent.

The tendermint [12] algorithm used by Cosmos project combines pbft and POS algorithms, selects some consensus nodes through token mortgage for BFT consensus, weakens the asynchronous assumption, and integrates the concept of lock on the basis of pbft. In some synchronous networks, consensus nodes can reach consensus through two-stage communication. The system can tolerate 1 / 3 of the faulty nodes without bifurcation. On the basis of tendermint, hotstuff [13] integrates the block chain structure of the blockchain with each stage of BFT. The signature confirmation of the previous block between nodes in each stage is carried out simultaneously with the construction of new blocks, which makes the implementation of the algorithm simpler. Hotstuff also uses threshold signature [14] to reduce the message complexity of the algorithm.

5. Paxos and raft

Consensus algorithm is a mechanism designed to ensure the accuracy and consistency of stored information. In traditional distributed systems, the most commonly used consensus algorithm is Paxos based algorithm. After the Byzantine general problem [3] was proposed, Lamport proposed Paxos algorithm in 1990 to solve the system consistency problem under specific conditions. Lamport reorganized and published Paxos paper [15] in 1998 and re described Paxos in 2001 [16]. Subsequently, Paxos dominated the field of consistency algorithms and was adopted by many companies, such as phxpaxos of Tencent, x-paxos of Alibaba, dynamodb of AWS of Amazon and Megastore of Google [17]. This kind of algorithm can quickly complete the data synchronization of distributed system when the number of nodes is limited and relatively trusted, and can tolerate crash fault at the same time. That is, in the traditional distributed system, it is not necessary to consider the malicious tampering of data by the participating nodes, but only to tolerate the downtime errors of some nodes. However, Paxos algorithm is too theoretical and difficult to understand and implement in engineering. Ongaro et al. Published a paper in 2013 and proposed raft algorithm [18]. Raft has the same effect as Paxos and is more convenient for engineering implementation.

10000 word long article: Interpretation of blockchain 7 consensus algorithms

The leader in raft occupies an absolutely dominant position and must ensure the absolute security of server nodes. Once the leader is maliciously controlled, it will cause huge losses. Moreover, the transaction volume is limited by the maximum throughput of the node. At present, many alliance chains will use raft algorithm to improve consensus efficiency without considering Byzantine fault tolerance.

6. Consensus algorithm combined with VRF

In the existing alliance chain consensus algorithm, if the number of nodes participating in the consensus increases, the communication between nodes will also increase, and the performance of the system will be affected. If some nodes are selected from many candidate nodes to form a consensus group for consensus, reducing the number of consensus nodes can improve the performance of the system. However, this will reduce the security, and the higher the proportion of malicious nodes in the candidate nodes, the higher the probability that the selected consensus group will not operate normally. In order to select the consensus group that can run normally from the candidate nodes and ensure the high availability of the system, on the one hand, it is necessary to design an appropriate random election algorithm to ensure the randomness of the selection and prevent the attack of malicious nodes on the system. On the other hand, we need to increase the proportion of honest nodes in the candidate nodes and increase the probability that honest nodes are selected into the consensus group.

At present, the public chain is often based on POS algorithm. The mortgage token increases the access threshold of consensus nodes, increases the evil cost of malicious nodes through economic game, and then randomly elects some nodes from qualified candidate nodes through random election algorithm.

Dodis et al. Proposed verifiable random functions (VRF) in 1999 [19]. Verifiable random function is an application of zero knowledge proof, that is, in the public-private key system, the person holding the private key can use the private key and a piece of known information to generate a random number according to specific rules. On the premise of not disclosing the private key, the person holding the private key can prove the correctness of random number generation to others. VRF can be constructed using RSA or elliptic curve. Dodis et al. Proposed the construction method of verifiable random function based on Diffie Hellman difficulty problem in 2002 [20]. At present, verifiable random function has been applied in the field of key transmission and blockchain [21]. The specific process of verifiable random function is as follows:

10000 word long article: Interpretation of blockchain 7 consensus algorithms

In the public chain, VRF has been applied in some projects. VRF is mostly combined with POS algorithm. All nodes that want to participate in the consensus pledge certain tokens as candidate nodes, and then select some consensus nodes randomly from many candidate nodes through VRF. All new nodes of zilliqa network must execute POW first. The existing nodes in the network verify the pow of the new node and authorize it to join the network. The consensus algorithm vbft designed by the blockchain project ontology combines VRF, POS and BFT algorithms. VRF randomly selects consensus nodes among many candidate nodes and determines the arrangement order of consensus nodes, which can reduce the impact of malicious bifurcation on the blockchain system and ensure the fairness and randomness of the algorithm. Algorand [22] proposed by the Turing prize winner Micali et al. Combines POS and VRF. Nodes can become candidate nodes by token pledge, and then select some nodes through the non interactive VRF algorithm to form a consensus committee, and then these nodes execute a pbft consensus algorithm to be responsible for the rapid verification of transactions, Algorand can ensure the normal operation of the system when the node is an honest node. Ouroboros [23] proposed by kiayias et al. Introduced VRF instead of pseudo-random number in the second version of praos [24] to select the master node in the partition. Taking the VRF algorithm used by algorand and other algorithms as an example, the main process is as follows:

10000 word long article: Interpretation of blockchain 7 consensus algorithms

10000 word long article: Interpretation of blockchain 7 consensus algorithms

In the VRF designed and used in the public chain, the probability that a node is selected as an accounting node is often positively correlated with the tokens it holds. The range of consensus nodes in the public chain cannot be determined in advance. All nodes that meet the token holding conditions may become consensus nodes. The system needs to select some nodes from the nodes with random number and participation for consensus. Compared with the public chain, the number of nodes participating in the consensus of the alliance chain is limited and the nodes are known. In this case, the nodes of the alliance chain can interact through the known node list, which can effectively prevent the witch attack that may be encountered in the VRF design of the public chain.

7. Formula algorithm combined with slicing technology

Slicing technology is a technology in database, which cuts the data in the database into multiple parts and then stores them in multiple servers. Through the distributed storage of data, the search performance of the server is improved. In blockchain, fragmentation technology is a mechanism to allocate transactions to multiple consensus groups composed of node subsets for confirmation, and finally summarize all results for confirmation. Fragmentation technology has been applied in blockchains, and many blockchains have designed their own fragmentation schemes.

Luu et al. Proposed elastico protocol in 2017 and first applied slicing technology to blockchain [25]. Elastico first competes to become an accounting node in the network through POW algorithm. Then, according to the predetermined rules, these nodes are assigned to different partition committees. Each partition Committee executes traditional Byzantine fault-tolerant consensus algorithms such as pbft to package and generate transaction sets. After more than nodes sign the transaction set, the transaction set is submitted to the consensus committee. After verifying the signature, the consensus committee finally packs all transaction sets into blocks and records them on the blockchain.

Elastico has verified the availability of fragmentation technology in blockchain. In a certain scale, sharding technology can expand throughput almost linearly. However, elastico uses POW to elect consensus nodes, which also leads to the long time of random number generation process and POW competing for consensus nodes, resulting in high transaction delay. Moreover, the pbft algorithm used in each partition has high communication complexity. When the number of nodes in a single slice is large, the delay is also high.

On the basis of elastico, kokoris kogias et al. Proposed omniledger [26], replaced POW with encrypted lottery protocol to select verifier groups, and then classified verifiers into different segments through randhound protocol [27]. OmniLedger。 Omniledger still adopts pbft based consensus algorithm as the consensus algorithm in sharding [28], and introduces Atomix protocol to deal with cross sharding transactions. The communication complexity between nodes in the consensus process is high. When the number of nodes in the shard increases and cross shard transactions increase, the System TPS will decrease significantly.

Wang et al. Proposed monoxide in 2019 [29]. The slicing technology is introduced into the pow blockchain system, and the Chu Ko Nu mining algorithm is proposed, which solves the problem of computing power dispersion caused by slicing, so that each miner can slice in different slices at the same time, and improves the TPS of pow without reducing safety.

8. Summary

This paper gives an overview of the consensus algorithm in the blockchain, including a detailed analysis of the basic consensus POW in the public chain and the basic formula algorithm pbft in the alliance chain, and then introduces the new and more advanced consensus algorithm, hoping to help readers explore the field of blockchain.

reference

[1]Antonopoulos A M. Mastering Bitcoin: Unlocking Digital Crypto-Currencies[J]. Oreilly Media Inc Usa, 2015.

[2]Karame G O, Androulaki E, Capkun S. Two Bitcoins at the Price of One? Double-Spending Attacks on Fast Payments in Bitcoin.[J]. 2012.

[3]Lamport L, Shostak R, Pease M. The Byzantine Generals Problem[J]. Acm Transactions on Programming Languages & Systems, 1982,4(3):382-401.

[4]Back A. Hashcash – A Denial of Service Counter-Measure: USENIX Technical Conference, 2002[C].

[5]Kraft D. Difficulty control for blockchain-based consensus systems[J]. Peer-to-Peer Networking and Applications, 2016,9(2):397-413.

[6]Andolfatto D. The False Analogy Between Gold and Bitcoin[J].

[7]Alfidi A. The Serious Disadvantages of Bitcoin[J].

[8]Miller A, Juels A, Shi E, et al. Permacoin: Repurposing Bitcoin Work for Data Preservation[J]. 2014:475-490.

[9]Stegaroiu C E. The Advantages And Disadvantages Of Bitcoin Payments In The New Economy[J]. Annals Economy, 2018,1.

[10] Fan Jie, Yi Letian, Shu Jiwu. Review of Byzantine system technology [J]. Journal of software, 2013 (06): 1346-1360

[11]Castro M, Liskov B. Practical Byzantine fault tolerance: Symposium on Operating Systems Design and Implementation, 1999[C].

[12]Buchman E. Tendermint: Byzantine fault tolerance in the age of blockchains[D]., 2016.

[13]Yin M, Malkhi D, Reiter M K, et al. Hotstuff: Bft consensus with linearity and responsiveness, 2019[C].

[14]Desmedt Y, Frankel Y. Shared generation of authenticators and signatures, 1991[C]. Springer.

[15]Lamport L. The part-time parliament[J]. Acm Transactions on Computer Systems, 1998,16(2):133-169.

[16]Lamport L. Paxos made simple[J]. ACM Sigact News, 2001,32(4):18-25.

[17]Chandra T D, Griesemer R, Redstone J. Paxos made live: An engineering perspective: Proceedings of the Twenty-Sixth Annual ACM Symposium on Principles of Distributed Computing, PODC 2007, 2007[C].

[18]Ongaro D, Ousterhout J K. In search of an understandable consensus algorithm., 2014[C].

[19]Li W, Andreina S, Bohli J, et al. Securing proof-of-stake blockchain protocols, Oslo, Norway, 2017[C]. Springer Verlag, 2017.

[20]Dodis Y. Efficient Construction of (Distributed) Verifiable Random Functions: International Workshop on Theory & Practice in Public Key Cryptography: Public Key Cryptography, 2002[C].

[21]Melara M S, Blankstein A, Bonneau J, et al. Coniks: Bringing key transparency to end users, Washington, DC, United states, 2015[C]. USENIX Association, 2015.

[22]Gilad Y, Hemo R, Micali S, et al. Algorand: Scaling Byzantine Agreements for Cryptocurrencies, Shanghai, China, 2017[C]. Association for Computing Machinery, Inc, 2018.

[23]Kiayias A, Russell A, David B, et al. Ouroboros: A Provably Secure Proof-of-Stake Blockchain Protocol, 2017[C].

[24]David B, Gaži P, Kiayias A, et al. Ouroboros Praos: An Adaptively-Secure, Semi-synchronous Proof-of-Stake Blockchain, 2018[C].

[25]Luu L, Narayanan V, Zheng C, et al. A secure sharding protocol for open blockchains, Vienna, Austria, 2016[C]. Association for Computing Machinery, 2016.

[26]Kokoris-Kogias E, Jovanovic P, Gasser L, et al. OmniLedger: A Secure, Scale-Out, Decentralized Ledger via Sharding, Los Alamitos, CA, USA, 2018[C]. IEEE Computer Society, 2018//.

[27]Syta E, Jovanovic P, Kogias E K, et al. Scalable Bias-Resistant Distributed Randomness, Los Alamitos, CA, USA, 2017[C]. IEEE Computer Society, 2017//.

[28]Kokoris-Kogias E, Jovanovic P, Gailly N, et al. Enhancing bitcoin security and performance with strong consistency via collective signing, Austin, TX, United states, 2016[C]. USENIX Association, 2016.

[29]Wang J, Wang H. Monoxide: Scale out blockchains with asynchronous consensus zones: 16th {USENIX} Symposium on Networked Systems Design and Implementation ({NSDI} 19), 2019[C].

This article is shared from Huawei cloud community “ten thousand words long text interpretation of blockchain seven types of consensus algorithm”, original author: aptx-486977.

Click focus to learn about Huawei cloud’s new technologies for the first time~