Bytom side chain vapor source code analysis – node block out process

Time:2020-9-10

In this article, the author will start from the creation of vapor node, and then expand the explanation of the source code involved in the process of vapor node block.

As the first part of the series of source code analysis of vapor, this paper first introduces vapor. Vapor is a high-performance side chain of Bytom, which is the mainstream public chain in China. It is an independent high-performance side chain developed from Bytom main chain. Vapor is one of the most important blockchain infrastructure of the platform. At present, dpos consensus algorithm is used to build large-scale commercial applications with high performance, high security and scalability.

Creation of vapor node and start of block module

Vapor entry function:

vapor/cmd/vapord/main.go

func main() {
    cmd := cli.PrepareBaseCmd(commands.RootCmd, "TM", os.ExpandEnv(config.DefaultDataDir()))
    cmd.Execute()
}

After passing in the parameter node, the runnode function is called and a new node is created.

vapor/cmd/vapord/commands/run_node.go

func runNode(cmd *cobra.Command, args []string) error {
    startTime := time.Now()
    setLogLevel(config.LogLevel)

    // Create & start node
    n := node.NewNode(config)
    ……
}

The structure of vapor node is as follows

vapor/node/node.go

type Node struct {
    cmn.BaseService

    config          *cfg.Config
    eventDispatcher *event.Dispatcher
    syncManager     *netsync.SyncManager

    wallet          *w.Wallet
    accessTokens    *accesstoken.CredentialStore
    notificationMgr *websocket.WSNotificationManager
    api             *api.API
    chain           *protocol.Chain
    blockProposer   *blockproposer.BlockProposer
    miningEnable    bool
}

Among them, what is related to block and consensus isblockProposerfield

Partial source code of new node

vapor/node/node.go

func NewNode(config *cfg.Config) *Node {
    //……
    node := &Node{
        eventDispatcher: dispatcher,
        config:          config,
        syncManager:     syncManager,
        accessTokens:    accessTokens,
        wallet:          wallet,
        chain:           chain,
        miningEnable:    config.Mining,

        notificationMgr: notificationMgr,
    }

    node.blockProposer = blockproposer.NewBlockProposer(chain, accounts, txPool, dispatcher)
    node.BaseService = *cmn.NewBaseService(nil, "Node", node)
    return node
}

You can see from this node.blockProposer It is essentially a vapor block generator. The module that actually controls the node to start the block is vapor / proposal / blockproposer/ blockproposer.go In:

func (b *BlockProposer) Start() {
    b.Lock()
    defer b.Unlock()

    // Nothing to do if the miner is already running
    if b.started {
        return
    }

    b.quit = make(chan struct{})
    Go b.generateblocks() // the key modules of block output function

    b.started = true
    log.Infof("block proposer started")
}

The block out module can be started through the API

vapor/api/miner.go

func (a *API) startMining() Response {
    a.blockProposer.Start()
    if !a.IsMining() {
        return NewErrorResponse(errors.New("Failed to start mining"))
    }
    return NewSuccessResponse("")
}

The above explanation is the source code involved in node creation and block module startup.

fromgenerateBlocks()Function start, will explain the specific source code of the vapor out block process.

The blocking mechanism of vapor

Vapor uses the consensus mechanism of dpos to block. Dpos is created by a trusted account (trustee, top 10 votes) elected by the community. In order to become the official trustee, users should go to the community to canvass votes and gain enough trust from users. Users vote based on the percentage of cryptocurrency they hold in total. Dpos mechanism is similar to the joint-stock company, ordinary shareholders can not enter the board of directors, they have to vote to elect representatives (Trustees) to make decisions on their behalf. Before explaining the flow of vapor out of the block, we should first understand the parameter setting of vapor in dpos.

The parameter information of dpos is located in vapor / consensus/ general.go

type DPOSConfig struct {
    NumOfConsensusNode      int64
    BlockNumEachNode        uint64
    RoundVoteBlockNums      uint64
    MinConsensusNodeVoteNum uint64
    MinVoteOutputAmount     uint64
    BlockTimeInterval       uint64
    MaxTimeOffsetMs         uint64
}

Next, the parameters are explained in detail

  • Numofconsensusnode is the number of consensus nodes in dpos, which is set to 10 in vapor. Ten consensus nodes responsible for block output are selected by voting.
  • Blocknumeachnode is the number of consecutive blocks from each consensus node, which is set to 12 in vapor.
  • Roundvoteblocknums is the number of blocks out of each round of voting, which is set to 1200 in the vapor. That is to say, the consensus node generated by each round of voting will be responsible for 1200 blocks.
  • Minconsensus nodevotenum is the minimum number of BTMS required to become a consensus node (unit: neu, one hundred million BTM). It is set to 10000000000 in the vapor. That is to say, if a node wants to become a consensus node, it needs to have at least 1 million BTM in the account.
  • Minvoteoutputmaoun is the minimum number of BTM (unit: neu) required by the node to vote. It is set to 100000000 in the vapor. If a node wants to participate in voting, it needs 1btm in the account
  • Blocktimeinterval is the shortest time interval between blocks, and vapor generates a block every 0.5 seconds.
  • Maxtimeoffsetms is the maximum number of seconds that the block time is allowed to be earlier than the current time, which is set to 2 seconds in vapor.

After the dpos parameter setting, you can see the core code of the block on the vaporgenerateBlocks

vapor/proposal/blockproposer/blockproposer.go

func (b *BlockProposer) generateBlocks() {
    xpub := config.CommonConfig.PrivateKey().XPub()
    xpubStr := hex.EncodeToString(xpub[:])
    ticker := time.NewTicker(time.Duration(consensus.ActiveNetParams.BlockTimeInterval) * time.Millisecond)
    defer ticker.Stop()

    for {
        select {
        case <-b.quit:
            return
        case <-ticker.C:
        }
        //1
        bestBlockHeader := b.chain.BestBlockHeader()
        bestBlockHash := bestBlockHeader.Hash()
        now := uint64(time.Now().UnixNano() / 1e6)
        base := now
        if now < bestBlockHeader.Timestamp {
            base = bestBlockHeader.Timestamp
        }
        minTimeToNextBlock := consensus.ActiveNetParams.BlockTimeInterval - base%consensus.ActiveNetParams.BlockTimeInterval
        nextBlockTime := base + minTimeToNextBlock
        if (nextBlockTime - now) < consensus.ActiveNetParams.BlockTimeInterval/10 {
            nextBlockTime += consensus.ActiveNetParams.BlockTimeInterval
        }
        
        //2
        blocker, err := b.chain.GetBlocker(&bestBlockHash, nextBlockTime)
        ……
        if xpubStr != blocker {
            continue
        }
        
        
        //3
        warnDuration := time.Duration(consensus.ActiveNetParams.BlockTimeInterval*warnTimeNum/warnTimeDenom) * time.Millisecond
        criticalDuration := time.Duration(consensus.ActiveNetParams.BlockTimeInterval*criticalTimeNum/criticalTimeDenom) * time.Millisecond
        block, err := proposal.NewBlockTemplate(b.chain, b.accountManager, nextBlockTime, warnDuration, criticalDuration)
        ……
        //4
        isOrphan, err := b.chain.ProcessBlock(block)
        ……
        //5
        log.WithFields(log.Fields{"module": logModule, "height": block.BlockHeader.Height, "isOrphan": isOrphan, "tx": len(block.Transactions)}).Info("proposer processed block")

        if err = b.eventDispatcher.Post(event.NewProposedBlockEvent{Block: *block}); err != nil {
            log.WithFields(log.Fields{"module": logModule, "height": block.BlockHeader.Height, "error": err}).Error("proposer fail on post block")
        }
    }
}

The code is simplified, omitting some unimportant parts, and dividing the important parts into five modules.

  1. Calculate and adjust the block time
  2. adoptGetBlockerGet the public key of the next block in order, and compare it with the current block to determine whether the block order of the current block is legal.
  3. adoptb.chain.ProcessBlockA block is generated based on the template.
  4. adoptchain.ProcessBlock(block)Try to process the block and add it to the local block chain.
  5. Use logrus framework to record new blocks and broadcast them like network.

b.chain.GetBlocker

in the light ofgenerateBlocks()Several important modules in the split explanation.

vapor/protocol/consensus_node_manager.go

Getblocker() passes in the hash of the current height block and the block out time of the next block.

//Returns a blocker with a specific timestamp
func (c *Chain) GetBlocker(prevBlockHash *bc.Hash, timeStamp uint64) (string, error) {
    consensusNodeMap, err := c.getConsensusNodes(prevBlockHash)
    //……

    prevVoteRoundLastBlock, err := c.getPrevRoundLastBlock(prevBlockHash)
    //……
    
    startTimestamp := prevVoteRoundLastBlock.Timestamp + consensus.ActiveNetParams.BlockTimeInterval
    //Get order, xpub is the public key
    order := getBlockerOrder(startTimestamp, timeStamp, uint64(len(consensusNodeMap)))
    for xPub, consensusNode := range consensusNodeMap {
        if consensusNode.Order == order {
            return xPub, nil
        }
    }
    //……
}
  • By callingc.getConsensusNodes()Get a map to store the consensus node.
  • Get the last block of the last round of voting, add the shortest block time interval, and calculate the start time stamp of this round.
  • callgetBlockerOrderBy using the start time stamp and the time stamp of the current block to be exported, the order of the block at this time point is calculated.
  • Final comparisonconsensusNodeMapinconsensusNode.Order, and returns the public key.

This module is to find the consensus node of the block corresponding to the current timestamp, and return the public key of the node. Because the nodes and order of blocks in dpos must be fixedgenerateBlocks()The consensus node that the module tries to block is not necessarily the legal one at the current time, so the module needs to verify the node qualification by comparing the public key.

proposal.NewBlockTemplate

vapor/proposal/proposal.go

func NewBlockTemplate(chain *protocol.Chain, accountManager *account.Manager, timestamp uint64, warnDuration, criticalDuration time.Duration) (*types.Block, error) {
    builder := newBlockBuilder(chain, accountManager, timestamp, warnDuration, criticalDuration)
    return builder.build()
}
func newBlockBuilder(chain *protocol.Chain, accountManager *account.Manager, timestamp uint64, warnDuration, criticalDuration time.Duration) *blockBuilder {
    preBlockHeader := chain.BestBlockHeader()
    block := &types.Block{
        BlockHeader: types.BlockHeader{
            Version:           1,
            Height:            preBlockHeader.Height + 1,
            PreviousBlockHash: preBlockHeader.Hash(),
            Timestamp:         timestamp,
            BlockCommitment:   types.BlockCommitment{},
            BlockWitness:      types.BlockWitness{Witness: make([][]byte, consensus.ActiveNetParams.NumOfConsensusNode)},
        },
    }

    builder := &blockBuilder{
        chain:             chain,
        accountManager:    accountManager,
        block:             block,
        txStatus:          bc.NewTransactionStatus(),
        utxoView:          state.NewUtxoViewpoint(),
        warnTimeoutCh:     time.After(warnDuration),
        criticalTimeoutCh: time.After(criticalDuration),
        gasLeft:           int64(consensus.ActiveNetParams.MaxBlockGas),
        timeoutStatus:     timeoutOk,
    }
    return builder
}

On the vapor, each block has a block header and a block body. The chunk header contains the version number, height, hash and timestamp of the previous block. The body includes the reference module of the blockchain, account manager, chunk header, transaction status (version number and verification status), utxo view, etc. The purpose of this part is to package all kinds of information of the block into a block through the template and give it to the followingProcessBlock(block)Processing.

b.chain.ProcessBlock

vapor/protocol/block.go

func (c *Chain) ProcessBlock(block *types.Block) (bool, error) {
    reply := make(chan processBlockResponse, 1)
    c.processBlockCh <- &processBlockMsg{block: block, reply: reply}
    response := <-reply
    return response.isOrphan, response.err
}
func (c *Chain) blockProcesser() {
    for msg := range c.processBlockCh {
        isOrphan, err := c.processBlock(msg.block)
        msg.reply <- processBlockResponse{isOrphan: isOrphan, err: err}
    }
}

Obviously, this is only the entrance of chain update, and the block data passes through theprocessBlockMsgStructure passed inc.processBlockChThis pipe. The data were then passedblockProcesser()After processing, themsg.replyPipeline, and the last one to handle this block isprocessBlock()Function:

func (c *Chain) processBlock(block *types.Block) (bool, error) {
    //1
    blockHash := block.Hash()
    if c.BlockExist(&blockHash) {
        log.WithFields(log.Fields{"module": logModule, "hash": blockHash.String(), "height": block.Height}).Debug("block has been processed")
        return c.orphanManage.BlockExist(&blockHash), nil
    }
    //2
    c.markTransactions(block.Transactions...)
    //3
    if _, err := c.store.GetBlockHeader(&block.PreviousBlockHash); err != nil {
        c.orphanManage.Add(block)
        return true, nil
    }
    //4
    if err := c.saveBlock(block); err != nil {
        return false, err
    }
    
    bestBlock := c.saveSubBlock(block)
    bestBlockHeader := &bestBlock.BlockHeader

    c.cond.L.Lock()
    defer c.cond.L.Unlock()
    //5
    if bestBlockHeader.PreviousBlockHash == c.bestBlockHeader.Hash() {
        log.WithFields(log.Fields{"module": logModule}).Debug("append block to the end of mainchain")
        return false, c.connectBlock(bestBlock)
    }
    //6
    if bestBlockHeader.Height > c.bestBlockHeader.Height {
        log.WithFields(log.Fields{"module": logModule}).Debug("start to reorganize chain")
        return false, c.reorganizeChain(bestBlockHeader)
    }
    return false, nil
}

processBlock()Function returnedboolIndicates whether a block is an isolated block.

  1. Judge whether the block is already in the chain through the hash of the block. If it already exists, an error is reported and false is returned (indicating that the block is not an orphan block)
  2. Mark the transactions in the block, which will be called laterc.knownTxs.Add()Add transactions to the transaction collection.
  3. Determine whether it is an orphan block. If so, the module processing in the management part of the orphan block is called and returns true.
  4. Save block insaveBlock()The signature will be verified in the and.
  5. bestBlockHeader.PreviousBlockHash == c.bestBlockHeader.Hash()The new block is added to the end of the chain.
  6. bestBlockHeader.Height > c.bestBlockHeader.HeightIndicates that a fork has occurred and needs to be rolled back.

summary

This article starts from the vapor setting out of the block to the end of the block process, and analyzes the source code involved in the block and block out part of the node at every level of detail. Although the length of this paper has been relatively long, there are still important issues not explained clearly. For example,generateBlocks()In the second point, the program will check the order of block output, but how to obtain the order of block output has not been analyzed in detail.

Then, the next article will analyze the details of dpos mechanism in vapor at source level.

Recommended Today

How to build desktop application easily with HTML, CSS and JavaScript

Can HTML, CSS and JavaScript really be used to build desktop applications? The answer is yes. In this article, we will focus on how electron uses web technologies such as HTML, CSS, and JavaScript to create desktop applications. Electron Electron can be used to build desktop applications using HTML, CSS, and JavaScript. These applications are […]