Libp2p RS version 0.3.0 introduction

Time:2021-7-31

V0.3.0 released on 4.23, usingAsyncRead & AsyncWriteTo replace ourReadEx & WriteEx & SplitEx; SimplifiedKad/DHTImplementation logic.

modify

ReadEx & WriteEx & SplitEx:

At first we tried to useasync-traitTo define their own IO operationsTraitFor more pure useasync/awaitTo write code.

withReadExFor example, it is roughly as follows:

#[async_trait]
pub trait ReadEx {
    async fn read(&mut self, buf: &mut [u8]) -> Result<usize, io::Error>;
}

Use what we defineTraitIt does bring us some benefits without writing state machine code. The same logic can be implemented in a more understandable way; Some problems are also introduced:

  1. It is difficult to separate reading and writing
    Let’s first look at why we need to separate reading and writingasync/awaitWay compared topollThe way, lost some control, could not be in oneFutureDo many things at the same time, such as reading and writing here:
let mut socket = ...;

//Poll mode
fn poll(socket: Pin<&mut Socket>, cx: Context) {
    match socket.poll_read(cx) {
        Poll::Pending => {
            //There is an opportunity to handle both reading and writing in the poll
            socket.poll_write(cx)
        }
        Poll::Ready => {...}
    }
}

// async/await
async fn handle_socket(socket: &mut Socket) {
    socket.read().await;
    //There is no way to continue the write operation when the read is not ready
    //Since both read and write require & mut T, there is no way to use select to achieve the purpose
    // let read_fut = socket.read();
    // let write_fut = socket.write();
    // select(read_fut, write_fut).await;
}

For this reason, we need to separate reading and writing. At the same time, reading and writing are processed in different processes, and the code logic will be clearer.

To achieve read-write separation, of courseRuntimeThe underlying IO supports twoRuntimeThe camp provides different implementation methods:

  • async-stdwithCloneTo achieve the purpose of separation
  • tokioThen with the help ofBiLockRealize read-write separation

CloneOf course it’s a good way,libp2p-rsThe middle layer is divided into many layers. The IO provided by the lower layer to the upper layer has its own logic in it. In this way, there are various reasons that make it difficult for us to implementClone, even if we do it at some costCloneThat also means that a runtime is bound.

With the help ofBiLockThe way is more general, at this timeReadExThe disadvantages of the lock are obvious. After obtaining the lock, there is no way to stay at the bottom ioPendingThe lock is released in the state, which will lead to the possibility of deadlock between read and write processes.

For the above reasons, we have definedSplitEx, the IO abstraction provided by each layer needs to be implementedSplitEx, this can solve the problem, but it is not elegant enough and increases the workload.

2. Poor backward compatibility
Rust asynchronous programming official does not define standard reading and writingTraitfuturesIn LibraryAsyncRead & AsyncWriteIt can be said to be a de facto standard. Existing applications / systems are based onAsyncRead & AsyncWriteTo build, the existing application / system wants to switch tolibp2p-rsYou need to make some changes,RustMost of the code is added with generic parameterswhereThe form of the condition is limited, andRustThe modification of this limitation is often to lead the whole body.

The alternative is that we can alsoReadExOn the basis of packagingAsyncRead, but this requires additional performance overhead, which is certainly not what we want to see.

3. Cannot reusefuturesSome extended functions in the library

Kad:

Kad-DHTThe implementation of the protocol has also been modified. More specifically, fromKadDeleted inre-provide/publishFunction. We thinkre-provide/publishThe logic belongs to useKadApplications, notKadItself. In this way, it can be simplified accordinglyProvider/RecordData structure, and inRecordStore TraitAdd two ingc_xxxMethods to separateProvider/RecordPerform GC, which is received from the network by trackingProvider/RecordTime stamp to complete. Note that GC is performed only on providers / records received from the network.


Netwops is composed of a domestic senior cloud computing and distributed technology development team, which has very rich landing experience in finance, power, communication and Internet industries. At present, netwops has set up R & D centers in Shenzhen and Beijing, with a team size of 30 +, most of which are technicians with more than 10 years of development experience, respectively from professional fields such as Internet, finance, cloud computing, blockchain and scientific research institutions.
Netwops focuses on the R & D and application of secure storage technology products. Its main products include decentralized file system (DFS) and decentralized computing platform (DCP). It is committed to providing distributed storage and distributed computing platform based on decentralized network technology. It has the technical characteristics of high availability, low power consumption and low network, and is suitable for the Internet of things Industrial Internet and other scenarios.
Official account: Netwarps