Upload and download process of HDFS

Time:2022-6-7

HDFS upload process

First, the client requests the NN to upload files
NN after checking whether the response client can upload
After the client gets the consent, it requests NN to upload the DN of the first file
NN the address returned to the client DN
The client establishes a connection with one of DN1, then DN1 connects with DN2, and DN2 connects with DN3
DN3, DN2, DN1, the client responds that the connection is successful
The client starts uploading the first block to DN1 in packets. After receiving a packet, DN1 saves one and sends it to DN2. Then DN2 transmits one to DN3,
When all three DNS have saved blocks, the first block is uploaded and the second block can be uploaded

HDFS download process

The client requests NN to download a file,
NN finds the file address by querying the metadata and returns the file address to the client,
The client selects a dn to start requesting data
DN starts to transmit data to the client in the form of packet
The client gets all the packets and merges them into a complete file
**The download process can be performed concurrently, that is, different DNS can be found to download different block files at the same time