Breakpoint download, here is what you want to know

Time:2020-8-27

When you want to use onprogress to display progress in a recent download request, it is found that the total displayed in onprogress is 0.

Breakpoint download, here is what you want to know

Why?

I don’t know if you’ve ever encountered some downloads when you download files with download software. Some downloads can display the total size, while others can’t be displayed. It looks like there’s a bug. In fact, this reason is similar to the situation that total is always 0 in onprogress.
Breakpoint download, here is what you want to know

If you want to find out the reason, you should first understand the principle of downloading files, and the normal file download is breakpoint continuous transmission. You can start from this aspect.

In order to allow the file download to pause and then re download from the pause download section, it is necessary to learn about thecontent-lengthAccept-RangesContent-RangealsoRange

content-length: used for the response header, indicating the byte size of the response content

Accept-Ranges: used in the response header to inform the client that a range request can be made. The following value represents the unit of content returned, usually bytes, such as: accept- Ranges:bytes

Content-Range: used for response header to describe the scope and overall length of response request content, such asContent-Range: bytes 201-220/326Indicates that the server returns the contents in the range of 201 to 220bytes in the request resource. The total size of the request resource is 326 bytes. If the total size is unknown, it will be displayedContent-Range: bytes 201-220/\*

Range: used in the request header to tell the server which part of the content to return, such asRange:bytes=500-1000Tells the server that I want to take 500 to 1000 bytes of content in this file.

utilizeHTTPMediumRangeContent-RangeCan achieve breakpoint download.

We can use itajaxTo simulate a breakpoint download, the code is as followsnginxOne of the serversindex.htmlDocuments.

let entryContentLength = 0,
  entryContent = "";
getContentLength("http://localhost:8083/test/index.html").then(res => {
  if (res) {
    entryContentLength = res;
  } else {
    Entrycontentlength = unable to know the length;
  }
  sectionDownload(0, 20, "http://localhost:8083/test/index.html");
});

function sectionDownload(start, end, url) {
  const xhr = new XMLHttpRequest();
  xhr.open("GET", url);
  xhr.setRequestHeader("Range", `bytes=${start}-${end}`);
  xhr.onload = function() {
    if (xhr.status == 206) {
      entryContent += xhr.response;
      //Ask for a part of it
      sectionDownload(end + 1, end + 20, url);
    } else if (xhr.status == 416) {
      //Complete download after a series of operations
      console.log(
        "The content obtained is: n" + entrycontent,
        "+ entrycontentlength: \ \ n" + entrycontentlength
      );
    } else if (xhr.status == 200) {
      console.log(
        "The content obtained is: n" + entrycontent,
        "+ entrycontentlength: \ \ n" + entrycontentlength
      );
    } else {
      console.log(xhr);
    }
  };
  xhr.send();
}
function getContentLength(url) {
  return new Promise(resolve => {
    const xhr = new XMLHttpRequest();
    xhr.open("HEAD", url);
    xhr.onload = function() {
      resolve(xhr.getResponseHeader("content-length"));
    };
    xhr.send();
  });
}

Let’s talk about the logic of this code. This code first sends one to the serverHEADRequest to get the size of the response header content length, that is, the requestindex.htmlThe size of, and then start gettingindex.htmlContent, only 20 bytes at a time, and pieced togetherentryContentVariable. When no bytes are returned, thenentryContentIt’s the whole thing index.html The content of.

HTTP response header with content returned:

Breakpoint download, here is what you want to know

HTTP response header when request scope cannot be satisfied:

Breakpoint download, here is what you want to know

As you can see, when some bytes are returned in the response, the returned status is206When the range of bytes requested by the client exceeds the size of the requested resource, the status code returns416206The status code indicates that part of the data of the resource has been captured,416expressRangeThe requested resource range cannot be satisfied. We can judge whether to continue the request according to the return status, so as to determine whether the file download is completed.

So let’s go back to the first question. At this time, you will find that the total file size is obtained from the beginning. Why do some downloads display the size of downloaded files, while others do not.

At this point, if your server is turned ongzipCompression, you use itHEADWhen you ask for discovery, you will find outHTTPNo response headercontent-lengthBack, the picture below is meNginxIt’s ongzipAfter compressionHEADThe response returned by the server during the request. You can find that the response header does notcontent-length。 Do you know why the total size is not displayed when downloading files.

Breakpoint download, here is what you want to know

Sometimes if the server turns on compression or in order to reduce CPU pressure, it will not calculate the total size of the file. At this time, the total size of resources cannot be obtained from the response header

However, if you useNodeThe server,No plug-ins are usedYou find that even if you ask to bring itRange:bytes=xx-xxWhen the request header and the file content are still fully retrieved, you will find that,The ability of segmented request download depends on the server, nginx, Apache and other servers have their own implementation methodsNodeHow to implement the server?

The following code is based onkoaThe implementation of the framework has the function of downloading files in sections.

const fs = require("fs");
const path = require("path");
const Koa = require("koa");
const app = new Koa();
const PATH = "./public";

app.use(async ctx => {
  const file = path.join(__dirname, `${PATH}${ctx.path}`);
  //1. 404 inspection
  try {
    fs.accessSync(file);
  } catch (e) {
    return (ctx.response.status = 404);
  }
  //ctx.set('content-encoding', 'gzip');
  const method = ctx.request.method;
  const { size } = fs.statSync(file);
  //2. Respond to the head request and return the file size
  if ("HEAD" == method) {
    return ctx.set("Content-Length", size);
  }
  const range = ctx.headers["range"];
  //3. Inform the browser that partial requests can be made
  if (!range) {
    //Here, if the client is not a segmented request, the entire file is returned
    ctx.body = fs.createReadStream(file);
    return ctx.set("Accept-Ranges", "bytes");
  } else {
    const { start, end } = getRange(range);
    //4. Scope of inspection request
    if (start >= size) {
      ctx.response.status = 416;
      return ctx.set("Content-Range", `bytes */${size}`);
    }
    //5, 206 partial response
    ctx.response.status = 206;
    ctx.set("Accept-Ranges", "bytes");
    ctx.set("Content-Range", `bytes ${start}-${end ? end : size - 1}/${size}`);
    ctx.body = fs.createReadStream(file, { start, end });
  }
});

app.listen(3000, () => console.log("partial content server start"));

function getRange(range) {
  const match = /bytes=([0-9]*)-([0-9]*)/.exec(range);
  const requestRange = {};
  if (match) {
    if (match[1]) requestRange.start = Number(match[1]);
    if (match[2]) requestRange.end = Number(match[2]);
  }
  return requestRange;
}

As you can see, it is very simple to download in sectionsNodeAccording to the request headerRangeRead the file binary stream in segments.

reference resources:
https://blog.csdn.net/weixin_33836874/article/details/88720882