Optimization of packaging speed after upgrading webpack3. X to 4. X

Time:2021-9-18

preface

LastWe talked about the optimization of package size after upgrading webpack3. X to 4. X. let’s talk about it todaywebpack 4.x(webpack 4.43.0) is actually upgrading its packaging speed optimizationwebpack4Since then, the packaging speed has been greatly improved, but the search time (narrow the scope)loaderThere is still room for optimization in processing time (multi process) and secondary packaging time (CACHE).

Packaged Analytics

Before optimization, we need to know the performance of project packaging, which we use herespeed-measure-webpack-pluginPlug in for analysis

webpack.base.js

+const SpeedMeasurePlugin = require('speed-measure-webpack-plugin');

+const smp = new SpeedMeasurePlugin({
+  outputFormat: 'humanVerbose'
+});

const webpackConfig = merge(baseWebpackConfig, {
  // ..
});

-module.exports = webpackConfig;
+module.exports = smp.wrap(webpackConfig);

Take a look at the packaging. This time is time-consuming62,361 ms, listedPluginsandLoadersSpecific time-consuming details:

The figure is relatively long, but it can be seen that the time-consuming one isvue-loaderandts-loader

Optimization of packaging speed after upgrading webpack3. X to 4. X

Find time optimization – exclude / include

webpackStarting from the entry file, find modules according to dependencies. We need as few processing modules as possible. The most common is exclusionexclude: /node_modules/excludeandincludeWhen used at the same time, exclude has higher priority

webpack.base.js

module.exports = {
  module: {
    rules: [
      {
        test: /\.vue$/,
        loader: "vue-loader",
      },
      {
        test: /\.js$/,
        loader: "babel-loader",
        //Ensure node_ The < script > part of Vue single file component in modules can be translated
        exclude: (file) => /node_modules/.test(file) && !/\.vue\.js/.test(file),
      },
      {
        test: /\.tsx?$/,
        loader: "ts-loader",
        exclude: /node_modules/,
        options: {
          //Disable type checking, which can be used as a plug-in
          transpileOnly: true,
          experimentalWatchApi: true,
          appendTsSuffixTo: [/\.vue$/],
        },
      },
    ],
  },
};

tsFor type checking, plug-ins can be used separately:ForkTsCheckerWebpackPlugin

Loader processing time optimization – thread loader

Each worker is a separate node.js process, which has an overhead of ~600ms. There is also an overhead of inter-process communication.
eachworkerIs a separatenode.jsProcess, with an overhead of about600millisecond. Interprocess communication also has overhead.See more

webpack.base.js

// ...
const isProduction = process.env.NODE_ENV === "production";

if (isProduction) {
  const threadLoader = require("thread-loader");
  //Preheat worker
  threadLoader.warmup(
    {
      // pool options, like passed to loader options
      // must match loader options to boot the correct pool
    },
    [
      // modules to load
      // can be any module
      "babel-loader",
      "ts-loader",
      "vue-loader",
      "sass-loader",
    ]
  );
}

module.exports = {
  // ...
  module: {
    rules: [
      {
        test: /\.vue$/,
        use: isProduction ? ["thread-loader", "vue-loader"] : ["vue-loader"],
      },
      {
        test: /\.js$/,
        use: isProduction
          ? ["thread-loader", "babel-loader"]
          : ["babel-loader"],
        exclude: (file) => /node_modules/.test(file) && !/\.vue\.js/.test(file),
      },
      {
        test: /\.tsx?$/,
        loader: "ts-loader",
        exclude: /node_modules/,
        use: isProduction
          ? [
              "thread-loader",
              {
                loader: "ts-loader",
                options: {
                  happyPackMode: true,
                  transpileOnly: true,
                  experimentalWatchApi: true,
                  appendTsSuffixTo: [/\.vue$/],
                },
              },
            ]
          : [
              {
                loader: "ts-loader",
                options: {
                  transpileOnly: true,
                  experimentalWatchApi: true,
                  appendTsSuffixTo: [/\.vue$/],
                },
              },
            ],
      },
      {
        test: /\.s?css$/,
        use: isProduction
          ? [
              MiniCssExtractPlugin.loader,
              "css-loader",
              {
                loader: "thread-loader",
                options: {
                  workerParallelJobs: 2,
                },
              },
              "sass-loader",
            ]
          : ["vue-style-loader", "css-loader", "sass-loader"],
      },
    ],
  },
};

Error reporting when using with TS loader

ERROR in ./src/main.ts
Module build failed (from ./node_modules/thread-loader/dist/cjs.js):
Thread Loader (Worker 0)
Cannot read property 'errors' of undefined
    at PoolWorker.fromErrorObj (/Users/yourProjectPath/node_modules/thread-loader/dist/WorkerPool.js:262:12)
    at /Users/yourProjectPath/node_modules/thread-loader/dist/WorkerPool.js:204:29
    at mapSeries (/Users/yourProjectPath/node_modules/neo-async/async.js:3625:14)
    at PoolWorker.onWorkerMessage (/Users/yourProjectPath/node_modules/thread-loader/dist/WorkerPool.js:170:35)
    at successfulTypeScriptInstance (/Users/yourProjectPath/node_modules/ts-loader/dist/instances.js:119:28)
    at Object.getTypeScriptInstance (/Users/yourProjectPath/node_modules/ts-loader/dist/instances.js:34:12)
    at Object.loader (/Users/yourProjectPath/node_modules/ts-loader/dist/index.js:17:41)

ts-loaderset uphappyPackMode: trueSee more

Use with node sass

seeWebpack documentIt was mentioned thatnode-sasscollocationthread-loaderIt needs to be set when usingworkerParallelJobs: 2, the personal test found that this setting is not needed now, and the speed will be faster

[email protected] [email protected] [email protected]

Error reporting when using sass loader

The error information is as follows:

Module build failed (from ./node_modules/thread-loader/dist/cjs.js):
Thread Loader (Worker 3)
this.getResolve is not a function

There is no solution for the time being, willsass-loaderDowngrade tov7.3.1Can operate normally,Related issue

Secondary packaging time optimization cache

usewebpack4When packing the second time, we will find that it comes a lot faster than the first time becausewebpackbuilt-interser-webpack-pluginTo minimize JS files, multi process and cache will be enabled by default, and the second time, the project will be read directlynode_modules/.cache/terser-webpack-pluginDirectory, so it will be much faster than the first packaging

Similarly, we are usingbabel-loaderYou can also enable and set the cachecacheDirectory: truethat will do

If you want to cache otherloaderThe processing results you can usecache-loader

summary

The project is not large enough and the number of server logical cores (physical core * number of threads per core) is not large enough. Do not use multi process packaging. Reverse optimization refers to methread-loaderAfter multi process packaging, local construction increases the number of processes40sAs shown in the following figure:

Optimization of packaging speed after upgrading webpack3. X to 4. X

It took a long time to finish this article, which took more than half a month. Summarize the points of delay:

  • First, I saw it halfway[webpack] 538 – Guide to improving packaging speedI think this article is well written. I didn’t want to start optimization for a time. I think it’s not as good as others. Why do I write it. Later, after hard scalp optimization, I found that the article was outdated(HappyPack)And redundancy(terser-webpack-pluginBy default, the multi process part will be enabled, and there is no practical part. I immediately feel that it is meaningful to have my own existence (this article);
  • Second, after the optimization, it was found that the effect was not good. It was once suspected that it was necessary to write on the local construction or even reverse optimization. Fortunately, it passed on the test serverCI/CDMedian construction speed50sAbout, it’s OK. I finished it with the attitude of starting and ending

Finally,workerThe overhead of startup and inter process communication is not small. It should be used in combination with the actual situation of the project. It should not be used for use. Project quality is more important than performance.

Reference link