[roaming GitHub] no compile / no server to realize the commonjs modularization of browser

Time:2021-2-27

introduction

I often visit GitHub. In addition to some big projects with extremely high star, I also find many interesting small projects on GitHub. Project or idea is very interesting, or there is a good technical point, read to let a person have harvest. So I’m going to put it together as a series of “roaming GitHub” to share and interpret the interesting projects I encounter on GitHub from time to time. This series focuses on the principle of explanation, but not the details of the source code.

[roaming GitHub] no compile / no server to realize the commonjs modularization of browser

OK, let’s get to the point. The warehouse to be introduced in this issue is called one- click.js .

1. one- click.js What is it?

one- click.js It’s an interesting library. This is how GitHub introduces it

[roaming GitHub] no compile / no server to realize the commonjs modularization of browser

We know that if we want the modularized code of commonjs to run normally in the browser, we usually need to build / package tools, such as webpack, roll up, etc. And one- click.js It allows you to run the module system based on commonjs in the browser without these building tools.

Further, you don’t even need to start a server. Try, for example, you can try clone one- click.js Project, double-click (open with browser) directlyexample/index.htmlIt’s ready to run.

There is a sentence in repo that outlines its functions:

Use CommonJS modules directly in the browser with no build step and no web server.

For example——

Suppose that in the current directory(demo/)Now, we have three “module” files:

demo/plus.js

// plus.js
module.exports = function plus(a, b) {
    return a + b;
}

demo/divide.js

// divide.js
module.exports = function divide(a, b) {
    return a / b;
}

And the entry module filedemo/main.js

// main.js
const plus = require('./plus.js');
const divide = require('./divide.js');
console.log(divide(12, add(1, 2)));
// output: 4

The common usage is to specify the entry, compile it into a bundle with webpack, and then refer to it by the browser. And one- click.js So you can get rid of this, just use it in HTML:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>one click example</title>
</head>
<body>
    <script type="text/javascript" data-main="./main.js"></script>
</body>
</html>

be carefulscriptHow to use the labeldata-mainThe entry file is specified. At this time, you can directly open the local HTML file with the browser, and you can output result 7 normally.

2. How does the packaging tool work?

The previous section described one- click.js The core is to realize the front-end modularization without packaging / building.

Before introducing its internal implementation, let’s take a look at what the packaging tool does. As the saying goes, if you know yourself and the enemy, you can win a hundred battles.

Or our three JavaScript files.

plus.js:

// plus.js
module.exports = function plus(a, b) {
    return a + b;
}

divide.js:

// divide.js
module.exports = function divide(a, b) {
    return a / b;
}

And entrance module main.js :

// main.js
const plus = require('./plus.js');
const divide = require('./divide.js');
console.log(divide(12, add(1, 2)));
// output: 4

Recall that when we use webpack, we specify the entry( main.js )。 Webpack will package a bundle (for example bundle.js )。 Finally, we introduce the processed bundle.js That’s it. At this time bundle.js In addition to the source code, a lot of “private goods” of webpack have been added.

In brief, webpack involves the following work:

  1. Dependency analysisFirst of all, when packaging, webpack will get the module dependencies according to the parsing results. In brief, in commonjs, we can get the sub modules that the current module depends on according to the parsed require syntax.
  2. Scope isolation and variable injection: for each module file, webpack wraps it in a function. This can be donemodulerequireIn addition, the scope can be isolated to prevent the global pollution of variables.
  3. Provide module runtimeFinally, forrequireexportsIt also needs to provide a set of runtime code to implement the functions of module loading, executing and exporting.

If you are not familiar with the above two or three items, you can learn about the module runtime design of webpack from this article.

3. Challenges we face

Without a build tool, you can run the commonjs module directly in the browser. In fact, you need to find a way to complete the three tasks mentioned above

  • Dependency analysis
  • Scope isolation and variable injection
  • Provide module runtime

To solve these three problems is one- click.js The core task of the project. Now let’s look at how to solve it.

3.1. Dependency analysis

It’s a troublesome question. If you want to load modules correctly, you must know exactly the dependencies between modules. For example, the three module files mentioned above——main.jsrely onplus.jsanddivide.jsSo it’s runningmain.jsWhen using code in, you need to ensure thatplus.jsanddivide.jsHave been loaded into the browser environment. However, the problem is that without a compiler, we can’t automatically know the dependencies between modules.

For a module library like requirejs, it is to declare the dependency of the current module in the code, and then use asynchronous loading and callback. Obviously, there is no such asynchronous API in the commonjs specification.

And one- click.js A clever but costly way to analyze dependencies is to load the module file twice. When the module file is loaded for the first time, a mock function is provided for the module filerequireMethod. Whenever a module calls this method, it can know which submodules the current module depends on in require.

// main.js
const plus = require('./plus.js');
const divide = require('./divide.js');
console.log(minus(12, add(1, 2)));

For example, abovemain.jsWe can provide an example similar to the followingrequiremethod:

const recordedFieldAccessesByRequireCall = {};
const require = function collect(modPath) {
    recordedFieldAccessesByRequireCall[modPath] = true;
    var script = document.createElement('script');
    script.src = modPath;
    document.body.appendChild(script);
};

main.jsAfter loading, you do two things:

  1. Record the dependent sub modules in the current module;
  2. Load submodules.

In this way, we canrecordedFieldAccessesByRequireCallRecord the dependency of the current module in; load the sub module at the same time. There can also be recursive operations for submodules until no new dependencies appear. Finally, the design of each module is introducedrecordedFieldAccessesByRequireCallThe integration is our dependence.

Besides, if we want to knowmain.jsWhich methods in the sub module are actually called can be accessed through theProxyTo return a proxy object and count further dependencies:

const require = function collect(modPath) {
    recordedFieldAccessesByRequireCall[modPath] = [];
    var megaProxy = new Proxy(function(){}, {
        get: function(target, prop, receiver) {
            if(prop == Symbol.toPrimitive) {
                return function() {0;};
            }
            return megaProxy;
        }
    });
    var recordFieldAccess = new Proxy(function(){}, {
        get: function(target, prop, receiver) {
            window.recordedFieldAccessesByRequireCall[modPath].push(prop);
            return megaProxy;
        }
    });
    // ……  Some other treatments
    return recordFieldAccess;
};

The above code will record the properties used when you get the properties of the imported module.

The loading of all the above modules is the first time of “loading twice”, which is used to analyze the dependency relationship. The second time, based on the dependency of the entry module, the module can be loaded in reverse. for examplemain.jsrely onplus.jsanddivide.jsThen the actual loading order isplus.js -> divide.js -> main.js

It is worth mentioning that in the process of loading all modules for the first time, the execution of these modules will basically report an error (because the dependent loading order is wrong). We will ignore the execution error and only focus on the dependency analysis. When you get the dependencies, reload all the module files in the correct order. one- click.js The method is calledscrapeModuleIdempotent, specific source code can see here.

Here you may find: “it’s a waste. Every file has been loaded twice.”

That’s true, and that’s one- click.js Tradeoff:

In order to make this work offline, One Click needs to initialize your modules twice, once in the background upon page load, in order to map out the dependency graph, and then another time to actually perform the module loading.

3.2. Scope isolation

We know that modules have a very important feature – the scope between modules is isolated. For example, for the following common JavaScript script:

// normal script.js
var foo = 123;

When it is loaded into the browser,fooThe variable will actually become a global variable, which can be accessed through thewindow.fooAccess to, this will also bring global pollution, variables and methods between modules may conflict and overlap with each other.

In nodejs environment, due to the use of commonjs specification, when module files like the above are imported,fooThe scope of the variable is only in the source module and will not pollute the global. Nodejs actually uses a wrap function to wrap the code in the module. We all know that the function will form its own scope, so it achieves isolation.

Nodejs will be inrequireThe source code file is packaged at compile time, while the source code file will be rewritten by the packaging tools like webpack at compile time. And one- click.js Without a compiler, rewriting at compile time will not work. What should we do? Here are two common methods:

3.2.1. Dynamic code execution of JavaScript

One way is throughfetchRequest to get the text content in script, and then through thenew FunctionorevalIn this way, dynamic code execution can be realized. Here, thefetch + new FunctionHow to make an introduction:

Or the division module abovedivide.jsWith a little modification, the source code is as follows:

//When loaded as a script, the variable becomes window.outerVar  Global variables, causing pollution
var outerVar = 123;

module.exports = function (a, b) {
    return a / b;
}

Now let’s implement scope masking:

const modMap = {};
function require(modPath) {
    if (modMap[modPath]) {
        return modMap[modPath].exports;
    }
}

fetch('./divide.js')
    .then(res => res.text())
    .then(source => {
        const mod = new Function('exports', 'require', 'module', source);
        const modObj = {
            id: 1,
            filename: './divide.js',
            parents: null,
            children: [],
            exports: {}
        };

        mod(modObj.exports, require, modObj);
        modMap['./divide.js'] = modObj;
        return;
    })
    .then(() => {
        const divide = require('./divide.js')
        console.log(divide(10, 2)); // 5
        console.log(window.outerVar); // undefined
    });

The code is very simple, the core is through thefetchAfter obtaining the source code, through thenew FunctionConstruct it in a function, and “inject” some variables of module runtime when calling it. For the code to run smoothly, a simplerequireMethod to implement the module reference.

Of course, the above is a solution, but in one- click.js But it doesn’t work. Because of one- click.js Another goal is to be able to run offline, sofetchThe request is invalid.

So one- click.js How to deal with it? Now let’s understand:

3.2.2. Another way of scope isolation

Generally speaking, isolation requirements are very similar to sandbox, and a common way to create a sandbox on the front end is iframe. Next, for convenience, we call the window actually used by the user “main window”, and the embedded iframe is called “sub window”. Because of the nature of iframe, each subwindow has its ownwindowObject, isolated from each other, will not contaminate the main window, and will not contaminate each other.

The following is still loaded with divide.js Module as an example. First, we construct an iframe to load scripts

var iframe = document.createElement("iframe");
iframe.style = "display:none !important";
document.body.appendChild(iframe);
var doc = iframe.contentWindow.document;
var htmlStr = `
    <html><head><title></title></head><body>
    <script></script></body></html>
`;
doc.open();
doc.write(htmlStr);
doc.close();

This allows you to load the module script in isolated scopes. But obviously it doesn’t work properly, so the next step is to complete its module import and export function. The problem of module export is to let the main window access the module objects in the sub window. So we can mount the script of the sub window to the variable of the main window after it is loaded and run.

Modify the above code:

// …… Omit duplicate code
var htmlStr = `
    <html><head><title></title></head><body>
    <scrip>
        window.require = parent.window.require;
        window.exports = window.module.exports = undefined;
    </script>
    <script></script>
    <scrip>
        if (window.module.exports !== undefined) {
            parent.window.modObj['./divide.js'] = window.module.exports;
        }
    </script>
    </body></html>
`;
// …… Omit duplicate code

The core of it is through the imageparent.windowIn this way, the “penetration” between the main window and the sub window is realized

  • Mount the object of the sub window to the main window;
  • At the same time, it supports the sub window to call the method in the main window.

The above is just a rough implementation of principle. If you are interested in more rigorous implementation details, you can see the loadmoduleformoduledata method in the source code.

It is worth mentioning that in “3.1. Dependency analysis”, it is mentioned that all modules should be loaded first to obtain dependencies, and this part of loading is also carried out in iframe, so “pollution” should be prevented.

3.3. Provide module runtime

The first version of runtime includes the construction of module object, storage module object and a module import method(require)。 All kinds of module runtime implementations are generally similar. It should be noted that if the isolation method uses iframe, some runtime methods and objects need to be passed in the main window and sub windows.

Of course, details may also need to support module path resolution, circular dependency processing, error handling, etc. Because the implementation of this part is similar to many libraries, or it is not a special core, I will not introduce it in detail here.

4. Summary

Finally, the general operation process is summarized as follows:

  1. First, get the entry module from the page, and then click one- click.js Middle isdocument.querySelector("script[data-main]").dataset.main
  2. The module is loaded in iframe, and therequireMethod to collect module dependencies until no new dependencies appear;
  3. After collection, you get the complete dependency graph;
  4. According to the dependency graph, “reverse” loading the corresponding module file, using iframe to isolate the scope, at the same time pay attention to pass the module runtime in the main window to each sub window;
  5. Finally, when the entry script is loaded, all the dependencies are ready to be executed directly.

In general, it is difficult to achieve dependency analysis and scope isolation without the help of build tools and servers. And one- click.js These problems are solved by using the above-mentioned technical means.

So, one- click.js Can it be used in a production environment? Obviously not.

Do not use this in production. The only purpose of this utility is to make local development simpler.

So pay attentionThe author also said that the purpose of this library is only to facilitate local development. Of course, some of the technical means as learning materials, we can also learn about. Interested partners can visit one- click.js Learn more about the warehouse.


OK, that’s it for this issue of “roaming GitHub.”. From time to time, we will have a look, chat and learn some interesting projects on GitHub. We can not only learn some technical points, but also understand the author’s technical thinking. We welcome interested partners to pay attention.

[roaming GitHub] no compile / no server to realize the commonjs modularization of browser


Previous content

  • The implementation principle of [roaming GitHub] QuickLink and Its Inspiration to the front end
  • How to improve [roaming GitHub] JSON.stringify () performance?

Recommended Today

Quickly use the latest 15 common APIs of vue3

Before that, I wrote a blog to introduce the new features of vue3. I had a brief understanding of the features of vue3, and at the end of the article, I gave you a little experience in vue3Compsition APISimple use of Address of last article: follow Youda’s steps and experience the new features of vue3 […]