Data concurrent loading strategy based on dynamic static separation

Time:2021-1-17

Author: Mo Guanzhao

Please indicate the source, keep the original link and author information


preface

Nowadays, many large web applications, especiallySPAAll of them are adoptedSeparation of motion and staticsIt’s a good strategy. On the description of the separation of motion and static, here is a good blogStatic processing of website — strategy of separating dynamic from static

I’m a front-end student. I had the honor to work with a back-end student who pursues the ultimate performance to develop this kind of dynamic static separation web projectTraditional sequential modeSingle data concurrent mode (hereinafter referred to as single concurrent mode)Multi channel data concurrent mode (hereinafter referred to as multi-channel concurrent mode)Let’s talk about the front end of this kind of applicationloadMy experience in this field. The examples in this article are all from this project.

1. Traditional sequential mode

In general, the browser will first receive a static page, which will contain tags referenced by style files and script files (pictures are not discussed here). as fordataHere are two ways to know where to come from:

  • Script request get
    Usually, after the script is loaded, the script will execute a piece of code to send the request data to the server, and then retrieve the data through the callback function and do the initialization work. This process is as follows:Request page => Render Page => Load script => Request data => The data is initialized with the script => Initialization completeThat is to say, from loading the application to starting the application is executed in the form of sequential tasks.

  • Fill directly in hidden label
    The server can also directly fill the data into a hidden tag in the web page, and then send it back to the client, that is, in the above orderget dataPut inPage requestBefore. After that, you can get the corresponding data directly in the scriptDOMThe content of the file, that is, data, is used for initialization.

These two methods have their own advantages and disadvantages, because they are not the focus of this paper, so I will directly introduce them here. But I prefer the former.

1.1 workflow diagram

If you use the idea of workflow to understand it, it can be as follows (the first way)

Data concurrent loading strategy based on dynamic static separation

1.2 result analysis

Data concurrent loading strategy based on dynamic static separation

  • Here we only study the data and main.js The loading of the.

  • Base64.css is a Base64 string used to store some small pictures, and it allows delayed loading, so it can be classified as picture resources.

The overall situation is acceptable. After all, the back-end students have made great efforts in caching. Users will500msYou can see the content of the page. It’s time600msThen the program can be started.

The advantages of this mode are obvious. This sequential loading startup mode is easy to use and maintain, and can also give full play to the advantages of static dynamic separation.
However, we think that if we put the data request in the figure above in front of the script and make concurrent requests, it may reduce the time required for loading and starting the whole page, and the back-end students also think that this loading effect will be more effectivevisualneat……

So there is the following research.

2. Single concurrent mode

To realize the concurrent loading of data and script, the most important thing is to make the data load independent of script

  1. Add one in the headscript, insert a transmissionajaxRequest code to send data request to the server.

  2. Again, add onescriptSet its SRC as the value of the data requesturlTo reference external data resources.

In terms of execution efficiency, 1 is one more step than 2, so 2 is chosen for discussion in this paper.

2.1 difficulties and Solutions

How to ensure that the external download of script tag does not block the download of other resources?

Put the script in the head tag. When downloading the external script introduced by script, the browser is blocked. When the network is bad or the script file is too large, the page is in a blank pause state. This experience is very bad.

  • We usually put the script file at the bottom of the page to reduce the blocking effect caused by the download and operation of the script, and this can ensure that the page elements referenced in the script have been rendered.

  • The data request has nothing to do with the page elements. Here we hope that it can be placed in the header to ensure that it can start loading as soon as possible to request together with other resources, but not block the download of other resources.

The browser will immediately load and parse the scripts marked with async attribute, and the script will be executed asynchronously relative to the rest of the page (when the page continues to parse, the script will be executed).

The solution here is to useHTML5OfasyncAttribute, which is applied to the script related to data request, can achieve the effect of concurrent loading of script and data. The code is as follows:

script(src="/Table/Data" type="text/javascript" async="async")

How to ensure the correct page startup when the sequence of data and script loading is unknown?

javascriptIs an analytic language that is executed when it is loaded.

At this time, the data request becomes a script tag, that is to say, it can become an assignment related tagjavascriptCode, directly put the results in the public environment. If you don’t turn it into an assignment code, based onThe introduction aboveThe possible data will become a part of the environmentAnonymous objectAnd can’t be accessed again later. In this way, after the script is recorded, you can directly reference the result to start the page. So here’s the problem

  1. Based on the aboveasyncIn practice, we may not be able to 100% guarantee the sequence of data and script loading. It is true that the size of resources determines the loading time to a certain extent, but the network transmission also has many unstable factors.

  2. We can’t be directly in any onescriptDirectly refer to the other party’s resources in theundefinedIt’s a mistake.

  3. As a last resort, we should not use polling to solve the concurrency problem. This kind of application performance is too low, contrary to our original intention.

Since they are interdependent, and we only need one party to refer to the other party’s resources to complete the startup we need. Here, we only need to expose the resources to the public environment before loadingwindowAfter the loading party is aware of it, it can start it directly by reference.

For data and script, we define their resources as follows:

name resources describe
data allData(Object) Store all dynamic data
script mainInitByData(Function) Main boot function

In the data request, the code is:

var allData = window.allData = '{"name":"data"}';

//Check whether the resources of the script exist
if (typeof window.mainInitByData !== 'undefined') {
    mainInitByData(JSON.parse(allData));
};

The relevant fragments in the script are:

var mainInitByData = window.mainInitByData = function(data) {
    //TODO...
}

if (typeof window.allData !== 'undefined') {
    mainInitByData(JSON.parse(allData));
}

2.2 workflow diagram

Data concurrent loading strategy based on dynamic static separation

2.3 result analysis

Data concurrent loading strategy based on dynamic static separation

It is not difficult to find that after parallel processing, the efficiency of loading pages is greatly increased compared with the previous sequential mode. And the page program can also start smoothly (here you can try).

Unexpectedly, one or two months later, the back-end students put forward their hopeMulti channel dataConcurrent requests, because there is also some data in the dynamic dataRelative to a period of timeFor static, this part of data can be processed by cache, and other data can be directly obtained from other servers, which can further improve the concurrency efficiency. Things are getting more and more interesting, as well as the following research.

3. Multiple concurrent mode

3.1 “inheritance” single concurrency

At this point, suppose that we need to request three pieces of data a, B, C, where a is relatively static data, we can make the following definition:

name resources describe
Sub data a AData(Object) Store a’s relative static data
Sub data B BData(Object) Store dynamic data of B
Sub data C CData(Object) Store dynamic data of C
script mainInitByData(Function) Main boot function

If you continue to use the strategy of single concurrency, the relevant fragment code of the script is:

var mainInitByData = window.mainInitByData = function(dataA, dataB, dataC) {
    //TODO...
}

if (typeof window.dataA !== 'undefined' && window.dataB !== 'undefined' && window.dataC !== 'undefined') {
    var dataA = JSON.parse(dataA),
        dataB = JSON.parse(dataB),
        dataC = JSON.parse(dataC);
    mainInitByData(dataA, dataB, dataC);
}

The above data is only an example, which does not mean that this kind of problem can be solved. If one day the back end suddenly requests to load 10 pieces of data simultaneously, the code will become very redundant.

Since we have to deal with concurrency, the idea of single concurrency can be used, but the direction here is not right.

Let’s think about it from a different perspective. The script still checks with the data, but the data contains all the child data. Here I call it the parent data directly. What about the sub data?

3.2 deal with data integration with the idea of semaphore

The reason is that it is the idea of semaphore, not semaphore, because semaphore itself isMultithreading and multitasking synchronizationFor JavaScript with async tag, it isSingle thread asynchronousBut it doesn’t mean that JavaScript can’t use the idea of semaphore, which is to solve the concurrency problem. Please refer to the specific definition of semaphore.

In order to better describe the process of borrowing ideas, the following definitions are made:

  1. There is a common semaphore between the parent data and the child data. The child data uses this semaphore to integrate the data, while the parent data uses this semaphore to initialize and start the script.

  2. Each time the child data is loaded, the semaphore is released, and its own data is integrated into the parent data.

  3. It is assumed that the order of applying semaphores between child data is unknown, but it must precede the parent data.

  4. The integrated data and semaphore are placed in a JS objectintegrateDataThey are named asdatasem(its value is 1-number of sub data), i.eintegrateData = {data: {}, sem: -2}

It may be necessary to adjust the format of the sub data. Become the following types to facilitate integration

{"message":"success", "data": {....}}

Then the processing code for all sub data is as follows:

var result = 'JSON';
var integrateData = window.integrateData || (window.integrateData = { data: {}, sem: 1 - 3 });
var onDataCallback = window.onDataCallback || (window.onDataCallback = function(result_, integrateData) {
    function dataIsReady(integrateData) {
        return integrateData.sem > 0;
    }

    function dataReadyCallback(integrateData) {
        integrateData.sem--;
        //Parent data and script startup
        var mainInitBydata = window.mainInitBydata;
        if (typeof mainInitBydata === "function") {
            mainInitBydata(integrateData);
        }
        integrateData.sem++;
    }

    if (dataIsReady(integrateData)) {
        Alert ("illegal request");
        return;
    }
    var result = result_;
    if (typeof result_ === "string") {
        result = JSON.parse(result_);
    }

    //Data integration
    if (result.message === "success") {
        var data = result.data;
        for (var key in data) {
            integrateData.data[key] = data[key];
        }
    }
    //Release semaphore
    integrateData.sem++;
    //Check the semaphore
    if (dataIsReady(integrateData)) {
        dataReadyCallback(integrateData);
    }
});
onDataCallback(result, integrateData);

At this point, the relevant code in the script is:

var mainInitByData = window.mainInitByData = function(integrateData) {
    //TODO...
}

var integrateData = window.integrateData;

//There is no need to worry about conflicts here, because JS is executed by a single thread. After the integration of the child data, the parent data will be directly executed to check the script resources. Therefore, when SEM > 0, the parent data is in the ready state.
if (integrateData && integrateData.sem > 0) {
    mainInitBydata(integrateData)
}

3.3 workflow diagram

Data concurrent loading strategy based on dynamic static separation

3.4 result analysis

Data concurrent loading strategy based on dynamic static separation

In fact, compared with single concurrency, the efficiency is not much improved, mainly because the dynamic data involved is not large, and each time the request message and response message are sent, there will be a certain size of header, resulting in unnecessary overhead. But if the dynamic data is big enough, this strategy can play a big role. At the same time, the two-way check in single concurrent mode can also be realized with the idea of semaphore.

summary

To sum up the above models, we can draw the following conclusions:

pattern efficiency Ease of use Main factors affecting performance Applicable scenarios
order ordinary easily The sum of data and program sizes General small projects
Single concurrent Higher than sequential mode ordinary Data to program size ratio Most of the static and dynamic separation of website applications
Multichannel concurrency Generally higher than single concurrency, when the data is too small, the efficiency will be lower than single concurrency complex Proportion of data Website applications with huge data, especially the relatively uniform proportion of data

In addition, one of the major defects of single concurrency and multi concurrency is the complexity of codeCouplingFor multi-channel concurrency, if there is a gap between the sub data requestsDependencyMay also define a variety of different semaphores, which is not conducive to management.

Using existing tools such asEventProxy, which can well manage these concurrent requests, including the dependencies between tasks. By eventsubscribeAndtriggerThe form of can let the program know the current situation betterTasks completedTo trigger the corresponding callback function for processing.

I hope this article can bring some help to readers.

Finally, I’d like to make a small advertisement and welcome to follow my GitHub:https://github.com/zero-mo

Recommended Today

Practice of query operation of database table (Experiment 3)

Following the previous two experiments, this experiment is to master the use of select statements for various query operations: single table query, multi table connection and query, nested query, set query, to consolidate the database query operation.Now follow Xiaobian to practice together!Based on the data table (student, course, SC, teacher, TC) created and inserted in […]