Front end multi data rendering optimization

Time:2022-5-4

preface

When I made a request some time ago, I came across the function of a custom list. All his data display is stored through the JSON string, which is also parsed through JSON. At first, he had a data upper limit, but later, after raising the upper limit, there were problems such as Caton,
Therefore, this paper introduces some solutions to solve the rendering problem of a large amount of data in the front end

programme

innerHTML

The first is the rendering scheme a long time agoinnerHTMLPlug in. It is the official API with good performance

This is a simple example of HTML rendering (in the experiment, the data is at the level of 10W to expand the difference. In practice, it will be less than this level)

    const items = new Array(100000).fill(0).map((it, index) => {
    return `<div>item ${index}</div>`
}).join('')
content.innerHTML = items

Performance analysis from Google:
Front end multi data rendering optimization

After refreshing and scrolling the page within 10 seconds, you can see that the rendering of DOM blocks the page for 1300 Ms

In the performance test, the total blocking time is controlled within300 msA qualified state is within, and this time will be affected by the computer hardware

Summarize the advantages and disadvantages of this method:

  • Advantages: the performance is relatively acceptable, but there is also blocking when there are many data
  • Disadvantages:

    • There is a risk of injection, and the matching with the frame is poor
    • The performance problem of scrolling is not solved when there are too many DOMS

Batch insert

Insert by slicing. If there are 10W pieces of data, we will divide them into 10 times and insert 1W pieces of data in a cycle each time

    [...new Array(10)].forEach((_, i) => {
    requestAnimationFrame(() => {
        [...new Array(10000)].forEach((_, index) => {
            const item = document.createElement("div")
            item.textContent = `item ${i}${index}`
            content.append(item)
        })
    })
})

After Google analysis:
Front end multi data rendering optimization

This also includes the performance analysis of page refresh and scrolling. It can be seen that the blocking time is 1800 milliseconds, which is a little worse than innerHTML. This is in the order of 10W. The smaller the number, the smaller the time gap

About requestanimationframe

amongrequestAnimationFrameThis method will tell the browser that you want to execute the animation and request the browser to call the callback function to update the animation before the next redrawing.

Execution method: when the requestanimationframe (callback) is executed, the callback function will not be called immediately, but will be put into the callback function queue,
When the page is visible and the list of callback functions requested by the animation frame is not empty, the browser will regularly add these callback functions to the queue of the browser UI thread (the execution time of the callback function is determined by the system)

Generally speaking, it will not block the execution of other code, but the total execution time is not much different from that of innerHTML scheme

Summarize the advantages and disadvantages:

  • Advantage: it will not block the operation of code
  • Disadvantages:

    • The total time spent inserting is still not much different from innerHTML
    • Similarly, the performance problem of scrolling is not solved when there are too many DOMS

Other native ways

canvas

Canvas is a special tool for drawing, which can be used in animation, game picture, data visualization, picture editing and real-time video processing.

Recently, canvas is used to render pages in the web of the famous framework fluent

Similarly, we can also use canvas to render a large amount of data


<div style="max-height: 256px;max-width:256px;overflow: scroll;">
    <canvas id="canvas"></canvas>
</div>
    let ctx = canvas.getContext('2d');
[...new Array(100000)].map((it, index) => {
    ctx.fillText(`item ${index}`, 0, index * 30)
})

After practical attempts, the canvas is limited, and the maximum height of about 6W can no longer be enlarged. That is to say, under a large amount of data, the canvas is still limited

Further optimization

Here is an optimization idea to monitor the scrolling of the outer Dom and dynamically render the display of canvas according to the height, which can achieve the final effect, but the cost is still too high

  • Advantages: good performance in the number of renderings
  • Disadvantages:

    • It is uncontrollable to render like a virtual list (it is a better scheme in other scenes, such as animation, map, etc.)
    • The style in canvas is difficult to control

IntersectionObserver

Intersectionobserver provides a way to asynchronously observe the intersection state between the target element and the viewport. In short, it can listen to whether an element will be seen by us. When we see this element, we can execute some callback functions to handle some transactions.

be careful:
The implementation of intersectionobserver should adopt requestidlecallback(), that is, the observer will be executed only when the thread is idle. This means that the priority of this observer is very low. It will be executed only after other tasks are executed and the browser is free.

Through this API, we can try to implement a scheme similar to virtual list

Here I implement a virtual list demo sliding down. The main idea is to listen to all DOMS in the list. When they disappear, remove and remove the listener, and then add new Dom and listener

Front end multi data rendering optimization
Core code:

const intersectionObserver = new IntersectionObserver(function (entries) {
        entries.forEach(item => {
            //0 means disappear
            if (item.intersectionRatio === 0) {
                //Add at the end of last
                intersectionObserver.unobserve(item.target)
                item.target.remove()
                addDom()
            }
        })
    });

Google’s performance analysis (first page entry and continuous scrolling of 1000 items):

Front end multi data rendering optimization

It can be seen that there is basically no blocking. This scheme is feasible, and there is no problem between initial rendering and scrolling

Click to view the details. The demo only realizes the scroll down scheme:
https://codesandbox.io/s/snow…

Further optimization

Now the intersectionobserver has realized the function similar to the virtual list, but frequent addition and removal of monitoring will seem to have hidden dangers, so I plan to adopt the expansion scheme:

General idea:

The current list takes 10 as a team, and the current list renders 30 in total. When scrolling to the 20th, the event will be triggered, the 30th-40th will be loaded, 0-10 will be deleted, and then triggered in turn

Front end multi data rendering optimization

In this way, the number of triggers and monitoring times will decrease exponentially. Of course, the cost is that the number of DOM rendered by colleagues will increase. Later, we will increase the number of each team again to maintain one
The balance between DOM number and monitoring

compatible

For the compatibility of intersectionobserver, you can obtain the compatibility of most browsers through Polyfill, and support IE7 at least. For details, see: https://github.com/w3c/Inters…

Summarize the advantages and disadvantages:

  • Advantages: a virtual list scheme implemented by using the native API, without data bottleneck
  • Disadvantages:

    • The adaptability of the framework in production is not high enough, and the implementation is complex
    • There may be some problems when listening and disarming are triggered frequently in infinite scrolling

frame

All the methods mentioned above are implemented in non framework. Here, let’s take a look at the performance of the list in react

react

This is a list rendering with a length of 10000

function App() {
  const [list, setList] = useState([]);

  useEffect(() => {
    setList([...new Array(50000)]);
  }, []);

  return (
    <div className="App">
      {list.map((item, index) => {
        return <div key={index}>item {index}</div>;
      })}
    </div>
  );
}

When the demo is running, you can obviously feel that the page is stuck
Front end multi data rendering optimization

According to Google analysis, in the order of 50000, the rendering is still not completed in 10 seconds after the refresh
Of course, the performance in the framework is certainly not as strong as that in the native framework. This conclusion is expected

Online demo address: https://codesandbox.io/s/angr…

It should also be noted that the transmission of a large amount of data in the template:

//The order of magnitude of this list is thousands or even tens of thousands, which will lead to the doubling of Caton, 
<Foo list={list}/>

This conclusion is applicable both in Vue and react, so the transfer of a large amount of data must be assigned and obtained in memory, rather than through modules, render and other conventional methods

If the order of magnitude is 100, we can also consider optimization, which can add up to more

startTransition

There will also be a new API in react18startTransition:

startTransition(() => {
    setList([...new Array(10000)]);
})

The function of this API is the same as what I said aboverequestAnimationFrameSimilarly, it can not enhance the performance, but it can avoid jamming, give priority to rendering other components and avoid white screen

Virtual list

The concept of virtual list is formally introduced here

Store the positions of all list elements and render only the list elements in the viewport. When the viewport scrolls, calculate which elements should be rendered in the viewport according to the offset size of the scrolling and the positions of all list elements.

Understand the principle of a dynamic diagram:
Front end multi data rendering optimization

Minimum implementation scheme

Here we try to implement a minimum virtual list scheme by ourselves:

//This is a react demo. In the Vue project, the principle is similar, and there is basically no change except for the setting of the data source

//Data source and configuration properties
const totalData = [...new Array(10000)].map((item, index)=>({
    index
}))
const total = totalData.length
const itemSize = 30
const maxHeight = 300

function App() {
  const [list, setList] = useState(() => totalData.slice(0, 20));

  const onScroll = (ev) => {
    const scrollTop = ev.target.scrollTop
    const startIndex = Math.max(Math.floor(scrollTop / itemSize) -5, 0);
    const endIndex = Math.min(startIndex + (maxHeight/itemSize) + 5, total);
    setList(totalData.slice(startIndex, endIndex))
  }

  return (
          <div onScroll={onScroll} style={{height: maxHeight, overflow: 'auto',}}>
            <div style={{height: total * itemSize, width: '100%', position: 'relative',}}>
              {list.map((item) => {
                return <div style={{
                  position: "absolute",
                  top: 0,
                  left: 0,
                  width: '100%',
                  transform: `translateY(${item.index *itemSize}px)`,
                }} key={item.index}>item {item.index}</div>;
              })}
            </div>
          </div>
  );
}

You can view the online demo: https://codesandbox.io/s/agit…

This is the smallest virtual list instance, which is mainly divided into two parts

  1. It needs to be wrapped in a container and supported with CSS. The actual rendered items need to be displayed in the correct position by transform
  2. Monitor the scrolling of the external container. When scrolling, the original data source is dynamically sliced and the list to be displayed is replaced at the same time

To see his performance:
Front end multi data rendering optimization

There is basically no blocking, and occasionally there is a little frame loss

This demo is not a final form. There are still many places to optimize
For example, caching, logic extraction, CSS re simplification, controlled rolling trigger frequency, rolling direction control and so on, there are many points that can be optimized

Other libraries

  • The list of virtual real solutions is large
  • React window is a more lightweight alternative to the library recommended by react virtualized.
  • The hooks form of react virtual virtual virtual list is similar to the logical encapsulation in my demo

Chrome official support

virtual-scroller

On chrome dev summit 2018, Google Engineering Manager gray Norton introduced virtual scroller, a web scrolling component, which may become a web layered API in the future
API). Its goal is to solve the performance problem of long lists and eliminate off screen rendering.

However, after some development, after internal discussion, we still terminate the API and turn to the development of CSS
Link: https://github.com/WICG/virtu…
Chrome’s introduction to virtual scroller: https://chromestatus.com/feat…

content-visibility

This is the new CSS property developed later

Chromium 85 began to have the content visibility attribute, which may be the most effective CSS attribute for improving page loading performance. Content visibility allows user agents to skip element rendering (including layout and painting) unless necessary. If the page has a large amount of off screen content, the content visibility attribute can skip the rendering of off screen content, speed up the user’s first screen rendering time, and reduce the interactive waiting time of the page

Details: https://web.dev/content-visib…

The way to use it is to add CSS properties directly

#content {
  content-visibility: auto;
}

Unfortunately, its effect is to enhance the rendering performance, but it still gets stuck when a large amount of data is initialized, which is not as direct and effective as the virtual list

But we can consider using it when we reduce the first screen rendering time

summary

There are many solutions for performance optimization under multi data

  • requestAnimationFrame
  • canvas
  • IntersectionObserver
  • startTransition
  • Virtual list
  • content-visibility

In general, virtual lists are the most effective, and you can use the simplest demo level to temporarily optimize your code