Great Chrome browser (6): Chrome 94 has started webgpu trial, and the image rendering and machine learning capabilities of the web are stronger

Time:2021-11-25

What interesting new features did chrome 94 officially released on September 21 bring?

background

Over the past decade, web technology has advanced by leaps and bounds, among which Chrome has contributed a lot. Understanding chrome can help us understand the development trend of the front-end industry.

Therefore, I willGreat Chrome browserTo explain in detail the important features of each version of chrome, and share some thoughts:

By focusing on chrome writing, I can not only improve my professional ability, but also improve my personal influence.My long-term goal is to publish a book on chrome in 2025After all, publishing their own books is the highest pursuit of every writer.

I am a cold geese, a programmer who loves writing code and writing articles. Welcome to my WeChat official account.Cold goose talk

TL;TR

Detailed interpretation

WebGPU

Chrome 94 has added trial featuresWebGPU, it provides a web API using GPU, which can be used for image rendering (such as 3D rendering) and data parallel computing (such as machine learning).

Webgpu is undoubtedly a revolutionary feature for the web. In essence, the computer industry is promoted by Moore’s Law (Moore is one of the founders of CPU manufacturer intlel) and Huang’s Law (Huang is Huang Renxun, the founder of GPU manufacturer NVIDIA and the godfather of nuclear bomb). The improvement of chip computing power has brought us previous computer industry revolutions: personal computers Internet, mobile Internet, artificial intelligence. When the computing power of GPU becomes stronger and more important, the web can not be well used, which is obviously unreasonable.

The webgpu feature corresponds toWebGPUandWebGPU Shading LanguageEngineers from Google, Mozilla and apple are responsible for these two proposals. Both webgpu and webgpu Shading Language proposals were developed by W3CGPU for the web WorkgroupDrafted. The work composition was established in 2017. After four years of efforts, webgpu finally began to try. It’s not easy! The webgpu proposal defines the API for using GPU in the web, and the webgpu Shading Language (wgsl) proposal defines the programming language of GPU code. Webgpu is supported by four browser giants (Safari, Firefox and edge), so it is only a matter of time before webgpu becomes the official W3C standard.

Webgpu started trial in chrome 94 and is expected to be officially released in chrome 98 in February next year. Such an important feature may be officially released after only three months of trial before you can learn how to use it. It seems a little too hasty.

Webgpu is the successor of webgl. Their goals are similar, but webgpu provides more underlying GPU capabilities. Therefore, webgpu has stronger image rendering ability, better performance, easier to use, and is more suitable for data parallel computing and machine learning.

According to the test results of safari, the performance of webgpu is much higher than that of webgl on various devices:

Great Chrome browser (6): Chrome 94 has started webgpu trial, and the image rendering and machine learning capabilities of the web are stronger
Image source:WebGPU and WSL in Safari

Tensorflow.js, the front-end machine learning library, also tested webgpu, and found that the performance of webgpu is better, but the gap between webgpu and webgl is not particularly obvious, which needs further research:

Great Chrome browser (6): Chrome 94 has started webgpu trial, and the image rendering and machine learning capabilities of the web are stronger
Image source:Fast client-side ML with TensorFlow.js

As shown in the figure below, webgpu is implemented based on various GPU APIs (such as DirectX 12, metal, Vulkan).

Great Chrome browser (6): Chrome 94 has started webgpu trial, and the image rendering and machine learning capabilities of the web are stronger
Image source:Access modern GPU features with WebGPU

Webgpu provides the underlying API, which is very powerful and complex. Implementation using webgpuVector multiplicationThe code is as long as 200 lines. Visually, a third-party library will appear in the community to encapsulate webgpu, providing a simpler way to use it for different application scenarios.

According to the test, when using webgpu for vector multiplication calculation, with the increase of vector length, its performance is much better than using CPU:

Great Chrome browser (6): Chrome 94 has started webgpu trial, and the image rendering and machine learning capabilities of the web are stronger
Image source:Get started with GPU Compute on the web

WebCodecs

Chrome 94 officially releasedWebCodecs, so that we can directly use the image, audio and video encoding / decoding capabilities provided by chrome.

WebCodecsThe proposal for W3C was put forward by engineers from Google, Mozilla and Microsoft. It was tried out in chrome 86. Engineers from Firefox and edge jointly drafted the webcodecs proposal, so it is expected that the proposal will eventually become the W3C standard.

Codec iscoDer anddecOder composite word, namely codec, can be hardware or software, which is used to encode and decode specific data formats, such as MP3 / AAC / VP9 / h264, etc.

The browser supports pictures, audio and video in various formats, but we can’t call the corresponding codec directly before. Webcodecs is to solve this problem, so that web developers can directly use various codecs.

In order to optimize performance and maintain consistency, Google Docs announced in May this year that it would migrate to a canvas based rendering scheme. However, previously, Google Docs still used the < img > tag of HTML when processing gifs. Later, Google Docs used the image codec imagedecoder of webcodecs to decode the GIF, and then used canvas to render, which optimized the performance and simplified the code architecture.

Zoom uses webcodecs in its web meeting SDK and web video SDK. Since the source code is not open source, it is unknown how to use it. It should use video related codecs. It is worth mentioning that the breaking changes in the update of webcodes in chrome 93 have led to bugs in zoom’s web meeting SDK and web video SDK. It seems that there is a great risk to directly use the features not officially released in the third-party SDK.

Scheduling APIs: Prioritized scheduler.postTask

Chrome 94 officially releasedScheduling APIs: Prioritized scheduler.postTaskcharacteristic:

  • The scheduler.posttask method is added to support scheduling tasks according to priority and scheduling tasks with specified delay;
  • Three task priorities are defined, from high to low: user blocking, user visible and background;
  • Taskcontroller is added to support dynamic modification of task priority and task cancellation;

Prioritized Task SchedulingThe proposal for WICG was put forward by Google engineers and began trial on chrome 86. Since the proposal has not been participated and supported by other browser manufacturers, the biggest risk of this feature is whether it can become a general standard.

When we use lighthouse to detect the page, one of the detection items is “minimize main thread work”, which divides the work of the main thread into seven categories as shown in the figure below. It can be seen that the task undertaken by the main thread is extremely heavy:

Great Chrome browser (6): Chrome 94 has started webgpu trial, and the image rendering and machine learning capabilities of the web are stronger

Because the main thread needs to perform too many tasks, if the main thread is a low priority task or a time-consuming task(Long Task, i.e. tasks that take more than 50ms to execute) are excessively occupied, which will cause the browser to respond too slowly to user operations. For example, clicking a button for a long time does not respond, which will greatly damage the user experience.

Take the most extreme example. If the JavaScript code falls into an endless loop, the page will be basically completely stuck and unable to respond to user operations. If you are interested, you can execute the following code on the console:

while (true) {
  console.log("hello, Chrome!");
}

If non critical JavaScript tasks (such as logs) occupy too many main threads or key JavaScript tasks (such as responding to user operations) wait too long, it will hurt the user experience. In order to optimize the scheduling of the main thread,requestIdleCallbackandisInputPendingRequestidlecallback can use the idle time of the main thread rendering page to execute low priority tasks, and isinpending can be used to avoid time-consuming tasks blocking response user operations. Compared with requestidlecallback and isinpending, the scheduling function of prioritized task scheduling proposal is more complete and powerful. It supports scheduling tasks according to priority, specifying delay scheduling tasks, dynamically modifying task priority and canceling tasks.

By using scheduler.posttask, aribnb’s engineers divide time-consuming tasks into small tasks, postpone non critical tasks, and pre download pictures. From the optimization effect, the search results pageTotal Blocking Time(TBT)Reduced from 16S to 6S, greatly optimizing the user experience. However, the optimal value of total blocking time is within 300ms, so the optimization made by airbnb does not look ideal. Of course, this value should be relevant to the specific application.

Aribnb engineers useScheduling APIsPicture pre download is realized:

  • When the user slides to the second search result, the second picture is downloaded in advance;
  • When the user slides to the second picture of the second search result, the third, fourth and fifth photos are downloaded in advance;
  • We can see four image download requests in the network console. According to the waterfall, the four requests are requested in chronological order;

Great Chrome browser (6): Chrome 94 has started webgpu trial, and the image rendering and machine learning capabilities of the web are stronger
Source: [building a fast web experience with the posttask
Scheduler](https://medium.com/airbnb-eng…)

With the increasing complexity of web applications, the main thread undertakes more and more tasks. How to reduce the burden of the main thread will become a major issue faced by browsers and web developers. Scheduling tasks according to priority may become one of the best practices of web applications and web frameworks. In addition, similar proposals to reduce the burden of main threads will continue to emerge, which will further strengthen the ability of web applications.

Idle Detection API

Chrome 94 officially releasedIdle Detection APIIt is used to check whether the user is active, and its judgment basis is whether the user uses the keyboard, mouse, touch screen, etc.

Idle Detection APIThe proposal for WICG was put forward by Google engineers. It was put on trial in chrome 84 and officially released in chrome 94. At present, it seems that this feature is only supported by chrome. Firefox and safari have clearly expressed their opposition to this feature in order to protect users’ privacy.

Apple’s insistence on users’ privacy and security has become a part of its corporate culture, affecting its decisions on many products and technologies, which is determined by its business model that does not rely on advertising to make money. According to the latest data from StatCounter, chrome and safari have 64% and 18% market shares respectively, so Safari is chrome’s biggest competitor. With a strong opponent like Safari restricting chrome, it is conducive to ensuring the healthy development of the web.

Personally, I also object to this feature, the idle detection API (although my objection is useless, ha ha). It is suspected of violating the user’s privacy, can be used to track the user’s daily behavior, and may be used in abnormal scenes.

In order to protect the user’s privacy and security, calling idle detection API requires the user’s authorization:

//Obtain idle detection API authorization
const state = await IdleDetector.requestPermission();
if (state !== "granted") {
  // Need to request permission first.
  return console.log("Idle detection permission not granted.");
}

The idle detection API can be used in the following scenarios: for instant chat, social media and online games, checking whether users are active can help users judge whether their contacts are online. In fact, chat applications slack and Google chat have expressed interest in this feature.

In those years, the PC version of QQ can automatically switch the state to “leave” according to the user’s mouse and keyboard operation. However, the chat application in the mobile Internet era defaults to the user being online 24 hours a day. Mobile Internet not only brings convenience to users, but also kidnaps everyone’s time and energy. Can you do without mobile phones for 24 hours?

Google engineers provide a very intuitive demo applicationEphemeral Canvas, we can use it to draw pictures. When we don’t operate the computer within 60 seconds, the pictures will be erased automatically.

JS Self-Profiling API

Chrome 94 officially releasedJS Self-Profiling API, which is used to obtain performance data during JavaScript execution.

JS Self-Profiling APIThe proposal for WICG was put forward by Facebook engineers. It was tried out on chrome 78 and officially released on chrome 94.

Not surprisingly, Safari opposes this feature because of performance and security issues. The performance problem is easy to understand. Collecting performance data during JavaScript execution will lose performance. As for security, Safari engineers are worried that hackers may suffer timing attack by obtaining the compilation time of JavaScript.

It seems that chrome and safari have very different views on how to develop web technology. The former is much more radical, while the latter is relatively conservative. It can also be seen from the great Chrome browser series blog I wrote that Google engineers have developed many new browser features. As a writer tracking chrome features, I can’t learn them. For most developers who are addicted to writing code, I may not have heard of many features If you are interested in Chrome, please pay attention to my WeChat official account.Cold goose talk)。 This is determined by the different business models of Google and apple. Apple has built a closed and complete IOS / MacOS / ipados / watchos ecosystem, and its enthusiasm for web technology is not as high as its own son; Web technology is the foundation for Google to settle down. There are no web pages, and the search engine still searches for a ball? Therefore, chrome is far more than just a traffic portal for Google. Chrome’s commitment to promoting the development of web technology is not empty talk. It is no coincidence that Chrome’s product leader became Google’s CEO. For more information, see my blog 2 years agoHow did chrome succeed

Although Safari is not interested in JS self profiling API, engineers from Facebook and Microsoft said that they have identified some very serious performance problems through JS self profiling API, indicating that this API should still have two brushes. Because the JS self profiling API does not affect product functions, there is no big problem that Safari does not support it. Anyway, Apple’s self-developed chip is fast enough and probably won’t have performance problems?

Canvas color management

Chrome 94 supports the use of display P3 gamut when creating 2D canvas, which will enhance the color restoration ability of 2D canvas.

canvas.getContext('2d', { colorSpace: "display-p3"} );

Canvas color managementThe trial started on chrome 90 and chrome 94 was officially released. This feature I’m inGreat Chrome browser (4): Chrome 92 adds at and randomuuid methods, and canvas supports display P3 gamutIt’s introduced in the blog, but it’s embarrassing that I made a mistake. It’s not a feature officially released by chrome 92. This feature is supported by Firefox and safari, so it will become a common standard.

Previously, 2D canvas only supported the old sRGB color gamut, but today’s screens and cameras have long supported a larger color gamut.

What is the color gamut? Its English name is color gamut or color space, which is the color range that devices (displays, projectors, printers) can express. The color range visible to the human eye is limited, and the color range that the device can express is a subset of the color range visible to the human eye, and the color range that different color gamut standards such as sRGB and display P3 can express is also different.

The color gamut of display P3 is 25% larger than that of sRGB. When we compare the two, we will find that display P3 is much brighter than sRGB, and the difference is very obvious:

Great Chrome browser (6): Chrome 94 has started webgpu trial, and the image rendering and machine learning capabilities of the web are stronger
Image source:Get Started with Display P3

For image, video, design, games, maps, food and other applications, the importance of color accuracy is self-evident.

For some seemingly color independent applications, if our application cannot accurately restore the color of the object, it will also affect the business. In most cases, the obvious difference between the Buyer show and the seller show is caused by the seller’s excessive PS, but it may also be caused by the color not being accurately restored.

103 Early Hints for Navigation

Chrome 94 has added trial features103 Early Hints for Navigation

103 the IETF proposal corresponding to early hints for navigation isRFC 8297: 103 Early Hints, it is essentially an update to the HTTP protocol. The proposal was put forward by kazuho OKU, an engineer of fastly, a cloud service provider. 103 early hits for navigation started trial on chrome 94, and the officially released chrome version is uncertain.

It is worth mentioning that fastly’s CDN service failed in June this year, resulting in the downtime of well-known services such as Amazon, Hulu, reddit and Shopify, which shows its great influence. Fastly proposes that 103 early hints does not generate electricity completely with love, and this feature also helps its CDN service. After receiving the 103 status code of the source site, the CDN node can decide whether to reuse the resources cached by the CDN node in advance according to whether the header contains cache control: private to improve the response speed. It can also be seen from this simple example that sometimes some technical problems cannot be well solved in the existing technical standard system, and the standard itself needs to be changed. It is inferred that Chinese programmers should participate more in the formulation of international technical standards, which is of practical significance. Of course, in the short run, the input-output ratio may not be so high, but in the long run, it is also the only way.

The following figure intuitively shows the role of 103 early hits for navigation. It can preload subsequent resources when the previous HTTP request header and response have not returned, so as to greatly advance the subsequent HTTP requests, so as to reduce the overall request time and improve the network utilization.

  • Case 1: after the body returns, parse the HTML, and then initiate a request to obtain CSS and JS resources:


Great Chrome browser (6): Chrome 94 has started webgpu trial, and the image rendering and machine learning capabilities of the web are stronger
Source: [towards ever faster websites with early hints and
priority hints](https://www.fastly.com/blog/f…)

  • Case 2: after the header returns and before the body returns, initiate a request to obtain CSS and JS resources according to the preload information in the header:

Great Chrome browser (6): Chrome 94 has started webgpu trial, and the image rendering and machine learning capabilities of the web are stronger
Source: [towards ever faster websites with early hints and
priority hints](https://www.fastly.com/blog/f…)

  • Case 3: after the 103 status code is returned, initiate a request to obtain CSS and JS resources according to the preload information in the header:

Great Chrome browser (6): Chrome 94 has started webgpu trial, and the image rendering and machine learning capabilities of the web are stronger
Source: [towards ever faster websites with early hints and
priority hints](https://www.fastly.com/blog/f…)

Obviously, in the third case, the overall response time is much faster and the utilization of the network is also improved. We might as well take a look at the specific request process in case 3.

The browser initiates a request to visit example.com:

GET / HTTP/1.1
Host: example.com

The server returns 103, and the header contains the preload information. At this time, the browser can initiate style.css and script.js requests:

HTTP/1.1 103 Early Hints
Link: </style.css>; rel=preload; as=style
Link: </script.js>; rel=preload; as=script

The server returns 200, the header contains the preload information, and the HTML text also contains the required CSS and JS files (isn’t this nonsense?).

HTTP/1.1 200 OK
Date: Fri, 26 May 2017 10:02:11 GMT
Content-Length: 1234
Content-Type: text/html; charset=utf-8
Link: </style.css>; rel=preload; as=style
Link: </script.js>; rel=preload; as=script

<!doctype html>
[... rest of the response body is omitted from the example ...]

summary

Starting from chrome 94, chrome will speed up its update frequency from updating a version every 6 weeks to updating a version every 4 weeks. For a product with 2.6 billion users worldwide, accelerating the release frequency can not only provide users with a better experience, but also bring great challenges to the R & D team. Chrome can always maintain a stable iteration frequency, while maintaining transparency and providing rich documents, which provides convenience for developers in the chrome ecosystem, which is worth learning.

This is the sixth article of the great Chrome browser, which has created a personal record of writing a special series blog, compared with only five articles in the previous series of JavaScript in simple terms. After chrome accelerates the update frequency, I also have to adjust my writing plan to keep pace with the iteration of chrome.

Careful students may find that there are some subtle differences in my blog, and I will insist on doing so in the future:

  • Start to introduce the standard proposals corresponding to each chrome feature. It is also necessary for the top technology companies to master the technical standards and understand these proposals. These proposals come from different standardization organizations, such as W3C, WICG and IETF. At present, chrome has the dominance of various web technology standards, which has both advantages and disadvantages;
  • Start to introduce Safari, Firefox, edge and other large companies’ attitudes towards various proposals and the business reasons behind them. Technology can promote business progress, and business can affect technology development. The two cannot be separated. It is not enough to only understand technology without understanding the business logic behind it;
  • I began to express negative views on some practices of chrome. As the title of my blog series, I have a positive attitude towards chrome on the whole, because it has completely changed the front-end ecosystem, changed my industry and affected my personal career development. However, this does not mean that I support everything chrome does, I will ask myself to be more objective;
  • Start introducing the origin trial feature. Each version of chrome will release some trial features, which are often more important than formal features and can not be missed. This can also let you know the latest development trend of web technology at the first time;

This article introduces seven features, six of which involve new web technology standards. It can be seen that Chrome is really committed to make web great again (mwga), and Google programmers work hard for OKR. In contrast, Firefox and safari look much more Buddhist, which is related to the business model and investment level of each company. However, after mastering the dominance of web technology, chrome does whatever it wants. Some features are a little too fast despite the opposition of peers, which is not very good. Web is different from other platforms, because it is open, belongs to everyone, and has backward compatibility. Therefore, Web technology standard can not be has the final say, nor can it be too casual, otherwise it will be unfavorable to the long-term development of Web technology. Unfortunately, the current web technology standards have nothing to do with domestic programmers. We have no choice but to be a gourd eater… Wait until the domestic browser has a greater voice.

Another point is that for each feature, I spent a lot of time reading various materials to understand its principle, and then wrote it according to my personal understanding. I don’t have time to write code tests for many features, so my statement is bound to be wrong. You are welcome to comment and correct. Interested students can add my personal wechat communication:KiwenLau

Welcome to pay attentionCold goose talkOfficial account, concernGreat Chrome browserSeries blog, witness the big front-end star sea with me!

reference material

recruit

Alibaba’s business platform Division has long recruited P6 and above front-end leaders to participate in the construction of the most cutting-edge Alibaba front-end ecosystem and promote the development of industry technology. The internal push address is:[email protected]

Welcome to my WeChat official account.Cold goose talk