Front end graphics – drawing, screenshots, composite motion pictures

Time:2020-2-5

Preface

This paper is mainly related to the front-end graphics processing, and the topic focuses on canvas. Including canvas effect, html2canvas screenshot, gif.js composite action chart. It is necessary and interesting to study these things. The research is not deep, but there are still some results. Welcome to exchange and discuss with interested partners.

Thread

Graphics related mainly includes two aspects: drawing and screenshot.

The technical implementation of the screenshot is divided into static graph and dynamic graph.

Generally speaking, there are three aspects: dynamic effect, screenshot and dynamic graph.

Key points

  • Dynamic effect: mainly creativity, practice, optimization (encapsulation, etc.)
  • Screenshot: mainly canvas to image, DOM element analysis. In addition, there are some difficulties such as derivative blur, cross domain image, style compatibility, etc
  • Moving pictures: mainly screenshots and delayed playback. In terms of technical implementation, it is mainly the concept of worker multithreading and its use under the Vue framework.

Canvas dynamic effect

Canvas_API
Common canvas Optimization: fuzzy problem and rotation effect

Dynamic production ideas

As for the front-end dynamic effect, it mainly includes three parts: CSS3, SVG and canvas.

Among them, CSS3 has the lightest weight and the most powerful function of canvas. SVG has a lower demand for client performance than canvas. In mobile phone, SVG is often used to replace canvas or to implement the pre steps for canvas (such as html2canvas1.0.0).

Since I’m familiar with canvas, I’ll focus on canvas next.

First of all, we need to understand the corresponding API and know what we can do.

For example, canvas needs to go through its API before it knows that it can draw graphics, make dynamic effects, and turn into pictures!

Second, be creative and know what to do.

For example, the effect of online tree mirror is very cool, and it’s not difficult to realize. But without this idea, I can’t do it. Unfortunately, I don’t have a good idea about the idea. Of course, creativity is based on existing things. Maybe you know more about it, so naturally

Then we should have good logical thinking ability and turn creativity into a realizable plan.

It is better to have a good code style and realize reusability, so the cost of developing 10 dynamic effects is not 10 times of that of developing one. At present, I think of a way of thinking: the idea is realized by logic, the specific drawing is encapsulated into a common way and quoted on demand.

Dynamic effect realization

Here is a strong suggestion to find a good online dynamic effect, follow the implementation, and then carefully taste the improvement. I am studying the tree mirror effect, and I have learned a lot from it. Thank you very much for sharing such a good effect.

After finishing under the Vue framework, the dynamic effect of the tree mirror will look like this:
Front end graphics - drawing, screenshots, composite motion pictures
Front end graphics - drawing, screenshots, composite motion pictures
Front end graphics - drawing, screenshots, composite motion pictures

As for the knowledge involved in this dynamic effect, I have divided it into several modules:

  • Tree mirror branch constructor

This corresponds to the “idea becomes a realizable solution”. After the complex dynamic effect is split, it is essentially a tree (branch is set as 2, is binary tree, familiar with it or not). To achieve such a cool dynamic effect by changing the tree branch length, branch angle, etc., it really needs to draw only points and lines.

After the principle is clear, you can use js to calculate the coordinates, color values and other parameters of points and lines, that is, the role of treebranch here, and then directly draw parameters from variables.

  • Drawing implementation module drawtree

This method takes the parameters branch of the tree, then calls the base API to draw the line, and finally draws the pattern of the tree in a certain state.

  • Loop drawing for dynamic effect

This method is used to call drawtree circularly to draw a graph, modify the state of the tree and realize the dynamic effect of the tree mirror.

  • Interaction event

This method is used to bind user events, so that users can actively modify the relevant parameters and control the state of the tree by mouse or touch screen.

Screenshot of html2canvas

The first contact with html2canvas was at the end of November 2017. At that time, the task was to share pictures in wechat. For various reasons, the final task was to “realize front-end screenshots and add custom tags”. Coincidentally, at the beginning of December, the author made a major revision, which was compatible with many CSS3 styles, and also optimized some shortcomings of the previous version 0.5.0. The research on html2canvas also ended on December 11 after 1.0.0-alpha.3.

In view of the compatibility with the lower version of mobile phones, we did not use the 1.0.0 version of diaoyuantian, but the 0.5.0 version. We need to study the shortcomings of 0.5.0 compared with 1.0.0, and then repair them by ourselves

Screenshot principle

In view of the need to repair the shortcomings of 0.5.0, we need to dig the source code of html2canvas happily and quickly, and naturally understand the principle of realizing the screenshot function of html2canvas.

Compared with the two versions and the online materials, two highlights are found:
1. The front-end screenshot depends on the function of converting canvas to image resource.
At the end of the screenshot, we will return a canvas object, which is our screenshot area. Then turn canvas into a picture.
2. How can the screenshot area be drawn on canvas? By parsing DOM objects, it becomes a language that can be understood by canvas, and can be drawn once on canvas.

To understand these two points, we can grasp the essence of front-end screenshots. Then study the optimization points as you need.

Optimization point

Screenshot ambiguity

This problem is the most discussed one before version 1.0.0. It can be said that when it comes to canvas, it is almost necessary to talk about ambiguity.

Let’s not talk about the evolution process here, but the final solution: get the scale of DOM, and use the scale of DOM to set the scale of canvas.

The reason can be found in the devicepixelratio parameter of the lower mobile terminal.

Style compatibility

Many parts of the project use flex layout, later found that html2canvas-0.5.0 is not compatible!!!

If you understand the second point of the screenshot principle, I believe you will know what is going on. In fact, the front-end screenshot is not the real screenshot, but after understanding the dom of the area to be screenshot, it is implemented on canvas in imitation. There is a key point here is to be able to parse DOM structure, and the source code will find that there is no support for flex style parsing.
Front end graphics - drawing, screenshots, composite motion pictures
Front end graphics - drawing, screenshots, composite motion pictures

Although the reasons for the incompatibility of styles are found, they can’t be solved pertinently – after all, the task of developing parsing styles on the old version is too big and impractical, so we can only expect when we can use the new version.

Finally, we have replaced the flex layout in the DOM that needs screenshots (all evil compatibility).

Cross domain picture

At that time, the screenshot elements included the wechat head image and the two-dimensional code with parameters generated by wechat. As a result, the picture couldn’t come out. How angry!

Because we have realized the first principle of screenshot – the final picture is transferred through canvas, and it can be drawn on canvas. After the picture is transferred, wechat head image and two-dimensional code with parameters can not be loaded. Therefore, we study the link of image transfer in canvas. Here are two things:
1. Cross domain images can contaminate canvas
2. Contaminated canvas can’t be converted into pictures (troublesome homology strategy)

Html2canvas provides the configuration of these two parameters: usecors and allowtaint. By default, cross domain pictures are not loaded and pollution is not allowed.
If you want to convert to a picture allowtaint, you must be false. If you want to load a cross domain picture usecors, you must be true.

So either we can make the image non cross domain, or we have to solve the problem of cross domain image pollution canvas.

The problem of cross domain images is only one of blurring with screenshots. Naturally, there are many mature schemes. By modifying the source code of html2canvas, the problem of cross domain image pollution can be solved.

Front end graphics - drawing, screenshots, composite motion pictures

Through this scheme, we have solved the problem of wechat avatar, but the two-dimensional code with parameters is not enough. Here’s another thing to learn: loading cross domain resources requires back-end support. Obviously, the configuration with parameter QR code is not allowed. Then we can only make the picture non cross domain
Fortunately, the front-end still has the ability to generate QR code, so the front-end no longer requests the URL of QR code, but directly generates QR code in the front-end.

After a series of efforts, the problem of cross domain pictures has been solved successfully.

Gif.js generate dynamic graph

The motivation to study this comes from an example on the Internet, canvas + gif.js, to create its own digital rain head. My primary goal is to realize the use of gif.js under the Vue framework, so as to facilitate the research and debugging later (without refreshing the browser all the time). The results can be seen in my GitHub, and the route is’ ා / test / testctxgif ‘.

Trampled pit

You need a URL to use, not a local file path

At first, the research naturally put the online code into local access, but it can’t generate dynamic graph! Since then, I have studied gif.js.
Gif.js is a JS GIF encoder that can run directly on the browser. Using typed arrays and web workers to render every frame in the background is very fast.

Compare the instances that can be used normally on the Internet with the local copies that can’t be used normally, and try to hang the instance to a server to run with the error message – IIS server, it’s time to witness the miracle, the scheme is feasible!!!

Migrating code to Vue framework

The verification scheme is feasible. Only after it is hung on the server, it starts to move to the Vue framework. But the process is not easy.

  • Web worker – front end multithreading

First, the file does not load properly. It’s easy to say that gif.js can be used by importing directly, and the GIF object will be attached to the window; however, gif.worker.js can’t be loaded normally, so it can’t be executed normally.
Then I started to explore what is the sanctity of gif.worker.js and whether it can be modified.
Then I got to know webworkers, the Buddha. It’s just a sharp weapon at the front!

The worker interface of the web workers API represents a background task that can be easily created and can send messages back to its creator. Creating a worker program simply calls the worker () constructor and specifies a script to run in the worker thread.

Can the front end be multithreaded? This is a miracle!

  • Gif.worker.js file loading

Gif.worker.js obviously uses worker. The creation of worker is to pass in the URL of the script to be executed, and then the worker loads and calls it. It is obvious that the problem is that the worker’s loading mechanism can’t correctly load the gif.worker.js file under the Vue framework – relative path, hash route, there are many problems.

After trying to modify the path, change the route configuration and other methods fail, it’s a flash of inspiration – isn’t this the configuration given by webpack? Failed to load. I should go to webpack!

Then I found the worker loader, configured the configuration of webpack and modified the source code of gif.js properly. Finally, the file was loaded successfully. Run successfully under the Vue framework!


However, a serious problem has also been found: loading in the worker loader mode will seriously affect the performance of the worker. Compared with not using the Vue framework, the speed is slower. This is a big deal. Moving to the Vue framework is to improve development efficiency. It can’t be done more slowly~

The specific reason is not deep, but through the implementation effect, it should be that this reference mode of webpack destroys the performance of worker’s multithreading, and even blocks user operation. So I started to load gif.worker.js instead of webback.

As it turns out, it’s always good to have a dream. After the gif.js and gif.worker.js are moved to the static folder, they are finally successful, with fast speed and good performance.

Using gif.js to realize composite dynamic graph

After the feasibility verification and the configuration of the development environment are completed, we can finally practice it. The finished product is shown in the figure below.
Front end graphics - drawing, screenshots, composite motion pictures

Make canvas animation – use as cutout

Following the example on the Internet, I also implemented a simple canvas animation as the material. That is, the effect in the dynamic diagram. The specific implementation can be seen in the code on GitHub.

Gif.js for dynamic graph synthesis

Gif.js supports us to load canvas objects, image objects or directly copy pixel points from CTX, set frame delay, sampling interval, cycle mode, etc.

The principle is related to GIF: capture multiple pictures and display them in turn according to the set time interval, so as to achieve the effect of moving pictures.

Specific parameter settings, such as drawing size (mainly zoom effect), canvas object screenshot interval and so on, need to be further explored.

epilogue

It’s almost a year since I came into contact with canvas. Maybe it’s the reason why I started to get into the front end of canvas. I’m always interested in drawing related comparisons. Now it’s also a summary of this series of research. I hope to have a chance to contact more cool effects or game related in the future. Of course, the front end and algorithm also hope to have breakthroughs. Well, there must be some dreams. Maybe it will come true~
Front end graphics - drawing, screenshots, composite motion pictures

Recommended Today

Redis design and implementation 4: Dictionary Dict

In redis, the dictionary is the infrastructure. Redis database data, expiration time and hash type all take the dictionary as the underlying structure. Structure of dictionary Hashtable The implementation code of hash table is as follows:dict.h/dictht The dictionary of redis is implemented in the form of hash table. typedef struct dictht { //Hash table array, […]