Technology sharing | how does video correction work in art teaching?


In recent years, the online education industry has entered the fast lane of development. People’s demand for education has become more and more clear and demanding. After a storm, we found that onlyHigh quality products and emphasis on user experienceOnly in the post epidemic era can the institutions be more and more prosperous. In the quality education track, online art teaching is the coreArt treasure, painting LalaWith online calligraphy teaching as the core, he Xiaoxiang has led the subdivision track with its own strength, and promoted the progress of quality education with science and technology. It is worth mentioning that art treasure education took the lead in adopting the “art loop” real-time video perspective correction system in the industry to solve a series of problems such as perspective, deformation and color deviation caused by mobile device terminals in the teaching live broadcast scene, restore the offline teaching experience and interaction, and improve the concentration of students.In these online art and calligraphy teaching scenes, video correction technology has become a bonus on the audio and video technology. This article will introduce the practical experience of paileyun in video correction technology from the perspective of technology.

1、 What is video correction?

Video rectification technology based on free geometric transformation algorithm of parameter estimation, which is based on the technical optimization of real-time audio and video, and corrects the video picture through the unique image processing algorithm. For example, in online art classes, students paint on paper remotely, and the camera is facing the paper. It is usually difficult to ensure that the camera can shoot directly at the center of the paper. Once the camera deviates from the center of the paper, the paper will deform in the captured video. This technology can enable teachers to correct the distortion of students’ videos and adjust the captured trapezoidal paper to a rectangle.

2、 What are the key technical points of video correction?

When taking images with mobile phones and other devices, if the camera cannot directly face the object to be taken (such as the drawing board), a series of geometric distortions will be generated. This is because the real world is a three-dimensional space when the camera is shooting, and has become a two-dimensional plane when displayed on the mobile phone. For the real spatial relationship between the object and the camera, the geometric distortion parameters of the subject can be estimated from the geometric position relationship of the subject in space, and then the mapping relationship between the area to be corrected and the true or false subject can be recalculated. The original video is processed by using the low complexity and efficient processing algorithm to get the corrected video. Of which,Key technical pointsinclude:1. Original geometric distortion parameter estimationa. The photographed object, such as a drawing board, is mathematically modeled in the shooting space to establish a spatial coordinate system; b. According to the position relationship of the object in space, the acquisition parameters of the camera are estimated, especially the translation, rotation, scaling and projection parameters.

2. Original mapping calculation, map the camera acquisition parameters with the assumed aerial view camera acquisition parameters to obtain the real video rendering parameter matrix

3. Efficient and fast video processing algorithmIn real-time video communication, there is a huge amount of video data. This technology needs to map and transform each pixel in the video, and the amount of computation increases sharply with the increase of pixel points. Therefore, we have designed a highly parallel processing algorithm, so that each pixel can perform computation independently, and GPU can be used to accelerate processing from the surface, even ultra-high definition video can be processed at the millisecond level.

Relevant development guide pages:

Android: VideoQuadTransform

com. pano. rtc. api. Constants. Panooptiontype enumeration type reference developer pano. video

iOS: kPanoOptionQuadTransform

PanoEnumerates. H file reference developer pano. video

The above is some sharing about video correction. Welcome to experience. Follow the official account of paileyun pano. We will share more technical knowledge about audio and video development in subsequent articles.