• NDK — using camera and audiorecord to realize live streaming


    Last time, we created a new project in Android studio, integrating the tools needed to realize live streaming, which are: Rtmpdump: push stream X264: Video Coding FAAC: audio coding Article address:NDK — Construction of live streaming framework in Android studio Live streaming also needs the support of streaming media server. I use virtual machines here. […]

  • Audio video-h264 coding


    H264 coding principle andAudio video AAC codingThe principle is almost the same. The difference is that the processing of buffer buffer is realized through H264 encoder Brief logic of AAC coding: Source file = = “avframe = =” encoder = = “avpacket = = > output file Abbreviated logic of H264 encoding Source file = […]

  • Detailed explanation of masked self encoder Mae and implementation of pytoch code


    Supervised learning is a traditional method of training machine learning model. During training, every observed data needs to be labeled. What if we had a way to train machine learning models without collecting labels? What if we extract tags from the same data collected? This type of learning algorithm is called self supervised learning. This […]

  • Transformer architecture layer by layer function introduction and detailed explanation


    Over the years, deep learning has been developing. The deep learning practice highly emphasizes the use of a large number of parameters to extract useful information about the data set we are processing. By having a large number of parameters, we can more easily classify / detect something because we have more data that can […]

  • Ffmpeg learning 4 (audio and video codec)


    libavcodec: it mainly realizes the realization of a series of encoders A common structure of encoding / decoding (1) AVCodec: encoder information (2) AVCodecContext: encoder context (3)AVFrame: original frame data of audio and video, corresponding to:AVPacketThe frame inside is the compressed frame 2 memory allocation and release av_frame_alloc()\av_frame_free()Build and release:AVFrame //Assign codec context avcodec_alloc_context3()/avcodec_free_context Second […]

  • Understanding deep learning: twin brothers of neural networks – self encoder (Part 1)


    preface This article can be used as the foundation of knowledge. What is self encoder? Self encoder is a magical thing that can extract deep-seated features from data. For example, when we input an image, the self encoder can extract the features of the “face” on the image (coding process), and this feature is saved […]

  • Application of AI algorithm in video scalable coding


    [focus on rongyun global Internet communication cloud] mainly includes five parts: the characteristics of three commonly used scalable video coding; Encoder adopted by webrtc and its application mode; The application status of scalable coding in webrtc; Target detection and rate allocation based on scalable coding; The application prospect and research direction of the combination of […]

  • Unsupervised learning – self encoder | deep learning (Li Hongyi) (19)


    1、 Depth self encoder Autoencoder Auto encoder is an unsupervised learning method, which can be used to reduce the dimension of data. For our input data, we can obtain a low dimensional code through an encoder, and then reconstruct the original data through a decoder, which is trained together. The following figure shows this process […]

  • 2021 technology prospect – current situation and Prospect of AV1 in RTC application practice


    The rise of online conferences, online education, e-commerce live broadcasting and other scenes also makes the real-time interactive technology go from behind the scenes to in front of the stage and get more people’s attention. A series of technologies related to RTE such as encoding and decoding, network transmission and computer vision are also glowing […]

  • Net / RPC package


    The principle of RPC is very simple. The client encodes the request method, parameters and other information and transmits it to the server. After the server decodes it, it finds the corresponding method to execute, and encodes the result and returns it to the client. Server Important structure type Server struct { serviceMap sync. Map […]

  • In simple terms, the fourth chapter of neural network shows learning reading notes


    Chapter 4 represents learning In Chapter 2, it is mentioned that the first step of machine learning is to extract features. Representation learning is to automatically learn features from data and directly use them for subsequent tasks. 4.1 express learning 4.1.1 meaning of learning It means that learning needs to answer 3 questions: How to […]

  • Audio and video learning notes


    This is my learning audio and video notes on XX. Com, mainly including the introduction of audio and video and the actual combat of ffmpeg. The notes are arranged according to the class process. Everyone’s foundation is different. I only write what I think I need to write down in my notes; I have five […]