The birth of postgraduate entrance examination assistant app!

Time:2020-2-12

Background: In the second half of last year, driven by various factors, I was ready to go to the postgraduate entrance examination. With the recommendation of my former colleagues, I took part in the postgraduate entrance examination training course. The training course sent out paper books and online video viewing accounts. Because online video needs to be viewed through the whole process of networking, I had a whim. If I download these videos, I can take them out to watch and review when I am not online. In this background Next, I spent a few days to do a simple app to help review. A few days ago, I passed the postgraduate entrance examination. In these days, I used my spare time to improve the app function, and recorded the development process and some insights;

Effect:

 

 

Function brief introduction: online play (download). M3u8 video, PC browser view and offline watch!

Be careful:Due to the privacy of the training class video, no video playing screen is provided here, please understand!

 

How to achieve it?

The videos of many training classes on the market are generally not ordinary. MP4 videos, many of which are encrypted or special video formats known by non professionals. During the work period, we often use the bag grabbing tool. I want to try the bag grabbing tool first, maybe we can learn some information, so as to inspire the next step, OK, just do it

 

When we use Charles to capture the package, we find that there are constantly new requests under the domain name in the screenshot, there is a.M3U8 request address, and there is also a constantly generated.Ts.ts request address.

There were a lot of online articles related to technology realization. At that time, the preparation time was short. I sorted out my core needs. I searched for relevant “wheels” on the Internet, and directly assembled them if I could meet my needs,Ha-ha 

 

 

Technical points

1、 IOS audio and video playback

IOS audio and video playback technology is now very mature. Previously, there were functions related to video playback in the project. The main class used in the code is avplayer. I suggest that you first look at the apple document, which is the most comprehensive and authoritative. What you find on the Internet is not comprehensive. You need to integrate different articles to be comprehensive. Open the Xcode window developer documentation and search the class name

 

Official notes: this class can be used to process local and remote file based media playback, such as quiktime movies, MP3 audio files, and audio-visual media provided by HLS live broadcast; using this object, you can directly play the. M3u8 file mentioned above;

However, there will be a problem. When the network is not good, the experience will be terrible, and you can’t watch it offline. Can you process it into a commonly used. MP4 file, which is convenient to save and watch offline?

The basic idea is: batch download. Ts.ts file = > merge file = > transcode to. Mp4

Downloading ts.ts files in batch is the basic idea of downloading files on the network

Merge file: a nsmutabledata is used to store the merged file data. Each. TS fragment data is read circularly and appended to nsmutabledata

Transcoding to. MP4: this step needs to be done with the ffmpeg framework. The corresponding IOS library can be called in the command-line way. Here’s an example of transcoding:

2000K – Y / users / Mac account name / library / Developer / coresimulator / devic es/FDABA415-C26F-483F-B0F8-CDC02A03054B/data/Containers/Data/Application/85D2E7FC-F389-40D3-9607-03718564AFD3/Documents/Download/9f344d0e79cbceceed64ecaa62adf709/9f344d0e79cbceceed64ecaa62adf709.mp4

The call mode is as follows:

Ffmpeg - SS 00:00:00 - I source file. TS file path - B: V 2000K - y destination. MP4 file path

There is a problem. I transcode it and find that the. MP4 video file is surprisingly large, 724MB, while the. TS source file is only about 70mb. If I download the training class video, it will occupy a lot of mobile space. My mobile phone is 128GB, and the remaining space is less than 15GB.

I’ll remove all the parameters and use the default

ffmpge -i

Transcoding out is about 180MB, the file size transcoding out of different command parameters is very different, which involves the understanding of ffmpeg and some basic concepts of audio and video.

 

There are many commands for. Ts to. MP4. I found the following command line online:

Overwrite the target file, use H264? Mp4toannexb, use the source file sound codec, video codec ffmpeg - Y - I - vcodec copy - acodec copy - vbsf H264? Mp4toannexb, and set the code rate of the target file video to 2000kbps
ffmpeg -ss 00:00:00 -i  -b:v 2000K -y 
ffmpeg -y -i  -c:v libx264 -c:a copy -bsf:a aac_adtstoasc

 

The code of using the ffmpeg command line in IOS involves some methods of Objective-C C language. Share the following

//Input source file sandbox path
    char* input=(char*) [inputPath UTF8String];
    //Output destination file sandbox path
    char* output= (char*)[outpath UTF8String];
    //Ffmpeg command
    char* a[]={"ffmpeg","-y","-i",input,output};
    //Start ffmpeg
    ffmpeg_main(sizeof(a)/sizeof(*a),a);

 

 

 

Basic concepts

 

1. Description of video and video format

Video is a collection of more than n pictures. The function of playing video is to continuously play “many” pictures. The time interval between each picture is very short. We call each picture as each frame, and the number of pixels in each frame is called the resolution of this image. For example, we have a 1.mp4 video file, and its resolution is the resolution of each frame;

To get a video file, we need to know not only the common resolution and package format, but also the frame rate. The frame rate indicates how many images can be played in one second, and the frame rate of the video can be constant or dynamic. The code rate indicates the total amount of video data recorded in one second, such as a 24 minute, 900 MB video

Frequency, the code rate is 7200mbit / 1440s = 5000kbps = 5mbps, the unit is generally kbit / s or Mbit / s, a video has not only image but also audio, and the code rate is the sum of the two. Conversely, through the code rate, we can easily know the size of a video file, and its calculation formula is (audio coding rate + video coding rate) Kbps / 8 * video length s

We know that if a picture is not encoded, it takes up a lot of byte space. For example, if a picture has a pixel of 1024 * 1024 and a bit depth of 32 bits, then the picture takes up a space of 1024 * 1024 * 32 / 8 = 4096kb. If a video simply adds up the space occupied by all the pictures, then the space occupied will be surprisingly large, which is a big problem for storage and transmission.

The video files with the suffix of MP4 we see everyday are the files that encode the video stream and audio stream and then package them in the format of MP4

Mp4 is a video encapsulation format. We will also see some video encapsulation formats such as. MKV,. Avi, etc.

As mentioned before, video stream data needs to be encoded to reduce space consumption. The most famous video coding format is H.264

 

2. How is the video played?

This app plays. MP4 files offline. What’s the principle? I use “wheel” directly. This wheel is sjbideoplayer. The principle of video playing in the app is directly used by Apple’s official avplayer. It’s encapsulated on the basis of it. We don’t study encapsulation here. What can avplayer do?

In the previous advertising business, video was also processed with this method. Limited by the business needs, the use is not deep enough. Here, analyze sjbideoplayer to understand its strength!

a. Load the local video file, start playing at the beginning and fixed point, build a custom playback size arbitrarily, and add it to uiview.

b. Support continuation and rotation, rotation fixation

c. Normal playback control

 

Standing on the giant’s shoulder for too long, it’s too cold to be high, a little flustered, to understand the knowledge of the bottom playing first!

To get an mp4 video file and play it, it needs a decoding process. Like pictures, it needs decoding data. The general playback process is as follows:

 

 

 

3. HLS (HTTP live)

HLS is Apple’s dynamic rate adaptive technology, based on HTTP streaming media network transmission protocol. Its working principle is to divide the whole stream into small HTTP based files to download, and only download some at a time. When the media stream is playing, the client can choose to download the same resources from many different standby sources at different rates, allowing the streaming session to adapt to different data rates. The supported video stream is H.264. The playback link of m3u8 suffix we see on the video website is the video using HLS protocol, which includes an m3u8 index file, TS media fragmentation and key encryption string file (the encryption string file is optional).

HLS has the following advantages:

a. Users can cache one video segment after watching it, so as to avoid watching only one video segment but caching the entire video file, so as to reduce server pressure and traffic consumption. (let me think that when I was doing advertising business, in video advertising, it was similar to many advertising SDKs on the market. MP4 video format was used, but it was difficult to meet this scenario. If it was aggregated, the video cache efficiency of each family would be a pkable point, and we can think about this later.)

Add:HLS can do encryption to ensure the security of files and prevent them from being stolen

1. The common one is the anti-theft chain (strictly speaking, it is not encryption), that is, to dynamically generate a token for the URLs of m3u8 and TS files, such as this:
http://www.cuplayer.com/m3u8/hunan/desc.m3u8?stream_id=hunan

This URL changes dynamically with many parameters, such as time, user ID, IP address, content ID, which makes you unable to use this URL to steal the chain. This way can prevent other websites from directly using your URL to watch or download from ordinary users.

The URL of TS file also needs to add request token, It will change to a method similar to http: / / server / file. TS? Token = XXXX. In this way, the disk storage location of the TS file does not need to change, but the URL can change (you can use the query string method or the URL Rewrite method). Note that since the URL is generated by m3u8, it means that the m3u8 file is generated dynamically rather than statically

About CDN caching, first of all, m3u8 files must not be cached, otherwise, what about the dynamic URL of TS fragmentation?
Then the dynamic URL’s TS fragment CDN is definitely not cached by default, but you can customize it a little to let CDN ignore the token part of the URL. Many CDNs have their own anti-theft chain solutions.

2. DRM encryption. The way of anti-theft chain is a kind of general protection. If you want to completely protect your content, you must encrypt the TS content. M3u8 has this tag: ා ext-x-key. Generally speaking, it will provide a URL to obtain the encryption key, and then decrypt the TS segment to play the file

Encryption: https://blog.csdn.net/cnhome/article/details/73250495

Decryption: https://blog.csdn.net/sbdx/article/details/80595094

 

Video encryption scheme: https://blog.csdn.net/ai2000ai/article/details/83106101

 

b. According to the network bandwidth switching different bit rates, both speed and clarity.

 

What is m3u8?

Packet capturing analysis request link is a file with suffix. M3u8 content type of “application / vnd. Application. Mpegurl”

 

Its file content is. TS segment list. Each. TS segment corresponds to different data of the video stream. All. TS segments constitute the video data. The client continuously downloads the segments. Because the segment interval between the segments is very short, it finally looks like a complete playback stream, that is, the video being played.

The student’s mobile phone repeatedly loads the. M3u8 file in a polling like manner and appends the. TS segment to play the streaming media.

Understand the contents of. M3u8 file as follows:

#Ext-x-targetduration: indicates the maximum duration of each. TS segment

#Extinf: indicates the duration of the following. TS fragment

#Ext-x-version: indicates all compatible versions of the playlist

#Ext-x-endlist: indicates the end of the m3u8 file

There are also some tag descriptions that I haven’t published in my example. Please refer to the official description: https://tools.ietf.org/html/draft-pantos-http-live-streaming-06. Here is a Chinese Translation: https://www.cnblogs.com/shakin/p/3870442.html

The video I captured is a recorded video. The data content format of the playlist is a single rate video adaptation stream, and sometimes the network bandwidth of different users is different. The content in the. M3u8 file may have different rate files, and the user’s mobile phone will choose a file suitable for itself to play, so as to ensure the smooth live video stream; multi rate video The following marks will be added to the frequency stream:

#Ext-x-stream-inf: indicates the next URL file in the playlist to identify another playlist file

 The tag contains the following properties

Bandwidth specifies the code rate, which is used to adaptively select the corresponding code stream for playing in different network bandwidth

Program-id unique ID

Codecs specifies the encoding type of the stream

An example is as follows:

 #EXTM3U

 #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1280000

 http://example.com/1.m3u8

 #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1280000

 http://example.com/2.m3u8

The client will choose the highest bit rate. M3u8 request first

 

The single bit rate is as follows:

 #EXTM3U

 #EXT-X-TARGETDURATION:5220

 #EXTINF:5220,

 http://media.example.com/1.ts

 #EXT-X-ENDLIST

 

All the. M3u8 I come across here are unencrypted. The encrypted content will generally be more than the content in the red box below

 

Find time later to see how to deal with this kind of video.

 

 5. FFMPEG

Ffmpeg is a cross platform video and audio recording, conversion and streaming solution, where the function of video format conversion is needed.

There are many articles on the Internet about the use of ffmpeg on the IOS platform. How to use this “wheel” is not mentioned here.

It took me nearly 20 minutes to compile the IOS library on my Mac Pro. I put the ffmpeg compiled file on the baidu online disk: https://pan.baidu.com/s/1xgtgku5ikxaarqrqakehrsq extraction code: egt2, which is convenient for you to download and use directly.

Explain some static libraries

Libavformat: used to generate and parse various audio and video packaging formats.
Libavutil: the core tool library, to do some basic audio and video processing operations.
Libavcodec: encoding and decoding of various formats of audio and video.
Libswscale: image format conversion module.
Libavfilter: audio and video filter library.
Libpostproc: post processing module.
Libswresample: for audio resampling.
Libadvice: used for audio and video acquisition of hardware.
Libavresample: This is compiled from the old version. The new version uses libswresample instead

In IOS, ffmpeg library is used for video related operations

Generally speakingCommand line modeThere are many parameters involved in the command line. Please refer to official documents and online articles before coding. Open the terminal command on your Mac. It is recommended to test the corresponding command manually.

First, install ffmpeg

brew install ffmpeg

The second step is to prepare the video related files.

Step 3: test the command line of the development scenario

Format Description:
-I filename input file
-T duration set processing time
-SS position set start time
-b: V bitrate setting video rate    
-b: A bitrate setting
-R FPS set frame rate
-S WXH sets the frame size in WXH format, such as 640x480
-c: V codec setting video encoder
-c: A codec setting audio encoder
-Ar freq sets the audio sampling rate

Some examples:
-Codec copy forces the use of codec encoding and decoding. Copy means no recoding
-Vcodec copy - acodec copy as above
-VN cancel video output
-An cancel audio output
-BSF: V H264? Mp4toannexb video data is converted to original H264 data using bitstream filter H264? Mp4toannexb
-F MP4 specifies that the output format is mp4

Share some examples, also welcome to leave a message to discuss!

Ffmpeg - SS 00:00:15 - t 00:00:05 - I input.mp4 - vcodec copy - acodec copy output.mp4 video cutting starts from 15 seconds of input.mp4, and the video of 5 seconds after interception is saved in output.mp4

ffmpeg -i input.mp4 -vn -acodec copy output.m4a
-VN cancel video output-Acodec specifies audio encoding copy to extract audio files from a video file without recoding

ffmpeg -i input.mp4 -an -vcodec copy output.mp4
-An cancel audio output-Vcodec specifies that the video coding copy means that the audio in a video is muted without recoding, and only the video is retained

ffmpeg -i output.mp4 -an -vcodec copy -bsf:v h264_mp4toannexb output.h264
The video data is converted to the original H264 data by using the bitstream filter H264 ﹐ mp4toannexb. Extract video stream from MP4 file and export it to raw H264 data

ffmpeg -i test.aac -i test.h264 -acodec copy -bsf:a aac_adtstoasc -vcodec copy -f mp4 output.mp4
-F specifies the output format to generate MP4 files using aac audio data and H264 video

ffmpeg -i input.wav -acodec libfdk_aac output.aac
Transform the encoding format of audio files

ffmpeg -i input.wav -acodec pcm_s16le -f s16le output.pcm
Exporting PCM raw data from wav audio file

ffmpeg -i input.flv -vcodec libx264 -acodec copy output.mp4
Recode video file, copy audio stream and encapsulate it in MP4 format file

ffmpeg -i input.mp4 -vf scale=100:-1 -t 5 -r 10 image.gif
-VF sets the filter of video to 100 (scale filter of video filter is used), and the frame rate is changed to 10 (- R) and the duration is changed to 5 (-t) Converting an mp4 video to a GIF motion picture

ffmpeg -i input.mp4 -r 0.25 frames_%04d.png
Every 4 seconds, a video frame is captured to generate a picture, and the generated picture is incremented from frames_.png

ffmpeg -i frames_%04d.png -r 5 output.gif
Using a set of pictures can form a gif

ffmpeg -i input.wav -af ‘volume=0.5’ output.wav
Use the volume effect to change the volume in an audio media file

ffmpeg -i input.wav -filter_complex afade=t=in:ss=0:d=5 output.wav
Fade in the audio file for the first 5 seconds

ffmpeg -i input.wav -filter_complex afade=t=out:st=200:d=5 output.wav
Fade the audio from 200s for 5 seconds

ffmpeg -i input1.wav -i input2.wav -filter_complex amix=inputs=2:duration=shortest output.wav
Combine the two audio, and take the shorter audio time as the output

ffmpeg -i input.wav -filter_complex atempo=0.5 output.wav
The audio is processed at 0.5 times the speed to generate output.wav. The length of time is changed to 2 times of the original, and the pitch is unchanged

ffmpeg -i test.avi -i frames_0004.jpeg -filter_complex overlay after.avi
Add watermark to video

Ffmpeg - I test. Avi - VF "DrawText = fontfile = simhei. TTF: text = 'thunder': x = 100: y = 10: fontsize = 24: fontcolor = yellow: Shadow = 2" after.avi
Add text watermark

ffmpeg -i input.flv -c:v libx264 -b:v 800k -c:a libfdk_aac -vf eq=brightness=0.25 -f mp4 output.mp4
The brightness value range of video enhancement parameter is - 1.0 to 1.0

ffmpeg -i input.flv -c:v libx264 -b:v 800k -c:a libfdk_aac -vf eq=contrast=1.5 -f mp4 output.mp4
The contrast parameter contrast is added to the video, and the value range is from - 2.0 to 2.0

ffmpeg -i input.mp4 -vf “transpose=1” -b:v 600k output.mp4
Video spinner use

ffmpeg -i input.mp4 -an -vf “crop=240:480:120:0” -vcodec libx264 -b:v 600k output.mp4
Video clip effects using

ffmpeg -f rawvideo -pix_fmt rgba -s 480*480 -i texture.rgb -f image2 -vcodec mjpeg output.jpg
Convert an RGBA data to a JPEG image

ffmpeg -f rawvideo -pix_fmt yuv420p -s 480*480 -i texture.yuv -f image2 -vcodec mjpeg output.jpg
Convert a data in YUV format to a picture in JPEG format

ffmpeg -re -i ipnut.mp4 -acodec copy -vcodec copy -f flv rtmp://xxx
Push a video to a streaming server


ffmpeg -i http://xxx/xxx.flv -acodec copy -vcodec copy -f flv output.flv
Dump the stream on the streaming server locally

 

Additionally share some command assist tests of ffprobe and ffplay

Ffprobe common commands
Ffprobe filename
View audio and video file information

Ffprobe - show format filename
View the output format information, time length, file size, bit rate, number of streams, etc.

Ffprobe - print'format JSON - show'streams filename
Output details in JSON format

Ffprobe - show frame filename
Display frame information

Ffprobe - show_packets filename
View package information

Ffplay common commands
Ffplay filename
Play audio and video files

Ffplay filename -loop 10
Loop files 10 times

Ffplay filename -ast 1
If the parameter of the first audio stream is 2, the second audio stream will be muted.

Ffplay filename -vst 1
If the parameter of the first video stream is 2, the second video stream does not display a black screen

Ffplay. PCM file - f format - channels 2 channels - AR sampling rate
Parameters must be set correctly to play PCM file

Ffplay - F rawvideo - pixel? Format yuv420p - s 480480480. YUV file
-F rawvideo represents the original format
-Pixel format yuv420p
-S 480480480 w h

Ffplay - F rawvideo - pixel? Format RGB24 - s 480 * 480. RGB file
Play RGB raw data

Ffplay filename -sync audio
Specify ffplay to use audio as the reference for audio and video synchronization. The same is true for the default ffplay

Ffplay filename -sync video
Specify ffplay to synchronize audio and video based on video

Ffplay filename -ext video
Specify ffplay to use external clock as reference for audio and video synchronization

Official call document of ffmpeg: https://ffmpeg.org/documentation.html, which can be used to view relevant parameters

 

The above is a command-line call. You need to compile the command-line static library of the corresponding version of ffmpeg to make it easy to call in IOS. In addition, you can also useAPI modeCode call for

On the official website, http://ffmpeg.org/documentation.html

You can find the corresponding version of the API interface document.

 

 

Share another test demo of ffmpeg 3.0: https://pan.baidu.com/s/1rzpipny79_ze-gxsqqpryw extraction code: jfd2You can directly download and use, test and learn the relevant ffmpeg commands!

 

6. push flow

Use this link: https://blog.csdn.net/lg767201403/article/details/83009938 to test video streaming.