Optimize recorder H5 recording: upload the server while recording and transcoding, support wechat to provide Android IOS hybrid app source code

Time:2020-3-31

With the increase of the supporting functions of recorder H5 GitHub open source library, the efficiency of audio transcoding processing gradually fails to keep up with the demand. Recently, time has been taken to upgrade and optimize the audio transcoding part to support more practical functions.
In addition, the hybrid app of IOS has also been ported. Both Android and IOS have demo app.

Online test, GitHub address: https://github.com/xiaoyuecn/recorder

1、 Characteristics of recorder H5

(1) Browser support

  1. PC: including but not limited toChromeFirefox
  2. Android:ChromeFirefoxAndroid WebViewTencent Android X5 kernel (QQ, wechat) (do not support UC series browsers, such as UC, Alipay, browser of most domestic mobile phone manufacturers)
  3. IOS:SafariWechat (jssdk)
  4. Hybrid app: provide Android and IOS appJsBridgeJust connect the interface

(2) Functional support

  1. Already providedmp3 wav ogg(beta) webm(beta) amr(beta)Format support, support any format extension
  2. Provide real-time volume feedback
  3. Provide a beautiful real-time waveform drawing extension
  4. Recording permission request and recording function are designed separately. You can request permission first and start recording when necessary
  5. mp3The default size of the format recording file is 2KB / s, the voice quality is acceptable, and the voice quality can be reduced to 1KB / s
  6. ^ [new] ^mp3 wavThe format transcoding is “fast”, and the audio file can be obtained almost immediately when the recording is finished. In addition, the transcoding can be realized while recording into a small voice clip file, which can be uploaded to the server in real time
  7. Provide the demo with rich functions, support online browser testing, ^ [new] provide the Android (15 + hybrid app installation package and source code, ^ [new] provide the IOS (9.0 + hybrid app source code for self compilation

2、 Use preview screenshot

(1) Mobile H5

Optimize recorder H5 recording: upload the server while recording and transcoding, support wechat to provide Android IOS hybrid app source code

(2)IOS Hybrid App

Optimize recorder H5 recording: upload the server while recording and transcoding, support wechat to provide Android IOS hybrid app source code

(3)Android Hybrid App

Optimize recorder H5 recording: upload the server while recording and transcoding, support wechat to provide Android IOS hybrid app source code

3、 Application scenario

  • H5 recording on mobile terminal and PC terminal
  • H5 sending voice on mobile and PC
  • H5 speech recognition on mobile and PC
  • H5 voice chat on mobile and PC

4、 Optimization process record

(1) Why upgrade optimization

At the beginning of writing a recorder, it only pursues short voice recording, so it uses one-time asynchronous (setTimeout fragmentation) transcoding after the end of recording. One minute of voice transcoding on the PC side costs 100-300ms on MP3, and 600-1500ms on the mobile side. The experience delay is not large, and the code logic is very simple; if using the web Worker (multithreading) can almost achieve no delay, but the code seems to be much more complex, so it is not adopted.

Later, it was found in actual use:

  1. A slightly longer voice transcoding mobile terminal is very hard, 2-3 minutes of voice to spend 5-10 seconds for MP3 transcoding;
  2. When the real-time transcoding is sent to the server, except for the rapid encoding of wav format itself, all other formats will cause the recording to jam, because the encoding takes too much time and the recording data during the encoding cannot be received normally.

Based on the above two problems, it seems that only web worker can solve them.

(2) Start using web worker to speed up transcoding

“Acceleration” needs quotation marks. In fact, web worker support is enabled. The transcoding speed is the same, but the transcoding work is put in the background thread, which does not take up the page time.

However, transcoding can be achieved while recording, without waiting for the end of recording to pile up transcoding, the result is that transcoding is instantaneous at the end of recording, no matter how long the recorded voice is.

It can also be perfectly transcoded into a small voice clip file and uploaded to the server in real time, because transcoding does not take up the time of the web page, and the recording can receive data normally without any jam.

(3) The rest of the questions

To support transcoding while recording, it is not only necessary to enable web worker support, but also the support of audio encoder. If the encoder can only transcode the complete PCM data into audio files at one time, then the source code of the encoder needs to be modified to support streaming transcoding and PCM transcoding.

Fortunately, the MP3 encoder in the library supports streaming and can be used without modification; however, it seems that other formats can’t be supported without modification of the encoder source code; except for wav format, the coding speed of wav is surprisingly fast and there is no problem without optimization.

Therefore, web worker is used in MP3 format. MP3 and wav formats support transcoding while recording. Other formats are still asynchronous (setTimeout) transcoding when recording is finished.

5、 The significance of hybrid app

It’s meaningless (or redundant), but IOS doesn’t support all kinds of things: unofficial browsers don’t support it, WebView in app doesn’t support it, but Android supports all kinds of things. The original pure H5 shuttle (included in hybrid APP) is the most convenient, and apple can’t drag its feet.

Therefore, it is only necessary to be compatible with IOS hybrid app, but considering the complexity of simplifying JS (it’s easy to get rid of the pot when there’s a problem), whether Android or IOS can be implemented easily. In addition, the audio coding of Android and IOS is not easy, and it’s not easy to update. Using JS audio transcoding can greatly simplify the logic of app. Therefore, Android also provides hybrid app support.

6、 More support

GitHub warehouse has all the source code, detailed use introduction and demo, and use communication group. If there is any problem in the use process, you can raise issue or send message. It would be better if we could provide reward support. There is QR code in the warehouse. Haha hey ~ GitHub address: https://github.com/xiaoyuecn/recorder

Recommended Today

PHP Basics – String Array Operations

In our daily work, we often need to deal with some strings or arrays. Today, we have time to sort them out String operation <?php //String truncation $str = ‘Hello World!’ Substr ($STR, 0,5); // return ‘hello’ //Chinese string truncation $STR = ‘Hello, Shenzhen’; $result = mb_ Substr ($STR, 0,2); // Hello //First occurrence of […]