Good news for audio and video developers, quickly integrating AI dubbing capabilities

Time:2022-8-17

Audio and video content creators have an increasing demand for AI dubbing in editing applications. It has many advantages. It can not only solve the cost of hiring human dubbing and the limitations of the creator's own accent and language, but also greatly improve production efficiency. For example, the playback time of a short video is as short as tens of seconds, and a long vlog type video is 4-5 minutes. Using the AI ​​dubbing function, multiple editing needs can be achieved in a short time.

HMS Core Audio Editing Service(Audio Editor Kit) provides AI dubbing services to help developers easily build speech synthesis functions in applications, and output text with one click to convert speech. It supports a variety of styles such as cute children's voices, friendly female voices, sunny male voices, English male voices, female voices, etc. You can also customize the timbre you want. The timbre is natural and smooth, and the emotion is rich, which meets the popular scenes that developers pay attention to, such as audio reading, audio content production, and editing.

development practice

1. Development preparation

For detailed preparation steps, please refer to the official website of Huawei Developer Alliance:

https://developer.huawei.com/consumer/cn/doc/development/Media-Guides/config-agc-0000001154009063?ha_source=hms1

2. Edit engineering integration

2.1 Set the authentication information of the application

Developers need to set application authentication information through api_key or Access Token.

  • (Recommended) Set the Access Token through the setAccessToken method, and initialize the settings when the application starts
HAEApplication.getInstance().setAccessToken("your access token");
  • The api_key is set by the setApiKey method, which can be initialized and set once when the application starts, and there is no need to set it multiple times.
HAEApplication.getInstance().setApiKey("your ApiKey");

2.2 Initialize the environment

Initialize the audio editing management class, create the timeline, and the required swimlanes.

// Create an audio editing management class
HuaweiAudioEditor mEditor = HuaweiAudioEditor.create(mContext);
// Initialize the editor's runtime environment
mEditor.initEnvironment();
// create timeline
HAETimeLine mTimeLine = mEditor.getTimeLine();
// create swimlane
HAEAudioLane audioLane = mTimeLine.appendAudioLane();

Import music.

// Add audio resource at the end of the swimlane
HAEAudioAsset audioAsset = audioLane.appendAudioAsset("/sdcard/download/test.mp3", mTimeLine.getCurrentTime());

3. AI dubbing function integration

Call HAEAiDubbingEngine to implement AI dubbing function.

// Configure the AI ​​dubbing engine through this configuration class
HAEAiDubbingConfig haeAiDubbingConfig = new HAEAiDubbingConfig()
// set volume
.setVolume(volumeVal)
// set the speed of sound
.setSpeed(speedVal)
// set the speaker
.setType(defaultSpeakerType);
// Single AI dubbing task callback
HAEAiDubbingCallback callback = new HAEAiDubbingCallback() {
    @Override
    public void onError(String taskId, HAEAiDubbingError err) {
        // error handling
    }
    @Override
    public void onWarn(String taskId, HAEAiDubbingWarn warn) {}
    @Override
    public void onRangeStart(String taskId, int start, int end) {}
    @Override
    public void onAudioAvailable(String taskId, HAEAiDubbingAudioInfo haeAiDubbingAudioFragment, int i, Pair pair, Bundle bundle) {
        // Start receiving the file and save it as a file
    }
    @Override
    public void onEvent(String taskId, int eventID, Bundle bundle) {
        // The synthesis is complete.
        if (eventID == HAEAiDubbingConstants.EVENT_SYNTHESIS_COMPLETE) {
            // AI dubbing task processing ends, that is, all synthesized audio data is processed
        }
    }
    @Override
    public void onSpeakerUpdate(List speakerList, List lanList,
         List lanDescList) { }
};
// AI dubbing engine
HAEAiDubbingEngine mHAEAiDubbingEngine = new HAEAiDubbingEngine(haeAiDubbingConfig);
// Set the AI ​​dubbing task playback process to listen
mHAEAiDubbingEngine.setAiDubbingCallback(callback);
// Real-time AI dubbing and playing API, text is passed in the text of the voice to be converted, modeAI dubbing task playback mode
String taskId = mHAEAiDubbingEngine.speak(text, mode);
// Pause playback
mHAEAiDubbingEngine.pause();
// resume playback
mHAEAiDubbingEngine.resume();
// close compositing
mHAEAiDubbingEngine.stop();

Demo

For more details about Huawei Audio Editing Service, please refer to:
Service official website:https://developer.huawei.com/consumer/cn/hms/huawei-audio-editor/?ha_source=hms1
Get guidance documents:https://developer.huawei.com/consumer/cn/doc/development/Media-Guides/client-dev-0000001107465102?ha_source=hms1

Learn more details>>

accessHuawei Developer Alliance official website
Obtaindevelopment guidance document
Huawei Mobile Services Open Source Warehouse Address:GitHubGitee

Follow us to know the latest technical information of HMS Core for the first time~