Wkwebview audio and video media playback processing

Time:2021-11-11

1. Set wkwebviewconfiguration.

Realize the functions that media files can be played automatically and played using embedded HTML5
Use thisTest website

//Initialize configuration object
WKWebViewConfiguration *configuration = [[WKWebViewConfiguration alloc] init];
//The default is no, which determines whether to play video with embedded HTML5 or local full screen control
configuration.allowsInlineMediaPlayback = YES;
//Automatic play, no user gesture is required to start play
//Wkaudiovisualmediatypenone audio and video playback does not need to be triggered by user gestures, that is, it is automatic playback
configuration.mediaTypesRequiringUserActionForPlayback = WKAudiovisualMediaTypeNone;
configuration.allowsAirPlayForMediaPlayback = YES;
configuration.allowsPictureInPictureMediaPlayback = YES;
    
self.webView = [[WKWebView alloc] initWithFrame:self.view.bounds configuration:configuration];
self.webView.navigationDelegate = self;
Nsurl * url = [nsurl urlwithstring: @ "test URL"];
[self.webView loadRequest:[NSURLRequest requestWithURL:url]];
[self.view addSubview:self.webView];

Because the autoplay and playinline properties are not set for H5 video. We need to inject ourselves to achieve the effect.

NSString *jSString = @"document.getElementsByTagName('video')[0].setAttribute('playsinline','');";
NSString *jSString2 = @"document.getElementsByTagName('video')[0].autoplay=true;";
//For JavaScript injection
WKUserScript *wkUScript = [[WKUserScript alloc] initWithSource:jSString injectionTime:WKUserScriptInjectionTimeAtDocumentEnd forMainFrameOnly:YES];
WKUserScript *wkUScript2 = [[WKUserScript alloc] initWithSource:jSString2 injectionTime:WKUserScriptInjectionTimeAtDocumentEnd forMainFrameOnly:YES];
[configuration.userContentController addUserScript:wkUScript];
[configuration.userContentController addUserScript:wkUScript2];

2. Listen for the callback of the player in the web page

Two approaches can be used.

2.1 using HTML5 audio / video events

HTML5 audio / video event code can be completed by H5 colleagues or injected by app side.
The injection code is as follows:

NSString *jSString3 = @"document.getElementsByTagName('video')[0].addEventListener('canplay', function(e) {window.webkit.messageHandlers.readytoplay.postMessage(\"canplay\");})";
NSString *jSString4 = @"document.getElementsByTagName('video')[0].addEventListener('pause', function(e) {window.webkit.messageHandlers.pause.postMessage(\"pause\");})";
NSString *jSString5 = @"document.getElementsByTagName('video')[0].addEventListener('play', function(e) {window.webkit.messageHandlers.play.postMessage(\"play\");})";
NSString *jSString6 = @"document.getElementsByTagName('video')[0].addEventListener('ended', function(e) {window.webkit.messageHandlers.ended.postMessage(\"ended\");})";
WKUserScript *wkUScript3 = [[WKUserScript alloc] initWithSource:jSString3 injectionTime:WKUserScriptInjectionTimeAtDocumentEnd forMainFrameOnly:YES];
[configuration.userContentController addUserScript:wkUScript3];
WKUserScript *wkUScript4 = [[WKUserScript alloc] initWithSource:jSString4 injectionTime:WKUserScriptInjectionTimeAtDocumentEnd forMainFrameOnly:YES];
[configuration.userContentController addUserScript:wkUScript4];
WKUserScript *wkUScript5 = [[WKUserScript alloc] initWithSource:jSString5 injectionTime:WKUserScriptInjectionTimeAtDocumentEnd forMainFrameOnly:YES];
[configuration.userContentController addUserScript:wkUScript5];
WKUserScript *wkUScript6 = [[WKUserScript alloc] initWithSource:jSString6 injectionTime:WKUserScriptInjectionTimeAtDocumentEnd forMainFrameOnly:YES];
[configuration.userContentController addUserScript:wkUScript6];

The code of JS received by app side is as follows:
Comply with wkscriptmessagehandler protocol

@interface ViewController () <WKNavigationDelegate,WKScriptMessageHandler>
@end

Then add the protocol for wkwebviewconfiguration

//Add a protocol
[configuration.userContentController addScriptMessageHandler:self name:@"readytoplay"];
[configuration.userContentController addScriptMessageHandler:self name:@"play"];
[configuration.userContentController addScriptMessageHandler:self name:@"pause"];
[configuration.userContentController addScriptMessageHandler:self name:@"ended"];

Use the following methods to get player events

#pragma mark - WKScriptMessageHandler

//!  Wkwebview calls back this method when it receives a scriptmessage
- (void)userContentController:(WKUserContentController *)userContentController didReceiveScriptMessage:(WKScriptMessage *)message {
    if ([message.name caseInsensitiveCompare:@"readytoplay"] == NSOrderedSame) {
        NSLog(@"video is readytoplay");
    }
    if ([message.name caseInsensitiveCompare:@"play"] == NSOrderedSame) {
        NSLog(@"video is play");
    }
    if ([message.name caseInsensitiveCompare:@"pause"] == NSOrderedSame) {
        NSLog(@"video is pause");
    }
    if ([message.name caseInsensitiveCompare:@"ended"] == NSOrderedSame) {
        NSLog(@"video is ended");
    }
}

reference material:
HTML audio / video reference manual
Video property and event usage
Wkwebview wkscriptmessagehandler protocol for interaction between IOS and JS

2.2 there is another kind that app can implement by itself. It uses avaudiosession to listen:

To listen with avaudiosession, you must use avaudiosessioncategoryoptionmixwithothers. This will cause switching to other audio and video apps without interrupting the player. For example, Netease cloud music, bilibilibili.
A phone call will interrupt the player.

NSError *sessionError = nil;
[[AVAudioSession sharedInstance] setCategory:AVAudioSessionCategoryPlayback
                                 withOptions:AVAudioSessionCategoryOptionMixWithOthers
                                       error:&sessionError];
[[AVAudioSession sharedInstance] setActive:YES error:nil];
[[NSNotificationCenter defaultCenter] addObserver:self selector:@selector(audioSessionSilenceSecondaryAudioHint:)
                                             name:AVAudioSessionSilenceSecondaryAudioHintNotification
                                           object:[AVAudioSession sharedInstance]];
- (void)audioSessionSilenceSecondaryAudioHint:(NSNotification *)notification
{
    NSDictionary *userInfo = notification.userInfo;
    NSLog(@"audioSessionSilenceSecondaryAudioHint  %@",userInfo);
}

Start playback output:

2021-04-01 15:22:31.302248+0800 webViewPlayMedia[18078:2811391] audioSessionSilenceSecondaryAudioHint  {
    AVAudioSessionSilenceSecondaryAudioHintTypeKey = 1;

End playback output:

2021-04-01 15:22:31.382646+0800 webViewPlayMedia[18078:2811391] audioSessionSilenceSecondaryAudioHint  {
    AVAudioSessionSilenceSecondaryAudioHintTypeKey = 0;

3. Get the video playback address and play it with a custom player

- (void)webView:(WKWebView *)webView didFinishNavigation:(null_unspecified WKNavigation *)navigation {
    NSLog(@"WKPhoneWebView didFinishNavigation");
    
    NSString *JsStr = @"(document.getElementsByTagName(\"video\")[0]).src";
    [self.webView evaluateJavaScript:JsStr completionHandler:^(id _Nullable response, NSError * _Nullable error) {
        if(![response isEqual:[NSNull null]] && response != nil){
            //Intercepted the video address
            NSLog(@"response == %@",response);
        }else{
            //No video link
        }
    }];
}

4. Pit

4.1 when playing the video, there will be an error prompt:
2021-04-01 09:34:57.361477+0800 webViewPlayMedia[17109:2655981] [assertion] Error acquiring assertion: <Error Domain=RBSAssertionErrorDomain Code=3 "Required client entitlement is missing" UserInfo={RBSAssertionAttribute=<RBSDomainAttribute| domain:"com.apple.webkit" name:"MediaPlayback" sourceEnvironment:"(null)">, NSLocalizedFailureReason=Required client entitlement is missing}>
2021-04-01 09:34:57.361610+0800 webViewPlayMedia[17109:2655981] [ProcessSuspension] 0x1043dc990 - ProcessAssertion: Failed to acquire RBS MediaPlayback assertion 'WebKit Media Playback' for process with PID 17110, error: Error Domain=RBSAssertionErrorDomain Code=3 "Required client entitlement is missing" UserInfo={RBSAssertionAttribute=<RBSDomainAttribute| domain:"com.apple.webkit" name:"MediaPlayback" sourceEnvironment:"(null)">, NSLocalizedFailureReason=Required client entitlement is missing}

However, if the background property is set, it still cannot be released, but it does not affect the playback.
The problem ishttps://stackoverflow.com/questions/66493177/required-client-entitlement-is-missing-in-wkwebviewIt has also been put forward, but there is no solution.

4.2 ios13.2 13.3 the system mobile phone will continuously report errors when loading wkwebview:
2021-04-01 15:55:11.083253+0800 webViewPlayMedia[342:59346] [Process] kill() returned unexpected error 1

In this system version, wkwebview cannot be played if wkwebviewconfiguration is configured.

Information:Received console warning: when I load wkwebview in ios13.2, [process] kill() returned unexpected error 1
This error has been fixed in version 13.4.

Recommended Today

Modify user information changeinfo

When judging the persistence layer: Problem: there is such a problem when modifying user information. For example: the user’s email is not required. It was not empty originally. At this time, the user deletes the mailbox information and submits it. At this time, if it is not empty to judge whether it needs to be […]