Screen sharing access guide

Time:2020-10-3

There are many application scenarios for screen sharing, such as:

  • Online classroom, teachers analyze desktop teaching technology;
  • Online meeting, participants share conference content in full screen;
  • Online medical treatment, agenda analysis of patients’ electronic cases;
  • Remote assistance, remote help others solve problems;

In order to effectively solve these pain points of users, the screen sharing function came into being. The new version of screen sharing is not in the form of application plug-ins. As we all know, before chrome 72, the integration of screen sharing function must integrate plug-ins, which increases the difficulty of developers. The domestic network can not access Google store and download plug-ins, which leads to the limitation of this function and is not adopted by everyone. The new version does not need plug-ins. It only needs to call a few codes to have the screen sharing function. The screen sharing can also choose to turn on and off the screen sound. If the screen shares multimedia with sound, this function is just right. And the screen sharing of anyrtc does not collect audio by default. If you want to output audio when sharing the screen, remember to turn on this function.

The above is about screen sharing on the web side, and the following is about screen sharing on the native side. Windows and MacOS are not much introduced here. It is easy and convenient to call the interface. When Android and IOS use screen sharing, you must open the custom audio and video acquisition module, collect audio and video from the outside, and plug audio and video streams into the SDK. You can go to the interface document for details.

Working principle of screen sharing on Web

Web side screen sharing is actually realized by creating a video track object of screen sharing. You can call the createscreenvideotrack method to create a local video track object for screen sharing. In the process of collecting screen sharing, the browser will ask which screens need to be shared and obtain the screen information according to the user’s choice

In the anyrtc web SDK, aArRTCClientObject can only publish one video track at a time. So if you want to publish local camera video tracks as well as screen sharing, you need to create twoArRTCClientObject, join the same channel, send the shared track of screen and video track of camera along the way.

Chrome screen sharing

Let’s take chrome as an example. If you share the screen on chrome, you can directly call createscreen videotrack.

This feature requires chrome 72 or above. If the software version you are using does not meet this requirement, please use the screen sharing plug-in to share the screen on chrome.

ArRTC.createScreenVideoTrack({
  //You can configure the encoding parameters here. Refer to the API documentation for details
  encoderConfig: "1080p_1",
}).then(localScreenTrack => {
  /** ... **/
});

Share audio

Anyrtc web SDK supports the sharing of screen and local background sound on Chrome browser 74 and above on Windows platform. You need to call thewithAudioThe parameter is set toenable

After this setting, the method returns a list of video track objects shared by the screen and audio track objects for playing background sound locally.

ArRTC.createScreenVideoTrack({
  encoderConfig: "1080p_1",
}, "auto").then([screenVideoTrack, screenAudioTrack] => {
  /** ... **/
});

be careful:

  • After using this method, the end user needs to check the pop-up box of screen sharingShare audioIn order to really take effect.
  • If you choose to share a single application window, you cannot share audio.

Screen sharing access guide

Share the screen and turn on the video at the same time

Because of oneArRTCClientThe object can only send one video track, so if you want to share the screen and start the camera video collection at the same time in one sender, you need to create two arrtcclients, one to send the screen shared track and the other to send the camera track.

async function startScreenCall() {
  const screenClient = ArRTC.createClient({ mode: "rtc", codec: "vp8" });
  await screenClient.join("<TOKEN>", "<CHANNEL>");

  const screenTrack = await ArRTC.createScreenVideoTrack();
  await screenClient.publish(screenTrack);

  return screenClient;
}

async function startVideoCall() {
  const videoClient = ArRTC.createClient({ mode: "rtc", codec: "vp8" });
  await videoClient.join("<TOKEN>", "<CHANNEL>");

  const videoTrack = await ArRTC.createCameraVideoTrack();
  await videoClient.publish(videoTrack);

  return videoClient;
}

Promise.all([startScreenCall(), startVideoCall()]).then(() => { /** ... **/ });

If you subscribe to yourself, you will generate additional charges, as shown in the following figure:

Screen sharing access guide

Development considerations

  • OneArRTCClientObject can only send one video track
  • The uid of users who are responsible for publishing screen sharing should not be fixed at the same value, otherwise the shared flow of the same uid may cause mutual kicking in some scenarios.
  • When the screen is shared, the client of the local streamDo not subscribe to local screen sharing streamsOtherwise, the charge will be increased.
  • When the screen is shared on Windows platform, if QQ chat window is shared, the screen will be black.

More fun uses

  • Video stream: one client publishes two video streams, one big stream and one small stream. Other clients can pull out the large and small streams according to their own network conditions.
  • Stream fallback: after the client turns on the stream fallback function, the SDK automatically detects the uplink and downlink of the current network. According to the packet loss rate, network delay and other factors, it automatically changes the pulled flow into a small stream. If the small stream can not meet the current network conditions, it will automatically switch the audio streaming to ensure the priority of audio communication.
  • Custom audio and video stream. When the local camera can’t meet the current demand, the collected audio and video stream can be put into the SDK. The SDK is responsible for the transmission of custom audio and video stream.
  • Smooth up and down the wheat, the wheat time is maintained at 200ms, click on the wheat can immediately carry out audio and video communication, widely used in live connected wheat, voice black and other scenes.
  • Mix, play online or local music, with whom the local voice is sent to the far end.
  • Face recognition, which reports a series of detected results to local users, including the distance between the face and the device screen. This function can be used to remind users to pay attention to eye hygiene and keep a certain distance from the screen.

There are still some ways to play, waiting for your needs, oh, as long as the demand is reasonable, we can achieve it for you~

Anyrtc entrepreneurship support program

  • 300000 free minutes to help start-ups develop rapidly.

Any RTC start-up support program can enjoy 300000 free minutes as long as it passes the enterprise audit and contacts customer service to join the anyrtc entrepreneurship support program. The number of minutes can reduce the cost of real-time audio and video technology services, and start the project quickly with zero cost.

  • Exclusive technical guidance and support

Anyrtc provides one-to-one customer service for start-ups, providing professional and serious services to customers and answering your questions in time. And to provide customers with exclusive technical guidance, faster start, easy online!

contact us

Tel: 021-65650071

QQ Consulting Group: 580477436

Arcall technology exchange group: 597181019

Email address: [email protected]

Technical issues: Developer Forum

For more help go to: www.anyrtc.io

Recommended Today

Java security framework

The article is mainly divided into three parts1. The architecture and core components of spring security are as follows: (1) authentication; (2) authority interception; (3) database management; (4) authority caching; (5) custom decision making; and;2. To build and use the environment, the current popular spring boot is used to build the environment, and the actual […]