[technology sharing] Introduction to RTC and Agora RTC SDK integration


Recently, I was lucky to see the RTC speech notice issued by Agora and attended several cloud lectures. As I have been engaged in the communication industry before and have done the research and development of communication equipment of telecom operators, I have always been interested in RTC technology. Here I would like to share my understanding.

RTC real time communication, real-time communication technology. When it comes to real-time communication, what you are most familiar with is that you call. Of course, the telecommunication network has experienced 2G, 3G, 4G, 5g, and the main wireless side and traffic development of 4G and 5g. In fact, there is no significant change in real-time communication. It is still voice and video, so I will not repeat it here.

Later, just real-time communication couldn’t meet the increasingly rich life needs of the masses, so Ott flourished, such as the familiar microblog and wechat, because many things don’t need real-time high-speed someone, such as just drying their current mood / food / interesting jokes, and they don’t need the other party’s second reply or immediate response expression.

Of course, there are still many scenarios that need to be used in real-time communication, such as voice / video conference during the outbreak of the epidemic due to the explosion of remote office, live broadcast with goods in the past two years, and real-time communication has once again come into people’s attention; however, if these are based on the existing real-time communication mechanism of operators, it is difficult to meet the needs of the people who are increasingly rich / changing, and After all, telephone communication is much more expensive than traffic, so real-time communication technology based on non carrier communication network has been greatly developed.

Many companies have seen this kind of demand, the most famous of which is the webrtc Technology launched by Google and the best-known Agora sound network in China. Webrtc actually expands the development field of RTC. It is a real-time communication solution initiated by Google. Based on the browser of the system, developers can quickly build an audio and video communication application, and the project is completely open source.

After Google launched webrtc, W3C standard organization and ieft organization quickly established standard group to improve the whole technical standard system, which also provided a good foundation for the generalization of webrtc, and webrtc has actually become a communication standard.

The overall architecture idea and flow chart of webrtc are as follows. Webrtc encapsulates media processing well

[technology sharing] Introduction to RTC and Agora RTC SDK integration

However, in the initial webrtc, there was no signaling protocol defined, so developers can play it freely. In fact, there are many signaling processing methods for different protocols, such as HTTP restful, JSON over websocket, SIP over HTTP or sip plugin. Generally speaking, a central server that can be accessed by both sides is needed to store the information of both sides of the communication (caller and callee). At the same time, the central server can help them exchange the information needed for connection. The purpose of signaling protocol is to ensure the existence of communication peer (including IP address, port, audio and video codec format negotiation, etc.) that can be obtained by both sides of media communication, In this way, the caller and callee can send media data to each other.

[technology sharing] Introduction to RTC and Agora RTC SDK integration

(the example in the figure is P2P direct media interworking between browsers. This situation can be achieved without symmetric NAT. Stun / turn technology is needed in the case of NAT, which will not be described in detail here.)

Of course, there is also a way of media interworking, that is, signaling and media go through the central end, and the central end has public network IP, so that the problem of NAT traversing can be avoided (Agora sound network is handled in this way).
[technology sharing] Introduction to RTC and Agora RTC SDK integration
Of course, through the lecture, we learned that Agora has gone beyond the scope of browser and webrtc to mobile Android, IOS, windows app, and embedded Linux app; (of course, supporting different browsers can realize Android and IOS, but after all, mobile and embedded terminals use less web pages and more apps).

Well, to get to the point, what I said above is actually to talk about how to really get through a point-to-point call and how the process is. This will be helpful for the following SDK integration development. Otherwise, I just get through, and I don’t know which technical paths need to be understood and mastered.

I used the IOT SDK RTSA Lite this time. First, I downloaded the RTSA source code from GitHub. Because it was only for verification, I used the Ubuntu virtual machine to compile and run. The compiled device, whether it is a virtual machine or a physical device, should be connected to the Internet. There are many audio and video codes or libraries that need to be downloaded in the compilation. Otherwise, you need to manually download and upload them for deployment.


After downloading, enter the RTSA Lite directory to see the directory distribution and cross_ compile.zh.md This is the description of cross compilation. If you transplant it to raspberry pie, please refer to this; Agora_ The so dynamic library is placed in the SDK to provide the API interface for developers_ Party is the parsing library of HTTP, JSON and media, which should be used for signaling negotiation;

[technology sharing] Introduction to RTC and Agora RTC SDK integration

Finally, the example is compiled in the directory. The compilation is very fast, and it is finished in seconds
[technology sharing] Introduction to RTC and Agora RTC SDK integration

Before executing the example, you need to activate the license. You need to create an account and project on Agora’s official website. You can directly go to the official website to operate. The only thing is that when you apply for a license, you’d better add Agora’s R & D communication group. Problems you encounter can be solved quickly in it. This is not an advertisement, because I encountered binding P when I applied for the license Project. according to README.md Step modification license.cfg , activate, and you’re ready to run. However, the example program only sends video and audio. Of course, the API provides an API for receiving operations. We can use Agora_ rtc_ You can find it in API. H (as shown in the figure below). You can write a copy of the example, but there is no code attached here.
[technology sharing] Introduction to RTC and Agora RTC SDK integration

In the lecture, I also asked that the SDK also provides a data channel, which is very useful in the embedded end. In addition to real-time communication, it may be necessary to transfer some instructions or files or pictures to each other, which can be transmitted through the data channel. I have seen Ericsson’s use case in Pakistan exhibition, and used the datachannel of webrtc to remotely control the excavator Row mining job, will try to use the data channel to complete some applications, and then share after completion.

In the end, Agora’s SDK is relatively simple and easy to integrate. With the development of demand and technology, I personally think that real-time communication will appear as a basic ability in various application scenarios, such as online games, voice customer service, robot control, video monitoring, office meetings, and so on, not to mention, this year Google’s webrtc has opened a lot of API capabilities so that it can work with AR Thanks to the technology development in this area, there is still a lot to be done. In fact, with regard to real-time communication, in addition to simple and easy integration at the end side, the communication quality also needs to be guaranteed. It is learned that the sound network has deployed servers that want to avoid scale, and uses sd-rtn Technology (I was lucky to have done technical analysis and research for a short time before) to optimize the transmission path to ensure the communication quality. In the future, if you just make a phone call, you may use free wechat calls. However, if you need to add real-time communication capabilities to your own applications, you still need to understand the principle and development of RTC technology.

This article is a personal original, first published in the voice network developer community[https://rtcdeveloper.com/t/to…]