How to realize multi person real-time annotation when sharing screen?

Time:2021-4-21

How to realize multi person real-time annotation when sharing screen?

preface

With the development of real-time communication technology, online education, video conferencing, online finance and other real-time interactive applications are used more and more. It is increasingly common to share documents and desktop online for teaching, collaboration and product demonstration. Diversified scenarios also give rise to more personalized demand for real-time interactive technology, including real-time annotation of shared content on the screen.

Pano launchedThe industry’s first real-time annotation product based on real-time video, screen sharing and document content streamTo solve the real-time interaction and real-time marking pain points in complex online interaction scenarios of multiple industries, and better improve the efficiency and effect of online interaction.

At present, pano SDK has launched real-time annotation API, which provides convenient access for developers and can quickly launch annotation function to meet business needs. This paper takes the Windows C + + interface as an example to see how to quickly access the shared annotation function of pano SDK.

preparation

technological process

The shared annotation API is used to annotate on the shared window and synchronize the annotation content at each end. The tagging function needs to ensure that the onwhiteboard available event notification is received. After sharing is enabled, developers can specify the shared stream to be marked. The access of tagging function is divided into tagging initiator and tagging receiver.

Sponsor

1) Get shared annotation object

two ) Format annotation tools, etc

three ) Start tagging ( startAnnotation )

4) Modify the format and dimension of dimension tool

five ) End annotation ( stopAnnotation )

Receiver

one ) Receive shared annotation start event (onshare annotation start)

2) Get shared annotation object

3) Format annotation tools, etc

4) Open the annotation window (start annotation)

5) Modify the format and dimension of dimension tool

6) Receive shared annotation end event (onshare annotation stop)

7) Get shared annotation object

8) Close the annotation window (stopannotation)
How to realize multi person real-time annotation when sharing screen?

API introduction

1. Get shared annotation object

To use the shared annotation object, you need to get the annotation manager object first. The manager object can be obtained from the panortcengine. After obtaining the shared annotation object from the annotation manager, the user ID corresponding to the shared flow needs to be specified in the interface.

RtcEngine

/**

  • @~english
  • @brief get annotation manager.
  • @return
    • non-null: a pointer to annotation manager
    • others: Failure
  • @note DO NOT delete this pointer, RtcEngine can handle its life cycle.
  • @~chinese
  • @ Brief get annotation Manager
  • @return
    • Non null pointer: pointer to the annotation manager.
    • Null pointer : fail
  • @Note do not delete this pointer, rtcegine will maintain its life cycle.

*/

virtual AnnotationManager* getAnnotationManager() = 0;

AnnotationManager

/**

  • @~english
  • @brief get share annotation object.
  • @param userId User ID
  • @return
    • non-null: a pointer to the share annotation object
    • others: Failure
  • @note DO NOT delete this pointer, AnnotationManager can handle its le cycle.
  • @~chinese
  • @ Brief get shared annotation object
  • @Param userid user ID
  • @return
    • Non null pointer: pointer to the shared annotation object.
    • Null pointer: failed
  • @Note do not delete this pointer, the annotation manager maintains its lifecycle.

*/

virtual PanoAnnotation* getShareAnnotation(uint64_t userId) = 0;

2. Start and end shared annotation

After obtaining the shared annotation object, you can start / end the annotation, and notify each end of the shared annotation to start / end

a. Start annotation calls startannotation, and the interface needs to pass in a annotation view. Call stopannotation to end the shared annotation.

PanoAnnotation

/**

  • @~english
  • @brief Start annotation and set render window
  • @param window Platform specified window object, it should HWND on Windows platform.
  • @return
    • OK: Success
    • Others: Fail
  • @note
    • Please make sure that whiteboard service is available before start annotation
    • For share annotation at presenter, pass NULL for window parameter
  • @~chinese
  • @Brief opens the annotation and sets the display window
  • @The window object related to param window platform. In Windows platform, this parameter must be hWnd object.
  • @return
    • OK: success
    • Others: failed
  • @note
    • Before opening the label, you need to ensure that the whiteboard service is normal.
    • For the shared end of shared annotation, the window parameter is passed to null

*/

virtual QResult startAnnotation(void *window) = 0;

/**

  • @~english
  • @brief Stop the annotation.
  • @return
    • OK: Success
    • Others: Fail
  • @~chinese
  • @ Brief stop tagging
  • @return
    • OK: success
    • Others: failed

*/

virtual QResult stopAnnotation() = 0;

The window object passed in by windows is of hWnd type, which is the same as the window object specified in subscribescreen. If it is a shared end, null is passed in. Pano will automatically create a transparent window consistent with the shared area.

b. In the callback of the annotation manager, there are events to inform the start and end of the shared annotation. After receiving the event, app can join the shared annotation or close the annotation view. After receiving onshare annotation start, you can get the corresponding annotation object through the userid in the callback. After that, call startAnnotation to add the annotation. After receiving onshare annotation stop, you can call stopannotation to close the annotation.

AnnotationManager::Callback

/*!

  • @~english
  • @brief Notification of share annotation start
  • @param userId User ID
  • @~chinese
  • @Brief start sharing annotation notification
  • @Param userid user ID

*/

virtual void onShareAnnotationStart(uint64_t userId) {}

/*!

  • @~english
  • @brief Notification of share annotation stop
  • @param userId User ID
  • @~chinese
  • @ Brief terminate shared annotation notification
  • @Param userid user ID

*/

virtual void onShareAnnotationStop(uint64_t userId) {}

3. Set annotation tools, formats, operations, etc

  • Annotation provides a variety of tools and operations. Set or call through the corresponding interface. Currently, the functions include brush, line, rectangle, ellipse, text, etc. Color, line width, bold italics, undo / redo, clear, etc. Fill type objects are not supported. Support user role settings (admin, attendee, viewer);
  • To clear all the marked contents, it needs the permission of admin role to clear the non self marked contents. Viewer roles can only be viewed, not annotated;
  • By setting the viewer role, you can turn off the annotation drawing function of a user. Set to attendee or admin to restart the annotation drawing function;
  • The snapshot function can quickly get the screenshot of the current annotation and shared content.

PanoAnnotation

/**

  • @~english
  • @brief Set annotation role type.
  • @param type The annotation role type.
  • @~chinese
  • @Brief sets the annotation role type.
  • @Param type annotates roles.

*/

virtual QResult setRoleType(WBRoleType type) = 0;

/**

  • @~english
  • @brief set tool type
  • @param type tool type
  • @return
    • OK: Success
    • Others: Fail
  • @~chinese
  • @ Brief setting tool type
  • @Para type tool type
  • @return
    • OK: success
    • Others : fail

*/

virtual QResult setToolType(WBToolType type) = 0;

/**

  • @~english
  • @brief set line width
  • @param size line width. Valid value range [1, 20]
  • @return
    • OK: Success
    • Others: Fail
  • @~chinese
  • @ Brief setting line width
  • @ Para size. Valid value range [ one , twenty ]
  • @return
    • OK: success
    • Others : fail

*/

virtual QResult setLineWidth(int size) = 0;

/**

  • @~english
  • @brief Set color
  • @param color Color
  • @return
    • OK: Success
    • Others: Fail
  • @~chinese
  • @Brief set color
  • @Para color
  • @return
    • OK: success
    • Others: failed

*/

virtual QResult setColor(WBColor color) = 0;

/**

  • @~english
  • @brief Set font style
  • @param style font style
  • @return
    • OK: Success
    • Others: Fail
  • @~chinese
  • @ Brief set font style
  • @Para style font style
  • @return
    • OK: success
    • Others: failed

*/

virtual QResult setFontStyle(WBFontStyle style) = 0;

/**

  • @~english
  • @brief Set font size
  • @param size font size
  • @return
    • OK: Success
    • Others: Fail
  • @~chinese
  • @Brief set font size
  • @Font size
  • @return
    • OK: success
    • Others: failed

*/

virtual QResult setFontSize(int size) = 0;

/**

  • @~english
  • @brief undo
  • @return
    • OK: Success
    • Others: Fail
  • @~chinese
  • @Brief undoes the last action
  • @return
    • OK: success
    • Others: failed

*/

virtual QResult undo() = 0;

/**

  • @~english
  • @brief redo
  • @return
    • OK: Success
    • Others: Fail
  • @~chinese
  • @Brief redoes the last undone operation of the annotation
  • @return
    • OK: success
    • Others: failed

*/

virtual QResult redo() = 0;

/**

  • @~english
  • @brief clear annotation content by specific user ID
  • @param userId user ID
  • @return
    • OK: Success
    • NO_PRIVILEGE: need ADMIN role to call this API
    • Others: Fail
  • @note ADMIN role is required if the userId is not local user
  • @~chinese
  • @Brief clear the specified user annotation content
  • @Param userid user ID
  • @return
    • OK: success
    • NO_ Privilege: no permission
    • Others: failed
  • @ Note only the admin role can clear the contents of non local users

*/

virtual QResult clearUserContents(uint64_t userId) = 0;

/**

  • @~english
  • @brief clear annotation content
  • @return
    • OK: Success
    • NO_PRIVILEGE: need ADMIN role to call this API
    • Others: Fail
  • @note this API need ADMIN role
  • @~chinese
  • @Brief clear annotation content, need the admin role to call successfully
  • @return
    • OK: success
    • NO_ PRIVILEGE : No permission
    • Others: failed
  • @Note this interface can only be called by the admin role

*/

virtual QResult clearContents() = 0;

/**

  • @~english
  • @brief save annotation contents to image.
  • @param outputDir output directory
  • @return
    • OK: Success
    • Others: Fail
  • @note snapshot result and image filename is returned in callback onSnapshotComplete
  • @~chinese
  • @Brief saves the annotation content to the image.
  • @Param outputdir output path
  • @return
    • OK: success
    • Others: failed
  • @The note snapshot result and image file name are returned through the callback function onsnapshotcomplete

*/

virtual QResult snapshot(const char *outputDir) = 0;

4. Callback event

The callback event of annotation object is registered through setcallback. The current callback event is annotation role change and snapshot completion notification.

PanoAnnotation::Callback

/**

  • @~english
  • @brief Notification of annotation role type changed
  • @param newRole The new role type.
  • @~chinese
  • @ Brief annotation role type change notification
  • @Param new role.

*/

virtual void onAnnoRoleChanged(WBRoleType newRole) {}

/**

  • @~english
  • @brief Notification of annotation snapshot complete
  • @param result Snapshot result
  • @param filename Snapshot image full path with name.
  • @~chinese
  • @Brief annotation snapshot completion notification
  • @Param result snapshot result
  • @ Param file name snapshot file name

*/

virtual void onSnapshotComplete(QResult result, const char *filename) {}

The above is the detailed tutorial of windows real-time sharing annotation. If you need to access other terminals, you can also contact the official website customer service to obtain technical documents_Follow us, we will share more about the exploration and practice of audio and video technology, as well as the detailed development tutorial based on pano SDK._

About paileyun

Established in 2019, paileyun is the first real-time interactive communication cloud service provider with video conference background in China, bringing together a large number of senior technical experts focusing on audio, video, whiteboard, network, AI and other fields. Through the pano SDK, enterprise developers can quickly realize interactive small class, super small class, double division large class, voice chat room, video social networking, live broadcast Lianmai, game voice, video customer service, telemedicine, office cooperation and other scenes around the world.