Eventbridge integrated cloud service practice

Time:2022-5-17

Author: Li Kai (Kaiyi)

Evenbridge integration overview

Eventbridge is a serverless event bus launched by Alibaba cloud. Its goal is to expand the event ecology, break the data island between systems and establish an event integration ecology. Provide unified event standardized access and management capabilities, improve the integration and integrated channels, help customers quickly realize the core atomic functions driven by events, and quickly integrate eventbridge into BPM, RPA, CRM and other systems.

Eventbridge expands the eventbridge event ecology through event standardization, access standardization and component Standardization:

  • Event standardization: embrace cloudevents 1.0 open source community standard protocol, support cloudevents community SDK and API natively, and fully embrace the open source community event standard ecology;
  • Access standardization: provide standard event push protocol putevent, and support two event access models: pull and push, which can effectively reduce the difficulty of event access and provide a perfect standardized process of event access on the cloud;
  • Component standardization: encapsulate the standard event downstream component tool chain system, including schema registration, event analysis, event retrieval, event dashboard, etc. Provide perfect event tool chain ecology.

In the field of integration, eventbridge focuses on creating two core scenarios: Event Integration and data integration, which will be described in detail below.

Event integration

At present, eventbridge has 80 + event sources of cloud products and 800 + event types. The whole event ecology is also gradually enriched.

So, how does eventbridge realize the event integration of cloud products?

  • First, you can see an event bus named default on the eventbridge console, and events of cloud products will be delivered to this bus;

  • Then click Create rule to select the concerned cloud product and its related events for event monitoring and delivery.

Let’s take two examples to see how eventbridge integrates events.

OSS event integration

Take OSS event source as an example to explain how to integrate OSS events.

OSS events are now mainly divided into four categories: those related to operation audit, cloud monitoring, configuration audit, and cloud products, such as putobject uploading files. The event sources of other cloud products are similar, which can be basically divided into these types of events.

The following is a demonstration of the event driven online file decompression service:

  • Under the OSS bucket, there will be a zip folder to store the files to be unzipped, and an unzip folder to store the unzipped files;
  • When the file is uploaded to the OSS bucket, the file upload event will be triggered and delivered to the cloud service private bus of eventbridge;
  • Then, an event rule will be used to filter the events of the zip bucket and deliver them to the HTTP endpoint of the decompression service;
  • After receiving the event, the decompression service will download and decompress the file from OSS according to the file path in the event, and transfer the file to the unzip directory after decompressing;
  • At the same time, there will be an event rule to listen to the file upload event in the unzip directory and push the event to the nail group after conversion.

Let’s take a look at how it is achieved:

Go to the link below to view the video:

​https://www.bilibili.com/video/BV1s44y1g7dk/​

1) First, create a bucket. There is a zip directory below to store the uploaded compressed files and an unzip directory to store the unzipped files.

  1. Deploy the decompression service and expose the address accessed by the public network.

The source code address of the decompression service is:

​https://github.com/AliyunContainerService/serverless-k8s-examples/tree/master/oss-unzip?spm=a2c6h.12873639.article-detail.15.5a585d52apSWbk​

You can also use ask to deploy directly. The yaml file address is:

​https://github.com/AliyunContainerService/serverless-k8s-examples/blob/master/oss-unzip/hack/oss-unzip.yaml​

3) Create an event rule to listen to the events of uploaded files in the zip directory and deliver them to the HTTP # endpoint of the decompression service.

Here, subject is used to match the zip directory.

4) Then create an event rule to listen to the events in the unzip directory and post the decompression events to the nail group.

Here, subject is also used to match the unzip directory.

For the configuration of variables and templates, please refer to the official documents:

https://help.aliyun.com/document_detail/181429.html​。

Eventbridge will extract parameters from the event through jsonpath, then put these values into variables, and finally render the final output through the definition of the template and deliver it to the event target. The event format of OSS event source can also refer to the official document:​​https://help.aliyun.com/document_detail/205739.html#section-g8i-7p9-xpk​​* *, and use jsonpath to define variables according to actual business needs. 5) Finally, upload a file to the OSS console for verification.

You can see the just uploaded eventbridge The zip has been unzipped and uploaded. You can also receive the notice of unzipped completion in the nail group. In addition, you can also view the delivered track of the content of the event on the event tracking side.

You can see that there are two upload events: one is the event uploaded through the console, and the other is the event uploaded after decompressing the file.

You can view the track, which has been successfully delivered to the HTTP endpoint of the decompression service and the nailing robot.

Integrate cloud products by customizing event sources and event targets of cloud products

The demo just demonstrated is the event source of integrating cloud services. Let’s take a demo to see how to integrate cloud products by customizing the event source and the event target of cloud products.

Go to the link below to view the video:

​https://www.bilibili.com/video/BV1QF411M7xv/​

The final effect of this demo is to automatically clean the data through eventbridge and deliver it to RDS. The event content is a JSON with two fields, one name and one age. Now we hope to filter out users older than 10 and store them in RDS.

The overall architecture is shown in the figure. An MNS queue is used as the custom event source, and the events are filtered and converted through the eventbridge, and finally directly output to the RDS.

1) First, you have created an MNS queue, an RDS instance and a database table. The table structure is as follows:

2) Create a customized event bus, select the event provider as MNS, and the queue is the queue created in advance;

After the creation, we can see a running event source in the event source;

3) Next, create rules and post them to RDS

The configured event mode is as follows:

{
    "source": [
        "my.user"
    ],
    "data": {
        "messageBody": {
            "age": [
                {
                    "numeric": [
                        ">",
                        10
                    ]
                }
            ]
        }
    }
}

For numerical matching, please refer to the official document:

https://help.aliyun.com/document_detail/181432.html#section-dgh-5cq-w6c

  1. Click next, select the event target as the database, fill in the database information, configure the conversion rules, and complete the creation.

5) Finally, first send a message with MNS queue. The age of this message is greater than 10.

You can see that this event is output to RDS.

Next, send another message less than 10 to MNS queue.

This event is filtered out and not output to RDS.

You can also view events through event tracking:

You can see that an event has been successfully delivered to RDS, and an event has been filtered and not delivered.

data Integration

Event flow is a more lightweight and real-time end-to-end event flow test channel provided by eventbridge for data integration. The main goal is to synchronize events between two endpoints, and provide filtering and conversion functions at the same time. Currently, Alibaba cloud supports event flow among its messaging products.

Different from the event bus model, the event bus is not required in the event flow. Its 1:1 model is lighter, and the way of directly reaching the target also makes the event more real-time; Through event flow, we can realize protocol conversion, data synchronization and cross regional backup between different systems.

The following will explain how to use event flow to route rocketmq messages to MNS queue and integrate the two products through an example.

The overall structure is shown in the figure. The message with MNS tag in rocketmq is routed to mnq queue through eventbridge.

Let’s see how to achieve it:

Go to the link below to view the video:

​https://www.bilibili.com/video/BV1D44y1G7GK/​

  • First, create an event flow, select the source rocketmq instance, and fill in the tag as MNS.

  • Leave the event pattern content blank to match all.

  • Select MNS as the target and select the target queue to complete the creation.

  • After creating, click start to start the event flow task.

After the event flow is started, we can send messages to the source rocketmq topic through the console or SDK. When a tag is MNS, we can see that the message is routed to MNS; When a tag is not MNS, the message will not be routed to MNS.

summary

This article mainly shares with you how to integrate cloud product event sources through eventbridge, how to integrate cloud product event targets, and how to integrate message products through event flow. For more information about eventbridge, scan the QR code below to join the nail group ~

ClickHere, learn more about eventbridge~

Release the latest information of cloud native technology, collect the most complete content of cloud native technology, regularly hold live streaming and live broadcasting of cloud native life, and release the best practices of Alibaba products and users. Explore cloud native technology with you and share the cloud native content you need.

Follow [Alibaba cloud native] official account to get more real-time information about cloud native!