Event driven microservice event driven design

Time:2020-11-19

This article is the second in the “event driven microservices” series, focusing on event driven design. If you want to understand the overall design, please see the first “event driven microservice – overall design”

Procedure flow

Let’s take a concrete example to illustrate event driven design. The program in this paper has two microservices, one is order service, the other is payment service. The user calls the use case createorder() of the order service to create an order. The order service does not have payment information for the time being. The order service then issues a command to the payment service. The payment service completes the payment and sends the payment created message. When the order service receives an event, add payment to the order table_ ID and modify the order status to paid.
The following is the component diagram:
Event driven microservice event driven design

event processing

Events are divided into internal events and external events. Internal events exist within a microservice and are not shared with other microservices. If it is described in the language of DDD, it is a domain event within the bounded context. External events are events published from one microservice and received by other microservices. If there is a context boundary between the context (d), it is used to describe the event. The two events are handled differently.

Internal events:

There are already mature methods for handling internal events. Its basic idea is to create an event bus to listen to events. Then register a different event handler to handle the event. This idea is widely used in various fields.

The following is the interface of the event bus, which has two functions: publish event and add event handler. An event can have one or more event handlers.

type EventBus interface {
    PublishEvent(EventMessage)
    AddHandler(EventHandler, ...interface{})
}

The key part of the code for the event bus is to load the event handler. Let’s take the order service as an example. Here is the code to load the event handler, which is part of the initialization container code. In this code, it only registers one event, payment create event, and its corresponding event handler, payment created event handler.

func loadEventHandler(c servicecontainer.ServiceContainer) error {
    var value interface{}
    var found bool

    rluf, err := containerhelper.BuildModifyOrderUseCase(&c)
    if err != nil {
        return err
    }
    pceh := event.PaymentCreatedEventHandler{rluf}
    if value, found = c.Get(container.EVENT_BUS); !found {
        message := "can't find key=" + container.EVENT_BUS + " in container "
        return errors.New(message)
    }
    eb := value.(ycq.EventBus)
    eb.AddHandler(pceh,&event.PaymentCreatedEvent{})
    return nil
}

Since the corresponding use cases are called when handling events, the use cases need to be injected into the event handler. In the previous code, first get the use case from the container, then create the event handler, and finally add the event and its corresponding processor to the event bus.

The event is published by calling publishevent() of the event bus. The following example is to listen to the payment created event from outside through message middleware in order service. After receiving it, it is converted into an internal event and sent to the event bus, so that the registered event processor can handle it.

eb := value.(ycq.EventBus)
    subject := config.SUBJECT_PAYMENT_CREATED
    _, err := ms.Subscribe(subject, func(pce event.PaymentCreatedEvent) {
        cpm := pce.NewPaymentCreatedDescriptor()
        logger.Log.Debug("payload:",pce)
        eb.PublishEvent(cpm)
    })

So how is the event handled? The key is the publishevent function. When an event is published, the event bus will call all handle() functions registered with the event handler in turn. The following is the code of publishevent(). In this way, each event handler only needs to implement the handle() function.

func (b *InternalEventBus) PublishEvent(event EventMessage) {
    if handlers, ok := b.eventHandlers[event.EventType()]; ok {
        for handler := range handlers {
            handler.Handle(event)
        }
    }
}

Here is the code for the paymentcreated EventHandler. Its logic is relatively simple, that is, getting the required payment information from Event, and then calling the corresponding use cases to complete the UpdatePayment () function.

type PaymentCreatedEventHandler struct {
    Mouc usecase.ModifyOrderUseCaseInterface
}
func(pc PaymentCreatedEventHandler) Handle (message ycq.EventMessage) {
    switch event := message.Event().(type) {

    case *PaymentCreatedEvent:
        status := model.ORDER_STATUS_PAID
        err := pc.Mouc.UpdatePayment(event.OrderNumber, event.Id,status)
        if err != nil {
            logger.Log.Errorf("error in PaymentCreatedEventHandler:", err)
        }
    default:
        logger.Log.Errorf("event type mismatch in PaymentCreatedEventHandler:")
    }
}

I used a third-party library here called jetbasrawi/ go.cqrs “To handle the eventbus. Jetbasrawi is a library of event sourcing. Event traceability and event driven are easy to confuse. They look a bit like each other, but they are actually two completely different things. Event driven is a call mode between microservices, which exists between microservices, corresponding to the call mode of RPC. Event traceability is a programming mode, which you can use or not use within microservices. But I couldn’t find an event driven library for a moment, so I found an event traceability library to use. In fact, it’s easy to write one myself, but I don’t think it’s better than jetbasrawi, so I’d better use it first. However, event traceability is more complex than event driven, so jetbasrawi may be a bit overkill.

External events:

The difference between external events is that they are transmitted between microservices, so message middleware is needed. I have defined a common interface to support different message middleware. Its two most important functions are publish() and subscribe().

package gmessaging

type MessagingInterface interface {
    Publish(subject string, i interface{}) error
    Subscribe(subject string, cb interface{} ) (interface{}, error)
    Flush() error
    // Close will close the decorated connection (For example, it could be the coded connection)
    Close()
    // CloseConnection will close the connection to the messaging server. If the connection is not decorated, then it is
    // the same with Close(), otherwise, it is different
    CloseConnection()
}

As a result of the definition of a common interface, it can support a variety of message oriented middleware. Here I choose “NATs” message middleware. It was chosen because it was a project of the cloud native Computing Foundation (CNCF), and it was powerful and fast. If you want to understand the concept of cloud nativity, please refer to “different interpretations and correct meanings of cloud Nativity”

The following code is the implementation of Nats. If you want to use other message middleware, you can refer to the following code.

type Nat struct {
    Ec *nats.EncodedConn
}

func (n Nat) Publish(subject string, i interface{}) error {
    return n.Ec.Publish(subject,i)
}

func (n Nat) Subscribe(subject string, i interface{} ) (interface{}, error) {
    h := i.(nats.Handler)
    subscription, err :=n.Ec.Subscribe(subject, h)
    return subscription, err
}

func  (n Nat) Flush() error {
    return n.Ec.Flush()
}

func  (n Nat) Close()  {
    n.Ec.Close()
}

func  (n Nat) CloseConnection()  {
    n.Ec.Conn.Close()
}

“Publish (subject string, I interface {})” has two parameters, and “subject” is the queue or topic of message middleware. The second parameter is the message to send, which is usually in JSON format. When using message oriented middleware, a connection is needed. Here we use the “connection” as the key* nats.EncodedConn ”It is an encapsulated link, which contains a JSON decoder, which can support the conversion between structure and JSON. When you call the publish function, the structure is sent, and the decoder automatically converts it into JSON text and sends it out. “Subscribe (subject string, I interface {})” also has two parameters. The first is the same as that of publish(), and the second is the event driver. When the JSON text is received, the decoder automatically converts it into a structure (struct) and then invokes the event handler.

I have written the code related to message middleware into a separate third-party library, so that you can use this library whether you use the framework or not. For details, see “jfeng45 / gmessaging”

command

Commands are very similar to events in code implementation, but they are quite different in concept. For example, a make payment is an order. You ask a third party (payment service) to do something, and you know who the third party is. Payment created is an event in which you are reporting that something has been done, and other third-party programs may decide whether to take the next action according to its results. For example, when the order service receives the payment completion event, it can change its order status to paid. Here, the sender of the event does not know who is interested in the message, so the transmission is broadcast. And the action (payment) has been completed, but the command is not yet completed, so the receiver can choose to refuse to execute a command. We usually talk about the event driven loose coupling, while RPC is tightly coupled, which refers to the event mode rather than the command mode. Command mode is tightly coupled because you already know who to send it to.

In practical applications, most of the commands we see are used within a microservice, and few commands are sent between microservices. Events are mainly passed between microservices. However, due to the confusion between events and commands, many “events” passed between microservices are actually “commands”. So instead of using event driven methods to make programs loosely coupled, you need to further check whether you misused “command” as “event.”. They are strictly distinguished in this procedure.

As like as two peas, the following is the interface of the command bus (Dispatcher).

type Dispatcher interface {
    Dispatch(CommandMessage) error
    RegisterHandler(CommandHandler, ...interface{}) error
}

We can define it as the following. Is it very similar to the event bus? The interface below is equivalent to the one above.

type CommandBus interface {
    PublishCommand(CommandMessage) error
    AddHandler(CommandHandler, ...interface{}) error
}

As like as two peas, the other way of defining events and commands, such as definition, processing, implementation and transmission, is almost identical. I won’t talk about the details. You can compare the code by yourself. Can we use only one event bus to process time and commands at the same time? In theory, there is no problem. That’s what I thought at the beginning, but because of the current interface (“jetbasrawi/ go.cqrs “) is not supported. If you want to change it, you need to redefine the interface, so it is temporarily abandoned. In addition, they are very different in concept, so it is necessary to define different interfaces in implementation.

Event and Command Design

Let’s talk about the problems that should be paid attention to when designing event driven.

Structural design

The addition of event driven mode compared with RPC is event and command. Therefore, the first thing to consider is what and how to extend the RPC program structure. “Event” and “command” are essentially part of the business logic, so they should belong to the domain layer. Therefore, in the program structure, two directories “event” and “command” are added to store events and commands respectively. The structure is shown in the figure below.
Event driven microservice event driven design

Different processing methods of sending and receiving

Now when dealing with external events, the way of sending and receiving is different.

The following is the code of the sender (the code is in the payment service project). The whole code function is to create payment, and then publish the “payment completed” message. It sends events directly through message middleware interface.

type MakePaymentUseCase struct {
    PaymentDataInterface dataservice.PaymentDataInterface
    Mi                   gmessaging.MessagingInterface
}
func (mpu *MakePaymentUseCase) MakePayment(payment *model.Payment) (*model.Payment, error) {
    payment, err := mpu.PaymentDataInterface.Insert(payment)
    if err!= nil {
        return nil, errors.Wrap(err, "")
    }
    pce := event.NewPaymentCreatedEvent(*payment)
    err = mpu.Mi.Publish(config.SUBJECT_PAYMENT_CREATED, pce)
    if err != nil {
        return nil, err
    }
    return payment, nil
}

The following is a code example of the receiver. It first receives the time with the message interface, then transforms the external events into internal events, and then calls the event bus interface to issue events inside the micro service.

eb := value.(ycq.EventBus)
    subject := config.SUBJECT_PAYMENT_CREATED
    _, err := ms.Subscribe(subject, func(pce event.PaymentCreatedEvent) {
        cpm := pce.NewPaymentCreatedDescriptor()
        logger.Log.Debug("payload:",pce)
        eb.PublishEvent(cpm)
    })

Why is there such a difference? When receiving, can we directly call use cases to handle external events instead of generating internal events? When sending, if there is no other internal event handler, calling message middleware directly to send is the simplest method (this sending process is lightweight and takes a short time). While receiving may need to deal with more complex business logic. Therefore, you want to divide this process into two parts: receiving and processing, so that the complex business logic can be processed in another process, so as to shorten the receiving time and improve the receiving efficiency.

Do you want a separate event handler for each event?

The current design is to have a separate file for each event and event drive. I’ve seen some people store all payment related events in a single file, such as paymentevent. The same is true for the event driver. Both methods are feasible. Now it seems clear to generate separate files, but if there are many events in the future, it may be easier to manage multiple events in one file. However, it will not be too late to change them.

Where is the event processing logic?

For a good design, all business logic should be concentrated together, so that it is easy to manage. In the current architecture, business logic is placed in use cases, but business logic is also needed in event processors. What should be done? The main function of the payment event processor is to modify the payment information in the order. The business logic of this part has been embodied in the modify order use case. Therefore, the payment event processor only needs to call the makepayment() function of the modified order. In fact, the logic of a business processor should not be encapsulated in a simple use case. Can we define the handle() function directly in the use case, so that the use case becomes an event handler? This design is feasible, but I think it is more logical to make the event handler a separate file. Because you must have the function of modifying the order payment, but the event handler is only available in the event driven mode. They belong to two different levels. Only when they are placed separately can the hierarchy be clear.

Source program:

Complete source link:

  • “Order service”
  • “Payment service”

Indexes:

1 “event driven microservices – overall design”

2 “jetbasrawi/go.cqrs”

3 “CNCF”

4 “different interpretations and correct meanings of cloud origin”

5 “NATS”

6 “jfeng45/gmessaging”

Recommended Today

The use of springboot Ajax

Ajax overview What is Ajax? data Ajax application scenarios? project Commodity system. Evaluation system. Map system. ….. Ajax can only send and retrieve the necessary data to the server, and use JavaScript to process the response from the server on the client side. data But Ajax technology also has disadvantages, the biggest disadvantage is that […]