Full record of. Net core microservices Introduction (6) — eventbus event bus

Time:2020-11-26

Tips: This article has been added to the reading list of series articles. You can click to see more related articles.

preface

The construction of Ocelot + consult has been completed in the previous article (5) – Ocelot API gateway (2) of. Net core microservices introduction. This article briefly introduces the eventbus.

Eventbus – event bus

  • First of all, what is an event bus?

Post a quote:

The event bus is an implementation of the observer (publish subscribe) pattern. It is a centralized event processing mechanism, which allows different components to communicate with each other without relying on each other to achieve the purpose of decoupling.

If you have not been exposed to eventbus, it may not be easy to understand. In fact, eventbus is widely used in client development (Android, IOS, web front-end, etc.), which is used for communication between multiple components (or interfaces). Anyone who understands it will understand…

  • So why do we use eventbus?

Take the current project as an example. We have an order service and a product service. The client has an order function. When the user places an order, it calls the order order interface of the order service. Then the order interface needs to call the inventory reduction interface of product service, which involves the call between services. How to call between services? Direct restapi? Or more efficient grpc? Maybe these two have their own usage scenarios, but they both have a problem of coupling between services, or it is difficult to achieve asynchronous call.

Let’s imagine: suppose we call the order service when we place an order. The order service needs to call the product service, the product service also calls the logistics service, and the logistics service calls the XX service… If the processing time of each service needs 2 seconds, and asynchronous is not used, the experience can be imagined.

If you use eventbus, the order service only needs to send an “order event” to the eventbus. The product service will subscribe to the “order event”. When the product service receives the order event, it is good to reduce the inventory by itself. In this way, the coupling of direct call between two services is avoided, and asynchronous call is realized.

Since asynchronous calls between multiple services are involved, distributed transactions have to be mentioned. Distributed transaction is not a unique problem of microservices, but a problem of all distributed systems.
For distributed transactions, check out cap principles and base theory to learn more. Today’s distributed systems are more likely to pursue the ultimate consistency of transactions.

The following uses the excellent project “cap” developed by Chinese people to demonstrate the basic use of eventbus. The reason why “cap” is used is that it can not only solve the final consistency of distributed system, but also is an eventbus, which has all the functions of eventbus!
The author introduces: https://www.cnblogs.com/savorboard/p/cap.html

Use of cap

  • Environmental preparation

Prepare the required environment in docker. First of all, the database. I use PostgreSQL for the database, and other things are OK. Cap support: sqlserver, mysql, PostgreSQL, mongodb.
About running PostgreSQL in docker, you can see my other blog: https://www.cnblogs.com/xhznl/p/13155054.html

Then MQ, here I use rabbitmq, Kafka can also.
Docker runs rabbitmq:

docker pull rabbitmq:management
docker run -d -p 15672:15672 -p 5672:5672 --name rabbitmq rabbitmq:management

Default user: guest, password: Guest

The environment is ready. Docker is so convenient…

  • Code modification:

In order to simulate the above business, you need to modify a lot of code. If the following code is missing, go to GitHub directly.

Nuget installation:

Microsoft.EntityFrameworkCore
Microsoft.EntityFrameworkCore.Tools
Npgsql.EntityFrameworkCore.PostgreSQL

Cap related:

DotNetCore.CAP
DotNetCore.CAP.RabbitMQ
DotNetCore.CAP.PostgreSql

Order.API/Controllers/OrdersController . CS add single interface:

[Route("[controller]")]
[ApiController]
public class OrdersController : ControllerBase
{
    private readonly ILogger _logger;
    private readonly IConfiguration _configuration;
    private readonly ICapPublisher _capBus;
    private readonly OrderContext _context;

    public OrdersController(ILogger logger, IConfiguration configuration, ICapPublisher capPublisher, OrderContext context)
    {
        _logger = logger;
        _configuration = configuration;
        _capBus = capPublisher;
        _context = context;
    }

    [HttpGet]
    public IActionResult Get()
    {
        String result = $"[order service]{ DateTime.Now.ToString ("yyyy-MM-dd HH:mm:ss")}——" +
            $"{Request.HttpContext.Connection.LocalIpAddress}:{_configuration["ConsulSetting:ServicePort"]}";
        return Ok(result);
    }

    /// 
    ///Order release order event
    /// 
    /// 
    /// 
    [Route("Create")]
    [HttpPost]
    public async Task CreateOrder(Models.Order order)
    {
        using (var trans = _context.Database.BeginTransaction(_capBus, autoCommit: true))
        {
            //Business code
            order.CreateTime = DateTime.Now;
            _context.Orders.Add(order);

            var r = await _context.SaveChangesAsync() > 0;

            if (r)
            {
                //Issue an order
                await _capBus.PublishAsync("order.services.createorder", new CreateOrderMessageDto() { Count = order.Count, ProductID = order.ProductID });
                return Ok();
            }
            return BadRequest();
        }

    }

}

Order.API/MessageDto/CreateOrderMessageDto.cs:

/// 
///Single event message
/// 
public class CreateOrderMessageDto
{
    /// 
    ///Product ID
    /// 
    public int ProductID { get; set; }

    /// 
    ///Purchase quantity
    /// 
    public int Count { get; set; }
}

Order.API/Models/Order . CS Order entity class:

public class Order
{
    [Key]
    [DatabaseGenerated(DatabaseGeneratedOption.Identity)]
    public int ID { get; set; }

    /// 
    ///Order time
    /// 
    [Required]
    public DateTime CreateTime { get; set; }

    /// 
    ///Product ID
    /// 
    [Required]
    public int ProductID { get; set; }

    /// 
    ///Purchase quantity
    /// 
    [Required]
    public int Count { get; set; }
}

Order.API/Models/OrderContext . CS database context:

public class OrderContext : DbContext
{
    public OrderContext(DbContextOptions options)
       : base(options)
    {

    }

    public DbSet Orders { get; set; }

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {

    }
}

Order.API/appsettings . JSON adds database connection string:

"ConnectionStrings": {
  "OrderContext": "User ID=postgres;Password=pg123456;Host=host.docker.internal;Port=5432;Database=Order;Pooling=true;"
}

Order.API/Startup . CS modify the configureservices method and add cap configuration:

public void ConfigureServices(IServiceCollection services)
{
    services.AddControllers();

    services.AddDbContext(opt => opt.UseNpgsql(Configuration.GetConnectionString("OrderContext")));

    //CAP
    services.AddCap(x =>
    {
        x.UseEntityFramework();

        x.UseRabbitMQ("host.docker.internal");
    });
}


The above is the modification of order service.

Product.API/Controllers/ProductsController . CS add inventory reduction interface:

[Route("[controller]")]
[ApiController]
public class ProductsController : ControllerBase
{
    private readonly ILogger _logger;
    private readonly IConfiguration _configuration;
    private readonly ICapPublisher _capBus;
    private readonly ProductContext _context;

    public ProductsController(ILogger logger, IConfiguration configuration, ICapPublisher capPublisher, ProductContext context)
    {
        _logger = logger;
        _configuration = configuration;
        _capBus = capPublisher;
        _context = context;
    }

    [HttpGet]
    public IActionResult Get()
    {
        String result = $"[product service]{ DateTime.Now.ToString ("yyyy-MM-dd HH:mm:ss")}——" +
            $"{Request.HttpContext.Connection.LocalIpAddress}:{_configuration["ConsulSetting:ServicePort"]}";
        return Ok(result);
    }

    /// 
    ///Inventory reduction subscription order event
    /// 
    /// 
    /// 
    [NonAction]
    [CapSubscribe("order.services.createorder")]
    public async Task ReduceStock(CreateOrderMessageDto message)
    {
        //Business code
        var product = await _context.Products.FirstOrDefaultAsync(p => p.ID == message.ProductID);
        product.Stock -= message.Count;

        await _context.SaveChangesAsync();
    }

}

Product.API/MessageDto/CreateOrderMessageDto.cs:

/// 
///Single event message
/// 
public class CreateOrderMessageDto
{
    /// 
    ///Product ID
    /// 
    public int ProductID { get; set; }

    /// 
    ///Purchase quantity
    /// 
    public int Count { get; set; }
}

Product.API/Models/Product . CS product entity class:

public class Product
{
    [Key]
    [DatabaseGenerated(DatabaseGeneratedOption.Identity)]
    public int ID { get; set; }

    /// 
    ///Product name
    /// 
    [Required]
    [Column(TypeName = "VARCHAR(16)")]
    public string Name { get; set; }

    /// 
    ///Inventory
    /// 
    [Required]
    public int Stock { get; set; }
}

Product.API/Models/ProductContext . CS database context:

public class ProductContext : DbContext
{
    public ProductContext(DbContextOptions options)
       : base(options)
    {

    }

    public DbSet Products { get; set; }

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        base.OnModelCreating(modelBuilder);
        
        //Initialize seed data
        modelBuilder.Entity().HasData(new Product
        {
            ID = 1,
            Name = product 1,
            Stock = 100
        },
        new Product
        {
            ID = 2,
            Name = product 2,
            Stock = 100
        });
    }
}

Product.API/appsettings . JSON adds database connection string:

"ConnectionStrings": {
  "ProductContext": "User ID=postgres;Password=pg123456;Host=host.docker.internal;Port=5432;Database=Product;Pooling=true;"
}

Product.API/Startup . CS modify the configureservices method and add cap configuration:

public void ConfigureServices(IServiceCollection services)
{
    services.AddControllers();

    services.AddDbContext(opt => opt.UseNpgsql(Configuration.GetConnectionString("ProductContext")));

    //CAP
    services.AddCap(x =>
    {
        x.UseEntityFramework();

        x.UseRabbitMQ("host.docker.internal");
    });
}


The above is the modification of product service.

The modification of order service and product service is completed here. There are many modifications, but the function is very simple. That is, they add their own database tables, and then the order service adds an order interface, and the order interface will send out an “order event”. The product service has added the inventory reduction interface, which will subscribe to the “order event”. Then when the client calls the order interface to place an order, the product service will subtract the corresponding inventory, and the function is so simple.

The basic use of EF database migration will not be introduced. Use docker to rebuild the image, run the order service, product service:

docker build -t orderapi:1.1 -f ./Order.API/Dockerfile .
docker run -d -p 9060:80 --name orderservice orderapi:1.1 --ConsulSetting:ServicePort="9060"
docker run -d -p 9061:80 --name orderservice1 orderapi:1.1 --ConsulSetting:ServicePort="9061"
docker run -d -p 9062:80 --name orderservice2 orderapi:1.1 --ConsulSetting:ServicePort="9062"

docker build -t productapi:1.1 -f ./Product.API/Dockerfile .
docker run -d -p 9050:80 --name productservice productapi:1.1 --ConsulSetting:ServicePort="9050"
docker run -d -p 9051:80 --name productservice1 productapi:1.1 --ConsulSetting:ServicePort="9051"
docker run -d -p 9052:80 --name productservice2 productapi:1.1 --ConsulSetting:ServicePort="9052"

last Ocelot.APIGateway/ocelot . JSON adds a routing configuration:

OK, so far, the whole environment is a little complicated. Ensure that our PostgreSQL, rabbitmq, consult, gateway and service instances are running normally.

After the service instance runs successfully, the database should look like this:




Product table seed data:

cap.published Table and cap.received The table is automatically generated by cap, which uses local message table + MQ to realize asynchronous guarantee.

Run the test

This time, postman is used as the client to call the single interface (9070 is the former Ocelot gateway port)

Order library published table:

Order table:

Product library received table:

Product library product table:

Try again:

OK, done. Although the function is very simple, we realize the decoupling of services, asynchronous invocation, and final consistency.

summary

Note that the above example is purely to illustrate the use of eventbus, and the actual order process will never do so! I hope you don’t take it seriously…

Some people may say that if the order is successful, but the inventory reduction fails due to insufficient inventory, what to do? Do you want to roll back the data in the order table? If this idea arises, it shows that there is no real understanding of the idea of ultimate consistency. First of all, we will check the inventory quantity before placing an order. Since we are allowed to place an order, we must have sufficient inventory. The transaction here refers to: the order is saved to the database, and the order event is saved to the database cap.published Table (save to cap.published Tables can theoretically be sent to MQ) both of which either succeed or fail together. If the transaction is successful, then the business process can be considered successful. As to whether the inventory reduction of product service is successful, it is a matter of product service (in theory, it should also be successful, because the message has been guaranteed to be sent to MQ, and the product service must receive the message). Cap also provides failure retrial and failure callback mechanism.

If you have to roll back the data, you can do it ICapPublisher.Publish Method provides a callbackname parameter, which can be triggered when inventory is reduced. Its essence is also completed by publishing and subscribing. This is not recommended. I won’t go into details. I’m interested in studying it myself.
In addition, cap can’t guarantee that messages are not repeated. In actual use, we need to consider the repeated filtering and idempotency of messages.

This article has a lot of content. I don’t know if it has been expressed clearly. If you have any questions, please comment and exchange. If there are any mistakes, please point out.

The next article plans to write about authorization and certification.

Code in: https://github.com/xiajingren/NetCoreMicroserviceDemo

To be continued

Recommended Today

PHP 12th week function learning record

sha1() effect sha1()Function to evaluate the value of a stringSHA-1Hash. usage sha1(string,raw) case <?php $str = “Hello”; echo sha1($str); ?> result f7ff9e8b7bb2e09b70935a5d785e0cc5d9d0abf0 sha1_file() effect sha1_file()Function calculation fileSHA-1Hash. usage sha1_file(file,raw) case <?php $filename = “test.txt”; $sha1file = sha1_file($filename); echo $sha1file; ?> result aaf4c61ddcc5e8a2dabede0f3b482cd9aea9434d similar_text() effect similar_text()Function to calculate the similarity between two strings. usage similar_text(string1,string2,percent) case […]