How to add protobuf support for asp.net core

Time:2020-2-8

Preface

In some applications with high performance requirements, using protocol buffer serialization is better than JSON. Moreover, the backward compatibility of protocol buffer is better.

Because asp.net core adopts a new middleware mode, protobuf net is only used to decorate the objects to be serialized, and corresponding formatter is added during MVC initialization.

There’s no time to explain. Get in the car.

Get zaabee.aspnetcoreprotobuf through nuget


Install-Package Zaabee.AspNetCoreProtobuf

Modify the configureservices method in the startup.cs file


public void ConfigureServices(IServiceCollection services)
{
  services.AddMvc(options => { options.AddProtobufSupport(); });
}

Fix… At this time, you can use the content type of application / x-protobuf to let asp.net core use protobuf for serialization / deserialization.

Test code

Add the following DTOs to the asp.net core project


[ProtoContract]
public class TestDto
{
  [ProtoMember(1)] public Guid Id { get; set; }
  [ProtoMember(2)] public string Name { get; set; }
  [ProtoMember(3)] public DateTime CreateTime { get; set; }
  [ProtoMember(4)] public List<TestDto> Kids { get; set; }
  [ProtoMember(5)] public long Tag { get; set; }
  [ProtoMember(6)] public TestEnum Enum { get; set; }
}

public enum TestEnum
{
  Apple,
  Banana,
  Pear
}

Create a new xUnit project, reference microsoft.aspnetcore.testhost through nuget, and create a test class


public class AspNetCoreProtobufTest
{
  private readonly TestServer _server;
  private readonly HttpClient _client;

  public AspNetCoreProtobufTest()
  {
    _server = new TestServer(
      new WebHostBuilder()
        .UseKestrel()
        .UseStartup<Startup>());
    _client = _server.CreateClient();
  }

  [Fact]
  public void Test()
  {
    // HTTP Post with Protobuf Response Body
    _client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/x-protobuf"));

    var dtos = GetDtos();
    var stream = new MemoryStream();
    ProtoBuf.Serializer.Serialize(stream, dtos);

    HttpContent httpContent = new StreamContent(stream);

    // HTTP POST with Protobuf Request Body
    var responseForPost = _client.PostAsync("api/Values", httpContent);

    var result = ProtoBuf.Serializer.Deserialize<List<TestDto>>(
      responseForPost.Result.Content.ReadAsStreamAsync().Result);

    Assert.True(CompareDtos(dtos,result));
  }

  private static bool CompareDtos(List<TestDto> lstOne, List<TestDto> lstTwo)
  {
    lstOne = lstOne ?? new List<TestDto>();
    lstTwo = lstTwo ?? new List<TestDto>();

    if (lstOne.Count != lstTwo.Count) return false;

    for (var i = 0; i < lstOne.Count; i++)
    {
      var dtoOne = lstOne[i];
      var dtoTwo = lstTwo[i];
      if (dtoOne.Id != dtoTwo.Id || dtoOne.CreateTime != dtoTwo.CreateTime || dtoOne.Enum != dtoTwo.Enum ||
        dtoOne.Name != dtoTwo.Name || dtoOne.Tag != dtoTwo.Tag || !CompareDtos(dtoOne.Kids, dtoTwo.Kids))
        return false;
    }

    return true;
  }

  private static List<TestDto> GetDtos()
  {
    return new List<TestDto>
    {
      new TestDto
      {
        Id = Guid.NewGuid(),
        Tag = long.MaxValue,
        CreateTime = DateTime.Now,
        Name = "0",
        Enum = TestEnum.Apple,
        Kids = new List<TestDto>
        {
          new TestDto
          {
            Id = Guid.NewGuid(),
            Tag = long.MaxValue - 1,
            CreateTime = DateTime.Now,
            Name = "00",
            Enum = TestEnum.Banana
          },
          new TestDto
          {
            Id = Guid.NewGuid(),
            Tag = long.MaxValue - 2,
            CreateTime = DateTime.Now,
            Name = "01",
            Enum = TestEnum.Pear
          }
        }
      },
      new TestDto
      {
        Id = Guid.NewGuid(),
        Tag = long.MaxValue - 3,
        CreateTime = DateTime.Now,
        Name = "1",
        Enum = TestEnum.Apple,
        Kids = new List<TestDto>
        {
          new TestDto
          {
            Id = Guid.NewGuid(),
            Tag = long.MaxValue - 4,
            CreateTime = DateTime.Now,
            Name = "10",
            Enum = TestEnum.Banana
          },
          new TestDto
          {
            Id = Guid.NewGuid(),
            Tag = long.MaxValue - 5,
            CreateTime = DateTime.Now,
            Name = "11",
            Enum = TestEnum.Pear
          }
        }
      }
    };
  }
}

Why use protobuf?

Because fast In our test of using business data, protobuf’s serialization / deserialization performance is about three times that of json.net, and the volume after serialization is about one-half that of JSON, which can improve the throughput performance of webapi to a certain extent.

In addition, there is a loss of precision in JSON’s handling of floating-point numbers, because the number type security integer of JS is 53 bits. When we use the snowflake algorithm to provide the global incremental ID, the primary key will be repeated due to the loss of precision. In addition, due to the same reason, the delivery of datetime type will also cause time matching error due to the inconsistent milliseconds. The general solution is to use string passing, but after all, it’s a partial method that doesn’t solve the problem from the root, so we still use protobuf to deal with it directly.

Disadvantages of protobuf

The dto layer must refer to protobuf net to add features, which to some extent leads to code intrusion. Basically, dto belongs to poco. If you rely on the third-party package, you always feel a bit chaste In addition, the serialized data of protobuf does not have visualization, so if you use message queuing or request monitoring, you need to comprehensively consider whether protobuf is suitable for use scenarios.

principle

Asp.net core is based on middleware. It comes with the default JSON formater (based on JSON. Net). Asp.net core will select the corresponding formater according to content type to process object serialization, including inputformatter (deserialization) and outputformatter (serialization). Therefore, in addition to protobuf, we can add or replace other serialization methods, such as using Jil to replace json.net to improve the performance of JSON.

The source code of the above implementation and demo and test has been put on GitHub.

summary

The above is the whole content of this article. I hope that the content of this article has a certain reference learning value for everyone’s study or work. If you have any questions, you can leave a message and exchange. Thank you for your support for developepaar.