Im-cloud builds distributed push middleware based on swoole native collaboration

Time:2019-8-29

github:http://github.com/brewlin/im-…

  • Im-cloud builds distributed push middleware based on swoole native collaboration
  • Installation and deployment of im-cloud distributed middleware
  • IM-cloud <> GOIM distributed middleware concurrent pressure measurement comparison
  • Analysis of im-cloud distributed middleware (1) – Communication Protocol
  • Analysis of im-cloud distributed middleware (2) – Implementation of cloud node
  • Analysis of im-cloud distributed middleware (3) – Implementation of job node
  • IM-cloud Distributed Middleware Analysis (IV) – Logic Node Implementation

I. Overview

Be based onswooleNative consortium builds commercial instant push IM service middleware, which is used as middleware alone instead of industry transaction. It can flexibly expand nodes to increase performance processing. It does not depend on external framework, and its core code is components built by native swoole.

Learn from goim(Bilbili Produced Million Message Second Push with Production VerificationHigh-performance distributed im middleware implemented by pure swoole can improve high-performance concurrent delivery and achieve better performance

II. Service Business Node

cloud,job,logicHorizontal expansion can be achieved at all nodes.

  • For example, n can be started when the consumption capacity is insufficient.jobNode Enhancement and Consumption Ability
  • Start multiple cloud nodes as client client client load balancing, multiplewebsockettcpClient is distributed to multiple cloud nodes to improve the processing capability of cloud node Center
  • logicProvide external restapi as the main business node
  • High PerformanceCollaboration, Horizontal Expansion, Distributed Service Architecture, Access Service Governance

@cloud

cloudAs a central service nodegrpc-serverNode, external receiving TCP, Websocket clientLong connectionIt can be scaled up horizontally to multiple nodes and registered with a service center, for exampleconsul。 Each cloud node maintains its own client

@job

jobNodes consume queue data as consumer nodes and then proceedgrpcCommunicate with cloud servicespush push room broadcastAs node middleware, consumekafakarockermq。。。For example, multiple nodes can be extended to improve concurrent consumption capability

@logic

logicNode provides rest API interface, as production node and grpc client, can write to the queue as producer, can also expand their business for grpc directly call cloud node Center for push.

Component dependency packagespackage

Independent configuration between services, dependency management with composer and composer component development

  • coreFor the core basic components, the underlying design draws lessons fromswoftSource Code Design
  • grpcDefine the grpc interface specification composer package, usingprotobufBuild, encapsulate connection pools
  • discoveryService Discovery Registration Component, Registrationgrpc-serverDiscover encapsulation of services, etc.
  • processProcess management module, which can register and start custom processes and submit them to swoole master process management declaration cycle
  • queueMessage queue management module, which provides message queue interface, is implemented at the bottom level.Connection poolInterface, no need to manage connections, different message queues can be switched according to type(done rabbitmq,soon kafak)
  • redisRedis client encapsulated with connection pool version
  • taskAsynchronous Task Component, which encapsulates the interface for delivering task process tasks, currently only supports deliveryworker->taskCustom process delivery and delivery to custom processes are not supported

IV. Data Flow

IM-cloud Connection Flow Chart

Im-cloud builds distributed push middleware based on swoole native collaboration

Im-cloud data flow chart

Im-cloud builds distributed push middleware based on swoole native collaboration

Im-cloud business process

Im-cloud builds distributed push middleware based on swoole native collaboration