Analysis of im-cloud distributed middleware (3) – Implementation of job node

Time:2019-8-25

github:http://github.com/brewlin/im-…

  • Im-cloud builds distributed push middleware based on swoole native collaboration
  • Installation and deployment of im-cloud distributed middleware
  • IM-cloud <> GOIM distributed middleware concurrent pressure measurement comparison
  • Analysis of im-cloud distributed middleware (1) – Communication Protocol
  • Analysis of im-cloud distributed middleware (2) – Implementation of cloud node
  • Analysis of im-cloud distributed middleware (3) - Implementation of job node
  • IM-cloud Distributed Middleware Analysis (IV) – Logic Node Implementation

1. Overview

Job node acts as consumer, consumes data produced by logic, and then pushes it to cloud node through grpc. cloud node really processes client data. Job node defaults to multi-process consumption to start four worker processes, and defaults to 10 grpc connection pools.

  • Data flow chart

Analysis of im-cloud distributed middleware (3) - Implementation of job node

2. @Consumer Consumption Center

By default, four worker processes are started to consume logic requests, time-consuming processing is put into task process processing, and forwarded to cloud node

Listen for worker startup events

Needconfig/queue.php ,config/event.phpRegister the appropriate events and related configurations

use App\Consumer\Consumer;
use Core\App;
use Core\Swoole\WorkerStartInterface;
use Swoole\Server as SwooleServer;

class WorkerStartListener implements WorkerStartInterface
{
    const INIT_LOGIC = 1;

    public function onWorkerStart(SwooleServer $server, int $workerId): void
    {
        if(App::isWorkerStatus()){
            // The n worker processes initiated are consumed as consumer processes, and each process is blocked directly until data is consumed.
            consumer()->consume(new Consumer());
        }
    }

}

Mainstream Consumption Process

  • 1. Establish a protocol for each consumer data request to process the relevant data
  • 2. Deliver each data to the worker process for real grpc and cloud push requests
Co::create(function()use($data){
    if(empty(CloudClient::$serviceList)){
        Log::error("cancle task deliver discovery cloud node is empty");
        return;
    }
    Task::deliver(Job::class,"push",[CloudClient::$serviceList,$data]);
},false);
return Result::ACK;
  • 3. By doing the above, concurrency can be accelerated and consumption speed can be accelerated. The task process also performs concurrent processing to increase the parallel processing capacity. If the task process blockades, the task task task delivery blockage will also occur. Therefore, a concurrent process processing is also needed in the worker process.

Recommended Today

Making table style with HTML5

No more nonsense. The specific code is as follows: <!DOCTYPE html> <html> <head> <meta charset=”utf-8″> < title > Table < / Title > <style type=”text/css”> *{ margin: 0; padding: 0; } body{ font: italic 20px Georgia, serif; letter-spacing: normal; background-color: #f0f0f0; } #content{ width: 750px; padding: 40px; margin: 0 auto; background-color: #fff; border-left: 30px solid […]