Case study on asynchronous processing of time-consuming tasks using task process in PHP spool

Time:2021-4-21

This paper introduces the asynchronous processing of time-consuming tasks using task process in PHP spool. The details are as follows:

As we know, there are two major processes in spool, namely, master master process and manager management process.

In the master main process, there will be a main reactor thread and multiple reactor threads, which are mainly used to maintain TCP connection, process network IO and send and receive data.

The role of manager is to fork and manage worker and task processes.

The function of the worker process is to receive the data passed by the reactor thread, process the data, and return the processing results to the reactor thread.

Task process is used to deal with some relatively time-consuming tasks. Task and worker process are independent, which will not affect worker process to process client requests.

1、 Application scenarios of task process:

1. Relatively time-consuming e-mail group sending, such as so and so activities, needs to send active e-mail to 100W users.

2. Push some big V news, for example, a big V sent a new message, fans need to get the news in time.

2、 The relationship between worker and task is as follows

1. In the worker process, tasks can be delivered by calling task (), and the task process responds to the delivered tasks through ontask event.

2. Task process through direct return or call finish () To tell the worker process that the task processing is completed, and the worker process responds to the completion of the task through the onfinish event.

3、 Before using task:

1. Configure task in server_ worker_ Num number.

2. Set the server’s ontask and onfinish event callback functions.

4、 Simple calculation example of using task to accumulate and sum

<?php
$server = new swoole_server('0.0.0.0', 6666);
 
$server->set([
  'worker_num' => 2,
  'task_worker_num' => 16,
]);
 
$server->on('WorkerStart', function ($server, $worker_id) {
  //Note here that we use task worker to determine whether it is a task process or a worker process
  // we need to call task () in the worker process, otherwise we will report a warning.
  //It will be executed twice here, because we set up the worker_ Num is 2
  if (!$server->taskworker) {
    echo  ' Delivery task starts ...',  PHP_ EOL ;
    //Deliver 32 cumulative computing tasks to 16 task processes
    for ($ix = 0; $ix < 32; $ix++) {
      //Note that the delivery here is asynchronous
      $server->task([mt_rand(1, 100), mt_rand(1000, 9999)]);
    }
    echo  ' End of delivery task ...',  PHP_ EOL ;
  }
});
 
//The server service must have an onReceive callback
$server->on('Receive', function ($server, $fd, $reactor_id, $data) {
 
});
 
//Note that the task process is completely in synchronous blocking mode
$server->on('Task', function ($server, $task_id, $src_worker_id, $data) {
  echo  " task  {$ task_ id }  The process is working ...",  PHP_ EOL ;
  $start = $data[0];
  $end = $data[1];
  $total = 0;
  for (; $start <= $end; $start++) {
    $total += $start;
  }
  echo "task {$task_ ID} process completes work... ", PHP_ EOL;
  return $total;
});
 
$server->on('Finish', function ($server, $task_id, $data) {
  echo "task {$task_ ID} process processing completed, the result is {$data} ", PHP_ EOL;
});
 
$server->start();

Note that we post tasks to the task pool by calling task (), and the bottom layer of spool will poll to post tasks to each task process.

When the number of tasks you post exceeds the processing speed of ontask, it will cause the task pool to be full, and then cause the worker process to block. Therefore, it is necessary to set tasks reasonably_ worker_ The relationship between num number and processing speed.

Of course, we can also manually post tasks to the specified task process. The second parameter of the task() function can specify the task process ID to be delivered, and the ID range is 0 to (task)_ worker_ num – 1)。

5、 The task is segmented and manually controlled to be delivered to the task process

<?php
$server = new swoole_server('0.0.0.0', 6666);
 
$server->set([
  'worker_num' => 1,
  'task_worker_num' => 10,
]);
 
$server->on('WorkerStart', function ($server, $worker_id) {
  //In order to facilitate the demonstration, the worker_ Num is set to 1, which will only be executed once
  if (!$server->taskworker) {
    //Through swoole_ Table shared memory, shared data in different processes
    $server->result = new swoole_table(10240);
    //Used to save the number of tasks completed
    $server->result->column('finish_nums', swoole_table::TYPE_INT);
    //Used to save the final calculation results
    $server->result->column('result', swoole_table::TYPE_INT);
    $server->result->create();
    //Calculate the sum of 1000, and assign the task to 10 task processes
    $num = 1000;
    $step = $num / $server->setting['task_worker_num'];
    for ($ix = 0; $ix < $server->setting['task_worker_num']; $ix++) {
      $start = $ix * $step;
      $server->task([$start, $start + $step], $ix);
    }
  }
});
 
$server->on('Receive', function ($server, $fd, $reactor_id, $data) {
 
});
 
//Note that the task process is completely in synchronous blocking mode
$server->on('Task', function ($server, $task_id, $src_worker_id, $data) {
  echo "task {$task_ ID} process is working... Calculate {$data [0]} - {$data [1]} ", PHP_ EOL;
  $start = ++$data[0];
  $end = $data[1];
  $total = 0;
  for (; $start <= $end; $start++) {
    $total += $start;
  }
  echo "task {$task_ ID} process completes work... ", PHP_ EOL;
  return $total;
});
 
$server->on('Finish', function ($server, $task_id, $data) {
  echo "task {$task_ ID} process processing completed, the result is {$data} ", PHP_ EOL;
  $server->result->incr('finish_nums', 'finish_nums');
  $server->result->set('result', ['result' => $data + $server->result->get('result', 'result')]);
 
  if ($server->result->get('finish_nums', 'finish_nums') == $server->setting['task_worker_num']) {
    Echo "final calculation result: {$server > result > get ('result ','result')}", PHP_ EOL;
  }
});
 
$server->start();

For more information about PHP, readers who are interested in it can see the following topics: summary of PHP network programming skills, summary of PHP socket usage, introductory course of PHP object-oriented programming, tutorial of PHP data structure and algorithm, and summary of PHP programming algorithm

I hope this article is helpful for PHP programming.

Recommended Today

Libp2p RS version 0.3.0 introduction

V0.3.0 released on 4.23, usingAsyncRead & AsyncWriteTo replace ourReadEx & WriteEx & SplitEx; SimplifiedKad/DHTImplementation logic. modify ReadEx & WriteEx & SplitEx: At first we tried to useasync-traitTo define their own IO operationsTraitFor more pure useasync/awaitTo write code. withReadExFor example, it is roughly as follows: #[async_trait] pub trait ReadEx { async fn read(&mut self, buf: &mut […]