Talking about reactor

Time:2020-11-26

preface

Environmental description

php --ri swoole

swoole

Swoole => enabled
Author => Swoole Team <[email protected]>
Version => 4.4.4
Built => Aug 27 2019 11:

php -version
PHP 7.3.7 (cli) (built: Jul  5 2019 12:44:05) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.3.7, Copyright (c) 1998-2018 Zend Technologies
    with Zend OPcache v7.3.7, Copyright (c) 1999-2018, by Zend Technologies

adoptswooleHow not to use itnginxandphp-fpmQuickly start an HTTP service

<?php
$http = new Swoole\Http\Server("0.0.0.0", 9501);

$http->on('request', function ($request, $response) {
    $response->header("Content-Type", "text/html; charset=utf-8");
    We should also eat with tears;
});

$http->start();

For cliphp server.phpStart service

curl http://localhost:9501
Standing flag, with tears, eating excrement should also be done

After the service is started, how about its performance?
So here comes the question…!Let’s not worry. Let’s give you a conclusion. The server started by spoole is much larger than the HTTP service under FPM. (you can check the results by local AB)
Talking about reactor

What is the bottleneck of PHP web?

We often see the following dialogue in forum, post bar and community
Dogegg A: PHP is the best language in the world
Dogegg B: screw you, PHP is poor
Dog egg C: yo! PHP is still alive.
Dogegg D: the end result of PHP is Java

。。。。。
Over the past few years, I’ve lost sight of it. I just want to say that PHP is indeed the best language in the world.
Talking about reactor
As for whether PHP has problems, I can only say that there are problems. Each language has its own more or less problems. You let those people who blow go every day ask them whether they are happy with go curd!
PHP is most ridiculed by the performance problem, you said nothing wrong, pursuitthe acmeThe performance of PHP is really not good (except for the brain heap machine)

lnmp

Before we go into PHP performance issues in detail, let’s review it againlnmp
L:linux
N:nginx
M:mysql
P:php
Let’s talk about the four things of Kangkang. We find that no matter what language is basically dependent onLinuxNginxMysql。 Then these three must have no pot, and the problem fallsphpGo ahead.
nginxIt is impossible to parse PHP directly, so with the help ofphp-fpm
The whole workflow is as follows:
Talking about reactor

php-fpmIt is a multi process and single thread model. If a request is stuck for 60s, then thisphp-fpmIt belongs to the state of waiting for a job in the manger. Then we come to the conclusion that the maximum concurrent short board carried by the server is the number of FPM. Then someone will say that it is better to set the number of FPM online without brain? So congratulations, you can go straight to the next job!
Each FPM occupies an average of 20 to 30 MB of memory, so the maximum number of FPMs supported by the machine is as follows:
FPM number = machine memory * 1024m * 0.8/30 (20)

Multiply by 0.8, mainly consider to be a person to leave a line, do not want to squeeze out the server memory for what, the impact is not good

Let’s try to think about a question: How did nginx evolve? Can PHP be the same as nginx in web application
select -> poll -> epoll

Yes, swoole can! PHP FPM can’t do it, but swote can.
Talking about reactor

reactor

There are a lot of basic concepts about reactor, which can be summarized as follows
1. I / O multiplexing
2. Event registration, distribution and scheduling
3. Asynchronous non blocking
We are familiar with the realization of reactornginxredisWait (mainly I know these two)
Let’s come to Kangkang nextswooleHow to combinereactor

The running flow chart of server in swote

Before reading the source code, let’s take a look at the flow chart of the server

Talking about reactor
Talking about reactor
Talking about reactor

come fromhttps://wiki.swoole.com/wiki/…

Explore the truth

I can see from the demo abovenew Swoole\Http\Server("0.0.0.0", 9501)Create aserver

We can locate the source code toswoole_server.ccOfstatic PHP_METHOD(swoole_server, __construct)

static PHP_METHOD(swoole_server, __construct)
{
    ......
    //Initialization
    zval *zserv = ZEND_THIS;
    char *host;
    size_t host_len = 0;
    zend_long sock_type = SW_SOCK_TCP;
    zend_long serv_port = 0;
    zend_long serv_mode = SW_MODE_PROCESS;

    //See wood has to be executed only by cli
    if (!SWOOLE_G(cli))
    {
        zend_throw_exception_ex(swoole_exception_ce, -1, "%s can only be used in CLI mode", SW_Z_OBJCE_NAME_VAL_P(zserv));
        RETURN_FALSE;
    }

    if (sw_server() != NULL)
    {
        zend_throw_exception_ex(swoole_exception_ce, -3, "server is running. unable to create %s", SW_Z_OBJCE_NAME_VAL_P(zserv));
        RETURN_FALSE;
    }

    .....
    // serv_ mode SWOOLE_ BASE、SWOOLE_ See the official wiki explanation for process

    if (serv_mode != SW_MODE_BASE && serv_mode != SW_MODE_PROCESS)
    {
        php_swoole_fatal_error(E_ERROR, "invalid $mode parameters %d", (int) serv_mode);
        RETURN_FALSE;
    }

    //Request memory
    serv = (swServer *) sw_malloc(sizeof(swServer));
    if (!serv)
    {
        zend_throw_exception_ex(swoole_exception_ce, errno, "malloc(%ld) failed", sizeof(swServer));
        RETURN_FALSE;
    }

    swServer_init(serv);
   
    ....
}

We don’t see it in the construction methodreactorRelated code, we continue to pursue$http->start()Locate to
static PHP_METHOD(swoole_server, start)

static PHP_METHOD(swoole_server, start)
{
    zval *zserv = ZEND_THIS;
    //Read the standard naming, get the server and check how familiar the service is
    swServer *serv = php_swoole_server_get_and_check_server(zserv);

    if (serv->gs->start > 0)
    {
        php_swoole_fatal_error(E_WARNING, "server is running, unable to execute %s->start", SW_Z_OBJCE_NAME_VAL_P(zserv));
        RETURN_FALSE;
    }
    if (serv->gs->shutdown > 0)
    {
        php_swoole_fatal_error(E_WARNING, "server have been shutdown, unable to execute %s->start", SW_Z_OBJCE_NAME_VAL_P(zserv));
        RETURN_FALSE;
    }

    if (SwooleTG.reactor)
    {
        php_swoole_fatal_error(E_WARNING, "eventLoop has already been created, unable to start %s", SW_Z_OBJCE_NAME_VAL_P(zserv));
        RETURN_FALSE;
    }

    .....

    //Pre work before startup of swote service
    php_swoole_server_before_start(serv, zserv);
    //Start sever
    if (swServer_start(serv) < 0)
    {
        php_swoole_fatal_error(E_ERROR, "failed to start server. Error: %s", sw_error);
    }

    RETURN_TRUE;
}

According to my intuition of moving Shishan for many years, preparation work must be done before restart, so the initialization of reactor must be inphp_swoole_server_before_start

void php_swoole_server_before_start(swServer *serv, zval *zobject)
{
    /**
     * create swoole server
     */
    if (swServer_create(serv) < 0)
    {
        php_swoole_fatal_error(E_ERROR, "failed to create the server. Error: %s", sw_error);
        return;
    }
    .....
int swServer_create(swServer *serv)
{
    serv->factory.ptr = serv;

    serv->session_list = (swSession *) sw_shm_calloc(SW_SESSION_LIST_SIZE, sizeof(swSession));
    if (serv->session_list == NULL)
    {
        swError("sw_shm_calloc(%ld) for session_list failed", SW_SESSION_LIST_SIZE * sizeof(swSession));
        return SW_ERR;
    }

    if (serv->enable_static_handler && serv->locations == nullptr)
    {
        serv->locations = new std::unordered_set<std::string>;
    }

    if (serv->factory_mode == SW_MODE_BASE)
    {
        return swReactorProcess_create(serv);
    }
    else
    {
        return swReactorThread_create(serv);
    }
}

We can see that the creation of reactor is different according to different execution modes, but this time we will only look at itswReactorProcess_createstartanalysisForcing this code
Talking about reactor

reactor

int swReactorProcess_create(swServer *serv)
{
    serv->reactor_num = serv->worker_num;
    serv->connection_list = (swConnection *) sw_calloc(serv->max_connection, sizeof(swConnection));
    if (serv->connection_list == NULL)
    {
        swSysWarn("calloc[2](%d) failed", (int )(serv->max_connection * sizeof(swConnection)));
        return SW_ERR;
    }
    //create factry object
    if (swFactory_create(&(serv->factory)) < 0)
    {
        swError("create factory failed");
        return SW_ERR;
    }
    serv->factory.finish = swReactorProcess_send2client;
    return SW_OK;
}

In fact, this function will onlyswserverThis structure initializes the corresponding properties and callback functions
1.reactor_num
2.conection_ List initialization good space
3. The end callback function of finish
4.factory

int swFactory_create(swFactory *factory)
{
    factory->dispatch = swFactory_dispatch;
    factory->finish = swFactory_finish;
    factory->start = swFactory_start;
    factory->shutdown = swFactory_shutdown;
    factory->end = swFactory_end;
    factory->notify = swFactory_notify;
    factory->free = swFactory_free;
    return SW_OK;
}

ReinitializationswServerYou can see it laterswServer_startThe parameter is constructed by initializationswServer

int swServer_start(swServer *serv)
{
    //Factory can be located to swfactory_ create
    swFactory *factory = &serv->factory;
    int ret;

    //The detection before startup determines whether the parameters under different modes and the upstream callback function of PHP are constructed, such as ontask..
    ret = swServer_start_check(serv);
    if (ret < 0)
    {
        return SW_ERR;
    }
    //Detection hook
    if (SwooleG.hooks[SW_GLOBAL_HOOK_BEFORE_SERVER_START])
    {
        swoole_call_hook(SW_GLOBAL_HOOK_BEFORE_SERVER_START, serv);
    }
    // sw_ atomic_ cmp_ Set is understood as a lock, that is, only one service can exist at the same time 
    //cannot start 2 servers at the same time, please use process->exec.
    if (!sw_atomic_cmp_set(&serv->gs->start, 0, 1))
    {
        swoole_error_log(SW_LOG_ERROR, SW_ERROR_SERVER_ONLY_START_ONE, "must only start one server");
        return SW_ERR;
    }
    //We should all know how to skip this standard output
    //run as daemon
    if (serv->daemonize > 0)
    {
        /**
         * redirect STDOUT to log file
         */
        if (SwooleG.log_fd > STDOUT_FILENO)
        {
            swoole_redirect_stdout(SwooleG.log_fd);
        }
        /**
         * redirect STDOUT_FILENO/STDERR_FILENO to /dev/null
         */
        else
        {
            serv->null_fd = open("/dev/null", O_WRONLY);
            if (serv->null_fd > 0)
            {
                swoole_redirect_stdout(serv->null_fd);
            }
            else
            {
                swSysWarn("open(/dev/null) failed");
            }
        }

        if (swoole_daemon(0, 1) < 0)
        {
            return SW_ERR;
        }
    }

    //master pid
    //Get the corresponding master process and start time
    serv->gs->master_pid = getpid();
    serv->stats->start_time = time(NULL);

    /**
     * init method
     */
     //Continue to initialize TCP related functions
    serv->send = swServer_tcp_send;
    serv->sendwait = swServer_tcp_sendwait;
    serv->sendfile = swServer_tcp_sendfile;
    serv->close = swServer_tcp_close;
    serv->notify = swServer_tcp_notify;
    serv->feedback = swServer_tcp_feedback;
    //Apply for the corresponding memory space of the worker
    serv->workers = (swWorker *) SwooleG.memory_pool->alloc(SwooleG.memory_pool, serv->worker_num * sizeof(swWorker));
    if (serv->workers == NULL)
    {
        swSysWarn("gmalloc[server->workers] failed");
        return SW_ERR;
    }

    if (swMutex_create(&serv->lock, 0) < 0)
    {
        return SW_ERR;
    }

    /**
     * store to swProcessPool object
     */
    
    serv->gs->event_workers.ptr = serv;
    serv->gs->event_workers.workers = serv->workers;
    serv->gs->event_workers.worker_num = serv->worker_num;
    serv->gs->event_workers.use_msgqueue = 0;

    uint32_t i;
    for (i = 0; i < serv->worker_num; i++)
    {
        serv->gs->event_workers.workers[i].pool = &serv->gs->event_workers;
        serv->gs->event_workers.workers[i].id = i;
        serv->gs->event_workers.workers[i].type = SW_PROCESS_WORKER;
    }

    /*
     * For swoole_server->taskwait, create notify pipe and result shared memory.
     */
    if (serv->task_worker_num > 0 && serv->worker_num > 0)
    {
        serv->task_result = (swEventData *) sw_shm_calloc(serv->worker_num, sizeof(swEventData));
        if (!serv->task_result)
        {
            swWarn("malloc[serv->task_result] failed");
            return SW_ERR;
        }
        serv->task_notify = (swPipe *) sw_calloc(serv->worker_num, sizeof(swPipe));
        if (!serv->task_notify)
        {
            swWarn("malloc[serv->task_notify] failed");
            sw_shm_free(serv->task_result);
            return SW_ERR;
        }
        for (i = 0; i < serv->worker_num; i++)
        {
            if (swPipeNotify_auto(&serv->task_notify[i], 1, 0))
            {
                sw_shm_free(serv->task_result);
                sw_free(serv->task_notify);
                return SW_ERR;
            }
        }
    }

    /**
     * user worker process
     */
    if (serv->user_worker_list)
    {
        i = 0;
        for (auto worker : *serv->user_worker_list)
        {
        //Here, we can see if there is a class representative explaining the generation mechanism of worker ID, and then we need two wokers_ Num
            worker->id = serv->worker_num + serv->task_worker_num + i;
            i++;
        }
    }
    serv->running = 1;
    //factory start
    if (factory->start(factory) < 0)
    {
        return SW_ERR;
    }
    //Registration signaling mechanism
    swServer_signal_init(serv);

    //write PID file
    if (serv->pid_file)
    {
        ret = sw_snprintf(SwooleTG.buffer_stack->str, SwooleTG.buffer_stack->size, "%d", getpid());
        swoole_file_put_contents(serv->pid_file, SwooleTG.buffer_stack->str, ret);
    }
    if (serv->factory_mode == SW_MODE_BASE)
    {
        ret = swReactorProcess_start(serv);
    }
    else
    {
        ret = swReactorThread_start(serv);
    }
    //failed to start
    if (ret < 0)
    {
        return SW_ERR;
    }
    swServer_destory(serv);
    //remove PID file
    if (serv->pid_file)
    {
        unlink(serv->pid_file);
    }
    return SW_OK;
}

The code is very long, and we can summarize it into a few things

  1. Start detection
  2. Check hook & execute hook
  3. Lock judgment
  4. IfdaemonStart change standard output
  5. Initialization function and initialization basic information (PID, time, memory of walker, etc.)
  6. Creating pipes and shared memory
  7. Create signal processing mechanism
  8. Create & start walker, task, reactor
  9. The flower is over

So let’s look at it alone8

    if (serv->factory_mode == SW_MODE_BASE)
    {
        ret = swReactorProcess_start(serv);
    }
    else
    {
        ret = swReactorThread_start(serv);
    }

Again, we just watchSW_MODE_BASE

int swReactorProcess_start(swServer *serv)
{
    //The main SW is required here_ MODE_ Single thread mode in base
    serv->single_thread = 1;

    //Monitoring TCP
    if (serv->have_stream_sock == 1)
    {
        for (auto ls : *serv->listen_list)
        {
        //Filtering UDP
            if (swSocket_is_dgram(ls->type))
            {
                continue;
            }
            //Processing of multiplexing port 
#ifdef HAVE_REUSEPORT
            if (serv->enable_reuse_port)
            {
                if (close(ls->socket->fd) < 0)
                {
                    swSysWarn("close(%d) failed", ls->socket->fd);
                }
                continue;
            }
            else
#endif
            {
                //Monitoring socket
                if (swPort_listen(ls) < 0)
                {
                    return SW_ERR;
                }
            }
        }
    }

    swProcessPool *pool = &serv->gs->event_workers;
    if (swProcessPool_create(pool, serv->worker_num, 0, SW_IPC_UNIXSOCK) < 0)
    {
        return SW_ERR;
    }
    swProcessPool_set_max_request(pool, serv->max_request, serv->max_request_grace);

    /**
     * store to swProcessPool object
     */
    serv->gs->event_workers.ptr = serv;
    serv->gs->event_workers.max_wait_time = serv->max_wait_time;
    serv->gs->event_workers.use_msgqueue = 0;
    serv->gs->event_workers.main_loop = swReactorProcess_loop;
    serv->gs->event_workers.onWorkerNotFound = swManager_wait_other_worker;

    uint32_t i;
    for (i = 0; i < serv->worker_num; i++)
    {
        serv->gs->event_workers.workers[i].pool = &serv->gs->event_workers;
        serv->gs->event_workers.workers[i].id = i;
        serv->gs->event_workers.workers[i].type = SW_PROCESS_WORKER;
    }

    //single worker
    if (swServer_is_single(serv))
    {
        return swReactorProcess_loop(&serv->gs->event_workers, &serv->gs->event_workers.workers[0]);
    }

    for (i = 0; i < serv->worker_num; i++)
    {
        if (swServer_worker_create(serv, &serv->gs->event_workers.workers[i]) < 0)
        {
            return SW_ERR;
        }
    }

    //task workers
    if (serv->task_worker_num > 0)
    {
        if (swServer_create_task_workers(serv) < 0)
        {
            return SW_ERR;
        }
        swTaskWorker_init(serv);
        if (swProcessPool_start(&serv->gs->task_workers) < 0)
        {
            return SW_ERR;
        }
    }

    /**
     * create user worker process
     */
    if (serv->user_worker_list)
    {
        serv->user_workers = (swWorker *) sw_malloc(serv->user_worker_num * sizeof(swWorker));
        if (serv->user_workers == NULL)
        {
            swSysWarn("gmalloc[server->user_workers] failed");
            return SW_ERR;
        }
        for (auto worker : *serv->user_worker_list)
        {
            /**
             * store the pipe object
             */
            if (worker->pipe_object)
            {
                swServer_store_pipe_fd(serv, worker->pipe_object);
            }
            swManager_spawn_user_worker(serv, worker);
        }
    }

    /**
     * manager process is the same as the master process
     */
    SwooleG.pid = serv->gs->manager_pid = getpid();
    SwooleG.process_type = SW_PROCESS_MANAGER;

    /**
     * manager process can not use signalfd
     */
    SwooleG.use_signalfd = 0;

    swProcessPool_start(&serv->gs->event_workers);
    swServer_signal_init(serv);

    if (serv->onStart)
    {
        swWarn("The onStart event with SWOOLE_BASE is deprecated");
        serv->onStart(serv);
    }

    if (serv->onManagerStart)
    {
        serv->onManagerStart(serv);
    }

    swProcessPool_wait(&serv->gs->event_workers);
    swProcessPool_shutdown(&serv->gs->event_workers);

    swManager_kill_user_workers(serv);

    if (serv->onManagerStop)
    {
        serv->onManagerStop(serv);
    }

    return SW_OK;
}

Continue to summarize what the letter did

  1. Monitoring TCP Port & port multiplexing
  2. Create a walker process
  3. Create task process
  4. signal processing
  5. Wait for the end of the wokers process
  6. Post shuntdown processing of wokers process technology
  7. Processing of onmanagerstop
  8. The flowers are over

The startup code of the key server is finished. It must be in the rain and fog. Don’t ask me, I am also. We actually want to see itreactorstayswooleWhat’s the role of the server service? What are we looking at
Talking about reactor
In fact, only after seeing the startup code can we know what we wantepollThe core of the model isnotifyWe can go back and look at the summary

1. Thread creation of reactor

int swReactor_create(swReactor *reactor, int max_event)
{
    int ret;
    bzero(reactor, sizeof(swReactor));

#ifdef HAVE_EPOLL
    ret = swReactorEpoll_create(reactor, max_event);
#elif defined(HAVE_KQUEUE)
    ret = swReactorKqueue_create(reactor, max_event);
#elif defined(HAVE_POLL)
    ret = swReactorPoll_create(reactor, max_event);
#else
    ret = swReactorSelect_create(reactor);
#endif

    reactor->running = 1;

    reactor->onFinish = reactor_finish;
    reactor->onTimeout = reactor_timeout;
    reactor->is_empty = swReactor_empty;
    reactor->can_exit = SwooleG.reactor_can_exit;

    reactor->write = swReactor_write;
    reactor->close = swReactor_close;

    reactor->defer = defer_task_add;
    reactor->defer_tasks = nullptr;

    reactor->default_write_handler = swReactor_onWrite;

    Socket::init_reactor(reactor);
    System::init_reactor(reactor);
    swClient_init_reactor(reactor);

    if (SwooleG.hooks[SW_GLOBAL_HOOK_ON_REACTOR_CREATE])
    {
        swoole_call_hook(SW_GLOBAL_HOOK_ON_REACTOR_CREATE, reactor);
    }

    return ret;
}

2. The function of TCP

    serv->send = swServer_tcp_send;
    serv->sendwait = swServer_tcp_sendwait;
    serv->sendfile = swServer_tcp_sendfile;
    serv->close = swServer_tcp_close;
    serv->notify = swServer_tcp_notify;
    serv->feedback = swServer_tcp_feedback;

3. AboutswServer_tcp_notify

/**
 * use in master process
 */
static int swServer_tcp_notify(swServer *serv, swConnection *conn, int event)
{
    swDataHead notify_event = {};
    notify_event.type = event;
    notify_event.reactor_id = conn->reactor_id;
    notify_event.fd = conn->fd;
    notify_event.server_fd = conn->server_fd;
    return serv->factory.notify(&serv->factory, &notify_event);
}

4. AboutswFactory_create

int swFactory_create(swFactory *factory)
{
    factory->dispatch = swFactory_dispatch;
    factory->finish = swFactory_finish;
    factory->start = swFactory_start;
    factory->shutdown = swFactory_shutdown;
    factory->end = swFactory_end;
    factory->notify = swFactory_notify;
    factory->free = swFactory_free;
    return SW_OK;
}

5. About notify

/**
 * only stream fd
 */
static int swFactory_notify(swFactory *factory, swDataHead *info)
{
    swServer *serv = (swServer *) factory->ptr;
    swConnection *conn = swServer_connection_get(serv, info->fd);
    if (conn == NULL || conn->active == 0)
    {
        swWarn("dispatch[type=%d] failed, connection#%d is not active", info->type, info->fd);
        return SW_ERR;
    }
    //server active close, discard data.
    if (conn->closed)
    {
        swWarn("dispatch[type=%d] failed, connection#%d is closed by server", info->type, info->fd);
        return SW_OK;
    }
    //converted fd to session_id
    info->fd = conn->session_id;
    info->server_fd = conn->server_fd;
    info->flags = SW_EVENT_DATA_NORMAL;

    return swWorker_onTask(factory, (swEventData *) info);
}

6. AboutonReceiveonTaskonFinish

It’s time to start taking notes
Talking about reactor

Reactor

1. Reactor is a thread form, which can be single thread or multi thread, depending on the number of Walker
2. Responsible for maintaining the clientTCPConnect, process networkIO, processing protocol, sending and receiving data
3. Do not execute any PHP code

Worker

1. Worker is a process form, which can be single process or multi process
2. Accept byReactorThread submits request packets and executesPHPThe callback function processes the data
3. Generate response data and send it toReactorThread, byReactorThread sent toTCPclient

To sum up is to do moreswooleinreactornamelynginx, workernamelyphp-fpmIn addition, with a coroutine, walker can beasynchronousIn theory, there is no upper limit on the concurrency. (someone started to spray: are you vegan when the file handle is opened!)

To the end

swooleThe source analysis of the article may be slow, because the more you see the more difficult to find, or because your foundation is not good enough.
Talking about reactor
But flag still needs to be completed. The next step may start withredisStart with, because the book is more comprehensive (copy is more smooth and smooth).
After reading the above, let’s give you a little question
YesswooleYou know whymysqlredisThe connection needs to be inonStartWhen to deal with it?