From simple to highly concurrent server (1)


A single threaded echo server

Let’s start with a simple server.
It can take a customer’s connection, receive a message, send it back, close the connection – done. We use BSD socket, which is common on Linux and IOS / OSX, to write the code of this server. The main part is as follows: (c + + syntax)

#include <stdio.h>
#include <stdlib.h>

#include <netdb.h>
#include <netinet/in.h>

#include <unistd.h>
#include <string.h>

void fuck_you(void)

int main(int argc, char *argv[])
    int server_sock = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);

    struct sockaddr_in server_addr;
    struct sockaddr_in client_addr;
    int portno = 5432;
    bzero((char *) &server_addr, sizeof(server_addr));
    server_addr.sin_family = AF_INET;
    server_addr.sin_addr.s_addr = INADDR_ANY;
    server_addr.sin_port = htons(portno);

    //Note 1: bind socket to specific IP address
    int bind_succ = bind
        ( server_sock
        , (struct sockaddr *) &server_addr
        , sizeof(server_addr)
    if (bind_succ < 0) fuck_you();

    //Note 2: start monitoring
    int listen_succ = listen(server_sock, 1024);

    if (listen_succ < 0) fuck_you();

    //Note 3: This is a cycle that will never stop
        int client_addr_size;
        printf("before accept.\n");
        //Note 4: the thread is blocked here
        int client_sock = accept
            (server_socket // accept the connection from the server socket
            , (struct SOCKADDR *) & client_addr // here is the address of the client initiating the connection
            , (socklen_t*)& client_addr_size
        printf("handle client sock: %d", client_sock);
        //Prepare buffer for receiving messages
        const int buffer_size = 1024;
        char * recv_buffer = (char*)malloc(buffer_size);
        printf("before recv, buffer ready, address: %p\n", recv_buffer);
        //Note 5: call recv to receive the information sent by the client. This process will block
        int msg_size = recv
            (Note: it is received by the client UU socket.)
            , recv_buffer // put the received content in the buffer
            , buffer_size // tells the system how large the buffer we set is
            , 0
        fwrite(recv_buffer, sizeof(char), msg_size, stdout);
        //Note 6: send the received information as it is
        int byte_sent = send
            ( client_sock
            , recv_buffer
            , msg_size
            , 0
        //Note 7: disconnect the client actively

This code is of course very coarse (error: coarse), there may be memory leakage, if the message sent by the customer is too long, it will not receive completely All kinds of problems, but it basically shows how a server program works.

The following are some important works to realize a TCP server mentioned in the code:

  1. Bind the listening address and start listening (notes 1 and 2)

  2. Waiting for client connection (Note 4)

  3. Receiving data sent by client (Note 5)

  4. Send reply (Note 6)

In fact, these four points are also the things that any server should accomplish.
If you use UDP, you don’t need to wait for the client to connect. This is because UDP is a packet oriented transport protocol rather than a connection oriented transport protocol. Using TCP, you need to wait for the client to connect. In fact, it involves the process of establishing a TCP connection, which is called “3-way handshake”.
However, this handshake process belongs to the standard part of TCP protocol, so it is actually completed by the operating system (all operating systems supporting TCP / IP protocol stack will complete this process for programmers). We just need to callacceptThis API is equivalent to telling the system “start to handle the handshake for me now, let me know if someone wants to shake hands with you.”.

Threads and blocking

The handshake procedure call accept will block the execution of the whole program. What does blocking mean?
When we write code, we write a loop, just like note 3 in the code:

    printf("I just can't stop speaking!\n");

Even if you don’t run the program, you should expect to keep typing a line on the screen. This means that if the program is not blocked, it will continue to execute. Strictly speaking,printfIt will also block, but the blocking time is very short, and it can automatically remove the blocking state. The specific explanation will be later.

And callacceptYou can’t automatically unblock it – if you run the code successfully, you’ll see that the screen outputbefore accept.After that, there was no immediate outputhandle client sock: ——The program staysacceptThe place called can also be considered asacceptNo results have been returned.

The essence of blocking is that the operating system suspends the thread executing your code, while the thread is the basic unit for the operating system to schedule the CPU, which usually means that the operating system takes the CPU to do other things, and your program cannot use the CPU for calculation, so it can only pause. Until a client successfully connects to your server.

In order to simulate this, we can use Python + gevent to simulate many (300) clients to initiate TCP connections simultaneously and continuously:

from __future__ import print_function
from gevent.socket import socket as gsocket
import gevent
import socket

def do_connect(addr, index):
    if 0: client_sock = socket.socket()
    while True:
        client_sock = gsocket(socket.AF_INET,
        print('client {0} connected.'.format(index))
        client_sock.send('Hello World')
        data = client_sock.recv(1024)
        print('recv data: {0}'.format(data))

if __name__ == '__main__':
    server_addr = ('', 5432)
    greenlets = list()
    for i in xrange(300):
        g = gevent.spawn(do_connect, server_addr, i)

Then, if there is no accident, you can see that the program continues to be executed, repeated very regularly, and always processed one by one in order. If you pay attention to the output of the client, you may see that the later initiated connections have timed out, and you will see a lot of tracebacks.

This is certainly not the experience we get when we visit the website everyday: we can connect and see the content of the webpage soon (there are many exceptions, of course, in China). So this is not an ideal high concurrency server.

Why does the client that sends the connection earlier not timeout, and the client that sends the connection later will timeout? The reason is that when the server is blocking and waiting for IO, a single thread cannot respond to other requests.

In order to verify this conclusion, you can put thegevent.sleepFor example, if you change the time to 20 seconds, you will find that more connections will timeout – because the server spends more time waiting for the client to send data, the number of connections that can be accepted in the service window before the same timeout is less.

(to be continued)