Websocket protocol

Time:2020-1-17

Reference articles

Websocket RFC GitHub Chinese Translation

Websocket RFC document

Implementation of workerman websocket protocol

Protocol composition

The protocol consists of an open handshake, followed by a basic message framing, layered TCP

Problems solved

Based on the browser mechanism, the two-way communication between client and server is realized

Protocol overview

  1. From client handshake
GET /chat HTTP/1.1
Host: server.example.com
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==
Origin: http://example.com
Sec-WebSocket-Protocol: chat, superchat
Sec-WebSocket-Version: 13
  1. Handshake from server
HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=
//Optional header, indicating the client allowed to pass
Sec-WebSocket-Protocol: chat

Above, head order does not matter

Once both the client and the server send the handshake signal, if the handshake is successful, the data transmission part starts. This is the channel of communication between the two parties, independent of the other party, and data can be sent at will.

The response of the server is not random. Please refer to page 6 / 7 of the RFC document for the following rules:

  1. Get client requestedSec-Weboscket-KeyField values, removing closing whitespace
  2. Spliced with a globally unique identifier258EAFA5-E914-47DA-95CA-C5AB0DC85B11
  3. sha1Encryption (short format)
  4. Base64 encryption

PHP program description:

$client_key = 'dGhlIHNhbXBsZSBub25jZQ==';
$client_key = trim($client_key);
$guid       = '258EAFA5-E914-47DA-95CA-C5AB0DC85B11';
$key        = $client_key . $guid;
$key        = sha1($key , true);
$key        = base64_encode($key);

The value obtained from the above results is that the server returns to the client to shake handsSec-Websocket-AcceptHeader field value

Close links

Received a0x8After the control frame, the link may be broken immediately, or after receiving the remaining data.

  • There can be a message body indicating the reason for the message, which can be recorded as a log.
  • The application must not send more data frames after sending off frames.
  • If an endpoint receives a close frame and has not previously sent a close frame, it must send a close frame.
  • After receiving the closing frame, the endpoint can delay responding to the closing frame and continue to send or receive data frames, but it does not guarantee that an endpoint that has sent the closing frame will continue to process data.
  • The endpoint that sends and receives the closed frame is considered closedwebsocketConnection, which must close the underlyingTCPConnect.

Design concept

Frame based rather than stream / text or binary frames

Link requirement

For client requirements

  • Handshake must be a valid HTTP request
  • The requested method must beGETAndHTTPVersion must be1.1
  • RequestedREQUEST-URIMust meet the requirements specified in the document (see page 13 for details)
  • Request must containHosthead
  • Request must containUpgrade: websocketHeader, value must bewebsocket
  • Request must containConnection: UpgradeHeader, value must beUpgrade
  • Request must containSec-WebSocket-Keyhead
  • Request must containSec-WebSocket-Version: 13Header, value must be13
  • Request must containOriginhead
  • The request may containSec-WebSocket-ProtocolHead, specify sub agreement
  • The request may containSec-WebSocket-Extensions, providing for the extension of the agreement
  • The request may contain other fields, such ascookieetc.

If the server does not meet the above requirements, the client will disconnect

  • If the response does not containSec-WebSocket-ProtocolSub protocol specified in, client disconnected
  • If the responseHTTP/1.1 101 Switching ProtocolsStatus code is not101, client disconnected

For server requirements

  • If the request isHTTP/1.1Or higherGETRequest, containsREQUEST-URIIt should be correctly analyzed according to the document requirements
  • Host field must be verified
  • UpgradeHeader field value must be case insensitivewebsocket
  • Sec-WebSocket-keyD decoding length is16Byte
  • Sec-WebSocket-VersionThe value must be13
  • HostIf not included, links should not be interpreted as browser initiated behavior
  • Sec-WebSocket-ProtocolFor the sub protocol of client request listed in, the server shall arrange and respond in order of priority
  • Optional other fields

Response requirements:

  • VerificationOriginField, return the appropriate error code if the request does not meet the requirements (for example: 403)
  • Sec-WebSocket-KeyThe value is one.base64The encrypted value does not need to be decoded by the server, but is only used to create a server handshake
  • VerificationSec-WebSocket-VersionValue, if not13, an appropriate error code is returned (for example:HTTP/1.1 426 Upgrade Required)
  • Resource name validation
  • Sub Protocol Validation
  • Extensions validation

If the above verification is passed, the server accepts the link. Then the response must meet the following requirements. See page 23 for details:

  1. Required, status lineHTTP/1.1 101 Switching Protocols
  2. Required, protocol upgrade headerUpgrade: websocket
  3. Required, indicates the header field of connection upgradeConnection: Upgrade
  4. Must,Sec-WebSocket-AcceptHeader field, please refer toProtocol overviewPart
  5. Optional:Sec-WebSocket-Protocolshead

The complete response code is as follows (respond in strict accordance with the following format!! Head order doesn’t matter! The key is to pay attention to the following line break! Strictly control the quantity!) :

HTTP/1.1 101 Switching Protocols\r\n
Connection: Upgrade\r\n
Upgrade: websocket\r\n
Sec-WebSocket-Accept: 3nlEzv+LqVBYnTHclAqtk62uOTQ=\r\n
//The following header field is optional
Sec-WebSocket-Protocols: chat\r\n\r\n

Basic framework agreement

Data transmission part pairpositionGrouped!! Because it isbitData encapsulation at the layer level, so if it is taken out directly, the acquired data will be processed data and need to be decrypted. The following figure isTransfer data format

  0                   1                   2                   3
  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
 +-+-+-+-+-------+-+-------------+-------------------------------+
 |F|R|R|R| opcode|M| Payload len |    Extended payload length    |
 |I|S|S|S|  (4)  |A|     (7)     |             (16/64)           |
 |N|V|V|V|       |S|             |   (if payload len==126/127)   |
 | |1|2|3|       |K|             |                               |
 +-+-+-+-+-------+-+-------------+ - - - - - - - - - - - - - - - +
 |     Extended payload length continued, if payload len == 127  |
 + - - - - - - - - - - - - - - - +-------------------------------+
 |                               |Masking-key, if MASK set to 1  |
 +-------------------------------+-------------------------------+
 | Masking-key (continued)       |          Payload Data         |
 +-------------------------------- - - - - - - - - - - - - - - - +
 :                     Payload Data continued ...                :
 + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - +
 |                     Payload Data continued ...                |
 +---------------------------------------------------------------+

1. Introduction to the meaning of special terms

  1. 1bit,FIN
  2. Each 1 bit, rsv1, rsv2, rsv3
  3. 4bit, opcode (defined in ABNF below)

    • %X0 consecutive frame
    • %X1 text frame
    • %X2 binary frame
    • %X3 to% X7 reserved frame
    • %X8 link off
    • %x9 ping
    • %xA pong
    • %Xb-f reserved control frame
    • All of the above are hexadecimal values
  4. 1bit, mask

    • The data sent by the client to the server needs to be set to 1
    • In other words, the data is masked
  5. Please refer to the RFC document (page 31) for the specific range of 7bit, 7 + 16bit, 7 + 64bit and payload length

    • Playload length = Extended Payload length + Application Payload length
    • Payload length = extended data length + application data length
    • The extended data length may be 0, so when the extended data length = 0, the payload length = the application length
    • The length unit of payload data isByte
  6. 0/4 byte, masking-key

    • The data sent by the client to the server is masked, with a length of 32bit
    • The data sent by the server to the client is unmasked, with a length of 0 bit
  7. x + y Byte, Payload Data

    • Payload data
  8. x Byte, Extension Data

    • Extended data
  9. y Byte, Application Data

    • Applied data

2. understanding

The following is shown in the figurewebsocketThe data transmitted by the protocol iswebsocketThe data processed by the protocol cannot be obtained directlyEffective data。 If you want to get valid data, you need to follow thewebsocketInterpretation of the provisions of the agreement.

In the figure, from left to right, arrange from high position to low position.

What is low and high??

It’s like a decimal number, if there’s a description like this:3Represents a bit,2Ten,1It means 100 digits. What’s the number?? Answer:123

It’s easy to understand,Individual, 10, 100It describes the order of arrangement; similarly, in the field of program, the order from low to high is also described! HoweverIndividual, 10, 100What is described is10 binary systemAndLow, highWhat is described is2 binary systemThe specific description isBit 0, bit 1, bit 2And so on.The order in the current example is low to high), here is a picture description:

Websocket protocol

UnderstoodLow, high, the data sequence described in the figure above is clear.

as everyone knows,Bit (bit)It is the minimum storage unit in memory and can only be stored0、1Two values. So in order to get and set the value of a bit, bit operation is needed. Because it ispositionTherefore, the content described in the figure is based on the complement.

The data sent by the client to the server is masked!It needs to be parsed, and the data process needs to be parsed:

//Parsing client encrypted data according to websocket specification
function decode(string $buffer){
    //Buffer [0] gets the first byte, 8bit
    //Compared with that picture, it shows fin + rsv1 + rsv2 + RSV 3 + opcode
    //Why to convert to ASCII value
    //To ensure that the bit operation result is correct!
    //For details of PHP bit operation, please refer to: https://note.youdao.com/share/? Id = 927bfc2f40a8d62f4c9165de30a41e75 & type = note#/
    //Here's a simple explanation
    //The following code will have a code like $first_byte > > 7
    //In PHP, the operands are treated as integers (int) 
    //So if you don't convert to ASCII, the process will be
    // (int) $buffer[0] >> 7
    //Such a result will be wrong!!
    //Ord ((int) $buffer [0])! = = ord ($buffer [0]) is the best proof
    //Because ASCII values are not the same, binary values (strictly speaking, I think it should be said: complement) are not the same
    //This violates the websocket protocol
    //Causes parsing errors
    $first_byte  = ord($buffer[0]);
    //Buffer [1] gets the second byte, 8bit
    //Compared with that picture, it shows mask + payload len
    $second_byte = ord($buffer[1]);
    
    //Get the first value on the left
    $fin = $first_byte >> 7;
    //To get the value represented by payload len
    //Bit 7 needs to be set to 0
    //Because bit 7 is the mask, bits 0 - 6 are the complement of payaod len
    //So to get the value of payload len
    // 0111 1111 => 127
    $payload_len = $second_byte & 127;
    
    //The data sent by the client to the server is masked
    //So get the client data after mask key + mask processing
    //Get mask key + payload data
    if ($payload_len === 127) {
        //If payload len = 127 bytes
        //Payload len itself occupies 7 bits
        //Extended payload light occupies 64bit
        $mask_key       = substr($buffer , 10 , 4);
        $encoded_data   = substr($buffer , 14);
    } else if ($payload_len === 126) {
        //If payload len = 126 byte
        //Payload length itself occupies 7 bits
        //Extended payload light occupies 16bit
        $mask_key       = substr($buffer , 4 , 4);
        $encoded_data   = substr($buffer , 8);
    } else {
        //If payload len = 126 byte
        //Payload length itself occupies 7 bits
        //Extended payload light occupies 0 bit
        $mask_key       = substr($buffer , 2 , 4);
        $encoded_data   = substr($buffer , 6);
    }
    
    //Decoding payload data
    $decoded_data = "";
    
    //Decode each payload data
    //Decoding rules are detailed in RFC documents
    for ($index = 0; $index < count($encoded_data); ++$index)
    {
        $k              = $index % 4;
        $valid_data     = $encoded_data[$index] ^ $mask_data[$k];
        $decoded_data  .= $valid_data;
    }
    
    //This is the real data sent by the client!!
    return $decoded_data;
}

Instead, if the server wants to send data towebsocketClient, the data should also be processed accordingly! Processing flow:

//Encapsulate the message sent to the client according to the websocket specification
function encode($msg){
    if (!is_scalar($msg)) {
        Print_r ("only scalar data can be sent");
    }
    
    //Data length
    $len = strlen($msg);
    
    //Only transfer text frame here! First byte, text frame 1000 0001 = > 129
    //If you need a binary frame, for example, to transfer large files, please implement it separately
    $first_byte = chr(129);
    
    if ($len <= 125) {
        //Payload length = maximum range supported by 7bit!
        $second_byte = chr($len);
    } else {
        if ($len <= 65535) {
            //Payload length = 7, extended payload length = 16bit, maximum supported range 65535
            //Finally, 16bit is interpreted as an unsigned integer, sorted as: big endian byte order (network byte order)
            $second_byte = chr(126) . pack('n' , $len);
        } else {
            // payload length = 7,extended payload length = 64bit
            //The last 64 bits are interpreted as unsigned integers, big endian (network byte order)
            $second_byte = chr(127) . pack('J' , $len);
        }
    }
    
    //Note that the data sent to the client does not need to be processed
    //Check websocket document for details!!
    $encoded_data = $first_byte . $second_byte . $buffer;
    
    //This is the data sent to the client!   
    return $encoded_data;
}

Message fragmentation

Fragmentation purpose

The main purpose of message segmentation is to allow a message of unknown size to be sent when the message starts but does not need to buffer the whole message; the message without segmentation needs to buffer the whole message in order to get the message size;

Fragmentation requirements:

  • First segment fin = 0, opcode! = 0x0, followed by multiple segments fin = 0, opcode = 0x0, ending at segments fin = 1, opcode = 0x0
  • Extended data may occur in any partition
  • The control frame may be injected into the middle of the fragmentation message, and the control frame itself must not be segmented
  • Message fragments must be delivered to the recipient in the order in which they were sent by the sender
  • A message in a fragment must not be interleaved with another message in the fragment unless an extension to interpret the interleave has been negotiated.
  • Websocket server shall be able to process the control frame in the middle of the fragmentation message
  • A sender can create fragments of any size for uncontrolled messages (uncontrolled frames)
  • Unable to process control frame
  • If any reserved bit values are used and the meaning of these values is unknown to the middleware, a middleware must not change the fragmentation of a message.
  • In a connection context, an extension has been negotiated and the middleware does not know the semantics of the negotiated extension. A middleware must not change the fragmentation of any message. Similarly, a middleware that does not see the websocket handshake (and is not informed about its content) and leads to a websocket connection must not change the fragmentation of any messages of the link.
  • Because of these rules, all fragments of a message are of the same type, set with the opcode of the first fragment. Because the control frame cannot be fragmented, the type of fragmentation used in a message must be either text, binary, or a reserved opcode.

ping

Received aping(0x9)Control frame, must return apong(0xa)Control frame, indicating that the process is still in progress!! It’s actually a heartbeat test

pong

  1. Can be received atping(0x9)After the control frame, it is returned as a response message.
  2. It can also be sent in one waypongFrame, indicating that the sender process is still in progress, as one-way heartbeat

Status code

  1. 1000, normally closed
  2. 1001, leaving
  3. 1003, closing connection
  4. 1004, reserved
  5. 1005, reserved
  6. 1006, reserved
  7. 1007, the endpoint is terminating the connection because it received a message that did not match the message type.
  8. 1008, the endpoint is terminating the link because it received a message that violated its rules.
  9. 1009, endpoint is terminating link because received message is too large
  10. 1010, endpoint terminating link due to extension issues
  11. 1011, endpoint terminating link, unexpected error
  12. 1015, reserved
  13. … omitted, refer to RFC document for details

tail

The above personal understanding is for reference only. If there is any mistake, please correct it. To be continued

Recommended Today

Summary: NPM common commands and operations

The full name of NPM is (node package manager), which is a package management and distribution tool installed with nodejs. It is very convenient for JavaScript developersDownload, install, upload and manage installed packages。 First of all, the following variables will be used: < name > | < PKG > module name < version > version […]