In depth interpretation of HTTP’s past and present lives

HTTPThe full name of is called hypertext transfer protocol. It is an application layer transfer protocol based on the world wide web. At the beginning of its establishment, it was mainly to integrate hypertext markup language（HTML）Document fromWebThe server passes to the client’s browser. The original version wasHTTP 0.9, was produced in the late 1980s, and later upgraded to 1.0 in 1996.

But here it is WEB2.0Since then, our page has become complex, not only some simple words and pictures, but also ourHTMLThe page has CSS，JavascriptTo enrich our page display, whenajaxWith the emergence of, we have another method to obtain data from the server, which are actually based onHTTPAgreed. Similarly, in the era of mobile Internet, our pages can run in the mobile browser, but withPC Compared with the mobile phone, the network situation is more complex, which makes us have toHTTPIn the process of in-depth understanding and continuous optimization. So it appeared in 1997HTTP1.1, and then in 2014,HTTP1.1It’s been updated all the time.

Then in 2015, in order to adapt to the rapid transmissionwebApplications and the needs of modern browsers inGoogleofSPDYNew technologies have been developed on the basis of the projectHTTP2agreement.

Four years later, in 2019,GoogleA new protocol standard has been developedQUICAgreement, it isHTTP3The cornerstone of its purpose is to improve users and websitesAPISpeed and security of interaction.

HTTP1.0

EarlierHTTP1.0Version, is aNo state, no connectionApplication layer protocol.

HTTP1.0It is specified that the browser and the server maintain a short connection, and each request of the browser needs to establish a connection with the serverTCPConnect and disconnect the server immediately after processingTCPConnected (no connection), the server does not track each client and does not record past requests (stateless).

TCPConnection time = 1.5RTT

• One go（SYN）
• Second circuit（SYN + ACK）
• Three go（ACK）

RTT(Round Trip Time)It refers to the time of communication

This statelessness can be achieved bycookie/sessionMechanism for identity authentication and status recording. The following two problems are more troublesome.

There is a very important premise to understand these two problems: the client establishes a connection to the server according to the domain name. GenerallyPCThe client browser will target a single domain nameserverAt the same time, 6 ~ 8 connections are established, and the number of connections at the mobile phone end is generally controlled at 4 ~ 6

• Unable to reuse connection, each request goes through three handshakes and slow start. The impact of three handshakes is more obvious in high latency scenarios, and slow start has a greater impact on large file requests.

  TCPThe connection will “tune” itself over time. At first, it will limit the maximum speed of the connection. If the data is successfully transmitted, it will increase the transmission speed over time. This tuning is calledTCPSlow start.

• Team head blocking（head of line blocking）, becauseHTTP1.0Specifies that the next request cannot be sent until the previous request response arrives. Assuming that the previous request response does not arrive, the next request will not be sent, and the subsequent requests will be blocked.

To solve these problems,HTTP1.1There it is.

HTTP1.1

aboutHTTP1.1, not only inheritedHTTP1.0Simple characteristics, but also overcome many problemsHTTP1.0Performance issues.

• Cache processing, inHTTP1.0Mainly used inheaderInsideIf-Modified-Since,ExpiresAs a criterion for cache judgment,HTTP1.1More cache control strategies are introduced, such asEntity tag，If-Unmodified-Since, If-Match, If-None-MatchWait for more alternative cache headers to control the cache policy.
• Bandwidth optimization and network connection usage，HTTP1.0In, there are some phenomena of wasting bandwidth. For example, the client only needs a part of an object, but the server sends the whole object, and does not support the function of breakpoint retransmission,HTTP1.1Is introduced in the request headerrangeHeader field, which allows only a certain part of the resource to be requested, that is, the return code is206（Partial Content）This makes it easy for developers to make free choices to make full use of bandwidth and connections.
• Management of error notification, inHTTP1.124 error status response codes are added in the, such as409（Conflict）Indicates that the requested resource conflicts with the current state of the resource;410（Gone）Indicates that a resource on the server has been permanently deleted.
• Host header processing, inHTTP1.0It is considered that each server is bound with a uniqueIPAddress, therefore, in the request messageURLHost name was not passed（hostname）。 However, with the development of virtual host technology, there can be multiple virtual hosts on a physical server（Multi-homed Web Servers）And they share oneIPAddress.HTTP1.1Both request and response messages should be supportedHostIf there is no header field in the request messageHostThe header field reports an error（400 Bad Request）。
• Persistent connection，HTTP1.1Added oneConnectionFields, by settingKeep-AliveCan keepHTTPThe connection is continuously disconnected to avoid repeated establishment and release of establishment every time the client and server requestTCPConnection, which improves the utilization of the network. If the client wants to shut downHTTPConnection, which can be carried in the request headerConnection: falseTo tell the server to close the request.
• Pipelining, based onHTTP1.1Long connections, making request pipelining possible. Pipelining enables requests to be transmitted “in parallel”. For example, suppose the subject of the response is ahtmlPage, which contains a lot ofimg, this timekeep-aliveIt plays a great role in “parallel” sending multiple requests.

The “parallel” here is not really parallel transmission becauseThe server must send back the corresponding results according to the order of client requests, so as to ensure that the client can distinguish the response content of each request.

As shown in the figure, the client sends two requests at the same time to obtainhtmlandcss, let’s say the servercssThe resource is ready first, and the server will send it firsthtmlResendcss。

In other words, only whenhtmlAfter the response resources are completely transmitted,cssOnly the resources that respond can start transmission. In other words, it is not allowed to have two at the same timeParallel response。

In addition,pipeliningThere are also some defects:

• pipeliningOnly applicable tohttp1.1, generally speaking, supporthttp1.1ofserverAll ask for supportpipelining;
• Only idempotent requests（GET，HEAD）Can usepipeliningFor example, non idempotent requestsPOSTCannot be used because there may be a sequence dependency between requests;
• MosthttpProxy server does not supportpipelining;
• And not supportedpipeliningThere is a problem with the old server negotiation;
• May lead to newFront of queue blockingProblems;

soHTTP1.1Still can’t solve the team head jam（head of line blocking）Problems. At the same time, “pipelining” technology has various problems, so many browsers either don’t support it at all, or close it directly by default, and the opening conditions are very harsh… And it doesn’t seem to be useful in fact.

What about the parallel requests we see on the Google console?

As shown in the figure, the green part represents a waiting time from the request to the server response, while the blue part represents the download time of the resource. In theory,HTTPThe response should be that the resources of the previous response are downloaded before the resources of the next response can be downloaded. Here, however, the parallel downloading of response resources occurs. Why?

In fact, althoughHTTP1.1Pipelining is supported, but the server must also respond back one by one, which is a big defect. In fact, browser manufacturers at this stage have taken another approach, which allows us to open multiple browsersTCPSession. In other words, the parallelism we see in the figure above is actually differentTCPConnectedHTTPRequests and responses. This is the familiar browser’s restriction on loading 6 ~ 8 resources in parallel in the same domain. And this is the realparallel！

HTTP2.0

HTTP 2.0Compared withHTTP 1.x , greatly improvedwebPerformance. In andHTTP/1.1On the basis of full semantic compatibility, the network delay is further reduced. For front-end developers, it undoubtedly reduces the optimization work in the front-end. This article willHTTP 2.0The agreement summarizes three basic technical points, contacts relevant knowledge and exploresHTTP 2.0How to improve performance.

begin to display one ‘s talent

HTTP/2: the Future of the InternetThis isAkamaiAn official demonstration established by the company to illustrateHTTP/2Compared with the previousHTTP/1.1Significant improvement in performance. Request 379 pictures at the same time, fromLoad timeAs can be seen from the comparison ofHTTP/2Advantage in speed.

At this point, if we openChrome Developer ToolsseeNetworkAs you can see in the column,HTTP/2With respect to network requestsHTTP /1.1The obvious difference.

HTTP/1:

HTTP/2:

Multiplexing

http1.1Initially, only 2 domains are supported for the same domain nametcp, but for performance reasonsrfcLater, it is modified to use 6-8. In addition,keep-aliveWithhttp pipeliningEssentiallymultiplexingBut because there will behead of line blockingProblem, mainstream browsers are prohibited by defaultpipelining, andhttp2.0It’s really solvedholproblem

Multiplexing allows simultaneous transmission through a single channelHTTP/2The connection initiates multiple request response messages.

As we all know, inHTTP/1.1In the agreement“Browser clients have a certain number of requests under the same domain name at the same time. Requests exceeding the limit will be blocked”。

Clients that use persistent connections SHOULD limit the number of simultaneous connections that they maintain to a given server. A single-user client SHOULD NOT maintain more than 2 connections with any server or proxy. A proxy SHOULD use up to 2*N connections to another server or proxy, where N is the number of simultaneously active users. These guidelines are intended to improve HTTP response times and avoid congestion.

The figure summarizes the number of restrictions imposed by different browsers.

This is why some sites have multiple static resourcesCDNOne of the reasons for domain names is to takeTwitterFor example,http://twimg.com, the purpose is to solve the problem of browser request restriction blocking for the same domain name in a disguised form. andHTTP/2 Multiplexing of（Multiplexing）It is allowed to pass through a single at the same timeHTTP/2The connection initiates multiple request response messages.

thereforeHTTP/2 It is easy to implement multi stream parallelism without relying on the establishment of multiple streamsTCP connect,HTTP/2holdHTTPThe basic unit of protocol communication is reduced to frames one by one, which correspond to messages in the logical flow. In parallel on the sameTCPTwo way message exchange on the connection.

Binary framing

Without changeHTTP/1.xSemantics, methods, status codesURIAnd the header field,HTTP/2How to achieve “breakthrough”HTTP1.1Performance constraints, improve transmission performance and achieve “low latency and high throughput”?

One of the keys is in the application layer（HTTP/2）And transport layer（TCP or UDP）Add a binary framing layer between.

In the binary framing layer,HTTP/2All transmitted information is divided into smaller messages and frames（frame）And encode them in binary format, whereHTTP1.x The first part of the information will be encapsulated inHEADER frame, and the correspondingRequest BodyThen package toDATA frameInside.

Here are some concepts:

• Flow（stream）: bidirectional byte stream on established connection.
• Message: a complete series of data frames corresponding to the logical message.
• Frame（frame）：HTTP2.0The smallest unit of communication. Each frame contains a frame header and at least identifies the stream to which the current frame belongs（stream id）

HTTP/2Communication is done on a connection that can carry any number of two-way data streams.

Each data stream is sent in the form of a message, which consists of one or more frames. These frames can be sent out of order, and then according to the stream identifier of each frame header（stream id）Reassemble.

For example, each request is a data stream, which is sent as a message, and the message is divided into multiple frames, which are recorded in the frame headerstream idIt is used to identify the data stream to which it belongs. Frames of different genera can be randomly mixed together in the connection. The receiving party maystream idThe frames are assigned to different requests.

In addition, multiplexing (connection sharing) may cause critical requests to be blocked.HTTP2.0Priority and dependency can be set for each data stream in the. The data stream with high priority will be preferentially processed by the server and returned to the client. The data stream can also rely on other sub data streams.

soHTTP2.0The real parallel transmission is realized, and it can be in oneTCPAny number onHTTPRequest. This powerful function is based on the characteristics of “binary framing”.

Summary:

• Single connection with multiple resources can reduce the link pressure on the server, occupy less memory and increase the connection throughput
• becauseTCPThe reduction of connection improves the network congestion, and the reduction of slow start-up time makes the recovery speed of congestion and packet loss faster

Header compression

HTTP/1.1Not supportedHTTPHeader compression, for this purposeSPDYandHTTP/2emerge as the times require,SPDYThe is universalDEFLATEAlgorithm, andHTTP/2Is designed specifically for header compressionHPACKAlgorithm.

stayHTTP1.xIn, header metadata is sent in plain text, which usually adds 500 ~ 800 bytes to each request.

for instancecookie, by default, the browser will set thecookieAttached toheaderSend it to the server. (due tocookieIt is relatively large and is sent repeatedly every time. Generally, it does not store information, but is only used for status recording and identity authentication)

HTTP2.0useencoderTo reduce the need for transmissionheaderSize, communication partiescacheA copyheader fieldsTable, which avoids duplicationheaderThe size of transmission is reduced. An efficient compression algorithm can greatly compressheader, reduce the number of packets sent, thereby reducing the delay.

Server push

Server push is a mechanism that sends data before the client requests. stayHTTP/2In, the server can send multiple responses to a request from the client.Server PushGive WayHTTP1.x In this era, the optimization means of using embedded resources has become meaningless; If a request is initiated by your home page, the server is likely to respond to the content of the home pagelogoAnd style sheets, because it knows that the client will use these things. This is equivalent to in a HTMLAll resources are collected in the document, but compared with it, server push has another great advantage: it can be cached! It also makes it possible for different pages to share cache resources under the condition of following homology.

HTTP3

Http / 3 has not been officially launched yet, but since 2017, http / 3 has been updated to 34 drafts, the basic features have been determined, and the package format may change in the future.

Therefore, this http / 3 introduction will not involve the package format, only its features.

Http / 2 with a fly in the ointment

HTTP/2Through new features such as header compression, binary coding, multiplexing and server push, it has been greatly improvedHTTP/1.1Performance, and the fly in the ointment isHTTP/2The protocol is based onTCPSo there are three defects.

• Team head obstruction;
• TCPAndTLSDelay in handshake;
• Network migration requires reconnection;

Team head blocking

HTTP/2Multiple requests are running in oneTCPIn the connection, then whenTCPIn case of packet loss, the wholeTCPAll have to wait for retransmission, then the message will be blockedTCPAll requests in the connection.

becauseTCPIs a byte stream protocol,TCPThe layer must ensure that the received byte data is complete and orderly if the serial number is lowTCPSegments are lost in network transmission, even if the serial number is higherTCPThe segment has been received, and the application layer cannot read this part of data from the kernelHTTP From the perspective, the request is blocked.

Delay in handshake between TCP and TLS

launchHTTPWhen requesting, you need to go throughTCPThree handshakes andTLSFour handshakes（TLS 1.2）Therefore, a total of 3 are requiredRTTTo send the requested data.

In addition,TCPDue to the “congestion control” feature, the connection has just been establishedTCPThere will be a “slow start” process, which will be rightTCPThe connection produces a “deceleration” effect.

Network migration requires reconnection

OneTCPThe connection is composed of four tuples (source)IPAddress, source port, destination IPAddress, target port), which means that if IPIf the address or port changes, it will lead to the need forTCPAndTLSRe handshake, which is not conducive to the scenario of mobile devices switching networks, such as4G Switch the network environment toWIFI。

These questions areTCPThe protocol has inherent problems, regardless of the application layerHTTP/2You can’t escape in any design. To solve this problem, we mustReplace transport layer protocol withUDP, this bold decision,HTTP/3 Yes!

QUICCharacteristics of the protocol

We know that,UDPIt is a simple and unreliable transmission protocol, andUDP Packages are unordered and have no dependencies.

and,UDPIt doesn’t need to be connected, so it doesn’t need the process of shaking hands and waving hands, so it’s naturalTCPCome on.

of course,HTTP/3 It’s not just simply replacing the transport protocol withUDP, also based onUDP protocol is implemented in the “application layer”QUICagreement, it has similarTCPThe network characteristics of connection management, congestion window and flow control are equivalent to unreliable transmission UDP The protocol has become “reliable”, so there is no need to worry about packet loss.

QUICThe protocol has many advantages. Here are some examples, such as:

• No head block;
• Faster connection establishment;
• Connection migration;

No queue head blocking

QUICThe agreement is similarHTTP/2 StreamWith the concept of multiplexing, multiple can be transmitted concurrently on the same connectionStream，StreamIt can be regarded as oneHTTP Request.

becauseQUICThe transport protocol used isUDP，UDPDon’t care about the order of packets. If packets are lost,UDPI don’t care.

howeverQUICThe protocol will ensure the reliability of data packets. Each data packet has a serial number and unique identification. When a packet in a stream is lost, even if other packets of the stream arrive, the data cannot be read by HTTP / 3 untilQUICRetransmit the lost message and the data will be handed over to HTTP/3。

As long as the data packets of other streams are completely received,HTTP/3You can read the data. This andHTTP/2Different,HTTP/2As long as a packet in one stream is lost, other streams will also be affected.

So,QUICMultiple on connectionStreamThere is no dependency between them. They are independent. Packet loss of a stream will only affect that stream, and other streams will not be affected.

Faster connection establishment

aboutHTTP/1andHTTP/2 agreement,TCPandTLSIt is layered and belongs to the transport layer implemented by the kernelopenssl The library implements the presentation layer, so they are difficult to merge together. They need to shake hands in batchesTCPShake hands againTLS handshake.

Http / 3 is required before transmitting data QUIC Protocol handshake, this handshake process only needs1 RTT, the purpose of handshake is to confirm the “connection” between the two sidesID“, connection migration is based on connectionIDImplemented.

howeverHTTP/3ofQUICThe agreement is not withTLSLayered, butQUICInternal containsTLS, it will carry in its own frame TLSThe “record” in and quic use TLS1.3Therefore, only 1 is requiredRTTThe connection establishment and key negotiation can be completed “at the same time”. Even during the second connection, the application packet can shake hands with quic (connection information)+TLSSend together to achieve0-RTT Effect of。

As shown in the right part of the figure below,HTTP/3 When the session is resumed, the payload data is sent together with the first packet, which can be achieved 0-RTT：

Connection migration

As we mentioned earlier, based onTCPTransmission protocol HTTPProtocol, because it is through Quad (source)IP, source port, destination IP. destination port) determine one TCPConnect, then when the mobile device’s network from4GSwitch toWIFIWhen, it means IPIf the address changes, you must disconnect and then re-establish the connection, and the process of establishing the connection includes TCP Three handshakes and TLS The delay of four handshakes, andTCPThe slow start deceleration process gives users the feeling that the network is suddenly stuck, so the connection migration cost is very high.

andQUICThe protocol does not “bind” the connection in the form of quads, but throughConnection IDTo mark the two endpoints of communication, the client and server can select a group respectivelyIDTo mark themselves, so even after the mobile device’s network changes, it leads to IP The address changes, as long as the context information (such as the connection) remains ID、TLSKey, etc.), you can “seamlessly” reuse the original connection, eliminate the cost of reconnection, and achieveConnection migrationThe function of.

Http / 3 protocol

Finished understandingQUIC Let’s take a look at the characteristics of the agreementHTTP/3Agreement in HTTPWhat changes have been made in this layer.

HTTP/3withHTTP/2 The structure of binary frame is the same, but the difference isHTTP/2 It needs to be defined in the binary frame of Stream, and HTTP/3 You don’t need to redefine yourselfStream, direct useQUIC Inside StreamSoHTTP/3The structure of the frame is also simplified.

As can be seen from the above figure,HTTP/3The frame header has only two fields: type and length.

According to different frame types, it is generally divided into two categories: data frame and control frame,HEADERSFrame（HTTP Head) andDATA Frame（HTTP Inclusion) belongs to data frame.

HTTP/3 The head compression algorithm has also been upgraded toQPACK。 AndHTTP/2MediumHPACK The coding method is similar,HTTP/3MediumQPACK Static table, dynamic table andHuffman code.

For static table changes,HTTP/2 Medium HPACK The static table has only 61 entries, whileHTTP/3Medium QPACK The static table is expanded to 91 items.

HTTP/2and HTTP/3ofHuffmanThe encoding is not much different, but the encoding and decoding methods of dynamic table are different.

In the so-called dynamic table, after the first request response, both parties will delete the data that is not included in the static tableHeaderItems update their own dynamic table, and then only use one number for subsequent transmission. Then the other party can find the corresponding data from the dynamic table according to this number, so it is not necessary to transmit long data every time, which greatly improves the coding efficiency.

As you can see,The dynamic table is sequential. If the first request loses packets, the subsequent request cannot be decodedHPACKBecause the other party has not established a dynamic table, subsequent request decoding will block the retransmission of the lost packets in the first request。

HTTP/3ofQPACKIf this problem is solved, how is it solved?

QUICThere are two special one-way flows. The so-called single flow can send messages only at one end, and two-way flow means that messages can be sent at both ends for transmissionHTTP Two way flow is used for messages. The usage of these two one-way flows is as follows:

• One is calledQPACK Encoder Stream, used to add a dictionary（key-value）Pass it to the other party, for example, when facing a table that does not belong to a static table HTTPRequest header, the client can send the dictionary through this stream;
• One is called QPACK Decoder Stream, which is used to respond to the other party and tell it that the newly issued dictionary has been updated to its own local dynamic table. This dictionary can be used for coding later.

These two special one-way flows are used toSynchronize dynamic tables on both sides, the encoder will use dynamic table encoding only after receiving the notification of update confirmation from the decoderHTTPHead.

summary

HTTP/2Although it has the ability of concurrent transmission of multiple streams, the transport layer isTCP The agreement has the following defects:

• Team head blocking，HTTP/2Multiple requests running in oneTCPIn connection, if the serial number is lower TCPSegments are lost in network transmission, even if the serial number is higherTCP The segment has been received, and the application layer cannot read this part of data from the kernel. From the perspective of HTTP, multiple requests are blocked;
• TCPandTLSHandshake delay，TCL Three handshakes and TLS Four handshakes, a total of3-RTTTime delay of;
• Connection migration requires reconnection, mobile device from4GSwitch network environment toWIFIDue toTCPIs based on a quad to confirm a TCPIf the network environment changes, it will lead to IPAddress or port changes, soTCPOnly disconnect and then re-establish the connection, and the cost of switching network environment is high;

HTTP/3 The transport layer is removed from theTCPReplaced with UDP, and inUDP Developed on the protocol QUICProtocol to ensure reliable data transmission.

QUICCharacteristics of the agreement:

• No queue head blocking，QUICMultiple on connectionStreamThere is no dependency between them, they are independent, and there will be no underlying protocol restrictions. If a packet loss occurs to a stream, it will only affect the stream, and other streams will not be affected;
• Fast connection establishmentBecauseQUICInternal inclusionTLS1.3Therefore, only 1 is requiredRTT You can “simultaneously” complete the establishment of connections andTLS Key agreement, even during the second connection, the application packet can communicate with QUIC Handshake information (connection information)+TLS Send together to achieve 0-RTTThe effect of.
• Connection migration，QUICThe protocol does not “bind” the connection in the form of quadruple, but through “connect” ID “To mark the two endpoints of communication. The client and server can select a group respectively ID To mark themselves, so even after the mobile device’s network changes, it leads to IP The address changes, as long as the context information (such as the connection) remains ID、TLSKey, etc.), the original connection can be reused “seamlessly” to eliminate the cost of reconnection;

in additionHTTP/3 ofQPACKTwo special one-way flows are used to synchronize the dynamic tables of both sidesHTTP/2ofHPACKTeam head blocking problem.

The following figure comes from another cloud shot, indicatingHTTP/2andHTTP/3Packet loss and its impact when multiplexing two requests:

HTTP/2Multiplex 2 requests. The response is decomposed into multiple packets. Once a packet is lost, both requests are blocked

HTTP/3Reuse 2 requests. Although the light packets are lost, the dark packets transmit well.

reference resources:

[http1.0 HTTP1.1 http2.0 comparison of main features](https://segmentfault.com/a/11…)

What are the major improvements of HTTP / 2 over 1.0?

What do you think of HTTP / 3?