HTTP
The full name of is called hypertext transfer protocol. It is an application layer transfer protocol based on the world wide web. At the beginning of its establishment, it was mainly to integrate hypertext markup language(HTML
)Document fromWeb
The server passes to the client’s browser. The original version wasHTTP 0.9
, was produced in the late 1980s, and later upgraded to 1.0 in 1996.
But here it is WEB2.0
Since then, our page has become complex, not only some simple words and pictures, but also ourHTML
The page has CSS
,Javascript
To enrich our page display, whenajax
With the emergence of, we have another method to obtain data from the server, which are actually based onHTTP
Agreed. Similarly, in the era of mobile Internet, our pages can run in the mobile browser, but withPC
Compared with the mobile phone, the network situation is more complex, which makes us have toHTTP
In the process of in-depth understanding and continuous optimization. So it appeared in 1997HTTP1.1
, and then in 2014,HTTP1.1
It’s been updated all the time.
Then in 2015, in order to adapt to the rapid transmissionweb
Applications and the needs of modern browsers inGoogle
ofSPDY
New technologies have been developed on the basis of the projectHTTP2
agreement.
Four years later, in 2019,Google
A new protocol standard has been developedQUIC
Agreement, it isHTTP3
The cornerstone of its purpose is to improve users and websitesAPI
Speed and security of interaction.
HTTP1.0
EarlierHTTP1.0
Version, is aNo state, no connectionApplication layer protocol.
HTTP1.0
It is specified that the browser and the server maintain a short connection, and each request of the browser needs to establish a connection with the serverTCP
Connect and disconnect the server immediately after processingTCP
Connected (no connection), the server does not track each client and does not record past requests (stateless).
TCP
Connection time = 1.5RTT
- One go(
SYN
)- Second circuit(
SYN + ACK
)- Three go(
ACK
)
RTT(Round Trip Time)
It refers to the time of communication
This statelessness can be achieved bycookie/session
Mechanism for identity authentication and status recording. The following two problems are more troublesome.
There is a very important premise to understand these two problems: the client establishes a connection to the server according to the domain name. GenerallyPC
The client browser will target a single domain nameserver
At the same time, 6 ~ 8 connections are established, and the number of connections at the mobile phone end is generally controlled at 4 ~ 6
-
Unable to reuse connection, each request goes through three handshakes and slow start. The impact of three handshakes is more obvious in high latency scenarios, and slow start has a greater impact on large file requests.
TCP
The connection will “tune” itself over time. At first, it will limit the maximum speed of the connection. If the data is successfully transmitted, it will increase the transmission speed over time. This tuning is calledTCP
Slow start. - Team head blocking(
head of line blocking
), becauseHTTP1.0
Specifies that the next request cannot be sent until the previous request response arrives. Assuming that the previous request response does not arrive, the next request will not be sent, and the subsequent requests will be blocked.
To solve these problems,HTTP1.1
There it is.
HTTP1.1
aboutHTTP1.1
, not only inheritedHTTP1.0
Simple characteristics, but also overcome many problemsHTTP1.0
Performance issues.
- Cache processing, in
HTTP1.0
Mainly used inheader
InsideIf-Modified-Since,Expires
As a criterion for cache judgment,HTTP1.1
More cache control strategies are introduced, such asEntity tag
,If-Unmodified-Since
,If-Match, If-None-Match
Wait for more alternative cache headers to control the cache policy. - Bandwidth optimization and network connection usage,
HTTP1.0
In, there are some phenomena of wasting bandwidth. For example, the client only needs a part of an object, but the server sends the whole object, and does not support the function of breakpoint retransmission,HTTP1.1
Is introduced in the request headerrange
Header field, which allows only a certain part of the resource to be requested, that is, the return code is206(Partial Content)
This makes it easy for developers to make free choices to make full use of bandwidth and connections. - Management of error notification, in
HTTP1.1
24 error status response codes are added in the, such as409(Conflict)
Indicates that the requested resource conflicts with the current state of the resource;410(Gone)
Indicates that a resource on the server has been permanently deleted. - Host header processing, in
HTTP1.0
It is considered that each server is bound with a uniqueIP
Address, therefore, in the request messageURL
Host name was not passed(hostname
)。 However, with the development of virtual host technology, there can be multiple virtual hosts on a physical server(Multi-homed Web Servers
)And they share oneIP
Address.HTTP1.1
Both request and response messages should be supportedHost
If there is no header field in the request messageHost
The header field reports an error(400 Bad Request
)。 - Persistent connection,
HTTP1.1
Added oneConnection
Fields, by settingKeep-Alive
Can keepHTTP
The connection is continuously disconnected to avoid repeated establishment and release of establishment every time the client and server requestTCP
Connection, which improves the utilization of the network. If the client wants to shut downHTTP
Connection, which can be carried in the request headerConnection: false
To tell the server to close the request. - Pipelining, based on
HTTP1.1
Long connections, making request pipelining possible. Pipelining enables requests to be transmitted “in parallel”. For example, suppose the subject of the response is ahtml
Page, which contains a lot ofimg
, this timekeep-alive
It plays a great role in “parallel” sending multiple requests.
The “parallel” here is not really parallel transmission becauseThe server must send back the corresponding results according to the order of client requests, so as to ensure that the client can distinguish the response content of each request.
As shown in the figure, the client sends two requests at the same time to obtainhtml
andcss
, let’s say the servercss
The resource is ready first, and the server will send it firsthtml
Resendcss
。
In other words, only whenhtml
After the response resources are completely transmitted,css
Only the resources that respond can start transmission. In other words, it is not allowed to have two at the same timeParallel response。
In addition,pipelining
There are also some defects:
pipelining
Only applicable tohttp1.1
, generally speaking, supporthttp1.1
ofserver
All ask for supportpipelining
;- Only idempotent requests(
GET,HEAD
)Can usepipelining
For example, non idempotent requestsPOST
Cannot be used because there may be a sequence dependency between requests; - Most
http
Proxy server does not supportpipelining
; - And not supported
pipelining
There is a problem with the old server negotiation; - May lead to new
Front of queue blocking
Problems;
soHTTP1.1
Still can’t solve the team head jam(head of line blocking
)Problems. At the same time, “pipelining” technology has various problems, so many browsers either don’t support it at all, or close it directly by default, and the opening conditions are very harsh… And it doesn’t seem to be useful in fact.
What about the parallel requests we see on the Google console?
As shown in the figure, the green part represents a waiting time from the request to the server response, while the blue part represents the download time of the resource. In theory,HTTP
The response should be that the resources of the previous response are downloaded before the resources of the next response can be downloaded. Here, however, the parallel downloading of response resources occurs. Why?
In fact, althoughHTTP1.1
Pipelining is supported, but the server must also respond back one by one, which is a big defect. In fact, browser manufacturers at this stage have taken another approach, which allows us to open multiple browsersTCP
Session. In other words, the parallelism we see in the figure above is actually differentTCP
ConnectedHTTP
Requests and responses. This is the familiar browser’s restriction on loading 6 ~ 8 resources in parallel in the same domain. And this is the realparallel!
HTTP2.0
HTTP 2.0
Compared withHTTP 1.x
, greatly improvedweb
Performance. In andHTTP/1.1
On the basis of full semantic compatibility, the network delay is further reduced. For front-end developers, it undoubtedly reduces the optimization work in the front-end. This article willHTTP 2.0
The agreement summarizes three basic technical points, contacts relevant knowledge and exploresHTTP 2.0
How to improve performance.
begin to display one ‘s talent
HTTP/2: the Future of the InternetThis isAkamai
An official demonstration established by the company to illustrateHTTP/2
Compared with the previousHTTP/1.1
Significant improvement in performance. Request 379 pictures at the same time, fromLoad time
As can be seen from the comparison ofHTTP/2
Advantage in speed.
At this point, if we openChrome Developer Tools
seeNetwork
As you can see in the column,HTTP/2
With respect to network requestsHTTP /1.1
The obvious difference.
HTTP/1:
HTTP/2:
Multiplexing
http1.1
Initially, only 2 domains are supported for the same domain nametcp
, but for performance reasonsrfc
Later, it is modified to use 6-8. In addition,keep-alive
Withhttp pipelining
Essentiallymultiplexing
But because there will behead of line blocking
Problem, mainstream browsers are prohibited by defaultpipelining
, andhttp2.0
It’s really solvedhol
problem
Multiplexing allows simultaneous transmission through a single channelHTTP/2
The connection initiates multiple request response messages.
As we all know, inHTTP/1.1
In the agreement“Browser clients have a certain number of requests under the same domain name at the same time. Requests exceeding the limit will be blocked”。
Clients that use persistent connections SHOULD limit the number of simultaneous connections that they maintain to a given server. A single-user client SHOULD NOT maintain more than 2 connections with any server or proxy. A proxy SHOULD use up to 2*N connections to another server or proxy, where N is the number of simultaneously active users. These guidelines are intended to improve HTTP response times and avoid congestion.
The figure summarizes the number of restrictions imposed by different browsers.
This is why some sites have multiple static resourcesCDN
One of the reasons for domain names is to takeTwitter
For example,http://twimg.com, the purpose is to solve the problem of browser request restriction blocking for the same domain name in a disguised form. andHTTP/2
Multiplexing of(Multiplexing
)It is allowed to pass through a single at the same timeHTTP/2
The connection initiates multiple request response messages.
thereforeHTTP/2
It is easy to implement multi stream parallelism without relying on the establishment of multiple streamsTCP
connect,HTTP/2
holdHTTP
The basic unit of protocol communication is reduced to frames one by one, which correspond to messages in the logical flow. In parallel on the sameTCP
Two way message exchange on the connection.
Binary framing
Without changeHTTP/1.x
Semantics, methods, status codesURI
And the header field,HTTP/2
How to achieve “breakthrough”HTTP1.1
Performance constraints, improve transmission performance and achieve “low latency and high throughput”?
One of the keys is in the application layer(HTTP/2
)And transport layer(TCP or UDP
)Add a binary framing layer between.
In the binary framing layer,HTTP/2
All transmitted information is divided into smaller messages and frames(frame
)And encode them in binary format, whereHTTP1.x
The first part of the information will be encapsulated inHEADER frame
, and the correspondingRequest Body
Then package toDATA frame
Inside.
Here are some concepts:
- Flow(
stream
): bidirectional byte stream on established connection. - Message: a complete series of data frames corresponding to the logical message.
- Frame(
frame
):HTTP2.0
The smallest unit of communication. Each frame contains a frame header and at least identifies the stream to which the current frame belongs(stream id
)
HTTP/2
Communication is done on a connection that can carry any number of two-way data streams.
Each data stream is sent in the form of a message, which consists of one or more frames. These frames can be sent out of order, and then according to the stream identifier of each frame header(stream id
)Reassemble.
For example, each request is a data stream, which is sent as a message, and the message is divided into multiple frames, which are recorded in the frame headerstream id
It is used to identify the data stream to which it belongs. Frames of different genera can be randomly mixed together in the connection. The receiving party maystream id
The frames are assigned to different requests.
In addition, multiplexing (connection sharing) may cause critical requests to be blocked.HTTP2.0
Priority and dependency can be set for each data stream in the. The data stream with high priority will be preferentially processed by the server and returned to the client. The data stream can also rely on other sub data streams.
soHTTP2.0
The real parallel transmission is realized, and it can be in oneTCP
Any number onHTTP
Request. This powerful function is based on the characteristics of “binary framing”.
Summary:
- Single connection with multiple resources can reduce the link pressure on the server, occupy less memory and increase the connection throughput
- because
TCP
The reduction of connection improves the network congestion, and the reduction of slow start-up time makes the recovery speed of congestion and packet loss faster
Header compression
HTTP/1.1
Not supportedHTTP
Header compression, for this purposeSPDY
andHTTP/2
emerge as the times require,SPDY
The is universalDEFLATEAlgorithm, andHTTP/2
Is designed specifically for header compressionHPACKAlgorithm.
stayHTTP1.x
In, header metadata is sent in plain text, which usually adds 500 ~ 800 bytes to each request.
for instancecookie
, by default, the browser will set thecookie
Attached toheader
Send it to the server. (due tocookie
It is relatively large and is sent repeatedly every time. Generally, it does not store information, but is only used for status recording and identity authentication)
HTTP2.0
useencoder
To reduce the need for transmissionheader
Size, communication partiescache
A copyheader fields
Table, which avoids duplicationheader
The size of transmission is reduced. An efficient compression algorithm can greatly compressheader
, reduce the number of packets sent, thereby reducing the delay.
Server push
Server push is a mechanism that sends data before the client requests. stayHTTP/2
In, the server can send multiple responses to a request from the client.Server Push
Give WayHTTP1.x
In this era, the optimization means of using embedded resources has become meaningless; If a request is initiated by your home page, the server is likely to respond to the content of the home pagelogo
And style sheets, because it knows that the client will use these things. This is equivalent to in a HTML
All resources are collected in the document, but compared with it, server push has another great advantage: it can be cached! It also makes it possible for different pages to share cache resources under the condition of following homology.
HTTP3
Http / 3 has not been officially launched yet, but since 2017, http / 3 has been updated to 34 drafts, the basic features have been determined, and the package format may change in the future.
Therefore, this http / 3 introduction will not involve the package format, only its features.
Http / 2 with a fly in the ointment
HTTP/2
Through new features such as header compression, binary coding, multiplexing and server push, it has been greatly improvedHTTP/1.1
Performance, and the fly in the ointment isHTTP/2
The protocol is based onTCP
So there are three defects.
- Team head obstruction;
TCP
AndTLS
Delay in handshake;- Network migration requires reconnection;
Team head blocking
HTTP/2
Multiple requests are running in oneTCP
In the connection, then whenTCP
In case of packet loss, the wholeTCP
All have to wait for retransmission, then the message will be blockedTCP
All requests in the connection.
becauseTCP
Is a byte stream protocol,TCP
The layer must ensure that the received byte data is complete and orderly if the serial number is lowTCP
Segments are lost in network transmission, even if the serial number is higherTCP
The segment has been received, and the application layer cannot read this part of data from the kernelHTTP
From the perspective, the request is blocked.
Delay in handshake between TCP and TLS
launchHTTP
When requesting, you need to go throughTCP
Three handshakes andTLS
Four handshakes(TLS 1.2
)Therefore, a total of 3 are requiredRTT
To send the requested data.
In addition,TCP
Due to the “congestion control” feature, the connection has just been establishedTCP
There will be a “slow start” process, which will be rightTCP
The connection produces a “deceleration” effect.
Network migration requires reconnection
OneTCP
The connection is composed of four tuples (source)IP
Address, source port, destination IP
Address, target port), which means that if IP
If the address or port changes, it will lead to the need forTCP
AndTLS
Re handshake, which is not conducive to the scenario of mobile devices switching networks, such as4G
Switch the network environment toWIFI
。
These questions areTCP
The protocol has inherent problems, regardless of the application layerHTTP/2
You can’t escape in any design. To solve this problem, we mustReplace transport layer protocol withUDP
, this bold decision,HTTP/3
Yes!
QUIC
Characteristics of the protocol
We know that,UDP
It is a simple and unreliable transmission protocol, andUDP
Packages are unordered and have no dependencies.
and,UDP
It doesn’t need to be connected, so it doesn’t need the process of shaking hands and waving hands, so it’s naturalTCP
Come on.
of course,HTTP/3
It’s not just simply replacing the transport protocol withUDP
, also based onUD
P protocol is implemented in the “application layer”QUIC
agreement, it has similarTCP
The network characteristics of connection management, congestion window and flow control are equivalent to unreliable transmission UDP
The protocol has become “reliable”, so there is no need to worry about packet loss.
QUIC
The protocol has many advantages. Here are some examples, such as:
- No head block;
- Faster connection establishment;
- Connection migration;
No queue head blocking
QUIC
The agreement is similarHTTP/2 Stream
With the concept of multiplexing, multiple can be transmitted concurrently on the same connectionStream
,Stream
It can be regarded as oneHTTP
Request.
becauseQUIC
The transport protocol used isUDP
,UDP
Don’t care about the order of packets. If packets are lost,UDP
I don’t care.
howeverQUIC
The protocol will ensure the reliability of data packets. Each data packet has a serial number and unique identification. When a packet in a stream is lost, even if other packets of the stream arrive, the data cannot be read by HTTP / 3 untilQUIC
Retransmit the lost message and the data will be handed over to HTTP/3
。
As long as the data packets of other streams are completely received,HTTP/3
You can read the data. This andHTTP/2
Different,HTTP/2
As long as a packet in one stream is lost, other streams will also be affected.
So,QUIC
Multiple on connectionStream
There is no dependency between them. They are independent. Packet loss of a stream will only affect that stream, and other streams will not be affected.
Faster connection establishment
aboutHTTP/1
andHTTP/2
agreement,TCP
andTLS
It is layered and belongs to the transport layer implemented by the kernelopenssl
The library implements the presentation layer, so they are difficult to merge together. They need to shake hands in batchesTCP
Shake hands againTLS
handshake.
Http / 3 is required before transmitting data QUIC
Protocol handshake, this handshake process only needs1 RTT
, the purpose of handshake is to confirm the “connection” between the two sidesID
“, connection migration is based on connectionID
Implemented.
howeverHTTP/3
ofQUIC
The agreement is not withTLS
Layered, butQUIC
Internal containsTLS
, it will carry in its own frame TLS
The “record” in and quic use TLS1.3
Therefore, only 1 is requiredRTT
The connection establishment and key negotiation can be completed “at the same time”. Even during the second connection, the application packet can shake hands with quic (connection information)+TLS
Send together to achieve0-RTT
Effect of。
As shown in the right part of the figure below,HTTP/3
When the session is resumed, the payload data is sent together with the first packet, which can be achieved 0-RTT
:
Connection migration
As we mentioned earlier, based onTCP
Transmission protocol HTTP
Protocol, because it is through Quad (source)IP
, source port, destination IP
. destination port) determine one TCP
Connect, then when the mobile device’s network from4G
Switch toWIFI
When, it means IP
If the address changes, you must disconnect and then re-establish the connection, and the process of establishing the connection includes TCP
Three handshakes and TLS
The delay of four handshakes, andTCP
The slow start deceleration process gives users the feeling that the network is suddenly stuck, so the connection migration cost is very high.
andQUIC
The protocol does not “bind” the connection in the form of quads, but throughConnection IDTo mark the two endpoints of communication, the client and server can select a group respectivelyID
To mark themselves, so even after the mobile device’s network changes, it leads to IP
The address changes, as long as the context information (such as the connection) remains ID
、TLS
Key, etc.), you can “seamlessly” reuse the original connection, eliminate the cost of reconnection, and achieveConnection migrationThe function of.
Http / 3 protocol
Finished understandingQUIC
Let’s take a look at the characteristics of the agreementHTTP/3
Agreement in HTTP
What changes have been made in this layer.
HTTP/3
withHTTP/2
The structure of binary frame is the same, but the difference isHTTP/2
It needs to be defined in the binary frame of Stream
, and HTTP/3
You don’t need to redefine yourselfStream
, direct useQUIC
Inside Stream
SoHTTP/3
The structure of the frame is also simplified.
As can be seen from the above figure,HTTP/3
The frame header has only two fields: type and length.
According to different frame types, it is generally divided into two categories: data frame and control frame,HEADERS
Frame(HTTP
Head) andDATA
Frame(HTTP
Inclusion) belongs to data frame.
HTTP/3
The head compression algorithm has also been upgraded toQPACK
。 AndHTTP/2
MediumHPACK
The coding method is similar,HTTP/3
MediumQPACK
Static table, dynamic table andHuffman
code.
For static table changes,HTTP/2
Medium HPACK
The static table has only 61 entries, whileHTTP/3
Medium QPACK
The static table is expanded to 91 items.
HTTP/2
and HTTP/3
ofHuffman
The encoding is not much different, but the encoding and decoding methods of dynamic table are different.
In the so-called dynamic table, after the first request response, both parties will delete the data that is not included in the static tableHeader
Items update their own dynamic table, and then only use one number for subsequent transmission. Then the other party can find the corresponding data from the dynamic table according to this number, so it is not necessary to transmit long data every time, which greatly improves the coding efficiency.
As you can see,The dynamic table is sequential. If the first request loses packets, the subsequent request cannot be decodedHPACK
Because the other party has not established a dynamic table, subsequent request decoding will block the retransmission of the lost packets in the first request。
HTTP/3
ofQPACK
If this problem is solved, how is it solved?
QUIC
There are two special one-way flows. The so-called single flow can send messages only at one end, and two-way flow means that messages can be sent at both ends for transmissionHTTP
Two way flow is used for messages. The usage of these two one-way flows is as follows:
- One is called
QPACK Encoder Stream
, used to add a dictionary(key-value
)Pass it to the other party, for example, when facing a table that does not belong to a static tableHTTP
Request header, the client can send the dictionary through this stream; - One is called
QPACK Decoder Stream
, which is used to respond to the other party and tell it that the newly issued dictionary has been updated to its own local dynamic table. This dictionary can be used for coding later.
These two special one-way flows are used toSynchronize dynamic tables on both sides, the encoder will use dynamic table encoding only after receiving the notification of update confirmation from the decoderHTTP
Head.
summary
HTTP/2
Although it has the ability of concurrent transmission of multiple streams, the transport layer isTCP
The agreement has the following defects:
- Team head blocking,
HTTP/2
Multiple requests running in oneTCP
In connection, if the serial number is lowerTCP
Segments are lost in network transmission, even if the serial number is higherTCP
The segment has been received, and the application layer cannot read this part of data from the kernel. From the perspective of HTTP, multiple requests are blocked; TCP
andTLS
Handshake delay,TCL
Three handshakes andTLS
Four handshakes, a total of3-RTT
Time delay of;- Connection migration requires reconnection, mobile device from
4G
Switch network environment toWIFI
Due toTCP
Is based on a quad to confirm aTCP
If the network environment changes, it will lead toIP
Address or port changes, soTCP
Only disconnect and then re-establish the connection, and the cost of switching network environment is high;
HTTP/3
The transport layer is removed from theTCP
Replaced with UDP
, and inUDP
Developed on the protocol QUIC
Protocol to ensure reliable data transmission.
QUIC
Characteristics of the agreement:
- No queue head blocking,
QUIC
Multiple on connectionStream
There is no dependency between them, they are independent, and there will be no underlying protocol restrictions. If a packet loss occurs to a stream, it will only affect the stream, and other streams will not be affected; - Fast connection establishmentBecause
QUIC
Internal inclusionTLS1.3
Therefore, only 1 is requiredRTT
You can “simultaneously” complete the establishment of connections andTLS
Key agreement, even during the second connection, the application packet can communicate withQUIC
Handshake information (connection information)+TLS
Send together to achieve0-RTT
The effect of. - Connection migration,
QUIC
The protocol does not “bind” the connection in the form of quadruple, but through “connect”ID
“To mark the two endpoints of communication. The client and server can select a group respectivelyID
To mark themselves, so even after the mobile device’s network changes, it leads toIP
The address changes, as long as the context information (such as the connection) remainsID
、TLS
Key, etc.), the original connection can be reused “seamlessly” to eliminate the cost of reconnection;
in additionHTTP/3
ofQPACK
Two special one-way flows are used to synchronize the dynamic tables of both sidesHTTP/2
ofHPACK
Team head blocking problem.
The following figure comes from another cloud shot, indicatingHTTP/2
andHTTP/3
Packet loss and its impact when multiplexing two requests:
HTTP/2
Multiplex 2 requests. The response is decomposed into multiple packets. Once a packet is lost, both requests are blocked
HTTP/3
Reuse 2 requests. Although the light packets are lost, the dark packets transmit well.
reference resources:
[http1.0 HTTP1.1 http2.0 comparison of main features](https://segmentfault.com/a/11…)