Expect Climbing Pit of HTTP Protocol

Time:2022-8-9

foreword

Today, a very difficult problem is encountered when connecting to an open interface of a third-party platform. Assemble the message according to the interface document, and useHttpClientinitiatePOSTThe request has been timed out, and the other server has not given any response.

The code to initiate the request is as follows:

using (var httpClient = new HttpClient())
{
    var msg = new HttpRequestMessage()
    {
        Content = new StringContent(postJson, Encoding.UTF8, "application/json"),
        Method = HttpMethod.Post,
        RequestUri = new Uri(apiUrl),
    };
    
    // This will block until timeout
    var res =  httpClient.SendAsync(msg).ConfigureAwait(false).GetAwaiter().GetResult();

    if (res.StatusCode != HttpStatusCode.OK)
    {
        throw new Exception(res.StatusCode.ToString());
    }

    return res.Content.ReadAsStringAsync().ConfigureAwait(false).GetAwaiter().GetResult();
}

The asynchronous request timeout cancellation error is as follows:

In this case, first of all doubt whether the other party's service is a problem
However, after confirmation, the other party's service is OK, and use the requestedurlandmessagepaste toPostManMake a request and get a return message quickly, everything is normal.

Ruled out the problem of the other party's service, is that our code problem?
but aboveHttpClientinitiatePostI don't know how many times I wrote the requested code, and it has always been fine. Why can't it work today? I can guarantee that there is nothing wrong with it.

How to solve this situation?

Climbing process

When encountering this kind of problem, compared with most people, they start to exchange various parameters and exchange various libraries, which may eventually become obscured. But here I believePostManCan request success, powerfulHttpClientIt must be possible, it must be which parameter problem, experienced veterans will first think: Is the protocol of the interface correct?HeaderIf there are any special requirements, please check the document here, there are no special requirements.

Control Variable

Since we don't know why and can't guess, thenControl Variableto resolve. All I can think of here is to grab packets, grabPostManThe successful request message and our failed message, compare the difference.

The packet capture tool usesFiddler

Postman message

POST http://xxx.xxx.xxx.xxx:30000/parking/carin/V1 HTTP/1.1
Content-Type: application/json
User-Agent: PostmanRuntime/7.29.2
Accept: */*
Postman-Token: 14547b64-d8f6-4b0b-9fa9-48c9ec74a8f6
Host: xxx.xxx.xxx.xxx:30000
Accept-Encoding: gzip, deflate, br
Connection: keep-alive
Content-Length: 563

{"data": ...the specific json content is omitted here}

HttpClient message

POST http://118.31.110.35:30000/parking/carin/V1 HTTP/1.1
Content-Type: application/json; charset=utf-8
Host: 118.31.110.35:30000
Content-Length: 563
Expect: 100-continue
Connection: Keep-Alive

{"data": ...the specific json content is omitted here}

Check for differences

  1. becausebodyThe contents are the same, so there is no need to compare them here.
  2. If there are differences in the headers of the two requests, we will smooth the differences one by one.
  3. Content-TypeexistHttpClientThere is more charset=utf-8 in the middle, this should not affect, the http protocol is utf8 by default.
  4. User-AgentexistHttpClientIf there is no, then we add exactly the sameUser-Agent, the test still times out.
  5. AcceptexistHttpClientNo, smoothing, testing, still timed out.
  6. Postman-TokenexistHttpClientNo, smoothing, testing, still timed out.
  7. Accept-EncodingexistHttpClientNo, smoothing, testing, still timed out.

Here's what Postman has, weHttpClientAlthough it has been guaranteed that most of the parameters are the same, the control variable method requires all parameters to be the same. There is no guarantee here, because HttpClient has an additional Expect header, and we have not guaranteed the same.

  1. HttpClientin the request headerExpect: 100-continueexistPostmanIt does not exist in the message, remove itExpect, test, success! !
  2. then we lockExpect: 100-continueCaused our request to be unresponsive, restore all the previous smoothing operations, just removeExpect: 100-continue, the test is still successful.

This article is an original article by Gui.H, published on the public account: Beauty of dotnet, reprint and indicate the source

Blog Park debut:https://www.cnblogs.com/sprin…

To finally solve the problem in the preface, just add a line of code

msg.Headers.ExpectContinue = false;

ExpectContinues property documentation:

So far the problem is solved, the control variable yyds

what is Expect

referenceExpectDefinition
https://developer.mozilla.org…

Expectis a request header that contains an expectation that the server can only handle the request properly if the expectation is met.

Expect

There is only one desired condition specified in the specification, namelyExpect: 100-continue, the server can respond as follows:

  • 100If the expected conditions in the message header can be met, so that the request can proceed smoothly,
  • 417(Expectation Failed) If the server cannot meet the expected conditions; it can also be any other status code (4xx) indicating an error on the client side.

For example, if the requestContent-LengthIf the value is too large, it may be rejected by the server.

What's good about Expect

Let the client determine whether the server is willing to receive the data before sending the request data. If the server is willing to receive the data, the client will actually send the data. If the client directly sends the request data, but the server rejects the request, this behavior It will bring a lot of resource overhead.

What's wrong with Expect

Not all servers will respond correctly with 100-continue, such as lighttpd, which will return 417 Expectation Failed.

Reason for timeout

HttpClientcarried by defaultExpecthead, we request to bringExpect: 100-continueIf the message is not sent to the server immediately, the server needs toExpect: 100-continueRespond, but the other server does not supportExpectOf course it can't respond, and in the question mentioned in the preamble, that is,HttpClientWaiting for the other server to respondExpect, and then send the message, and the other server seems to see why we haven't sent the message, both parties are waiting for data, and finallyHttpClientTimeout~

The above is purely personal understanding, if there are any inaccuracies, please correct me~

This article is bymdniceMulti-platform release