Netty series: building an HTTP upload file server

Time:2021-12-1

brief introduction

In the last article, we talked about how to download files from the HTTP server, the problems that should be paid attention to in building the download file server, and the get method used. This article will discuss the common post method of submitting data to the server and how to upload files to the server.

Upload data with get method

According to the HTTP specification, put generally uploads data to the server. Although it is not advocated, get can also be used to upload data to the server.

Let’s take a look at the problems needing attention in the construction of get client.

The get request is actually a URI, followed by the requested parameters. Netty provides a querystringencoder specifically used to build the parameter contents:

//HTTP request
        QueryStringEncoder encoder = new QueryStringEncoder(get);
        //Add request parameters
        encoder.addParam("method", "GET");
        encoder.addParam("name", "flydean");
        encoder.addParam("site", "www.flydean.com");
        URI uriGet = new URI(encoder.toString());

With the request URI, you can create an httprequest. Of course, this httprequest also needs to have the corresponding HTTP head data:

HttpRequest request = new DefaultHttpRequest(HttpVersion.HTTP_1_1, HttpMethod.GET, uriGet.toASCIIString());
        HttpHeaders headers = request.headers();
        headers.set(HttpHeaderNames.HOST, host);
        headers.set(HttpHeaderNames.CONNECTION, HttpHeaderValues.CLOSE);
        headers.set(HttpHeaderNames.ACCEPT_ENCODING, HttpHeaderValues.GZIP + "," + HttpHeaderValues.DEFLATE);
        headers.set(HttpHeaderNames.ACCEPT_LANGUAGE, "zh-CN,zh;q=0.8,zh-TW;q=0.7,zh-HK;q=0.5,en-US;q=0.3,en;q=0.2");
        headers.set(HttpHeaderNames.REFERER, uriSimple.toString());
        headers.set(HttpHeaderNames.USER_AGENT, "Netty Simple Http Client side");
        headers.set(HttpHeaderNames.ACCEPT, "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");

        headers.set(
                HttpHeaderNames.COOKIE, ClientCookieEncoder.STRICT.encode(
                        new DefaultCookie("name", "flydean"),
                        new DefaultCookie("site", "www.flydean.com"))
        );

We know that there are only two parts of data in httprequest: httpversion and httpheaders. Httpversion is the version number of the HTTP protocol, and httpheaders is the set header content.

For get requests, because all the contents are contained in the URI, there is no need for additional HTTP content. Just send httprequest directly to the server.

channel.writeAndFlush(request);

Then, let’s look at how the server handles the get request after receiving it.

After the server receives the MSG of the httpobject object object, it needs to convert it into an httprequest object, and you can get the corresponding information through protocolversion(), uri() and headers().

For the parameters in the URI, netty provides the querystringdecoder class, which can easily resolve the parameters in the URI:

//Resolve parameters in URL
            QueryStringDecoder decoderQuery = new QueryStringDecoder(request.uri());
            Map<String, List<String>> uriAttributes = decoderQuery.parameters();
            for (Entry<String, List<String>> attr: uriAttributes.entrySet()) {
                for (String attrVal: attr.getValue()) {
                    responseContent.append("URI: ").append(attr.getKey()).append('=').append(attrVal).append("\r\n");
                }
            }

Upload data by post method

For the post request, it has one more httpcontent than the get request, that is, in addition to the basic httprequest data, it also needs a postbody.

If it is just an ordinary post, that is, the content of the post is in the form of key = value, it is relatively simple. If the post contains files, it will be more complex. You need to use enctype = “multipart / form data”.

Netty provides an httppostrequestencoder class to quickly encode the request body. First look at the complete constructor of the httppostrequestencoder class:

public HttpPostRequestEncoder(
            HttpDataFactory factory, HttpRequest request, boolean multipart, Charset charset,
            EncoderMode encoderMode)

Where request is the httprequest to be encoded, multipart indicates whether it is in the format of “multipart / form data”, charset encoding method, and charsetutil.utf by default_ 8。 Encodermode is the encoding mode. At present, there are three encoding modes: rfc1738, rfc3986 and HTML5.

The default encoding mode is rfc1738, which is also the encoding mode for most forms to submit data. However, it is not applicable to OAuth. If OAuth is to be used, rfc3986 can be used. HTML5 disables the mixed mode of multipart / form data.

Finally, let’s talk about httpdatafactory. Factory is mainly used to create interfacehttpdata. It has a minsize parameter. If the created httpdata size is greater than minsize, it will be stored on disk. Otherwise, it will be created directly in memory.

Interfacehttpdata has three types of httpdata: attribute, fileUpload and internalattribute.

Attribute is the attribute value passed in the post request. FileUpload is the file passed in the post request, and the internalattribute is used inside the encoder, which is not discussed here.

Therefore, attribute and fileUpload can be divided into the following types according to the size of the minsize parameter passed in:

MemoryAttribute, DiskAttribute or MixedAttribute
MemoryFileUpload, DiskFileUpload or MixedFileUpload

In this section, let’s first look at the processing method of not uploading files in post requests. First, create HTTP request and postbody encoder:

//Build HTTP request
        HttpRequest request = new DefaultHttpRequest(HttpVersion.HTTP_1_1, HttpMethod.POST, uriSimple.toASCIIString());

 HttpPostRequestEncoder bodyRequestEncoder =
                new HttpPostRequestEncoder(factory, request, false);

Add headers to the request:

//Add headers
        for (Entry<String, String> entry : headers) {
            request.headers().set(entry.getKey(), entry.getValue());
        }

Then add the form attribute to the bodyrequestencoder:

//Add form attribute
        bodyRequestEncoder.addBodyAttribute("method", "POST");
        bodyRequestEncoder.addBodyAttribute("name", "flydean");
        bodyRequestEncoder.addBodyAttribute("site", "www.flydean.com");
        bodyRequestEncoder.addBodyFileUpload("myfile", file, "application/x-zip-compressed", false);

Note that we added the method, name and site attributes to the bodyrequestencoder above. Then a fileUpload is added. However, because our encoding method is not “multipart / form data”, what is passed here is only the file name, not the whole file.

Finally, we will call the finalizerequest method of bodyrequestencoder to return the final request to be sent. In the process of finalizerequest, you can also set whether transfer encoding is chunked according to the size of the transferred data.

If the transmitted content is large, it needs to be transmitted in segments. At this time, transfer encoding = chunked needs to be set, otherwise it will not be set.

Last send request:

//Send request
        channel.write(request);

On the server side, we also need to construct an httpdatafactory, and then use this factory to construct an httppostrequestdecoder to decode the data from the encoder:

HttpDataFactory factory =
            new DefaultHttpDataFactory(DefaultHttpDataFactory.MINSIZE);
//Post request
decoder = new HttpPostRequestDecoder(factory, request);

Because the message received by the server can be httpcontent or lasthttpcontent according to the length of the sent message. If it is httpcontent, we will cache the parsing results in a StringBuilder and send them together after receiving lasthttpcontent.

After receiving httpcontent, we call the decoder.offer method to decode httpcontent:

decoder.offer(chunk);

There are two containers inside the decoder that store httpdata data, namely:

List<InterfaceHttpData> bodyListHttpData
and
Map<String, List<InterfaceHttpData>> bodyMapHttpData

Decoder.offer is to parse chunk, and then fill the parsed data into bodylisthttpdata and bodymaphttpdata.

After parsing, you can read the parsed data.

You can traverse the bodylisthttpdata through the hasnext and next methods of the decoder to obtain the corresponding interfacehttpdata.

The data type of interfacehttpdata can be obtained through data. Gethttpdatatype(). As mentioned above, there are two types: attribute and fileUpload.

Upload files by post method

If you want to post a file, the client can pass multipart = true when creating httppostrequestencoder:

 HttpPostRequestEncoder bodyRequestEncoder =
                new HttpPostRequestEncoder(factory, request, true);

Then call setbodyhttpdata and finalizerequest methods respectively to generate httprequest, and then write to the channel:

//Add body HTTP data
        bodyRequestEncoder.setBodyHttpDatas(bodylist);
        //Finalize request to determine whether chunk is required
        request = bodyRequestEncoder.finalizeRequest();
        //Send request header
        channel.write(request);

Note that if transfer encoding = chunked, the httprequest is only the information of the request header. We also need to manually write the httpcontent into the channel:

//Judge whether the bodyrequestencoder is chunked and send the request content
        if (bodyRequestEncoder.isChunked()) {
            channel.write(bodyRequestEncoder);
        }

On the server side, judge the gethttpdatatype of interfacehttpdata. If it is fileUpload type, it means that the uploaded file is obtained, and the contents of the file can be read through the following methods:

FileUpload fileUpload = (FileUpload) data;
responseContent.append(fileUpload.getString(fileUpload.getCharset()));

In this way, we can get the files transmitted from the client on the server.

summary

Http file upload needs to consider many problems. If you don’t understand, you can refer to my example. Or leave a message for me to discuss.

Examples of this article can be referred to:learn-netty4

This article has been included inhttp://www.flydean.com/21-netty-http-fileupload/

The most popular interpretation, the most profound dry goods, the most concise tutorial, and many tips you don’t know are waiting for you to find!

Welcome to my official account: “those things in procedure”, understand technology, know you better!

Recommended Today

Pkce of OAuth 2.0 extension protocol

Hello everyone, I’m the lab researcher of this issue – wait until dark. Today, our research object is OAuth extension protocol pkce. In draft OAuth 2.1, the authorization code + pkce authorization mode is recommended. Why is pkce so important? Next, let’s go to the lab and find out! preface The full name of pkce […]