Lua web rapid development guide (7) – efficient interface call – httpc Library

Time:2019-12-7

Httpc library is based on HTTP client library written by socket which is implemented in CF framework

The httpc library has built-in SSL support, which enables you to request third-party interfaces without using a proxy

Httpc supports header, args, body and timeout request settings, and perfectly supports various httpc call modes

API introduction

Httpc library needs to be imported manually before use:local httpc = require "httpc".

httpc.get(domain, HEADER, ARGS, TIMEOUT)

Calling the get method will make an HTTP get request to the domain

Domain is a string conforming to the URL definition specification;

Header is a key value array, which is generally used to add custom headers;

Args is the key value array of the request parameter. For get method, it will be automatically formatted as:args[n][1]=args[n][2]&args[n+1][1]=args[n+1][2];

Timeout is the maximum timeout of httpc request;

httpc.post(domain, HEADER, BODY, TIMEOUT)

Calling the post method will send an HTTP post request to the domain. Thecontent-typeWill be set to:application/x-www-form-urlencoded.

Domain is a string conforming to the URL definition specification;

Header is a key value array, which is generally used to add custom headers; content type and content length settings are not supported;

Body is a key value array. For post method, it will be automatically formatted as:body[n][1]=body[n][2]&body[n+1][1]=body[n+1][2];

Timeout is the maximum timeout of httpc request;

httpc.json(domain, HEADER, JSON, TIMEOUT)

The JSON method will make an HTTP post request to the domaincontent-typeWill be set to:application/json.

Header is a key value array, which is generally used to add custom headers; content type and content length settings are not supported;

JSON must be a string type;

Timeout is the maximum timeout of httpc request;

httpc.file(domain, HEADER, FILES, TIMEOUT)

The file method will make an HTTP post request to the domain

Header is a key value array, which is generally used to add custom headers; content type and content length settings are not supported;

Files is a key value array. Each item contains: name, filename, file content, type and other attributes. The file type is optional

Timeout is the maximum timeout of httpc request;

Httpc return value

All httpc request interfaces will have two return values:code, response. code is HTTP protocol status code, and response is response body (string type)

If the parameter is incorrect, the connection is disconnected and other errors occur, the code will be nil, and the response will be the error message

“One time HTTP request”

What is a one-time httpc request?

Every time we use httpc library to request the third-party HTTP interface, we will close the connection after the interface returns. This is not a problem in daily use

But when we need to request the same interface multiple times, it is not so efficient to close the connection after each request. Now we try to use an HTTP class object to solve this problem

Note: the httpc class object cannot use the same connection for the interface of different domain names, which will return an error call to the user

An introduction to the use of class object in httpc Library

To use httpcclassThe class library of httpc needs to be imported. The import method is:local httpc = require "httpc.class".

Before using httpc to initiate a request, you need to create an httpc object, such as:local hc = httpc:new {}. after the creation and initialization of httpc object, the usage is the same as the above API

hcAndhttpcIt has the same API, but needs to use different calling methods. For example:hc:gethc:posthc:jsonhc:file.

oncehcCall to display after usehc:close()Method to close the created httpc object and destroy the httpc connection

Begin to practice

Now, let’s apply the API usage we learned above to practice

1. Start a web server of httpd Library

staymain.luaStart one inhttpdServer.

local httpd = require "httpd"
local json = require "json"

local app = httpd:new("httpd")


app:listen("", 8080)

app:run()

1. Add an API route for IP address home location query

Let’s use it firsthttpdThe database starts a server service and provides an IP home location query interface

app:api('/ip', function(content)
    local httpc = require "httpc"
    local args = content.args
    if not args or not args['ip'] then
        return json.encode({
            code = 400,
            MSG = "bad interface call method",
            data = json.null,
            })
    end
    local code, response = httpc.get("http://freeapi.ipip.net/"..args["ip"])
    if code ~= 200 then
        return json.encode({
            code = 401,
            MSG = "failed to get data",
            data = json.null,
            })
    end
    return response
end)

Now the code is finished! Let’s open the browser and input:http://localhost:8090/ip?ip=8.8.8.8View the return data

2. Query the home location of multiple IP addresses

One response for one request is the essence of HTTP protocol! However, we often encounter the business scenario of batch request, so we can design an example of batch request / return

Let’s assume that the client will send a post request. The body is JSON type and contains an IP array: ip_list = {1.1.1.1, 8.8.8.8, 114.114.114.114}

After receiving the array, the server needs to return the IP home information to the client at one time

app:api('/ips', function(content)
    local httpc = require "httpc.class"
    if not content.json then
        return json.encode({
            code = 400,
            MSG = "bad call parameters",
            data = json.null,
        })
    end
    local args = json.decode(content.body)
    if type(args) ~= 'table' or type(args['ip_list']) ~= 'table' then
        return json.encode({
            code = 400,
            MSG = "bad parameter type",
            data = json.null,
        })
    end
    local hc = httpc:new {}
    local ret = { code = 200 , data = {}}
    for _, ip in ipairs(args['ip_list']) do
        local code, response = hc:get("http://freeapi.ipip.net/"..ip)
        ret['data'][#ret['data']+1] = json.decode(response)
    end
    hc:close()
    return json.encode(ret)
end)

Because normal browser post can’t send JSON, let’s usecurlCommand line tools to test:

curl -H "Content-Type: application/json" -X POST -d '{"ip_list":["1.1.1.1","8.8.8.8","114.114.114.114"]}' http://localhost:8090/ip

The returned data is as follows:

{"code":200,"data":[["CLOUDFLARE.COM","CLOUDFLARE.COM","","",""],["GOOGLE.COM","GOOGLE.COM","","","level3.com"],["114DNS.COM","114DNS.COM","","",""]]}

3. Continuous optimization

The above example seems to be perfect! We have made three requests by using the method of connection holding, which has reduced the connection consumption (TCP handshake) of the request by 50%

But for those of us who need performance very much: each request needs to wait until the last request is processed before starting a new request, which is obviously not enough for us

In this case,httpcThe library provides a callmulti_requestThe specific usage is here

This method allows us to send hundreds of requests at the same time to solve the problem of single connection blocking

4. Concurrent requests

Now, let me usehttpcLibrarymulti_requestMethod to request multiple interfaces simultaneously, reducing the problems caused by connection blocking

app:api('/ips_multi', function (content)
    local httpc = require "httpc"
    if not content.json then
        return json.encode({
            code = 400,
            MSG = "bad call parameters",
            data = json.null,
        })
    end
    local args = json.decode(content.body)
    if type(args) ~= 'table' or type(args['ip_list']) ~= 'table' then
        return json.encode({
            code = 400,
            MSG = "bad parameter type",
            data = json.null,
        })
    end
    local requests = {}
    local responses = { code = 200, data = {}}
    for _, ip in ipairs(args["ip_list"]) do
        requests[#requests+1] = {
            domain = "http://freeapi.ipip.net/"..ip,
            method = "get",
        }
    end
    local ok, ret = httpc.multi_request(requests)
    for _, res in ipairs(ret) do
        responses['data'][#responses['data'] + 1] = res
    end
    return json.encode(responses)
end)

OK, now let’s use it againcurlTools to test:

curl -H "Content-Type: application/json" -X POST -d '{"ip_list":["1.1.1.1","8.8.8.8","114.114.114.114"]}' http://localhost:8090/ips_multi

We can see from the request response time of CF that the response time consumption is reduced by 50% again

[[email protected]:~/Documents/core_framework] $ ./cfadmin
[2019 / 06 / 16 17:45:21] [info] httpd is listening: 0.0.0.0:8090
[2019 / 06 / 16 17:45:21] [info] httpd is running web server service
[2019/06/16 17:45:23] - ::1 - ::1 - /ips_multi - POST - 200 - req_time: 0.140253/Sec
[2019/06/16 17:45:38] - ::1 - ::1 - /ips - POST - 200 - req_time: 0.288286/Sec

Complete code

local httpd = require "httpd"
local json = require "json"

local app = httpd:new("httpd")

app:api('/ip', function(content)
    local httpc = require "httpc"
    local args = content.args
    if not args or not args['ip'] then
        return json.encode({
            code = 400,
            MSG = "bad interface call method",
            data = json.null,
            })
    end
    local code, response = httpc.get("http://freeapi.ipip.net/"..args["ip"])
    if code ~= 200 then
        return json.encode({
            code = 401,
            MSG = "failed to get data",
            data = json.null,
            })
    end
    return response
end)

app:api('/ips', function(content)
    local httpc = require "httpc.class"
    if not content.json then
        return json.encode({
            code = 400,
            MSG = "bad call parameters",
            data = json.null,
        })
    end
    local args = json.decode(content.body)
    if type(args) ~= 'table' or type(args['ip_list']) ~= 'table' then
        return json.encode({
            code = 400,
            MSG = "bad parameter type",
            data = json.null,
        })
    end
    local hc = httpc:new {}
    local ret = { code = 200 , data = {}}
    for _, ip in ipairs(args['ip_list']) do
        local code, response = hc:get("http://freeapi.ipip.net/"..ip)
        ret['data'][#ret['data']+1] = json.decode(response)
    end
    hc:close()
    return json.encode(ret)
end)

app:api('/ips_multi', function (content)
    local httpc = require "httpc"
    if not content.json then
        return json.encode({
            code = 400,
            MSG = "bad call parameters",
            data = json.null,
        })
    end
    local args = json.decode(content.body)
    if type(args) ~= 'table' or type(args['ip_list']) ~= 'table' then
        return json.encode({
            code = 400,
            MSG = "bad parameter type",
            data = json.null,
        })
    end
    local requests = {}
    local responses = { code = 200, data = {}}
    for _, ip in ipairs(args["ip_list"]) do
        requests[#requests+1] = {
            domain = "http://freeapi.ipip.net/"..ip,
            method = "get",
        }
    end
    local ok, ret = httpc.multi_request(requests)
    for _, res in ipairs(ret) do
        responses['data'][#responses['data'] + 1] = res
    end
    return json.encode(responses)
end)

app:listen("", 8090)

app:run()

Continue learning

In the next chapter, we will learn how to write websocket using httpd library