Detailed explanation of curl command and @& special character processing

Time:2021-8-31

catalogue

  1. Examples of common usage
  2. Processing of special characters

1、 Examples of 10 common uses of curl

preface

In Linux, curl is a file transfer tool that uses URL rules to work on the command line. It can be said that curl is a very powerful HTTP command line tool. It supports file upload and download. It is a comprehensive transmission tool, but traditionally, it is used to call URL a download tool.

Syntax:# curl [option] [url]

This article mainly shares 10 common usages of Linux command line curl for your reference. Let’s take a look at the detailed introduction below:

1. Get page content

When we use curl without any options, we will send a get request to get the linked content to the standard output by default.

curl http://www.codebelief.com

2. Display HTTP header

If we only want to display the HTTP header without displaying the file content, we can use the – I option:

curl -I http://www.codebelief.com

Output is:

HTTP/1.1 200 OK 
Server: nginx/1.10.3 
Date: Thu, 11 May 2017 08:24:45 GMT 
Content-Type: text/html; charset=utf-8 
Content-Length: 24206 
Connection: keep-alive 
X-Powered-By: Express 
Cache-Control: public, max-age=0 
ETag: W/"5e8e-Yw5ZdnVVly9/aEnMX7fVXQ" 
Vary: Accept-Encoding

You can also display HTTP headers and file contents at the same time. Use the – I option:

curl -i http://www.codebelief.com

Output is:

HTTP/1.1 200 OK 
Server: nginx/1.10.3 
Date: Thu, 11 May 2017 08:25:46 GMT 
Content-Type: text/html; charset=utf-8 
Content-Length: 24206 
Connection: keep-alive 
X-Powered-By: Express 
Cache-Control: public, max-age=0 
ETag: W/"5e8e-Yw5ZdnVVly9/aEnMX7fVXQ" 
Vary: Accept-Encoding

 
 
......

3. Save Link to file

We can use the > symbol to redirect the output to a local file.

curl http://www.codebelief.com > index.html

You can also use curl’s own-o/-OOption to save the contents to a file.

  • -o(lowercase o): the result will be saved to the file name provided on the command line
  • -O(uppercase o): the file name in the URL will be used as the file name to save the output
curl -o index.html http://www.codebelief.com curl -O http://www.codebelief.com/page/2/

be careful:use-OOption, you must ensure that the file name is included at the end of the link, otherwise curl cannot save the file correctly. If there is no file name in the link, you should make

use-o Option to specify the file name manually or use the redirection symbol.

4. Download multiple files at the same time

We can use-oor-OOption to specify multiple links at the same time, and write the command in the following format:

curl -O http://www.codebelief.com/page/2/ -O http://www.codebelief.com/page/3/

Or:

curl -o page1.html http://www.codebelief.com/page/1/ -o page2.html http://www.codebelief.com/page/2/

5. Use – L to follow the link redirection

If we directly use curl to open some links that are reset backward, we can’t get the web content we want in this case. For example:

curl http://codebelief.com

You will get the following prompt:

301 Moved Permanently 
 
301 Moved Permanently 
nginx/1.10.3

When we open the link through the browser, we will automatically jump tohttp://www.codebelief.com。 At this point, what we want curl to do is follow the jump of the link like a browser to get the final web content. We can add the – L option to the command to follow the link redirection:

curl -L http://codebelief.com

In this way, we can get the repositioned web content.

6. Use – A to customize the user agent

We can use-ACustomize the user agent. For example, the following command will disguise as Android Firefox browser to request the web page:

curl -A "Mozilla/5.0 (Android; Mobile; rv:35.0) Gecko/35.0 Firefox/35.0" http://www.baidu.com

Next, we will use – h to achieve the same purpose.

7. Use – h to customize the header

When we need to pass a specific header, we can follow the following command:

curl -H "Referer: www.example.com" -H "User-Agent: Custom-User-Agent" http://www.baidu.com

As you can see, when we use-HWhen customizing a user agent, you need to use“User-Agent: xxxThe format of the.

We can directly pass cookies in the header in the same format as the above example:

curl -H "Cookie: JSESSIONID=D0112A5063D938586B659EF8F939BE24" http://www.example.com

Another method is described below.

When we use curl to access a page, cookies are not saved by default. In some cases, we want to save cookies for the next visit. For example, when we log in to a website, we want to keep the login status when we visit the website again. At this time, we can save the cookie at the time of login and read it the next time.

-cFollowed by the file name to save.

curl -c "cookie-example" http://www.example.com

We talked about using-H To send cookies. This way is to write the cookie string directly in the command. If you use – B to customize cookies, the command is as follows:

curl -b "JSESSIONID=D0112A5063D938586B659EF8F939BE24" http://www.example.com

If you want to read cookies from the file, – H is powerless, and you can use it at this time-bTo achieve this goal:

curl -b "cookie-example" http://www.example.com

Namely-b It can be followed by either the cookie string or the file name where the cookie is saved.

10. Use – D to send a post request

We take the login web page as an example to illustrate the method of sending post request using curl. Suppose there is a login page www.example.com/login. You only need to submit your user name and password to log in. We can use curl to complete this post request, – D to specify the data to be sent, and – x to specify the way to send the data:

curl -d "userName=tom&passwd=123456" -X POST http://www.example.com/login

in use-dIf omitted-X, the default mode is post:

curl -d "userName=tom&passwd=123456" http://www.example.com/login

Force get mode

When sending data, you can use not only post but also get, for example:

curl -d "somedata" -X GET http://www.example.com/api

Or use-GOptions:

curl -d "somedata" -G http://www.example.com/api

Read data from file

curl -d "@data.txt" http://www.example.com/login

Login with cookie

Of course, if we visit the website again, it will still become unlisted. We can use the method mentioned above to save the cookie and bring the cookie every time we visit the website to maintain the login status.

curl -c "cookie-login" -d "userName=tom&passwd=123456" http://www.example.com/login

When you visit the web site again, use the following command:

curl -b "cookie-login" http://www.example.com/login

In this way, you can keep visiting the login page.

2、 Special character processing

1. Get request processing

1.1 integrate the wholeURLBoth are enclosed in double quotation marks’ ‘or single quotation marks”
curl ' http://restapi.amap.com/v3/geocode/geo?address= Lihai Xinyuan, 178 Shenzhen road, Laoshan District, Qingdao, Shandong & output = JSON & key = ABC '
curl " http://restapi.amap.com/v3/geocode/geo?address= Lihai Xinyuan, 178 Shenzhen road, Laoshan District, Qingdao, Shandong & output = JSON & key = ABC“
1.2. You can use ‘\’ before special characters.
curl  http://restapi.amap.com/v3/geocode/geo?address= Lihai Xinyuan, 178 Shenzhen road, Laoshan District, Qingdao City, Shandong Province \ & output = JSON \ & key = ABC

2. Post request processing

Pass the account password, but it contains special characters

curl -d name=john [email protected]&3*J https://www.mysite.com
2.1 curl > 7.18.0 with options--data-urlencode, it can solve this problem. Using this, I can simply send a post request as
curl -d name=john --data-urlencode [email protected]&3*J https://www.mysite.com
2.2 data after using URLEncode
curl -d 'name=john&passwd=%4031%263*J' https://www.mysite.com
2.3 you can use ‘\’ before special characters.
passwd=\@31\&3*J
2.4 variables
export CURLNAME="john:@31&3*J"
curl -d -u "${CURLNAME}" https://www.example.com
2.5 variables
~]$ export A=g

~]$ export B=!

~]$ export C=nger


   curl http://<>USERNAME<>1:[email protected]<>URL<>/<>PATH<>/

Standing on the shoulders of giants picking apples

https://www.jb51.net/article/118402.htm

https://www.itranslater.com/qa/details/2582815762230543360

Recommended Today

The selector returned by ngrx store createselector performs one-step debugging of fetching logic

Test source code: import { Component } from ‘@angular/core’; import { createSelector } from ‘@ngrx/store’; export interface State { counter1: number; counter2: number; } export const selectCounter1 = (state: State) => state.counter1; export const selectCounter2 = (state: State) => state.counter2; export const selectTotal = createSelector( selectCounter1, selectCounter2, (counter1, counter2) => counter1 + counter2 ); // […]