Detailed explanation of common commands of curl in Linux

Time:2021-11-16

Download a single file and print the output to standard output (stdout) by default


curl http://www.centos.org

Save the downloaded file to the specified file through the – O / – O option:

-o: Saves the file to a file with the file name specified on the command line

-O: Save the file locally using the default file name in the URL

#Download the file locally and name it mygettext.html
curl -o mygettext.html http://www.gnu.org/software/gettext/manual/gettext.html 
#Save the file locally and name it gettext.html
 curl -O http://www.gnu.org/software/gettext/manual/gettext.html

The steering character “>” can also be used to steer the output

Get multiple files at the same time


 curl -O URL1 -O URL2

If you download multiple files from the same site at the same time, curl will try to reuse the connection.

Redirect with – L option

By default, curl will not send HTTP location headers. When a requested page moves to another site, it will send an HTTP load

Header as a request, and then redirect the request to the new address.

For example, when visiting google.com, the address will be automatically redirected to google.com.hk.


curl http://www.google.com
 <HTML>
 <HEAD>
  <meta http-equiv="content-type" content="text/html;charset=utf-">
  <TITLE> Moved</TITLE>
 </HEAD>
 <BODY>
  <H> Moved</H>
  The document has moved
  <A HREF="http://www.google.com.hk/url?sa=p&amp;hl=zh-CN&amp;pref=hkredirect&amp;pval=yes&amp;q=http://www.google.com.hk/&amp;ust=amp;usg=AFQjCNFoumfjyJpNDPuFKTibavEaA">here</A>.
 </BODY>
 </HTML>

The above output indicates that the requested file has been transferred tohttp://www.google.com.hk

This is mandatory redirection by using the – L option

1 # let curl use address redirection, and the google.com.hk site will be queried

2 curl -L http://www.google.com

Breakpoint continuation

You can use the breakpoint continuation function for large files by using the – C option, such as:

#The process ends when the file is downloaded
 $ curl -O http://www.gnu.org/software/gettext/manual/gettext.html
 ##############       .%
 #Continue to download the file by adding the - C option. The downloaded file will not be downloaded again
 curl -C - -O http://www.gnu.org/software/gettext/manual/gettext.html
 ###############      .%

Use network speed limit for Curl

Use the — limit rate option to limit the maximum network usage of curl

#The maximum download speed will not exceed 1000B / second  
curl –limit-rate 1000B -O http://www.gnu.org/software/gettext/manual/gettext.html
Download files modified within the specified time

When downloading a file, the last modification date of the file can be judged. If the file has been modified within the specified date, it will be downloaded, otherwise it will not be downloaded.
This function can be achieved by using the – Z option:

#The yy.html file will not be downloaded until it is updated after December 21, 2011
 curl -z 21-Dec-11 http://www.example.com/yy.html

Curl authorization

When accessing the page requiring authorization, you can provide the user name and password through the – U option for authorization

curl -u username:password URL
 # The common practice is to enter only the user name on the command line, and then prompt for the password, so as to ensure that the password will not be disclosed when viewing the history
4 curl -u username URL

Download files from FTP server

Curl also supports FTP download. If a file path is specified in the URL instead of a specific file name to be downloaded, curl will list all file names in the directory instead of downloading all files in the directory

 # List public_ All folders and files under HTML
 curl -u ftpuser:ftppass -O ftp://ftp_server/public_html/
 # Download the xss.php file
 curl -u ftpuser:ftppass -O ftp://ftp_server/public_html/xss.php
Upload files to FTP server

The – t option uploads the specified local file to the FTP server

#Upload the myfile.txt file to the server

curl -u ftpuser:ftppass -T myfile.txt ftp://ftp.testserver.com

#Upload multiple files at the same time

curl -u ftpuser:ftppass -T "{file1,file2}" ftp://ftp.testserver.com

#Get the content from standard input and save it to the file specified by the server

curl -u ftpuser:ftppass -T - ftp://ftp.testserver.com/myfile_1.txt

Get more information

Get more link information by using – V and – trace

Query words through dictionary

 # Query the meaning of bash words
 curl dict://dict.org/d:bash 
#Lists all available dictionaries
 curl dict://dict.org/show:db
  # Query the meaning of bash word in FOLDOC dictionary
curl dict://dict.org/d:bash:foldoc

Set proxy for Curl

-The X option adds proxy functionality to the curl

#Specify proxy host and port
 curl -x proxysever.test.com:3128 http://google.co.in 

Other website collation

Saving and using website cookie information

#Save the cookie information of the website to the sugar cookies file
curl -D sugarcookies http://localhost/sugarcrm/index.php 
#Use last saved cookie information
 curl -b sugarcookies http://localhost/sugarcrm/index.php
Transfer request data

By default, curl uses the get method to request data. In this method, the data is directly transmitted through the URL
You can specify to use post to transfer data through — data / – D

 # GET
 curl -u username https://api.github.com/user?access_token=XXXXXXXXXX
 # POST
 curl -u username –data “param=value&param=value” https://api.github.com
 # You can also specify a file and pass the contents of the file to the server as data
 curl –data @filename https://github.api.com/authorizations

Note: by default, if there are special characters in the past data passed through post, the special characters need to be escaped and passed to the server. If the value contains spaces, the spaces need to be converted to% 20, such as:

curl -d “value%201” http://hostname.com

In the new version of curl, a new option — data URLEncode is provided. The parameters provided through this option will automatically escape special characters.

curl –data-urlencode “value 1” http://hostname.com

In addition to using get and post protocols, you can also specify other protocols through the – x option, such as:

curl -I -X DELETE https://api.github.cim

Upload file

 curl –form “[email protected]http://hostname/resource