Linux awk command details

Time:2021-10-19

Awk is another powerful data processing tool in Linux system besides sed command. The following Liangxu Xiaobian will be from several aspectsLinux awkThe order is explained in detail. I hope it will be helpful to you.

Linux命令

Similar to the SED command, the awk command also scans the file line by line (from the first line to the last line) to find the line containing the target text. If the match is successful, the user will perform the desired operation on the line; Otherwise, no processing will be done to the row.

When awk reads each line of the file content, it will compare whether the line matches the given pattern. If it matches, it will execute the processing process. Otherwise, it will not do any processing on the line. If no processing script is specified, the matching line will be displayed to the standard output, that is, the default processing action is print print line; If no pattern matching is specified, all data will be matched by default.

Awk has two special modes: begin and end, which are placed before no data is read and executed after all data is read. Before reading the file content, the instructions after begin will be executed, then read the file content and judge whether it matches the specific mode. If it matches, execute the action instructions after the normal mode, finally execute the end mode command, and output the results after document processing.

The workflow of awk is shown in Figure 1.

Linux命令

The basic syntax format of awk command is as follows:

[ [email protected] ~]#Awk [options] ‘script command’ file name

The common options and functions of awk command are shown in Table 2.

Options Function
-F fs Specify FS as the separator of the input line. The default separator of awk command is space or tab
-f file Read the awk script instruction from the script file instead of entering the processing script in the command parameters
-v var=val Before executing the process, set a variable VaR and give its device initial value val
-W compat Running awk in compatibility mode, the gun extension option is ignored
-W copyleft Output brief gun copyright information
-W dump-variables[=file] Print global variables (variable name, type, value) to the file. If no file name is provided, it will be automatically output to the file named dump variables

The script command of awk mainly consists of two parts: matching rules and executing commands.

The matching rule of awk is the same as the address part of SED command. It is used to specify that script commands can act on specific lines in text content, which can be specified by string or regular expression. In addition, the entire script command uses single quotation marks (”), and the execution command part needs to be enclosed by curly brackets “{}”.

Note that when the awk program is executed, if no execution command is specified, the matching line will be output by default; If no matching rule is specified, all lines in the text are matched by default.

One of the main features of awk is its ability to process data in text files. It will automatically assign a variable to each data element in each row.

The built-in variables and functions of awk are shown in Table 3.

Variable Function
$0 Current record
$1~$n The nth field of the current record
ARGC Number of command line parameters
FILENAME The name of the current input document
FNR The current record number of the current input document, especially when there are multiple input documents
NR Enter the current record number of the stream
NF Number of fields in the current record
FS Field separator
OFS Enter the field separator, which defaults to space
ORS Enter the record separator, which defaults to newline \ n
RS Enter the record separator, which defaults to newline \ n

[example 1] output the current line number of the current document.
Create a sample file named text, and write the following contents in the sample file:

http://www.lxlinux.net/

1234

Enter the following command in the terminal page:

[[email protected] ~]# awk ‘{print FNR}’ text
1
2
3

Expressions are composed of variables, constants, functions, regular expressions and operators. Variables in awk include character variables and numeric variables. If the variable defined in awk is not initialized, the initial value is null or 0. Quotation marks must be used when characters are manipulated. Operators and their meanings are shown in Table 3.

Operator meaning
+ plus
reduce
* ride
/ except
% Seeking remainder
^ exponentiation
++ Self adding 1
Self subtraction 1
+= Add and assign to variable
= Assign value to variable after subtraction
A bird’s nest= Multiply and assign to variable
/= Assign value to variable after division
> greater than
< less than
>= Greater than or equal to
<= Less than or equal to
= be equal to
!= Not equal to
Matching regular expressions
!〜 Mismatch regular expression
&& And
II or

For example, awk assignment operator, a + 5 is equivalent to a = a + 5, as follows:

[[email protected] ~]# awk ‘BEGIN{a=2; a+=2;print a}’
4

In addition, awk also supports if conditional judgment, while and for loop statements.

  1. If conditional judgment statement. There are two basic formats for if statements:

If (expression)
Action 1
else
Action 2

perhaps

If (expression) action 1; Else action 2

If the judgment result of the expression is true, execute action 1; otherwise, execute action 2.

For example, if it is judged that the available capacity of boot partition is less than 20MB, it will give an alarm, otherwise it will display OK, as shown below:

[[email protected] ~]# df | grep “boot” | awk ‘{if ($4<20000) print “Alert”; else print “OK”}’
OK

  1. While loop statement. There are two basic formats of while loop statements:

While (condition)
action

perhaps

do
action
While (condition)

The while loop statement is as follows:

[[email protected] ~]# awk ‘i=1 {} BEGIN { while(i<=10) {++i; print i}}’
1
2
3
4
……
10
11

  1. For loop statement. The basic format of the for loop statement is as follows:

For (variable; condition; counter)
action

The for loop statement is as follows:

[[email protected] ~]# awk ‘BEGIN { for(i=1; i<=10; i++) print i}’
1
2
3
……
9
10

The above isLiangxu tutorial networkExplain the Linux awk command shared by all friends.

This article is composed of blog one article multi posting platformOpenWriterelease!

Recommended Today

SQL exercise 20 – Modeling & Reporting

This blog is used to review and sort out the common topic modeling architecture, analysis oriented architecture and integration topic reports in data warehouse. I have uploaded these reports to GitHub. If you are interested, you can have a lookAddress:https://github.com/nino-laiqiu/TiTanI recorded a relatively complete development process in my hexo blog deployed on GitHub. You can […]