Linux awk time calculation script and detailed explanation of awk commands

Time:2021-10-27

In Linux, if planning time is a hassle,

The awk script is as follows


BEGIN {FS=":";OFS=":"}
 {total_seconds=total_seconds+$3}
 total_seconds>=60 {total_seconds=total_sconds-60
  $2=$2+1
 }
{total_minutes=total_minutes+$2
 $2=$2+1
 }
{total_minutes=total_minutes+$2}
 total_minutes>=60{total_minutes=total_minutes-60
 $1=$1+1}
{total_hours=total_hours+$1}
END{print $1,$2,$3}

The following is a detailed explanation of the Linux awk commands collected for you

brief introduction

Awk is a powerful text analysis tool. Compared with grep search and sed editing, awk is particularly powerful when it analyzes data and generates reports. In short, awk is to read the file line by line, slice each line with a space as the default separator, and then analyze and process the cut part.

Awk has three different versions: awk, nawk and gawk, which generally refers to gawk, which is the GNU version of awk.

Awk gets its name from the first letter of the surnames of its founders Alfred aho, Peter Weinberger and Brian Kernighan. In fact, awk does have its own language: awk programming language, which has been officially defined as “style scanning and processing language” by the three creators. It allows you to create short programs that read input files, sort data, process data, perform calculations on input, generate reports, and countless other functions.

usage method

awk ‘{pattern + action}’ {filenames}

Although the operation may be complex, the syntax is always the same, where pattern represents what awk looks for in the data, and action is a series of commands executed when matching content is found. Curly braces ({}) need not always appear in a program, but they are used to group a series of instructions according to a specific pattern. Pattern is the regular expression to be represented, enclosed by slashes.

The most basic function of awk language is to browse and extract information based on specified rules in files or strings. After awk extracts information, other text operations can be carried out. Complete awk scripts are often used to format information in text files.

Generally, awk is a processing unit based on the behavior of a file. Awk receives each line of the file and then executes the corresponding command to process the text.

Call awk

There are three ways to call awk

1. Command line mode

Copy codeThe code is as follows:
awk [-F  field-separator]  ‘commands’  input-file(s)

Where commands is the real awk command and [- f field separator] is optional. Input file (s) is the file to be processed.
In awk, each item in each line of a file separated by a field separator is called a field. Typically, the default field separator is a space without specifying the – f field separator.

2. Shell script mode

Insert all awk commands into a file and make the awk program executable, and then the awk command interpreter, as the first line of the script, is called again by typing the script name.
Equivalent to the first line of the shell script: #/ bin/sh
You can change it to: #/ bin/awk

3. insert all awk commands into a single file, and then call:


awk -f awk-script-file input-file(s)

The – f option loads the awk script in the awk script file, and the input file (s) is the same as the above.

This chapter focuses on the command line mode.

Entry example

Suppose the output of last – N 5 is as follows

[ [email protected] ~]#Last – N 5 < = = take out only the first five lines
root     pts/1   192.168.1.100  Tue Feb 10 11:21   still logged in
root     pts/1   192.168.1.100  Tue Feb 10 00:46 – 02:28  (01:41)
root     pts/1   192.168.1.100  Mon Feb  9 11:41 – 18:30  (06:48)
dmtsai   pts/1   192.168.1.100  Mon Feb  9 11:41 – 11:41  (00:00)
root     tty1                   Fri Sep  5 14:09 – 14:10  (00:01)

If only the 5 most recently logged in accounts are displayed

#last -n 5 | awk  ‘{print $1}’
root
root
root
dmtsai
roo

The awk workflow is as follows: read in a record separated by ‘\ n’ newline character, then divide the record into fields according to the specified field separator, fill in the fields, $0 represents all fields, $1 represents the first field and $n represents the nth field. The default domain separator is “blank key” or “[tab] key”, so $1 represents the login user, $3 represents the login user IP, and so on.

If only the account of / etc / passwd is displayed

#cat /etc/passwd |awk  -F ‘:’  ‘{print $1}’ 
root
daemon
bin
sys

This is an example of awk + action. Action {print $1} is executed for each line.

-F specifies that the field separator is’: ‘.

If only the account of / etc / passwd and the shell corresponding to the account are displayed, and the account and shell are separated by tab

#cat /etc/passwd |awk  -F ‘:’  ‘{print $1″\t”$7}’
root    /bin/bash
daemon  /bin/sh
bin     /bin/sh
sys     /bin/sh

If only the account of / etc / passwd and the shell corresponding to the account are displayed, and the account and shell are separated by commas, add column name and shell in all rows, and add “blue, / bin / nosh” in the last row.

cat /etc/passwd |awk  -F ‘:’  ‘BEGIN {print “name,shell”}  {print $1″,”$7} END {print “blue,/bin/nosh”}’
name,shell
root,/bin/bash
daemon,/bin/sh
bin,/bin/sh
sys,/bin/sh
….
blue,/bin/nosh

The workflow of awk is as follows: first execute searching, then read the file, read in a record separated by / N newline character, then divide the record into fields according to the specified field separator, fill in the fields, $0 represents all fields, $1 represents the first field, $n represents the nth field, and then start executing the action corresponding to the mode. Then start reading the second record… Until all records are read, and finally execute the end operation.

Search all lines in / etc / passwd that have the root keyword

#awk -F: ‘/root/’ /etc/passwd
root:x:0:0:root:/root:/bin/bash
This is an example of the use of pattern. Only the rows that match the pattern (here root) will execute the action (without specifying the action, the content of each row will be output by default).

Search supports regular, for example, find: awk – F: ‘/ ^ root /’ / etc / passwd starting with root

Search all lines with the root keyword in / etc / passwd and display the corresponding shell

# awk -F: ‘/root/{print $7}’ /etc/passwd            
/bin/bash

Action {print $7} is specified here

Awk built in variable
Awk has many built-in variables to set environment information. These variables can be changed. Some of the most commonly used variables are given below.

ARGC                Number of command line parameters
ARGV                Command line parameter arrangement
ENVIRON             Supports the use of system environment variables in queues
FILENAME            Awk browse file name
FNR                 Number of records browsing the file
FS                  Sets the input field separator, which is equivalent to the command line – f option
NF                  Number of fields for browsing records
NR                  Number of records read
OFS                 Output field separator
ORS                 Output record separator
RS                  record separator

  In addition, the $0 variable refers to the entire record$ 1 represents the first field of the current line, $2 represents the second field of the current line,… And so on.

Statistics of / etc / passwd: file name, line number of each line, number of columns of each line, and corresponding complete line content:

#awk  -F ‘:’  ‘{print “filename:” FILENAME “,linenumber:” NR “,columns:” NF “,linecontent:”$0}’ /etc/passwd
filename:/etc/passwd,linenumber:1,columns:7,linecontent:root:x:0:0:root:/root:/bin/bash
filename:/etc/passwd,linenumber:2,columns:7,linecontent:daemon:x:1:1:daemon:/usr/sbin:/bin/sh
filename:/etc/passwd,linenumber:3,columns:7,linecontent:bin:x:2:2:bin:/bin:/bin/sh
filename:/etc/passwd,linenumber:4,columns:7,linecontent:sys:x:3:3:sys:/dev:/bin/sh

Using printf instead of print can make the code more concise and easy to read

 awk  -F ‘:’  ‘{printf(“filename:%10s,linenumber:%s,columns:%s,linecontent:%s\n”,FILENAME,NR,NF,$0)}’ /etc/passwd

Print and printf

Awk provides both print and printf printout functions.

The parameters of the print function can be variables, numeric values or strings. Strings must be quoted in double quotation marks and parameters separated by commas. Without commas, parameters are concatenated and indistinguishable. Here, the function of comma is the same as that of the separator in the output file, except that the latter is a space.

The usage of printf function is basically similar to that of printf in C language. It can format strings. When the output is complex, printf is easier to use and the code is easier to understand.

Awk programming

Variables and Assignment

In addition to the built-in variables of awk, awk can also customize variables.

The number of accounts in / etc / passwd is counted below

awk ‘{count++;print $0;} END{print “user count is “, count}’ /etc/passwd
root:x:0:0:root:/root:/bin/bash
……
user count is  40

Count is a custom variable. In the previous action {}, there was only one print. In fact, print is only one statement, while action {} can have multiple statements to; Separate the numbers.

Count is not initialized here. Although it is 0 by default, it is appropriate to initialize it to 0:

awk ‘BEGIN {count=0;print “[start]user count is “, count} {count=count+1;print $0;} END{print “[end]user count is “, count}’ /etc/passwd
[start]user count is  0
root:x:0:0:root:/root:/bin/bash

[end]user count is  40

Count the number of bytes occupied by files in a folder

ls -l |awk ‘BEGIN {size=0;} {size=size+$5;} END{print “[end]size is “, size}’
[end]size is  8657198

If displayed in M:

ls -l |awk ‘BEGIN {size=0;} {size=size+$5;} END{print “[end]size is “, size/1024/1024,”M”}’
[end]size is  8.25889 M
Note that the statistics do not include subdirectories of folders.

Conditional statement

The conditional statements in awk are borrowed from C language. See the following Declaration:


if (expression) {
  statement;
  statement;
  ... ...
}

if (expression) {
  statement;
} else {
  statement2;
}

if (expression) {
  statement1;
} else if (expression1) {
  statement2;
} else {
  statement3;
}

Count the number of bytes occupied by files in a folder and filter 4096 files (usually folders):


ls -l |awk 'BEGIN {size=0;print "[start]size is ", size} {if($5!=4096){size=size+$5;}} END{print "[end]size is ", size/1024/1024,"M"}' 
[end]size is 8.22339 M

Circular statement

The circular statements in awk also learn from C language and support while, do / while, for, break and continue. The semantics of these keywords are exactly the same as those in C language.

array

Because the subscripts of arrays in awk can be numbers and letters, the subscripts of arrays are usually called keys. Values and keywords are stored in an internal hash table for key / value. Because hashes are not stored in order, you will find that they are not displayed in the order you expect when displaying the contents of the array. Arrays, like variables, are created automatically when used, and awk will also automatically determine whether they store numbers or strings. Generally speaking, arrays in awk are used to collect information from records. They can be used to calculate sums, count words, track the number of times templates are matched, and so on.

Displays the account for / etc / passwd

awk -F ‘:’ ‘BEGIN {count=0;} {name[count] = $1;count++;}; END{for (i = 0; i < NR; i++) print i, name[i]}’ /etc/passwd
0 root
1 daemon
2 bin
3 sys
4 sync
5 games
……

Here we use the for loop to traverse the array