shell

Time:2021-1-18

1、 Shell overview

  • Shell is a = = command line interpreter = =, which provides users with an interface system level program to send requests to the Linux kernel to run programs. Users can use shell to start, suspend, stop and even write some programs.
  • Shell is also a powerful programming language, easy to write, easy to debug and flexible. Shell is a script language for interpretation and execution, in which Linux system commands can be called directly.

2、 How to execute shell script

1. Echo output command
Echo [option] [output content]
Options: 
    -e: Character conversion with backslash control
Control character effect
\ \ Output itself
a Output warning tone
b Backspace key, that is, the left delete key
c Cancels the line break at the end of the output line. Consistent with “- n” option
e Escape key
f Page feed
n Line break
r enter key
t Tab, or tab
v vertical tab
0nnn Output characters according to octal ASCII code table. Where 0 is a digit zero and NNN is a three digit octal number
xhh Output characters according to the hexadecimal ASCII code table. Where HH is a two digit hexadecimal number
[[email protected] ~]# echo -e "ab\bc"
ac
[[email protected] ~]# echo -e "a\tb\tc\nd\te\tf"
a    b    c
d    e    f
[ [email protected]  ~]#Echo - e "\ E [1; 31m how are you 
#Output colored results
#30m = black, 31m = red, 32m = green, 33m = yellow 
#34m = blue, 35m = magenta, 36m = cyan, 37m = white
2. The first script
[[email protected] ~]# vim helloworld.sh
#!/bin/Bash
echo "hello world"
3. Script execution

There are two ways

  • Give executive authority, run directly

    chmod 755 helloworld.sh
    ./helloworld.sh
  • Execute script through bash call

    bash helloworld.sh

3、 Basic functions of Bash

1. Historical order

History [option] [history command save file] 
Options:
    -c: Clear history command 
    -w: Write the history command in the cache to the history command save file ~ /. Bash_ history

By default, 1000 historical commands will be saved, which can be modified in the environment variable configuration file / etc / profile.

Calling history command:
  • Use up and down arrows to call previous history commands
  • Use! N to repeat the nth history command
  • Use! To repeat the previous command
  • Repeat the last command that starts with the! String

2. Command alias

#Set command alias
Alias = original command
#Query command alias
alias
#Remove alias
Unalias alias
#Make alias permanent
vim /root/.bashrc

3. Common shortcut keys

Shortcut key effect
Ctrl+A Move the cursor to the beginning of the command line. If the command we enter is too long, we want to use it when we want to move the cursor to the beginning of the command line.
Ctrl+E Move the cursor to the end of the command line.
Ctrl+C Force the termination of the current command.
Ctrl+L Clear screen, equivalent to clear command.
Ctrl+U Delete or cut the command before the cursor. I entered a long line of commands, do not use backspace key to delete one character by one, using this shortcut key will be more convenient.
Ctrl+K Delete or cut the content after the cursor.
Ctrl+Y Paste the contents cut by Ctrl + U or Ctrl + K.
Ctrl+R Search in the history command. After pressing Ctrl + R, the search interface will appear. As long as you enter the search content, you will search from the history command.
Ctrl+D Exit the current terminal.
Ctrl+Z Pause and put in the background. This shortcut involves the content of work management, which we will introduce in detail in the chapter of system management.
Ctrl+S Pause screen output.
Ctrl+Q Restore screen output.

4. Output redirection

Standard input and output:
equipment Device file name File descriptor type
keyboard /dev/stdin 0 Standard input
monitor /dev/stdout 1 standard output
monitor /dev/stderr 2 Standard error output
Output redirection
Type Symbol effect
Standard output redirection Command > file In the way of overlay, the correct output of the command is output to the specified file or device.
Standard output redirection Command > > file In the way of appending, the correct output of the command is output to the specified file or device.
Standard error output redirection Error command 2 > file In the way of overlay, the error output of the command is output to the specified file or device.
Standard error output redirection Error command 2 > > file In the way of appending, the error output of the command is output to the specified file or device.
Command > file 2 > & 1 In the way of overlay, the correct output and error output are saved in the same file.
Command > > file 2 > & 1 By appending, the correct output and error output are saved in the same file.
Correct output and error output are saved at the same time Command & > file In the way of overlay, the correct output and error output are saved in the same file.
Command & > > file By appending, the correct output and error output are saved in the same file.
Command > > file 1 2 > > file 2 Append the correct output to file 1 and the wrong output to file 2.

5. Sequential execution of multiple commands

Multiple command executor format effect
; Command 1; Command 2 Multiple commands are executed in sequence, and there is no logical connection between them
&& Command 1 & Command 2 Logic and logic. When command 1 is executed correctly, command 2 will be executed; when command 1 is not executed correctly, command 2 will not be executed
\ \ Command 1\ \ Command 2 Logical or. When command 1 is not executed correctly, command 2 will be executed; when command 1 is executed correctly, command 2 will not be executed
[[email protected] ~]# ls ; date
helloworld.sh
Sat Aug 15 11:51:42 CST 2020
[[email protected] ~]# ls && echo yes
helloworld.sh
yes
[[email protected] ~]# ls 1 || echo no
ls: cannot access 1: No such file or directory
no

6. Pipe symbol

Command 1 | Command 2
#The correct output of command 1 is used as the operation object of Command 2
[[email protected] ~]# ps -ef | grep bash | grep -v "grep"
root     12026 12023  0 11:03 pts/1    00:00:00 -bash
1000     23691 22856  0 Jul26 ?        00:00:00 bash
Grep search content
Options: 
    -i: Ignore case 
    -n: Output line number 
    -v: Reverse lookup 
    --Color = Auto: the search keywords are displayed in color

7. Special symbols

Symbol effect
Single quotation marks. In single quotation marks, all special symbols, such as $and ‘, have no special meaning
“” Double quotation marks. In double quotation marks, special symbols have no special meaning, but $and ‘and’ are exceptions. They have the special meanings of “call variable value”, “reference command” and “escape character”
Back quotes. The contents enclosed in back quotation marks are system commands, which are executed first in bash. &Dollar; () works the same way, but $() is recommended because the back quotation marks are very easy to read wrong
$() It is used to refer to system commands just like the back quotation mark
# In shell scripts, the line beginning with # represents comments
$ The value used to call the variable, if needednameYou need to use the$nameTo get the value of the variable
\ Escape character, the special symbol that follows will lose its special meaning and become ordinary character. For example, \ & dollar; will output the $sign, and improper doing is variable reference
#Back quotes and $()
[[email protected] ~]# echo `ls`
a b c
[[email protected] ~]# echo $(date)
Sat Aug 15 12:12:25 CST 2020
#Single quotation mark and double quotation mark
[[email protected] ~]# name=pd
[[email protected] ~]# echo '$name'
$name
[[email protected] ~]# echo "$name"
pd
[[email protected] ~]# echo '$(date)'
$(date)
[[email protected] ~]# echo "$(date)"
Sat Aug 15 12:13:43 CST 2020

4、 Bash variables

1. User defined variables

A variable is a unit of computer memory in which values can be changed. When shell scripts need to save some information, such as a file name or a number, they store it in a variable. Each variable has a name, so it’s easy to refer to. Variables can be used to save useful information and make the system know the user’s relevant settings. Variables can also be used to save temporary information.

Variable setting rules:
  • The variable name can be composed of letters, numbers and underscores, but it cannot start with a number. If the variable name is 2Name, it is wrong
  • In Bash, the default type of variable is string type. If you want to carry out numerical operation, you must specify the variable type as numeric type
  • Variables are connected by equal sign, and there can be no spaces on the left and right sides of the equal sign
  • If the value of a variable has spaces, it needs to be enclosed in single or double quotation marks
  • In the value of a variable, you can use escape characters
  • If you need to increase the value of the variable, you can stack the value of the variable. However, variables need to be enclosed in double quotation marks with “& dollar; variable name” or with ${variable name}
  • If you are assigning the result of a command to a variable as a variable value, you need to include the command in back quotation marks or $()
  • The name of environment variable is suggested to be capitalized, which is easy to distinguish
Variable classification:
  • User defined variables
  • Environment variable: this kind of variable mainly stores the data related to the system operating environment
  • Positional parameter variable: this kind of variable is mainly used to pass parameters or data to the script. The variable name cannot be customized, and the function of the variable is fixed
  • Predefined variable: it is a variable that has been defined in bash. The name of the variable cannot be defined and the function of the variable is fixed
Local variable:

2. Environmental variables

User defined variables only take effect in the current shell, while environment variables will take effect in the current shell and all subshell of the shell. If the environment variable is written to the corresponding configuration file, the environment variable will take effect in all shells.

Set environment variables:
#Declare variables
Export variable name = variable value 
#Query variables
env 
#Delete variable
Unset variable name
Common system environment variables:
#Path: the path for the system to find the command 
[[email protected] ~]# echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/opt/mysql/bin:/opt/go/bin:/gopath/bin:/root/bin
#Path variable overlay 
[[email protected] ~]# PATH="$PATH":/root/sh
[[email protected] ~]# echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/opt/mysql/bin:/opt/go/bin:/gopath/bin:/root/bin:/root/sh
#PS1: variables that define the system prompt
\d: Display date in the format of "Sunday" 
\h: Displays the abbreviated host name. For example, the default host name is "localhost" 
\t: Display 24-hour time in the format of "HH: mm: SS" 
\T: Display 12 hour time in the format of "HH: mm: SS" 
\A: Display the 24-hour time in the format of "HH: mm" 
\u: Displays the current user name 
\w: Displays the full name of the current directory 
\W: Displays the last directory of the current directory 
\#: which command to execute 
$: prompt. If the user is root, the prompt will be "#", if the user is normal, the prompt will be "$"

#Examples
[[email protected] ~]# PS1='[\[email protected]\t \w]$' 
[[email protected]:27:42 ~]#PS1='[\[email protected]\@ \h \# \W]$'
[[email protected]:28 PM peiqi 12 ~]#PS1='[\[email protected]\h \W]$'
[[email protected] ~]#

3. Position parameter variable

Position parameter variable effect
$n N is a number, & dollar; 0 stands for the shell script itself, & dollar; 1 – & dollar; 9 stands for the first to ninth parameters, and more than ten parameters need to contain ${10} in curly brackets
$* This variable represents all the parameters in the command line, $* takes all the parameters as a whole
[email protected] This variable also represents all the parameters in the command line, but [email protected] treats each parameter differently
$# This variable represents the number of all parameters in the command line
Example 1:
[[email protected] ~]#vim test1.sh
    #!/bin/bash
    n1=$1
    n2=$2
    #The sum of the variables sum is N1 plus N2
    sum=$(( $n1 + $n2))
    #Print the value of the variable sum
    echo $sum
[[email protected] ~]#bash test1.sh 10 20
30
Example 2:
[[email protected] ~]#vim test2.sh
    #!/bin/bash 
    #Use $# to represent the number of all parameters
    echo "$#"
    #Use $* to represent all parameters  
    echo "$*" 
    #Using [email protected] also represents all parameters 
    echo "[email protected]" 
[[email protected] ~]#bash test2.sh 4 5 6
3
4 5 6
4 5 6

Example 3: & dollar; * and & dollar; @ differences

[[email protected] ~]#vim test3.sh
    #!/bin/bash
    echo '----------------- "$ *" -----------------'
    #All the parameters in "$*" are considered as a whole, so the for loop will only loop once 
    for i in "$*" 
        do 
            echo "$i" 
        done
    echo '----------------- "$ @" -----------------'
    #Each parameter in "[email protected]" is regarded as independent, so if there are several parameters, they will loop several times
    for j in "[email protected]" 
        do 
            echo "$j" 
        done
    echo '----------------- $ * -----------------'
    #Note the difference between "$*" and $*, while "[email protected]" and [email protected] are not
    for k in $* 
        do 
            echo "$k"
        done
[[email protected] ~]#bash test3.sh 1 2 3
----------------- "$ *" -----------------
1 2 3
----------------- "$ @" -----------------
1
2
3
----------------- $ * -----------------
1
2
3

4. Predefined variables

Predefined variables effect
$? The return status of the last executed command. If the value of this variable is 0, the last command is executed correctly; if the value of this variable is not 0 (which number is determined by the command itself), the last command is executed incorrectly
$$ The process number (PID) of the current process
$! The process number (PID) of the last process running in the background
[[email protected] ~]#vim test.sh
    #!/bin/bash
    #Output PID of current process
    #This PID is test.sh The PID of the generated process when the script is executed 
    echo "The current process is $$" 
    #Use the find command to search in the root directory helloworld.sh file
    #Symbol & means to put a command into the background for execution
    find /root -name helloworld.sh & 
    echo "The last one Daemon process is $!"
[[email protected] ~]#bash test.sh 
The current process is 23943
The last one Daemon process is 23944
[[email protected] ~]#/root/helloworld.sh
Receive keyboard input:
Read [option] [variable name] 
Options: 
    -P "prompt information": output prompt information while waiting for read input 
    -T seconds: the read command will wait for user input all the time. Use this option to specify the waiting time 
    -N number of characters: the read command only accepts the specified number of characters and will be executed 
    -s: Hide the input data, suitable for the input of confidential information
[[email protected] ~]# vim read.sh
    #!/bin/bash 
    #Prompt "please enter name" and wait 30 seconds to save the user's input into the variable name 
    read -t 30 -p "Please input your name: " name
    #Age is privacy, so we use the "- s" option to hide the input
    read -s -p "Please enter your age: " age
    echo -e "\n"
    #Use the "- N 1" option to receive only one input character and execute (no carriage return)
    read -n 1 -p "Please select your gender[m/f]: " gender
    echo -e "\n"
    echo "Name:$name"
    echo "Age:$age"  
    echo "Gender:$gender"
[[email protected] ~]# bash read.sh 
Please input your name: pd
Please enter your age: 
Please select your gender[m/f]: m
Name:pd
Age:18
Gender:m

5、 Operators of Bash

1. Numerical operations and operators

The declare command is used to declare shell variables:
Declare [+ / -] [option] variable name 
Options: 
    -: setting type properties to variables 
    +: cancels the type property of a variable 
    -i: Declare variable as integer 
    -x: Declare a variable as an environment variable 
    -p: Displays the declared type of the specified variable
    
[ [email protected]  ~]#Declare - I a // declare integer variables
[ [email protected]  ~]#A = 10 // change variable content
[[email protected] ~]# echo $a
10

Linux declare command

Numerical operation:

Method 1

[[email protected] ~]# a=11
[[email protected] ~]# b=22
[[email protected] ~]# declare -i c=$a+$b

Method 2: expr or let numerical operation tool

[[email protected] ~]# a=11
[[email protected] ~]# b=22
#The value of D is the sum of a and B. Note that there must be spaces on both sides of the + sign
[[email protected]eiqi ~]# d=$(expr $a + $b)

Method 3: “& dollar; ((expression))” or “$[expression]”

[[email protected] ~]# a=11
[[email protected] ~]# b=22
[[email protected] ~]# e=$(($a+$b))
[[email protected] ~]# f=$[$a+$b]
Operator:
priority operator explain
13 -、+ Monocular negative and monocular positive
12 !、~ Logical not, bitwise negation or complement
11 *、/、% Multiplication, division and modulus
10 +、- Add, subtract
9 <<、>> Shift left and right by bit
8 <=、>=、<、> Less than or equal to, greater than or equal to, less than or greater than
7 ==、!= Equal to, not equal to
6 & Bitwise AND
5 ^ Bitwise XOR
4 \ Bitwise OR
3 && Logic and
2 \ \ logic
1 =、+=、-=、*=、/=、%=、&=、 ^==、<<=、>>= Assignment, operation and assignment
#Although multiplication and division have higher priority than addition, the operation priority can be adjusted through parentheses 
[[email protected] ~]# a=$(( (11+3)*3/2 )) 
#14 is not divisible by 3, the remainder is 2 
[[email protected] ~]# b=$(( 14%3 ))  
#The result of logic and operation is 1 only if both sides of and are 1, otherwise the result of and is 0 
[[email protected] ~]# c=$(( 1 && 0 ))

2. Variable test and content replacement

Variable replacement method Variable y is not set The variable y is null Variable y has a value set
X = ${Y-new value} X = new value X is empty x=$y
X = ${Y: – new value} X = new value X = new value x=$y
X = ${y + new value} X is empty X = new value X = new value
X = ${Y: + new value} X is empty X is empty X = new value
X = ${y = new value} X = new value, y = new value X is null and Y is unchanged X = $y, y value unchanged
X = ${Y: = new value} X = new value, y = new value X = new value, y = new value X = $y, y value unchanged
X = ${y? New value} New value output to standard, error output (screen) X is empty x=$y
X = ${Y:? New value} New value output to standard, error output New value output to standard, error output x=$y
[ [email protected]  ~]#Unset y # delete variable y
[ [email protected]  ~]#X = ${Y-new} to test
[ [email protected]  ~]#Echo $X # because the variable y does not exist, so x = new
new
[ [email protected]  ~]#Y = # assign a null value to the variable y
[ [email protected]  ~]#X = ${Y-new} to test
[ [email protected]  ~]#Echo $X # because the variable y is empty, X is empty

[ [email protected]  ~]#Y = old # assign a value to the variable y
[ [email protected]  ~]#X = ${Y-new} to test
[ [email protected]  ~]#Echo $X # because the variable y has a value, x = $y
old

6、 Environment variable configuration file

1. Introduction of environment variable configuration file

Source command:
[ [email protected]  ~]#Source profile
or 
[ [email protected]  ~]#. profile
Introduction to environment variable configuration file:

The environment variable configuration file mainly defines the system default environment variables that are effective for the operating environment of the system, such as path, histsize, PS1, hostname and other default environment variables.

  • /etc/profile
  • /etc/profile.d/*.sh
  • ~/.bash_profile
  • ~/.bashrc
  • /etc/bashrc

2. Function of environment variable configuration file

/Etc / profile function:
  • User variable
  • Logname variable
  • Mail variable
  • Path variable
  • Hostname variable
  • Histsize variable
  • umask
  • Call / etc / profile. D / *. Sh file
~/.bash_ Profile function:
  • The ~ /. Bashrc file is called
  • Add the $home / bin directory after the path variable
~/Function of. Bashrc:
  • Define default alias
  • Call / etc / bashrc
/The functions of etc / bashrc are as follows
  • PS1 variable
  • umask
  • Path variable
  • Call / etc / profile. D / *. Sh file

3. Other profiles and login information

Environment variable profile in effect at logout:
  • ~/.bash_logout
Other profiles:
  • ~/bash_history
Shell login information:
  • Local terminal welcome message / etc / issue

    Escape character effect
    d Displays the current system date
    s Displays the name of the operating system
    l Display login terminal number, this is more commonly used
    m Display hardware architecture, such as i386, i686, etc
    n Display host name
    o Display domain name
    r Show kernel version
    t Displays the current system time
    u Displays the serial number of the currently logged in user
  • Remote terminal welcome message / etc/ issue.net

    • Escape character in / etc/ issue.net Cannot be used in file
    • Whether to display this welcome message is determined by the SSH configuration file / etc / SSH / sshd_ Config decides to add “banner / etc”/ issue.net “Line” can be displayed (remember to restart SSH service)
  • Welcome message after login / etc / MOTD

    • Whether it is local login or remote login, this welcome message can be displayed

7、 Basic regular expressions

1. Regular expressions and wildcards

  • Regular expressions are used to match strings in a file. Grep, awk, SED and other commands can support regular expressions.
  • The wildcard is used to match the qualified file names, and the wildcard is an exact match. The commands ls, find, CP do not support regular expressions, so you can only use shell’s own wildcards to match.

2. Basic regular expressions

character explain
\ Marks the next character as a special character, text, back reference, or octal escape character. For example, “n” matches the character “n”, “n” matches the newline character, “\” matches “\”, “\ (” matches “(“
^ Matches the start of the input string. If setRegExpObjectMultilineAttribute, ^ also matches the position after “n” or “R”
$ Matches the position at the end of the input string. If setRegExpObjectMultilineProperty, $also matches the position before “n” or “R”
* Zero timesormany timesMatches the preceding character or subexpression. For example, “Zo *” matches “Z” and “zoo.”. *Equivalent to {0,}
{n} nIs a nonnegative integer,just rightmatchingnTimes. For example, “O {2}” does not match “O” in “Bob”, but matches two “O” in “food”
{n,} nIs a nonnegative integer,at leastmatchingnTimes. For example, “O {2,}” does not match “O” in “Bob”, but matches all “O” in “Foooood”. “O {1,}” is equivalent to “O +”. “O {0,}” is equivalent to “O *”
{n,m} MandnIs a nonnegative integer, wheren <= m. Match at leastnAt mostmTimes. For example, “O {1,3}” matches the first three “O” in “food”. “O {0,1}” is equivalent to “O?”. Note: spaces cannot be inserted between commas and numbers
. Matches any single character except “RN” (the newline character). To match any character including “RN”, use a pattern such as “[SS]”
[xyz] A character set that matches any character it contains. For example, “ABC” matches “a” in “plain”
1 Reverse range character to match any character that is not in the specified range. For example, ‘[^ A-Z]’ matches any character that is not in the range of ‘a’ to ‘Z’
#Match rows that contain at least one a 
grep "a*" test_rule.txt
#Matches a string containing at least two consecutive a's 
grep "aa*" test_rule.txt  
#"S.. D" matches words that must have two characters between the letters s and D 
grep  "s..d" test_rule.txt  
#Match any character between S and D 
grep "s.*d" test_rule.txt  
#Match everything
grep ".*" test_rule.txt  
#Match lines beginning with capital "m" 
grep "^M" test_rule.txt 
#Match lines ending in lowercase "n"
grep "n$" test_rule.txt 
#Matches blank lines 
grep -n "^$" test_rule.txt 
#Match the letters s and I, either a or o
grep "s[ao]id" test_rule.txt
#Match any number 
grep "[0-9]" test_rule.txt
#Matches lines that start with lowercase letters
grep "^[a-z]" test_rule.txt
#Matches lines that do not start with lowercase letters
grep "^[^a-z]" test_rule
#Match lines that do not start with a letter 
grep "^[^a-zA-Z]" test
#Match the line ending with 
grep "\.$" test_rule.txt
#Match the string with a letter appearing three times in succession
grep "a\{3\}" test_rule.txt 
#Matches a string containing three consecutive numbers 
grep "[0-9]\{3\}" test_rule.txt
#Match lines that start with at least three consecutive numbers 
grep "^[0-9]\{3,\}[a-z]" test_rule.txt
#There is at least one a and at most three a between S and I 
grep "sa\{1,3\}i" test_rule.txt

8、 Character capture command

1. Cut field extraction command

Cut file name 
Options: 
    -F column number: which column to extract 
    -D separator: splits the column according to the specified separator
vim strudent.txt
    ID Name Gender Mark
    1 peiqi F 86
    2 qiaozhi M 90
    3 pd M 83
[[email protected] ~]# cut -d " " -f 2 student.txt 
Name
peiqi
qiaozhi
pd
[[email protected] ~]# cut -d " " -f 3,4 student.txt 
Gender Mark
F 86
M 90
M 83
[[email protected] ~]# cut -d ":" -f 1,3 /etc/passwd

2. Awk command

#Row matching statement awk '' can only use single quotation marks
Awk [option parameter]'pattern1 {action1} pattern2 {action2}...'File name
Option parameters:
    -F FS: Specifies the break separator of the input file. FS is a string or a regular expression, such as - F:
    -V var = value: assign a user-defined variable
Pattern: the content that awk looks up in the data is the matching pattern
Action: a series of commands executed when a match is found
Case practice:
  • Data preparation

    [[email protected] ~]# cp /etc/passwd ./
  • Search the passwd file for all lines that start with the root keyword, and output the seventh column of the line

    [[email protected] ~]# awk -F: '/^root/ {print $7}' passwd
  • Search all the lines ending in / bin / bash in passwd file, and output the first and seventh columns of the line, which are divided by “‘”

    [[email protected] ~]# awk -F: '/\/bin\/bash$/ {print $1" - "$7}' passwd 
  • Only the first and seventh columns of / etc / passwd are displayed, separated by tab, and the column name “user, shell” is added in front of all rows, and “PD, / bin / PD” is added in the last row

    #Note: begin is executed before all data rows are read; end is executed after all data rows are read
    awk -F: 'BEGIN{print "user\tshell"} {print $1"\t"$7} END{print "pd\t/bin/pd"}' passwd
  • Increase the user ID in passwd file by 1 and output

    [[email protected] ~]# awk -v i=1 -F: '{print $3+1}' passwd
Built in variables of awk:
variable explain
FILENAME Current file name
NR The number of records that have been read out is the line number, starting from 1
NF The number of fields in a record
Case practice:
  • Statistics passwd file name, line number of each line, number of columns of each line

    Awk - F: '{print "file name:" file name "\ \ T" "line:" NR "\ \ T" "column:" NF}' passwd
    File name: passwd line: 1 column: 7
    ...
    ...
    File name: passwd line: 23 column: 7
  • Cutting IP

    [[email protected] ~]# ifconfig eth0 | grep "inet" | awk -F " " '{print $2}'
    172.17.229.160
  • Inquiry test.txt The line number of the empty line

    [[email protected] ~]# awk '/^$/ {print NR}' test.txt 
    4
    6
  • There are documents grade.txt The contents are as follows:

    Zhang San 40
    Li Si 50
    Wang Wu 60

    Find the sum of the second column and output

    awk -F " " '{sum+=$2} END{print sum}' grade.txt
  • Relational operators

    # a.txt
    ID    Name    PHP    Linux    MySQL    Avg
    1   peiqi    82  95      86      87.66 
    2   pd      74  96      87      85.66 
    3   jaz     99  83      93      91.66
    [[email protected] ~]# cat s.txt | awk '$6>=87.66 {printf $2 "\n"}' | grep -v Name
    peiqi
    jaz

3. Sed command

Sed is a stream editor that processes one line at a time. When processing, the current processing line is stored in the temporary buffer, which is called “mode space”, and then the content of the buffer is processed with the SED command. After the processing, the content of the buffer is sent to the screen. The next line is then processed, and this is repeated until the end of the file. The contents of the file do not change unless you use redirection to store the output.

Sed [option parameter] '[action]' file name
#Option parameters:
    -e: Edit sed action directly in command line mode
    -n: Generally, the SED command will output all the data to the screen. If this option is added, only the lines processed by the SED command will be output to the screen
    -i: Using the modified result of sed to modify the file reading the data directly, instead of outputting it from the screen
# action:
    a: Append to add one or more rows after the current row. When adding multiple lines, except for the last line, the end of each line needs to be used to represent the unfinished data
    c: Line replacement: replace the original data line with the string after C. when replacing multiple lines, except for the last line, the end of each line needs to be used to represent the unfinished data
    i: To insert one or more rows before the current row. When inserting multiple rows, except for the last row, the end of each row needs to be represented by the incomplete data
    d: Delete, delete the specified row
    p: Print, output the specified line
    s: String replacement, replacing one string with another. The format is "line range s / old string / new string / g"
Data preparation:
# grade.txt
ID    Name    PHP    Linux    MySQL    Avg
1   peiqi    82  95      86      87.66 
2   pd      74  96      87      85.66 
3   jaz     99  83      93      91.66
  • Look at the second line of the file

    [[email protected] ~]# sed -n '2p' grade.txt 
    1   peiqi    82  95      86      87.66 
  • Insert 4A 4 Alex 88 93 79 86.66 into the grade.txt Below line 4

    sed '4a 4 alex 88 93 79 86.66' grade.txt
  • Delete all lines in the file that contain Peiqi

    sed '/peiqi/d' grade.txt
  • Delete the data from lines 2 to 4, and delete the content (- I parameter)

    sed -i '2,4d' grade.txt
  • Insert two rows of data before the second row

    sed '2i hello \
    world' grade.txt
  • Will grade.txt Replace Peiqi in the file with Qiaozhi

    #G means global, replace all
    sed 's/peiqi/qiaozhi/g' grade.txt
  • At the same time grade.txt Replace Peiqi and Jaz in the file with null

    sed 's/peiqi//g;s/jaz//g' grade.txt

9、 Character processing command

1. Sort command sort

Sort file name
#Options: 
    -f: Ignore case 
    -n: Sort by numeric type, and sort by string type by default 
    -r: Reverse sort 
    -t: Specifies the separator. The default separator is tab 
    -K, n [, M]: sort according to the specified field range. Start with field n and end with field m (default to end of line)
Data preparation:
[[email protected] ~]# vim sort.txt
    bb:40:5.4
    bd:20:4.2
    xz:50:2.3
    cl:10:3.5
    ss:30:1.6
  • Sort in reverse order according to the third column after division

    [[email protected] ~]# sort -t : -nr -k 3 sort.txt
    bb:40:5.4
    bd:20:4.2
    cl:10:3.5
    xz:50:2.3
    ss:30:1.6

2. Statistics command WC

WC instruction can be used to calculate the number of bytes, words, or columns of a file. If the file name is not specified, or the given file name is “-“, WC instruction will read data from the standard input device.

WC [options] file name
#Options: 
    -l: Only count the number of rows 
    -w: Only count the number of words 
    -m: Count characters only
[[email protected] ~]# wc -l -w -m sort.txt 
 5  5 50 sort.txt

10、 Conditional judgment

1. Two judgment formats

[[email protected] ~]# test -e /etc/passwd
[[email protected] ~]# echo $?
0
[[email protected] ~]# [ -e /etc/passwd ]
[[email protected] ~]# echo $?
0
#If the first command is executed correctly, print yes, otherwise print No
[[email protected] ~]# [ -e /etc/pwd ] && echo "yes" || echo "no"
no

2. Judge by file type

Test options effect
-B document Judge whether the file exists and whether it is = = block device = = file
-C file Judge whether the file exists and whether it is = = character device = = file
-D document Judge whether the file exists, and whether it is = = directory = = file
-E file Determine whether the file exists
-F file Judge whether the file exists, and whether it is = = normal = = file
-L file Judge whether the file exists and whether it is = = symbolic link = = file
-P file Judge whether the file exists and whether it is = = pipeline = = file
-S file Judge whether the file exists and whether it is = = pipeline = = file
-S file Determine whether the file exists and whether it is a = = socket = = file

3. Judge according to file authority

Test options effect
-R file Judge whether the file exists and whether the file has = = read permission==
-W file Determine whether the file exists and whether the file has = = write permission==
-X file Judge whether the file exists, and whether the file has = = execution permission==
-U file Determine whether the file exists and whether the file has suid permission
-G file Determine whether the file exists and whether the file has sgid permission
-K file Determine whether the file exists and whether the file has sbit permission

4. Compare between the two files

Test options effect
File 1 – NT file 2 Judge whether the modification time of file 1 is newer than that of file 2
File 1 – ot file 2 Judge whether the modification time of file 1 is older than that of file 2
File 1 – EF file 2 Judging whether the inode number of file 1 is consistent with that of file 2 can be understood as whether two files are the same file. This judgment is a good way to judge hard links

5. Comparison between two integers

Test options effect
Integer 1 – EQ integer 2 Judge whether integer 1 is equal to integer 2
Integer 1 – NQ integer 2 Judge whether integer 1 is not equal to integer 2
Integer 1 – GT integer 2 Judge whether integer 1 is greater than integer 2
Integer 1 – integer 2 Judge whether integer 1 is less than integer 2
Integer 1 – integer 2 Judge whether integer 1 is greater than or equal to integer 2
Integer 1 – Le integer 2 Judge whether integer 1 is less than or equal to integer 2

6. Judgment of string

Test options effect
-Z string Determine whether the string is empty
-N string Determine whether the string is non empty
String 1 = = string 2 Determine whether string 1 is equal to string 2
String 1! = string 2 Determine whether string 1 is not equal to string 2

7. Multiple condition judgment

Test options effect
Judgment 1 – a judgment 2 Logic and, judgment 1 and judgment 2 are true, and the final result is true
Judgment 1 – O judgment 2 Logic or, if one of judgment 1 and judgment 2 holds, the final result is true
! judgment Logical negation makes the original judgment negative

11、 Process control

1. If statement

Single branch if conditional statement:
If; then 
    program 
fi 
#Or 
If [conditional judgment] 
then 
    program 
fi

Several points should be paid attention to in single branch conditional statements

  • The if statement ends with fi, which is different from the curly bracket ending in general language
  • [conditional judgment formula] is to use the test command to judge, so there must be a space between the brackets and the conditional judgment formula
  • Then followed by the program executed after meeting the conditions, can be placed after [], with; segmentation. It can also be written in a new line, so it is unnecessary
#!/bin/bash 
#Statistics root partition usage 
#The root partition utilization rate is assigned to rate as a variable value 
rate=$(df -h | grep "/dev/vda1" | awk '{print $5}' | cut -d "%" -f 1) 
if [ $rate -ge 80 ];then 
    echo "Warning! /dev/vda1 is full of!" 
fi
Double branch if conditional statement:
If; then 
    The procedure to be carried out when the condition is established 
else 
    When the condition does not hold, another procedure is executed 
fi

Example 1: Backup MySQL database

#!/bin/bash 
#Backup MySQL database

#Synchronization system time 
ntpdate asia.pool.ntp.org &>/dev/null
#Assign the current system time to the variable date according to the format of "month day" 
date=$(date +%Y%m%d) 
#Count the size of MySQL database, and give the size variable 
size=$(du -sh /var/lib/mysql) 

if [ -d /tmp/dbbak ];then 
    echo "Date: $date!" > /tmp/dbbak/dbinfo.txt 
    echo "Data size: $size" >> /tmp/dbbak/dbinfo.txt 
    cd /tmp/dbbak 
    tar -zcf mysql-lib-$date.tar.gz /var/lib/mysql dbinfo.txt &>/dev/null 
    rm -rf /tmp/dbbak/dbinfo.txt 
else 
    mkdir /tmp/dbbak 
    echo "Date: $date!" > /tmp/dbbak/dbinfo.txt 
    echo "Data size: $size" >> /tmp/dbbak/dbinfo.txt 
    cd /tmp/dbbak 
    tar -zcf mysql-lib-$date.tar.gz /var/lib/mysql dbinfo.txt &>/dev/null 
     rm -rf /tmp/dbbak/dbinfo.txt
fi

Example 2: judge whether MySQL is started or not

#!/bin/bash
port=$(netstat -anp | grep mysql | grep tcp6 | awk '{print $4}' | cut -d ":" -f 4) 
if [ "$port" == 3306 ];then 
    echo "mysql is running..."
else 
    service mysqld start
    echo "mysql is starting..." 
fi
Multi branch if conditional statement:
If; then 
    When condition 1 is true, program 1 is executed 
Elif; then 
    When condition 2 is true, execute procedure 2
else 
    When all conditions do not hold, the program is executed at last 
fi

2. Case statement

Case statement and if elif… All else statements are multi branch conditional statements, but different from if multi branch conditional statements, case statements can only judge one kind of conditional relations, while if statements can judge many kinds of conditional relations.

Case $variable name in 
"Value 1") 
    If the value of the variable is equal to the value 1, program 1 is executed 
;; 
"Value 2") 
    If the value of the variable is equal to the value 2, program 2 is executed 
;;  
*) 
    If none of the variables have the above values, execute this program 
;; 
esac

3. For loop

Grammar 1:
For variable in value 1 value 2 value 3 
    do 
        program 
    done

Example 1: printing time

#!/bin/bash
for time in morning noon afternoon evening 
    do 
        echo "This time is $time!" 
    done 

Example 2: batch decompression script

#!/bin/bash
cd /opt 
ls *.tar.gz > tar.log 
for i in $(cat tar.log) 
    do 
        tar -zxf $i -C /test &>/dev/null 
    done 
rm -rf /opt/tar.log
Grammar 2:
For ((initial value; cycle control condition; variable change)) 
    do 
        program 
    done

Example: from 1 to 100

#!/bin/bash
total=0 
for (( i=1;i<=100;i=i+1 )) 
    do 
        total=$(( $total+$i )) 
    done 
echo "The sum of 1+2+...+100 is: $total"

4. While loop and until loop

While loop:

A while loop is an indefinite loop, also known as a conditional loop. As long as the conditional judgment is true, the cycle will continue until the conditional judgment is not true. This is different from the fixed loop of for.

While 
    do 
        program 
    done

Example: from 1 to 100

#!/bin/bash
i=1 
total=0
while [ $i -le 100 ] 
    do 
        total=$(( $total+$i )) 
        i=$(( $i+1 )) 
    done 
echo "The sum of 1+2+...+100 is: $total"
Until loop:

As opposed to the while loop, the until loop is executed as long as the conditional judgment is not true. Once the cycle condition is established, the cycle is terminated.

Until [conditional judgment] 
    do 
        program 
    done

Example: from 1 to 100

#!/bin/bash
i=1 
total=0
#The loop stops until the value of variable I is greater than 100 
until [ $i -gt 100 ] 
    do 
        total=$(( $total+$i )) 
        i=$(( $i+1 )) 
    done 
echo "The sum of 1+2+...+100 is: $total"

  1. a-z