Notes on Shell Scripting

Time:2020-12-20

Recently, I’m learning shell programming, through MOOCSLearn Shell Scripting with 360 ArchitectsSo I type out some examples in the course and record them here. We will continue to write some script examples to play with.

Example 1:
The variable string =’bigdata process framework is Hadoop, Hadoop is an open source program ‘. After executing the script, the string variable string is printed out, and the following options are given to the user:
(1) Print variable length
(2) Delete all Hadoop in the string
(3) Replace the first Hadoop with MapReduce
(4) Replace all Hadoop with MapReduce
The user can execute the corresponding function by inputting array 1|2||3|4. Input q|q to exit interactive mode

#!/bin/bash

string='Bigdata process framework is Hadoop,Hadoop is an open source programm'

function print_tips()
{
    echo '====================================='
    Echo "variable string = ${string}"
    Echo 'input 1, print variable length'
    Echo 'input 2 to delete all Hadoop'
    Replace the first one with "echo" and input "Hadoop '
    Echo 'input 4, replace all Hadoop with MapReduce'
    Echo 'enter q|q to exit the script'
    echo '====================================='
}

function print_strlen()
{
    Echo "string length is: ${#string}"
}

function print_delete_hadoop()
{
    Echo "delete all Hadoop in the string, the string is: ${string // Hadoop}"
}

function print_replace_hadoop()
{
    Echo "replace the first Hadoop with MapReduce, and the string is ${string / Hadoop / MapReduce}"
}

function print_replace_all_hadoop()
{
    Echo "replace all Hadoop with MapReduce, and the string is ${string // Hadoop / MapReduce}"
}

#print_tips
#print_strlen
#print_delete_hadoop
#print_replace_hadoop
#print_replace_all_hadoop

while True
do
  print_tips
  # shellcheck disable=SC2162
  Read - p 'please enter the command:' input
  case $input in
  1)
    print_strlen
    ;;
  2)
    print_delete_hadoop
    ;;
  3)
    print_replace_hadoop
    ;;
  4)
    print_replace_all_hadoop
    ;;
  q|Q)
      Echo 'exit script'
    exit
    ;;
  *)
    Echo 'invalid instruction'
    ;;
  esac

done

Example 2
Write a script, create the directory / TMP / scripts, switch the working directory to this directory, and copy the words.txt Go to the current directory and rename it as test. Change the other user permissions of the files in the test and the directory under it to no permission.

#!/bin/bash

echo '============================='
Echo 'create directory'
mkdir /tmp/scripts

echo '=============================='
Echo 'switch to new directory'
cd /tmp/scripts
directory=`pwd`
Echo "the current working directory is: ${directory}"

user=`whoami`
userDirectory="/Users/${user}"
echo ${userDirectory}
echo '==============================='
Echo 'copy TXT culture to / TMP / scripts directory'
cp "${userDirectory}/words.txt" /tmp/scripts/test.txt

echo '================================'
Echo 'modify file permissions'
chmod 700 /tmp/scripts/test.txt

Example 3
According to the system time, how many weeks are left in this year and how many weeks have passed

#!/bin/bash

date=`date "+%j"`
Echo "this year has passed $(($date / 7)) weeks"
Echo "there are still $((((365 - $date) / 7)) weeks left this year"

Example 4
Judge whether the nginx process exists. If not, the process will be pulled up automatically

#!/bin/bash

check=`ps -ef | grep nginx | grep -v color | wc -l`

if [ $check -eq 1 ];then
        `sudo nginx`
        Echo 'nginx is not enabled. Now it is enabled.'
else
        Echo 'nginx enabled'
fi

Example 5
Prompt the user to enter a positive integer num, and then calculate 1 + 2 + 3 + +The value of num must be judged whether num is a positive integer. If it does not match, the user can re-enter it

#!/bin/bash

while True
do
    Please enter a positive integer: "num"
    expr $num + 1 &>/dev/null
    if [ $? -eq 0 ];then
        if [ $num -gt 0 ];then
          sum=0
            for((i=1;i<=num;i++))
            do
                sum=`expr $sum + $i`
            done
            echo "1+2+3+...+${num}=${sum}"
            exit
        else
            Echo "the input character is not a positive integer, please re-enter"
        fi
    else
        Echo "the input character is not a positive integer, please re-enter"
    fi
done

Example 6

Example of find command usage

1. Search the file (non directory) in the / etc directory. The file name ends with conf and is greater than 10k, and then delete it

find /etc -type f -name '*.conf' -size +10k -exec rm -f {}

2. Delete the files that end in log in / var / log directory and the change time is more than seven days

find /var/log -name '*.log' -mtime +7 -exec rm -rf {}

3. The search condition is the same as that in example 1, except that it is not deleted, but copied to / root / conf directory

find /etc -size +10k -type f -name '*.conf' -exec cp {} /root/conf

Example 7

Examples of grep command usage

1. Display the lines in the / proc / meminfo file that begin with big and small s

cat /proc/meminfo |grep -i ^"s"

2. Display lines not ending in / bin / bash in the / etc / passwd file

cat /etc/passwd|grep -v "/bin/bash"$

3. Find two or three digits in / etc / passwd

cat /etc/passwd | grep -o "\<[1-9][0-9]\{1,2\}\>" | sort -nr | uniq

Example 8

Examples of SED command usage

1. Display 10-30 lines of / cat / passwd file

sed -n '10,30p' /ect/passwd

2. Display the line containing root from the file / cat / passwd to the line containing shutdown

sed -n '/root/,/shutdown/p' /etc/passwd

Insert the file at the beginning of Hello

sed -i '1s/^/hello \n/g' passwd

4. Replace the nologin of passwd file with nologin111

sed -i 's/nologin/nologin111/g' passwd

5. Delete the line containing redis in passwd file

sed -i '/redis/d' passwd

Example 9

Processing a MySQL like configuration file my.cnf For example
Write script to achieve the following functions: output file has several segments, and for each segment can count the total number of configuration parameters

The following file configuration my.cnf

[client]
#password    = your_password
port        = 3306
socket        = /tmp/mysql.sock

[mysqld]
port        = 3306
socket        = /tmp/mysql.sock
datadir = /www/server/data
default_storage_engine = InnoDB
performance_schema_max_table_instances = 400
table_definition_cache = 400
skip-external-locking
key_buffer_size = 64M
max_allowed_packet = 100G
table_open_cache = 256
sort_buffer_size = 1M
net_buffer_length = 4K
read_buffer_size = 1M
read_rnd_buffer_size = 256K
myisam_sort_buffer_size = 16M
thread_cache_size = 32
query_cache_size = 32M
tmp_table_size = 64M
sql-mode=NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES

explicit_defaults_for_timestamp = true
#skip-name-resolve
max_connections = 500
max_connect_errors = 100
open_files_limit = 65535

log-bin=mysql-bin
binlog_format=mixed
server-id = 1
expire_logs_days = 10
slow_query_log=1
slow-query-log-file=/www/server/data/mysql-slow.log
long_query_time=3
#log_queries_not_using_indexes=on


innodb_data_home_dir = /www/server/data
innodb_data_file_path = ibdata1:10M:autoextend
innodb_log_group_home_dir = /www/server/data
innodb_buffer_pool_size = 256M
innodb_log_file_size = 128M
innodb_log_buffer_size = 32M
innodb_flush_log_at_trx_commit = 1
innodb_lock_wait_timeout = 50
innodb_max_dirty_pages_pct = 90
innodb_read_io_threads = 2
innodb_write_io_threads = 2

[mysqldump]
quick
max_allowed_packet = 500M

[mysql]
no-auto-rehash

[myisamchk]
key_buffer_size = 64M
sort_buffer_size = 1M
read_buffer = 2M
write_buffer = 2M

[mysqlhotcopy]
interactive-timeout

Expected output results:
1.client 2
2.mysqld 41
3.mysqldump 2
4.mysql 1
5.myisamchk 4
6.mysqlhotcopy 1

#!/bin/bash

file='my.cnf'

function get_all_paragraph()
{
    echo "`sed -n '/\[*\]/p' $file | sed -e 's/\[//g' -e 's/\]//g'`"
}

function get_paragraph_config()
{
    echo "`sed -n '/\['$1'\]/,/\[*\]/p' $file | grep -v '\[*\]' | grep -v ^'#' | grep -v ^$ | wc -l`"
}

paragraphs=`get_all_paragraph`

index=0
for item in $paragraphs
do
  index=`expr $index + 1`
  count=`get_paragraph_config $item`
  echo "$index . $item $count"
done

Example 10

1. Delete all comment lines and empty lines in the configuration file

sed -i '/^\s*$/d' *.conf && sed -i '/^[[:space:]]*#/d' *.conf

2. In the configuration file, add * before all lines that do not start with ා. Note: the lines beginning with #, are not added

sed -i '/^[^#]/s/^/*&/g' passwd
sed -i '/^[^#]/i *' passwd

Example 11

1. Print the 7th field in / etc / passwd in string format with “: as the separator

awk 'BEGIN{FS=":"}{printf "%s\n",$7}' /etc/passwd | grep -v ^$

2. Print the third field in / etc / passwd in decimal format with “: as the separator

awk 'BEGIN{FS=":"}{printf "%d\n",$3}' /etc/passwd | grep -v ^$

3. Match all lines containing root string in / etc / passwd file

awk '/root/{print $0}' /etc/passwd

4. Match all lines starting with root in the / etc / passwd file line

awk '/^root/{print $0}' /etc/passwd

5. Use awk to calculate the number of lines in / etc / passwd that begin with ා

awk 'BEGIN{i=0}/^#/{i++}END{print i}' /etc/passwd

6. Calculate the average student course score. The student course file is as follows

Allen 80 90 96 98
Mike 75 84 66 78
Jerry 91 58 45 87
Bill 84 58 68 55
Li 69 59 78 63 81

awk '{printf "%s:%0.2f\n",$1,($2+$3+$4+$5)/4}' words.txt

7. Calculate 1 + 2 + 3 + 4 + +The sum of 100, please use while, do, while, for three loops

$>>vim for.awk

BEGIN{
        sum=0
        for(i=0;i<=100;i++)
        {
                sum+=i
        }
        print sum
}

$>>awk -f for.awk
$>>vim while.awk

BEGIN{
        sum=0;
        i=1;
        while(i<=100)
        {
                sum+=i
                i++
        }
        print sum
}

$>>awk -f while.awk
$>>vim dowhile.awk

BEGIN{
        sum=0
        i=1
        do{
                sum+=i
                i++
        }while(i<=100)
        print sum
}

$>>awk -f dowhile.awk

Example 12

Implement a script to view the running status of the process under configuration. The output is as follows:

PROCESS_NAME GROUP STATUS PID CPU MEMORY START_TIME
nginx web RUNNING 59 0.0 0.0 8/26 09:10:05 2020

The configuration information is as follows:

[service]
Web
DB
Code

[Web]
nginx
apache

[DB]
mysql

[Code]
php
python

The implementation code is as follows:

#! /bin/bash

conf_name='status.conf'

#Get the group list
function get_conf_group_list() {
    group_list=`sed -n '/\[service\]/,/\[*\]/p' $conf_name | grep -v '\[*\]' | grep -v '^$'`
    echo $group_list
}

#Get the information under a group
function get_conf_item_list() {
    item_list=`sed -n '/\['$1'\]/,/\[*\]/p' $conf_name | grep -v '\[*\]' | grep -v '^$'`
    echo $item_list
}

#Get all item data
function get_all_item() {
  index=1
  for item in `get_conf_group_list`
  do
    group_item=`get_conf_item_list $item`
    for item in $group_item
    do
      echo $item
    done
  done
}
#Get service PID
function get_service_pid() {
    service_ps=`ps -ef | grep $1 | grep -v $$ | grep -v grep | awk '{print $2}'`
    echo "$service_ps"
}

#Check if the service is on
function check_service_is_alive() {
    check=`ps -ef | grep $1 | grep -v $$ | grep -v grep | wc -l`
    if [ $check -eq 0 ]; then
      return 1
#        echo "$1:STOPPED"
    else
      return 0
#        echo "$1:RUNNING"
    fi
}

#Get the running status of process CPU
function get_pid_cpu(){
  pid=`ps aux | grep $1 | grep -v $$ | grep -v grep | grep $2 | awk '{print $3;exit}'`
  echo $pid
}

#Get the running status of process memory
function get_pid_memory(){
  pid=`ps aux | grep $1 | grep -v $$ | grep -v grep | grep $2 | awk '{print $4;exit}'`
  echo $pid
}

function get_pid_lstart() {
    PID = ` PS - p $1 - O lstart | grep - V started | sed - E's / three // g '- E's // - / g'`
    echo $pid
}

awk 'BEGIN{printf "%-15s%-10s%-10s%-10s%-10s%-10s%-10s\n","PROCESS_NAME","GROUP","STATUS","PID","CPU","MEMORY","START_TIME"}'

for item in `get_conf_group_list`
do
  for sonItem in `get_conf_item_list $item`
  do
    check=`check_service_is_alive $sonItem`
    #Stop allowing
    if [ $? -eq 1 ]; then
      awk -v var1=$sonItem -v var2=$item 'BEGIN{printf "%-15s%-10s%-10s%-10s%-10s%-10s%-10s\n",var1,var2,"STOPPED","NULL","NULL","NULL","NULL"}'
    else
      for pid in `get_service_pid $sonItem`
      do
        cpu=`get_pid_cpu $sonItem $pid`
        memory=`get_pid_memory $sonItem $pid`
        lstart=`get_pid_lstart $pid`
        awk -v var1=$sonItem -v var2=$item -v var3=$pid -v var4=$cpu -v var5=$memory -v var6=$lstart 'BEGIN{printf "%-15s%-10s%-10s%-10s%-10s%-10s%-10s\n",var1,var2,"RUNNING",var3,var4,var5,var6}'
      done
    fi
  done
done

This work adoptsCC agreementThe author and the link to this article must be indicated in the reprint