Missing lesson in Computer Education – MIT – L2 – shell tools and scripts

Time:2021-4-25

https://missing.csail.mit.edu/
https://missing-semester-cn.g…
https://www.bilibili.com/vide…

note

Shell script

Special variables

  • $0Script name
  • $1reach$9-Parameters of the script.$1Is the first parameter, and so on.
  • [email protected]-All parameters
  • $#-Number of parameters
  • $?-The return value of the previous command
  • $$-The process ID of the current script
  • !!-Complete previous command, including parameters. Common application: when you fail to execute command due to insufficient authority, you can usesudo !!Try again.
  • $_-The last parameter of the previous command. If you are using an interactive shell, you can clickEscThen type.To get this value.

Process replacement

A similar feature of unpopular isProcess replacement ( process substitution ), <( CMD )Will carry outCMDAnd output the results to a temporary file, and<( CMD )Replace with a temporary file name. This is useful when we want the return value to be passed through a file instead of stdin. For example,diff <(ls foo) <(ls bar)The folder is displayedfooandbarThe difference between the two files.

Globbing

convert image.{png,jpg}
#Opening of exhibition
convert image.png image.jpg

cp /path/to/project/{foo,bar,baz}.sh /newpath
#Opening of exhibition
cp /path/to/project/foo.sh /path/to/project/bar.sh /path/to/project/baz.sh /newpath

#It can also be used in combination with general configuration
mv *{.py,.sh} folder
#All *. Py and *. Sh files are moved

mkdir foo bar

#The following command will create foo / A, foo / B,... Foo / h, bar / A, bar / B,... Bar / H files
touch {foo,bar}/{a..h}
touch foo/x bar/y
#Shows the difference between Foo and bar files
diff <(ls foo) <(ls bar)
#Output
# < x
# ---
# > y

shebang

Note that scripts don’t have to be written in Bash to be called in the terminal. For example, this is a python script that outputs the input parameters in reverse order

#!/usr/local/bin/python import sys
for arg in reversed(sys.argv[1:]):
    print(arg) 

Shell knows to use the Python interpreter instead of the shell command to run this script, because the first line of the script is emptyshebang)。

stayshebangUse in lineenvCommand is a good practice, it will use the program in the environment variable to parse the script, so as to improve the portability of your script.envWe will make use of what we introduced in the first lecturePATHEnvironment variables. For example, aenvThis is what shebang looks like#!/usr/bin/env python

shellcheck

to writebashScripts can be awkward and counterintuitive at times. for exampleshellcheckSuch tools can help you locate errors in SH / bash scripts. For example:

Missing lesson in Computer Education - MIT - L2 - shell tools and scripts

Missing lesson in Computer Education - MIT - L2 - shell tools and scripts

Shell tools

See how commands are used

Find files

Comparison between locate and find

Find code

An example of grep:

Missing lesson in Computer Education - MIT - L2 - shell tools and scripts

Example of RG:

#Find all the files that use the requests library
rg -t py 'import requests'
#Find all files not written to shebang (including hidden files)
rg -u --files-without-match "^#!"
#Find all foo strings and print the next five lines
rg foo -A 5
#Print matching Statistics (number of matching rows and files)
rg --stats PATTERN

Find shell commands

It is worth noting that when you enter a command, if you add a space at the beginning of the command, it will not be added to the shell record. This feature is used when you enter commands that contain passwords or other sensitive information. If you accidentally forget to add a space in front of it, you can edit it.bash_historyor.zhistoryTo manually remove that item from history.

Folder navigation

Oh-my-zsh? Novice on the roadSetting up Windows Terminal, WSL and Oh-my-Zsh

After class exercises

Exercise 1

Missing lesson in Computer Education - MIT - L2 - shell tools and scripts

Exercise 2

macro.sh:

macro() {
    macro_dir=$(pwd)
    echo "I am in $macro_dir" | tee /mnt/f/code/learn/missing-semester/l2-shell-tools/macro.txt
}

polo.sh:

polo() {
    cd "$macro_dir" || exit
    macro
}

Missing lesson in Computer Education - MIT - L2 - shell tools and scripts

Exercise 3

ex3_solution.sh:

#!/usr/bin/env bash

./ex3_problem.sh > ex3_result.txt 2> ex3_result.txt
state=$?
count=0

while [[ state -eq 0 ]]; do
    ./ex3_problem.sh >> ex3_result.txt 2>> ex3_result.txt
    state=$?
    count=$((count + 1))
done

cat ex3_result.txt
echo "ex3_problem ran $count times before failure"

Missing lesson in Computer Education - MIT - L2 - shell tools and scripts

Exercise 4

$ tree ex4_html_folder
ex4_html_folder
├── 1.html
├── 1.txt
├── a
│   ├── a 1.html
│   ├── a 1.txt
│   ├── a 2.txt
│   └── a 3.txt
└── b
    ├── b 1.html
    ├── b 2.html
    └── b 3.html

2 directories, 9 files

reference resourcestldr xargsExamples of usage are given

 - Delete all files with a .backup extension (-print0 uses a null character to split file names, and -0 uses it as delimiter):
   find . -name {{'*.backup'}} -print0 | xargs -0 rm -v

tldr tarAn example of tar command is given

 - [c]reate an archive from [f]iles:
   tar cf {{target.tar}} {{file1}} {{file2}} {{file3}}

 - E[x]tract a (compressed) archive [f]ile into the target directory:
   tar xf {{source.tar[.gz|.bz2|.xz]}} --directory={{directory}}

 - Lis[t] the contents of a tar [f]ile [v]erbosely:
   tar tvf {{source.tar}}

So the answer to this question is as follows:

find . -name "*.html" -print0 | xargs -0 tar cf html.tar

Verify:

$ tar tvf html.tar
-rwxrwxrwx yzj/yzj           0 2021-01-29 15:00 ./ex4_html_folder/1.html
-rwxrwxrwx yzj/yzj           0 2021-01-29 15:25 ./ex4_html_folder/a/a 1.html
-rwxrwxrwx yzj/yzj           0 2021-01-29 15:25 ./ex4_html_folder/b/b 1.html
-rwxrwxrwx yzj/yzj           0 2021-01-29 15:25 ./ex4_html_folder/b/b 2.html
-rwxrwxrwx yzj/yzj           0 2021-01-29 15:25 ./ex4_html_folder/b/b 3.html

$ mkdir ex4_html_folder_extracted
$ tar xf html.tar --directory=ex4_html_folder_extracted
$ tree ex4_html_folder_extracted
ex4_html_folder_extracted
└── ex4_html_folder
    ├── 1.html
    ├── a
    │   └── a 1.html
    └── b
        ├── b 1.html
        ├── b 2.html
        └── b 3.html

3 directories, 5 files

The above solution is to change the separator of the find command output from the original newline character to null, and then let xargs also use null as the separator. You can also use the – D option to specify the newline character as the separator

find . -name "*.html" | xargs -d "\n" tar cf html.tar

Exercise 5

#List files in the most recent modified order
$ find . -type f -print0 | xargs -0 ls -lt --color
-rwxrwxrwx 1 yzj yzj 10240 Jan 29 15:27  ./html.tar
-rwxrwxrwx 1 yzj yzj     0 Jan 29 15:25 './ex4_html_folder/a/a 1.html'
-rwxrwxrwx 1 yzj yzj     0 Jan 29 15:25 './ex4_html_folder_extracted/ex4_html_folder/a/a 1.html'
-rwxrwxrwx 1 yzj yzj     0 Jan 29 15:25 './ex4_html_folder/a/a 3.txt'
-rwxrwxrwx 1 yzj yzj     0 Jan 29 15:25 './ex4_html_folder/a/a 2.txt'
-rwxrwxrwx 1 yzj yzj     0 Jan 29 15:25 './ex4_html_folder/a/a 1.txt'
-rwxrwxrwx 1 yzj yzj     0 Jan 29 15:25 './ex4_html_folder/b/b 1.html'
-rwxrwxrwx 1 yzj yzj     0 Jan 29 15:25 './ex4_html_folder/b/b 2.html'
-rwxrwxrwx 1 yzj yzj     0 Jan 29 15:25 './ex4_html_folder/b/b 3.html'
-rwxrwxrwx 1 yzj yzj     0 Jan 29 15:25 './ex4_html_folder_extracted/ex4_html_folder/b/b 1.html'
-rwxrwxrwx 1 yzj yzj     0 Jan 29 15:25 './ex4_html_folder_extracted/ex4_html_folder/b/b 2.html'
-rwxrwxrwx 1 yzj yzj     0 Jan 29 15:25 './ex4_html_folder_extracted/ex4_html_folder/b/b 3.html'
-rwxrwxrwx 1 yzj yzj     0 Jan 29 15:01  ./ex4_html_folder/1.txt
-rwxrwxrwx 1 yzj yzj     0 Jan 29 15:00  ./ex4_html_folder/1.html
-rwxrwxrwx 1 yzj yzj     0 Jan 29 15:00  ./ex4_html_folder_extracted/ex4_html_folder/1.html
-rwxrwxrwx 1 yzj yzj   837 Jan 29 10:14  ./ex3_result.txt
-rwxrwxrwx 1 yzj yzj   291 Jan 29 10:11  ./ex3_solution.sh
-rwxrwxrwx 1 yzj yzj   205 Jan 29 09:58  ./ex3_problem.sh
-rwxrwxrwx 1 yzj yzj    58 Jan 29 09:52  ./macro.txt
-rwxrwxrwx 1 yzj yzj    49 Jan 29 09:48  ./polo.sh
-rwxrwxrwx 1 yzj yzj   129 Jan 29 09:44  ./macro.sh
-rwxrwxrwx 1 yzj yzj    50 Jan 28 21:41  ./mcd.sh
-rwxrwxrwx 1 yzj yzj   509 Jan 28 21:10  ./example.sh

#Find the most recently modified file
$ find . -type f -print0 | xargs -0 ls -lt --color | head -n1
-rwxrwxrwx 1 yzj yzj 10240 Jan 29 15:27 ./html.tar

Recommended Today

Build HTTP service with C + + Mongoose

Mongoose source code address:https://github.com/cesanta/mo… Mongoose user manual:https://www.cesanta.com/devel… Mngoose set up HTTP service #include <string> #include “mongoose.h” using namespace std; static const char *s_http_port = “8000”; static void ev_handler(mg_connection *nc, int ev, void *ev_data) { struct http_message *hm = (struct http_message *) ev_data; if (ev == MG_EV_HTTP_REQUEST) { std::string uri; if (hm->uri.p && hm->uri.p[0] == ‘/’) […]