Linux13 — file search and compression, pipeline

Time:2022-5-8

1 file search

#Find command: in Linux system, find files according to our requirements


# 1. Format:
    Find [query path] [matching pattern] [matching rule]

# 2. Match pattern:
    -Name: match by file name 

    -Type: match by file type
      Common file types:
        f: Ordinary file
        d: Normal folder
        b: Block device file
        c: Character device file
        s: Socket file
        l: Link file
        
    -Perm: match by file permissions
     
           
    -User: match by file owner
    -Nouser: query the files # that have no owner, that is, the files that have been deleted by the owner
    
    -Group: match by file group
    -Nogroup: query files that do not belong to a group
     
    -Size: match by file size
        +Query more than n files
        -Query files smaller than n
         
    -Mtime: match by file modification time
    -CTime: match by file creation time
    -Atime: match by file access time
        +N query files n days ago
        -N query files within n days
    
    
    -a: And # and default
    -o: Or # or
    
    -Exec: process the matched content # xargs
     
     
# 3. Matching rules (combined with regular matching)
    *: matches any number of arbitrary characters (matches zero or more arbitrary characters)
    ? :  Match any character
    
    
#Supplement:
#Common file permissions:
    755: default permissions for folders
    644: default permissions for files

#Delete users and user groups
    Delete user: userdel #-r completely delete
    Delete user group: groupdel
    
#Various times to view files
     stat

Find case

# 1. Query the hosts file in the / etc directory  
    find /etc -name "hosts"

# 2. Query the files beginning with ifcfg in / etc directory
    find /etc/ -name "ifcfg*"
            
# 3. Query the / etc directory to Conf end file
    find /etc/ -name "*.conf"

# 4. Find out which files contain host in the file name in the / etc directory
    find /etc/ -name "*host*"

# 5. Query all ordinary files in / etc directory.
    find /etc/ -type f

# 6. Query all folders in / etc directory
    find /etc/ -type d
            
# 7. Query all block device files in the / dev / directory
    find /dev/ -type b
    
# 8. Query all character device files in the / dev directory
    find /dev/ -type c
    
# 9. Query out the ordinary files whose file name contains nginx in the / etc directory
    find /etc/ -name "*nginx*" -type f
            
# 10. Query out the files with 755 permissions under / root directory
    find /root/ -perm 755
            
# 11. Query the file whose owner is test01
    find /root/ -user test01
            
# 12. Query the deleted files of the owner
    find /root/ -nouser 

# 13. Query files belonging to test group
    find /root/ -group test
            
# 14. Query the deleted files belonging to the group
    find /root/ -nogroup
            
# 15. Query files modified 2 days ago
    find /root/ -mtime +2
            
# 16. Query files created within 2 days
    find /root/ -ctime -2

# 17. Query files accessed within 2 days
    find /root/ -atime -2
            
# 18. Query files larger than 1m
    find /root/ -size +1M
            
# 19. Query files less than 1m
    find /root/ -size -1M
            
# 20. Query files created within 3 days and delete them
    find /tmp/ -ctime -3 -type f -exec rm -rf {} \;
        -Exec: content after processing query
        {}: represents the queried content
        \;    :  fixed collocation
        
    find /tmp/ -ctime -3 -type f | xargs -I {} rm -rf {} 
        
        
# 21. Add all ordinary files created 3 days ago Bak suffix
    find /tmp -ctime -3 -type f -exec mv {} {}.bak \;
    
    find /tmp -ctime -3 -type f | xargs -I {} mv {} {}.bak
        
        
#Supplement:
    Xargs: format everything into one line
    Xargs - I {}: format all contents into one line and put them into {}, and other commands can take out the contents from {} for command execution
    
    Because the execution result of the previous command may contain line breaks, and when the execution result is passed as the second command parameter, the command cannot be executed (some commands cannot have line breaks)
    Therefore, you can filter the parameters through xargs and replace the newline with a space

2 compression software

1.gzip

# gzip
    Compression software makes the file into a compressed package, which will delete the original file and generate a new compressed package file
    
#Format:
    Compression: gzip [file path]
    Decompression: gzip - D [compressed package path]
    
#Disadvantages:
    Gzip cannot compress directories

2.bzip2

# bzip2
    Compression software makes the file into a compressed package, which will delete the original file and generate a new compressed package file
    
#Format:
    Compression: bzip2 [file path]
    Decompression: bzip2 - D [compressed package path]
    
#Disadvantages:
    Bzip2 cannot compress a directory
    
#Compare
    Gzip has a higher compression ratio than bzip2

3. Tar common

# tar
    Package files or directories without deleting or compressing the original files;
    But it can be used with gzip or bzip2 for packaging and compression
    
#Format:
    Tar [required parameters] [selection parameters] [package name] [original file or directory]
 
    #Required parameters:
        -f: Specifies the name of the compressed package
        -c: Package file # lowercase

        -z: Specify gzip compression. Generally, files compressed with gzip are compressed with tar. GZ as extension
        -j: Specifies to use bzip2 compression. Generally, files compressed with bzip2 are compressed in tar. Bz2 as extension
        
        -x: Decompression: there is no need to specify the compression type of the compressed package. It will automatically match the type of the compressed package and decompress by itself
        
        -v: Displays the process of compressing a compressed package
        -P: When the root directory exists in the compressed package, the root directory # capitalization is automatically removed
        -t: View the contents of the compressed package
        
    #Note: when parameters are used together, f is placed last, followed by the name of the compressed package
    
    #Select parameters:
        -C: Extract to the specified directory # uppercase
    
         
# eg:
#Package and compress a file (using gzip compression)
    tar -zcvf rumenz.tar.gz rumenz.txt
    
# tar. GZ extract to the / tmp directory
    tar -xzvf rumenz.tar.gz -C /tmp
      
#List the contents of the compressed package
    tar -ztvf rumenz.tar.bz2 rumenz.txt
    
    #Note:
    Because we use gzip compressed rumenz tar. gz,
    So check out rumenz tar. GZ package files, you have to add Z this option
      
#Compress all ordinary files in the / etc directory into / TMP / etc.tar GZ file
    tar -czvPf /tmp/etc.tar.gz $(find /etc/ -type f | xargs)
    
#Supplement: $(): equivalent to parentheses in mathematics. Give priority to the contents in parentheses before executing tar

4. Zip common

Compressed files or folders

#Install compression and decompression commands
    yum -y install zip unzip
    
    
#Compress
Zip [options] compressed package name source file or source directory list
    -R recursive compression
    -m 	 After compressing the file, deleting the original file # is equivalent to moving the file to the compressed file
    -v 	 Displays detailed compression process information

    eg: zip -r etc.zip /etc
    
    
#Decompress
Unzip [parameter] compressed package name
    -D directory name 	 Unzip the compressed file to the specified directory
    -N when decompressing, existing files will not be overwritten
    -O when decompressing, overwrite the existing files without user confirmation
    
    -V view the details of the compressed file, including the file size, file name and compression ratio in the compressed package, but do not decompress it
    -T check whether the compressed file is correct and complete, but do not unzip it

 
# eg: 
    zip -r tmp.zip /tmp 
    unzip tmp.zip
    
    zip install.log.zip install.log 
    unzip install.log.zip

3 pipeline

It is used to transmit data and hand over the command execution result before the pipeline to the command after the pipeline for processing

#Format:
    |

#Case
# 1. Delete the files created within one day under the / tmp directory
    find /tmp/ -ctime -1 -type f | xargs -I {} rm -rf {}
      
# 2. Copy all ordinary files in etc to the / tmp directory
    find /etc/ -type f | xargs -I {} cp -r {} /tmp/

Recommended Today

Pandas – basic operation of dataframe

We learned in the last sectionSeriesThe basic operations of structure addition, deletion, modification and query are mastered in this sectionDataFrameIt will be very easy to add, delete, modify and check~ First, let’s construct aDataFrame : data = [[1,2,3], [4,5,6], [7,8,9]] index = [‘a’, ‘b’, ‘c’] columns = [‘A’, ‘B’, ‘C’] df = pd.DataFrame(data=data, index=index, columns=columns) […]