Linux knowledge

Time:2021-3-25

Basics

The five units of a computer include:input unit output unit control unit arithmetic logic unit Storage unitTop five
part. The CPU occupies the control and arithmetic logic unit, and the storage unit contains memory and auxiliary memory;

Unit of measurement

Carry system Kilo Mega Giga Tera Peta Exa Zetta
Binary 1024 1024K 1024M 1024G 1024T 1024P 1024E
decimal system 1000 1000K 1000M 1000G 1000T 1000P 1000E

System call

Linux knowledge

  • The core layer of the operating system is written directly referring to the hardware specifications, so the same operating system program cannot run under different hardware architectures. For example, windows 8.1 for personal computers can’t run directly on computers with arm architecture (mobile phones and tablet hardware).
  • The operating system is just managing the whole hardware resources, including CPU, memory, I / O devices and file system files. If there is no other application assistance, the operating system can only make the computer host ready! No other functions can be run. So now you know why you need to install software like photoimpact or Photoshop to run web images on windows?
  • Application development refers to the development interface provided by the operating system, so the application can only run on the operating system, not on other operating systems.
    Now you know why when you buy the online game CD, the CD will clearly write which operating system the software is suitable for, right? Why can’t some games be installed on Linux?
  • Core functions

Since the core is mainly responsible for the allocation and management of resources related to the whole computer system, we know the whole computer system
The most important thing is CPU and memory. Therefore, the core should at least have these functions:

  • System call interfaceAs just mentioned, this is to make it easier for program developers to make further use of hardware resources through communication with the core. Therefore, it is necessary to have this simple interface to facilitate program developers.
  • Process controlHave you ever heard of the so-called “multitasking environment”? A computer may have a lot of work at the same time to run to the CPU waiting for processing, the core at this time must be able to control these work, let the CPU resources for effective allocation! In addition, a good CPU scheduling mechanism (that is, the order in which the CPU runs first) will effectively speed up the overall system performance!
  • Memory managementControl the memory management of the whole system, this memory control is very important, because all the program code and data of the system must be stored in the memory first. Usually, the core will provide the function of virtual memory. When the memory is insufficient, it can provide the function of swap.
  • File system managementFile system management, such as data input and output (I / O) and so on! There is also support for different file formats, etc. if your core does not know a file system, then you will not be able to use the file format! For example, Windows 98 does not know the hard disk with NTFS file format;
  • Device driversAs mentioned above, the management of hardware is one of the main tasks of the core. Of course, the device driver is what the core needs to do! Fortunately, there is a so-called “loadable module” function, you can edit the driver into a module, you do not need to re compile the core!

GNU program and the foundation of FSF

  • Advocated by Richard Stallman in 1984GNU(GNU’s not Unix) program,
    The goal of this project is to create a free and open UNIX operating system.
  • establishFSF (Free Software Foundation)More engineers and volunteers are invited to write the software.
  • In order to prevent the free software developed by GNU from being used by others to become patented software, he and his lawyer drafted the famous general public license
    License, GPL) * *, and call it copyleft.
  • By the Finns in 1991Linus TorvaldsDevelop Linux operating system. In short, the main success of Linux lies in:
    MINIX (Unix), GNU, Internet, POSIX and virtual team.
  • Several important softwares developed by GNU, such as:

    • Emacs
    • GNU C (GCC)
    • GNU C Library (glibc)
    • Bash shell
  • Linux itself is the most popular operating system, and its development website is set up inhttp://www.kernel.orgWe also call the data at the bottom of the Linux operating system“Kernel”。
  • Since Linux kernel 3.0, the odd and even core version planning has been abandoned. The new planning uses mainline version as the basis, and provides long term support version to strengthen the continuous maintenance of some functions.
  • The composition of Linux distributions includes a complete system made of “Linux kernel + free software + documentation (tools) + fully installable program”.
  • Same: (1) same usehttp://www.kernel.org(2) support the same standards, such asFHS(File system Hierarchy Standard)LSB(Linux Standard Base)(3) using almost the same free software (such as GCC / glibc / VI / Apache / bind / sendmail…) in GNU; (4) using almost the same operation interface (such as bash / KDE / gnome, etc.). Different: the kernel used and the version of each software may be different; different developers join different application tools and use different suite management modes (dpkg and RPM)

As a matter of fact, brother bird prefers to buy basic books when buying science and technology books, because he has learned the basic knowledge well, so maybe he can find another one for other parts
Keyword, Google again, a lot of data can let you to analyze and judge! You would say, in that case, Nagi
Is there a lot of projects in basic books, not Google? Don’t forget, “what keywords are you going to use in the beginning
Google! Therefore, the key to reading basic books is to let yourself master those books“keyword”Hello! plus
Oil!

In class, some students often ask, “teacher, how many times do you have to listen to your class before you can learn?” bird
My standard answer is, “you’ll never learn!” Because you use “listen” and don’t do it, you will never know “Jing”
How to write “Yan”! Most of the time, there are some unexplained emergencies in the computer / network, which have never been touched,
How is it possible to understand? So “it’s never going to be heard!” Why experiment? Because you’ll have experience after the experiment
Write it down? Otherwise, the experimental results are available in textbooks! It’s not like reciting. What about the dry hemp experiment? Is it a waste of money? ^_^

Here are some useful FAQs and how to websites for your reference:

The more interesting one above is that oneTLDP(The Linux Documentation Project)He listed almost all of them
We can see the literature data and various how to methods on Linux. Although they are in English, they are of great reference value
Value!

Hardware / disk

Basics

Linux knowledge

This small block is the smallest physical storage unit of disk, which is calledSectorThe circle composed of sectors of the same concentric circle is calledTrack. Since there may be multiple disks in the disk, the same track on all disks can be combined to form the so-calledCylinder. There are two kinds of physical quantity design of sector, which are512BytesAnd4KBytes

Usually, data reading and writing will start from the outer ring to the inner ring

So is every sector equally important? In fact, the first sector of the whole disk is particularly important, because it records the important information of the whole disk! The important information contained in the first sector of the early disk is calledMBR (Master BootRecordHowever, due to the continuous expansion of disk capacity in recent years, there are some problems in reading and writing, and even some disk partitions larger than 2TB have made some operating systems unable to access. Therefore, a new disk partition format, calledGPT (GUID partition table)! These two partition formats and restrictions are not quite the same!

MSDOS (MBR) and GPT partition table

In the diagram in the previous section, do we see “start and end tracks”? Usually, a disk may have multiple disks, and the same track of all disks is called a magnetic trackCylinder (cylinder)Usually that isThe smallest unit of a file systemThat is to sayMinimum unit of partitionla Why “usually”? Recently, GPT, a partition table with 64 bit recording function, is availableSectorNumber as a partition unit! Great! So, we just use the reference cylinder or sector number to deal with it!

Format and limitation of MSDOS (MBR) partition table

In order to be compatible with Windows disk, the early Linux system used MBR (master boot record) to deal with boot manager and partition table! The boot manager record area and partition table are all placed in the first sector of the disk, which is usually 512bytes in size Therefore, the first sector 512bytes will have these two data:

  • mainMaster boot record (MBR): can install boot management program, there are446 Bytes
  • Partition table: record the status of the whole hard disk partition, yes64 Bytes

Since the partition table is located in a block with only 64 bytes capacity, it has the largest capacityThere are only four groups of recording areasEach group of recording area records the cylinder number of the beginning and end of the section. If you look at the hard disk as a long bar, and then the cylinder as a straight bar, then the 64bytes record section is a bit like the following figure:

Linux knowledge

  • P1:/dev/sda1
  • P2:/dev/sda2
  • P3:/dev/sda3
  • P4:/dev/sda4

Since the partition table only has 64 bytes, it can only hold four partition records at most, and the records of these four partitions are calledPrimaryorExtendedPartition. According to the above illustration and explanation, we can get several key information:

  • The maximum number of primary and extended partitions is four (limited by hard disk)
  • There can be at most one extended partition (limited by the operating system)
  • Logical partition is a partition continuously cut by extended partition;
  • After being formatted, the partition for data access is the main partition and the logical partition. Extended partition cannot be formatted;
  • The number of logical partitions varies according to the operating system. In Linux system, SATA hard disk can break through the limit of more than 63 partitions;

Linux knowledge

  • P1:/dev/sda1
  • P2:/dev/sda2
  • L1:/dev/sda5
  • L2:/dev/sda6
  • L3:/dev/sda7
  • L4:/dev/sda8
  • L5:/dev/sda9

The first four numbers are reserved for primary or extended! So the device name number of logical partition starts from the 5th
In fact, the extended partition does not only occupy one block, but will be distributed in the first few sectors of each partition to record the partition information!

GUID partition table, GPT disk partition table

Because in the past, the size of a sector was only 512bytes, but now there are 4K sector designs! In order to be compatible with all disks, the so-called logical block address (LBA) is usually used in the definition of sectors. GPT uses this LBA for all disk blocks (512bytes by default) And the first LBA is called lba0 (numbered from 0).

Unlike MBR, which only uses the first 512bytes block to record partition information, GPT uses 34 LBA blocks to record partition information! At the same time, unlike in the past, when only one MBR block was killed, the last 33 LBAs of the whole disk were used as another backup in addition to the first 34 LBAs of GPT! This may be safer!

The detailed structure looks like the following:

Linux knowledge

  • Lba0 (MBR compatible block)

Similar to the MBR mode, this compatible block is also divided into two parts. One is the block similar to the previous 446 bytes, which stores the boot manager of the first stage! In the record area of the original partition table, this compatible mode only puts a partition with a special flag to indicate that the disk is in GPT format. If you don’t understand the GPT partition table, you won’t know this disk. Unless the user has a special request to process this disk, the management software can’t modify the partition information to further protect this disk!

  • Lba1 (GPT header record)

This part records the location and size of the partition table itself, the location of the GPT partition for backup (that is, the last 34 LBA blocks mentioned above), and the check mechanism code (CRC32) of the partition table. The operating system can judge whether the GPT is correct according to the check code. If there is an error, you can also get the backup GPT (the last backup block of the disk) through this record area to restore the normal operation of GPT!

  • Lba2-33 (division of actual records)

Starting from lba2 block, each LBA can record 4 segmentation records, so in the default case, there can be a total of 4 * 32 = 128 segmentation records! Because each LBA has 512bytes, each record uses 128 bytes of space. In addition to the identification code and related records required by each record, GPT provides 64bits in each record to record the start / end magnetic area number. Therefore, for a single partition slot, the maximum capacity limit of GPT partition table is “2 ^ 64 * 512bytes = 2 ^ 63″* 1kbytes = 2 ^ 33 * TB = 8 ZB “, pay attention to 1zb = 2 ^ 30tb! Do you think it’s big enough?

At present, GPT partition can provide up to 128 records by default. In the core device records of Linux, although in the past, only 15 partition slots could be reached for a single disk, due to the Linux kernel’s processing through udev and other methods, Linux now has no such limitation! In addition, there are no so-called concepts of main, extension and logical segmentation in GPT segmentation. Since each record can exist independently, of course, each record can be regarded as a main segmentation! Every partition can be used to format!

GPT partition has no concept of extension and logical partition. You can imagine that all partitions are primary partitions!

  • Each partition has its own boot sector
  • The system disk in the figure is the first and second partition,
  • The actual bootable core file is placed in each partition!
  • The loader only knows the bootable core files in its own system disk and other loaders;

Linux knowledge

  1. The boot process consists of: BIOS — > MBR — > > boot loader –;
  2. The main functions of boot loader are: providing menu, loading core and transferring control to other loaders
  3. Boot loader can be installed in MBR and boot sector
  4. Linux operating system files use the directory tree system, and the corresponding disk needs to have “Mount” action;
  5. Novice simple partition, it is recommended as long as there are / and swap two partitions

Partition format

In fact, there are three kinds of “equipment types”. Our practice machine actually uses standard partition and LVM. The meanings of the three types of equipment are as follows:

  • Standard zoningThis is what we have been talking about! Partition like / dev / vda1 is enough.
  • LVM: This is a device setting that can flexibly increase / decrease the file system capacity. We will continue to introduce LVM in the following chapters!
  • LVM in short supplyThe translation of this term is very strange! In fact, this is an advanced version of LVM! Different from the traditional LVM direct allocation of fixed capacity, this “LVM tight supply” project allows you to allocate disk capacity to you according to the capacity you use. Therefore, if there is less data in the LVM device, your disk can actually store more data! And will not be without reason to occupy! This part will be emphasized when we talk about LVM later!

In addition, the file system in the figure refers to the file system that we can format when “formatting”. Let’s talk about each file system project (the detailed project will be explained in the following chapters)

  • ext2/ext3/ext4: the type of file system used in the early days of Linux. Because ext3 / ext4 file system has more log records, the recovery of the system is faster. However, due to the increasing disk capacity, ext4 family seems to be a bit unstoppable, so unless you have special setting requirements, ext4 projects are rarely used recently!
  • swap: disk emulation becomes memory. Swap doesn’t use the mount of directory tree, so you don’t need to specify the mount point with swap.
  • BIOS Boot: This is the project that GPT partition table may use. If you use MBR partition, you don’t need this project!
  • xfs: This is the default file system of CentOS, which was developed by large server at first! He is very good at large capacity disk management, and the speed of formatting is quite fast, which is very suitable for today’s environment with several terabytes of disk! So we mainly use this stuff!
  • vfat: a file system type supported by both Linux and windows. If you have windows and Linux operating systems in your host hard disk, you can create a VFAT file system for data exchange!

Linux knowledge

Partition GPT with parted in Linux

The file name of each hardware device in Linux

After choosing the hardware you need, you need to understand the role of each hardware in Linux. Here, brother bird once again emphasizes: “in Linux system, every device is treated as a file.” for example, the file name of the hard disk of the SATA interface is / dev / SD [A-D], where the letter in brackets is any one of A-D, that is, / dev / SDA, / dev / SDB, / dev / SDC, and / dev / SDD.

Tips, a bracket expression, will be used frequently in later chapters. Please pay special attention to it. In addition, it is emphasized that in Linux system, almost all hardware device files are in the / dev directory, so you will see the file names of / dev / SDA, / dev / sr0, etc.

What about printers and floppy disks? They are / dev / lp0, / dev / fd0 respectively! OK, what about the other peripherals? Here are some common devices and their file names in Linux:

equipment The file name of the device in Linux
SCSI / SATA / USB hard disk drive /dev/sd[a-p]
USB flash drive /Dev / SD [A-P] (same as SATA)
Virti / O interface /Dev / VD [A-P] (used in virtual machine)
Floppy disk machine /dev/fd[0-7]
printer /Dev / LP [0-2] (25 pin printer) / dev / USB / LP [0-15] (USB interface)
mouse /Dev / input / mouse [0-15] (general) / dev / psaux (PS / 2 Interface) / dev / mouse (current mouse)
CDROM/DVDROM /Dev / SCD [0-1] (general) / dev / Sr [0-1] (general, CentOS is more common) / dev / CDROM (current CDROM)
Tape drive /Dev / ht0 (IDE interface) / dev / st0 (SATA / SCSI interface) / dev / tape (current tape)
IDE hard disk drive /Dev / HD [A-D]

Today, due to the IDE interface disk drive has been almost eliminated, too rare! So now even the disk file name of IDE interface is simulated as / dev / SD [A-P]! In addition, if you use a cloud machine that you have applied with an ISP, you may get a virtual machine. In order to speed up, the disk in the virtual machine is generated by the emulator. The disk file name generated by the emulator is the file name of / dev / VD [A-P] series! Pay attention! Pay attention!

Recommended Today

SQL exercise 20 – Modeling & Reporting

This blog is used to review and sort out the common topic modeling architecture, analysis oriented architecture and integration topic reports in data warehouse. I have uploaded these reports to GitHub. If you are interested, you can have a lookAddress:https://github.com/nino-laiqiu/TiTanI recorded a relatively complete development process in my hexo blog deployed on GitHub. You can […]