NFS notes (I) working principle and detailed configuration of NFS server

Time:2022-6-10

1、 How NFS works

1. What is an NFS server

NFS is the abbreviation of network file system. Its biggest function is to enable different machines and different operating systems to share files with each other through the network.

Server for NFS allows PC to mount the directory shared by server for NFS in the network to the file system at the local end. From the perspective of the local end system, the directory of the remote host is like its own disk partition, which is very convenient to use;

2. NFS mount principle

Mounting structure diagram of NFS server:

NFS notes (I) working principle and detailed configuration of NFS server

As shown above:

After we set up a shared directory /home/public in server for NFS, other NFS clients that have access to server for NFS can mount this directory to a mount point of their file system, which can be defined by themselves. As shown in the above figure, the directories mounted by client a and client B are different. After mounting, we can see all the data of the server /home/public locally. If the client configured on the server side is read-only, the client can only be read-only. If read / write is configured, the client can read / write. After mounting, NFS client views disk information command: \df – H.

Since NFS transmits data between the server and the client through the network, the corresponding network port is required for data transmission between the two. Which port does the NFS server use for data transmission? Basically, the port of NFS server is set at 2049, but the file system is very complex. Therefore, NFS has other programs to start additional ports. These additional ports used to transmit data are randomly selected, which are ports smaller than 1024; Since it is random, how does the client know which port is used by the NFS server? At this time, it needs to be implemented through the remote procedure call (RPC) protocol!

3. How RPC and NFS communicate

NFS supports quite a number of functions, and different functions will be started with different programs. Each time a function is started, some ports will be enabled to transmit data. Therefore, the ports corresponding to NFS functions are not fixed. Clients need to know the relevant ports on the NFS server to establish connections for data transmission. RPC is a service used to uniformly manage NFS ports, and the unified external port is 111, RPC records NFS port information, so that we can communicate port information between the server and the client through RPC. The main function of PRC is to specify the port number corresponding to each NFS function, and notify the client that the client can connect to the normal port.

How does RPC know the port of each NFS function?

First, after NFS is started, some ports will be randomly used. Then NFS will register these ports with RPC, and RPC will record these ports. RPC will open port 111 and wait for the client’s RPC request. If the client has a request, the server’s RPC will inform the client of the previously recorded NFS port information. In this way, the client will get the port information of the NFS server and transfer the data with the actual port.

Prompt: before starting NFS server, first start the RPC service (i.e. portmap service, the same below). Otherwise, NFS server cannot register with the RPC service area. In addition, if the RPC service is restarted, all the previously registered NFS port data will be lost. Therefore, the NFS program managed by RPC service should also be restarted to re register with RPC at this time. Special note: generally, after modifying the NFS configuration document, NFS does not need to be restarted. Execute /etc/init D/nfs reload or exportfs – RV to make the modified /etc/exports effective.

4. Communication process between NFS client and NFS server

NFS notes (I) working principle and detailed configuration of NFS server

1) First, start the RPC service on the server and open port 111

2) Start NFS service on the server side and register port information with RPC

3) The client starts the RPC (portmap service) and requests the server’s NFS port from the server’s RPC (portmap) service

4) The RPC (portmap) service on the server feeds back NFS port information to the client.

5) The client establishes an NFS connection with the server through the obtained NFS port and transfers data.

NFS notes (I) working principle and detailed configuration of NFS server

2、 NFS deployment

1. View system information

[[email protected] ~]#cat/etc/redhat-release

CentOS release 7.3.1611 (AltArch)

[email protected] ~]#uname-a

Linux server7.ctos.zu 3.10.0-514.el7.centos.plus.i686 #1SMP Wed Jan2512:55:04UTC2017i686 i686 i386 GNU/Linux

It is a habit to check the system version and kernel parameters first. The kernel of the same software is different in different versions, so the deployment method is also different. Do not cause unnecessary errors because of this.

2. NFS software installation

To deploy NFS services, you must install the following two software packages: NFS utils: NFS master program and rpcbind: PRC master program;

Both NFS server side and client side need to install these two software.

Note: the RPC server for NFS is named portmap under centos5 and rcpbind under centos6 and centos7

NFS package

NFS utils: NFS main program, including rpc nfsd  rpc. Mount two deamons

Rpcbind:rpc main program

2.1. View NFS software package

       [[email protected] ~]# rpm -qa | egrep “nfs|rpcbind”

2.2. Installing NFS and RPC services

       [[email protected] ~]# yum install nfs-utils  rpcbind

       [[email protected] ~]# rpm -qa  | egrep “nfs|rpcbind”

  rpcbind-0.2.0-38.el7_3.1.i686

  nfs-utils-1.3.0-0.33.el7_3.i686

  libnfsidmap-0.25-15.el7.i686

Check what files are installed in the computer for the two software packages;

[[email protected] ~]# rpm -ql nfs-utils

3. Start services for NFS

3.1. Start rpcbind service before starting NFS service

View rcpbind status

[[email protected] ~]# systemctl status rpcbind

● rpcbind.service – RPC bind service

   Loaded: loaded (/usr/lib/systemd/system/rpcbind.service; indirect; vendor preset: enabled)

Active: active (running) since – 2017-09-04 10:03:20 CST; 1s ago

  Process: 3583 ExecStart=/sbin/rpcbind -w $RPCBIND_ARGS (code=exited, status=0/SUCCESS)

 Main PID: 3584 (rpcbind)

   CGroup: /system.slice/rpcbind.service

           └─3584 /sbin/rpcbind -w

Note: after rpcbind is successfully installed, it is enabled by default, and it is started automatically after startup. If not, let’s restart the rcpbind service

[[email protected] ~]# systemctl restart  rpcbind

View PRC port

[[email protected] ~]# yum install net-tools lsof

[[email protected] ~]# lsof  -i:111

COMMAND  PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME

systemd    1 root   56u  IPv6  43164      0t0  TCP *:sunrpc (LISTEN)

systemd    1 root   57u  IPv4  43165      0t0  TCP *:sunrpc (LISTEN)

rpcbind 3584  rpc    4u  IPv6  43164      0t0  TCP *:sunrpc (LISTEN)

rpcbind 3584  rpc    5u  IPv4  43165      0t0  TCP *:sunrpc (LISTEN)

rpcbind 3584  rpc    8u  IPv4  44975      0t0  UDP *:sunrpc

rpcbind 3584  rpc   10u  IPv6  44977      0t0  UDP *:sunrpc

[[email protected] ~]# netstat -tlunp |grep rpcbind

udp        0      0 0.0.0.0:111             0.0.0.0:*                         3584/rpcbind

udp        0      0 0.0.0.0:791             0.0.0.0:*                           3584/rpcbind

udp6       0      0 :::111                  :::*                                3584/rpcbind

udp6       0      0 :::791                  :::*                                3584/rpcbind

View the port information registered with PRC by service for NFS before NFS is started

[[email protected] ~]# rpcinfo -p localhost

   program vers proto   port  service

    100000    4   tcp    111  portmapper

    100000    3   tcp    111  portmapper

    100000    2   tcp    111  portmapper

    100000    4   udp    111  portmapper

    100000    3   udp    111  portmapper

   100000    2   udp    111  portmapper

3.2 start NFS service after RPC service is started

View status

[[email protected] ~]# systemctl status  nfs

 nfs-server.service – NFS server and services

   Loaded: loaded (/usr/lib/systemd/system/nfs-server.service; disabled; vendor preset: disabled)

   Active: inactive (dead)

It is not started by default. The system does not start after the system is started and restarted. Start the NFS service and set it to start.

[[email protected] ~]# systemctl start nfs

[[email protected] ~]# systemctl enable nfs

Created symlink from /etc/systemd/system/multi-user.target.wants/nfs-server.service to /usr/lib/systemd/system/nfs-server.service.

[[email protected] ~]# systemctl status  nfs

● nfs-server.service – NFS server and services

   Loaded: loaded (/usr/lib/systemd/system/nfs-server.service; enabled; vendor preset: disabled)

Active: active (exited) since – 2017-09-04 10:15:21 CST; 19s ago

 Main PID: 3654 (code=exited, status=0/SUCCESS)

   CGroup: /system.slice/nfs-server.service

After starting NFS, we check the port information registered by RPC again

[[email protected] ~]# rpcinfo -p localhost

   program vers proto   port  service

    100000    4   tcp    111  portmapper

    100000    3   tcp    111  portmapper

    100000    2   tcp    111  portmapper

    100000    4   udp    111  portmapper

    100000    3   udp    111  portmapper

    100000    2   udp    111  portmapper

    100024    1   udp  56626  status

    100024    1   tcp  42691  status

    100005    1   udp  20048  mountd

    100005    1   tcp  20048  mountd

    100005    2   udp  20048  mountd

    100005    2   tcp  20048  mountd

   100005    3   udp  20048  mountd

    100005    3   tcp  20048  mountd

    100003    3   tcp   2049  nfs

    100003    4   tcp   2049  nfs

    100227    3   tcp   2049  nfs_acl

    100003    3   udp   2049  nfs

    100003    4   udp   2049  nfs

    100227    3   udp   2049  nfs_acl

    100021    1   udp  57225  nlockmgr

    100021    3   udp  57225  nlockmgr

    100021    4   udp  57225  nlockmgr

   100021    1   tcp  35665  nlockmgr

    100021    3   tcp  35665  nlockmgr

  100021    4   tcp  35665  nlockmgr

After confirming that the startup is useless, let’s take a look at which ports NFS has opened

[[email protected] ~]# netstat -tulnp |grep -E ‘(rpc|nfs)’

tcp        0      0 0.0.0.0:42691           0.0.0.0:*               LISTEN      3634/rpc.statd

tcp        0      0 0.0.0.0:20048           0.0.0.0:*               LISTEN      3642/rpc.mountd

tcp6       0      0 :::39614                :::*                    LISTEN      3634/rpc.statd

tcp6       0      0 :::20048                :::*                    LISTEN      3642/rpc.mountd

udp        0      0 127.0.0.1:842           0.0.0.0:*                           3634/rpc.statd

udp        0      0 0.0.0.0:20048           0.0.0.0:*                           3642/rpc.mountd

udp        0      0 0.0.0.0:111             0.0.0.0:*                           3584/rpcbind

udp        0      0 0.0.0.0:791             0.0.0.0:*                           3584/rpcbind

udp        0      0 0.0.0.0:56626           0.0.0.0:*                           3634/rpc.statd

udp6       0      0 :::56122                :::*                                3634/rpc.statd

udp6       0      0 :::20048                :::*                                3642/rpc.mountd

udp6       0      0 :::111                  :::*                                3584/rpcbind

udp6       0      0 :::791                  :::*                                3584/rpcbind

4. Detailed explanation of common NFS processes

[[email protected] ~]# ps -ef |egrep “rpc|nfs“

rpc       3584     1  0 10:03 ?        00:00:00 /sbin/rpcbind -w

rpcuser   3634     1  0 10:15 ?        00:00:00 /usr/sbin/rpc.statd –no-notify

root      3637     2  0 10:15 ?        00:00:00 [rpciod]

root      3642     1  0 10:15 ?        00:00:00 /usr/sbin/rpc.mountd

root      3652     1  0 10:15 ?        00:00:00 /usr/sbin/rpc.idmapd

root      3657     2  0 10:15 ?        00:00:00 [nfsd4_callbacks]

root      3663     2  0 10:15 ?        00:00:00 [nfsd]

root      3664     2  0 10:15 ?        00:00:00 [nfsd]

root      3665     2  0 10:15 ?        00:00:00 [nfsd]

root      3666     2  0 10:15 ?        00:00:00 [nfsd]

root      3667     2  0 10:15 ?        00:00:00 [nfsd]

root      3668     2  0 10:15 ?        00:00:00 [nfsd]

root      3669     2  0 10:15 ?        00:00:00 [nfsd]

root      3670     2  0 10:15 ?        00:00:00 [nfsd]

root      3705  3267  0 10:23 pts/0    00:00:00 grep -E –color=auto rpc|nfs

nfsd

The main NFS service provider. The main function of this daemon is to manage whether the client can use the server file system mount information, including the ID of the login user.

  rpc.mountd

The main function of this daemon is to manage NFS file systems. When the client side successfully passes rpc After NFSD logs in to the host, it will also go through the authentication program for file use permission before it can use the NFS server to provide the specified file. It will read the NFS configuration file /etc/exports to compare the permissions of the client. After this pass, the client will obtain the permission to use NFS files.

rpc. Lockd (not necessary)

This daemon is used to manage the locking of files. When multiple clients try to write a file at the same time, it can cause some problems to the file. rpc. Lockd can be used to overcome this problem. But rpc Lockd must be enabled on both the client and server.

rpc. Statd (not necessary)

This daemon can be used to check the consistency of files. If a file is damaged because the client uses the same file at the same time, rpc Statd can be used to detect and attempt to recover the file

5. Configure services for NFS

NFS software is very simple. The main configuration file: /etc/exports is empty by default. If there is no such file, you can use VIM to actively create it. As for the setup of NFS server, it is also very simple. As long as you edit the main configuration file /etc/exports, start rpcbind (if it has been started, do not restart), and then start NFS, NFS will succeed.

How should /etc/exports be set?

[[email protected] etc]# vi /etc/exports

/tmp/data      192.168.1.0/24(ro)          client-A.ctos.zu(rw,sync)

#[shared directory] [client address 1 (permission)] [client address 2 (permission)]

The above is a simple case configuration. The top of each line is the directory to be shared. Note that the unit is the directory

Shared directory: a directory on our local computer that we want to share with other hosts on the network. If I want to share the /tmp/data directory, this option can directly write to the /tmp/data directory, which can be shared to different hosts according to different permissions.

Client address 1 (parameter 1, parameter 2): a network or a single host can be set for the client address. Parameters: for example, read / write permission RW, synchronous update sync, and compressed visiting account all_ Square, the compressed anonymous account anonuid=uid, anongid=gid, etc;

The client address can be set in the following ways:

1) . you can use a complete IP or a network number, such as 192.168.100.100 or 192.168.8.0/24

2) . you can use the host name, but the host name must be in /etc/hosts, or you can use DNS to find the name. Anyway, the key is to find the IP. If it is a host name, you can also support wildcards, such as’ * ‘or’? ‘ Acceptable; For example: host[1-8] ctos. zu,server?. test. com

NFS permission settings

NFS configuration permission settings, that is, the parameter set in brackets () in the configuration format of /etc/exports file;

Parameter commandParameter usage

RW stands for read / write

Roread only indicates read only permission

When sync requests or writes data, it will not return until the data is synchronously written to the hard disk of NFS server

no_ root_ If the user of squas accessing the NFS server shared directory is root, it has root permission on the directory. This configuration was originally prepared for diskless users. Users should avoid using!

root_ Square for users accessing NFS server shared directories, if they are root, they will be compressed into the identity of nobody.

all_ No matter how the identity of the user accessing the NFS server shared directory includes root, its permissions will be compressed into anonymous users, and their UDI and GID will become uid and GID of nobody or nfsnobody accounts. When multiple NFS clients read and write NFS server data at the same time, this parameter is very useful to ensure that everyone has the same permission to write data.

However, different systems may have different UIDs and GIDS for anonymous users. Because here we need the same users between the server and the client. For example, if the server specifies that the uid of the anonymous user is 2000, then the client must have an account of 2000

Anonuidanonuid is the anonymous uid and GID. Describes the permissions that the client uses to access the server. By default, it is nfsnobody. Uid65534.

Anongid is the same as anongid, which is to replace uid with gid

Configuration instance:

/home/test  1192.168.1.0/24(rw,sync,all_squash,anonuid=2000,anongid=2000)

###Note that the red part cannot have spaces!! A configuration commonly used in production environment, which is suitable for multiple clients to share an NFS directory. All_ Square means that no matter what identity the client accesses, it will be compressed into all_ The user and group identities followed by square. Anonuid and anongid numbers are used here=

Summary:

Server share configuration format:

1) Basic format: shared directory ip/24 (shared attribute) – > note that there are no spaces

2) Share permission settings:

RW read / write properties

The sync file is not returned until it is actually written to the disk

all_ Square: all access users are compressed into subsequent users.

Anonuid: default compressed user

Anongid: default compressed user group

What is the identity of the client?

By default, the client access server uses the user nfsnobody to access. Uid and GID are 65534. When the server is shared by default, all is added_ Squash this parameter. Anonuid is 65534 (that is, nfsnobayd users). Of course, if the nfsnobody in the system is another uid, it may cause access permission problems. Therefore, it is better that we can set up a user to access, and unify uid and GID.

How is the mount?

There are two important documents that can solve this problem/ Var/lib/nfs/etab and /var/lib/nfs/rmtab can be used to view what directories are shared on the server, how many clients are shared, and the specific information of client attachment.

1. The etab file can see which directories are shared on the server, who can use them, and what parameters are set.

2. The rmtab file is used to view the mounting of the shared directory.