Installation and configuration of Rsync server in centos6

Time:2021-7-30

1、 Introduction to Rsync

The Rsync / remote tool can be used to quickly synchronize data between different hard disks in the LAN, or multiple remote hard disks.

Rsync is a tool used to replace RCP. Rsync uses the so-called “Rsync algorithm” to synchronize the files between the local and remote hosts. This algorithm only transmits different parts of the two files, not the whole file every time, so it is very fast. You can refer to how Rsync works a practical overview to further understand the operation mechanism of Rsync.

Rsync supports most UNIX like systems and has been well tested on Linux, Solaris and BSD. In addition, it also has corresponding versions under the windows platform, and the well-known ones are cwrsync and sync2nas.

The original authors of Rsync are Andrew Tridgell and Paul Mackerras, which are currently http://rsync.samba.org maintain.

The basic features of Rsync are as follows:

1. The entire directory tree and file system can be saved in an image;

2. It is easy to maintain the permissions, time, soft and hard links of the original documents;

3. It can be installed without special permission;

4. Optimized process and high file transmission efficiency;

5. You can use RCP, SSH and other methods to transfer files. Of course, you can also connect directly through socket;

6. Support anonymous transmission to facilitate website mirroring.

When Rsync is used for remote synchronization, two methods can be used: remote shell (SSH is recommended, and SSH is responsible for user authentication) and C / S (that is, the client connects to the remote Rsync server, and Rsync server is responsible for user authentication).

Regardless of the local synchronization directory or remote synchronization data, all files will be copied once in the first run, and only the changed files (for the new files) or the changed parts of the files (for the original files) will be copied in the future run.

Rsync has no speed advantage during the first replication and is not as fast as tar. Therefore, when the amount of data is large, you can consider using tar for the first replication and then Rsync for data synchronization.

2、 System environment

System platform:CentOS release 6.3 (Final)

Rsync version:rsync-3.0.9-2.el6.rfx.x86_64.rpm

Rsync server:TS-DEV (172.16.1.135)

Rsync client:TS-CLIENT (172.16.1.136)

3、 Server side installation of Rsync service

3.1. Check whether Rsync is installed

# rpm -qa|grep rsync

If installed, uninstall using the RPM – e command.

3.2. Download RPM package

# wget http://pkgs.repoforge.org/rsync/rsync-3.0.9-2.el6.rfx.x86_64.rpm

3.3. Installing Rsync

# rpm -ivh rsync-3.0.9-2.el6.rfx.x86_64.rpm

4、 Configure Rsync service

4.1. Steps to configure Rsync server

  • First, select the server startup method
  • The Rsync server with heavy load should run independently
  • The xinetd run mode can be used for Rsync servers with light load
  • Create the configuration file rsyncd.conf
  • For Rsync servers with non anonymous access, an authentication password file should also be created
  • 4.2. Running Rsync service with xinetd

    CentOS runs the Rsync service in xinetd mode by default. Xinetd configuration file for Rsync
    In / etc / xinetd.d/rsync. To configure the Rsync service running with xinetd, execute the following command:

    # chkconfig rsync on# service xinetd restart

    Administrators can modify the / etc / xinetd.d/rsync configuration file to suit your needs. For example, you can modify the configuration line

    server_args = –daemon

    Add the service option of Rsync later.

    4.3. Run Rsync service independently

    The simplest way to run Rsync service independently is to execute the following command:

    # /usr/bin/rsync –daemon

    You can write the above command to the / etc / rc.local file to run the Rsync service every time you start the server. Of course, you can also write a script to automatically start the rysnc service at startup.

    4.4. Configuration file rsyncd.conf

    Rsyncd.conf needs to be configured for both Rsync service running modes, and its format is similar to the main configuration file of samba.
    The configuration file rsyncd.conf is in the / etc directory by default. To put all files related to Rsync service in a separate directory, execute the following command:

    # mkdir /etc/rsyncd# touch /etc/rsyncd/rsyncd.conf# ln -s /etc/rsyncd/rsyncd.conf /etc/rsyncd.conf

    The configuration file rsyncd.conf consists of global configuration and several module configurations. The syntax of the configuration file is:

  • The module starts with [module name]
  • The format of parameter configuration line is name = value, where value can have two data types:
  • String (strings can be delimited without quotes)
  • Boolean (1 / 0 or yes / no or true / false)
  • In # or; Start behavior comment
  • \Is a continuation character

    Global parameters

    All configuration lines except [module] in the file are global parameters. Of course, you can also define module parameters in the global parameters section. At this time, the value of this parameter is the default value of all modules.

    parameter explain Default value
    address When running independently, the IP address used to specify the server to run. This parameter is ignored by xinetd runtime and replaced with the – address option on the command line. Local all IP
    port Specifies the port number on which the Rsync daemon listens. This parameter is ignored by xinetd runtime and replaced with the – port option on the command line. 873
    motd file Specifies a message file whose contents are displayed to the client when the client connects to the server. nothing
    pid file The daemon of Rsync writes its PID to the specified file. nothing
    log file Specifies the log file for the Rsync daemon without sending the log to syslog. nothing
    syslog facility Specifies the message level when Rsync sends log messages to syslog. daemon
    socket options Specify custom TCP options. nothing

    Module parameters

    Module parameters are mainly used to define which directory of Rsync server should be synchronized. The format of module declaration must be in the form of [module]. This name is the name seen in Rsync client, similar to the share name provided by Samba server. The data actually synchronized by the server is specified through path. You can specify multiple modules according to your needs. The following parameters can be defined in the module:

    a. Basic module parameters

    parameter explain Default value
    path The specified Rsync parameter must be on the current Rsync server. nothing
    comment Assign a description to the module, which is displayed to the customer together with the module name when the customer gets the module list through connection. nothing

    b. Module control parameters

    parameter explain Default value
    use chroot If true, Rsync will first chroot to the directory specified by the path parameter before transferring the file. The reason for this is to achieve additional security protection, but the disadvantage is that root permission is required, and the directory file pointed to by the symbolic connection outside the path cannot be backed up. true
    uid Specifies that the module transfers files with the specified uid. nobody
    gid Specifies that the module transfers files with the specified GID. nobody
    max connections Specify the maximum number of concurrent connections of the module to protect the server. Connection requests exceeding the limit will be told and then try again. 0 (no limit)
    lock file Specifies a lock file that supports the max connections parameter. /var/run/rsyncd.lock
    list Specifies whether the module should be listed when the customer requests a list of available modules. If this option is set to false, you can create hidden modules. true
    read only Specify whether to allow customers to upload files. If true, upload is not allowed; If it is false and the server directory also has read-write permission, upload is allowed. true
    write only Specify whether to allow customers to download files. If true, downloading is not allowed; If it is false and the server directory also has read permission, it is allowed to download. false
    ignore errors Specifies whether I / O errors are ignored when a delete operation is run on the Rsync server. In general, Rsync will skip the – delete operation in case of I / O errors to prevent serious problems caused by temporary resource shortage or other I / O errors. true
    ignore nonreadable Specifies that the rysnc server completely ignores files that users do not have access to. This makes sense when there are some files in the directory to be backed up that should not be obtained by the backup person. false
    timeout This option can override the IP timeout specified by the customer. This ensures that the Rsync server will not wait forever for a crashed client. For anonymous Rsync servers, the ideal number is 600 (in seconds). 0 (unlimited)
    dont compress Used to specify files that are not compressed before transfer. This option can define a list of command options that customers are not allowed to use for this module. You must use the full name of the option, not the short name. When an option is rejected, the server reports an error message and exits. For example, to prevent the use of compression, it should be: “Don compress = *”. *.gz *.tgz *.zip *.z *.rpm *.deb *.iso *.bz2 *.tbz

    c. Module file filter parameters

    parameter explain Default value
    exclude Specify multiple files or directories (relative paths) separated by spaces and add them to the exclude list. This is equivalent to using – exclude in the client command to specify the schema. empty
    exclude from Specify a file name containing the exclude rule definition from which the server reads the exclude list definition. empty
    include Specify multiple files or directories (relative paths) separated by spaces and add them to the include list. This is equivalent to using – include in the client command to specify the schema. empty
    include from Specify a file name containing the include rule definition from which the server reads the include list definition. empty

    A module can only specify one exclude parameter and one include parameter. You can define complex exclude / include rules by combining include and exclude. These parameters are equivalent to the corresponding Rsync client command options. The only difference is that they act on the server side. For details on how to write rule files, please refer to http://www.howtocn.org/rsync:use_ rsync。

    d. Module user authentication parameters

    parameter explain Default value
    auth users Specify a list of user names separated by spaces or commas, which are the only users allowed to connect to the module. The users here have nothing to do with the system users. The user name and password are stored in clear text in the file specified by the secrets file parameter. (anonymous)
    secrets file Specify an Rsync authentication password file. This file works only if auth users is defined. empty
    strict modes Specify whether to monitor the permissions of the password file. If true, the password file can only be accessed by the user running the Rsync server, and no other user can access the file. true

    The permission of Rsync authentication password file must be 600, otherwise the client will not be able to connect to the server. Each line in the Rsync authentication password file specifies a user name: password pair in the format:

        username:passwd

    Generally speaking, the password should not exceed 8 characters. If you only configure an anonymous Rsync server, you do not need to set the above parameters.

    e. Module access control parameters

    parameter explain Default value
    hosts allow Use a host list to specify which host clients are allowed to connect to the module. Hosts that do not match the host list will be rejected. *
    hosts deny Use a host list to specify which host clients are not allowed to connect to the module. empty

    The client host list definition can be in the following form:

    A single IP address. For example: 192.168.0.1 the entire network segment. For example: 192.168.0.0/24192.168.0.0/255.255.255.0 resolvable single hostname. For example: CentOS, all hosts in the centos.bsmart.cn domain. For example, *. Bsmart. CN “*” means all. Multiple list items should be separated by spaces.

    f. Module log parameters

    parameter explain Default value
    transfer logging Causes the Rsync server to log the transfer operation to the transfer log file. false
    log format Specifies the field for the transfer log file. ”%o %h [%a] %m (%u) %f %l”

    “% t [% P]” will be added at the beginning of each log line when the “log file” parameter is set.

    The log format definer that can be used is as follows:
  • %A – Remote IP address
  • %H – remote host name
  • %L – number of characters in file length
  • %P – PID of this Rsync session
  • %O – operation type: “send” or “recv”
  • %F – file name
  • %P – module path
  • %M – module name
  • %T – current time
  • %U – authenticated user name (null if anonymous)
  • %B – number of bytes actually transmitted
  • %C – when sending a file, record the check code of the file

    5、 Application case of Rsync server

    5.1. Configure Rsync service on server ts-dev

    a. Edit profile

    # vi /etc/rsyncd/rsyncd.conf

    # Minimal configuration file for rsync daemon# See rsync(1) and rsyncd.conf(5) man pages for help# This line is required by the /etc/init.d/rsyncd script
    # GLOBAL OPTIONSuid = root gid = root
    use chroot = no
    read only = yes #limit access to private LANshosts allow=172.16.0.0/255.255.0.0 192.168.1.0/255.255.255.0 10.0.1.0/255.255.255.0hosts deny=* max connections = 5

    pid file = /var/run/rsyncd.pid

    secrets file = /etc/rsyncd/rsyncd.secrets #lock file = /var/run/rsync.lock
    motd file = /etc/rsyncd/rsyncd.motd #This will give you a separate log filelog file = /var/log/rsync.log #This will log every file transferred – up to 85,000+ per user, per synctransfer logging = yes log format = %t %a %m %f %bsyslog facility = local3timeout = 300
    # MODULE OPTIONS[davidhome] path = /home/david/ list=yes ignore errors auth users = david comment = David home exclude = important/ [chinatmp]path = /tmp/china/list=noignore errorsauth users = chinacomment = tmp_china

    b. Create the / etc / rsyncd / rsyncd.secrets file

    # vim /etc/rsyncd/rsyncd.secrets

    David: asdf # format user name: password
    China: JK # this user is not required to be a system user

    c. For password security, we set the permission to 600

    # chown root:root /etc/rsyncd/rsyncd.secrets

    # chmod 600 /etc/rsyncd/rsyncd.secrets

    d. Establish the welcome information file / etc / rsyncd / rsyncd.motd seen by the client connected to the server

    # vim /etc/rsyncd/rsyncd.motd

    +++++++++++++++++++++++++++
    + David Camp +
    +++++++++++++++++++++++++++

    e. Start Rsync

    # /etc/init.d/xinetd restart

    f. Check whether port 873 is up

    # netstat -an | grep 873

    If Rsync is started successfully, you can see that port 873 is already listening.

    g. Server side file details

    5.2. Client configuration

    a. Client installation Rsync

    # yum -y install rsync

    b. Synchronize data through Rsync client

    Scenario 1:

    # rsync -avzP [email protected]::davidhome /tmp/david/

    Password: Here you need to enter the password of David, which is provided by the server. In the previous example, we use asdf, and the entered password is not displayed; Enter after losing;
    Note:This command means to log in to the server with the David user and synchronize the David home data to the local directory / TMP / David /. Of course, the local directory can be defined by yourself, such as Dave; When there is no davidhome directory under the current operating directory on the client, the system will automatically create one for you; When it exists in the directory David home, you should pay attention to its write permission.

    explain:
    -A parameter, equivalent to – rlptgod, – R is recursive, – L is linked file, which means copying linked file- P means to keep the original permissions of the file- T keep the original time of the document- G. keep the original user group of the file- O keep the original owner of the document- D is equivalent to block device file;
    -Z compression during transmission;
    -P transmission progress;
    -V transmission progress and other information have something to do with – P. try it yourself. You can see the document;

    Scenario 2:
    # rsync -avzP –delete [email protected]::davidhome /tmp/david/

    This time, we introduce a – delete option, which means that the data on the client side should be completely consistent with that on the server side. If there are files in the / TMP / David / directory that do not exist on the server, they will be deleted. The ultimate goal is to make the data in / TMP / David / directory completely consistent with that on the server; Be careful when using. It’s best not to take the directory with important data as the local update directory, otherwise all your data will be deleted;

    Scenario 3:
    # rsync -avzP –delete –password-file=/tmp/rsync.password [email protected]::davidhome /tmp/david/

    This time, we added an option – password file = rsync.password. When we log in to the Rsync server as David user to synchronize data, the password will read the file / TMP / rsync.password. The content of this file is only the password of David user. We should do the following;

    # touch /tmp/rsync.password
    # chmod 600 /tmp/rsync.password
    # echo “asdf”> /tmp/rsync.password
    # rsync -avzP –delete –password-file=/tmp/rsync.password [email protected]::davidhome /tmp/david/

    Note:So you don’t need a password; In fact, this is more important because it is necessary for the server to plan tasks through crond;

    5.3. Rsync client automatically synchronizes data with server

    Edit crontab
    # crontab -e
    Add the following code:

    10 0 * * * rsync -avzP –delete –password-file=/tmp/rsync.password [email protected]::davidhome /tmp/david/

    It means that the following commands are executed at 0:10 every day.

    6、 Error analysis

    @ERROR: chdir failed
    rsync error: error starting client-server protocol (code 5) at main.c(1530) [receiver=3.0.6]

    rsync: opendir “.” (in xxxxxxx) failed: Permission denied (13)

    terms of settlement:

    1. Remove all restrictions of SELinux on Rsync:
    # /usr/sbin/setsebool -P rsync_disable_trans 1
    # service xinetd restart

    2. Be cruel and prohibit the whole SELinux:
    # vim /etc/selinux/config
    Change SELinux = forcing to SELinux = disabled
    After saving and exiting, restart the machine.

    At this point, the Rsync server is configured.

    7、 Reference

    For the use of Rsync command, refer to: http://www.howtocn.org/rsync:use_ rsync

    For a detailed description of Rsync service, please refer to: http://www.howtocn.org/rsync:use_ rsync_ server