1、 Introduction to Rsync
The Rsync / remote tool can be used to quickly synchronize data between different hard disks in the LAN, or multiple remote hard disks.
Rsync is a tool used to replace RCP. Rsync uses the so-called “Rsync algorithm” to synchronize the files between the local and remote hosts. This algorithm only transmits different parts of the two files, not the whole file every time, so it is very fast. You can refer to how Rsync works a practical overview to further understand the operation mechanism of Rsync.
Rsync supports most UNIX like systems and has been well tested on Linux, Solaris and BSD. In addition, it also has corresponding versions under the windows platform, and the well-known ones are cwrsync and sync2nas.
The original authors of Rsync are Andrew Tridgell and Paul Mackerras, which are currently http://rsync.samba.org maintain.
The basic features of Rsync are as follows:
1. The entire directory tree and file system can be saved in an image;
2. It is easy to maintain the permissions, time, soft and hard links of the original documents;
3. It can be installed without special permission;
4. Optimized process and high file transmission efficiency;
5. You can use RCP, SSH and other methods to transfer files. Of course, you can also connect directly through socket;
6. Support anonymous transmission to facilitate website mirroring.
When Rsync is used for remote synchronization, two methods can be used: remote shell (SSH is recommended, and SSH is responsible for user authentication) and C / S (that is, the client connects to the remote Rsync server, and Rsync server is responsible for user authentication).
Regardless of the local synchronization directory or remote synchronization data, all files will be copied once in the first run, and only the changed files (for the new files) or the changed parts of the files (for the original files) will be copied in the future run.
Rsync has no speed advantage during the first replication and is not as fast as tar. Therefore, when the amount of data is large, you can consider using tar for the first replication and then Rsync for data synchronization.
2、 System environment
System platform:CentOS release 6.3 (Final)
Rsync client:TS-CLIENT （172.16.1.136）
3、 Server side installation of Rsync service
3.1. Check whether Rsync is installed
# rpm -qa|grep rsync
If installed, uninstall using the RPM – e command.
3.2. Download RPM package
# wget http://pkgs.repoforge.org/rsync/rsync-3.0.9-2.el6.rfx.x86_64.rpm
3.3. Installing Rsync
# rpm -ivh rsync-3.0.9-2.el6.rfx.x86_64.rpm
4、 Configure Rsync service
4.1. Steps to configure Rsync server
4.2. Running Rsync service with xinetd
CentOS runs the Rsync service in xinetd mode by default. Xinetd configuration file for Rsync
In / etc / xinetd.d/rsync. To configure the Rsync service running with xinetd, execute the following command:
Administrators can modify the / etc / xinetd.d/rsync configuration file to suit your needs. For example, you can modify the configuration line
Add the service option of Rsync later.
4.3. Run Rsync service independently
The simplest way to run Rsync service independently is to execute the following command:
You can write the above command to the / etc / rc.local file to run the Rsync service every time you start the server. Of course, you can also write a script to automatically start the rysnc service at startup.
4.4. Configuration file rsyncd.conf
Rsyncd.conf needs to be configured for both Rsync service running modes, and its format is similar to the main configuration file of samba.
The configuration file rsyncd.conf is in the / etc directory by default. To put all files related to Rsync service in a separate directory, execute the following command:
The configuration file rsyncd.conf consists of global configuration and several module configurations. The syntax of the configuration file is:
All configuration lines except [module] in the file are global parameters. Of course, you can also define module parameters in the global parameters section. At this time, the value of this parameter is the default value of all modules.
|address||When running independently, the IP address used to specify the server to run. This parameter is ignored by xinetd runtime and replaced with the – address option on the command line.||Local all IP|
|port||Specifies the port number on which the Rsync daemon listens. This parameter is ignored by xinetd runtime and replaced with the – port option on the command line.||873|
|motd file||Specifies a message file whose contents are displayed to the client when the client connects to the server.||nothing|
|pid file||The daemon of Rsync writes its PID to the specified file.||nothing|
|log file||Specifies the log file for the Rsync daemon without sending the log to syslog.||nothing|
|syslog facility||Specifies the message level when Rsync sends log messages to syslog.||daemon|
|socket options||Specify custom TCP options.||nothing|
Module parameters are mainly used to define which directory of Rsync server should be synchronized. The format of module declaration must be in the form of [module]. This name is the name seen in Rsync client, similar to the share name provided by Samba server. The data actually synchronized by the server is specified through path. You can specify multiple modules according to your needs. The following parameters can be defined in the module:
a. Basic module parameters
|path||The specified Rsync parameter must be on the current Rsync server.||nothing|
|comment||Assign a description to the module, which is displayed to the customer together with the module name when the customer gets the module list through connection.||nothing|
b. Module control parameters
|use chroot||If true, Rsync will first chroot to the directory specified by the path parameter before transferring the file. The reason for this is to achieve additional security protection, but the disadvantage is that root permission is required, and the directory file pointed to by the symbolic connection outside the path cannot be backed up.||true|
|uid||Specifies that the module transfers files with the specified uid.||nobody|
|gid||Specifies that the module transfers files with the specified GID.||nobody|
|max connections||Specify the maximum number of concurrent connections of the module to protect the server. Connection requests exceeding the limit will be told and then try again.||0 (no limit)|
|lock file||Specifies a lock file that supports the max connections parameter.||/var/run/rsyncd.lock|
|list||Specifies whether the module should be listed when the customer requests a list of available modules. If this option is set to false, you can create hidden modules.||true|
|read only||Specify whether to allow customers to upload files. If true, upload is not allowed; If it is false and the server directory also has read-write permission, upload is allowed.||true|
|write only||Specify whether to allow customers to download files. If true, downloading is not allowed; If it is false and the server directory also has read permission, it is allowed to download.||false|
|ignore errors||Specifies whether I / O errors are ignored when a delete operation is run on the Rsync server. In general, Rsync will skip the – delete operation in case of I / O errors to prevent serious problems caused by temporary resource shortage or other I / O errors.||true|
|ignore nonreadable||Specifies that the rysnc server completely ignores files that users do not have access to. This makes sense when there are some files in the directory to be backed up that should not be obtained by the backup person.||false|
|timeout||This option can override the IP timeout specified by the customer. This ensures that the Rsync server will not wait forever for a crashed client. For anonymous Rsync servers, the ideal number is 600 (in seconds).||0 (unlimited)|
|dont compress||Used to specify files that are not compressed before transfer. This option can define a list of command options that customers are not allowed to use for this module. You must use the full name of the option, not the short name. When an option is rejected, the server reports an error message and exits. For example, to prevent the use of compression, it should be: “Don compress = *”.||*.gz *.tgz *.zip *.z *.rpm *.deb *.iso *.bz2 *.tbz|
c. Module file filter parameters
|exclude||Specify multiple files or directories (relative paths) separated by spaces and add them to the exclude list. This is equivalent to using – exclude in the client command to specify the schema.||empty|
|exclude from||Specify a file name containing the exclude rule definition from which the server reads the exclude list definition.||empty|
|include||Specify multiple files or directories (relative paths) separated by spaces and add them to the include list. This is equivalent to using – include in the client command to specify the schema.||empty|
|include from||Specify a file name containing the include rule definition from which the server reads the include list definition.||empty|
A module can only specify one exclude parameter and one include parameter. You can define complex exclude / include rules by combining include and exclude. These parameters are equivalent to the corresponding Rsync client command options. The only difference is that they act on the server side. For details on how to write rule files, please refer to http://www.howtocn.org/rsync:use_ rsync。
d. Module user authentication parameters
|auth users||Specify a list of user names separated by spaces or commas, which are the only users allowed to connect to the module. The users here have nothing to do with the system users. The user name and password are stored in clear text in the file specified by the secrets file parameter.||(anonymous)|
|secrets file||Specify an Rsync authentication password file. This file works only if auth users is defined.||empty|
|strict modes||Specify whether to monitor the permissions of the password file. If true, the password file can only be accessed by the user running the Rsync server, and no other user can access the file.||true|
The permission of Rsync authentication password file must be 600, otherwise the client will not be able to connect to the server. Each line in the Rsync authentication password file specifies a user name: password pair in the format:
Generally speaking, the password should not exceed 8 characters. If you only configure an anonymous Rsync server, you do not need to set the above parameters.
e. Module access control parameters
|hosts allow||Use a host list to specify which host clients are allowed to connect to the module. Hosts that do not match the host list will be rejected.||*|
|hosts deny||Use a host list to specify which host clients are not allowed to connect to the module.||empty|
The client host list definition can be in the following form:
A single IP address. For example: 192.168.0.1 the entire network segment. For example: 192.168.0.0/24192.168.0.0/255.255.255.0 resolvable single hostname. For example: CentOS, all hosts in the centos.bsmart.cn domain. For example, *. Bsmart. CN “*” means all. Multiple list items should be separated by spaces.
f. Module log parameters
|transfer logging||Causes the Rsync server to log the transfer operation to the transfer log file.||false|
|log format||Specifies the field for the transfer log file.||”%o %h [%a] %m (%u) %f %l”|
“% t [% P]” will be added at the beginning of each log line when the “log file” parameter is set.
5、 Application case of Rsync server
5.1. Configure Rsync service on server ts-dev
a. Edit profile
# vi /etc/rsyncd/rsyncd.conf
# GLOBAL OPTIONSuid = root gid = root
use chroot = no
read only = yes #limit access to private LANshosts allow=172.16.0.0/255.255.0.0 192.168.1.0/255.255.255.0 10.0.1.0/255.255.255.0hosts deny=* max connections = 5
pid file = /var/run/rsyncd.pid
secrets file = /etc/rsyncd/rsyncd.secrets #lock file = /var/run/rsync.lock
motd file = /etc/rsyncd/rsyncd.motd #This will give you a separate log filelog file = /var/log/rsync.log #This will log every file transferred – up to 85,000+ per user, per synctransfer logging = yes log format = %t %a %m %f %bsyslog facility = local3timeout = 300
# MODULE OPTIONS[davidhome] path = /home/david/ list=yes ignore errors auth users = david comment = David home exclude = important/ [chinatmp]path = /tmp/china/list=noignore errorsauth users = chinacomment = tmp_china
b. Create the / etc / rsyncd / rsyncd.secrets file
# vim /etc/rsyncd/rsyncd.secrets
David: asdf # format user name: password
China: JK # this user is not required to be a system user
c. For password security, we set the permission to 600
# chown root:root /etc/rsyncd/rsyncd.secrets
# chmod 600 /etc/rsyncd/rsyncd.secrets
d. Establish the welcome information file / etc / rsyncd / rsyncd.motd seen by the client connected to the server
# vim /etc/rsyncd/rsyncd.motd
+ David Camp +
e. Start Rsync
# /etc/init.d/xinetd restart
f. Check whether port 873 is up
# netstat -an | grep 873
If Rsync is started successfully, you can see that port 873 is already listening.
g. Server side file details
5.2. Client configuration
a. Client installation Rsync
# yum -y install rsync
b. Synchronize data through Rsync client
# rsync -avzP [email protected]::davidhome /tmp/david/
Password: Here you need to enter the password of David, which is provided by the server. In the previous example, we use asdf, and the entered password is not displayed; Enter after losing;
Note:This command means to log in to the server with the David user and synchronize the David home data to the local directory / TMP / David /. Of course, the local directory can be defined by yourself, such as Dave; When there is no davidhome directory under the current operating directory on the client, the system will automatically create one for you; When it exists in the directory David home, you should pay attention to its write permission.
-A parameter, equivalent to – rlptgod, – R is recursive, – L is linked file, which means copying linked file- P means to keep the original permissions of the file- T keep the original time of the document- G. keep the original user group of the file- O keep the original owner of the document- D is equivalent to block device file;
-Z compression during transmission;
-P transmission progress;
-V transmission progress and other information have something to do with – P. try it yourself. You can see the document;
# rsync -avzP –delete [email protected]::davidhome /tmp/david/
This time, we introduce a – delete option, which means that the data on the client side should be completely consistent with that on the server side. If there are files in the / TMP / David / directory that do not exist on the server, they will be deleted. The ultimate goal is to make the data in / TMP / David / directory completely consistent with that on the server; Be careful when using. It’s best not to take the directory with important data as the local update directory, otherwise all your data will be deleted;
# rsync -avzP –delete –password-file=/tmp/rsync.password [email protected]::davidhome /tmp/david/
This time, we added an option – password file = rsync.password. When we log in to the Rsync server as David user to synchronize data, the password will read the file / TMP / rsync.password. The content of this file is only the password of David user. We should do the following;
# touch /tmp/rsync.password
# chmod 600 /tmp/rsync.password
# echo “asdf”> /tmp/rsync.password
# rsync -avzP –delete –password-file=/tmp/rsync.password [email protected]::davidhome /tmp/david/
Note:So you don’t need a password; In fact, this is more important because it is necessary for the server to plan tasks through crond;
5.3. Rsync client automatically synchronizes data with server
# crontab -e
Add the following code:
It means that the following commands are executed at 0:10 every day.
6、 Error analysis
rsync: opendir “.” (in xxxxxxx) failed: Permission denied (13)
terms of settlement:
1. Remove all restrictions of SELinux on Rsync:
# /usr/sbin/setsebool -P rsync_disable_trans 1
# service xinetd restart
2. Be cruel and prohibit the whole SELinux:
# vim /etc/selinux/config
Change SELinux = forcing to SELinux = disabled
After saving and exiting, restart the machine.
At this point, the Rsync server is configured.
For the use of Rsync command, refer to: http://www.howtocn.org/rsync:use_ rsync
For a detailed description of Rsync service, please refer to: http://www.howtocn.org/rsync:use_ rsync_ server