<<
Servers often need to keep files or directories consistent with each other. For example, large software download websites typically use multiple servers to provide download services. When files on one server are updated, the other servers also need to be updated, and the update should only apply to new or modified files; otherwise, it results in wasted network bandwidth and time. rsync is an excellent piece of software that effectively maintains file and directory consistency.
rsync, remote synchronize
As the name suggests, it is a software tool that implements remote synchronization functionality. While synchronizing files, it can preserve the original file permissions, timestamps, soft and hard links, and other metadata. It can also transfer files via ssh, providing very good confidentiality. Additionally, it is free software. rsync’s official website: http://rsync.samba.org/, where you can get the latest version. Of course, because rsync is such a useful tool, many Linux distributions include it. If your Linux does not have rsync installed, you can install it manually using the following method:
I. Installation Process
1. Download rsync
Currently (September 2003), the latest rsync version is 2.5.6. Download it from the rsync official website:
# wget http://ftp.samba.org/ftp/rsync/rsync-2.5.6.tar.gz
2. Extract
# tar -xzpvf rsync-2.5.6.tar.gz
3. Compile and Install
# cd rsync-2.5.6/
# ./configure –prefix=/usr/local/rsync
# make
# make install
If no errors occurred during the above process, the installation is complete. The rsync command is now available, located in /usr/local/rsync/bin. You can use the rsync command to fetch data from servers running the rsync service.
If you want to turn the current machine into an rsync server, you will need to perform some additional configuration.
II. Configuring the rsync Service
Configuring a simple rsync service is not complicated; you need to modify or create some configuration files.
1. rsyncd.conf
# vi /etc/rsyncd.motd
rsyncd.conf is the main configuration file for the rsync service. It controls various properties of the rsync service. Below is an example of an rsyncd.conf file:
# First, define global variables
secrets file = /etc/rsyncd.secrets
motd file = /etc/rsyncd.motd
read only = yes
list = yes
uid = nobody
gid = nobody
hosts allow = 192.168.100.90 # Which computers can access the rsync service
hosts deny = 192.168.100.0/24 # Which computers cannot access the rsync service
max connections = 2
log file = /var/log/rsyncd.log
pid file = /var/run/rsyncd.pid
lock file = /var/run/rsync.lock
# Then, define the rsync directories
[terry]
comment = Terry ‘s directory from 192.168.100.21
path = /home/terry
auth users = terry,rsync
[test]
comment = test rsync
path = /home/test
In the above configuration file, within the 192.168.100.0/24 subnet, only the machine 192.168.100.90 is allowed to access this rsync server’s service. The latter part of the configuration file defines two rsync directories: the ‘terry’ directory can only be accessed by users who know the ‘terry’ or ‘rsync’ account credentials, while the ‘test’ directory can be accessed without an account. rsync also provides other options when defining directories for stricter control.
2. rsyncd.secrets
# vi /etc/rsyncd.secrets
rsyncd.secrets stores the usernames and passwords for the rsync service. It is a plaintext file. Below is an example of an rsyncd.secrets file:
terry:12345
rsync:abcde
Because rsyncd.secrets stores the rsync service’s usernames and passwords, it is very important. Therefore, the file’s permissions must be set to 600, allowing only the owner to read and write:
# chmod 600 /etc/rsyncd.secrets
3. rsyncd.motd
# vi /etc/rsyncd.motd
rsyncd.motd records the welcome message for the rsync service. You can enter any text information in it, such as:
Welcome to use the rsync services!
4. services
# vi /etc/services
services is not an rsync configuration file, and this step is optional. The benefit of modifying the services file is that the system knows port 873 corresponds to the service name rsync. The method for modifying services is to ensure the following two lines exist in services; if not, add them manually:
rsync 873/tcp # rsync
rsync 873/udp # rsync
5. /etc/xinetd.d/rsync
# vi /etc/xinetd.d/rsync
Create a file named /etc/xinetd.d/rsync and enter the following content:
service rsync
{
disable = no
socket_type = stream
wait = no
user = root
server = /usr/local/rsync/bin/rsync
server_args = –daemon
log_on_failure += USERID
}
After saving, you can start the rsync service. Enter the following command:
# /etc/rc.d/init.d/xinetd reload
Now the rsync service is running on this machine (192.168.100.21). Next is how to use it.
III. rsync Command Usage
After configuring the rsync server, you can issue rsync commands from the client to perform various synchronization operations. rsync has many functional options. Let’s introduce some commonly used options below:
The rsync command format can be:
1. rsync [OPTION]… SRC [SRC]… [USER@]HOST:DEST
2. rsync [OPTION]… [USER@]HOST:SRC DEST
3. rsync [OPTION]… SRC [SRC]… DEST
4. rsync [OPTION]… [USER@]HOST::SRC [DEST]
5. rsync [OPTION]… SRC [SRC]… [USER@]HOST::DEST
6. rsync [OPTION]… rsync://[USER@]HOST[:PORT]/SRC [DEST]
rsync has six different working modes:
1. Copying local files; this mode is activated when neither the SRC nor DEST path information contains a single colon “:” separator.
2. Using a remote shell program (like rsh, ssh) to copy content from the local machine to a remote machine. This mode is activated when the DEST path address contains a single colon “:” separator.
3. Using a remote shell program (like rsh, ssh) to copy content from a remote machine to the local machine. This mode is activated when the SRC path address contains a single colon “:” separator.
4. Copying files from a remote rsync server to the local machine. This mode is activated when the SRC path information contains a “::” separator.
5. Copying files from the local machine to a remote rsync server. This mode is activated when the DEST path information contains a “::” separator.
6. Listing files on a remote machine. This is similar to an rsync transfer, but the local machine information is omitted in the command.
Let’s illustrate with examples:
# rsync -vazu -progress [email protected]:/terry/ /home
v for verbose output
a for archive mode operation, copying directories and symbolic links
z for compression
u for update only, preventing local new files from being overwritten; note the clocks on both machines should be synchronized
-progress for showing progress
The above command keeps the /home/terry directory on client 192.168.100.90 synchronized with the terry directory on the rsync server. Before executing synchronization, this command will prompt you for the password for the ‘terry’ account, which we defined earlier in the rsyncd.secrets file. If you want to put this command into a script and execute it periodically, you can use the –password-file option, as shown in the specific command below:
# rsync -vazu -progress –password-file=/etc/rsync.secret
[email protected]:/terry/ /home
To use the –password-file option, you must first create a file to store the password, specified here as /etc/rsync.secret. Its content is very simple, as follows:
terry:12345
Similarly, modify the file properties as follows:
# chmod 600 /etc/rsyncd.secrets
IV. Example: Using rsync to Keep Files Synchronized Between Linux Servers
Now, suppose there are two Linux servers, A (192.168.100.21) and B (192.168.100.90). The directories /home/terry on Server A and /home/terry on Server B need to stay synchronized, meaning that when files on Server A change, the files on Server B should change accordingly.
Following the method above, we install rsync on Server A and configure it as an rsync server, setting up the /home/terry directory as a shared rsync directory. Then, install rsync on Server B. Since B acts only as a client, no configuration is needed. Then, on Server B, create the following script:
#!/bin/bash