How to Rsync between two folders automatically on file changes

A common usage of rsync is to copy files from one location to another, rsync is the successor of scp command . However, there are some advantages that rsync has over scp:

1. Incremental transfer: rsync can transfer only the differences between two files, which makes it faster and more efficient than scp. This is particularly useful when transferring large files or directories with many files that have already been transferred before.
2. Bandwidth-friendly: rsync can limit the amount of bandwidth it uses during the transfer, which makes it a better choice for transferring files over slow or limited bandwidth connections. scp does not have this feature.
3. Resuming interrupted transfers: rsync can resume interrupted transfers from where it left off, which is useful if a transfer is interrupted for any reason. scp does not have this capability.
4. Preserving permissions: rsync can preserve file permissions and ownership during transfer, which is useful when transferring files between systems with different user and group IDs. scp does not have this capability.

How to use Rsync commands

There’s a lot of great rsync tutorials online, so I wont repeat the basics here, but if you want to learn them check our these links , or this youtube video:

Rsync between two directories when a file changes (Rsync push / interrupt driven)

A common scenario is to initiate the rsync command any time a file or directory changes. On linux based system this can be accomplished between one of two ways, via a push (interrupt-based mechanism) or polling.

Let’s first look at the interrupt driven (push based) , first we need a reliable way to detect file changes, and one of the common ways inLinux is to use inotifywait . Inotifywait is normally part of a a package called inotify-tools, install it via the command below

sudo apt update
sudo apt install inotify-tools
inotifywait --help

Next you can craft a rsync command using inotifywait to watch one or more files/directories for changes and then trigger an rsync action action, here’s an example”

#!/bin/bash

#save as syncwatch.sh


# Set the local and remote directories
local_dir="/path/to/local"
remote_dir="user@host_or_ip:/path/to/remote"

# Set the rsync options
rsync_options="-az  --progress "


# Start the loop to monitor for changes
while true; do
    # Use the inotifywait command to monitor for file changes in the local directory
    inotifywait -r -e modify,create,delete,move ${local_dir}

    # Trigger rsync to copy all changed files from local to remote
    # echo rsync ${rsync_options} ${local_dir}/ ${remote_dir}
    rsync ${rsync_options} -e 'ssh -p 22' ${local_dir}/ ${remote_dir}
done

This example code (lets call it syncwatch.sh ) will watch a given folder and then when changes are detected it will automatically rsync the contents to the remove server. You can leave this script running by running it with nohup synchwatch.sh &, to send it to the background

Rsync between two directories when a file changes (Rsync polling)

An alternate approach is to poll/ or periodically request to run the rsync command . The traditional way to do this is via the linux cron tab , which provides an easy mechanism to handle the scheduling of periodic scripts.

First here’s a simple script, let’s call it syncpoll.sh, the retrieves files from a remote server on a periodic basis.

#!/bin/bash

# Remote server settings
remote_user="user"
remote_server="hostname_or_ip"
remote_folder="/path/to/remote/folder"

# Local machine settings
local_folder="/path/to/local"

# Run rsync to copy the remote folder to local folder
# ssh -p port left in there in case of non-standard ssh port
rsync -avz -e 'ssh -p 22' "${remote_user}@${remote_server}:${remote_folder}" "${local_folder}"

#Optional you may need to change ownership or permission
#chown -R www-data:www-data ${local_folder}

# Print message to indicate the script has finished running
echo "rsync from remote server to local completed at $(date)"

Save as a file like syncpoll.sh, change the execute permissions like chmod +x syncpoll.sh Next to run the periodic command above simple edit the crontab , like crontab -e then add the following line.

*/15 * * * * /bin/bash /path/to/script/syncpoll.sh

This command will run the script ever 15 minutes, for different intervals, simple modify the crontab , if you’re not sure how to use a crontab try one of the may gui’s like https://crontab-generator.org/ or heck even just ask ChatGPT (“write me a crontab entry for every Sunday at 2pm“)

Rsync wrap up

Well there you have it two ways to let rsync keep you files/folder and data using two particle methods.