LinuxCommandLibrary

lsyncd

Synchronize directories quickly to remote destinations

TLDR

Watch the source for changes and run rsync to synchronize files to the destination on every change

$ lsyncd -rsync [path/to/source] [host::share_name]
copy

Use SSH instead of rsyncd shares
$ lsyncd -rsyncssh [path/to/source] [host] [path/to/destination]
copy

SYNOPSIS

lsyncd [OPTIONS...] CONFIG_FILE

PARAMETERS

-help, -h
    Display a help message and exit.

-version, -v
    Show the lsyncd version and exit.

-log FILE, -L FILE
    Specify the path to the log file.

-log-level LEVEL, -l LEVEL
    Set the logging verbosity (e.g., error, warning, notice, info, debug).

-daemon, -d
    Run lsyncd as a background daemon process.

-pidfile FILE, -p FILE
    Write the process ID to the specified file.

-config FILE, -c FILE
    Specify the Lua configuration file to use. This is mandatory for operation.

-delay SECONDS
    Override the default delay setting from the config file, waiting this many seconds before rsyncing.

-dry-run
    Perform a dry run, simulating synchronization without making actual changes.

-no-daemon
    Prevent lsyncd from daemonizing, useful for debugging.

-syslog
    Send log messages to syslog instead of a file or stdout.

DESCRIPTION

lsyncd is a lightweight daemon that provides real-time file synchronization between local directories and remote hosts. It efficiently monitors local filesystem changes using inotify events (on Linux) or fsevents (on macOS) and then leverages rsync as its underlying transfer mechanism to propagate those changes. This makes it an ideal solution for live mirroring, continuous backups, and maintaining synchronized data across servers with minimal latency. Its core strength lies in its event-driven nature, avoiding the overhead of frequent polling, and its flexibility, which is primarily configured through a powerful Lua scripting interface. It's designed for simple setup and high performance, making it popular for replicating web content, logs, or other critical data.

CAVEATS

lsyncd relies on inotify limits for watched files/directories, which may need tuning for very large file trees. It requires rsync to be installed and accessible on both the local and remote systems. Initial synchronization of large datasets is often best handled by a separate, one-time rsync command, as lsyncd is designed for incremental changes. Its Lua-based configuration, while powerful, can have a learning curve. Care must be taken with rsync options, especially regarding file deletions, to ensure desired behavior.

CONFIGURATION FILE

lsyncd's behavior is almost entirely defined by a Lua-scripted configuration file. This file specifies source directories, destination hosts, rsync parameters, SSH settings, logging options, and rules for what to include or exclude during synchronization. This powerful scripting capability allows for highly customized mirroring setups.

HOW IT WORKS

At its core, lsyncd establishes watches on directories using the inotify API (or fsevents on macOS). When the kernel notifies lsyncd of changes (e.g., file creation, modification, deletion, move), it collects these events. After a configurable delay (to batch multiple rapid changes), lsyncd executes an rsync command, passing the detected changes and specified options to efficiently synchronize the local and remote locations.

HISTORY

lsyncd emerged as a pragmatic solution to real-time file synchronization challenges. It was conceived to be lightweight and efficient by combining existing, robust Linux primitives: inotify for event detection and rsync for data transfer. This approach avoids the resource intensity of polling-based synchronization tools and the complexity of full-blown distributed file systems. The use of Lua for configuration provided a flexible and powerful way to define synchronization tasks, making it adaptable to various scenarios since its early development.

SEE ALSO

rsync(1), inotifywait(1), fswatch(1), cron(8), unison(1)

Copied to clipboard