fdupes

Find duplicate files in a directory

$ fdupes [/path/to/directory]

$ fdupes -r [/path/to/directory]

$ fdupes -d [/path/to/directory]

$ fdupes -dN [/path/to/directory]

$ fdupes -S [/path/to/directory]

$ fdupes -m [/path/to/directory]

$ fdupes [/path/dir1] [/path/dir2]

$ fdupes -r [/path/original] [/path/copy]

fdupes [options] directory...

fdupes identifies duplicate files in specified directories by comparing file sizes and MD5 signatures, followed by a byte-by-byte comparison for verification. It groups duplicate files together for review or automatic deletion.
The tool first compares file sizes, then calculates partial and full MD5 hashes, and finally performs byte comparison to ensure accuracy. This multi-stage approach balances speed and accuracy.
fdupes can operate across multiple directories, finding duplicates within and between them. It's useful for cleaning up backup copies, photo libraries, and reclaiming disk space.

-r, --recurse

Search directories recursively.

Recursive for following directories only.

Prompt for files to delete.

With -d, delete without prompting.

Show size of duplicate files.

Show summary statistics.

List sets on single lines.

Exclude empty files.

Treat hard links as duplicates.

Omit first file in each set.

Quiet mode (for -d operation).

Reverse order (delete first file).

Replace duplicates with hard links.

Byte-by-byte comparison can be slow for large files. Delete operations are irreversible. Hard link mode changes file relationships. Cannot compare across different filesystems for hard linking.

fdupes was created by Adrian Lopez in 1999 as a utility for finding duplicate files. It has been maintained and enhanced over the years, becoming a standard tool for duplicate detection on Unix-like systems. The tool influenced many similar utilities and remains actively used for disk cleanup operations.

rmlint(1), find(1), md5sum(1), diff(1)