uniq

uniq [options] [input [output]]

uniq filters adjacent matching lines from input, writing unique lines to output. It only compares consecutive lines, so input typically needs to be sorted first.Commonly used with sort in a pipeline: sort file | uniq removes all duplicates. Adding -c shows frequency counts, useful for analyzing log files or finding common patterns.The -d option shows only lines that appear more than once, while -u shows only lines appearing exactly once. These are mutually exclusive perspectives on the data.Field and character skipping options allow ignoring prefixes (like timestamps or line numbers) when comparing for uniqueness.

-c, --count

Prefix lines with occurrence count

Only print duplicate lines, one per group

Print all duplicate lines

Only print unique lines (occurring once)

Ignore case when comparing

Skip first N fields when comparing

Skip first N characters when comparing

Compare only first N characters

Line delimiter is NUL, not newline

Uniq only removes adjacent duplicates. Without sorting first, non-adjacent duplicates remain. Always use sort | uniq or sort -u for true deduplication.sort -u is often more efficient than sort | uniq as it removes duplicates during sorting rather than in a separate pass.The count option (-c) prefixes with spaces and count, which may need processing for further use.

sort(1), comm(1), cut(1), wc(1)