Text Processing
sed applies editing commands to each line. The substitute command s/old/new/ replaces the first match per line; g replaces all matches, I ignores case. -i edits the file in place instead of printing the result.$ sed 's/old/new/g' [file] $ sed -i 's/old/new/g' [file] $ sed -i 's/old/new/gI' [file] sd does the same job with simpler, regex-by-default syntax.Any character can delimit the s command: s|/usr/bin|/usr/local/bin| avoids escaping slashes in paths.
d deletes matching lines; -n with p prints only selected lines. Addresses can be patterns, line numbers, or ranges.$ sed '/pattern/d' [file] $ sed -n '/pattern/p' [file]
awk splits every line into fields: $1 is the first field, $0 the whole line, NR the line number. -F changes the field separator from whitespace to anything else.$ awk '{print $1}' [file] $ awk -F: '{print $1, $3}' /etc/passwd $ awk '{print NR, $0}' [file] cut is the lightweight alternative for simple column extraction, by delimiter (-d , -f ) or character position (-c ).
An awk program is condition { action } : lines matching the condition run the action (default: print the line). $ awk '/pattern/ {print $2}' [file] $ awk 'NR>=5 && NR<=10' [file] Aggregate across lines with variables and an END block. $ awk '{sum += $1} END {print sum}' [file] $ awk '{sum += $1} END {print sum/NR}' [file]
Alphabetical by default; -n sorts numerically, -r reverses, -u drops duplicates, -t and -k sort by a specific field. -h understands human-readable sizes like 2K and 1G.
uniq only compares neighboring lines, so sort first. -c counts occurrences, -d shows only duplicated lines, -u only unique ones.The sort | uniq -c | sort -rn pipeline is the classic frequency counter: it ranks every distinct line by how often it occurs.
tr maps characters from one set to another, -d deletes them, -s squeezes repeats into one.$ tr 'a-z' 'A-Z' < [file] $ tr -d '[:digit:]' < [file]
diff -u is the standard patch-style comparison; -y shows files side by side; cmp compares bytes and is ideal for binary files.$ diff -u [file1] [file2] $ diff -y [file1] [file2] comm shows lines unique to each sorted file and lines they share, in three columns. Suppress columns by number.$ comm -12 [file1] [file2] Both comm and join require their input files to be sorted.
paste glues files together line by line; -s joins all lines of one file into a single line. join matches lines from two files on a common field, like a database join.$ paste -d',' [file1] [file2] $ join -t: -1 1 -2 3 [file1] [file2]
Align fields into a table, or wrap long lines. fmt rewraps paragraphs intelligently, fold cuts hard at the width (-s breaks at spaces).
nl numbers only non-empty lines by default; -ba numbers all lines, like cat -n .
Lines, words, and bytes; -m counts characters in multi-byte encodings.
tac prints a file last line first, rev reverses each line's characters, shuf randomizes line order.
Copied to clipboard
