POSIX text filters, one filter per page
Thirty lessons on the classic text filters: cutting fields with -d and -f, lists and ranges, characters versus bytes and the one-character delimiter law; sorting with keys, numbers, human sizes, reverse, unique and stable, and the locale trap LC_ALL=C closes; uniq's adjacent-lines rule with counting and skipping; tr translation, ranges, classes, squeezing, deleting and complements; paste's parallel merge, cycling delimiters and the -sd idiom; head's minus counts, tail's plus counts and following, wc's counts and the missing newline; comm's three columns and join's keys, strays and set work; column -t from util-linux and pr's page headers; and two real pipelines, the top-ten counter and the two-file key join, with one real terminal task per page.
One filter, one command, one real output. That's a page.
POSIX text filters, one filter per page
Thirty lessons on the classic text filters: cutting fields with -d and -f, lists and ranges, characters versus bytes and the one-character delimiter law; sorting with keys, numbers, human sizes, reverse, unique and stable, and the locale trap LC_ALL=C closes; uniq's adjacent-lines rule with counting and skipping; tr translation, ranges, classes, squeezing, deleting and complements; paste's parallel merge, cycling delimiters and the -sd idiom; head's minus counts, tail's plus counts and following, wc's counts and the missing newline; comm's three columns and join's keys, strays and set work; column -t from util-linux and pr's page headers; and two real pipelines, the top-ten counter and the two-file key join, with one real terminal task per page.
Set in Space Grotesk, Inter and JetBrains Mono (SIL Open Font License).
Every fact in this book is as the official source states it, fetched and read during this build: the GNU Coreutils Manual 9.12 nodes for cut, sort, uniq, tr (including its Character arrays, Translating, and Squeezing repeats and deleting sub-nodes), paste, head, tail, wc, comm, join (including its Working with fields, Paired and unpaired lines, and Set operations sub-nodes) and pr; the POSIX.1-2017 utility pages for the eleven POSIX tools in this book (cut, sort, uniq, tr, paste, head, tail, wc, comm, join, pr); and the Linux man-pages man7 column(1) page, column being a util-linux tool and not a POSIX utility. Every command and output pair on every page was run on this machine's GNU coreutils 9.4, with column from util-linux 2.39.3, and its output confirmed before printing. No facts are sourced from Reddit, Stack Overflow or tutorials. An independent guide, not affiliated with or endorsed by the GNU Project, The Open Group, the Linux man-pages maintainers or the util-linux project.
Your purchase is for personal use only. You do not have redistribution rights: please do not share, resell, or republish this book or its pages.
© 2026 Steve Hodgkiss. All rights reserved. Personal use only; no redistribution rights.
Edition 1.0 · stevehodgkiss.net
Contents
Per GNU Coreutils Manual, cut invocation: with -f, use character as the input field separator, default TAB; fields, bytes and characters are numbered starting at 1.
The -d and -f pair
cut answers one question: which parts of each line do you want? -d, names the comma as the one delimiter, and -f1 asks for field one, counted from 1.
Given eu-west,web,2 and us-east,cache,5, the command prints eu-west and us-east, nothing else. With no -d at all the delimiter is the TAB character, the documented default.
A delimiter and a field list: every cut you will ever write is this page plus a longer list.
Make a three-column comma file, print field 1 with cut -d, -f1, then field 3, and confirm both outputs against the file.
Per GNU Coreutils Manual, sort invocation: unless otherwise specified, comparisons use the collating sequence specified by the LC_COLLATE locale, and the footnote advises setting LC_ALL to C when sort produces output sorted differently than expected.
sort and the locale
Plain sort compares under the LC_COLLATE locale, and locales disagree. The same four lines, a A B b, came out A B a b under LC_ALL=C and a A b B under en_US.utf8 on this machine.
Case folded together, order changed: a pipeline two steps later, comm or join, would silently disagree with a file sorted under a different locale.
The manual's own advice: when the order surprises you, pin it with LC_ALL=C, bytewise and boring.
Sort the lines a, A, B, b twice, once under LC_ALL=C and once under a UTF-8 language locale, and diff the two orders.
Per GNU Coreutils Manual, uniq invocation: -c prints the number of times each line occurred along with the line; Per GNU Coreutils Manual, sort invocation: -r with -n ranks numerically from the largest initial number.
Count with -c
uniq -c prefixes each line with the number of times it occurred: five log lines become counts of 2, 1 and 2. The count sits in front, so the next sort reads it.
sort -rn ranks numerically, greatest first. Ties keep order only by the rest of the line, which is why 2 POST /login can top 2 GET /.
Count, then rank: two commands that between them answer most how-often questions ever asked of a log.
Pipe a five-line method log through sort, uniq -c and sort -rn, and confirm the counts add back up to five.
Per GNU Coreutils Manual, tr invocation and the Translating sub-node: tr translates each input character in string1 to the corresponding character in string2; characters not in string1 pass through unchanged.
tr swaps letters
tr is a cipher wheel. tr el ip maps e to i and l to p, position for position; everything else passes through untouched. hello becomes hippo.
It reads standard input and writes standard output, no files listed on the command line. The two strings are arrays of characters, not patterns: no regex has ever run inside tr.
Every character in SET1 needs its counterpart in SET2, one for one, and that is the whole contract.
Pipe the word hello through tr el ip, then through tr abc xyz with a word containing all three letters, and predict both outputs before running.
head, tail, wc
Per GNU Coreutils Manual, head and tail invocations: head prints the first ten lines by default and a minus-prefixed -n prints all but the last lines; tail prints the last lines, a plus-prefixed -n starts at line K, and -f follows a growing file while -F follows the name with retry. Per wc invocation: counts print in the order newlines, words, bytes, and a file ending without a newline has its trailing partial line uncounted.
- 01head -n, and minus
- 02wc counts
Per GNU Coreutils Manual, head invocation: head prints the first part, 10 lines by default; -n num outputs the first num lines, and if num is prefixed with a minus, prints all but the last num lines. Both forms confirmed on coreutils 9.4.
head -n, and minus
head prints the first ten lines by default; -n 3 prints exactly three. Say -n rather than -3: the manual keeps the old spelling only for obsolete hosts.
The surprise is the minus: head -n -5 prints everything except the last five lines. Of twelve lines, seven survive, because 12 minus 5 is 7.
First N, or all but last N: head reads the number's sign as carefully as its value.
Print the first three of twelve sequence lines, then all but the last five, and check the second output is seven lines long.
Per GNU Coreutils Manual, wc invocation: the counts are printed in the order newlines, words, characters, bytes, maximum line length; a word is a nonempty sequence of non white space delimited by white space; if a file ends in a non-newline character its trailing partial line is not counted. Confirmed on coreutils 9.4.
wc counts
wc prints its counts in a fixed order: newlines, words, bytes, then the rest. On one two three with its newline that is 1, 3, 14, and the 14 is honest arithmetic: 13 characters plus the newline.
-l, -w, -c each pick a single count. But -l counts newline characters, not lines: text without a final newline scores 0, its last line invisible.
When a count looks one short, check the file's last byte before you blame the tool.
Run wc on a printf line and verify the byte count by hand, then run wc -l on the same text without its trailing newline.
Per GNU Coreutils Manual, comm invocation: before comm can be used, the input files must be sorted using the collating sequence specified by LC_COLLATE; sort with no options always outputs a file suitable as comm input; wrongly sorted inputs are diagnosed when an input is found to contain unpairable lines.
comm wants sorted
comm is a promise between three parties: both files must be sorted under LC_COLLATE, and comm walks them in lockstep on that assumption.
Break it and comm notices, in its own way: our run against an unsorted file2 printed comm: file 2 is not in sorted order and produced unusable output.
The manual's guarantee: sort with no options always makes comm-ready input. LC_ALL=C on both sides keeps the two files honest.
Run comm on one sorted and one unsorted file and read the diagnosis, then sort both under LC_ALL=C and run it again.
Per GNU Coreutils Manual nodes for cut, sort, uniq and head: cut -f selects fields, sort orders them, uniq -c counts adjacent repeats, sort -rn ranks numerically reversed, head keeps the first lines; the pipeline chains the four nodes' documented behavior. Confirmed on coreutils 9.4.
The top-ten pipeline
The one pipeline to know by heart: cut -d' ' -f1 | sort | uniq -c | sort -rn | head. Field out, order up, count, rank, cap.
Each stage is one page of this book doing its documented job; the count padding you see is uniq -c's own field width, which the final sort -rn reads as its key.
Top ten of anything countable: swap the cut, keep the shape.
Run the counting pipeline over five fake log lines, then over a real access log if you have one, and cap it at ten with head.