sed and awk starters, one command per page
Thirty-two commands for developers who still edit text by hand: sed's cycle over the pattern space, s and its flags, addresses and ranges, -n with p, d and =, -i with a backup, -E for sane regexps, y and a, i and c, the hold space, N and D; then awk's pattern and action, records and fields, NR and NF, -F and OFS, BEGIN and END, arrays, and the string functions, one command per page.
A diagram, the classic trap, and one command to go try this week. That's a page.
sed and awk starters, one command per page
Thirty-two commands for developers who still edit text by hand: sed's cycle over the pattern space, s and its flags, addresses and ranges, -n with p, d and =, -i with a backup, -E for sane regexps, y and a, i and c, the hold space, N and D; then awk's pattern and action, records and fields, NR and NF, -F and OFS, BEGIN and END, arrays, and the string functions, one command per page.
Set in Space Grotesk, Inter and JetBrains Mono (SIL Open Font License).
sed behaviour checked against the GNU sed 4.10 manual (gnu.org, software/sed/manual, fetched and read during this build): Introduction; The s Command; Often-Used Commands; Addresses; Regular Expressions; Advanced sed: cycles and buffers; Sample Scripts. awk behaviour checked against the GNU gawk manual (gnu.org, software/gawk/manual, fetched and read during this build): Getting Started; Running awk; Reading Input Files; Printing Output; Patterns, Actions, and Variables; Arrays in awk; Functions. Concepts are named as the manuals name them. Teaching conventions are named as conventions. This book quotes no verbatim passages and is an independent guide not affiliated with or endorsed by the Free Software Foundation.
General information only. Not professional advice; verify against your own sed and awk versions and the current manuals.
Your purchase is for personal use only. You do not have redistribution rights: please do not share, resell, or republish this book or its pages.
© 2026 Steve Hodgkiss. All rights reserved. Personal use only; no redistribution rights.
Edition 1.0 · stevehodgkiss.net
Contents
The model
What sed is, the cycle that reads a line, runs the script, and prints, the pattern space where the work happens, and what awk adds: records split into fields, with a pattern and an action.
- 01The stream, not the file
- 02Records and fields
sed is a stream editor, used to perform basic text transformations on an input stream (a file or input from a pipeline), working in one pass over the input. Its ability to filter text in a pipeline particularly distinguishes it. Per the GNU sed manual, Introduction.
The stream, not the file
Let's say the log has 40,000 lines and every timestamp says 2025, and someone suggests find and replace in an editor.
sed is a stream editor: it performs basic text transformations on an input stream, a file or pipeline input, in one pass, says the manual. Text flows in, changed text flows out. No file is opened, nothing is loaded, the editor never starts.
The text flows through. sed changes it in flight.
Run cat on any text file piped through sed 's/2025/2026/' this week and watch the transformed stream scroll by. Nothing on disk changed.
In awk, the input is read in units called records and processed one record at a time; by default each record is one line. Each record is automatically split into chunks called fields, and the fields are separated by whitespace. Per the GNU gawk manual, Reading Input Files.
Records and fields
Let's say you need the third column of a CSV and you open a spreadsheet for it.
awk already did the split. Input is read as records, one line by default, and each record is automatically split into fields separated by whitespace, says the manual. The columns arrive pre-cut. The rest of awk is deciding what to do with column three.
Records in, fields split, work with the piece you came for.
Pipe any multi-column command output through awk '{ print $2 }' this week and get exactly the second column. No spreadsheet.
sed: substituting
The substitute command and everything that hangs off it: the g flag, captured groups with backslashes, the ampersand, any delimiter, and -E for regexps without the backslash fog.
- 01s: the substitute command
- 02The g flag
The s command (as in substitute) is probably the most important in sed. Its syntax is s/regexp/replacement/flags. It attempts to match the pattern space against the regexp; if the match is successful, that portion of the pattern space which was matched is replaced with replacement. If no address is specified, the command is performed on all lines, but it modifies only the first instance on each line; use the g modifier to affect every instance. Per the GNU sed manual, The s Command.
s: the substitute command
Let's say you ran s/old/new/ on a line with two olds, one changed, and you called sed buggy.
It did exactly what it says. s/regexp/replacement/ replaces the matched portion, and by default modifies only the first instance on each line, says the manual. The command runs on every line when there's no address, but inside each line, first match only.
First match per line. That's the contract.
On a scratch file this week, run sed 's/a/X/' on a line with three a's, count one X, then meet the g flag on the next page.
The g flag on the s command applies the replacement to all matches to the regexp, not just the first. The number flag replaces only the number-th match. For GNU sed, mixing g and a number ignores matches before the number-th, then replaces all matches from the number-th on. Per the GNU sed manual, The s Command.
The g flag
Let's say the CSV needs semicolons and only the first comma changed, three hundred times over.
The g flag applies the replacement to all matches to the regexp, not just the first, says the manual. And a number flag targets only the number-th match. First, every, or the second specifically: the flags are the whole conversation.
Default is first. g is all. A number is that one.
Fix every comma in a scratch CSV this week with sed 's/,/;/g'. Then try s/,/;/2 and see exactly the second comma change.
sed: choosing lines
Addresses pick which lines a command touches: line numbers, dollar, slash patterns, ranges with a comma, first~step, and the commands that consume them: -n with p, d, =, q, and -i editing in place.
- 01Addresses choose the lines
- 02The -n and p partnership
Addresses can be line numbers, matching only that line; dollar, matching the last line of input; or first~step, a GNU extension matching every step-th line starting with line first, so 1~2 selects odd-numbered lines. If no address is specified, the command is performed on all lines. Line numbers can also be used on the s command. Per the GNU sed manual, Addresses: selecting lines and Selecting lines by numbers.
Addresses choose the lines
Let's say the change is only supposed to touch line 144, and it touched all of them.
Commands take an address in front. A line number matches only that line, dollar matches the last, first~step matches every step-th line starting at first, and no address means all lines, says the manual. The address goes first: line, range, or pattern, then the command.
Address first, command second. All lines is the default you forgot.
Change only line 2 of a scratch file this week with sed '2s/old/new/', then the last line with sed '$s/old/new/'. Two lines, two commands.
By default sed prints all processed input. The -n option suppresses automatic printing, and the p command prints the pattern space; together they print specific lines. The command sed -n '45p' file prints only line 45. The p flag on the s command prints the new pattern space if the substitution was made. Per the GNU sed manual, Command-Line Options and The s Command.
The -n and p partnership
Let's say you want just line 45 and the terminal shows you all two thousand.
Two halves of one move. By default sed prints all processed input; -n suppresses automatic printing, and p prints the pattern space, says the manual. Mute everything, unmute what you meant. The p flag on s does the same for lines you actually changed.
-n mutes the room. p unmutes the one.
Print exactly line 45 of any big file this week: sed -n '45p'. Then print only changed lines: sed -n 's/old/new/p'.
sed operates by performing its cycle on each line of input: read into the pattern space, run the script, print unless -n. awk's basic function is to search for lines containing patterns and perform actions on them, each record by default one line, automatically split into fields. Programs in awk follow the pattern-action form; the default action is print. Per the GNU sed manual, How sed Works; per the GNU gawk manual, Getting Started.
The whole machine
Let's say you got this far and the two tools still blur together.
They're two small machines, and you've walked both. sed runs a cycle over each line, read, script, print; awk runs pattern-action rules over records already split into fields, say the manuals. Transform lines with sed, work columns with awk, chain them in one pipe when the job needs both.
A cycle and a loop. That's the whole machine.
Teach the two machines to one workmate this week from memory: sed's cycle, awk's pattern and action. Teaching it is the last rep.