systemd starters, one unit per page
Twenty-seven steps for developers who've run systemctl start but never owned the layer beneath: PID 1 and its control groups, unit files and their load path stack, the [Unit] [Service] [Install] sections, systemctl status, enable versus start, static and masked, reload versus daemon-reload, WantedBy and its symlink, ExecStart's absolute path, Type= simple exec forking oneshot, Restart=on-failure, User= and Environment=, drop-in overrides, targets as runlevels, Wants=, Requires=, After=, OnCalendar= and AccuracySec, monotonic timers, Persistent=true, the timer-service pair, the journal and journalctl -u -f -b, systemctl --user and linger, and socket activation, one unit per page.
A diagram, the classic trap, and one thing to go try this week. That's a page.
systemd starters, one unit per page
Twenty-seven steps for developers who've run systemctl start but never owned the layer beneath: PID 1 and its control groups, unit files and their load path stack, the [Unit] [Service] [Install] sections, systemctl status, enable versus start, static and masked, reload versus daemon-reload, WantedBy and its symlink, ExecStart's absolute path, Type= simple exec forking oneshot, Restart=on-failure, User= and Environment=, drop-in overrides, targets as runlevels, Wants=, Requires=, After=, OnCalendar= and AccuracySec, monotonic timers, Persistent=true, the timer-service pair, the journal and journalctl -u -f -b, systemctl --user and linger, and socket activation, one unit per page.
Set in Space Grotesk, Inter and JetBrains Mono (SIL Open Font License).
systemd behaviour checked against the official systemd documentation (systemd.io and the freedesktop.org man pages, latest stable), fetched and read during this build: systemd(1); bootup(7); systemd.unit(5); systemd.service(5); systemd.exec(5); systemd.timer(5); systemd.time(7); systemd.target(5); systemd.socket(5); systemd-journald.service(8); journalctl(1); systemctl(1). Concepts, directives and defaults are named as the documentation names them. Teaching conventions (one unit a day) are named as conventions. This book quotes no verbatim passages and is an independent guide not affiliated with or endorsed by the systemd project.
General information only. Not professional advice; verify against your own systemd version and the current documentation.
Your purchase is for personal use only. You do not have redistribution rights: please do not share, resell, or republish this book or its pages.
© 2026 Steve Hodgkiss. All rights reserved. Personal use only; no redistribution rights.
Edition 1.0 · stevehodgkiss.net
Contents
The model
What systemd is: PID 1 tracking everything in cgroups, units as one file format with a load path stack, the three sections, status, and the enable/start distinction that trips everyone once.
- 01PID 1, the manager
- 02Units, not scripts
- 03systemctl status
- 04start is not enable
systemd is a suite of basic building blocks for a Linux system. It provides a system and service manager that runs as PID 1 and starts the rest of the system. It keeps track of processes using Linux control groups, and implements transactional dependency-based service control logic. Per systemd.io, the official project page.
PID 1, the manager
Let's say someone says "the service manager" and you nod. Worth knowing what's at the bottom of it.
systemd runs as PID 1, the first process, and starts the rest of the system. Everything it starts it tracks in a control group, so it always knows which processes belong to which service, even after forks. Activation is a transaction: the units you asked for plus their dependencies, checked before anything starts.
PID 1 starts everything, and the cgroup list never lies.
Run systemctl status with no arguments this week. The first lines describe PID 1 itself, then the state of the whole system.
A unit configuration file whose name ends in .service encodes information about a process controlled and supervised by systemd. A unit configuration file whose name ends in .target encodes information about a group of units. Other unit types include .socket, .timer, .mount. The unit file search path includes /etc/systemd/system, /run/systemd/system, /usr/local/lib/systemd/system, /usr/lib/systemd/system; files found in directories earlier in the list override files with the same name lower in the list. Per systemd.unit(5).
Units, not scripts
Let's say you go looking for the init script and there isn't one. There's a small file ending in .service.
systemd manages units: a service is a process, a target is a group of units, a socket, a timer, a mount point, a device. Same file format, same sections, one suffix each. When systemd looks for a unit file it searches a fixed path stack: /etc/systemd/system first, then /run, then /usr/local/lib, then /usr/lib/systemd/system. Earlier wins, so your copy in /etc shadows the package's copy without touching it.
Name a thing, give it a suffix, put it in /etc. That's a unit.
Pick a service you use and run systemctl cat it this week. Note which directory its unit file was actually loaded from.
systemctl may be used to introspect and control the state of the systemd system and service manager. The status command shows runtime status information about one or more units followed by most recent log data from the journal. Per systemctl(1).
systemctl status
Let's say a service is misbehaving and you're about to read logs in three places. Start with one command.
systemctl status name gives you the state, the Main PID, the cgroup's processes, and the last journal lines, on one screen. With no arguments it describes PID 1 itself. It also prints the unit file's path and whether it's enabled.
status first. It answers six questions before you ask them.
Run systemctl status on a service you rely on this week. Find the Main PID line and the journal tail below it.
Enabling units should not be confused with starting (activating) units, as done by the start command. Enabling and starting units is orthogonal: units may be enabled without being started and started without being enabled. Enabling simply hooks the unit into various suggested places, for example so that the unit is automatically started on boot. The enable command creates a set of symlinks as encoded in the [Install] sections of the unit files. Per systemctl(1).
start is not enable
Let's say you ran systemctl start, rebooted, and the service wasn't running. Nothing broke. You asked for the wrong thing.
start acts now: it activates the unit in this boot. enable acts later: it creates symlinks as the [Install] section directs, hooking the unit into boot. They're orthogonal: a unit can be enabled and not running, running and not enabled, both, or neither. Want both at once? That's what --now is for.
start is now, enable is at boot. Say which one you mean.
Run systemctl is-enabled on two services this week, then systemctl is-active on the same two. Compare the answers.
ExecStart= configures the commands that are executed when the service is started. Unless Type=oneshot, exactly one command must be given. If the first argument is an absolute path, it is directly executed; if it is a simple filename, it is searched in the search path. Lines for this setting are split into arguments; the first argument must be an absolute path to the executable. Per systemd.service(5).
ExecStart= the command
Let's say your unit fails instantly and the journal says status 203. The path was the problem.
ExecStart= is the command. One command unless it's a oneshot, and the first argument must be an absolute path to the executable, /usr/bin/python3, not python3. systemd runs it directly, without a shell, so no pipes, no globs, no dollar variables. You get those by wrapping in sh -c, or better, by using the right directive.
Absolute path, no shell. Every other surprise flows from that.
Take one unit you run and check its ExecStart this week: absolute path? Any shell syntax that shouldn't be there?
Wants= is a list of units that will be started alongside this unit when this unit is activated. This is the recommended way to hook the start-up of one unit to the start-up of another unit. If unit foo.service pulls in unit bar.service as configured with Wants= and no ordering is configured with After= or Before=, then both units will be started simultaneously and without any delay between them. Requirement dependencies do not influence the order in which services are started or stopped. Per systemd.unit(5).
Wants= pulls it in
Let's say your app unit Wants= the database and still starts before it's ready. Wants was never about order.
Wants= pulls the listed units into the same start transaction, and the docs call it the recommended hook. But if no ordering is set, both start simultaneously, no delay. Requirement and ordering are orthogonal knobs: Wants says together, After says in that order. The pattern is both.
Wants decides who comes. After decides who waits.
Pick a unit with dependencies and run systemctl list-dependencies it this week. Spot one Wants= that has no After=.
The -u, --unit=UNIT option of journalctl shows messages for the specified systemd unit, such as a service unit. The -f, --follow option shows only the most recent journal entries and continuously prints new entries as they are appended. The -b, --boot option shows messages from a specific boot; if the boot ID is omitted, logs for the current boot are shown. Per journalctl(1).
journalctl -u -f -b
Let's say you read the journal by scrolling. Three flags replace the scrolling.
-u name filters to one unit's messages, the first flag of nearly every session. -f follows live, tail-style, newest entries as they land. -b slices to one boot, current by default, so yesterday's noise is gone. They stack: journalctl -u app -b -f is this boot, this service, live. Add -e to jump to the end instead of following.
One unit, one boot, live. Three flags, one habit.
Run journalctl -u followed by a service you run, with -b and -f, this week. Watch a restart land in real time.