Verifying AI-generated code, one check per page
Twenty checks for developers who generate a large share of their code with AI and don't fully trust it: the trust numbers (96% don't trust it, 48% verify), reading the diff before approving, the prompt as the first verification artifact, the failing test written first, repro over prose, edge cases, the hallucinated method, dependency claims, security review with OWASP's LLM guidance, the self-review trap, small diffs over big dumps, checking generated tests, the review queue, logging, flags, canaries, rollback, the verification budget, reading as the skill that compounds, one check per page.
A diagram, the classic mistake, and one check to go run. That's a page.
Verifying AI-generated code, one check per page
Twenty checks for developers who generate a large share of their code with AI and don't fully trust it: the trust numbers (96% don't trust it, 48% verify), reading the diff before approving, the prompt as the first verification artifact, the failing test written first, repro over prose, edge cases, the hallucinated method, dependency claims, security review with OWASP's LLM guidance, the self-review trap, small diffs over big dumps, checking generated tests, the review queue, logging, flags, canaries, rollback, the verification budget, reading as the skill that compounds, one check per page.
Set in Space Grotesk, Inter and JetBrains Mono (SIL Open Font License).
Facts checked against the survey publishers' own pages: SonarSource's 2026 State of Code Developer Survey (96% don't fully trust AI code, 48% always verify, AI is 42% of committed code, 61% say it looks correct but isn't reliable, toil steady at 24% of the work week, reviewing AI code the number one AI-era skill at 47%), the Stack Overflow 2025 Developer Survey (84% using or planning to use AI, almost right but not quite the top frustration at 66%), LinearB's 2026 Software Engineering Benchmarks Report (8.1M+ pull requests: 4.6x/5.3x pickup delay, 17.6h vs 3.4h, 96 vs 26 median lines, 32.7% vs 84.4% merged in 30 days), and OWASP's Top 10 for LLM Applications (LLM01 prompt injection, LLM03 improper output handling). Practice claims via PubMed (16719566, 31311973). Figures are quoted as published, source and year on the page. Teaching conventions are named as conventions. None of these organisations is affiliated with or endorses this book.
General information only. Survey findings describe the publishers' results at the time of writing; methods and numbers change between editions. Not security or legal advice for your specific system.
Your purchase is for personal use only. You do not have redistribution rights: please do not share, resell, or republish this book or its pages.
© 2026 Steve Hodgkiss. All rights reserved. Personal use only; no redistribution rights.
Edition 1.0 · stevehodgkiss.net
Contents
SonarSource's 2026 State of Code Developer Survey of over 1,100 professional developers found that 96% do not fully trust that AI-generated code is functionally correct, only 48% always check AI-assisted code before committing, AI accounts for 42% of committed code (expected to reach 65% by 2027), 61% agree AI often produces code that looks correct but isn't reliable, and reviewing and validating AI code was ranked the number one AI-era skill (47%). The Stack Overflow 2025 Developer Survey found 84% of developers using or planning to use AI tools, and 'dealing with AI solutions that are almost right, but not quite' was the top frustration, cited by 66%. That a per-change checklist is the response is teaching convention, not a survey finding.
The trust numbers
Let's say you use an AI tool every day, don't fully trust it, and commit most of it anyway. You're the majority.
SonarSource 2026: 96% don't fully trust AI code, 48% always verify, AI is 42% of committed code. Stack Overflow 2025: 84% use or plan to use AI, top frustration at 66%: almost right, but not quite. That distrust-to-verification gap is where incidents live.
Nobody trusts it. Everybody ships it.
Count your last ten AI-assisted commits and mark which ones you actually verified. Keep the number.
Before you merge
The spec, the diff, the test, the run: the four checks that happen before the approve button.
- 01Read the diff before approving
- 02The hallucinated method
LinearB's 2026 Software Engineering Benchmarks Report, analysing 8.1 million+ pull requests from 4,800+ teams, found AI pull requests wait 4.6x longer before review, and that once picked up they are reviewed faster, with AI-assisted PRs clearing review in 3.2 hours against 4.2 for unassisted work. That reading the diff before approving is the core review convention is standard code review practice, not a measurement.
Read the diff before approving
Let's say the PR has 700 changed lines, it's 4:45pm, tests are green, and you approved on the write-up.
The diff is the contract. The description is marketing. Read every line in Files changed first. LinearB 2026: AI PRs wait 4.6x longer for review, then clear it faster. Green plus a clean write-up is not a review, and every merged line is yours.
Green tests are one check. Your eyes are the other.
Open your next AI PR's Files changed tab first, before the description, and read every line before approving.
Hallucinated or non-existent library methods are a documented failure mode of large language models producing code. That checking each unfamiliar call against official documentation is the countermeasure is review convention, not a measurement.
The hallucinated method
Let's say the model called .parseEverything(), it read so cleanly you almost didn't check, and the runtime said no.
Squiggle every method you don't recognise, then verify each against the official docs. The model optimises plausibility, so plausible APIs are what it makes. The prose defends the method until the interpreter refuses.
If you can't find it in the docs, it isn't real.
Pick one call in the last AI-generated file you can't vouch for and grep the official documentation for it. Today.
Rollback capability and rehearsed rollback procedures are standard release engineering practice. That rollback is the final safety check for changes that could not be fully verified pre-merge is teaching convention, not a measurement.
Rollback is the last check
Let's say the deploy is Friday 5pm, the migration is generated, and rollback is theoretically possible, practically untested.
Can't roll back means can't verify in production, only witness it. Rehearse the return path. A rehearsed rollback makes a bad change three bad minutes. The untested rollback turns out, mid-incident, to have been a plan, not a capability.
Undo is a feature. Test it like one.
In staging today, roll back your newest deploy and time it. If it takes longer than five minutes, fix that first.
That stacking individual checks into a per-change checklist is the method of this book is teaching convention. The supporting findings are real: SonarSource 2026 (96% distrust, 48% verify), Stack Overflow 2025 (almost right, top frustration at 66%), LinearB 2026 (4.6x pickup delay). No survey measures this checklist itself.
One check per page
Let's say you've read nineteen checks and the honest question is which of them you'll still be running next month.
This whole book is one sentence: one check per page, run per change. Pick the five that fit your codebase and pin them above your desk. A checklist you run beats a system you admire, and the surveys say the demand is permanent, 42% of committed code heading to 65%. The unrun checklist is the unread book, and the gap between distrust and verification stays exactly as wide.
One check per page. All of them per change.
Write your own five-check list from these pages and pin it above your desk. Run it on the next change.