A Description Can't Replace the Code

“This is not a pipe.” — René Magritte, The Treachery of Images (1929)

For code that has to be right, reading it is irreducible — you can’t hand the intent-check to the AI. That’s the claim in Give Away the Writing, Keep the Reading. The obvious objection: why read the code when the model can describe what it did, and a second model can check the description? Here’s why that pipeline can’t close.

Bottom line up front

• Code that has to be right? Read it — don’t accept the AI’s description. The description drops the omissions it didn’t notice, and that’s where the bugs live.
• When you verify, point the check at code, not prose. Ask “show me the lines that handle X, Y, Z,” not “did you handle the edge cases?” The first you can check; the second is a summary that hides the gap. It only catches the concerns you can name — the case you didn’t think to ask about, you still catch by reading.
• A description doesn’t tell you whether it’s a promise. “Refactor” claims the behaviour is unchanged, and can be wrong about it; “move” and “clean up” say what the author did and make no claim about what the code still does. Same slot in the sentence, different obligations — the grammar won’t flag which one you got.
• A second model helps on what’s findable, not on intent. “Another model checked it” doesn’t clear your reading.
• Throwaway? Skip all of it. That’s for the scratch you’ll delete.

The compression is the problem

A description is a compression. It’s cheaper than the code because it drops the omissions it doesn’t know it made — the case not handled, the check not added. That’s where the security and performance bugs live, and a description can’t surface a gap it never noticed. Linguists have a name for the general shape of it: form versus meaning (Bender & Koller) — a description is form, and form can’t carry the meaning it never held.

The AI told me a batch of jobs were running our newest model. I read the config. Every one was still pinned to an older one. It had described the setup it assumed, not the one that was there. Nothing in the description read as uncertain, because the description didn’t know.

“It’s secure” is a verdict you can’t check. “Here are the measures I took” is a list of the known-knowns — it can name a gap it noticed, but nothing in it guarantees the ones it didn’t. There’s no informal description both short enough to beat reading and complete enough to catch the omissions. The only exception is formal — a spec or a proof — and that’s a precise artifact you still have to read. Short of that, a description complete enough to fully trust is the code.

There’s a subtler version of the gap, and I only found it by trying to automate around it. My reviewer reads the PR description to set a prior: if the author says the change preserves behaviour, then an apparent behaviour change is more interesting, not less. Which forces you to decide what counts as a claim of behaviour-preservation — and the words authors actually reach for split into two kinds that look identical on the page. Some get read as promises about behaviour — refactor and no functional change strongest, and in the PR titles I test against, rename, lint and dependency bump usually too. Write one of those and you’ll be taken to have claimed the observable behaviour survives, whether you meant to or not. The rest describe an action: extract, inline, move, reorganise, clean up, tidy, format. They say what the author did to the code and make no claim about what the code still does.

I mean the colloquial PR verbs there, not Fowler’s named refactorings — and the collision is the whole problem. Extract Function and Move Function are formal operations, defined since Opdyke as restructurings that preserve observable behaviour. Those words got borrowed out of a vocabulary where they carry a guarantee and into PR titles where they carry none, and they kept the respectable sound on the way over.

Treat the second kind as the first and every ordinary feature PR reads as a refactor. “Move retry logic into the saga so it runs on mount.” “Inline validation on the form.” “Format dates consistently across the app.” Each borrows a word from the tidy-up vocabulary; each describes new behaviour. Those three are pinned in my test suite, because the mistake is easy and it fails in the expensive direction — a false this is only a refactor makes every genuine feature diff look suspicious.

The point isn’t my keyword list. It’s that a description doesn’t announce which kind it is. Refactor and move occupy the same slot in the same sentence and do entirely different work: one is a claim that can turn out false, the other can’t be false because it never claimed anything. The reader supplies the difference, and a reader who supplies the wrong one has been misled by a description that never lied to them.

One consequence is worth taking even if you automate none of this. I had only ever used the description defensively — if the author already flagged it, don’t flag it back — so it could quiet a finding and never raise one. But a promise is checkable, and that cuts both ways: if the description says no functional change and the diff moves a comparison boundary, the contradiction is itself the finding. The author’s claim is evidence, and evidence can incriminate. The failure left over is an author who writes “refactor” on a change that isn’t one — which is precisely the case you wanted caught.

Nor can a test settle it alone

Deciding what “correct” means is itself an assumption you have to ground — software’s old oracle problem — so a test is only as good as the expectation baked into it. A green suite can still miss the bug at the seams: each unit right, the contract between them wrong. And every layer built from the same prompt is one more artifact that can be confidently wrong, each downstream of the same guess at your intent. A second model helps least where you need it most. On intent you never stated, it’s as blind as the first — neither can see what you didn’t say — and their errors correlate across families more than you’d hope (Kim et al. measured that on benchmarks; reading it onto intent is mine). The second opinion is worth less than it looks.

You can’t just move up a layer

The strongest objection is that this is nostalgia. We climbed off assembly and never looked back; we’ll climb off code the same way and program in plain English. Karpathy planted the flag in 2023: the hottest new programming language is English.

It’s a real argument, and it’s wrong about the one property that matters: a language defines what its source means. A compiler implements a language specification — a standard that fixes what conforming source means, independent of any one compiler. (Runtime behaviour still turns on inputs and environment, and a spec leaves corners undefined — but the mapping from source to meaning is pinned.) Assembly to C to Python, you gave up detail and kept that: a defined mapping from source to meaning.

English-to-code through a model gives it up. The fix isn’t to complain the model is random. You can’t even buy determinism with a sampler setting — greedy decoding is theoretically deterministic and production endpoints still aren’t, because batch size shifts with load. But that was never the point. The point is there’s no spec. A prompt isn’t a complete, checkable statement of what the code must do. English can’t say what you didn’t think to say, and nothing guarantees a prompt maps to the same code across a rephrasing or a model version. Determinism was never what the compiler gave you — a specification was. So this isn’t the next rung on the ladder. It’s sawing off the rung that made the ladder worth trusting.

Someone is trying to build the rung back. AWS’s Kiro runs an optional automated-reasoning pass over your written requirements, “before any design or code is generated.” Their engineering write-up is specific about the mechanism: a language model encodes the requirements into formal logic, expressed in SMT-lib, and an SMT solver reports the contradictions and the gaps, along with ambiguity and criteria you couldn’t test. That’s a different kind of check from the one this post rules out. A contradiction between two things you wrote is findable, and a solver settles it by proof instead of by a second opinion. I’m describing what their engineers say the pass does — they publish no evaluation of it, and I haven’t run it.

It is a specification, and I should concede that before saying what it doesn’t reach. An SMT-lib model a solver can check is the formal artifact this post carved out earlier. But it isn’t the kind of specification the ladder was built on. A language spec pins what source means; this one formalises a language model’s reading of what your requirements say, and their account describes no check that an implementation conforms to it. So the missing rung is still missing. The guess didn’t get removed. It got a checkable artifact next to it, one step earlier. Their own engineers are blunt about the limit: “The hard part of specification engineering is that there is no oracle for specification correctness other than the user.” They’re explicit that the technique can’t pick which of several plausible formal readings of a requirement is the one you meant.

Completeness, in their system, is defined over what the model knows: “there must be no state where the requirements don’t prescribe what the system should do.” Those symbols come from your requirements plus background assertions — implicit domain knowledge the tool supplies, which their engineers call the “dark matter” of requirements. So the boundary isn’t only what you typed; it’s that plus what the tool guessed you meant by it. It’s still a boundary. Introduce no symbol for timezones, and have the tool not think of them either, and no solver will tell you that you forgot timezones — not for want of a better solver. You also aren’t asked to read the model that draws the line: findings come back as “a simple two-option question,” no formal-methods background needed. That’s the point of the product, and it’s also the tell. A spec you answer questions about instead of reading is a description you trust. The strongest live attempt narrows my claim rather than breaking it.

For a fluent reader it isn’t even a burden. Well-written, convention-following code is as clear as English, often clearer, because it’s precise where English is vague. That fluency is trainable — the subject of Thinking at the Speed of Code.

Read the raw

Code stays the layer you can pin down and verify. A description is a claim; the code is the raw. The conversation with the AI isn’t the artifact. Throwaway, none of this matters — skip it. When it has to be right, you read the raw.

And if you want to record the intent so your reading has an anchor — an ADR, a line in the PR, a test — remember the record is a description too, and inherits the same trap: prose about the code drifts silently the moment the code moves. The only intent worth trusting is the kind the code can contradict — a test that fails, a type that won’t compile. Write it where a wrong version can’t stay green; everything else is a claim you still have to read the raw to check.

That’s a ranking, not a binary. A name carries intent and enforces nothing — the concept moves, the label stays, nothing complains. A shape — which arguments a function takes, which fields exist, what’s optional — makes a whole class of wrong versions awkward to write. A type won’t compile. A test fails out loud. One intent, four rungs, ordered by how hard it is for a wrong version to stay green: put it on the highest rung it will sit on, and don’t leave it in prose when it would have compiled. I work the bottom of that ladder in How to Think About Code (“Say What You Mean” — couple the rule to the code rather than restating it beside the code) and the top in “Tests as Promises” — write the guarantee down as the thing that breaks.

One caveat, because it cuts against where that lands. Code tells you what the system does — the indicative mood, in Zave and Jackson’s split — and never what it should do, which is the optative. Approving a change makes the code the current baseline; it doesn’t make it the authority on intent, because those are two different kinds of statement and no amount of reading one yields the other. So read the raw is a rule about what you can verify, not a claim that the code settles what was owed. When the two disagree, the raw tells you what shipped and the agreed intent tells you what should have — and the second is what you argue from, even though only the first can be checked.

---

Sources

• Barr, Harman, McMinn, Shahbaz & Yoo, The Oracle Problem in Software Testing: A Survey, IEEE TSE 41(5) (2015); Weyuker, On Testing Non-Testable Programs, Computer Journal 25(4) (1982).
• Perry & Evangelist, An Empirical Study of Software Interface Faults (1985; updated HICSS 1987) — faults cluster at the seams.
• Kim et al., Correlated Errors in Large Language Models, arXiv:2506.07962 (ICML 2025) — errors correlate across model families; reading that onto unstated intent is mine.
• Thinking Machines, Defeating Nondeterminism in LLM Inference (10 September 2025) — greedy sampling is “theoretically deterministic,” and yet “LLM APIs are still not deterministic in practice”: from 1,000 completions at temperature 0 they got 80 unique ones, the most common occurring 78 times. The cause is infrastructural rather than statistical — “the primary reason nearly all LLM inference endpoints are nondeterministic is that the load (and thus batch-size) nondeterministically varies,” over a forward pass that lacks “batch invariance.” Added after publication, to correct one sentence: a sampler setting does not buy you determinism. Their setup, not mine.
• Bender & Koller, Climbing towards NLU: On Meaning, Form, and Understanding, ACL (2020) — a system trained on form gets form, not meaning; the gap a description leaves is the same one.
• Martin Fowler, Refactoring (1999; 2nd ed. 2018), after William Opdyke’s 1992 thesis — refactoring is defined as restructuring that preserves observable behaviour, and Extract Function / Inline Function / Move Function are named operations in that catalogue. Cited for the collision: the formal vocabulary guarantees behaviour-preservation, the colloquial PR-title use of the same verbs does not.
• Zave & Jackson, Four Dark Corners of Requirements Engineering, ACM TOSEM 6(1) (1997) — the indicative/optative split: statements about what the system does are a different kind from statements about what it should do.
• Karpathy (2023) — “the hottest new programming language is English” (the foil).
• Dobe, Arora, Zetzsche, Labai & Delmas, Requirements analysis: catching requirement bugs before they become code, Kiro (AWS) engineering blog, 12 May 2026 — an optional automated-reasoning pass over written requirements, before any design or code is generated: a language model encodes them into SMT-lib, an SMT solver reports contradictions and unspecified states, and each finding surfaces as a two-option question. Cited for the mechanism and for its stated limit — “there is no oracle for specification correctness other than the user.” They publish no evaluation of their own system; the numbers in the post are borrowed from third parties and are about code, not requirements. Read from their engineering post; I haven’t used the tool.
• How to Think About Code (my own) — “Say What You Mean” (couple the rule to the code) and “Tests as Promises” (name the guarantee). The constructive counterpart to this post’s negative claim.

How sure I am: The oracle problem, the seam faults, and the correlated-error result are prior art (cited); reading correlated errors onto unstated intent is mine — a wager, flagged in the text. The indicative/optative split is prior art (Zave & Jackson, read to primary); the consequence I draw from it — that approving a change makes the code the baseline but never the authority on intent — is mine, and it’s a deliberate complication of this post’s own closing rule rather than a retraction of it. The name→shape→type→test ranking is my own framing of an ordering by checkability — it’s how the book teaches it, not something I’ve measured. The core impossibility — no informal description both short enough to beat reading and complete enough to catch the omissions — is an argument I reason to, not one I’ve proven; I’d retract it for a counterexample. Added after publication: the nearest thing to a counterexample I’ve found is a shipped product — AWS’s Kiro, which its vendor says checks written requirements with an SMT solver before generating anything — and it narrows the claim rather than breaking it. It does produce a formal specification, which is a real concession: what their account doesn’t describe is a check of code against one. I read the vendor’s own account of the mechanism, not the coverage of it, and their limits are stated by their own engineers. They publish no evaluation of the system, and I have not used it: that passage rests on their engineering post, not on measurement or on my hands on it. The promise-versus-action split is lived: it’s a working classifier in my reviewer, with those three sentences locked as regression tests because the naive version misread them. It’s a distinction that earns its keep in one codebase’s PR titles, not a claim about English. Also added after publication: the sentence about temperature was wrong. It said zero gets you near enough to deterministic; it doesn’t, and the corrected line now carries the reason and the citation. The correction cuts my way — it makes the sampler an even worse answer to the problem — which is exactly why it needed flagging rather than quietly fixing.

Related: The Other Intent (which part of a recorded intent keeps applying after merge, and why it has to be encoded rather than described); Augmented Reading (what to do when the code is too big to read and the description still can’t stand in for it); A Pool of Models Narrows the Overlap (the correlated-error result, measured on the review side); The Raw Is the Trust Boundary (the code is the raw; the description is a render of it).

Reading series, part 2 of 4 (start with Give Away the Writing, Keep the Reading). Next → Your Codebase Is the Prompt.