Teaching the Reviewer

“Men learn while they teach.” — Seneca

I was building a code-review tool. Nothing fancy — read the diff, flag what’s wrong, the job I do by hand a dozen times a week. Partway in I stopped on a question bigger than the tool. Could it review as well as I do, maybe better — and could teaching be how it got there?

The bet was that those are the same move. If the tool could explain a finding well enough that the reader got better, then I’d have handed it real judgment, not a heuristic. Teaching wasn’t a feature to bolt on later. It was the test of whether the judgment had transferred at all.

“Can AI review code” was never the interesting question. Everyone asks it. Mine: which slice of my review is mine to freeze, and how much of me won’t go.

Bottom line up front

• Split your review judgment in two. The part you can name — the rules, the patterns, the “we don’t do that here” — comes out of you, but not all the same way. The slice that mechanises freezes into a deterministic check, and the tool beats you on it: tireless, firing on every rule and every diff, catching what your eye skims. The rest of the nameable part gets captured as explicit rules the model applies — better than tacit, but the model can still skim those. The part you can’t name stays yours — is this the right change at all, is the shape wrong in a way you never wrote down.
• Build the reviewer to teach, not just to flag. You can’t freeze a rule you can’t say out loud — and by freeze I mean the strong form — pinning the judgment to an explicit rule a deterministic check runs the same way every time. (A rule you can name but can’t mechanise is still worth extracting — the model applies it explicitly — but that’s captured, not frozen: the model can still skim it.) Forcing the tool to explain each finding is what drags the tacit judgment into the open where it can be caught. The explanation is the extraction.
• It pays the teacher back. Naming your own judgment sharpens it; you understand your review better once you’ve had to teach it. (I expect it pays the reviewee too, and lifts the team’s floor. That part I’m claiming, not proving.)
• Don’t trust “as good as me” flat across the board. True on the findable axis, false on the essence. The tool is the instrument that shows you where your own line sits today.

The part that freezes

Brooks split software into the accident and the essence. The accident is what’s hard because of how we happen to build — nameable, attackable, the stuff better tools chip away. The essence is the conceptual core, hard no matter what. (Brooks, “No Silver Bullet,” IFIP 1986; IEEE Computer 1987.) Mapping your review judgment onto that split is mine, not his. It holds anyway.

The accident is everything you could put on a checklist. Null not handled. This should be pure. We don’t reach across that boundary. Error swallowed. You know these on sight, and still don’t always catch them. You skim the boring diff. You’re tired on a Friday. You’ve reviewed the same mistake forty times and your eye slides past the forty-first.

A tool doesn’t slide past. It does that in two different ways, though, worth keeping apart. The mechanised checks — the deterministic ones — fire every rule, every time, at 200 lines or 2,000; completeness there is guaranteed by construction. The LLM layer applies the rules that have to be read rather than matched, and it does that without the fatigue that makes me skim the boring diff. On the accident they don’t just match me — they beat me, raising the floor to my best nameable self and holding it there. That’s what code inspections were always reaching for: Fagan proceduralised review into roles, checklists, and exit criteria in 1976 — the nameable part made into a process that runs whether or not any one reviewer is on form. (Fagan, 1976 — prior art for proceduralised review, not proof that judgment extracts.) The tool is that, minus the meeting.

As I put it in an internal note of mine: a skill is judgment you froze — and the frozen kind fires every time, your blind spots bundled in.

The part that doesn’t

Then the residue. Is this the right change at all. Should this feature exist. This is clean and still the wrong shape, and I can’t name the rule it breaks because it doesn’t break one. It just reads wrong.

That part resists naming, and the reason is old. Polanyi: we can know more than we can tell. (Polanyi, The Tacit Dimension, 1966 — framing, not data.) Underneath sits chunking. Chase and Simon found chess experts reconstruct a real position far better than novices and lose almost all of that edge on a random board — they register meaningful structure at a glance. (Chase & Simon, 1973 — a narrow result; reading it as “they see before they can say,” the words lagging the recognition, is my extension.) Expert review has the same shape. You catch the wrong-shape in a glance and the words come later, if they come.

The machine-learning version rhymes, and it’s only an analogy. Distil a big model into a small one and there are no rules to read off. You train the student to match the teacher’s outputs, and it recovers the behaviour without ever stating what it learned — some judgment is a function you can only imitate, not a rule you can read. (Hinton, Vinyals & Dean, 2015.) That’s imitation, not freezing: the behaviour transfers, but nothing becomes an explicit rule anyone can state or check, so the teacher never owns the judgment either. You can’t freeze what you can’t articulate — you can only imitate it.

One caution on the word residue. What I couldn’t name marks where the tool’s coverage stops today, not a permanent human-only preserve. The line moves.

Teaching is how you freeze the part that freezes

Here’s the move I didn’t expect.

The tool kept waving through a pattern that should have stopped it. An array index used as a list key. A .slice() that quietly broke a memo. Each time it reasoned the same way: it works, the data’s static, ship it. I pushed back once, twice — and then heard what I was doing. I was inventing the safety the code never stated, supplying on its behalf a guarantee it never made.

So I said the rule out loud, for the first time. Dangerous-looking code is only safe if something visible says why — a comment, or a convention the reader already knows: as const, readonly, a memo wrapper. No marker, no pass. The reviewer doesn’t get to supply the justification the author left out.

I’d been running that test for years and had never written it down. The tool needing a decision is what made me say it. That is the whole mechanism in one moment: to get the nameable judgment out of me and into the tool, I had to make the tool explain itself, and explaining forced me to name the rule. The explanation is the extraction.

Many of the rules I “knew” I’d never actually stated. Building a reviewer that explains — this is wrong because X, here’s the shape it should have — is what dragged them into the open. Building the tool mapped my judgment back to me.

And this is what review was quietly doing all along. Bacchelli and Bird asked what teams expect from review versus what they get. It turns up several things — defects, alternatives, awareness of the change — but its realised value leans towards understanding and knowledge transfer more than teams expect going in. That expectation-versus-outcome gap is their finding; reading it as teaching-primary is mine. (Bacchelli & Bird, 2013.)

Teaching pays the teacher first, and that leg is solid. People who study in order to teach, or explain to someone they’re tutoring, understand the material better than people studying for a test. (The protégé effect — Chase et al., 2009; Bargh & Schul, 1980. Scope: it’s about the effort a teaching frame induces, not proof that rules extracted.) Naming my judgment to build the tool made the judgment clearer, and I reviewed better afterward.

Where the wobble goes

“Nameable” isn’t one thing either. A finding is three acts and they come apart: noticing — something here is off; deciding — off enough to raise, and this is what it does; naming — the label it gets filed under. Noticing is the part that mechanises, because a detector fires on the structure every time. Deciding is the part everyone assumes is the kernel, the judgment call you’d defend out loud. Naming is the one nobody thinks about, and it is the most enumerable of the three: a label is a closed list, if you bother to write the list. I hadn’t. Mine was free text.

So I pinned the first act and watched the other two. I fed a detector into the reviewer, which fixes noticing by construction, and ran it against one unchanged defect eleven times. The disposition held — eleven out of eleven, same call. The name didn’t. Five distinct slugs came back for that one defect (four, if you fold the two per-variant spellings together): branch-chain-dispatch-candidate, hardcoded-per-variant-dispatch, non-data-driven-branching, repeated-branch-structure, per-variant-branch-dispatch. Every one of them describes the thing correctly. No two of them are the same string, which means no two of them aggregate, dedupe, or suppress. The wobble hadn’t gone anywhere. It had moved to the one element I left un-enumerated. That is one defect at one site, not a study, but it points somewhere specific: you can’t freeze the perception, you can freeze the combination, and you can freeze the label too — if you write the list.

The honest line

“As good or better than me” is true, and it’s a trap if you don’t say where.

On the findable — the accident, the nameable — better. The mechanised half is complete by construction; the model half is tireless and unbored where I’d skim. On the essence — intent, what to build, the wrong-shape — no. That verdict is bounded lived experience, not a measured result: it’s what I saw building and running the thing, with no eval and no n behind it. The formal evidence is thin either way. One 2025 study of LLM reviewers had them classifying code correctness around 68% of the time and handing worse suggestions to as much as a quarter of the correct code they saw: useful, not trustworthy alone. (Cihan et al., 2025 — one narrow study, not a settled number.)

The real question isn’t whether the tool beats me. It’s how much of my review reduces to rules I can name — freeze the mechanisable ones, capture the rest — and how much is residue I can’t. There’s no clean number. But building the thing measures the line. Every rule I managed to name went into the tool. Everything left over — the part I kept reaching for and couldn’t say — is where my coverage ends for now, and that part is still mine.

One more line to draw once a rule is frozen into the tool: what gets to block. Only the mechanised checks can — a deterministic rule is checkable, its pass or fail isn’t a matter of opinion, and a red build on it is honest. The LLM’s judgment can’t, even when it’s applying a rule I named, because its output is a probable reading, not a checkable fact. Make the model’s opinion a required green and — this is my inference, not something I’ve measured — you don’t raise the floor, you lower the ceiling: a mandatory check trains the next human to defer to it, and a reviewer everyone defers to has replaced the reading instead of guarding it. Block the mechanised checks; keep the judgment advisory.

Give away the nameable. Keep the reading. Teaching is how you tell which is which.

---

How sure I am: the accident-vs-essence split, tacit knowledge and chunking, knowledge distillation, code review’s realised value, the protégé effect, and the (young) LLM-review literature are prior art, cited; mapping that split onto review judgment, and the account of building a reviewer that had to explain itself to drag the rule out of me, are my own framing and experience; that the same teaching also pays the reviewee and lifts the team’s floor is a wager — I believe it, and I’ve marked it in the essay as belief, not evidence. The naming-wobble result is mine and measured, but it is eleven runs against one unchanged defect at one site: enough to show the wobble relocated, nowhere near enough to size how often it does. If this is wrong somewhere, it’s there or in the wager.

Sources

• Frederick Brooks, No Silver Bullet — Essence and Accident in Software Engineering (IFIP 1986; IEEE Computer 1987) — the accident-vs-essence split the essay maps onto review judgment.
• Michael Fagan, Design and Code Inspections to Reduce Errors in Program Development (IBM Systems Journal, 1976) — review made an explicit process of roles, checklists, and exit criteria: the nameable part, made into a process.
• Michael Polanyi, The Tacit Dimension (1966) — “we can know more than we can tell”; the residue that resists naming.
• Chase & Simon, Perception in Chess (Cognitive Psychology, 1973) — experts reconstruct real board positions far better than random ones; meaningful structure seen at a glance.
• Hinton, Vinyals & Dean, Distilling the Knowledge in a Neural Network (2015) — a student model recovers a teacher’s behaviour from its outputs, with no rules to read off; arXiv:1503.02531.
• Bacchelli & Bird, Expectations, Outcomes, and Challenges of Modern Code Review (ICSE, 2013) — review’s realised value leans to understanding and knowledge transfer more than teams expect going in.
• Chase, Chin, Oppezzo & Schwartz, Teachable Agents and the Protégé Effect (J. Science Education & Technology, 2009); Bargh & Schul, On the Cognitive Benefits of Teaching (J. Educational Psychology, 1980) — preparing to teach sharpens the teacher.
• Cihan et al. (2025) — one narrow LLM-reviewer study: ~68% correctness classification, worse suggestions on a share of correct code; illustrative only, the literature is young; arXiv:2505.20206.

---

Related: Give Away the Writing, Keep the Reading; The Productivity Mirage (keep the rep, protect the practice); The Nit Was Never Optional (once the tool holds the rule, the “optional” hedge on it stops making sense).