Severity Is the Budget Key
updated 10 min
Confidence a procedure derived from three posts of my own and never run against a real budget
I told an agent that fidelity came first, cost second, speed third. It optimised for cost anyway, and the post about why is about the measurement asymmetry that let it. Later I decided the order itself was the mistake, and said so: a strict order asserts all recall is worth the same, and it isn’t.
That was the easy half. A strict order is at least a procedure — wrong, but runnable. I removed it and put nothing in its place.
Here is the replacement.
Bottom line up front
- A strict priority order is a procedure. “Fidelity matters more here than there” is not. Saying the order is wrong leaves you with a judgment you have to make every run, which in practice means you make it once, badly, and forget.
- Verification is the bill, not discovery. Mechanising the noticing on a fourteen-line change cut fresh tokens roughly sevenfold and did not cut the cost, because what was left was the one judgment call.
- The class of a finding already encodes how much verification it needs. A nit is cheap to fix and you still pay to check the fix. A cross-module contract bug is the opposite shape.
- So budget per class, not per run. Severity is the key you price against, because severity is what determines the expensive part.
- A class whose check costs more than the class returns is one you drop, visibly. That is not a reversal of the post arguing against suppressing nits — it is the precondition that post attached, failing.
A strict order is wrong and still does more work than no order
The failure in the original rule was specific. I had pinned fidelity first, cost second, speed third, and shipped it as advice. An agent took it exactly as written. Because the lower term never trades against the higher one, cost was unbudgeted — every cheap lever stayed optional forever, and that was correct under the rule I gave it.
The fix I published was to stop asserting a total order. That diagnosis holds. But look at what it leaves behind. A strict order tells you what to do at every decision point without further thought. “All recall is not worth the same” tells you that you now have a judgment to make, and says nothing about how to make it. Replace a bad procedure with a true observation and you have fewer errors of the kind the procedure caused, and a new error: nobody prices anything, because pricing is now a matter of taste and nobody wants to own it.
What I needed was another procedure. Not a total order — a key.
Verification is the bill
The number that makes this tractable comes from mechanising the noticing. I expected to find savings and found out where the money was instead. Fresh model tokens on a fourteen-line rename fell roughly sevenfold. The bill did not fall. It relocated. Noticing went nearly free and the cost that was left was the one judgment call, which was most of the roughly four dollars that fourteen-line change still ran.
Say that plainly: finding a candidate is cheap and deciding whether it is real is expensive. If that is true, then the thing you are actually buying, every run, is verification. Discovery is the part you can mechanise toward zero. Verification is the part you cannot, because it is the part a model has to think about.
That reframes the budget question. You are not buying review. You are buying a number of verifications, at a price that varies by what is being verified.
The class already prices the verification
Which brings in the second published piece. The post on why the nit calculus inverts is about generation getting cheap: when the fix is instant, the apology a nit used to carry stops making sense, because a nit you would have waved through is just a fix you have not made yet. But it carries a clause that matters more here than it did there — you still pay to check the fix is right.
That clause is the pricing function. A nit’s verification is shallow and local: read the replaced line, confirm it says what it should. A removed guard is deeper: you have to establish what the guard was for, which is a question about intent and often about code the diff does not touch. A cross-module contract break is deeper again.
So the class of a finding is not only a statement about how much the reader should care. It is a statement about how much the check costs. Severity and verification burden are two readings of the same property, which is why severity works as a budget key and an arbitrary label would not.
Put the two together and the procedure falls out. Verification is the bill. Class sets the verification. Therefore class prices the check, and you budget per class rather than once for the whole run.
What that looks like in practice
You stop asking “can we afford this review” and start asking “can we afford this class of finding, in this codebase, at this rate.”
Those are different questions with different answers, and the second one is answerable. A class has a rate — how often it fires here. It has a verification cost — what a correct check on it takes. It has a value — what catching one is worth in this codebase. Multiply and compare. The axis you privilege is yours to pick, but once you have picked it, the arithmetic is arithmetic.
The honest consequence is that the answer is local. The same class can clear its cost in one repository and fail in another, because the rate and the value both move. There is no global table of what a severity class is worth.
The floor, and why it is not a reversal
If a class can fail to clear its cost, then some class, somewhere, gets dropped. That sounds like it contradicts a post I have already shipped, so I want to be exact about why it does not.
The nit post argues against suppressing nits. It argues that conditionally — the case holds when applying-and-checking a nit is cheap. That is a precondition, stated at the time, and dropping a class is precisely that precondition failing. Not an exception to the rule. The rule, read where its own condition does not hold.
There is also already a delete route in the corpus. The post on rule sets as an attention problem has a third option beside mechanising and structuring: delete the rule. But it justifies deletion by capability — the model got good enough not to need the instruction, which is Anthropic’s result, not mine. The budget justification is the new thing here. A rule can be worth deleting because the model no longer needs it, and separately because checking what it finds costs more than finding it returns. Those are different arguments and only the first one was written down.
The plainest version of the floor is the one I would say out loud: if nits are costing more than we can pay, then they are no good. Not wrong. Not unimportant. No good — as in, not worth the money, here, this quarter.
What someone else’s receipt shows, and what it doesn’t
GitHub published something in September that bears on this, and I want to be careful about how much.
Erik Kristensen and Napalys Klicius wrote up how they made AI coding more cost-efficient without giving up task quality. The part worth having is a negative result they chose not to ship: the same tightened file-tool instructions helped one product surface and increased cost on another running the same harness, so they did not ship it. Their closing lesson is that evidence is local to the workload.
That is the floor as a shipped decision rather than an argument. Someone killed a change that was good on the metric they cared about elsewhere, because in this workload it cost more than it returned.
What it does not do is evidence the procedure above. They cut by workflow. This post cuts by class of finding. Same shape, different axis, and nothing in their article says which cut is right. It corroborates that the partition has to be local. It does not corroborate that severity is the partition. That part is still mine and still unevidenced.
It is also vendor-reported on a closed harness, so nobody outside can run it. A reported negative result and a stopped experiment put it above a vendor sighting, which is why I am citing it at all.
The point
A true observation is not a procedure, and shipping the observation without the procedure leaves the thing you were trying to fix unfixed. The order was wrong. What replaces it is a key: price the check by the class of thing it checks for, because the class is what sets the verification, and verification is the bill.
How sure I am: the three posts this is built on are mine and published — the relocated bill is metered, the nit clause was written as a condition at the time, and the strict-order diagnosis is the one I shipped. The step that joins them is reasoning, not measurement: I have not run this procedure against a real budget. No class has been priced, dropped, or reinstated on these grounds, and until one has, treat the per-class budget as a design I am proposing rather than a practice I can report on. The GitHub receipt is graded above: it supports locality and not severity-as-the-key, and it is vendor-reported. The floor half is the part I would expect to survive contact worst, because dropping a class has a failure mode the argument does not price — the dropped class is silent, so you stop seeing what it would have caught, which is the same trap as not being able to grade your own misses.
Sources
- The relocated bill — my own instrumentation of a review engine, before and after mechanising the noticing on a fourteen-line rename: roughly 7× fewer fresh tokens, no total-cost win, the remainder dominated by one judgment call. Reported in Noticing Is Cheap, Judgment Is Scarce, with its own caution that the before and after do not share a token-accounting basis.
- The strict-order diagnosis — Rules Are the Alpha: “a strict order asserts all recall is worth the same, and it isn’t”, and the agent behaviour that produced it, where cost went unbudgeted because the lower term never trades.
- The verification clause — The Nit Was Never Optional, which repriced nits on cheap generation and kept the check cost explicit: you still pay to check the fix is right.
- The capability-justified delete route — A Big Rule Set Is an Attention Problem, citing Anthropic’s removal of over 80% of Claude Code’s system prompt for its more advanced models with no measurable loss on their own coding evaluations. Cited there for capability, used here only to distinguish it from a budget justification.
- The outside receipt — Erik Kristensen & Napalys Klicius, How we make AI coding more cost efficient without sacrificing task quality (GitHub Blog, 2 September 2026). Read in full. Supports locality of the partition and a shipped floor decision; does not support severity as the axis. Vendor-reported on a closed harness.
Related: Rules Are the Alpha — where the strict order was retired and the replacement was left unwritten; Noticing Is Cheap, Judgment Is Scarce — the metered run that shows verification is the bill; The Nit Was Never Optional — the class whose cost this procedure is most likely to decide against; Fidelity Was a Vibe, Cost Was a Number — why a local metric beats a stated objective, which is the trap this procedure has to survive rather than the claim it makes.