A Cue Finds More of What It Can Already See

“The eye sees only what the mind is prepared to comprehend.” — Robertson Davies, Tempest-Tost (1951)

An AI reviewer runs on a real PR and finds three bugs. I drop a line into the file it reads before every review — the repo’s own AGENTS.md-style guidance file, the way you’d really steer it, not a one-off prompt trick — that says “there are at least four or five bugs in here, find them.” Same PR, same pinned commit, same base. Now it finds six.

And not one of the six is a bug it wasn’t already finding without the note. The cue didn’t hand it new sight. It made it say more of what it could already see. That gap — count up, coverage flat — is the whole post.

Bottom line up front

• A cue amplifies coverage the reviewer already has. On this PR, across every run, it never reached a blind spot — and signal-detection theory says why to expect that: a cue shifts the criterion, it doesn’t add sight. “Find the bugs” turns the lit part of the room up; it doesn’t light a new part.
• The extra findings are volume, not new bugs. Nearly every finding across nine runs mapped to a pool of about six real issues. Some “extra” findings were one root reported twice. “Found six” is not six distinct bugs.
• Two real bugs stayed missed in every single run, at every cue strength — including when I told the reviewer explicitly “a prior review already found these three, don’t repeat them, find the ones that were missed.”
• Grounding is the confabulation switch. A reviewer that can check its own claims against the code takes the cue cleanly. One that can’t invents findings from absence. Same cue, opposite outcome. (This one is a proxy result — see caveats.)
• Unioning decorrelated passes catches about 2.5x a single pass. That’s a separate lever from the cue, and the two stack.
• The takeaway: once verification is cheap, over-generating is nearly free. So spend on coverage and diversity — more passes, different framings, real grounding — not on a louder prompt.

The cue lifts the count, and breaks the cap

Three conditions, three runs each, everything else frozen. Findings per run:

• no cue: 3, 3, 3 (mean 3.0)
• mild cue (“at least two bugs”): 3, 3, 4 (mean 3.3)
• inflated cue (“four to five bugs”): 5, 6, 2 (mean 4.3)

The count climbs with the cue, and the inflated cue breaks a natural cap — left alone, the reviewer settles around three findings and stops. I re-ran the inflated cue out to six: 5, 6, 2, 4, 4, 4 (mean 4.2). The lift holds. The variance is genuinely higher than the flat control, but the lone “2” was a tail — five of six runs sit between four and six.

So the cue works, in the narrow sense that it moves the number. That’s exactly the trap.

The count is volume, not new bugs

Line the nine runs up against each other and almost every finding maps back to the same pool of about six real issues — six being the union across all nine runs, while any single run surfaces only three or four. The cue didn’t discover a seventh. It raised how many of the six surfaced in a given run, and it started splitting roots — one bug (“this identifier is fully qualified”) reported as two findings. “Found six” was three real bugs, one of them counted twice, plus reruns of things earlier passes already had.

And two real bugs — a controlled-input mismatch and a stale-response race — showed up zero times out of nine. No cue strength touched them. That’s the ceiling I saw: on this PR, across all nine runs, the cue amplified the reachable pool and never surfaced those two. But 0/9 can’t separate “unreachable” (a genuine blind spot) from “rare” (a real bug with a low base rate that would also show zero) — so the blind-spot claim leans on the mechanism, not the count. Read it as a mechanism-backed prediction — a cue shifts the criterion, it doesn’t add sight — not a measured law.

Grounding is the switch between finding and confabulating

Turning the cue up is only safe if the reviewer can check what it says. The grounded reviewer — the one with a full checkout, able to open the file and confirm — took the inflated cue cleanly: more findings, still grounded in real code, roughly no confabulation even at “four to five.”

Earlier, weaker stand-ins that saw only the diff did the reverse. Cued to find more, a diff-only reviewer invented a missing import — inferred from the fact that the diff didn’t show the import line, not from any real absence in the code. Same cue, opposite outcome. The difference is whether the reviewer can verify its own claim. Cheap grounding, not expensive verification, is what neutralizes the cue’s failure mode.

One seam, stated plainly: this contrast is a proxy, run on stand-ins, not yet on the production scout under a live cue. Whether the real diff-only scout confabulates under a cue the way the stand-ins did is a receipt I still owe. Read the switch as strongly indicated, not nailed.

A small check pays part of that receipt on a stand-in. Under the same inflated cue (“at least four issues”), I showed one reviewer the full file and another only the function body — with the helpers it calls defined off-screen. Both produced about four findings to meet the cue; the grounded one confabulated almost nothing (~0.2 invented “undefined/missing-import” findings per file), while the diff-only one filled most of its quota with inventions (~2.8 per file) about symbols that were defined all along. Same cue, opposite outcome — the switch, measured (n=6, one 7B; still a stand-in, not the production scout).

Told to find the missed ones, it still couldn’t

The sharpest test of the ceiling: I told the reviewer, explicitly, “a prior review already found these three; do not re-report them; find the two or more that were missed.” Two runs.

It still hit neither blind spot — zero, again. Pointing directly at “the missed ones” did not create sight. It re-reported all three “known” bugs anyway (an exclusion instruction — “don’t repeat X” — doesn’t stick; the salient findings recur regardless), and it filled the remaining slots from the reachable pool — more of what it could already see, dressed up as an answer to “what did you miss.”

Across mild, inflated, and directed conditions, those two bugs are zero for every run here. The pattern held across every run on this PR: a cue amplifies coverage; the mechanism says it does not extend sight.

The lever is coverage — and it stacks with the cue

Here’s the part that changes what you do. I ran a decorrelated loop on the real engine, no cue at all, deduping findings by rule and location:

• round 1: 12 findings — round 2: 11 (8 new, cumulative 20) — round 3: 14 (10 new, cumulative 30).

It never went dry. The union of three passes caught about 2.5x a single pass, and round three was still adding ten new findings — so the right N is four or five, or loop-until-dry, not three. Round-to-round overlap sat around 25–35%, which means real run-to-run variance, not a bookkeeping artifact.

One honest caveat, which is the point: these were deep-confirmed and deduped, not oracle-verified. “+18 unique” is not “+18 real” — without ground truth, some are false positives the verifier let through. Directionally strong, not proven. (This is exactly why good recall is unearnable — You Can’t Grade Your Own Misses carries that argument.)

Two things follow. The cue and the union are separate levers, and they stack — the union grows the reachable pool through decorrelated draws; the cue surfaces more of whatever pool you’ve got. And once verification is cheap, over-generating costs almost nothing, so recall is gated by coverage, not verifier spend. The move is parallel scouts feeding one deep verifier — different framings, real grounding — not one reviewer with a louder prompt. Diversity reaches the unseen. Intensity just says more about the seen.

---

How sure I am: Measured — the cue matrix (three conditions, and the inflated cue out to six runs), the directed-exclusion escalation (two runs), and the union loop (three rounds on the real engine). Real runs, one system, one PR each, genericized. Directional, not a published rate. Reasoned — the “spend on coverage, not a louder prompt” conclusion, and the parallel-scouts-into-one-deep design; these follow from the data plus the cheap-verification economics, but I haven’t run the scouts-into-deep configuration head-to-head yet. Single anecdote / proxy — the grounding switch (diff-only confabulation vs grounded clean use) comes from stand-ins, not the production scout under a live cue, and “the grounded runs didn’t confabulate” is read off the finding descriptions, not verified line-by-line against the code. Both are owed receipts, flagged as such — though a small laptop stand-in now measures the grounding switch (grounded ~0.2 vs diff-only ~2.8 confabulated “undefined/missing-import” findings per file under the same cue), paying part of that one.

Sources

• Cobbe et al., 2021, Training Verifiers to Solve Math Word Problems (arXiv:2110.14168; introduces GSM8K) — train a verifier to rank generations.
• Lightman et al., Let’s Verify Step by Step, 2023 (arXiv:2305.20050) / ICLR 2024 — process supervision; step-level verification beats outcome-only supervision.
• Weaver — Saad-Falcon et al., Shrinking the Generation–Verification Gap with Weak Verifiers, Stanford, 2025 (arXiv:2506.18203; NeurIPS 2025) — and Trust but Verify! A Survey on Verification Design for Test-time Scaling (Venktesh et al., 2025, arXiv:2508.16665) — verifying is easier than generating, which is why over-generate-then-verify pays.
• Huang et al., Large Language Models Cannot Self-Correct Reasoning Yet, 2023 (arXiv:2310.01798) / ICLR 2024 — self-verification needs an external signal; the verifier can’t be the voice that generated.
• Bebugging / defect (error) seeding — Harlan Mills, 1970s — planting known bugs to estimate what’s missed.
• Satisfaction-of-search / subsequent-search-miss (radiology; review: Adamo et al., 2021) — finding one target lowers the odds of finding the next.
• Signal-detection theory (Green & Swets, 1966) — a known-target-present expectation shifts the criterion, raising both hits and false alarms.
• My own reviewer-cue notes — the cue matrix, the directed-exclusion escalation, and the union loop, genericized. Working documents; no production recall figure claimed.

---

Related: You Can’t Grade Your Own Misses (you can’t measure the misses; this post is about cuing to find more of them, and how far that reaches), A Pool of Models Narrows the Overlap — It Doesn’t Erase It (coverage isn’t a verdict — the union under-covers too), Fidelity Was a Vibe, Cost Was a Number (why cheap verification makes coverage the thing worth buying), A Flip Is a Missing Rule (the union-of-N measurement here, reclassified — it buys coverage, and the run-to-run variance it reports is a pointer rather than noise to damp).