Appendix — the techniques, tested
These are the moves I actually reach for, pulled from my own session logs — not a tips list, a tested one. A few kinds of receipt sit behind them, and I’ve marked which is which: some were put through a quick with-vs-without A/B (I give the n); most were earned in the making of this essay (marked from the build); and one comes from a separate coding project. Either way the evidence is directional — small n, often one judge, scenarios I built myself — so read that limit once, here, and take the verdicts as a starting map, not settled science. Each comes with a Try: — the actual words I’d type, not a template to fill in. I’ve kept the failures. A technique that backfired is a more useful entry than one nobody checked, which is this whole essay pointed at its own toolkit.
Start here — the 60-second version
Two halves to this, and only one belongs in the machine.
The machine’s half — freeze in the small checklist. Put the following rules somewhere your agent always reads. They do not make the model smarter; they make it less likely to skip the checks you would have wanted it to run.
## Decision quality
On a non-trivial or hard-to-reverse decision, a user correction, or a repeated
failed fix, apply these checks before committing:
1. Verify what exists. Before proposing a new abstraction, file, dependency,
or approach, confirm the existing system cannot already do it.
2. Name assumptions with confidence: high / medium / low. Flag any low-confidence
assumption that would change the answer if wrong.
3. Trace consequences in practice: rollout, maintenance, consumer impact, and
what breaks at 3 a.m.
Lead with a recommendation. If uncertain, recommend the option safest to be
wrong about and say why. For genuinely irreversible decisions with missing
context, "pause — here are the options; don't choose yet" is itself a
recommendation.
## Decisions
Authoritative decisions live in the source — the spec, this file, the code — not in chat.
When something changes, update the source and mark it "SUPERSEDES: <old>". Re-read the
source before acting; a change mentioned only in conversation is a suggestion, not a decision.
## Review
For review that matters, get a second opinion from a DIFFERENT model — not more personas in
this session. Same-model agreement is weak signal; what's useful is where two models disagree.
It's evidence you weigh, not a verdict you hand off.
## Summaries
Treat any summary — a subagent's report, a test digest, your own recap — as a claim, not a
fact. Before acting on its key detail, check it against the raw. Fluency is not correctness.
## Long sessions
When a thread has run long, prefer a fresh session seeded from the source-of-truth file over
carrying full history. Re-ground from the artifact, not the conversation's memory of it.
## Scoping — order, not whether
For work you want done anyway, the only question is sequence; the axis is coupling, and the cost to
price is the human's downstream cost (which doesn't parallelize), not your effort. Do independent,
obviously-mechanical work now, in parallel — it's cheap, and deferred it's the stuff you lose (you
misjudge quick work as unimportant, so your "skip it" read is the unreliable part). Leave work coupled
to a still-unsettled hard call — doing it now is rework when the call lands; let the hard path incubate,
and absent a real deadline don't rush it. Review the settled state once. The context-switch tax
isn't loop length — it's leaving with a full head, several half-resolved forks to keep warm; so the
cheap place to stop is a clean seam, forks closed and the next task standing on its own. A
heuristic, not a law: it only has to beat the default, and genuine urgency still jumps the queue.
Your half — nobody installs this. Three moves carry most of it. Brief for a conversation, not compliance — put your real thinking in (not a one-line wish), and get the machine questioning it rather than just executing. Rough idea or finished plan, the move is the same: make sure something pushes back before it’s built, because it won’t always come for free. Treat everything it hands back as a claim, not a fact, and check the load-bearing parts against the raw: fluency isn’t correctness. And keep asking whether you’re still solving the right problem, not just whether the output looks good — that’s the one call that never delegates.
The one that licenses the rest
- Test the technique itself — with vs without, and a sham arm. Useful for: not fooling yourself about your own cleverness. Receipt: it’s the only reason any verdict below exists, including the ones that embarrassed me. (A/B — the battery) Try: “run this same task three ways — with the rule in context, without it, and with a decoy rule that shouldn’t matter — then have a fresh agent score all three blind and tell me which won and why.”
The ones that earned it
These overlap and double back in practice — a kit, not a sequence.
- Get the second opinion from a different model, not more personas. Useful for: catching what you’re blind to. Receipt: a different model family caught real over-claims in here that my own panel — one model in five hats — nodded straight past; on code, it flagged typed API methods that still registered untyped listeners and a
teardown()that never finished its lifecycle, both of which my own read had passed. Sequence it late and keep the pen: the different family goes last, after your own passes; let it find, then make the fixes in your own model so the prose still sounds like you, and do the final read on already-checked text. (from the build) Try: (hand your answer to a different family) “here’s a claim and its reasoning. find the over-claims and the logic gaps — quote the line, say why. don’t tell me where you agree.” - After a rewrite, ask what contract or depended-on behavior it dropped. Useful for: the capability a rewrite silently loses. A rewrite optimizes the paths you’re holding in mind and quietly drops the ones you’re not, and a fresh read of the new code won’t flag them; nothing there looks wrong. Diff against the prior artifact, not the task — hand old and new to a different family and ask what the old one guaranteed that the new one doesn’t. Not every difference matters (old code covers plenty by accident), so target contracts, tests, and depended-on cases; and verify the pointer: check the diff it claims against the actual old code. Receipt: rewriting a small library, this caught a real loss — a per-scope mode a caller depended on. I run it on rewrites by default, before trusting my own read. (separate coding project, n=1, not this essay) Try: (old + new in hand) “here’s the original and my rewrite. what contract or depended-on behavior did the rewrite drop? name each and the caller that relied on it; don’t praise the rewrite.”
- Ground every load-bearing claim in the source — then recheck the grounding against the raw. Don’t accept “I checked, it holds”: a model hallucinates the verification as readily as the answer, so a confident “it checks out” is a second fabrication wearing a checkmark. Ask it to ground the claim by default — attribute it to the exact span it rests on, the row or line or output, not its description — and then verify the pointer, not the whole source: does that line actually say it? Cheap to check, cheaper still handed to a different model. And check the join, not just the pieces — a hallucination can be true fragments recombined into a whole that was never there: a real paper cited for a claim it never makes, each part checking out while “the paper makes the claim” goes untested. Verify the relation, not just the entities it connects. Useful for: catching fabrication, and the second-order kind that hides behind “verified.” Receipt: three confident summaries lied to me while I made this essay; the catch was always the raw behind the claim, never the claim. (from the build; grounded attribution beats unaided self-correction — RARR, Gao et al. 2023; FActScore, Min et al. 2023; relational verification, SAFE, Wei et al. 2024.) Try: “ground the key claim — quote the exact row / line / output it rests on, not your summary — then check that span against the claim and tell me where they diverge. If you can’t point to it, you can’t claim it.”
- Make “done” a grade, and loop to it. Useful for: a stopping rule that isn’t a vibe. Receipt: “is it an A yet?” ended every section; “it feels done” would have stopped them three drafts early. (from the build) Try: “grade this A–F against [the goal]. list only what’s keeping it from an A, quote each line, name the fix. then fix the mechanical ones, panel the judgment calls, run a voice pass, hand it to a different model last, save anything only I can decide for the end, and re-grade until it stops dropping below A.”
- Don’t lead — withhold your own preference. Useful for: not getting your worse idea rubber-stamped — and, in open-ended brainstorming, for getting generative range instead of your own idea echoed back. Receipt: when I said “I’m adding a cache,” the model blessed the band-aid; when I just described the slow endpoint, it found the real fix. (A/B, n=2; the mechanism is sycophancy — RLHF nudges models to match your stated beliefs over the truth, Sharma et al. 2023.) Try: not “I’m going to add a cache here — sound right?” but “this endpoint is slow: [symptoms]. what’s the actual cause, and the smallest fix?”
- Ask for confidence, but the score isn’t the check; the raw is. Useful for: surfacing the unknowable instead of papering over it. Receipt: the plain answer asserted an unknowable fact; the calibrated one admitted it couldn’t know. (A/B, n=1) Caveat: the number is another fluent output — a self-reported score runs overconfident, and post-training doesn’t make it trustworthy (Xiong et al. 2023; Tian et al. 2023). A fluent wrong claim can arrive at high confidence; the tag is made of the same stuff as the answer, so use it to aim your reading, not to end it. Try: “answer, but tag each claim high / medium / low confidence, and flag anything you can’t actually know from what I gave you.”
- Re-run in a clean room. A fresh agent with none of the conversation. Useful for: telling a real result from an artifact of the chat. Receipt: the from-memory rewrite of this essay was exactly that — and it’s how the panels avoid grading their own homework. (from the build) Try: (open a fresh session, none of the thread) “here’s the task and the inputs, nothing else. solve it from scratch.”
The ones that only sometimes help
- A panel of personas — for coverage, not a vote. Five named experts on one model is one model in five hats; their agreement is theatre. Useful for: the considerations you’d have missed. Not for deciding — they converge, and the convergence is usually just correctness. Receipt: a blind cross-model run showed named personas vary the framing, rarely the decision. (from the build) Try: “answer as four people who’d genuinely disagree — name them (Ousterhout, Hickey, …). give me the considerations each would raise, not a tally.”
- Force alternatives — but only when the call is close. Useful for: a genuinely contested decision (it produced a cleaner rule on a real config question). Noise on an easy one (it buried a naming answer under a terminology sidebar). Receipt: one win, one loss. (A/B, n=2) Try: (on a call you’re genuinely torn on) “give me three real alternatives and the one you’d pick, with the tradeoff you’re accepting — a pick, not a list.”
The ones still unproven — interesting, not earned
- Make it a story / shape the register. Useful for: memorability — a blind judge picked the story version of this essay as the one more likely to be finished and quoted. But that version also got more revision passes than its plainer twin, so the effect is confounded with sheer iteration: I can’t separate “told as a story” from “worked on it longer.” Interesting, not earned — which is why it’s not in the list above. (A/B, n=1, confounded) Try: “retell this as a short story someone would actually finish and quote a line from — same substance, no lecture.”
- When a loop won’t hit the bar, a fixed try-count is a blind stop. The argument, not a result: pick N tries and you can’t tell from inside the run whether N was right; the win might come at N+1, and the count carries no signal about its own stopping point (pick N=1 and experience says the answer often comes later). So read what the run is doing instead: is it repeating a failure it’s already hit, or failing in a new way each time? That reads more of the run’s state than a number does; information, not a proof it stops better. Receipt: I built this as a failure-signature stop in a model-tuner and watched a live run stall. That earns the failure mode — loops do get stuck — not this rule’s superiority; shipping it isn’t evidence it works. Whether it beats the fair baseline (a fixed budget plus a normal no-improvement patience, not a static N picked to lose) I never measured. A failure guard observed in one live tuner; unproven as an optimization. (shipped + by hand, n=1; unmeasured vs a patience baseline; content-aware stopping — semantic early-stop, SHP.) Try: “loop and improve on each failure, and stop when we start seeing the same failures again. tell me when we do.”
The one I had to test five times
Step back, look upstream, change the problem — a checklist nudge that helps only where the default would skip the step.
The first tests said it failed. The next said it worked. Both were too tidy, which should always make you nervous. The problem was the test: I kept handing the model the code that already revealed the root cause. Once it can see the N+1 query, “look upstream” adds nothing — and tips into over-engineering.
The move is for a different moment — before you show the code, when you’re about to prescribe the fix yourself: “my endpoint’s slow, add a cache,” and the model can’t see the root either.
Tested there — four band-aid requests, five reps each, blind cross-family judge — the nudge made the model question the band-aid 20 of 20 times. But the plain model already does it 12 of 20; it won’t blindly build a famous anti-pattern. The nudge’s real value is the cases it flips, and they cluster on the dangerous, less-obvious one: a flaky-charge retry. Unprompted, all five reps wrote the retry loop; prompted, all five blocked on idempotency first. On a charge call, that’s the difference between a double-charge bug and not.
Try: “before you build that, what’s the actual root?” — but only before the code’s in front of the model. Once it can see the root itself, the nudge is on probation.
(Five designs to pin it down; final = 4 scenarios × 5 reps, blind cross-family COMPLY/QUESTION classification, matched by my own read 40/40. The mechanism is this essay’s own: a checklist item that fires the step you’d otherwise skip. Full notes in the research.)
The ones you’ve already seen
If you’ve read a “prompt like a pro” post, you’ve met the carousel: RAIN, CLAR, FLOW, PIVO, SEED, CRAFT — a new acronym a month, plus spin-offs (CRAFTe, CREATE). Line them up and it’s the same handful of pieces re-lettered — the labor goes into the initials, not the idea. None of the pieces are new: it’s retrieval, few-shot, a persona, and an output spec with the serial numbers filed off.
But before sorting them, notice what they’re all for: the transmissive use — getting the model to do a task you could have specified yourself. The constructive use — brainstorming, reaching a thought you couldn’t alone — runs on a different rule: give it the problem, the constraints, the uncertainty, but don’t hand it a finished answer disguised as a spec, or it just polishes the idea you walked in with (see Don’t lead, and the transmissive/constructive split the essay opens with). Everything below is for the transmissive mode; here’s what each piece actually changes.
Change what you get back:
- Context — the facts and constraints it can’t infer. Feed it the wrong context and you get the same fluent, confident answer built on false premises; the confidence is just the register, not a belief that shook. Put what it can’t know.
- Examples — a sample of what “good” looks like, and it earns its place: in the essay’s own run (one judge, so suggestive), stripping the worked examples dropped a brief’s score from 9.5 to 5.3. Not separate magic: like everything in the prompt, it shifts probability mass rather than adding capability (the essay’s mechanism point). It’s often the clearest way to state the task at all, pinning down intent that prose leaves open — though it can add little, or even mislead, when the model already does the task cold or the samples cue the wrong pattern. Show one when the target shape isn’t obvious.
- Role — a generic “act as a senior engineer” barely moves anything; the lift is corpus specificity, not the word “expert.” “Review this like Ousterhout” (or Hickey) points at a narrow, opinionated body of work — it reweights toward that person’s stance, changing the perspective, and mostly where they genuinely dissent. It shifts framing far more than the decision (the essay’s persona panel; Hu & Collier 2024). A job title does little; a named character is a lever on viewpoint.
Change how it’s delivered, not whether it’s right:
- Format, tone, audience — the shape, the register, who it’s for. Three names for one thing: how the answer comes back. Set it so you’re not reshaping by hand — but reformatting a wrong answer just gives you a tidier wrong answer.
Helps you give the model a clear task:
- A clear task helps the model as much as it clarifies you — vagueness fails at both ends (the essay’s second move). What it doesn’t hand you is the test for clear, so: one reading not several — no relative terms it has to resolve for you (“good,” “fast,” “a few” with no number) — and some way to tell whether the output did it, the bar requirements engineers have used for decades. That clarity is the one piece the checklist can’t supply for you; all it does is make you produce it.
Theater:
- Invented targets — “cut abandonment 25%,” “1.5% keyword density.” Ask for a number it can’t ground and you’ll often get a confident fake instead of a “can’t know.” Precision-shaped and empty.
So the honest version of the whole carousel is short: give it the context it lacks, show an example when the shape isn’t obvious, name a real opinionated persona if perspective matters, and say what shape you want back. The rest is you briefing yourself, and even done perfectly, it’s the floor. What moves the needle sits upstream of the prompt — the thinking you put in, the pushback you invite before it builds, the answer you reject. The catch: a cleanly stated task is what convinces you it’s the right task. Use the checklist if it helps — just don’t mistake it for the brief.
No Try: here — this one maps the pieces, it isn’t a move.
The honest size of all this
The failures are kept on purpose. The upstream-redirect nudge is the one I trusted least, so I tested it — and then four more times, because each result was either tidy-and-suspicious or measured in the wrong condition. The fifth test finally put it where the move is actually used — a band-aid request with the code unseen — and the hit reproduced: it reliably catches the band-aid the model would otherwise just build, most valuably the dangerous one. Five rounds to earn one honest, qualified verdict. That sequence — tidy, suspicious, reversed, reversed again, finally tested right — is this essay’s whole argument happening to its own appendix. An untested technique that sounds good is the fluent-but-unchecked thing the essay warns against; so is a tested one whose tidy number you don’t interrogate. That’s why even the embarrassing results stay in — and why the clean ones get the asterisk too. The clean result was never the prize; the honest one is.