Your Skills Are a Codebase, Not a Junk Drawer

“Perfection is finally attained not when there is no longer anything to add, but when there is no longer anything to take away.” — Antoine de Saint-Exupéry

I asked the agent to review a diff. Two skills woke up and started fighting over whose job it was. And the skill I’d built a week earlier for exactly this review? It never stirred.

That’s the moment every skill-hoarder hits. You keep adding skills because each one reads as free capability. Then the pile turns on you: two skills grab for the same task, the one you built for precisely this stays asleep, and a dozen more sit there doing nothing but costing you. A junk drawer — full, and useless.

Bottom line up front

Your skills are a codebase, not a junk drawer. A skill is code: it sprawls, rots, overlaps, and drifts the way code does, so you maintain the portfolio the way you maintain a codebase. You prune it; you don’t grow it. Three moves:

• Don’t build. Walk the resolution ladder first — most “I need a skill for X” is a routing rule, not a skill.
• Audit the portfolio as a system, not one skill at a time. Merge the overlaps, retire the dead weight, re-trigger the ones that never fire.
• Tune each skill against an eval. Your frozen judgment won’t tell you when it’s wrong — and check the thing is even wired, because “the file exists” is not “the behaviour it promises runs.”

The honest answer to “should I build a skill for this?” is usually no.

More skills is not more capability

Here’s the bit people miss. In Anthropic’s Agent Skills — the implementation I’m describing throughout this post — every skill’s description is preloaded into your context every session. That’s how progressive disclosure works there: the agent can’t read a whole skill to decide whether to use it, so it reads the name and the one-line description and picks from those. The body loads only after it’s chosen. Other platforms may wire this differently; the argument below assumes this always-on-description model.

So each skill you add is two things at once. It’s always-on token cost, sitting in the budget whether it fires or not. And it’s one more way for the agent to pick wrong for the task in front of it. Past a point, adding skills makes the agent less reliable, not more capable — more descriptions to tell apart, more near-misses, more turf wars like the one that opened this post. (That past-a-point tip into less-reliable was my inference from the mechanism — a small check now puts a curve on it: on a clean skill-selection task a 7B picked the right skill ~90% of the time with 3–10 skills loaded, ~70% at 30, and ~45% at 60. n=12×3 seeds, one model, and real selection is fuzzier than this matching task — harness in the repo.)

The cost of a skill was never the twenty minutes to write it. The cost is that it sits there forever, in every context window, competing.

Move 1: Don’t build — walk the ladder

Most “I need a skill for X” is not a skill gap. The distinction that saves you the most: a behavioural constraint is not a capability gap. “I keep forgetting to run the tests before I commit” is not a missing skill — you can run the tests, you just skip them. That’s a routing rule, a hook, a one-liner. You don’t build a whole skill to make yourself do a thing you already know how to do. A real capability gap — where you genuinely can’t do X — is the only case that earns a walk down the ladder, and even then it usually stops at rung two or three.

So before you build anything, walk the ladder — my own framework, lived not cited — cheapest rung first:

1. You already have it. The capability exists; you forgot it, or it’s filed under a name you didn’t search.
2. A routing rule. One line in CLAUDE.md or AGENTS.md — “when the user asks for a diff review, run the review skill.” No new skill, just a signpost.
3. Combine what you’ve got. Two existing skills chained do the job.
4. A config tweak. Tighten a trigger, widen a description, fix whatever made an existing skill miss.
5. A small addition to a skill you already own.
6. A new skill from scratch — the rung with the highest marginal cost. Every rung from the routing rule up adds some context and maintenance; a new skill adds the most, plus fresh overlap risk with everything already installed.

Six is the expensive one — and the one everybody reaches for first.

Move 2: Audit the portfolio as a system

Skills don’t rot in isolation. They rot against each other. Two skills grow until they overlap; a third contradicts a routing rule you wrote months later; four never fire because their triggers are too narrow; and a real gap hides behind all that noise.

You can’t see any of it one skill at a time. You have to lay the whole set out and read it as a system — the way you’d read a service map, not a single function. Merge the overlaps into one skill with one clear trigger. Retire the dead weight — a skill that hasn’t fired in months is a description tax you pay for nothing. Re-trigger the ones that should fire and don’t. Then, and only then, go hunting for the real gap — because half the time the “gap” was two half-skills that should have been one.

That’s what skill-audit does. It takes the portfolio and grades it on exactly those dimensions — overlap, conflict, dead weight, gap — with the resolution ladder above as its spine. It’s the tool I reach for when the drawer’s too full to see into. (The ladder and those audit dimensions are my own framework — lived, not cited.)

And it grades each skill against a bar, not just the portfolio against itself. The sharpest bar I’ve found isn’t in any style guide — it’s the system prompts and tool descriptions that ship with the frontier labs’ own agents. Two patterns show up there that few hand-written skills bother with. First, the triggers tend to be two-sided: they say when to fire and when to skip, and which side wins when two could fire. Skills that only say when to fire are the ones that turn on where a sibling should own the task — the turf war that opened this post. Second, the instructions make the model check before it asserts: verify the file still exists before you recommend it; never report success without evidence. A skill that skips that lies to you in a single confident run. Both are where a good skill and a junk-drawer skill part ways — so both are things a real audit grades. (My inference from lab prompts, not a survey: the officially-published system prompts are verifiable; the two-sided-trigger and check-before-assert details come from leaked prompt/tool-description collections I can’t independently verify. “Few hand-written skills do this” is my own observation.)

Move 3: Tune against an eval

A skill is judgment you froze. You captured how you’d handle something once, on one good day, and now it fires every time — your blind spots bundled in, running on cases you never imagined when you wrote it. Frozen judgment can be wrong. It can be stale. Either way it stays confident, and it never tells you which.

Code has this problem too, and we solved it decades ago: tests. A skill wants the same treatment — an eval harness, a set of cases with known-good answers, so you can tune the skill and watch it get better or worse instead of guessing. A skill without an eval is a claim you never checked. Not that every skill earns one — build the harness for the ones you actually lean on, and the ones that keep misfiring.

That’s skill-tuner: point it at a skill and its eval harness, and it tunes the triggers, the instructions, and the output against real cases. I learned the hard way that this reverses verdicts. I once had an eval that made a whole set of skills look useless — it was injecting every skill into every case, even the ones it should never touch, so it graded them on noise. Gate it to fire a skill only where the skill belongs, and the same skills came out clearly helping. The skill wasn’t broken. My measurement was. Fix the eval before you tune the skill. (My own episode.)

One thing that episode doesn’t buy you: gating the eval so a skill only runs where it belongs fixes how you grade its output, but it says nothing about when the skill fires. Tuning a trigger is a different job. To do it the harness needs both kinds of case — should-fire cases and shouldn’t-fire cases — so you can watch the trigger both catch the right task and stay asleep on the wrong one. A pass/fail gate on output alone can’t tell you a skill is waking up where a sibling should own the task; only a shouldn’t-fire case can.

And there’s a cheaper failure than a skill that scores badly — one an eval that assumes the skill ran will never catch: the skill that doesn’t run at all, while its file sits there describing what it would do. I turned up three of these in my review harness inside a week. A promised check whose agent was never dispatched — something cheaper had replaced it months back, and the only validation on the file confirmed that it existed. A routing field that documented how it steered the pipeline, which nothing at runtime read. A setting authors kept editing in one file while the code parsed it from another.

None of those is a stale comment. A stale comment misleads a person, and a person can shrug and go read the code. These are artifacts the harness is supposed to consume or honour, and the shape they share is this: an artifact your agent system treats as authoritative is an interface, and interfaces have wiring bugs. It can name a consumer that doesn’t exist, promise a control nothing reads, or quietly lose to a second copy of itself. Ordinary interface failures, in the one place nobody thought to type-check.

So the check you want isn’t “does the file exist” — mine passed that three times over. It’s three questions per authoritative artifact: who consumes it, is that consumer reachable, and is this the copy the code actually reads? You can answer all three by tracing, and you can prove it behaviourally: change the artifact and confirm something downstream changes. If it’s meant to control the harness and nothing changes, it isn’t a control. It’s set dressing. (Three failures in one harness — mine. A pattern worth checking for, not a prevalence claim.)

The shape of it

Skills are a portfolio. You prune it. This is the maintenance sibling to Skills Are One-Offs — that post is about how skills don’t lean on each other; this one is about keeping the set you’ve got healthy. The software-portfolio framing — that skills sprawl and rot like code — is mine. As I put it in The Brief: a skill is judgment you froze; it fires every time, your blind spots bundled in.

So before you build the next one, ask whether the drawer needs another thing in it — or whether it needs a clear-out. Usually it’s the clear-out. (That “usually” is my judgment from doing this, not a measured rate.)

How sure I am: The software-portfolio / tech-debt framing is mine; skill-audit, skill-tuner, and the eval-reversal are my own builds and lived evidence; the progressive-disclosure cost is documented (cited). The past-a-point degradation now has a small measured curve (7B: ~90% selection at 3–10 skills → ~45% at 60; n=12×3, one model, clean matching task — repo). The rest is argued from the mechanism, not measured.

Sources

• Agent Skills progressive disclosure — each skill’s name and description is preloaded into context every session so the agent can decide which to activate; the skill body loads only once chosen. This is the always-on cost that makes “more skills” a liability past a point. (Anthropic, Agent Skills documentation; agentskills.io.)
• The resolution ladder and the audit dimensions (overlap / conflict / dead weight / gap) are my own framework, shipped as skill-audit. Lived, not cited.
• Eval-driven tuning of skills ships as skill-tuner; the “fix the eval before the skill” episode is my own.
• The software-portfolio / tech-debt framing (skills sprawl and rot like code) is mine.
• “A skill is judgment you froze” — from The Brief.
• The single-skill design bar (two-sided triggers; verify-before-assert) — observed in published and leaked lab prompts: Anthropic’s officially-published system prompts are verifiable; the tool-description patterns come from leaked collections (e.g. asgeirtj/system_prompts_leaks, CC0) I can’t independently verify. Shipped as checks in skill-audit; my inference, not a style guide.

---

Related: Skills Are One-Offs (the sibling — which reuse layer a thing belongs on); Shrink, Reorganise, Relocate (pruning is the move that’s actually free; reorganising passes the synthetic test); A Pipeline Is What It Refuses to Forward (the same pruning question asked of a seam instead of a portfolio).