Bryan Low

Don't write CLAUDE.md

aiclaude-code

CLAUDE.md goes stale. You write a rule for today's model, then the model changes or the harness learns to do the thing on its own. The rule stays behind, wrong or redundant, and costs tokens every session. Then you add Codex, copy the rules into AGENTS.md, and now you have two nearly identical files drifting apart.

So I stopped treating either file as the source. I keep one baseline of what I want from a coding agent, and every now and then I audit the two files against it. The audit proposes what to add or remove, and I review the proposal before anything changes.

Neither file changes on its own. I call them outputs, since the actual deciding happens in the baseline.

The baseline

The baseline isn't tied to any tool. It holds preferences that should last, not workarounds for how one model happens to behave this month. Each entry has an instruction and the reason I'm keeping it.

- **Context hygiene**: route disposable work — searches, whole-file
  reads, log triage — to subagents, so only distilled results reach the
  main context. _Why: long contexts can bury relevant information._

The reason is there for future me. Six months later, I can see why I added the rule instead of guessing from the wording. Preferences usually hold up. Claims about how models behave need to be checked again.

I only change the baseline when what I want changes. A model release or a scary Reddit thread might make me run an audit, but it doesn't get to rewrite the baseline.

Projection

One rule decides what goes into each instruction file:

A line is projected only while the harness doesn't already cover it. If that coverage goes away, the line comes back.

This way I'm not spending context on instructions the harness already gives. If the two tools really differ, that difference goes into the projection, and I still only have one version of what I want.

When a rule comes from policy or configuration, its wording is generated from that data instead of yet another list I maintain by hand.

Coverage is the awkward part. A model telling me what its harness instructions say isn't ground truth. Prompts may be hidden, different model tiers may get different instructions, and a model can simply be wrong about whether two rules mean the same thing. So I treat the result as a review aid. If the evidence is weak or the models disagree, the line stays.

What earns a line

Every rule costs context and attention. The fiftieth rule isn't automatically harmful, but it still needs a good reason to be loaded in every session. Mine has to pass four checks:

  • Observed failure. It fixes a mistake I've watched happen more than once, not one I'm worried might happen. A one-off just needs a better prompt. Only a recurring gap gets a rule.
  • Non-inferable. The agent can't work it out at runtime from the code, the harness, or policy it's already under.
  • Durable intent. It's what I want, not a patch for a current quirk. Quirk patches go into on-demand docs with a date on them, where they're cheap to expire.
  • Not mechanically checkable. If a sandbox rule, a hook, or a linter can enforce it, it goes there. A hook fails loudly when the rule is broken, a line of prose doesn't.

Most things I want to write down fail at least one of these.

The audit

After a model or harness update, I run an audit. It reads the baseline and the current projections, asks each model tier whether it already covers each principle, then produces a matrix for me to review.

# agent-instructions-audit
 
For each baseline principle and output:
 
absent from any model tier, missing from projection -> ADD
covered by every model tier -> SHAVE
partial coverage or disagreement -> KEEP
projection contradicts baseline -> CONFLICT
 
Get a second model's opinion on proposed changes.
Report evidence and diffs. Change nothing without confirmation.

SHAVE needs every tier using that output to agree. If one tier looks like it needs the instruction, or the evidence is unclear, I keep it.

A principle, end to end

One baseline rule says:

- **State persistence**: where compaction is unreliable, persist the plan,
  decisions, and open threads before the context gets tight.

At the time of one audit, the two projections differed:

claude   covered   -> OMIT
codex    partial   -> KEEP

That won't stay true forever, and the next audit might disagree. If the evidence changes, I get a new proposal and decide whether to accept it.

Refreshing the whys

A rule might last longer than its reason. “I want to review my own commits” is a preference. “Long contexts can bury relevant information” is a claim, and claims need checking.

So I have a second skill for that:

# doctrine-refresh
 
Pull every empirical claim from the baseline.
Check each one against current evidence.
Mark it supported, contested, or obsolete.
Get a second model to review the result.
Report. Change nothing without confirmation.

If a reason is weak, the skill proposes new wording. If the whole case for a rule has fallen apart, it stops there. Evidence can tell me a reason is wrong, but it can't decide that I've changed my mind.

Evidence and limits

The research doesn't agree on whether these files help at all. Gloaguen et al. found no general improvement in task success and higher inference cost, though agents did follow recorded non-standard practices. Lulla et al. found lower median runtime and fewer output tokens with comparable task completion.

A study of 100 repositories found context bloat in 42% of agent configuration files and skill leakage in 35%, and conflicting instructions often showed up alongside them (dos Santos et al.). That's a good reason to prune. It doesn't prove my audit works.

Lost in the Middle, RULER, IFScale, and When Instructions Multiply give useful background on long-context use and following multiple instructions. Their tasks aren't coding-agent instruction files though, so I take them as reasons to be careful, not proof that fewer lines make a better agent.

I haven't measured any improvement in task success from this workflow. What I can see is smaller: the two files no longer drift apart, temporary harness quirks are easier to spot, and every standing rule has a reason I can go back to.

The work didn't go away. Instead of patching two files, I review one baseline and the evidence behind it. It's been a smaller, calmer job across several model updates.