A red skipped test versus a green real fix — the falsifier that turns one into the other.
Opinion 7 min read

Conduct harnesses: falsifiable rules for AI coding agents

Arnold Wender
Arnold Wender
#AI #Open Source #AI Agents #Developer Tools #Opinion

I spend most of my days now working alongside autonomous coding agents. They are fast, tireless, and — under pressure — quietly willing to cut a corner a good teammate never would. And they cut the same corners, over and over: they report a green test suite they bought with a skipped test, invent a config flag that reads plausibly and doesn’t exist, --force past a wall they never read, or quit at the first red line and call it done.

None of that shows up as an error. All of it shows up later — in production, on someone else’s afternoon.

Here is the gap I kept bumping into: the tooling we already have checks the code. Linters, type checkers, tests, CI — keep every one of them. None of them check the agent: the judgment before the keystroke, the honesty of the report, the decision to keep going or bail. That is a conduct problem, and conduct doesn’t live in a config file.

So I wrote a small thing to fix it — and then, because I couldn’t stop, I wrote it ten times.

What a conduct harness is

A conduct harness is a tiny codex you keep in the agent’s context while it works. Four disciplines:

  • Cleanliness — what you leave behind.
  • Judgment — how you decide under pressure.
  • Honesty — how you report.
  • Persistence — whether you abandon the work.

Plenty of CLAUDE.md files already say “please be careful.” The part I actually care about is different: every rule ships with a falsifier — the specific, observable thing that proves the rule was broken. Not a vibe. A violation you can point at, and a script or a reviewer can check.

That word is deliberate. Karl Popper drew the line between science and everything else at falsifiability: a claim that nothing could ever disprove isn’t a strong claim, it’s an empty one. The same is true of a rule you hand an agent. “Write clean code” can’t be broken on paper, so it’s never really followed. “No console.log you added survives into the diff you call done” — that one fails loudly.

The whole idea on one screen

Every edition ships the same worked example, because it is the mechanism. Give the agent a failing test and a deadline. The cheap rescue looks like this:

- expect(parseAmount("1.005")).toBe(1.01);
+ test.skip("rounds two decimals", () => {
+   expect(parseAmount("1.005")).toBe(1.01);  // flaky, revisit later
+ });

The suite is green. The bug is still there — parseAmount still rounds 1.005 down to 1.00 — and now nothing watches it. A regression shipped wearing a checkmark.

Under the harness, the rule “refuse the cheap rescue” carries the falsifier “a test skipped or a warning muted to force a green check.” So skipping the test trips the falsifier out loud — and the agent takes the other road instead:

- return Math.round(Number(s) * 100) / 100;   // 1.005 * 100 = 100.4999… → 100
+ return Math.round((Number(s) + Number.EPSILON) * 100) / 100;  // half-up on the cent

Same model, same task, opposite outcome. The suite is green because the code is correct, and the test still stands guard. That is the whole pitch. (And to be honest about it: this is an illustration of the mechanism, not a benchmark. I’m not selling you a number.)

Two layers, and why there are ten of them

Each rule is written twice. An identity name you remember under deadline, and an engineering name that’s actually testable. A rule the agent forgets the moment the context window fills is dead weight — so the memorable half matters as much as the checkable half.

That is also why there are ten cultural editions of the exact same spine. A discipline with a name your team believes in gets followed; a numbered rule gets skimmed. Pick the idiom that lands for you:

One spine underneath, verbatim. The skin is just the handle you’ll actually grip.

Where it came from

Honest origin: the idea arrived as a kind of download while I was building my own harness for my work. Something said do it, and I ran. A day later there were ten.

What it is, and what it isn’t

Each is MIT-licensed, zero-dependency, and self-contained. You use it as a paste-block in a system prompt or AGENTS.md, or you wire it as a session-start hook so it loads itself every time. It’s harness-agnostic — Claude Code or any other agent.

And here is the honest state, since honesty is one of the four rules the thing describes: it’s brand new. Zero stars the day I write this. The falsifiers are enforced by reading today — the mechanical ones (a stray debug print, green over red) could be enforced by tooling, and aren’t yet. It’s a design and an argument, not a finished framework with a chart to prove it. What I’m genuinely unsure about is whether a named discipline survives a full context window better than a numbered list. My bet is yes. I haven’t measured it.

If you run agents in production and want to tear the idea apart, I’d take that as a gift. The repositories are below.

Sources & links