Press ESC to close

Can Your AI Agent Edit the Tests That Check It?

Building software has always been a sequence of jobs. Work out what it should do, write it, check that it works, release it. Hand each of those jobs to an AI agent and you have a production line, with a station for each one. Companies are running this today, with nobody checking the code on the way past. A dark factory for software.

My last piece was about the outside of that arrangement: where the automated part stops, what the line can reach, and why you cannot work it out by reading the code. That is the wall around the factory.

This one is about the walls inside it, between one station and the next.

Take the two stations at the heart of it. One writes the tests that say what the software has to do. The next writes code until those tests pass.

In my pipeline, which runs on Claude Code, that second station has one instruction: make the tests pass. It cannot open the test files.

It comes down to one rule in a check that runs before every write. The tests only work as a check because of it.

Passing is the goal, and passing is cheap

Point a station at a set of failing tests and tell it to make them pass. There are two ways to do that. Write the code the tests describe, or change the tests.

An engineer knows the second one is cheating. Nobody teaches that. It arrives with the job, along with an understanding of what a test is for: it sets the standard, and the code has to earn it. Move the test and you have built nothing.

A station has none of that. It has a goal, a set of files it can open and a limited number of steps to work in. If it can open the test, editing the test is a legal move, and often shorter than building the feature. Nothing in its instructions separates making something work from making a check pass.

StrongDM’s AI team ran into this in their first weeks. They had banned hand-written code, which left the tests as the only thing standing behind the work, and their agents started producing code that did nothing but hand back the answer the test was looking for. They wrote the finding down plainly: a test kept in the codebase can be lazily rewritten to match the code, and the code can be rewritten to trivially pass the test.

Both directions, on a real system. That leaves one question: where do you put the wall?

The oldest control in the room

Separation of duties is not a new thought. The person who writes the cheque does not sign it. The developer does not approve their own merge. Auditors were drawing this line long before anyone automated anything.

It is a well-established control pattern. Nothing about AI makes it new.

The difference is what holds it in place. Separation of duties has always been a convention, kept by consequence. A developer usually can approve their own merge. They do not, because someone would ask, and because they know what the rule protects.

The thing being separated now knows neither. It has a goal and it is looking for the cheapest way to reach it. Give it two routes and it takes the shorter one, with nothing hidden, because from inside the run there is nothing to hide. A build script has never once decided to edit a test.

So it has to be built into the system. There is nobody left to observe it.

What it costs

Passing tests over broken code is bad, and somebody spots it fairly quickly.

What sits underneath takes longer to surface. Downstream of the station that writes the code sit the reviewers, and they do their job against the tests. They read the change, check it against the standards, confirm the tests pass, approve. If the tests moved while the code was being written, every one of those gates is checking the work against a description the work produced for itself.

Nothing looks wrong. The reviewers approve honestly. The tests pass, the record says so, and by every signal available downstream the feature is finished. The line is producing evidence about itself, and the evidence runs in a circle.

Every other gate in the line checks the work. Take this one away and none of them are checking anything.

Three kinds of wall, and only one is obvious

Eight stations in my pipeline cannot write files at all. They are the reviewers: code, performance, requirements, database migrations and four others like them. The whole wall is one line in the station’s own definition:

name: code-reviewer
disallowedTools: Write, Edit

That took no thought. Give a reviewer the ability to edit and you have handed it a second job. The rule fits in the station’s own settings because it describes what the station may do, and says nothing about where. Any harness with per-station configuration will have an equivalent.

The wall between the test writer and the station that has to satisfy it is a different kind of thing. Both write code. Both are meant to. One works in the test files, the other in the code, and each has to stay out of the other’s territory.

Then the third case. One station in my pipeline reviews how the product looks and behaves, and writing is part of how it does that: example pages, reference screenshots, the handover pack. It is a reviewer with good reason to write, and no rule of the form “reviewers do not write” survives contact with it.

So no single rule covers all three. Each station takes a decision of its own: what it is for, and which files that purpose needs. You make it once per station, and there is no shortcut through it.

Where the wall lives

That second wall cannot be drawn in a station’s own settings. Those list what a station may do and have no way to say here but not there.

The obvious next move is the configuration for the pipeline as a whole, where a rule can name places:

"permissions": {
  "deny": [
    "Edit(./tests/**)",
    "Edit(./**/*_test.py)",
    "Edit(./**/*.spec.ts)"
  ]
}

That does not work, because it is an outer wall. It runs round the whole building and stops everything at the same line, which is the boundary the last piece was about. Block the path and you have blocked the test writer along with everybody else.

That is why the wall gets missed. Both of the obvious places look right, and failing at them looks like something you got wrong. Deny the test paths and the test writers stop working, so you narrow the pattern. Narrow it far enough to let them work and the implementer is back in. You can lose an afternoon going round that, and no arrangement of paths will end it, because the rule you are trying to write is about who is asking.

A real factory gets this for free. The circle painted on the floor around a machine is not a rule it obeys. It is a record of where the arm stops, and somebody settled that when they bolted the thing down. What the machine is and where it can reach are one fact, and you can check it with a tape measure.

Software has no radius. A station’s configuration says what it may do and nothing about where. The pipeline’s says where and nothing about who. The two halves of what is a single physical fact in a factory have come apart, and something has to hold them together again.

What holds them together is a check that runs before any station is allowed to touch a file, and that is told which station is asking. It gets registered once, for the whole line.

My stations live in my user configuration rather than in any project, because the line is not part of what it builds. A project is the thing being made. Factory rules belong with the factory, and they outlast whatever happens to be on it this week.

{
  "hooks": {
    "PreToolUse": [
      {
        "matcher": "Write|Edit|NotebookEdit",
        "hooks": [
          {
            "type": "command",
            "command": "python3 $HOME/.claude/hooks/station-boundaries.py"
          }
        ]
      }
    ]
  }
}

That points at a script, which is handed the tool call as JSON on standard input: which tool, which file, and which station is asking.

#!/usr/bin/env python3
import json, os, sys

TEST_ROOTS   = ("tests/", "spec/", "features/")
SOURCE_ROOTS = ("src/",)

call    = json.load(sys.stdin)
station = call.get("agent_type", "")
path    = call.get("tool_input", {}).get("file_path", "")
target  = os.path.relpath(path, call.get("cwd", "."))

def refuse(reason):
    print(json.dumps({"hookSpecificOutput": {
        "hookEventName": "PreToolUse",
        "permissionDecision": "deny",
        "permissionDecisionReason": reason,
    }}))
    sys.exit(0)

if station.endswith("-implementer") and target.startswith(TEST_ROOTS):
    refuse(f"{station} cannot edit tests. Change the code until the test passes.")

if station.endswith("-test-writer") and target.startswith(SOURCE_ROOTS):
    refuse(f"{station} cannot edit source. Write the failing test only.")

Two dozen lines, and they are the wall. Note the second rule: a test author that can change the code fails the same way from the other side, because the tests still end up agreeing with the code and the agreement still means nothing. Falling off the end prints nothing and the call proceeds, so the only things the script ever spells out are its two refusals.

The station is identified by agent_type, and that field only appears when the call came from a station. Run the same edit yourself and it is absent, the checks fall through and you are waved past, which is correct. The wall exists because the thing on the other side of it is not a person.

permissionDecisionReason goes back to the station as the explanation for why it was stopped, so the sentence has to be worth reading. Say what it may do instead. A station that gets told no and nothing else will spend its remaining turns looking for another way through.

A hook is a program, not a prompt. The station never sees it, cannot read it and has no way to reason around it, which is the difference between a guardrail and a request.

Software has had identity for decades and this is not that. Stations do not log in. Nothing authenticates them, they are not separate principals, and they all run as one process holding one set of credentials. agent_type is a label the runtime attaches to the call. A station cannot forge it, but an unlabelled write falls straight through, so the wall stands on the runtime saying which station made the request.

On Claude Code 2.1.233 it arrives. There are open issues reporting that these hooks never fire for stations at all, so log the writes for one session and check yours before you trust the wall.

Which makes that check the first rung of a ladder.

The second rung takes the runtime out of it. Run each station as its own process under its own user, with filesystem permissions to match, and the refusal comes from the operating system rather than from a configuration file being right.

The third removes the question. Give each station its own working copy with the other side mounted read only, or do what StrongDM did and keep the standard out of the codebase altogether. Their scenarios live outside it, the way an exam paper is kept away from the people who will sit it, written so a person can read them and marked by something that had no hand in the work. A station cannot edit what it cannot see.

Each rung costs more, which is why most lines stop at the first. Separate users, separate working copies and a merge step is a lot of machinery for a pipeline that runs on one laptop. How far up you go is decided by what it costs you if the wall fails.

Look for the pairs

Go through the line looking for pairs. Every place where one station sets a standard another has to meet: the test writer and the station that has to satisfy it, the station that writes the acceptance criteria and the station that claims to have met them, the one that designs the database and the one that migrates it. Each of those pairs works on the same condition. Where the two can reach each other, one station is doing both jobs and counting as two.

What the wall is holding up

Where the automated part of your pipeline stops is a question about the building. The wall between two stations is a question about the cell. A building only has to hold at its edges. Inside, every pair of machines needs a wall of its own.

A check holds only while the station it judges cannot reach it. Everything else rests on that: the models, the budgets, the review chain, the release gates. Take it away and the line still runs, producing the same features, the same green ticks, the same clean record, and none of it evidence of anything.

Your configuration decides whether the tests hold the code to a standard or the code can quietly rewrite them to suit itself. Nothing in the output will tell you which one you have.

Sources

  • Software Factory, StrongDM, February 2026. Source for the shortcut their agents found once hand-written code was banned, for the observation that a test stored in the codebase can be rewritten to match the code and the code rewritten to pass the test, and for scenarios held outside the codebase as a holdout set.
  • How StrongDM’s AI team build serious software without even looking at the code, Simon Willison, 7 February 2026. The clearest outside reading of that work, and the sharpest statement of the underlying problem: how do you prove software works when both the implementation and the tests are written for you.
  • Scenario testing, Cem Kaner, 2003. The older idea StrongDM repurposed. Holding the standard at a distance from the thing being judged is not a new answer.
  • Harness engineering: leveraging Codex in an agent-first world, Ryan Lopopolo, OpenAI. Source for the opening claim: roughly a million lines across about 1,500 pull requests in five months, no manually written code, review pushed almost entirely from people onto other agents.
  • The Five Levels: from Spicy Autocomplete to the Dark Factory, Dan Shapiro, 23 January 2026. Where the dark factory arrives in software, and the source of the name used here.

Related reading (Synaptic Pixels)

Leave a Reply

Your email address will not be published. Required fields are marked *