Press ESC to close

The Software Factory You Didn’t Notice You’d Built

Sensible automations, each gated on its own, add up to a production line no one designed.

Every automated loop in your delivery pipeline was reviewed, gated and argued over before it went in. Wired together, one sensible addition after another, they have become a production line that nobody sat down and designed, and that nobody governs as a whole.

When this series opened with the retired human checkpoint, it ended on a warning: find the loops you have already closed before they start feeding each other. You can still reason about a single automated loop. Once a system of them is wired end to end that gets much harder, and usually no one notices the moment it does. The two pieces since took loops one at a time: the dumb loop that works, and the loop that rewrites its own instructions. This one is about the wiring. One loop’s output is the next loop’s input, and enough of them connected end to end make a production line.

The line assembled itself

No single decision built it. A retry loop wired to the test suite, because retrying against a verifier works. An optimiser that rewrites the instructions, because it arrived in a library upgrade. An agent opening its own pull requests and a review agent reading them. Auto-merge on green because the queue was the bottleneck, and an incident bot to roll back what the merge bot shipped. Each addition was sensible, and each was reviewed and gated in its own right.

Now trace one change from ticket to production and count the human reads. The plan one agent wrote was consumed by the coding agent. The code the coding agent produced was read by the review agent. The review agent’s verdict went to the merge gate. Every artifact still exists, but in the normal run each handoff is machine to machine. The reading that used to happen between stations has stopped, and no one announced it, because no step changed. Only the connections did.

In June, a team from Renmin University and Microsoft Research published Arbor, a research agent that beat Claude Code and Codex by 2.5x on the same compute. Its architecture is worth reading slowly. A long-lived coordinator holds the strategy and a growing tree of everything learned so far. Short-lived executors take jobs, run experiments in isolation and report evidence back. Coordination, dispatch, workstations, reporting lines, institutional memory. It is a plant. The difference between Arbor and your stack is that someone designed Arbor as a whole, and knows they are operating one.

Where the loop discipline runs out

A real discipline has grown up around single loops this year: verifiers in the loop, loop contracts, gates that halt and ask a human. Sonar’s line that loop engineering without verification is just automation is the same claim I made about the dumb loop, and the people building this tooling are right. If your loops lack verifiers, fix that first.

That discipline stops at the edge of each loop, and the wiring between them has none of it.

Three things exist at line level that no loop contains.

Throughput. Your delivery rate used to be set by how fast people could read. The humans were the rate limit, and without anyone writing it down they were also the inspection regime. Remove them from the handoffs and the line runs at machine speed. Volume is the point, and it is also what guarantees that any review process bolted on afterwards will never keep up. The line does not idle.

The handoffs. Every checkpoint you retired was a person who read an artifact before passing it on. The reading was never in anyone’s job description. It was invisible work, and it produced your real audit trail as a by-product of the line being slow. Full automation removes exactly that: the humans doing work nobody had specified. What replaces them is an open question, and I am going to leave it open.

Ownership of the outcome. When the shipped thing is wrong, which loop failed? The optimiser tuned the prompt against its score, and the score improved. The verifier ran the tests, and the tests passed. The merge gate was green. Every station did its job and the line still shipped the wrong thing. Responsibility is clear inside each loop and absent for the composition of them, which no one was ever asked to sign off.

Even the good loop-level fixes stop composing. When I wrote about the loop that rewrites its own prompt, the remedy was to build the audit surface back in deliberately: decide which prompts may rewrite themselves at all, record the exact words from every run, put a human gate in front of the rewrites that matter. Every part of that was buildable because the state that loop learned is text sitting where the prompt goes. Arbor’s tree gives the gate nothing to stand in front of, and no pile of station-level records, however complete, reconstructs what the line decided. A fire extinguisher on every machine is not a fire strategy for the plant.

The questions that only exist at line level

What does control mean across a line? Every loop has its own stop button. Pressing one halts a station and leaves work in flight at every other, and whether the line as a whole can be stopped safely is an architectural property nobody chose, because nobody knew they were choosing.

What does audit mean when the artifacts that matter cross handoffs no human reads? Storing them all is easy now. Reading them was the control, and the reader is gone.

Where does responsibility live when no single loop owns the outcome? Naming an owner for each loop just gives you a list of names.

I am not going to resolve these here, because each one is a different subject rather than a loop question scaled up, and each deserves more than a closing paragraph.

Manufacturing has a name for a plant that runs with nobody inside: a dark factory. The lights are off because no one is left who needs to see. In manufacturing that is a deliberate achievement, decades of discipline, commissioned as a whole and operated by people who know exactly what they are running. Software is arriving at the same place by accretion, one sensible loop at a time, which means the factories will be running before their owners notice they own one.

What to do before the next one

None of this is an argument to stop automating. It is an argument to treat the whole line as the thing you now own and govern, the same way you already govern each loop sitting on it. Draw it. Find every handoff a human used to read and no longer does. Decide, on purpose, where the line as a whole can be stopped, what record of its decisions you could actually reconstruct, and who signs off the composition rather than the parts. You built the factory one sensible loop at a time. Governing it is the one job you cannot automate your way out of.

Yours may already be running. That is where this series goes next: what a dark software factory looks like from the inside, who the last human before production is and what breaks first when you get it wrong.

Sources

Related reading (Synaptic Pixels)

Leave a Reply

Your email address will not be published. Required fields are marked *