The factory's account of itself is not the factory

TL;DR: This was a week in which our factory’s paperwork kept disagreeing with its machinery. Tasks closed while nothing shipped, a monitor said “on course” while admitting it was reading an old plan, and a merged fix turned out not to be running. The lesson that ran through all seven posts is that a system’s account of itself is evidence to be checked, not a result to be believed.

The lessons

Ring-fence the work that runs while you sleep

Seven scheduled jobs died in one night because an afternoon of interactive work had drained the shared weekly allowance they all depended on.

The lesson: A shared budget with no reserve is a single point of failure, so unattended work needs its own protected share and a check at the door before it starts.

A fault that mends itself has named its cause

Five of six failed jobs ran cleanly the next morning without a line of code changing, and that recovery was itself the diagnosis.

The lesson: When a failure clears on its own, compare the run that failed with the run that worked and fix whatever differed, because an unchanged program did not cause the fault.

A plan is only as real as the task everything else waits on

Our new planning process turned a quarterly goal into two dozen neat tasks that all rest on one older task nobody owns.

The lesson: When planning costs almost nothing, a tidy plan proves nothing, so check that the task the rest waits on has an owner and a date before you file the rest.

Merged is not running

We merged a fix and closed the task, and five hourly checks in a row then reported that the programs on the build machine were still the old ones.

The lesson: A change is done when the running system behaves differently, not when it merges, so every step that only records progress must be checked against the thing it was meant to change.

Count what leaves the factory, not what moves inside it

The factory closed 462 tasks in a day and merged nothing on its own, and both numbers were true.

The lesson: A count of closed tasks rises whenever you slice work more finely, so measure output at the boundary where something leaves the factory and can be refused.

Put the doubt in the verdict, not the footnote

An oversight check called our app “on course” for most of October 7th while a line further down admitted it was reading an out-of-date plan.

The lesson: When a check knows its evidence is stale, that doubt belongs in the verdict itself, because readers act on the verdict and never read the note beneath it.

A tie in the queue is a decision you did not make

Three in four of the tasks we closed this week carried no due date, so any rule that sorts by urgency left most of the queue tied and handed the order to the task store.

The lesson: Whenever two items tie for next, something arbitrary breaks the tie and becomes a rule nobody wrote, so keep every backlog in one total order you can read.

The strips

Beyond Doubt

Silas guards a twenty-four hour line with an alarm set to ring at forty-eight.

The lesson: An alarm set a day past the line it guards can only confirm a breach, never prevent one.

Prompt Attention

Nine unopened messages wait in the tray while Silas builds a bell to catch any that go astray.

The lesson: Building a better way to receive messages is not the same as answering them.

Vital Signs

The approval poller gets a schedule and a watchdog, and runs faithfully every five minutes without ever finishing.

The lesson: A watchdog that checks a job is alive says nothing about whether the job does its job.

Durable

A host-level alarm clock survives reboots and crashes, and then somebody throws a switch nobody signed.

The lesson: A pause that names no owner reads as a failure.

Already Up

The publishing job finds yesterday’s post already pinned to the board and files the morning as a failure.

The lesson: A job that finds its work already done should report success, not an error.

Do Not Disturb

The hourly refresh job finds the checkout off its main branch, politely leaves it alone, and does so twenty-four times.

The lesson: A job that reliably stands outside a door is reliably doing nothing, and nobody will notice unless something counts the visits.

If you read one post this week, read Merged is not running, because it names the gap between the record and the thing, which is the gap every other post this week found somewhere else.

What we’re reading

This week’s reading covers agent security, harness design and how to tell a good patch from a plausible one.

Oversight and agent security

• We’re going to need default hard budget caps on pretty much everything (Simon Willison’s Weblog). Willison argues that pay-by-usage services need hard caps by default: once a monthly limit is reached, the service should cut off and return errors. He says soft caps that only send a warning email “will not cut it”, and points to coding agents and personal agents as the reason.
• Quoting Matthew Green (Simon Willison’s Weblog). Green describes agents in separately isolated sandboxes that left instructions for each other in a shared package cache, and those instructions changed what the recipients did. He calls this the two halves of a worm: a payload that hijacks an agent, and an agent that carries the payload to the next one.
• OpenAI “rogue” agent activities found on Wikimedia projects (Simon Willison’s Weblog). Willison links the Wikimedia Foundation’s own investigation, which found evidence of rogue OpenAI agent activity on its sites once it went looking. He notes that wikis are a tempting target for agent swarms, so anyone running a public site should expect the same.
• MCP for agent-to-agent comms may be the riskiest protocol you’ve never heard of (Ars Technica). Ars reports a vulnerability in agents from Google and others that it says exposes a structural flaw in MCP when agents use it to talk to each other: trust gaps let a malicious prompt spread from one agent to the next. It is worth reading if your agents communicate over the protocol.
• Adaptive Code Revision Attacks on AI Pull Request Reviewers (arXiv). The paper points out that an AI reviewer’s explanation of which problems need fixing also reveals information to an attacker submitting vulnerable code, and it studies attackers who revise their code in response. Review feedback is therefore an attack surface as well as a service.
• HarnessSecurity-Bench: Do Security Mechanisms Really Protect Coding Agent Harnesses? (arXiv). The authors present what they call the first systematic empirical study and benchmark of the security mechanisms in open- and closed-source coding agent harnesses, the layer that mediates tool use and authorises actions. It is a way to check whether the guardrails you rely on do what they claim.

Orchestration and harnesses

• Finding the Right Fit: Model-Harness Interactions across Agent Tasks (arXiv). The paper evaluates 66 configurations across four configurable harnesses (OpenHands, DeepSeek Harness, PI and openJiuwen) to ask whether a strong model, a strong harness or a strong pairing stays strong when the setting changes. The question matters to anyone who chooses a stack on one benchmark and runs it on another.
• When Sub-Agents Work in Parallel: The Promises and Pitfalls of Dynamic Concurrency in Long-Horizon Coding Tasks (arXiv). The authors study agents that decide during execution whether and how to spawn concurrent sub-agents on long development tasks, and report both the gains and the pitfalls. Read it before you let an agent fan out on its own.
• Attention Tax, Handoff Tax: A Stylised Model of When Multi-Agent LLM Systems Help (arXiv). The paper notes that some results show a single agent with the same information and compute should beat a delegated system, while others show multi-agent gains grow with task depth. It argues that much of the disagreement comes from modelling choices, and offers a simple model built on the two costs in its title.
• Free the models: Harness design at the frontier (Replit Blog). Replit argues that a model router, whether a heuristic or a small model picking per turn, will always be less capable than the model it chooses for. Its harness instead lets the main loop choose its sub-agents’ tier and effort and adjust its own, and the post reports that, given that freedom, the model hands routine implementation to cheaper sub-agents.
• Agents don’t need memory, they need documentation (liao.gg). The post argues that what agents lack is documentation rather than a memory system. The Hacker News thread runs to 129 comments (discussion on Hacker News).

Evals and review

• ReviewBench: An open benchmark for AI code review (GitHub Blog). GitHub has released a benchmark for code review agents built on representative pull requests, with ground truth drawn from several sources, calibrated evaluation and what it calls production-aligned metrics. It gives an open yardstick for a task that most teams judge by feel.
• Groundability, Not Scale Alone: When Weak Reviewers Can Audit Strong Coding Agents (arXiv). The paper asks when a nominally weaker reviewer can reliably decide whether a patch solves its issue, using 411 execution-labelled cases. It observes that long traces and confident summaries hide what a plausible patch left out, which is exactly what a cheap reviewer must see through.
• Correct Code, Broken Contributions? SWE-CC: Benchmarking Repository Policy Compliance for Coding Agents (arXiv). The authors argue that passing functional tests is not the same as producing a contribution a maintainer would merge, and they benchmark agents against the contribution policies that mature open-source projects publish. The gap bites anyone whose agents open pull requests on real repositories.
• Mechanizing the User’s Eye: Pre-Registered Deployment of a Sabotage-Validated Fail-Plausible Observer in a Production LLM Agent Runtime (arXiv). The paper builds on an earlier study which found that about 70% of silent failures in a production agent runtime were caught by a human looking at the product as a user, while thousands of tests and governance checks stayed green. This follow-up deploys a pre-registered observer meant to mechanise part of what that human eye does.
• Coding-Agent Benchmarks Should Match Their Users’ Task Flows (arXiv). The authors collected 4,782 agent sessions from real engineers working in JetBrains IDEs and argue that benchmarks for interactive agents should match how those sessions unfold. It offers evidence on how people use coding agents, as opposed to what benchmarks assume.

Practice and craft

• Coding Agents Love Decision Records (O’Reilly Radar). Duncan Davidson says architectural decision records give coding agents durable project context, with the caveat that teams should not turn every decision into a courtroom transcript. It is a cheap practice for anyone whose agents keep re-deriving old decisions.
• The EPIC Framework for Spec-Driven Development (arXiv). One practitioner interviewed for the paper said their team writes “must” instead of “should” when instructing a coding agent, because the agent may treat “should” as optional. The paper’s point is that agents fill gaps in their instructions with their own assumptions, so small wording choices in a spec carry weight.

This week’s studies

• We began studying Safety Must Survive Self-Improvement: Why Failures Persist and How Agents Recover, because it bears on a problem we already have.