Our coding agents skipped the pre-push hook on 118 of 411 pushes

This week a lot of people on X argued about "software factories" for AI coding. Some companies sell pipelines of agents, and their critics say most of it is marketing and the parts that work are the deterministic ones.

We run something close to a factory. Coding agents write most of the code for our TypeScript framework, Kumiko, and the apps built on it, and an automated review has read 5,322 of our merged PRs since May 27. The rules for the agents are about 3,100 lines of Markdown in 32 files. I went through our agent transcripts to see which of those rules held. The critics are right, but "deterministic" covers less than I thought.

A note didn't hold

On November 1 we wrote down that the main agent starts coding agents itself, with no Opus agent in between that only passes the prompt along. Between then and the evening of November 5, 22 general-purpose agents started a coding agent anyway, 53 times in total.

That evening we added a hook that refuses the spawn when a subagent tries it. It has fired once since, in our own test. In the two days after, coding agents were started only by the main agent or by the one exception we allow.

So the hook can't take the credit, at least not yet. The same commit rewrote the rule, and something the hook can't touch changed too: the main agent used to hand work to a general-purpose agent about 20 times a day, and since then it's about 3 times a day. The hook only checks spawns from subagents. The drop came from the rewritten Markdown, or from a different mix of work in those two days.

A hook didn't hold either

Our pre-push hook runs typecheck, lint, unit tests and our own guards. Skipping it with --no-verify or PRE_PUSH_SKIP=1 is on the short list of hard bans in our agent config, with one exception: the failure is outside your diff, you can show who caused it, and you ran the skipped checks on your own files.

Since October 20 our transcripts contain 411 push commands, and 118 of them skipped the hook.

I read the reasons the agents wrote next to the skips. Most point to red outside their diff, often with the three proofs listed. Some name tests that time out when the machine is busy, and the exception explicitly excludes flaky tests. Some give no reason at all.

The infrastructure repo explains most of the skips. Its hook also scans the neighbouring repos, so it goes red on code the agent never touched. When a gate keeps failing for someone else's change, the exception gets used on every push, and after a while that's simply how pushing works in that repo. A memory note our agents keep for themselves even called PRE_PUSH_SKIP=1 the documented pattern. I corrected it today.

What held

The checks on GitHub. In every repo those pushes went to, main only takes a merge after the required checks pass, and the rule applies to admins too. All 118 skipped pushes still had to get through CI before anything merged. An agent on a laptop can't skip a check that runs somewhere else.

Our cast audit shows the same thing. It counts suspicious as casts in the framework against a frozen baseline and fails when a change adds one. It runs inside the same pre-push hook, so it got skipped along with everything else, but CI runs it too. The baseline stood at 349 in June, after we extended the audit's coverage, and it's at 69 now.

What I'd tell someone building one

Put the check that must hold on the server. Keep the local hook fast, and let it look only at the diff being pushed, or it will cry wolf until nobody listens. Write the Markdown anyway, as the reason behind a check, but don't count on it to be the check.

Speed matters for the same reason. When we moved our per-file guards into Biome plugins on November 4, lint got two to three times slower. A slower hook is one more reason to skip it, and we haven't fixed that yet.

What this doesn't prove

The numbers come from our agent transcripts in this workspace, counted per tool call. A push that was retried counts twice.

The reasons come from the short notes the agents wrote next to each push. I didn't replay the sessions, so I can't say how many of the claimed three proofs would hold up.

The spawn hook has two days of data and hasn't been triggered by real work yet.

We didn't compare bug rates before and after any of this, so I can't say what the skipped hooks cost us or how much CI caught in their place.

A question

If you run agents on your repos, which check do you treat as the one that can't be skipped? And how do you keep the local one from going red on code the agent didn't touch?