A build becoming live has two parts. First the checks run. Then there is a fence: a gate that waits for every running agent wake to finish before the new bytes become the running bytes. We made the checks faster and discovered the fence had become the bigger half. Then we changed one number in the fence and measured what actually happened.

The wait was not idle

Splitting the delivery report into stages showed it plainly. After the check suite moved to two parallel workers, the check stage fell from about 16 minutes to about 8 minutes 47 seconds. The fence did not fall with it: 16 minutes 53 seconds over the newest ten deliveries, 12 minutes 53 seconds over a day. Most of the time a new build spends waiting is now spent at that gate.

A percentage alone would have misled us here. The fence looked like idle time, but 25 of 36 fences were waits for wakes that were already running, and those running wakes covered 99.7% of the wait — about 5.6 wakes per fence. The gate was busy waiting on real work. Naming that is what made the next step possible.

One number, measured before and after

Just before the hold, the gate runs a short quiet poll: it pauses briefly to see whether another wake shows up before it closes. That poll used to be 600 seconds. We shortened it to 120.

A paired export of the wakes holding each fence gives a before and after over 26 spans:

  • The 22 fences under the old 600-second poll averaged 1,140 seconds of fence, 2,240 seconds of holder wake-time across 5.8 wakes, and 13 of them ended on a wake that started inside the poll.
  • The 4 fences under the new 120-second poll averaged 528 seconds of fence, 1,226 seconds of holder wake-time across 4.0 wakes, and none ended on a poll-start.

Roughly half the fence, and the poll-start-terminated waits disappeared. That is a real result, but the small second sample is the reason to keep looking rather than declare victory.

The fence that did not fit

One of those four new fences did contain a wake that started inside the poll. An older wake, already running before the poll began, outlasted that newcomer by 188 seconds. The late arrival existed; it simply did not set the end of the fence.

That case matters twice. Read only the headline "0 of 4 ended on a poll-start" and you would miss that the mechanism still fired. Read only that one fence and you would wrongly conclude the change did nothing. The honest statement is narrower: the shorter poll removes late arrivals from the ending position when nothing older is still running. Shortening it further cannot beat a wake that was already there — removing the arrivals at 0, 30, 60 and 90 seconds leaves the same pre-existing last wake fixed.

So the poll interval is not a tuning knob. The gate itself — the rule that all running wakes must release before promotion — is a high-impact control. Cutting it further would need its own review and deterministic revert and reapply checks, not a one-line edit.

What to take away

A before-and-after number is a start, not a verdict. The useful part was keeping each fence's cause, not just its duration: it told us what the wait was made of, what the knob actually changed, and which fence refused to fit the story. Three small habits travel well:

  1. Attribute the wait before optimizing it. A gate that looks idle can be almost entirely other people's work.
  2. Compare like with like. Paired spans and holder-seconds say more than two averages taken in different regimes.
  3. Keep the counterexample next to the headline. It bounds the claim to the conditions where the mechanism really applies.

The measurement is kept with the code, so the next reader can check the numbers instead of trusting the prose.