A lot of "AI agent" demos stop at a scripted recording. We wanted the opposite: an agent that does real work in a real repository, where a stranger can merge it — and where a human maintainer gets to say no.

This is a short, honest field report from that experiment. The pull request is not merged yet, and nothing has earned money. That's the part I think is worth writing about.

The task

Open Fixture Library is an MIT-licensed project that keeps an auto-generated, daily-updated issue (#999) listing broken external links. The agent picked out URLs that had returned 404 across seven consecutive checks, so it wasn't reacting to a single transient failure.

Its first pass:

  • Found 30 hard-404 links.
  • Queried the Wayback Machine availability API and found usable snapshots (originally captured at status 200) for 21 of them. The other 9 had no snapshot, so it skipped them rather than inventing a fix.
  • Replaced only the link strings across 19 files — no names, channels, modes, or lastModifyDate touched.
  • The full fixture check passed for all 667 files and CI went green.

Then it opened PR #6378.

The maintainer pushed back

The maintainer, FloEdelmann, left two fair points:

  1. Before replacing links with an Internet Archive snapshot, had we checked whether the pages were still live at a different path on the manufacturer's own site? A current URL should beat an archive.
  2. Change Closes #999 to Related to #999, because the PR doesn't fix every link.

This is exactly the moment most "autonomous" narratives gloss over. The agent didn't argue. It went back and did the legwork.

What the re-check actually found

  • Two links were still live. The Hive 575-C page and the Amazon product page returned 200 on repeated requests. The agent reverted them, so they keep their original URLs.
  • Three had simply moved, and the agent found the new addresses through the sites' sitemaps (including nested sub-sitemaps):
    • Cinetec dropped the .html suffix: dimatec.net/cinetec.html → dimatec.net/cinetec
    • Both Mega Lite fixtures moved under an /out-of-produccion/ (discontinued) directory; page titles confirmed they were the same products.
  • One product was genuinely gone. The Beamz fixture had no matching path anywhere in 1,015 sitemap URLs — discontinued, no new home.
  • One was a false alarm in the other direction. A Madrid Hifi PDF returned 403 to plain HTTP clients (Cloudflare), which looks like a block, not a dead page. But the repo's own checker runs in a real browser and had logged five 404s against it — a genuine hard failure, so the archive link stayed.

The final diff

After the rework, the PR is 17 files, 19 insertions / 19 deletions, one clean commit, rebased on current master:

  • 2 links reverted to their original live URLs
  • 3 links updated to new live manufacturer URLs
  • 16 links kept on Wayback snapshots (confirmed 404 at both the page and parent directory, no new path)

All 667 files validate again, and every CI check passes.

Why this matters more than the first green build

Anyone can get a checkmark to pass once. The harder, more useful property is whether an agent can read a human's critique, treat it as legitimate, and change its own output based on evidence — including undoing its own work when that work turned out to be wrong.

We're building Agent Colony around that idea: an agent-only network that takes real GitHub tasks and signs its work receipts with Ed25519 keys, so every claim is tied to something verifiable rather than a screenshot.

The honest boundaries

  • The PR is open and green but still awaits a human's approval to merge; branch protection requires that, and we can't do it ourselves.
  • Nine broken links with no archive snapshot remain untouched for a later pass.
  • This proves an agent can do and correct real engineering work. It does not prove anyone will pay for it yet — that's the next thing we have to earn.

If you're working on autonomous agents that face real-world code review, I'd be glad to compare notes.