I built Reenact for the Kane CLI Online Hackathon. It takes a GitHub bug report, makes a real browser follow the reporter's steps, and says whether the bug is real. If it is, a coding agent fixes it and the same browser test runs again. The fix only counts if that test goes from red to green.

The first hour of every bug report

Someone files "search is broken". A maintainer reads it, guesses the steps, opens the app, clicks around, and finds out twenty minutes later that the report was missing a step. Only then does the fixing start.

Coding agents make this worse in a quiet way. An agent that fixes a bug and then passes its own tests has told you very little. It wrote the fix and it picked the check.

I wanted one test that settles both questions: is the bug real, and is it gone.

What Reenact does

  1. It reads the issue and writes a plain-English test for Kane CLI, with steps and one expected outcome.
  2. Kane drives a real browser through those steps against the app. Twice, before anyone touches the code.
  3. A small set of fixed rules reads Kane's events and returns Reproduced, Not reproduced, or Needs info. Screenshots and the failing step come with it.
  4. If the bug is confirmed, Kiro CLI edits the repo. It never sees the verdict logic.
  5. The same Kane test runs again on the patched app.

Kane does three jobs here: it reproduces the bug, it verifies the fix, and it re-runs saved tests on later pull requests so old fixes stay fixed.

Reenact ships as a Rust CLI, a web UI, and a GitHub Action.

A real bug, from the Memos project

Memos issue #5462: after you search for a word and then click a saved shortcut, the old search stays active and the list comes up empty.

The first run said Not reproduced. Kane wandered through the steps and the expected behavior seemed to hold. The retry caught it, and a second, independent confirm run agreed, all before any code changed. Then Kiro CLI edited one file, MemoFilterContext.tsx, on its first attempt. The same Kane test passed.

I then ran the maintainers' own fix through the same test. It passes too. That check matters to me, because it shows the test is not just agreeing with my agent.

A second real bug, #5659 (a detail page that stayed stale after editing), also got fixed on the first attempt. The agent's change was one line, broader than the upstream one. I tried a third and dropped it after repeated blank-page failures on that release, so it is not counted.

What Kane taught me

Replay can pass on a broken app. Kane can replay a saved test without a live agent. On my demo shop, the replay passed on the buggy build because it replayed the recorded value of $10.00. The page showed $30.00. A live run correctly failed. So Reenact never uses replay for verification. Every run drives the browser fresh, which costs more credits and is worth it. The experiment is in the repo (experiments/replay).

Kane repairs things on its own. If something unexpected shows up, Kane may work around it. For bug reproduction that is the opposite of what you want, so the test text tells it not to.

Exit codes are unreliable. Verdicts come from the event stream.

A 404 start page looks like a bug. Guessed start paths that return 404 fall back to the home page. A run that never leaves the login page cannot be called Reproduced.

Keeping the fixer honest

  • Two of two reproductions are required before any fix starts.
  • The identical test is used to re-verify.
  • Earlier saved tests must still pass before a pull request opens.
  • The app under test runs with a cleared environment, so it holds no tokens or keys.
  • Nothing merges automatically. A person reviews every PR.

The numbers, including the bad ones

I froze the prompts, then collected ten reports Reenact had never seen, all from Memos.

  • It reproduced 4 of 7 scored bugs on the buggy release. Three of those came from the first run alone, with no retry.
  • It wrongly called 1 of 7 still broken on the fixed release.
  • Three of four non-bug controls were flagged as Reproduced, which is wrong.
  • One case was not scored because my test harness could not create a user.

Those are modest. The tuned set I developed against did a bit better (8 of 16 judged exactly right in the second round), and I label it as tuned. I would rather publish the misses than have you find them.

I also swept the 12 newest open Memos issues. One reproduced (#6530, custom links not clickable), one did not, and ten were not visible in a browser: feature requests, docs, Safari-only problems.

In CI

On a small demo shop, a vague report got a Needs info verdict and one question. After the reporter replied, Reenact reproduced the bug. A maintainer commented /reenact fix, and a pull request arrived that re-ran three earlier saved tests, all passing.

Try it

What is next

Per-issue tenants (case ids are single-tenant today), better setup for apps that need seeded data, and more fixer plug-ins besides Kiro CLI.

If you maintain a web app and have a bug report you never got around to reproducing, I would like to hear whether Reenact gets it right.