I asked a Pi agent to test LessCode against real pull requests. It sent one subagent to find TypeScript repositories and candidate PRs, then assigned the test runs to other subagents.

That was the moment I thought, Okay, this is doable. Finding useful test cases and running the tool against each one was a job I would normally do by hand. Now I could give that job to a workflow I’d built and see what came back from each run.

One agent found the cases. Others ran them.

I built LessCode for the Meta Global AI Developer Hackathon using Pi and Pi subagents. LessCode reviews JavaScript and TypeScript changes. It gives you a review that points to the source and can offer a patch when a supported edit passes its checks. For the hackathon, I used Meta’s Muse Spark Open through Baseten to propose findings and edits.

I call my Pi setup my software factory. Here’s what it did for this evaluation:

Role Job
Pi coordinator Organize the evaluation and assign the work.
Sourcing subagent Find TypeScript repositories and candidate PRs.
Testing subagents Run LessCode on their assigned changes and report what happened.

The evaluation got its own VM

My home lab runs Proxmox, so I created a VM just for this evaluation. The coordinator and its subagents worked inside that VM, apart from the rest of my lab.

The VM gave the whole job a place to run. Inside it, the assignments gave each runner a narrower job. I wanted to know which result came from which change, so I could look at each case on its own.

What one run taught me

One case I documented was a JavaScript refactor in is-number. I ran the same pinned change through LessCode’s engine, CLI, and MCP interface. All three finished without proposing a patch.

Then I read the reviews. Two of them said an expression using typeof num would throw a ReferenceError. I checked the functions in isolation. For the inputs I tested, they returned false.

That gave me a specific thing to improve. I added checks that let LessCode verify certain claims against evidence from the code it reviewed, then ran the same change again. This time, the CLI and MCP reports correctly described the behavior for those inputs. The CLI also withheld two claims it couldn’t support with citations.

What I’ll reuse

The part I want to keep is the ability to go back to the same case. One agent can find a change worth reviewing, another can run the tool, and I can take a claim from the output and check it myself. After I change the tool, I can rerun the exact revision and see whether that particular result improved.

I built a code reviewer at the hackathon. I came away with a way to test the reviewer that I want to use on my next project, too.

If you use subagents for testing, what do you save so you can rerun a case and compare the results later?