What happens when you give a vague ticket to a person?

They ask a question. Or they fill the gap with what they know about your team, your customers and that one time it all went wrong.

Now give the same ticket to an AI agent.

It doesn't ask. It fills the gap with a plausible guess, finishes, and reports back with total confidence:

Implemented and verified. ✅

And you find out what it guessed in review, reading a diff against a definition of done that nobody wrote down. 😅

Acceptance criteria are where that gets fixed, before the work starts instead of after. With an agent they matter more, not less. Every gap gets filled, and nobody tells you which ones.

Full disclosure: I make Arbituro, a work tracker built around criteria like these. Everything in this post works in any tracker, and only the last section is about Arbituro.

🤔 Three Questions for Every Criterion

A criterion an agent can meet answers three questions:

  1. What will be true? An outcome someone can observe, not an activity.
  2. How will we know? The evidence: a test, a check's result, a page someone opens, a command's output.
  3. Who decides? A named CI check, or a named person. Not "the team". Not "QA". And definitely not nobody.

Can't answer the third one? Then it isn't a criterion yet. It's a wish. 🌠

📏 Eight Rules

Rule #1: Write Outcomes, Not Activities

"Add tests" describes effort. "Signing up twice with the same address is refused" describes a result. An agent can always do the activity. Only the outcome tells it when to stop.

Rule #2: Replace Every Quality Word

Fast, clean, robust, properly, user-friendly. Each one is a gap the agent fills with its own standard, and its standard is... generous. 😇 Swap the word for a number, an example or a named judge.

Rule #3: One Thing per Criterion

A criterion with "and" in it can be half met, and then nobody can say met or not met. Split it.

Rule #4: Name the Edge Case You Care About

Agents cover the cases you wrote down. So write down the one you'd check first in review.

Rule #5: Say What's Out of Scope

Agents love to widen work. A rename here, a little tidy-up there, and suddenly the diff touches forty files. 🧹 One line saying what this job leaves alone keeps the diff reviewable.

Rule #6: Let a Check Decide Wherever One Can

A CI check decides "the test passes" on every push, and it never gets tired, bored or hungry. Keep people for judgement: wording, design, whether the change actually solves the problem.

Rule #7: Keep Standing Rules Out of Tickets

"Lint passes", "reviewed by a second engineer" and "coverage holds" apply to every change. Enforce them once, in CI and branch protection, so each ticket's criteria say only what's specific to it.

Rule #8: Two to Five per Job

Fewer, and something's usually missing. More, and it's probably two jobs in a trench coat. 🧥

🔧 Before and After

Enough theory. Let's fix some tickets.

Before: "Handle expired reset links properly."

After: Opening an expired or already used reset link shows "This link has expired" and a button that sends a new one. Decided by the e2e check.

"Properly" was carrying that whole ticket. 😂

Before: "Make search faster."

After: On the seeded test data of 50,000 orders, the first page of search results comes back in under 300 ms at the 95th percentile. Decided by the search-perf check.

Faster than what? Now we know.

Before: "Make checkout mobile-friendly."

After: At 375 px wide, the checkout form fits without scrolling sideways, and every button is at least 44 px tall. Priya decides, from screenshots at 375 px.

And Priya knows exactly what she's looking at.

Before: "Clean up the billing code."

After:

  1. Billing behaves exactly as before: every existing billing test passes. Decided by the test check.
  2. InvoiceService reaches the payment provider only through PaymentClient. Priya decides, from the diff.

Not in this job: renaming anything other code calls.

"Clean up" is how a two-line fix becomes a forty-file diff. That last line is doing real work.

Before: "Add tests for the duplicate email bug."

After:

  1. Signing up with an address that differs from an existing one only in case is refused with "That email is already registered". Decided by the test check.
  2. The new test fails on main, so it really catches the bug. Sam decides, from the agent's run of the test against main.

That second criterion is sneaky, and I love it. More on why in a minute.

Before: "Tell customers about the price change."

After: Customers on the old plan are sent the approved email about the new price. Met when the approved email is sent.

Wait, that one isn't code at all! Right. Agents draft emails and posts too, and done there means the message actually went out. A draft sitting somewhere doesn't count. ✉️

⚖️ Who Decides: a Check, a Person or a Send

A CI check suits anything a machine can verify. Name it exactly as it appears in the pull request's checks, and watch out for matrix jobs, which get names like test (20).

A check is only as good as the test behind it, though. When the agent writes the test too, the test is what you review. That's why "the new test fails on main" is up there: it catches a test that passes whatever the code does. Yes, that's a thing that happens. 🙃

A person suits judgement: wording, design, whether the fix solves the customer's problem. Name one person. When the agent believes the criterion is met, it says what it checked, and that person decides.

A send suits work whose result goes out to someone: a customer email, a release note in a team channel. It's met when the approved message is actually sent.

🧾 What a Claim Should Say

When an agent thinks a person-decided criterion is met, the useful report is what it checked, not that it's finished.

Here's our old friend again:

Implemented and verified. ✅

And here's a claim somebody can actually decide on:

Ran npm test -- signup on main: "refuses an address that differs only in case" fails (expected 409, got 201). On this branch it passes, along with the other 212 tests.

See the difference? The second one names the command, says what happened on main and says what happened on the branch. A claim should name the command it ran or the page it opened, and quote what it saw. One with nothing behind it hands the person who decides a decision with nothing to decide on.

🤝 Let the Agent Draft, and a Person Agree

Here's the twist: agents are really good at drafting criteria. Ask one to read the ticket and the code before it starts, and it'll often find the edge case you missed.

What it shouldn't do is set its own bar. That's like letting me write my own performance review. 😅 The agent proposes, a person accepts or edits, and then the work starts.

The same goes mid-work. If a criterion turns out to be wrong, the agent should say so and stop, not quietly work to a different one. Changing or dropping a criterion is a person's decision, made on the record with a reason, so the review matches what was agreed.

📋 A Template

Copy it, paste it, make it yours:

**Done when** 1. <What will be true>. Decided by the `<check>` check. 2. <What will be true>. <Name> decides, from <evidence>. 3. <What will be true>. Met when the approved <email or post> is sent. **Not in this job:** <what to leave alone> 

🏗️ How Arbituro Does It

Okay, the promised product bit. In Arbituro, every acceptance criterion says how it's decided: by a named person, by a CI check, or by an approved message being sent. Agents read the criteria before they start, whether they're Arbituro's own agents or your coding agent connected over MCP.

Arbituro's agent proposing four acceptance criteria for a new job, each naming the person who decides it, with a way to add your own and a button to agree to all four.

  • Drafting. When a job is filed, the project's agent suggests criteria, each naming who decides it, and you accept, edit or decline each one. A coding agent can propose criteria too, naming the check that should decide each, and a person accepts them.
  • Checks. A criterion that names a CI check is decided when that check finishes on the latest commit of a pull request linked to the job: met when it succeeds, not met when it fails.
  • People. For a criterion a person decides, an agent can only record what it checked. That person is asked, and decides in one press.
  • Changes. Agents never tick, change or remove a criterion. The person who decides one, or the job's owner, can waive it with a reason, so the record shows what changed and why.

🏁 Wrapping Up

Three questions, eight rules, and somebody (or something) named to decide each criterion. That's really all it takes to stop getting "Implemented and verified. ✅" as a status report.

Arbituro is free for up to three people at arbituro.com, and the acceptance criteria docs have the details. For the coding side, the other post in this series has Claude Code working a ticket like these over MCP: Stop Pasting Tickets into Claude Code.

What about you? How does your team write acceptance criteria, and have you tried handing them to an agent yet? Bonus points for the vaguest criterion you've ever seen. 😂 Tell me in the comments. 👇