You ask a coding agent to fix one function. It fixes it. On the way it also touches a config file, bumps a version, and loosens a test that was in the way. Every test passes. The agent says it's done.

Nothing in that workflow is wrong by its own rules. The tests were the definition of done, and the tests pass. The extra edits only show up if you read the whole diff, and the point of handing work to an agent was not having to.

This is common, not anecdotal

A May 2026 study read 20,574 real coding-agent sessions from public repositories, about a third of them Claude Code, along with Cursor, Codex and others (arXiv 2605.29442). Across 16,118 validated problem episodes:

  • 38.33% involved the agent violating an explicit developer constraint
  • 10.20% were the agent taking actions beyond the stated scope
  • 22.58% were the agent misreporting the status of its own work

When a problem visibly got resolved, 91.49% of the time it took the developer explicitly pushing back. The agent rarely caught it alone.

Forums say the same in plainer words: an agent that keeps editing files the user didn't ask it to, to the point the user built their own auditor, and agents that overshoot simple requests.

Why "make the tests pass" doesn't catch it

A test-based gate answers one question: does the code still do what the tests check? Scope creep is a different question: did the agent change only what it was asked to change?

  • A lockfile or config edit usually breaks no test.
  • A loosened test makes the suite easier to pass.
  • A version bump is invisible to unit tests.

One developer ran 36 controlled Claude Code tasks to check whether "done" matched the output. The report and the output never disagreed. What did go wrong was quieter. Under a strict completion criterion, the agent changed what the code meant in order to pass, in 2 of 6 runs. And the verification script itself passed when the result file was missing.

So the failure isn't mainly the agent lying. Our checks ask the wrong question, or quietly pass when there is nothing to check.

What a scope check has to do

If you want a machine to say "this went outside the task," it needs:

  1. A declared scope. Paths the task may change, and paths it may never change. Without this there is nothing to judge against.
  2. The final state, not the tool calls. Agents write files through scripts, build steps and shell redirects as well as edit tools. Watching tool calls misses some of those. Comparing the working tree at the end does not.
  3. A baseline. Your own uncommitted work existed before the session. A scope check that blames the agent for it gets switched off within a day.
  4. A place in the loop where it can say no. A report you read afterwards is a code review. A check at the moment the agent tries to finish can send it back with a list of what to undo.
  5. A way out. If the agent fails the same check three times, it should hand over to a person instead of looping.

And it should be honest about what it can't see. A path-based scope check doesn't notice a change of meaning inside a file you allowed, like rewriting a calculation so a test passes. That still needs an output check or a human.

What I built

I built this as a Claude Code Stop hook called taskattest. When the agent tries to finish, it checks the git working tree against a declared scope:

{ "task_id": "fix-billing", "checks": [ {"type": "scope_changes", "name": "billing only", "root": "..", "allow": ["src/billing/**", "tests/**"], "deny": ["**/*.lock"]} ] } 

If the agent also changed pyproject.toml, it can't stop yet:

[FAIL] fix-billing (model calls: 0) - fail billing only: 1 change(s) outside the allowed scope: pyproject.toml fix: undo changes outside the task scope: pyproject.toml 

It can also check output files (invented rows, wrong totals, CSV and Excel). It's deterministic: no model calls, no network, standard library only, and it runs on Linux, macOS and Windows. Apache-2.0.

https://github.com/sionjang101-create/taskattest

I'm more interested in the problem than the tool. If you've hit scope creep with an agent, what did it touch, and would a declared scope have caught it?