Treating "It Worked" as Verification Ships Code That Only Works With One Commit
quintetkitDEV Community
2 views
I shipped a bug in a script I sell. I had also reported that script as
"verified across four scenarios."
I had verified it. In a way that verified nothing.
What was broken
The script cleans up git worktree directories once their branch is merged.
Part of the decision was detecting squash merges.
# the broken check
cherry="$(git cherry "$base" "$branch")"
[ -z "$cherry" ] && return 1
printf '%s\n' "$cherry" | grep -q '^+' && return 1
return 0 # every line is "-", so it is merged
git cherry marks each commit - or + depending on whether an equivalent
exists on the base. All - means merged.
That is wrong. git cherry matches on patch-id, so a squash that collapses
several commits into one can never match.
Measured locally:
# after squash-merging a branch with one commit
$ git cherry main one
- 52b3d961...
# after squash-merging a branch with two commits
$ git cherry main two
+ 02577c73...
+ 221f83ce...
It only works by accident when the branch has a single commit.
Why I did not catch it
I had built a scratch repository and run four scenarios:
no work yet → keep ✓
ordinary merge → delete ✓
unmerged → keep ✓
squash merge → delete ✓
All four behaved as expected. And the branch in the fourth scenario
had exactly one commit.
The test setup read like this:
git checkout -qb squash main
echo x > x.txt && git add -A && git commit -qm c1 # ← one commit
git checkout -q main && git merge -q --squash squash && git commit -qm "squash"
Reaching for the smallest possible reproduction is a normal instinct.
That "smallest" happened to be the only shape in which the broken code passes.
A squash collapses several commits into one. A one-commit squash has none of the
properties of a squash. I thought I was testing squash merges; I was testing
something else that shared the name.
The fix was not "detect it more cleverly"
My first instinct was to find a more accurate check — compare diffs, compare tree
hashes.
I dropped that. A false "merged" verdict deletes someone's working tree.
An error that errs toward deleting costs far more than one that errs toward keeping.
So the answer was to not decide at all:
# Squash merges are deliberately not detected here.
#
# git cherry matches on patch-id, so a squash that collapses several commits
# into one can never match. Measured: a one-commit branch returns `-`, a
# two-commit branch returns `+ +`. It only ever worked by accident.
#
# A false "merged" verdict deletes a working tree, so we do not guess.
# With gh available, the PR state answers this exactly.
return 1
And then made the inability visible:
if ! command -v gh >/dev/null 2>&1; then
echo " Note: without gh, squash-merged branches cannot be detected."
echo " If any of the kept worktrees were squash-merged, remove them by hand."
fi
Silently doing nothing reads as being broken. If you cannot decide, say so.
Rebuild the scenarios around the property, not the boundary
Branch
Commits
Expected
Result
no work
0
keep
✓
ordinary merge
2
delete
✓
unmerged
2
keep
✓
squash (1 commit)
1
keep
✓
squash (3 commits)
3
keep
✓
The change was not "use more than one commit everywhere." It was splitting
one-commit squash and multi-commit squash into separate cases.
Thinking in boundary values gets you 0, 1, 2. That is not the same as asking what
the smallest shape is in which the property under test actually appears.
A squash's property is collapsing, and collapsing needs at least two things to collapse.
The same hole, in how I measured
Three separate times in the same session I made this mistake:
# wrong. $? belongs to head
./script.sh | head -5
echo "exit code: $?"
Through a pipe, $? is the last command's status. head always succeeds, so
a failing script is reported as exit code 0.
All three times the conclusion was the same: the tool was right, my measurement was wrong.
# right
./script.sh > /tmp/out.log 2>&1
echo "exit code: $?"
head -5 /tmp/out.log
The common thread is never doubting the measurement apparatus.
When a number matches expectations, nothing tests how it was obtained.
The checklist that came out of it
"It worked" is not verification. Can you explain why it passed?
Does the test input actually carry the property under test, or did minimising
the repro delete that property?
Decide in advance which way an error should fall. Deleting costs more than keeping.
When you cannot decide, do not go quiet about it.
Distrust the measurement path — pipes, redirection, $?. Especially when the
number is the one you wanted.
Summary
Detecting squash merges with git cherry is wrong: it matches on patch-id, so a
collapsed squash never matches
It passes for single-commit branches, which is exactly why people try it and ship it
When you minimise a reproduction, check you have not minimised away the property
If you cannot decide accurately, do not decide — but do not stay silent either
Doubt your measurement most when the result is what you expected
This script is something I sell. When a shipped thing turns out to be broken,
writing up how it broke leaves more trust behind than quietly swapping the file.
Related
What You Refuse to Check Decides the Quality of a Linter
Is Your AI-Generated Code Review Just a Formality?
Running Claude Code in True Parallelism - git worktree and Serial Steps
I publish the configuration for splitting Claude Code into separate personas —
Architect, Coder, Reviewer, Conflict Resolver — under MIT. Copy it, run
./setup.sh, and it works. It does not depend on your tech stack.
https://github.com/quintetkit/quartet
I built one real tool using nothing but this workflow. Every Issue, PR, review
and merge is still there. The parts that went wrong were not deleted.
https://github.com/quintetkit/mdlinkcheck
The version that adds a UI Designer persona, review criteria, a per-Issue
parallel execution script and a 10-chapter guide is on the
product page.
Hello, I’m simply asking because I’m aspiring to become a web developer, and I’m curious about how viable the field is nowadays. I don’t think you can really blame people for asking this either, because front-end development has been heavily trivialised by AI(not my opinion), or at least that’s what
Project Name: Podcode Repo/Website Link: https://podcode.io Description: Quick background, since this matters for the rest. I run a small private AI stack for my own work. Coding agents like Claude Code and Codex are part of my daily flow. They are great when they work. The thing that drove me nuts
ChatGPT.com reached about 1.09 billion monthly US visits in July 2026, a 48.38% year-over-year increase, according to Semrush Traffic Analytics data. In the same comparison, Bing.com traffic fell about 50.43%. The contrast does not show AI replacing conventional search overnight. Google and YouTube