A right-or-wrong score throws away the most useful part of a wrong answer

Two people answer "Sydney" to "What's the capital of Australia?" One was guessing. One was sure.
A right-or-wrong score gives them the same zero, and loses the only thing worth knowing.

If you've ever built a quiz, a grader, an interview rubric or even a code-challenge platform, you've had to decide what a wrong answer is worth. Most of us default to zero. I think that's the most wasteful line in the scoring function.

Not all wrong answers are the same

Take the Australia question. Plausible wrong answers include:

Sydney: the famous city. A familiarity error.
Melbourne: also big, and historically relevant. A near miss with some knowledge behind it.
Wellington: right idea (a capital), wrong country. A category confusion.
Vienna: a sound-alike, "Austria" vs "Australia". A reading error.

Four zeros. Four completely different things to fix.

Confidence is the hidden variable

There's a well-replicated finding in memory research called the hypercorrection effect: errors made with high confidence are more likely to be corrected after feedback than low-confidence errors (Butterfield & Metcalfe, 2001). Metcalfe's review Learning from Errors goes further: making errors, followed by corrective feedback, can help learning rather than hurt it.

The implication for a grader: a confident wrong answer is a teaching moment you should invest in. A low-confidence wrong answer is a guess, and the right move is often just to show the answer and move on.

A binary score can't tell those apart.

What you can capture cheaply

You don't need an AI tutor to keep more information. Three cheap signals:

Confidence. One tap before submitting: "guessing / fairly sure / certain".
Error type. For multiple-choice, tag each distractor with the mistake it represents when you write it.
Attempts. Was it right on the first try or the third?

That's enough to give very different feedback to the guesser and the confident Sydney.

A small scoring sketch

Here's a deliberately simple example. It's not a recommendation for any specific product, just a way to show how much more a wrong answer can carry:

const DISTRACTORS = { sydney: "familiarity", melbourne: "near-miss", wellington: "category", vienna: "misread", }; function feedback(answer, confidence) { if (answer === "canberra") return { correct: true }; const errorType = DISTRACTORS[answer] ?? "unknown"; const priority = confidence === "certain" ? "explain" : "show-answer"; return { correct: false, errorType, priority }; }

The score can still be zero. What changes is what you do with the zero: explain the familiarity trap to the certain player, and just reveal the answer to the guesser.

"But partial credit inflates scores"

Fair objection. If you award points for near misses, a leaderboard gets noisy and people learn to aim for "close enough".

The fix is to separate two things that usually live in one number:

The score, which can stay strict.
The diagnosis, which records what kind of wrong it was and drives feedback.

You don't have to be generous to be informative.

This applies way beyond quizzes

Code challenges: a solution that fails one edge case and one that doesn't compile are both "failed". They need different help.
Interviews: "didn't get it" hides whether the candidate had the right approach and slipped, or had no approach.
Linters and tests: a failing test that names the expectation teaches more than a red X.

Every time we flatten an outcome into pass/fail, we choose to forget why.

The cost of remembering is usually small: one extra field, one tag, one more line in the test message. The cost of forgetting shows up later, as people repeating the same mistake because nothing ever told them which mistake it was. If you're building anything that judges answers, store the why. Future you, and every learner on the other side, will thank you.

Writing distractors that diagnose

The cheapest place to capture error type is when you write the question, not after the answer. A distractor that represents a specific mistake turns every wrong click into a label you can act on.

A simple process:

List the common mistakes first, before writing any options. For capitals: famous city, former capital, neighbouring country, sound-alike.
Write one option per mistake. Each distractor should be the answer someone with that mistake would give.
Avoid joke options. An option nobody would pick carries no information and makes the question easier.
Review the distribution. After real players answer, look at which distractors get picked. One that nobody chooses can be replaced; one that half the players choose might mean the question is ambiguous.

For free-text answers the same idea applies, just later: group wrong answers by pattern and map each pattern to a feedback message. Even a handful of patterns covers a surprising share of what people type.

The result is a question that teaches you about your players as much as it tests them.