The Half-Life Layer — Or, "It Passed in March" Is Not the Same as "It Passes Now"

Somewhere in your repository there is a test that passed in March. It passed against a schema that has since gained two columns, a dependency that has since changed its default timeout, and a traffic profile that has since shifted from mostly reads to mostly writes. The test still passes. Nobody has run it against the current system in a way that would notice the difference.

That is not a broken test. It is a piece of evidence that has expired while continuing to be cited.

Evidence is a measurement, not a property

"It is tested" sounds like a property of the code. It is not. A passing test is a measurement: a specific artifact, executed against a specific commit, under a specific configuration, at a specific time. All four of those are part of the claim, and three of them change without anyone touching the test.

This is why the same sentence can be true in one review and false in the next without a single line of code changing. The test did not become wrong. The system underneath it moved, and the measurement stopped describing the system.

Auditors, customers, and incident reviews all ask the same question — "how do you know?" — and all three accept an answer that omits the timestamp. The half-life layer is the discipline of putting the timestamp back.

Three ways evidence expires

1. The substrate changes. Schema migrations, dependency upgrades, base image rebuilds, a service split in two, a region added, a queue replaced. The test still exercises the behavior it was written for, against a system that no longer has the same shape. A passing integration test against a service that has since been split is evidence about a service that no longer exists.

2. The population changes. A load test calibrated to 40 requests per second says nothing at 400. A threshold tuned when 90% of traffic was one large tenant becomes decorative when the tenant mix flattens. A retention policy verified against last year's data volume is a different policy at ten times the volume, because the job that enforces it now runs for six hours instead of twenty minutes and overlaps the next run.

3. The witness changes. The check now runs inside the component it is supposed to check — self-attestation. Or the dashboard is still green because the metric stopped being emitted and the panel renders the last value it had. Or the person who could read the output left, and the artifact is now a file nobody opens.

Two of these three leave no trace in version control. That is the whole problem: the diff is clean, the test is green, and the evidence is stale.

Claims have a shorter half-life than the artifacts under them

A schema's half-life might be a year. The test's half-life might be six months. The claim you built on top of the test — "this migration is reversible," "this cap holds," "this deletion is complete" — has the shortest half-life of all, because it depends on the schema, the test, the configuration, and the population at once.

So the useful question is not "when was this last verified?" It is "what would have to change for this measurement to stop being true, and has any of it changed since?" A date alone does not answer that. A date plus a list of the assumptions the measurement depended on does.

Write the assumptions down at the moment of measurement, when they are obvious. Six months later they are invisible, and the artifact looks like a permanent property instead of a snapshot.

The five-column evidence register

One row per claim you rely on — not per test, and not per document. A claim is the sentence you would say to a customer, an auditor, or an incident review.

• Claim. One sentence, falsifiable, in the words you would actually use. "Deleting an account removes all derived rows within 30 days" is a claim. "Data deletion is tested" is a category.
• Artifact. The thing that can be opened: a run log, a query result, a signed report, a diff. Not a ticket number that says the artifact exists.
• Measured against. The commit, the configuration, and the date — all three. If the measurement was taken against a staging environment, say so; staging is a different system with a different population.
• Expires when. The event that invalidates the measurement, stated as an event rather than a date: "when the schema for events changes," "when a second region is added," "when the model behind the classifier is swapped," "when any tenant exceeds 1 TB." An expiry stated as an anniversary is a calendar entry, not a trigger.
• Re-verify by / owner. A date and a person. Dates without owners are intentions; owners without dates are hobbies.

A row whose "Expires when" column is empty is a claim with no defined failure mode. That does not mean it is safe. It means nobody has thought about what would make it false, which is the more common condition.

Calendar review is the wrong mechanism

Annual reviews are popular because they are easy to schedule and easy to pass. They are also blind by construction: the thing that invalidates evidence is a change, and a calendar cannot see a change. A schema migration in June does not move the December review, and by December the reviewers are looking at a system they no longer recognize, approving a document that describes the old one.

Event-driven re-verification is cheaper and earlier. Attach the check to the change instead of the date:

• schema migration merged → re-run the deletion and retention measurements
• dependency major version bumped → re-run the timeout and retry-path tests
• model or classifier swapped → re-run the evaluation set and re-check the threshold
• new region or tenant tier added → re-run the blast-radius and cap tests
• pricing or plan limits changed → re-run the entitlement checks

The trigger has to live where the change happens — in the migration's checklist, the dependency bot's PR template, the model rollout runbook. A register that nobody consults before merging is a beautiful document with no effect.

Three signs you are citing expired evidence

1. The screenshot has no date. A dashboard screenshot proves a graph rendered. It does not prove the metric was collected at the moment of the claim, that the panel is wired to the current data source, or that the threshold on the graph matches the threshold in the code. Screenshots are the most common form of expired evidence because they are the easiest to produce and the hardest to falsify.

2. The test is green because it stopped running. Renamed suites, skipped tests, conditional CI jobs, and paths excluded from the default test command all produce the same output as a passing test: nothing. A suite that has not executed in ninety days should be reported as "unknown," not "green."

3. The number predates the change it is supposed to justify. A benchmark from before the model swap, a cost estimate from before the pricing change, a capacity plan from before the new tenant. The number is real; the claim it supports is about a system that no longer exists. This is the most expensive form, because the number is usually the one in the deck.

Re-verification is cheaper than re-discovery

Re-running a measurement you already designed is minutes to hours. Discovering that the measurement no longer holds is an incident. The asymmetry is enormous and routinely ignored, because re-verification has no visible output when it succeeds — the artifact is identical to the one you already had, minus the date.

That is why re-verification needs a small piece of theater to survive: a fresh artifact with a new timestamp, filed in the same place, with the trigger recorded. Not because the theater has value in itself, but because "we checked and nothing changed" is invisible, and invisible work gets cut first.

The uncomfortable part

Most compliance evidence is produced once and copied forward. The SOC 2 report from last year, the pen-test summary from two quarters ago, the architecture diagram from before the service split — all of it is accurate about a system that has since moved, and all of it is accepted because the reviewer also cannot test absence.

The half-life layer does not make evidence permanent. It makes the expiry visible, which is the only honest thing evidence can do. A claim with a stated half-life and a named re-verification trigger is worth more than a claim marked "verified," because it tells you when to stop believing it.

Friday exercise

Pick the three claims you would make to a customer under pressure. For each, write the artifact, the date it was measured, and the one change that would make it false. If you cannot name the change, the claim is not measured — it is remembered. If you can name it, put the re-verification on the checklist of the change, not on next year's calendar.

If you are citing a test, a benchmark, or a sign-off that was measured before the system changed shape, say so in the comments. We keep a five-column evidence register — the claim, the artifact, what it was measured against (commit + config + date), the change that would expire it, and who re-verifies by when — and I will send it over. No form, no email list.