Email tests tend to work perfectly on a laptop and become mysterious in CI. A workflow starts three jobs, each creates a user, and suddenly one test reads another's message. A retry makes the failure look random. It is not random; the test data had no identity.

The useful mental model is simple: every CI run needs an identity that follows the test from inbox creation to cleanup. A burner email address or a create temporary mail helper can be part of that setup, but the important part is the run contract around it. The inbox is a test fixture, not a shared bucket.

Why parallel CI makes email tests confusing

Most email checks begin with a locator such as “find the newest message for this address.” That is already a little fragile. Add parallel jobs, delayed delivery, and retries, and “newest” stops meaning “mine.” CI can get noisy quick when every job is looking at the same mailbox.

The common symptoms are familiar:

  • an OTP from a previous attempt is accepted;
  • a verification link opens the wrong preview environment;
  • cleanup deletes a mailbox while another job still uses it;
  • logs show a message ID, but nobody can connect it to a CI run.

The fix is not always a bigger polling timeout. A longer wait just makes the confusion take more time. Give the test data a stable correlation ID first.

Give every run a small identity

Use the CI provider's run number, attempt number, and job name to build one opaque ID. Keep it short enough for an email local-part or a log line:

function testRunId(env) { const run = env.CI_RUN_NUMBER || "local"; const attempt = env.CI_RUN_ATTEMPT || "1"; const job = (env.CI_JOB_NAME || "email-test") .toLowerCase() .replace(/[^a-z0-9]+/g, "-") .replace(/^-|-$/g, "") .slice(0, 24); return `${job}-${run}-${attempt}`; }

For a local run, add a random suffix so two terminals do not compete. In CI, the attempt number is important: a rerun should not silently reuse the first attempt's inbox unless that behaviour is explicit.

Derive all related values from this ID:

run ID: checkout-1842-2 test user: checkout-1842-2-user-7 message key: checkout-1842-2|user-7|signup

The format matters less than using it everywhere: address, metadata, logs, and artifacts. This make debugging less of a scavenger hunt. The values is intentionally boring, which is good for test identifiers.

Build the fixture lifecycle around that identity

Treat the email fixture as a resource with four explicit operations:

  1. Create an address or inbox owned by the run.
  2. Use it for one test flow, with a message query scoped to the run ID.
  3. Freeze the useful message metadata when the assertion succeeds or fails.
  4. Release the fixture during teardown, even when the test aborts.

The query should include more than an address: match the run marker, recipient, event type, and a time boundary. If the provider supports metadata, put the run ID there instead of relying only on the visible subject.

Keep the search vocabulary separate from protocol identifiers. A user may type temp mailid or temp org mail into a form, and that can be valid input coverage. Neither phrase should become the correlation key for a test. Search terms are not ownership.

For preview environments, put the run ID in the application-side event too. A message can then be checked against both the email service and the application event log. My earlier pattern for preview-environment email checks follows this same idea: one run ID is more useful than several loosely related timestamps.

Keep cleanup and artifacts predictable

Cleanup needs an owner and a deadline. A simple contract looks like this:

created_at = now expires_at = now + 30 minutes owner = ci run ID

The test job should release the fixture in a finally block. If a test fail halfway through, teardown still needs to run. A separate janitor can remove anything past expires_at, because CI jobs can be cancelled before teardown runs. The janitor should only removes resources with the matching test prefix or ownership metadata; broad cleanup is how one build breaks another.

When a check fails, save a small evidence bundle before releasing the inbox:

  • the run ID and test name;
  • the address or inbox identifier, redacted if needed;
  • matching message IDs and timestamps;
  • the polling deadline and number of attempts;
  • the application event ID when available.

Do not save a clickable verification token in a public CI artifact. Redact secrets while preserving enough text to tell whether the right message arrived.

If a flow uses OTPs, the same ownership rule applies. Inbox boundaries for OTP flows is a useful related example: isolating the inbox is part of isolating the authentication attempt.

A practical CI checklist

Before calling the email test reliable, check these points. These checks is small, but they catch the failures that are hardest to reproduce:

  • Does every run and retry get a distinguishable ID?
  • Can two jobs query the same address by accident?
  • Does the message query include a run marker and an event type?
  • Is the polling deadline recorded in the failure output?
  • Does teardown run after assertion failures and cancellations?
  • Can a janitor identify ownership without guessing from age alone?
  • Are tokens and private message content redacted in artifacts?
  • Can a developer reproduce the message lookup locally?

The last question matters more than it sounds. A helper that works in CI but is painful to use locally will be bypassed eventually. Keep the interface small and print the run ID at the start of every test, so a developer know what to search for.

Questions that come up in real pipelines

Is a temporary address enough isolation?

No. It isolates the recipient, but not necessarily the job, attempt, or message query. Run metadata and cleanup are still needed.

What if delivery is slower than the timeout?

Record the deadline and the observed attempts, then fail with evidence. Increasing the timeout for every test often hides a delivery regression. A few targeted retries are better, and the failure should say what was missing.

Run-aware email fixtures turn a noisy CI problem into an ordinary lifecycle problem. Create with an owner, query with a correlation ID, preserve useful evidence, and clean up by contract. Once those pieces are in place, parallel automation feels much less spooky—and the next failure has somewhere useful to start.