Email verification tests rarely fail because a selector is wrong. More often, the browser did its job and the test lost control of time: it read an old message, checked too early, or retried without explaining what changed.
That makes a throwaway email address look unreliable when the real problem is an ambiguous polling contract. In QA work, ambiguity is expensive. A test should tell us which inbox it owned, when the verification request was sent, which messages it observed, and why one message was accepted.
This article describes a small Playwright workflow for making that contract explicit. It is useful for signup, password-reset, and invitation flows, especially when email delivery is slower in CI than on a laptop.
Why email polling becomes flaky
The simplest helper usually looks like this: ask for the newest message, wait a little, and try again. It is easy to write, but the phrase “newest message” hides several decisions:
- Newest compared with what point in time?
- Does the message belong to this test or a previous run?
- Is a matching subject enough, or should the body prove ownership?
- How much time is left before the test gives up?
- What evidence survives when the retry finally passes?
Without answers, a retry can turn a real race into a green build. The test might consume a message created by another run, or it might accept the first message whose subject happens to contain “Verify”. The report then says only Timeout 30s exceeded, which is not a diagnosis.
I have found this is also where small naming mistakes spread. A debug note may say tamp mail com, a helper may call the inbox “latest”, and a CI log may omit the run id. None of those details alone breaks the product, but together they make failures hard to reproduce.
Define the polling contract
Before writing a helper, define the rules in plain language:
- Create or lease one inbox for the test attempt.
- Record
triggeredAtimmediately before the product action. - Ignore messages received before that timestamp.
- Require an ownership signal, such as the inbox address or a run token in the message.
- Poll until a fixed deadline, with a bounded interval between reads.
- Save observed candidates and rejection reasons as failure evidence.
The deadline matters more than a fixed retry count. Three attempts can take three seconds or three minutes depending on the mail client. A deadline gives the test a predictable budget and makes slow CI behavior visible.
If your team uses a service such as temp mail so for isolated test data, keep the provider access behind a small client interface. The Playwright test should care about the contract, not the provider's implementation details.
A Playwright helper with explicit evidence
The helper below uses a client with a list method. The exact API will differ between providers, but the decision points should stay stable.
type MailCandidate = { id: string; subject: string; receivedAt: string; text: string; }; type PollEvidence = { inbox: string; triggeredAt: string; deadline: string; seen: string[]; rejected: string[]; }; async function waitForVerification( mail: { list: (inbox: string) => Promise<MailCandidate[]> }, inbox: string, triggeredAt: Date, runToken: string, timeoutMs = 45_000, ) { const deadline = Date.now() + timeoutMs; const evidence: PollEvidence = { inbox, triggeredAt: triggeredAt.toISOString(), deadline: new Date(deadline).toISOString(), seen: [], rejected: [], }; while (Date.now() < deadline) { const messages = await mail.list(inbox); for (const message of messages) { if (evidence.seen.includes(message.id)) continue; evidence.seen.push(message.id); if (new Date(message.receivedAt) <= triggeredAt) { evidence.rejected.push(`${message.id}: before trigger`); continue; } if (!message.subject.includes("Verify")) { evidence.rejected.push(`${message.id}: subject mismatch`); continue; } if (!message.text.includes(runToken)) { evidence.rejected.push(`${message.id}: ownership token missing`); continue; } return { message, evidence }; } await new Promise((resolve) => setTimeout(resolve, 1_000)); } throw new Error(`Verification email not found: ${JSON.stringify(evidence)}`); } The seen list prevents a noisy provider from producing duplicate evidence. More importantly, each rejection has a reason. If the helper times out, a reviewer can tell whether delivery was late, the subject changed, or the application sent the message to a different inbox.
Classify failures before retrying
Do not make every email failure retryable. A useful classification is:
- Delivery delay: no candidate after the trigger, but the provider is healthy. Retry within the deadline.
- Ownership failure: a message arrived, but its token or recipient is wrong. Keep the failure; retrying may hide a real routing bug.
- Contract failure: the subject or content no longer matches the expected product behavior. Fail fast and attach the message evidence.
- Provider failure: the mail client returned an API or authentication error. Record it separately from a product assertion.
This distinction also improves dashboards. A CI report with “email timeout” is vague. A report with “ownership token missing in message 8f2” points an engineer toward the right system.
For larger suites, consider exporting the evidence as an attachment rather than printing full message bodies into logs. Sensitive links and tokens should not become permanent build artifacts. The same principle applies to the verifiable CI receipts you use for other automation steps: retain enough proof to audit the run, but avoid unnecessary secrets.
A small QA checklist
Before calling an email test stable, check these items:
- Does every attempt have a unique inbox or run token?
- Is the trigger timestamp captured before the UI action?
- Are old messages rejected explicitly?
- Does the message prove recipient or run ownership?
- Is the timeout a deadline instead of an unbounded loop?
- Are candidate ids and rejection reasons included in failure output?
- Can a provider error be distinguished from a product assertion?
- Are links and tokens redacted from long-lived CI logs?
It is worth testing the helper itself with a few fake timelines: an old matching email, a new message for another run, a late valid message, and a provider error. Those cases are more valuable than adding another blind retry to the browser spec. For Docker-based pipelines, Docker checks for email failures are a useful reminder that the environment is part of the test contract too.
Final takeaway
Reliable Playwright email tests do not need perfect delivery speed. They need explicit boundaries. Capture the trigger time, prove message ownership, poll to a deadline, and keep the evidence that explains every rejection.
Once those rules are in one helper, retries become a controlled recovery mechanism instead of a coin toss. That is a small change, but it turns “the email test is flaky” into a failure a QA engineer can actually investigate.


