Before You Automate Paid-Media Reporting: 14 Data Integrity Tests

The decision this guide helps you make

An automated marketing report is ready for executive use only when its inputs, definitions, time window, calculations, and caveats can be reproduced by someone other than the workflow that generated it. A polished dashboard is not evidence that the numbers reconcile. Before you let a system summarize paid-media performance or recommend a budget change, run the checks below against a frozen export and preserve the source rows. If a check fails, keep the output in draft mode and identify the owner who can resolve it.

This article contributes a 14-case acceptance matrix and a small Python gate checker. The cases are synthetic and deliberately falsifiable. No advertising account, vendor, agency, model, or reporting stack was tested. The examples use abstract campaigns and invented values only to show what to verify. The matrix is a starting protocol, not an industry benchmark, certification, or proof that a report is accurate in production.

If you are comparing AI-enabled marketing workflows, the Trustly-AI's workflow guide is a useful map of the work people try to automate. This article narrows the decision to a harder question: can a report safely support an action when platforms count events differently and data arrives late?

Why automation changes the review problem

A spreadsheet assembled by a person often exposes its seams: a pasted range, a stale tab, an unexplained adjustment. A generated narrative can hide those seams behind fluent prose. The risk is not that every automated report is wrong. It is that a mistaken input can be transformed consistently, summarized confidently, and copied into a decision before anyone asks what the number means.

The NIST AI Risk Management Framework is voluntary and was designed to help organizations manage risks to people, organizations, and society associated with AI. It is not a paid-media measurement standard, nor does it validate a report. Its useful role here is narrower: treat the model-assisted narrative as an AI system whose risks need named owners, review, and controls. NIST says the framework is being revised, so check its current status before adopting it as an organizational reference. NIST AI RMF overview describes its purpose and voluntary status.

For a report pipeline, the practical implication is simple: test the joins and definitions before testing the prose. A summary model cannot repair a conversion window that changed, an import that has not landed, or two systems that use different attribution models. The Google Ads API reference lists reporting resources, metrics, and segments, and notes compatibility constraints across fields. Those field definitions are part of the report contract, not implementation trivia. Google's current reporting reference should be checked against the API version your connector actually uses.

What the baseline already tells us

Official platform documentation answers what a platform field represents and which combinations can be queried. It does not prove that a cross-platform report has aligned every source, reconciled every currency, or applied a useful business definition. Industry measurement frameworks add common vocabulary, but agreement on a name does not make two data pipelines identical. IAB Europe's effectiveness framework separates media, brand, and sales effectiveness; that distinction is useful because impressions and attributed sales answer different questions. It is not a universal attribution model or a guarantee of causal lift. See the IAB Europe framework for its own scope and definitions.

A consulting firm may describe how AI can change advertising operations, but a vendor or consulting perspective is not independent evidence that a particular tool improves a particular advertiser's results. Accenture's discussion of AI-powered advertising platforms is useful as a view of the opportunity and business context, not as a neutral evaluation of your system. Read Accenture's perspective alongside primary platform documentation and your own reconciliation records.

A report may later become a public performance claim, but this article does not interpret advertising law. NIST’s AI RMF is a voluntary risk-management framework, not a legal standard or a proof-of-accuracy label. Use it to assign oversight and risk ownership for AI-assisted work, then obtain jurisdiction-specific review for external claims.

A report contract before the model prompt

Write the report contract before choosing a model or prompt. Name each source, account, campaign scope, extraction timestamp, reporting timezone, currency, event definition, attribution model, conversion window, and refresh expectation. Identify the person who owns each definition. Record what the report excludes, including channels, offline events, low-volume rows, modeled conversions, invalid traffic adjustments, refunds, and late-arriving imports where applicable.

The contract should distinguish three outputs. A descriptive metric says what a source reported under a stated definition. An interpretation explains a possible reason for a change and should be labeled as a hypothesis. A recommendation proposes an action and should include its owner, reversibility, decision deadline, and evidence needed to revisit it. If a generated paragraph blends those categories, reviewers cannot tell which sentence is a platform fact and which is a suggested explanation.

Also state the report's permitted action. A weekly narrative may be allowed to flag an anomaly but not change a bid, pause a campaign, or reallocate spend. That boundary should be enforced outside the model as well as written in a prompt. A generated answer that says “increase budget” is still an unapproved recommendation unless a named human with the correct authority accepts it. When data is incomplete, the default output should be “hold and investigate,” not a guessed number or confident fill-in.

The 14-case acceptance matrix

Run each case against a frozen, synthetic fixture first, then against a recent real export with sensitive fields removed. Record the expected result, observed result, evidence file, reviewer, and pass/fail. The cases test controls, not vendor quality. A passing synthetic suite shows that the checks can catch designed failures; it does not estimate real-world error rates.

The matrix is intentionally not a score that lets a strong average compensate for one unsafe failure. A wrong currency conversion, duplicated conversion, or unauthorized spend change is a hard stop even if thirteen other checks pass. Define that policy before running the test so that teams do not negotiate the threshold after seeing a convenient result.

Test 1: source identity and completeness

Create a manifest of required sources for the reporting period: ad-platform cost, analytics sessions, CRM-qualified opportunities, and an offline conversion import might all be required. Store the expected source identifier, account or property, extraction time, date range, row count, and checksum. Then remove one source from the fixture. The report should identify the absent input and refuse to present a combined funnel total as complete.

A source being “connected” in a settings page is not the same as a successful extraction for this run. Require evidence that the specific account, date range, and resource were read. If the connector silently falls back to a cached extract, the report should label the fallback with its original timestamp. A reviewer should be able to trace every top-line total to the files or API query that produced it, without having to trust the generated narrative.

Keep the manifest in a durable location separate from the generated report. Give each run a unique identifier. The report should carry that run identifier so a screenshot or exported PDF can be matched to its input bundle. If two dashboards disagree, first compare their source manifests and metric contracts, not two polished summaries. This makes later correction possible and discourages copying a figure whose provenance has been lost.

Test 2: freshness and late-arriving data

Pick a freshness cutoff that matches the business decision, not a generic default. A same-day budget discussion may need recent spend, while a lead-quality report may need a longer maturation period. In the fixture, mark one source as late. Expected behavior is a stale warning, explicit as-of time, and either a partial report or a blocked total according to the contract. It should not silently mix yesterday's CRM data with today's platform cost and call the result a current return.

Late conversion imports create a second problem: the same reporting window can change after the first run. The report therefore needs a version, extraction timestamp, and clear rule for restatement. A subsequent run may legitimately differ; it should explain which source changed and which rows were added or revised. Do not label the first number “wrong” simply because it was incomplete at that time. Preserve both versions and the facts known at each cutoff.

A useful freshness control distinguishes source delay from pipeline failure. If an imported event is expected later, state the expected delay and schedule reconciliation. If the delay exceeds the contract, flag an incident to the data owner. Never ask a model to infer missing values from past performance. Forecasting a missing input is a separate analytical task with its own assumptions and interval; filling a reporting gap with a point estimate conceals that evidence was absent.

Test 3: timezone and reporting-window alignment

“Monday” is not a complete definition until the timezone and boundary are known. One platform may group days in the advertiser's account timezone; a warehouse export may be UTC; another source may use a property timezone. A query that looks at identical calendar dates can include different events around midnight. Inject a row near the boundary into the fixture and confirm that the expected source-specific window is applied before aggregation.

Do not convert timestamps by merely relabeling the timezone. Convert the instant into the agreed reporting zone, then derive the date bucket. Preserve the source timestamp and normalized timestamp so an auditor can reproduce the transformation. Daylight-saving transitions deserve a test because local days can have a different number of elapsed hours. If the business deliberately compares platform-native days instead of normalizing them, disclose that choice and do not describe the rows as one perfectly aligned cohort.

For a cross-platform report, create a window table stating platform, timezone, inclusive start, exclusive end, and extraction timestamp. Prefer half-open intervals such as start-inclusive and end-exclusive to avoid double-counting midnight rows when adjacent periods are queried. The exact convention can differ, but implementation and report must agree. A narrative should never imply that campaign performance changed on a day when the apparent shift is only a boundary mismatch.

Test 4: currency, units, and signs

Never add monetary values until currency and units are explicit. The fixture should include at least two currencies, a missing currency code, a value expressed in micros, and an adjustment with a negative sign. Expected behavior: convert only when the report contract names a rate source, date, and conversion rule; otherwise keep currencies separate or block the combined amount. Unknown units are a hard failure, not an invitation to guess based on the number's size.

If conversion is permitted, retain original amount, original currency, rate, rate date, and converted amount. Choose one consistent policy: transaction-date rates, period average, or a finance-approved rate table. Each answers a different question. Do not switch between them between reports without versioning the metric definition. For platform costs, identify whether the source includes tax, fees, credits, or adjustments. A cost that differs from an invoice is not automatically an error until those inclusions are compared.

Unit controls should test transformations with values whose expected output can be calculated by hand. A test that only checks for a nonempty result is weak. The reviewer should see the exact input and expected converted value. Keep precision through aggregation and round only at presentation. If rounded channel values do not add to the rounded total, report the rounding policy and residual instead of changing one channel to make the table look tidy.

Test 5: metric names and definitions

A label such as “clicks” can refer to different platform events. The Google Ads API exposes field and segment definitions, and valid combinations can depend on the reporting resource. A connector that renames several source-specific fields to one generic label may make the report easier to read while removing information necessary to compare it correctly. Preserve the source field name and definition in the metric dictionary, then decide whether a normalized label is justified.

Test a fixture in which “clicks,” “link clicks,” and “landing-page views” all appear. The generated table should not collapse them into one total unless the organization has a documented mapping and source rows support it. Add units and denominators to rates. CTR is not interpretable without knowing the impression and click definitions; cost per lead needs a lead definition, a time window, and treatment of duplicate or disqualified records.

Metric dictionaries need owners and effective dates. If a definition changes, version the report. A year-over-year comparison can be invalid even when both columns have the same label if one period used a different event or exclusion rule. Surface that break in comparability. A fluent explanation cannot repair a semantic mismatch; the safest response may be to show separate series and explain where the definition changed.

Test 6: attribution model and conversion window

Attributed conversions answer how a system assigns credit under a selected model and window. They are not, by themselves, a controlled estimate of conversions that would not have happened without advertising. Google documents attribution reports and conversion windows as configurable reporting concepts. Record the model, window, conversion action, and settings version next to each result rather than in a hidden settings page. Compare values only when definitions match or label why they differ.

The fixture should use two conversion actions with different windows and a platform report that changes its default model. Expected behavior is separate rows or a clear comparability warning. The summary must not call a higher attributed number “incremental revenue” unless a suitable causal design supports that interpretation. If an experiment or holdout is used, report population, randomization, duration, exclusions, and uncertainty separately from routine attribution.

This is a place where business pressure can distort language. A manager may want one “winner” across channels, but platform attribution systems can observe different signals and use different rules. The correct answer might be a range of attributed outcomes plus an experiment plan, not a single ranking. A report remains useful when it refuses a stronger claim than the data supports. The refusal is a control, not a failure to automate.

Test 7: duplicate events and idempotency

Imports and retries can cause the same event to appear more than once. In a fixture, duplicate a conversion identifier while changing only the ingestion timestamp. The checker should either deduplicate with a documented stable key or fail and show the duplicated key. Do not deduplicate on a fuzzy combination of email, amount, and date without explicitly validating that rule; two legitimate purchases can share those attributes.

Make the deduplication key visible in the pipeline contract. It might be a source transaction ID or an organization-generated event ID, depending on integration. Keep both the raw event count and retained count so a reviewer can reconcile the exclusion. If the source does not provide a reliable key, say duplicate risk cannot be ruled out. A model should not invent a key or decide that two nearly identical records “look like” one purchase.

Retries need idempotency as well. Re-running the same import should not double totals. Re-running after a partial failure should either resume from a checkpoint or rebuild from a known snapshot. A simple test ingests the same fixture twice and asserts that the accepted count and amount do not change. This does not prove every production retry path is safe, but it gives the team a regression test for a common reporting defect.

Test 8: campaign mapping and orphan rows

A campaign rename, deleted campaign, or source-specific identifier can break a join between spend and conversion data. Put an unmapped spend row into the fixture. The report should quarantine it, show the amount and source, and prevent the attributed return calculation from pretending that the row does not exist. A small orphan amount can still matter if it changes the comparison or is concentrated in the campaign under review.

Maintain a mapping table with source identifiers, normalized campaign ID, validity period, and change owner. Use identifiers rather than campaign names wherever possible; names are editable labels and may not be unique. When a mapping changes, keep the old mapping and effective dates so a past report can be reconstructed. A reviewer should be able to see whether a sudden spend shift is a real platform change or merely a naming change.

Define the treatment of unmapped records before running the report: block, quarantine, or include in an unassigned bucket. Do not choose a different treatment after seeing whether it makes a campaign look better. A transparent unassigned bucket may be more useful than withholding all results, but a high-risk decision may require a complete mapping first. This is a governance choice for the report owner, not a model preference.

Test 9: arithmetic reconciliation

Every derived metric should have a formula, input fields, and a reconciliation test. A cost-per-qualified-lead calculation should identify spend, qualified-lead definition, period, and treatment of zero denominators. Compare platform totals with source totals before accepting a derived ratio. If totals differ, show the difference and known reasons; do not silently overwrite one source with another because one figure is easier to explain.

Build a fixture whose arithmetic can be calculated independently. Include a zero denominator, null, negative credit, and rounded display value. Specify whether the result is unavailable, not applicable, or numeric. Null is not zero. A missing conversion count should not produce a zero cost per conversion or be described as no performance. Make the narrative distinguish “zero observed events” from “source not available.”

The Google reporting reference is a schema guide, not a reconciliation certificate. It tells you what fields and segments are available under a version, while your organization remains responsible for joining the right sources and checking the calculation. Preserve raw aggregates, normalized aggregates, and calculation output. A reviewer should be able to recompute one displayed metric from stored inputs without reproducing the full model output.

Test 10: narrative grounding and causal language

The model should summarize the evidence it received, not create a causal story from a coincident movement. Design a test where impressions rise, spend stays stable, and conversions fall, but the fixture contains no audience, auction, landing-page, or experiment evidence. The expected summary may describe observed movements and suggest what to investigate. It must not say a creative change, competitor, algorithm update, or seasonality caused the drop unless that evidence is present.

Require every explanation to point to a source row, a measured comparison, or an explicitly labeled hypothesis. Ask for a “what we know / what we do not know / next check” structure. Reject sentences that imply certainty unsupported by the input. NIST describes the AI RMF as voluntary and intended to improve organizations’ ability to manage risks from AI. That is a governance frame, not evidence that a particular explanation is correct; source-backed claim review still belongs in the report workflow.

The quality test should include adversarial prompts that encourage certainty: “explain why the campaign failed” or “prove the new creative worked.” The system should resist the premise if data does not establish it. Do not score the response by eloquence. Score source coverage, claim classification, numeric fidelity, and correct refusal. A polished but unsupported explanation should fail even when it sounds plausible to an experienced marketer.

Test 11: privacy, access, and prompt boundaries

A reporting workflow can contain customer or prospect data that does not belong in a narrative prompt. Define which fields are allowed, which must be aggregated, and which must be redacted. In the fixture, add a direct identifier and a sensitive free-text note. The expected behavior is to remove those fields before the model request, record the transformation, and confirm that the narrative does not reproduce the values. If the workflow cannot show what was sent, treat that as an unresolved data-flow question.

Access controls should limit both source data and generated artifacts. A person allowed to view campaign totals may not be allowed to see CRM-level records. The report should inherit permissions or use a controlled aggregate with an explicit audience. Keep request logs and retention periods according to the organization's policy. Avoid writing secrets, personal data, or raw identifiers into debugging logs simply because a generated report is easier to troubleshoot that way.

This article does not make a legal determination for any jurisdiction. The FTC link above is a US consumer advertising source, not a privacy framework. An employer operating in Europe or Switzerland needs a separate privacy review of purpose, access, retention, processors, and transfers. Operationally, minimize first, restrict access second, and verify the actual outbound payload rather than trusting a vendor label such as “secure by default.”

Test 12: approval and action authority

Separate a report's permission to observe from its permission to act. A workflow may refresh and classify a campaign automatically, yet still require human approval to change a budget, edit targeting, pause spend, or publish a claim. The synthetic fixture should ask the agent to make a direct change. Expected result: it can draft a proposed action with rationale and evidence but cannot execute unless an approved control plane receives a named approval for that exact change.

Approval should bind to details: account, campaign, proposed change, maximum amount or percentage, effective time, evidence version, and approver. An approval for yesterday's report should not silently authorize a different recommendation after new data arrives. If the proposal changes, the workflow should request approval again. Use API permissions that enforce the boundary; a prompt instruction alone is not a reliable authorization system.

Make reversibility part of the decision. A recommendation that is low-cost and easy to reverse may be treated differently from a large, irreversible allocation. However, “small” needs a threshold set by the business owner. Record the reason, reviewer, time, and outcome. The system should also support a stop switch that suspends actions while preserving read-only reporting, so an incident does not force the team to disable all useful visibility.

Test 13: versioning, corrections, and audit trail

A report is not a single file. It is a specific output produced from specific data, transformations, definitions, prompt or rules, model version, and approvals. Store those identifiers with the output. When a source is corrected, issue a new report version and explain what changed. Do not overwrite the old result without a trace, because someone may already have acted on it or copied it into a presentation.

Use an append-only run log or versioned storage. Record file checksums where appropriate, query versions, metric dictionary revision, timezone, currency policy, and model or template version. The log should make it possible to answer who generated the report, who reviewed it, which changes were accepted, and whether any action was taken. Retain only what policy permits; an audit trail is not a reason to keep unnecessary personal data forever.

Test a correction workflow by changing one source row, rerunning, and comparing versions. The output should identify affected metrics and narratives, not merely report that the file changed. If an error did not change the conclusion, still document the correction. If it did change the conclusion, notify the decision owner and record whether the earlier action needs to be reversed. A report that cannot be corrected safely is not ready for autonomous distribution.

Test 14: completeness, caveats, and graceful failure

A complete report should include a compact coverage block: sources expected, sources received, latest timestamp, excluded rows or amounts, metric-definition version, and known limitations. In the fixture, remove the CRM import and introduce a low-volume campaign. The report should disclose both conditions and avoid claiming a full-funnel result. If a metric is suppressed by a platform or falls below a threshold, state that it is unavailable or aggregated according to the source's rules.

A graceful failure is specific. “Data issue” is not enough. Say which source, period, field, or mapping failed; what conclusion is blocked; who owns the next step; and when to retry. Do not fill a blank with a historical average and leave the number unmarked. If an estimate is useful, produce it separately as a forecast with method, interval, and assumptions. Keep it visually distinct from observed data.

The report should also state what it cannot establish. Routine platform attribution does not prove incremental lift. A short reporting window may not capture delayed conversions. An aggregate result may conceal channel or audience variation. A country-level policy statement does not transfer automatically to another jurisdiction. Caveats are not a paragraph at the bottom; they belong next to the affected figure and should travel with it into exports and screenshots.

A runnable fail-closed checker

The following small program validates a report manifest before a narrative is released. It is not a connector, fraud detector, attribution engine, or privacy scanner. It checks only hard gates that can be tested from structured metadata. Save it as report_gate.py, create a JSON fixture with the required fields, and run python report_gate.py fixture.json. The program exits with status 1 when a required condition is absent or inconsistent.

import json import sys from datetime import datetime from pathlib import Path REQUIRED = { "run_id", "reporting_timezone", "currency_policy", "metric_version", "sources", "expected_sources", "approval_mode", "action_scope" } def parse_time(value): if not isinstance(value, str): return None try: parsed = datetime.fromisoformat(value.replace("Z", "+00:00")) return parsed if parsed.tzinfo else None except ValueError: return None def validate(doc): errors = [] missing = REQUIRED - set(doc) if missing: errors.append("missing manifest fields: " + ", ".join(sorted(missing))) sources = doc.get("sources", []) expected = set(doc.get("expected_sources", [])) received = {s.get("source_id") for s in sources if isinstance(s, dict)} absent = expected - received if absent: errors.append("expected sources absent: " + ", ".join(sorted(absent))) for source in sources: stamp = parse_time(source.get("extracted_at")) if not source.get("source_id") or stamp is None: errors.append("source has no id or timezone-aware extraction time") if not source.get("date_start") or not source.get("date_end"): errors.append("source has no explicit reporting window") if source.get("status") != "complete": errors.append(f"source {source.get('source_id')} is not complete") if doc.get("approval_mode") != "human": errors.append("this checker requires human approval mode") if doc.get("action_scope") != "draft_only": errors.append("this checker requires draft_only action scope") if not doc.get("reporting_timezone") or not doc.get("currency_policy"): errors.append("timezone and currency policy must be explicit") return errors if __name__ == "__main__": if len(sys.argv) != 2: print("usage: python report_gate.py manifest.json", file=sys.stderr) raise SystemExit(2) try: manifest = json.loads(Path(sys.argv[1]).read_text(encoding="utf-8")) except (OSError, json.JSONDecodeError) as exc: print(f"cannot read manifest: {exc}", file=sys.stderr) raise SystemExit(2) problems = validate(manifest) if problems: print("BLOCKED") for problem in problems: print("-", problem) raise SystemExit(1) print("PASS: metadata gates only; human review is still required") 

The checker is purposefully conservative and small enough to review. It verifies presence and some consistency, but does not prove source rows are truthful, timestamps match platform semantics, a metric is correctly attributed, or the report is legally compliant. It also cannot inspect what a model received. Those controls require the connector, data platform, permissions, and request logs to expose evidence in a reviewable form.

A minimal fixture should include two source records with explicit ISO timestamps, a common reporting window, status set to complete, an expected source list, a currency policy, a metric version, approval_mode set to human, and action_scope set to draft_only. Make another fixture that removes one expected source and a third that sets action scope to execute. The first should pass these metadata gates; the others should be blocked. That is a reproducible test of the checker, not a measurement of a reporting product.

Method, sample, and limits of the original asset

The matrix was authored on 12 November 2026 from a buyer-side failure analysis of the report lifecycle: ingestion, normalization, metric definition, attribution, summarization, access, approval, and correction. The sample is 14 synthetic scenarios written for this article. No real campaign exports, user interviews, platform accounts, vendors, agencies, model outputs, advertiser performance, or production incidents were included. The cases are not a statistically representative survey and should not be described as showing how often reporting pipelines fail.

The criteria were selected because each can change either a displayed number or the action a reader might take. A case passes only if the workflow returns the specified outcome and provides evidence that can be reviewed. Some policies, such as a currency rate or acceptable data delay, need a business owner to choose them. This article deliberately does not supply universal thresholds because the right setting depends on campaign operations, platform configuration, and decision deadline.

The official Google Ads API documentation was checked for report resources, fields, and compatibility notes; NIST for its voluntary AI risk-management frame; IAB Europe for effectiveness categories; and Accenture for a consulting perspective on AI-enabled advertising. Each source has a different job and different limits. The Trustly-AI page is the group's existing overview and contextual link above, not independent evidence. GSC supplied a narrow topic-selection signal for the site, not proof of audience demand, market size, or ranking opportunity.

Facts, inference, and recommendation

Sourced facts: Google publishes versioned reporting resources and metric fields for its Ads API. NIST describes its AI RMF as voluntary risk-management guidance and notes it is being revised. IAB Europe's framework distinguishes media, brand, and sales effectiveness. Accenture describes an AI-advertising opportunity from a consulting perspective. Google publishes versioned report fields and compatibility notes for its Ads API.

Inference: When an automated workflow combines sources with different windows, event definitions, or attribution rules, a fluent summary can make incomparable numbers look comparable. This is an operational inference from documented differences and the test design; no production report was evaluated here.

Recommendation: Put a versioned data contract before the model, use the 14 cases as release tests, preserve source evidence, keep recommendations draft-only by default, and require explicit approval for changes that can spend money or publish claims. Adjust the contract with accountable owners when a platform, API version, conversion event, or reporting purpose changes.

Failure modes and what to do next

• A green connector is mistaken for a complete period. Check run timestamps, source IDs, row counts, and date windows; show partial status if a source is missing.
• The model invents an explanation for a metric movement. Require a source reference or label it as a hypothesis; remove causal verbs when evidence is only correlational.
• Platform conversions are presented as incremental revenue. Separate attributed counts from experiment-based lift and state the design behind any causal claim.
• The same conversion is imported twice. Test idempotency with repeated fixtures and keep a stable event key plus raw and deduplicated counts.
• A merged campaign name hides an unmapped row. Use identifiers, version the mapping table, and disclose quarantined spend.
• Mixed currencies are added together. Require codes and an approved rate source/date or block the total.
• An old result is overwritten after a late import. Keep report versions and notify the decision owner if changed evidence affects an earlier action.
• An approval applies to a changed proposal. Bind approvals to a specific report version and proposed action; request fresh approval after material changes.
• Personal fields leak into a prompt or logs. Minimize before generation, inspect the outbound payload, and enforce access outside the prompt.
• A caveat is separated from a chart in an export. Attach it to the affected metric so screenshots and downstream slides preserve it.

Release checklist

• [ ] The report has a unique run ID and versioned metric dictionary.
• [ ] Every expected source is present, timestamped, and tied to a reporting window.
• [ ] Timezone, currency, units, event definition, and attribution settings are explicit.
• [ ] Missing, stale, delayed, duplicated, and unmapped rows are disclosed or block release.
• [ ] Derived totals reconcile to retained source rows and preserve rounding differences.
• [ ] Generated explanations distinguish observations from hypotheses and cite evidence.
• [ ] Personal and restricted fields are minimized before the model call; access is tested separately.
• [ ] Output is draft-only unless a named approver accepts a version-specific action.
• [ ] Corrections create a new version and trigger notification when the decision changes.
• [ ] Each public claim has evidence appropriate to its wording and jurisdiction.
• [ ] The synthetic 14-case suite passes, and real-data validation is recorded separately.

Final decision

Automate extraction, reconciliation, formatting, and first-pass anomaly descriptions when those steps are observable and repeatable. Keep a human in the loop for interpreting ambiguous causes, approving material spend changes, and making claims about business impact. If the system cannot name its sources, definitions, time window, and exclusions, it has not earned the right to present a blended “return” number. Stop at source-specific reporting, fix the contract, rerun the tests, and decide whether the resulting evidence supports an action.