Request logs are the first thing you reach for when something breaks and the last thing you want leaked. They are your best debugging tool and a concentrated store of exactly the data you told customers you would protect. The resolution is not a philosophy. It is a field list.
Split the log in two: metadata and content.
Metadata: log it generously
- Identity — request ID, key, organization. Which call was this, and whose?
- Shape — model, provider, prompt / completion / total tokens.
- Outcome — status, latency, cache-hit status.
- Money — the settled cost of that one call.
- Time — start and end timestamps.
Cost by model, latency percentiles, error regressions, cache effectiveness, per-key spend — all answerable from that list, without a single character of prompt text. The standards agree: OpenTelemetry's GenAI span convention requires the operation and provider names, recommends the model, finish reasons, response ID and usage token pair, and treats message content as opt-in because it is likely to contain sensitive information.
Content: a decision, not a default
The prompt and the completion buy you a narrow slice of extra value — reproducing one specific bug, sampling for quality review — and nearly all of the risk. On the gateway in the guide below, content is never written to the log; metadata always is, because metadata is the billing record. The data policy changes what the dashboard shows, not what is stored, and it is an admin-only lever. Full-content logging is a roadmap item, and the honest interim rule is: if you need the exact prompt weeks later, capture it on your own side, deliberately, with your own redaction.
Three disciplines that make the log trustworthy
- Capture the request ID. Every response carries it in a header. Store it beside your own trace ID on the success path and the exception path. Skip this and every investigation degrades to searching by timestamp.
- Absent is not zero. When a call cannot be priced, the cost header is omitted and a status header says
unpriced. Default the missing value to zero in your client and your spend dashboard drifts from the ledger forever, silently. - Redact before you copy. CSV exports, forwarded headers, a second observability stack — your copy can carry content too. Redaction after the copy is theatre.
And keep the categories apart: who changed a budget or rotated a key belongs to the audit trail, a separate store with different volume and retention. Retention itself is a policy with an owner — bounded windows, exports for what must outlive them. Keep aggregates long, raw rows short, content shortest of all.
The full guide — field table, the four-step debugging workflow, the gateway-logs-versus-self-hosted-traces comparison, and a five-step verification routine — is here: https://nrouter.ai/blog/guides/llm-observability-request-logs