Short answer: for e-commerce product search, keep an append-only audit stream beside the retrieval architecture's index, and make every chunk, embedding, filter, and ranking decision replayable from a versioned event. That is the least complex way to explain a bad result without rebuilding yesterday's catalog from memory.
Logs are evidence.
I build RAG features in Python, so I tend to start in a notebook and then regret any assumption the notebook made silently. Product search makes those assumptions expensive: a catalog row can change price, stock, language, or policy text while a vector index still serves an older chunk. An audit log turns that drift into data we can inspect. The goal is not to log every internal detail. It is to preserve the inputs and decisions that change retrieval.
How should e-commerce product search retrieval architecture use audit logs?
Treat the log as a companion to the index, not as a dump of debug strings. A catalog event enters a normalizer, receives a stable product and document version, gets chunked, and is written to an embedding queue. The index stores vectors and searchable fields; the audit stream stores what was accepted, rejected, or replaced and why. A query trace then records which index generation answered it.
This separation matters during reindexing. If a seller edits a 2,000-word product description, the right question is not “which vector changed?” It is “which source version produced this chunk, under which chunking and embedding configuration, and did the old version remain searchable?” Those fields let an engineer replay one product instead of replaying the entire feed. In practice, I want the replay to answer a support ticket without a heroic incident session: take the request ID, resolve the index generation, locate the returned SKU, then compare its source version and hash with the event that created it. If a filter removed the item, the trace should show that branch; if reranking moved it, the before-and-after candidate positions should be visible. A vague line such as “semantic score was low” cannot tell a catalog operator whether the source was stale, the chunk boundary was wrong, or a policy filter did exactly what it was configured to do.
Here is a small, runnable event writer. It uses JSON Lines so the same records can land in object storage, a stream, or a relational staging table without changing the producer.
from __future__ import annotations import hashlib import json from dataclasses import asdict, dataclass from datetime import datetime, timezone from pathlib import Path @dataclass(frozen=True) class ChunkAudit: product_id: str source_version: str chunk_id: str text_sha256: str chunker_version: str embedding_model: str index_generation: str action: str reason: str recorded_at: str def make_event(product_id: str, source_version: str, text: str, chunk_number: int, index_generation: str) -> ChunkAudit: digest = hashlib.sha256(text.encode("utf-8")).hexdigest() return ChunkAudit( product_id=product_id, source_version=source_version, chunk_id=f"{product_id}:{source_version}:{chunk_number}", text_sha256=digest, chunker_version="catalog-chunker-3", embedding_model="embedding-model-v1", index_generation=index_generation, action="upsert", reason="source_changed", recorded_at=datetime.now(timezone.utc).isoformat(), ) def append_event(path: Path, event: ChunkAudit) -> None: with path.open("a", encoding="utf-8") as stream: stream.write(json.dumps(asdict(event), sort_keys=True) + "\n") event = make_event("sku-1842", "catalog-2026-08-31-07", "USB-C dock", 0, "gen-42") append_event(Path("audit.jsonl"), event) The example deliberately records a content hash rather than raw customer or seller text. You can keep the source document under its own retention and access policy, then use the hash to prove which text was indexed. I started with raw payloads once; a privacy review made it clear that an audit trail should answer an engineering question without becoming a second catalog.
What belongs in a retrieval audit record?
At minimum, preserve four identities: the source version, chunk identity, transformation configuration, and index generation. Add a reason code for actions such as source_changed, metadata_only, deleted, or rejected_schema. For query-time debugging, attach a request ID, normalized query hash, filter expression, candidate count, and the IDs returned before and after reranking. Store model names and configuration versions, not just a vague “semantic search” label.
A useful retention split is cheap and boring: keep compact events for the full legal or operational window, keep detailed query traces for a shorter window, and keep the source snapshot only as long as policy permits. Hashes, timestamps, and version IDs make the compact tier useful. They also make duplicate events detectable when a queue retries.
The log needs ordering semantics. Use an event ID and an ingestion timestamp, but do not pretend wall-clock time is a total order across workers. A source sequence number or partition offset is better for replay. If two updates arrive out of order, the indexer should apply the newest source version and write an explicit superseded event; silently overwriting the record creates an audit hole.
Choosing an index design without hiding its failure modes
Dense vectors are good at paraphrases such as “quiet keyboard for a shared office,” while lexical retrieval catches exact model numbers and material names. Hybrid retrieval usually gives a product catalog a more predictable baseline than choosing one signal. The audit record should capture the candidate sets from each branch, because a ranking complaint can originate in tokenization, filters, vector recall, or the merge function.
The cost axis is mostly about multiplication: number of chunks times vector dimensions times replicas, plus refresh and storage overhead. Smaller chunks can improve pinpoint matches but increase vector count and write traffic. Larger chunks reduce index entries but mix warranty, compatibility, and shipping text into one retrieval unit. Measure recall and index footprint together on a fixed evaluation set; a tiny gain in recall is not automatically worth another replica.
I keep an eval fixture with around 200 anonymized queries and expected product IDs. It is not a benchmark anyone should generalize from; your mileage may vary. It is a change detector. Every chunker or embedding version gets a side-by-side run, and the audit log lets me inspect the exact products that moved from rank 3 to rank 30.
A table helps make the trade-off explicit:
| Decision | Helps | Costs or risks | Audit evidence |
|---|---|---|---|
| 256-token chunks | Precise passages | More vectors and writes | Chunk count, boundaries |
| 768-token chunks | Lower index cardinality | Mixed intents in one hit | Token count, section label |
| Hybrid retrieval | Exact terms plus paraphrase recall | Two candidate pipelines | Branch scores and counts |
| Full query traces | Fast incident analysis | Retention and privacy work | Request ID, filters, result IDs |
Where this architecture should not be used
The catch is operational weight. If the catalog is a few thousand stable items and search is purely lexical, an append-only event stream plus vector index may be needless machinery; a database change log and a well-tested query plan are easier to own. Stick with a simpler design when there is no semantic retrieval, no frequent reindex, and no requirement to explain historical results.
It is also not suitable when audit data cannot be protected to the same standard as the catalog. Redact query text, separate identities from search traces, and define deletion propagation before launch. A log that cannot honor retention rules is a liability, regardless of retrieval quality.
An operational checklist that survives reindexing
Start with one immutable source version and one index generation. Before promoting a generation, replay a sample of audit events and compare product IDs, filter behavior, and cost metrics with the prior generation. During rollout, write both generations into the query trace so a rollback has evidence rather than guesswork.
Watch queue lag, rejected schema events, duplicate event IDs, chunk-count distribution, and the share of queries with zero candidates. Alert on changes in those distributions, not only on HTTP failures. When an on-call engineer receives a report that “the red jacket disappeared,” they should be able to follow request ID to candidate branches, then from a product ID to its source version and chunk hash.
Keep the notebook useful: export the same JSON Lines fixture that production replays, pin configuration versions, and make the evaluation command deterministic. The best architecture is the one a tired engineer can explain at 02:00, including why a particular chunk exists and what it costs to keep it searchable.
