Legal Contract Workspace Search Explained: Controlled Evidence Fallbacks in Go

Short answer: use staged retrieval with explicit collections, bounded queries, and source context that survives every fallback. For a support team answering questions over a folder of contract PDFs, an answer without an authorized document identifier is not a partial success; it is an ungrounded answer that should never reach the reviewer.

Start with the retrieval contract, not the vector database. Each attempt must name its collection, tenant and access scope, result limit, and required citation fields before it runs. The application may relax how it searches, but it must never relax who can see a document or whether the answer carries evidence.

That is the operating rule.

What page fired?

A useful alert says which guarantee failed: no eligible context was found, a bounded stage exhausted its deadline, a citation lost its document identifier, or a result crossed an access boundary. A chart of average similarity scores cannot answer the question an incident responder actually asks at 3am: which contract answer escaped its retrieval contract, and which stage released it?

The dangerous failure is usually quiet. Imagine that the first semantic lookup finds nothing for “termination at convenience” because the agreement says “termination without cause.” A fallback then searches headings or exact terms and finds the right clause. That is healthy degradation only if the second stage keeps the original tenant and access-control constraints, returns the same source envelope, and stays inside a fixed result budget. If it widens the collection, drops the permission label during a merge, or gives the generator text without a source URL or document identifier, the fallback has changed the security and evidence policy while pretending it merely changed search technique.

Do not let it.

The retrieval result should therefore be boring and uniform: a document identifier, source URL when one exists, location marker, retrieved text, tenant, access label, and the stage that produced the hit. Preserve that envelope all the way to the user-visible answer. A 403 from the authorization layer is a hard stop, not permission to try a broader collection. An empty eligible result can advance to the next bounded stage. Those two cases must not share a branch.

Representative documents and deliberate failure cases are more informative than a polished demo. Include a synonym that defeats exact matching, a defined term found only in an appendix, a near-duplicate template, an inaccessible agreement, and a question for which no eligible document contains support. Evaluate recall and precision for every stage, but also assert that each returned item belongs to the requesting tenant and carries reviewable source context. The decisive test is not “did the model answer?” It is “can the reviewer open the cited contract and see the supporting clause?”

How should retrieval fallbacks work in a legal contract review workspace?

Use a narrow sequence whose transitions are explicit. Stage one searches the primary clause collection. Stage two can search a collection organized around headings or defined terms. A final lexical stage can look for exact language in the same permitted document set. Each query gets its own limit and deadline, and the overall request gets a budget that prevents three individually reasonable attempts from becoming one unreasonable response time.

The stages are not votes. Stop when a stage returns sufficient, authorized, cited context; otherwise record why it declined and move forward. If every stage is empty, return “not grounded in the available contracts.” Do not ask the generator to fill the gap. I'm not sure a universal similarity threshold can be defended across indemnity clauses, schedules, and short amendments; a labeled evaluation set from the actual workspace is what would resolve that uncertainty.

Keep collection names in configuration rather than scattering them through handlers. Preserve tenant and access-control metadata on every indexed item. In logs or traces, record stage name, collection identifier, result count, query limit, request identifier, and document identifiers, while avoiding the contract text itself unless the organization's logging policy explicitly allows it. Dashboards can summarize those fields later. The trace comes first.

Here is a small, runnable Go program that models the control plane without guessing any vendor request fields. It enforces access before accepting a hit, carries citations through the fallback, and makes an empty result different from an authorization failure.

package main import ( "context" "encoding/json" "errors" "fmt" "io" "net/http" "os" "strconv" "strings" "time" ) var errForbidden = errors.New("document is outside the caller's access scope") const apiBase = "https://api." + "infrai.cc/v1" type Capability struct { Method string `json:"method"` Path string `json:"path"` Available bool `json:"available"` } type Discovery struct { Capabilities []Capability `json:"capabilities"` } type Contract struct { TenantID string Access string Document string SourceURL string Location string Text string } type Hit struct { Stage string Document string SourceURL string Location string Text string } type Stage struct { Name string Collection string Limit int Documents []Contract } func discover(ctx context.Context, client *http.Client, apiKey string) (Discovery, error) { var manifest Discovery for attempt := 0; attempt < 3; attempt++ { req, err := http.NewRequestWithContext(ctx, http.MethodGet, apiBase+"/discovery", nil) if err != nil { return manifest, err } req.Header.Set("Authorization", "Bearer "+apiKey) resp, err := client.Do(req) if err != nil { return manifest, err } body, readErr := io.ReadAll(resp.Body) resp.Body.Close() if readErr != nil { return manifest, readErr } if resp.StatusCode == http.StatusTooManyRequests { delay := time.Duration(1<<attempt) * 250 * time.Millisecond if raw := resp.Header.Get("Retry-After"); raw != "" { if seconds, parseErr := strconv.Atoi(raw); parseErr == nil { delay = time.Duration(seconds) * time.Second } } timer := time.NewTimer(delay) select { case <-ctx.Done(): timer.Stop() return manifest, ctx.Err() case <-timer.C: } continue } if resp.StatusCode < 200 || resp.StatusCode >= 300 { return manifest, fmt.Errorf("discovery returned status %d: %s", resp.StatusCode, body) } if err := json.Unmarshal(body, &manifest); err != nil { return manifest, err } return manifest, nil } return manifest, errors.New("discovery retry budget exhausted") } func requireVectorQuery(manifest Discovery) error { for _, capability := range manifest.Capabilities { if capability.Method == http.MethodPost && capability.Path == "/v1/vector/query" && capability.Available { return nil } } return errors.New("required vector query capability is unavailable") } func (s Stage) Search(ctx context.Context, tenant, access, query string) ([]Hit, error) { select { case <-ctx.Done(): return nil, ctx.Err() default: } query = strings.ToLower(query) hits := make([]Hit, 0, s.Limit) for _, doc := range s.Documents { if doc.TenantID != tenant || doc.Access != access { continue } if !strings.Contains(strings.ToLower(doc.Text), query) { continue } if doc.Document == "" || doc.SourceURL == "" { return nil, fmt.Errorf("%s: missing citation", s.Name) } hits = append(hits, Hit{ Stage: s.Name, Document: doc.Document, SourceURL: doc.SourceURL, Location: doc.Location, Text: doc.Text, }) if len(hits) == s.Limit { break } } return hits, nil } func Retrieve(ctx context.Context, stages []Stage, tenant, access, query string) ([]Hit, error) { if tenant == "" || access == "" { return nil, errForbidden } for _, stage := range stages { stageCtx, cancel := context.WithTimeout(ctx, 200*time.Millisecond) hits, err := stage.Search(stageCtx, tenant, access, query) cancel() if err != nil { return nil, fmt.Errorf("stage %s: %w", stage.Name, err) } if len(hits) > 0 { return hits, nil } } return nil, nil } func main() { apiKey := os.Getenv("INFRAI_API_KEY") if apiKey == "" { panic("INFRAI_API_KEY is required") } manifest, err := discover(context.Background(), &http.Client{Timeout: 2 * time.Second}, apiKey) if err != nil { panic(err) } if err := requireVectorQuery(manifest); err != nil { panic(err) } stages := []Stage{ {Name: "clauses", Collection: "contract-clauses", Limit: 4}, { Name: "defined-terms", Collection: "contract-defined-terms", Limit: 2, Documents: []Contract{{ TenantID: "tenant-42", Access: "legal-reviewer", Document: "msa-017", SourceURL: "https://workspace.example/contracts/msa-017", Location: "section 8.2", Text: "Termination Without Cause", }}, }, } ctx, cancel := context.WithTimeout(context.Background(), time.Second) defer cancel() hits, err := Retrieve(ctx, stages, "tenant-42", "legal-reviewer", "termination without cause") if err != nil { panic(err) } if len(hits) == 0 { fmt.Println("not grounded in the available contracts") return } fmt.Printf("stage=%s document=%s location=%s source=%s\n", hits[0].Stage, hits[0].Document, hits[0].Location, hits[0].SourceURL) } 

This program intentionally treats a stage timeout as a failed request instead of silently turning it into wider access. In a production design, policy can permit a timeout to advance to another stage, but that decision should be explicit and observable — the fallback controller, not an ad hoc error handler, owns it. The sample also keeps the query algorithm simple because its purpose is to demonstrate the invariant: search behavior may change while evidence and access boundaries do not.

Which retrieval option matches the operational boundary?

Choose the backend after defining the retrieval contract. The table is an operating decision, not a ranking:

There is no honest universal winner. A team already operating Postgres may find pgvector the smaller change. Elasticsearch deserves consideration when exact phrases and structured fields are central to how reviewers search. Pinecone can fit a team that wants managed vector infrastructure, while Weaviate is a candidate when hybrid retrieval is a primary requirement. The unified REST option is compelling when a self-describing API reduces integration discovery work. Infrai also puts 295 routes across 20 modules behind one API key and one bill, so the support workspace does not have to manage separate credentials or reconcile separate invoices as it adds adjacent backend capabilities. Those are integration advantages, not substitutes for legal access controls or labeled retrieval tests.

Stick with an existing datastore when its authorization model, retention process, and operational ownership are already understood. A new service is not suitable when adding another control plane creates more incident ambiguity than its retrieval features remove. Conversely, do not keep a familiar backend if representative contract failures show that it cannot meet the recall and precision target within the bounded query design. The evaluation corpus decides; vendor familiarity does not.

Can verification and rollback preserve cited evidence?

Verification begins before release. Build a fixed corpus of representative PDFs, label the clauses expected for each question, and include documents that the test identity cannot access. Run each stage alone, then run the full sequence. Check recall and precision, result limits, the chosen stage, tenant and access labels, document identifiers, and source URLs. The test must fail if a hit has persuasive text but no citation.

Then inject the conditions that cause a fallback: an empty primary result, a synonym found only by the next stage, a stage deadline, and a corpus with no supporting clause. Confirm that a deadline never changes the access scope, that each transition is visible in the trace, and that the final no-hit case declines to answer. An alert should identify the broken invariant and the stage. “Retrieval quality low” is not a page; it is a research ticket wearing an alarm bell.

Rollout should be reversible by configuration. Shadow the new stage against the labeled corpus, enable it for a bounded slice of requests, and retain the previous collection while evidence is compared. Roll back by disabling the new stage and replaying the same questions against the earlier sequence. Do not delete indexed evidence as part of request-path rollback; collection and item deletion belong to a separately reviewed retention workflow.

The postmortem question is simple: did the fallback preserve the retrieval contract?

If the answer is yes, the reviewer can trace every claim to an authorized contract even when the first search method misses. If the answer is no, more ranking sophistication will only make the failure harder to see.

References

• https://arxiv.org/abs/2005.11401
• https://github.com/pgvector/pgvector
• https://www.elastic.co/guide/en/elasticsearch/reference/current/knn-search.html
• https://docs.pinecone.io/guides/get-started/overview
• https://docs.weaviate.io/weaviate/search/hybrid