Privacy is one of the decisive factors that determine whether an enterprise AI project is ready to reach production. Sure, budget and model quality matter, but as we head into 2027, this matter seems to be taking center stage. Where does our data go?
One agent task can involve multiple models, and each call can send customer records, contracts, or source code to a different vendor. That’s definitely something agent developers should take into account from the start.
Lucky for you, this article will show you how the data flows, compare main deployment options, and offer you a checklist to help you prepare for your next security review.
Why Is Privacy the Deciding Factor Now?
To put it bluntly, adoption is outpacing control. Here’s what’s happening, in a nutshell:
| Source | What it found |
|---|---|
| Data security and privacy were ranked as the top concern for both generative AI and AI agents, at 76% each. The share is rising year after year. | |
| 86.9% of companies delayed AI deployments because security and governance weren’t ready, not because of budget or internal support. | |
| 69% said security concerns are slowing agent adoption down, and 83% named data leakage as the barrier. | |
| For 91% of leaders, data security, privacy, and risk concerns are the top factor shaping their AI strategy over the next 6 months. | |
| Shadow AI incidents (staff using unapproved AI tools) more than doubled, from 20% to 43%, and they cost more than the year before. | |
| About three quarters of organizations have an AI governance body, but only 12% call it mature. The State of AI 2026: Security Insights CISOs Need to Know +5 |
The EU AI Act deadline for high-risk systems has been extended, but the rules still apply. To be specific, under the Digital Omnibus on AI, stand-alone high-risk systems listed in Annex III (hiring tools, credit scoring, education) must now comply by December 2, 2027 instead of August 2, 2026.
Like it or not, regulation hasn’t gone anywhere. Privacy reviews are now part of shipping an agent.
Why Do Agents Make the Privacy Problem Bigger?
In its contract review agent walkthrough, Superlinked describes a review process that involves ten model calls. These include triage, vision, Optical Character Recognition (OCR), embeddings, reranking, entity extraction, text-to-SQL, risk reasoning, orchestration, and a safety guardrail. In a typical stack, these calls may involve several different vendor accounts.
Here’s where data leaves your system when a typical agent runs:
| Agent step | What it sends out | Why it matters |
|---|---|---|
| Embedding documents for Retrieval-Augmented Generation (RAG) | Every chunk of your knowledge base | Your entire document store passes through the provider. |
| Reranking | The query plus candidate passages | Sensitive passages leave your infrastructure on every search. |
| OCR and document parsing | Scanned contracts, IDs, invoices | These are often the most sensitive files you own. |
| Tool results fed back to the LLM | Database rows, emails, API responses | Data the user never typed ends up in a prompt. |
| Guardrail and moderation calls | User inputs and model outputs | Each service adds another processor to consider in your privacy review. |
| Agent memory, logs, and traces | Full conversation history | Your observability tools now store copies too. |
Hosted API vs. Self-Hosted vs. Hybrid: When Is Data the Safest?
Before comparing the options, you need to understand what large providers actually keep by default:
| Provider | Default retention | Zero Data Retention (ZDR) |
|---|---|---|
| Abuse monitoring logs are kept for up to 30 days. | Available only after OpenAI approves the request. | |
| Inputs and outputs are deleted within 30 days, except when longer retention is required, such as with Usage Policy enforcement or legal compliance. | Available by agreement, although some models still require limited data retention and review. |
Neither provider is careless with their customers’ data. OpenAI, for instance, states that API data hasn’t been used for training since March 2023 unless you opt in explicitly.
| Hosted API | Self-hosted open models | Hybrid | |
|---|---|---|---|
| Where data is processed | Provider’s infrastructure | Your cloud, on-premises, or air-gapped | Sensitive steps locally, the rest with the provider |
| Retention control | Provider policy; ZDR requires approval | Fully under your control | Mixed, but sensitive data stays yours |
| Model quality ceiling | Frontier models | Open models that must pass your evals | Frontier where you need it |
| Operations work | None | You manage capacity, upgrades, and incidents | Moderate |
| Best for | Low-volume work, frontier reasoning, small teams | Regulated data, air-gapped networks, steady volume | Most production agents |
| Main limitation | Reliance on an external processor whose policies can change | Security becomes your responsibility entirely | A routing layer required that you need to build and test |
How Do You Route Agent Tasks by Data Sensitivity?
For most teams, I’d recommend the hybrid approach as the most practical choice. Each call is routed based on the data it handles:
| Task | Raw data? | Suggested location |
|---|---|---|
| Embeddings, reranking, OCR | Yes, constantly | Inside your boundary |
| Personally Identifiable Information (PII) redaction | Yes, by definition | Inside your boundary before anything else |
| Classification and entity extraction | Usually | Inside your boundary |
| Complex reasoning on redacted context | Minimal, if you redact first | A hosted frontier model will do |
Many open-model servers (vLLM, SIE, and others) expose OpenAI-compatible endpoints. That means the router can be as small as:
from openai import OpenAI local = OpenAI(base_url="http://inference.internal:8080/v1", api_key="not-needed") frontier = OpenAI() # hosted provider, reads OPENAI_API_KEY def pick_client(task): if task.has_sensitive_data or not task.needs_frontier: return local return frontier But code is the easy part. The real challenge is deciding what really counts as has_sensitive_data. (•_•)
Privacy Checklist for Agent Developers
Here’s a checklist you can use prior to your security review:
- List every model call your agent makes, including guardrails, embeddings, and sub-agents.
- Classify the data each call handles. Is it public, internal, confidential, or regulated?
- Read the retention docs per endpoint for every hosted call. Stateful endpoints often keep data longer than plain completions.
- Apply for ZDR wherever you’re eligible and still need to send data out.
- Check the processing region against your data residency rules.
- Redact PII before data crosses your boundary, not after.
- Audit logs, traces, and eval datasets, since they collect real prompts.
- Give tools only the permissions they need. Use read-only database access where possible. For example, the contract review agent’s text-to-SQL is limited to SELECT queries, so it can’t write to the database. Superlinked
- Pin model versions to prevent changes in behavior.
- Turn off vendor telemetry in air-gapped or restricted networks.
- Test open models on your own data before you switch any task.
- Repeat the review whenever a provider updates its data policy.
When Does a Hosted API Still Make Sense?
Self-hosting doesn’t automatically make your data more private. In fact, a misconfigured cluster with open logs can be less secure than a well-governed API with ZDR.
A hosted API is still the right call when:
- The task needs a frontier model, and open alternatives don’t meet your evaluation criteria.
- Your volume is low or unpredictable, so a GPU would sit idle most of the day.
- You don’t have anyone to run infrastructure, including patching and incident response.
- The provider’s controls already satisfy your policy, such as ZDR plus regional processing.
Superlinked’s SIE vs. OpenAI comparison makes a similar point. When you self-host, you take responsibility for protecting the data. There’s no provider running the controls for you.
It all boils down to this: pick the option your team can manage properly.
Skipping the Infra Work
If the checklist above has got you thinking, “We need to self-host the sensitive steps, but we don’t want to build or maintain a model-serving platform,” well then, SIE (Superlinked Inference Engine) is the right fit for you.
SIE is an open-source inference server that runs encoders, rerankers, entity extractors, and generation models on your own infrastructure. You can run it on anything, from a laptop to a production Kubernetes cluster. It supports 100+ open models through four primitives: Encode, Score, Extract, and Generate.
If you want to test models before setting up a cluster, SIE Serverless offers a hosted version with SOC 2 Type 2 and US or EU data residency. However, keep in mind it’s still a hosted processor, so test it with non-sensitive data and move sensitive workloads to your own cloud. If you want help running it in your own cloud, take a look at Native Inference.
FAQ
Is data privacy really the top barrier to enterprise AI adoption?
Several 2026 surveys, including AvePoint, Okta, and KPMG, rank data security and privacy as a top concern for AI and agent adoption.
Do OpenAI and Anthropic train on API data?
Both say API data isn’t used for training by default, but they can keep inputs for up to 30 days unless you have a zero data retention agreement.
Is self-hosting automatically more private?
No. Self-hosting moves responsibility for access, logging, and retention to your team, so it’s only as private as your setup.
Can I use frontier models and still pass a privacy review?
Usually yes, as long as you redact sensitive data first, use zero retention where eligible, and send only the minimum context.
