How to deploy an AI assistant: Pattern 2, the control/data split (customer cloud)

A reference architecture for running the data plane of an AI assistant in the customer's own cloud account, with the provider keeping only a control plane

This document describes the second of three deployment patterns for an AI assistant platform. It is written for architects and platform engineers whose customer, or whose own organisation, has said: "our documents, our search index and our encryption key stay in our cloud account, under a key we can revoke without asking anyone."

The structure is the same as Pattern 1: a one-picture architecture, the request flow, the components, use cases, design considerations, what crosses the boundary, and when to choose it. The design is independent of any product; the technologies named can be replaced.

Terms used throughout. The provider is whoever builds and operates the platform software: a vendor, or an internal platform team. The customer is the organisation whose users and data the platform serves. In this pattern the customer also operates most of the system.

Prerequisites. On the customer's side: a cloud account, a Kubernetes service, managed Postgres and object storage, a key management service, and access to a cloud model provider in the chosen region. On the provider's side: a build pipeline that can sign artifacts and an OCI registry. The customer needs a team that can run Kubernetes and its supporting services in production.

The three patterns at a glance

The three questions still decide: where does the data live, who operates it, what crosses the boundary. For this pattern the third question is the one to get right, and the next section shows why.

The design to avoid: split by component

The obvious way to meet the customer's rule is to split by component: move the pieces that store customer content into the customer's account and keep the rest.

The customer runs retrieval over their own vector store, a connector module that reaches their systems from inside their network, a cache, their documents bucket, Vault and a KMS key they can revoke. The provider reaches these over one private link with mutual TLS. Software is pull-only. Stored data never leaves the customer's account.

Now trace one chat message and mark every arrow where customer content crosses an account boundary with ◆.

Five crossings per question. None of them is a breach: each is encrypted, disclosed, inside the residency boundary and gone within minutes. But the design cannot support the sentence the customer wants: "your data never reaches our account." Stored data does not; data in flight does, five times.

Two rules follow, and they apply to any architecture:

1. Mark every arrow where content crosses a boundary. If you cannot draw it, you do not know it.
2. "Defensible" is the floor. The useful question is whether the customer can verify the promise without trusting the provider.

Architecture

The fix is to split by plane, not by component.

• The control plane is everything that is the same for every customer and carries no content: building and signing software, publishing it, issuing licences, collecting health metadata, taking support requests. The provider keeps this.
• The data plane is everything that touches a user, a document, a system or a prompt: entry point, gateway, sign-in, chat, Orchestrator, tools and actions, retrieval, connectors, data, keys, and the model call. The customer runs all of it, in their account.

Three things change. There is no connectivity link, because there is nothing for the provider to reach, and the provider has no inbound path at all. The model call stays home: the Orchestrator calls the customer's own cloud model providers in the customer's account, under the customer's contract. There is nothing to crypto-shred at exit, because the provider holds no configuration, sessions or buffer. The ◆ marker appears nowhere.

Request flow

Every hop stays inside the customer's boundary. The provider's column in the detailed topology has no live-request arrows at all.

Components

Use cases

A customer whose data must stay in their own cloud account

The defining case. Documents, index and keys in the customer's account, encrypted under a key the customer can revoke, with no content in the provider's infrastructure. The customer can operate Kubernetes and its supporting services, or has a managed-service team that can.

A provider who must support several clouds

Because the customer runs the data plane, the provider's software has to run on whichever cloud the customer uses. The design keeps every cloud-specific dependency behind an interface: the model router, the identity binding for the registry pull, and the key management.

A regulated business unit inside one company

An internal platform team is the provider; a business unit with its own cloud account and its own compliance regime is the customer. The pull-only delivery and the egress proxy log give the business unit's auditors proof without trusting the platform team.

Design considerations

Security and privacy

Delivery without access. The provider never pushes anything. Its pipeline signs every chart and image, attaches an SBOM and a vulnerability-scan attestation, and publishes to an OCI registry. In the customer's cluster, Argo CD polls the registry over the customer's own outbound HTTPS and applies new versions; a release gate lets the customer review the SBOM and attestation first; Kyverno verifies the signature with a bundled, offline trust root before any pod starts. The customer holds every admin credential.

For cross-cloud registry access, two methods work: federate the cluster's workload identity into a pull-only role at the provider, or issue a per-customer pull-only robot token the customer keeps in their own Vault. Both are scoped to that customer's repositories and can only pull.

Identity inside the boundary. The customer runs the identity broker, and the identity provider sits inside their boundary. The IdP token is verified once at login; after that the browser holds an opaque, hash-stored session. CLI tokens and API keys resolve the same way. No sign-in assertion crosses to the provider.

Tools and actions in the customer's account. The sandbox is self-hosted in the customer's account with no platform credentials and default-deny egress; a managed sandbox service would be a third party that sees content. The action gateway applies the customer's policy and approval steps. Tool connectors run against an allowlist the customer sets. Per-user tokens are sealed in a credential vault with a key id. Metering writes to a local store with limits per organisation and workspace.

The model router points at the customer's own providers. Several providers behind one interface, each in the customer's account, pinned to a region the customer chooses, reached over private endpoints, with invocation logging as the customer's own setting. The prompt never touches the provider's account. The nuance to state plainly: the model provider still sees prompts, under the customer's contract rather than the vendor's.

Data and authorization. Postgres with pgvector, the documents bucket, Redis and Vault live in the customer's account. Authorization is one decision point: policy compiled into every query, with row-level security beneath it. Encryption uses a customer-owned KMS key. Revoke it and everything stops and becomes unreadable, including to provider support.

Reliability

High availability and disaster recovery are the customer's: multi-zone, a second region per their policy, immutable backups. The provider states no recovery targets here because there is no provider data plane to fail over. The provider commits only to registry availability, and an optional local registry mirror removes even that runtime dependency.

Operations

Telemetry is local by default: Fluent Bit, OpenTelemetry, Loki, Jaeger, VictoriaMetrics, a local Grafana. Cloud and cluster audit logs stay in the account. The egress proxy log is the customer's proof: it shows exactly two external names ever reached for the provider (the registry and, if opted in, the heartbeat), plus the customer's own model, identity and system endpoints.

Support works two ways, both optional: a redacted support bundle the customer reads before sending, or a break-glass session the customer opens, approves, time-boxes and records in their own audit log. The break-glass session is the one path that could ever show content to the provider, and the diagram says so.

Cost

The customer pays for the whole data plane: cluster, databases, model consumption under their own contract, a second region if they want one. Managed cloud services soften the operating cost but do not remove it. The provider's cost is small and fixed: a pipeline, a registry, a dashboard.

Performance

Model capacity, quota and regional placement are the customer's. The provider cannot add quota or fix an outage at the customer's model provider. Everything else scales as in Pattern 1, inside the customer's cluster.

Design alternatives

• Split by component (the first design above). Fewer things for the customer to run, but five content crossings per question.
• A managed sandbox service. Less to operate, but a third party that sees content.
• Push-based delivery with a provider credential into the customer's cluster. Simpler for the provider, but gives it an inbound path, which this pattern exists to remove.
• Automatic heartbeat. Useful for support, but it must stay opt-in and payload-free, or the egress proxy log stops being proof.

What crosses the boundary

Compare this with the first design's five content crossings per question. That comparison is the whole argument for splitting by plane.

Trade-offs

• The customer runs the whole data plane. This is close to the operating burden of an on-premises deployment, eased by managed cloud services.
• Support works half-blind: heartbeat and redacted bundles by default.
• Several model providers to support, with differences in models, limits and behaviour.
• No provider recovery targets. The provider commits only to registry availability.
• Version skew: N and N-1 are supported; a customer who stalls their release gate drifts out of the window.
• Not zero third parties. The customer's model provider still processes prompts.

When to choose this pattern

Choose the control/data split when the customer's rule is "our documents, index and keys in our account, under a key we can revoke, and no content in your infrastructure", and they can operate Kubernetes and its supporting services. If they want the provider to run everything, Pattern 1. If not even a prompt may leave their building, Pattern 3.

Deployment outline

The provider prepares; the customer deploys.

1. Provider: release pipeline. Build, sign every chart and image, attach the SBOM and scan attestation, publish to the OCI registry, issue the signed licence and a pull-only credential per customer.
2. Customer: foundation. Cloud account, network, Kubernetes cluster, managed Postgres with pgvector, object storage, Vault, a customer-owned KMS key, the model accounts with private endpoints.
3. Customer: delivery path. Argo CD pointed at the provider's registry over outbound HTTPS, the release gate, Kyverno with the bundled trust root, the optional registry mirror. Verify that the egress proxy log shows only the registry.
4. Customer: identity and edge. The identity broker inside the boundary, federated to the customer's identity provider; the customer's domain, certificate, WAF and load balancer; the API gateway.
5. Customer: application and tools. Argo CD applies the signed charts: Chat Gateway, Orchestrator, retrieval, connectors, mesh, the tools-and-actions layer, the model router configured for the customer's providers.
6. Customer: observability and support. Local telemetry, audit logs to the customer's SIEM, the opt-in heartbeat if wanted, the break-glass procedure written into the customer's runbook.
7. Customer: resilience. High availability, a second region if policy requires it, immutable backups, and a test that revoking the KMS key stops everything as intended.

Detailed topology

The full design in ten customer bands plus the provider's two control-plane bands. The hop numbers match the request flow table above. Open the figure at full size to read the component notes.

What's next

This is part 2 of a three-part series. The other parts are also linked in the series box at the top of this article.

• Pattern 1, the shared service (SaaS): the provider runs everything inside a residency boundary.
• Pattern 3, the isolated site (on-premises and air-gapped): everything runs in the customer's building, with models on local GPUs.