Quick answer: Rate-limit a multi-tenant MCP gateway at two levels. Give each tenant a bucket keyed to a distinct access key, and enforce a shared budget that matches upstream capacity. Per-tenant limits stop one noisy agent from starving the others; the shared budget stops well-behaved tenants from overwhelming the backend together. Return 429 with Retry-After on each rejection, and count rejections per tenant so a misconfigured agent shows up early.
Why a shared gateway needs per-tenant limits
When several teams or agents share one MCP gateway, they share one pool of upstream connections and backend quotas. Without tenant-aware limits, capacity is first-come, first-served: one agent in a retry loop can hold most connections and burn the shared budget, leaving other tenants on 429s though they changed nothing. The failure is quiet, since each request the noisy tenant sends is valid and the gateway stays healthy. A per-tenant limit makes shared capacity a defended resource, so excess traffic is rejected at the gateway before it reaches the backend.
Identify the tenant before you count
A limiter is only fair if it knows who is calling. Authenticate each agent with a distinct key and resolve it to a stable tenant identifier before touching any counter; shared keys make per-tenant accounting impossible. Powerduck Cloud keeps the hosted MCP endpoint off by default until you expose it and issues a separate MCP access key for each agent, so every caller has a distinct identity the gateway can count and revoke independently. The comparison hub places this access model among the wider trade-offs.
Choose the limit shape that matches the traffic
MCP traffic is not only request rate. Streamable HTTP and SSE hold long-lived connections, so concurrent connections are a separate dimension from requests per second, and a heavy tool call costs far more than a light one.
| Limit dimension | What it controls | Watch out for |
|---|---|---|
| Requests per interval | Raw call volume per tenant | Misses long-lived and heavy calls |
| Concurrent connections | Open SSE and HTTP streams | A few agents can hold every connection |
| Weighted token bucket | Cost-weighted calls | Needs a weight per tool |
| Shared upstream budget | Total backend capacity | Binds every tenant together |
A token bucket allows a controlled burst at a steady average, which fits agents that call in batches. A sliding window smooths output more strictly but costs more to track; a fixed window lets a tenant spike at the boundary. Most gateways use per-tenant token buckets and cap connections separately.
Two budgets, not one
Per-tenant buckets alone are not enough. If every tenant stays under its own limit but the totals exceed backend capacity, the upstream still rejects requests that land on tenants who did nothing wrong. Run a global budget that mirrors the upstream ceiling, checked after the tenant bucket in a layered token bucket. When the shared bucket is empty, the gateway returns its own 429 instead of relaying the upstream error. To give one tenant headroom, raise that tenant's bucket rather than loosen the global ceiling.
Return clear rejections and watch the counters
A rejection should say what happened and when to retry. Return 429 with Retry-After, and distinguish a tenant-level rejection from a shared-capacity one in a header or log, since the remedies differ: fix the agent's cadence in the first case, add capacity in the second. Track per-tenant request count, rejection count, peak connections, and share of the global budget, and alert on a climbing rejection rate, which usually signals a retry loop. On a multi-instance gateway, keep counters in a shared store; per-node counters multiply the real limit by the node count.
Honest limits
Rate limiting enforces fairness, not availability. If the backend is down or the global budget sits above real capacity, correct per-tenant limits still produce 429s. Limiting is also not authorization: a caller within quota can still invoke every tool its key grants, so scope and rate stay separate. Request-count limits treat every call as equal and underweight a tenant that sends few calls but runs the heaviest tools; weight the bucket or cap concurrency if that fits.
Frequently asked questions
How is a tenant-level 429 different from an upstream 429? A tenant-level 429 means that caller spent its allocation and should slow down. An upstream 429 means shared backend capacity is exhausted, and the gateway should absorb it rather than forward traffic. The fix is per-tenant first, platform-wide second.
Should I key limits to the API key or the IP address? The API key or resolved tenant identity. IPs are shared behind NAT and load balancers, so an IP-based limiter either groups unrelated agents or misses the real source.
Do per-tenant limits remove the need for a global budget? No. They cap each tenant but not the sum. A global budget matching the upstream ceiling is still needed to keep well-behaved tenants from exceeding backend capacity together.
The bottom line
Treat a multi-tenant MCP gateway as a capacity boundary with two doors, the tenant bucket and the shared upstream budget, and make every caller present a distinct key first. Per-tenant limits without a global budget protect tenants from each other but not their combined load; a global budget without per-tenant limits lets the loudest agent decide who gets served. Keep counters shared across instances, return actionable 429s, and watch per-tenant rejection rates.