Originally published on Riz Dynamics AI. Azure AI Gateway is in public preview and may change before general availability.
Most teams piloting Microsoft Foundry hit the same wall eventually: a model deployment with no spending ceiling, and no clean way to add one without touching application code. Azure AI Gateway -- a separate product from Azure API Management, currently in public preview -- sits in front of your Foundry deployments and gives you that ceiling, plus routing and failover across models.
Here's how to set it up, step by step.
1. Add a Model and Create an Alias
Add your Foundry model backend to the gateway, then create an alias on top of it. An alias is the stable endpoint your application code calls -- you can swap, add, or retire backend models behind it without changing a single line of client code.
2. Configure Rate Limits and Cost Limits
This is where the actual cost control lives. Set a token rate limit (for example, 1,000 tokens/minute) and a hard cost limit (for example, $2/day) per alias. Exceed either, and requests get throttled at the gateway -- before they hit your bill.
3. Configure Authentication and API Keys
Set up Entra role assignment, generate API keys, and configure the system-assigned managed identity. One gotcha: you may hit an Entra directory-permission error during this step that only a tenant admin can resolve -- don't assume it's something you can self-serve past.
4. Monitor Token Consumption and View API Logs
The gateway's Monitoring tabs -- Overview, Models, Tools, Policies, Traces -- give you visibility into what's actually consuming tokens and cost. Note: these stay empty until you connect Application Insights.
5. Configure Model Routing and Failover
Add a second backend model, set priority/weight, configure retries and an exclusion window for failed backends. This is the multi-model resilience piece, separate from the cost controls above.
The gap worth knowing about
Azure AI Gateway handles cost and traffic control well, but it isn't a content-safety layer. There's no native prompt filtering or PII redaction built in. If that's part of your risk surface, pair it with Azure AI Content Safety.
Read the full walkthrough with screenshots of every configuration screen on rizdynamics.com.

