AI Gateway
An intelligent control plane for your AI applications
Connect to any model, dynamically route requests, and manage usage, billing, and logs from one unified gateway.
Reduce Costs & Latency
Improve Reliability with Dynamic Controls
Add Observability
Dynamic Routing
Automatically route requests based on latency, cost, or availability. Adjust rules instantly from the dashboard or API — no redeploys, no downtime.
Core Capabilities
Global Network Performance
Built on Cloudflare's infrastructure. Ensures low-latency, globally distributed access with automatic scalability and built-in security.
Caching
Reduces redundant API calls. Saves money and improves response time by storing and reusing frequent requests automatically.
Built-in Observability
Logs, metrics, and usage analytics. Includes fallback routing, rate limiting, and safety guardrails to manage cost, behavior, and compliance across multiple providers.
Security controls and guardrails
Protect your AI applications from leaking or sending sensitive information. Protects your AI app from malicious traffic without needing to configure or maintain anything extra.
Unified Billing
Manage all your costs with one simple bill and access every provider through a single API. Spend less time managing and more time shipping.
Reducing latency and cost of AI apps by caching API responses
Usage analytics — monitoring prompt performance, token counts, and behavior
Building custom dashboards and alerting systems directly from AI Gateway logs
Control your AI infrastructure
Examples showing how to configure caching, routing, and monitoring for AI workloads.
// wrangler.jsonc // Simple configuration { "ai": { "binding": "AI" } } // Pass through the Gateway from your Worker with Workers AI // index.js const resp = await env.AI.run( "@cf/meta/llama-3.1-8b-instruct", { prompt: "tell me a joke", }, { gateway: { id: "my-gateway", }, }, ); // Use with OpenAI SDK import OpenAI from "openai"; const openai = new OpenAI({ apiKey: "my api key", // defaults to process.env["OPENAI_API_KEY"] baseURL: await env.AI.gateway("my-gateway").getUrl("openai"), });
Rightblogger
Without AI Gateway, it’s difficult to see which applications are driving the majority of the costs with the OpenAI API … We can choose to limit the number of requests used by certain tools to control costs.
Powerful primitives, seamlessly integrated
Built on systems powering 20% of the Internet, AI Gateway runs on the same infrastructure Cloudflare uses to build Cloudflare. Enterprise-grade reliability, security, and performance are standard.
Compute
Storage
AI
Media
Network
SASE / Zero Trust