AI Gateways & LLM Routing — Market

Verified claims and product-axis read for AI Gateways & LLM Routing. Every fact below is sourced; every product judgment traces back to underlying signals.

Verified facts

• Portkey, LiteLLM, and Helicone all expose Prometheus-compatible metrics endpoints. ↗ (other)
• OpenRouter's terms allow models to be deprecated with 14-day notice to API consumers. ↗ (other)
• OpenRouter natively exposes provider-side privacy policies (training-data opt-out) per model. ↗ (other)
• Helicone introduced cost alerts via Slack/webhook in 2024. ↗ _(historical_event)_
• OpenRouter offers BYOK (bring your own key) at 0% markup, monetizing only on prepaid credits. ↗ (other)
• LiteLLM project maintainer count grew to 12+ regular committers in 2025. ↗ _(historical_event)_
• LiteLLM hosts router benchmarks showing <5ms p99 added latency on small-payload requests. ↗ _(technical_spec)_
• Portkey integrates with AWS Bedrock, Azure OpenAI, and GCP Vertex out of the box. ↗ (other)
• Portkey claims 99.99% uptime SLA on its Enterprise tier with regional failover. ↗ (other)
• LiteLLM Proxy publishes weekly Docker pulls exceeding 500K on Docker Hub. ↗ (other)

Top products (engine read)

OpenAI-compatible Multi-Provider LLM Gateway — gateway / proxy

Opportunity: Devs want one drop-in OpenAI-compatible endpoint that abstracts provider sprawl, handles failover, and removes provider lock-in (fact 4f1adaf4: Portkey 250+ LLMs; fact 495572d5: LiteLLM Proxy 100+ providers).

This is the spine of the ai-gateways category. Portkey (7k+ stars per fact 6c959edf), LiteLLM, and Helicone are all hiring backend/platform engineers around it; community posts show indie 'AISBF' and 'AI Cost Firewall' clones validating product-market pull. The category is crowded — defensibility now shifts to ecosystem (caching, guardrails, observability) rather than the proxy itself.

LLM Guardrails / Agent Security Proxy — policy-enforcement gateway

Opportunity: Agent runtimes need an in-band judge to block prompt injection / data exfil / unsafe tool calls — Portkey ships 40+ pre-built deterministic & LLM-based checks (fact 6a2a6008-c864-48b8-b687-9d49125b6013); Brex open-sourced CrabTrap to fill the gap.

Strong tailwind: agentic workloads in prod are forcing gateways to grow a security plane. Helicone is hiring a 'Staff AI Agentic Security Engineer' and OpenRouter a 'Trust and Safety Lead' — every gateway is now also a policy gateway. Big-co (Brex) building in-house signals an enterprise willingness to pay.

Semantic Caching for LLM Gateway — cache layer

Opportunity: Cached requests are where gateways monetize latency + cost; Portkey advertises sub-1ms p50 added latency on cached calls (fact fd1ed84d-7563-4781-96e8-78ed2551fa44).

Semantic caching is moving from a Portkey feature to a standalone wedge — indie 'AI Cost Firewall' and AISBF lead with it as the headline. Hot, but commoditizing fast; whoever owns embeddings-aware cache + invalidation wins.

See the Products and Strategy modules for the full product list and forward-looking judgment.