Kosmoy vs LiteLLM: Proxy vs AI Management Platform (2026)

Each spoke is one capability, scored 0–10; the further a point sits from the centre, the stronger the product. LiteLLM's shape is a sharp spike: Gateway & Policy Control (9 vs 8) and Observability & FinOps (8 vs 7) both go to LiteLLM, and its self-hosted, air-gap-capable footprint earns a 9 on sovereignty — one point behind Kosmoy only because Kosmoy has no vendor-licensed tier at all. Kosmoy's shape is the wider web: inventory, security, guardrails, containment and compliance are where the gap runs the other way.

Cost and adoption friction. The core is MIT-licensed and free; the enterprise tier is a license key with a free 7-day trial, SSO free up to five users, and AWS/Azure Marketplace procurement (enterprise docs). Kosmoy has no self-service tier; procurement runs through an enterprise sales process.

Community and velocity. ~53.6k GitHub stars, ~9.8k forks, weekly stable releases, a public registry of custom guardrails and an ecosystem security working group — no proprietary vendor matches that surface area of contributors and integrations. The Rust migration announced June 25, 2026 targets vendor-reported sub-1ms gateway overhead, staged through December 2026 (announcement).

FinOps granularity. Budgets and spend attribution per organization, team, project, key and tag, soft-budget alerts, programmatic spend reports, Prometheus and OpenTelemetry metrics, and tool-call tracing since Logs v2 in January 2026 (release notes). Kosmoy tracks cost, usage and agent traces per model, app and user — solid FinOps, a less granular attribution hierarchy.

MCP gateway maturity. An MCP server registry with per-server access groups, OAuth 2.0 including On-Behalf-Of (production v2 shipped July 2026), and controls over which MCP servers are exposed to the public internet — the most mature MCP story among open-source gateways (MCP docs).

Inventory beyond the proxy. LiteLLM's model catalog, MCP registry and Agent Hub cover what is routed through LiteLLM. Kosmoy's four registries also cover what is not: connectors pull agents from Azure AI Foundry, Bedrock, Vertex, Salesforce and ServiceNow into one master list, each entry with an owner and a risk tier. If the question is “what AI do we run?”, only one of these products tries to answer it.

Compliance evidence. LiteLLM's enterprise audit logs cover gateway admin actions, with durable log export for compliance storage — useful inputs, not a compliance product. It does not document EU AI Act, ISO/IEC 42001 or NIST AI RMF mapping, risk classification or evidence packs as of July 15, 2026. Kosmoy produces framework-ready evidence bundles from registry state plus gateway logs: one source, every audit.

Agent containment. LiteLLM offers scoped virtual keys, per-agent rate limits and key revocation — coarse, policy-level controls — and its per-session-VM coding-agent runtime (May 2026) is an early project, not a containment product. Kosmoy's Action Capsule is architectural: each agent, MCP server or private model runs in a kernel-enforced sandbox whose only egress is its paired gateway, with per-task credentials and a kill switch.

A platform, not a kit. LiteLLM's guardrails framework is genuinely capable — built-in checks, realtime guardrails, policy templates — but it is an enforcement point your team assembles and operates, along with the Postgres, Redis, upgrades and on-call that the enterprise license explicitly leaves with the customer. Kosmoy ships inventory, gateway, guardrails, evidence and containment as one supported product. Which model is better is a staffing question as much as a technical one — see build vs buy.

Both products expose an OpenAI-compatible endpoint, so applications move between them by changing a base URL — migration in either direction is configuration, not a rewrite. In practice many Kosmoy prospects arrive already running LiteLLM, and the sensible pattern is rarely rip-and-replace: LiteLLM stays as the developer-facing proxy for experimentation, while Kosmoy becomes the governed path to production — the inventory, risk tiers, compliance evidence and containment around whatever routes the traffic. LiteLLM's Rust migration, if it lands on schedule in December 2026, only strengthens the case for keeping it as a fast data path inside a governed perimeter.