AI FINOPS · LAYERS 2 & 3
Control AI spend before the cloud bill arrives.
AI cost is decided per call — so it has to be measured and controlled per call. Kosmoy attributes every request to its model, app, team and agent, warns before budgets break, and stops spend at the gateway when they do.
An AI FinOps platform measures, allocates and controls what an enterprise spends on AI. It answers three questions the provider invoice cannot: who spent it, on which use case, and was it worth it — and it gives someone the authority to say stop. Kosmoy does this in the request path: every LLM, MCP and agent call crosses the gateway, so cost is attributed, budgeted and — past the cap — blocked in real time.
This page covers the category. For the per-call telemetry itself — dashboards, forecasts, export — see the Cost Tracking module. For how vendors compare, see the AI FinOps buyer’s guide.
Why AI cost breaks cloud cost habits.
Cloud FinOps assumes capacity you provision. AI spend is consumption a model — or an agent — decides at runtime.
Tokens, not instances
Spend scales with every request — input tokens, output tokens, retries. There is no reserved-instance ceiling; a busy afternoon is a bigger bill.
Model choice is a 30× lever
The same task costs cents on a small model and tens of cents on a frontier one. Whoever picks the model sets the unit cost — often without knowing it.
Context length compounds
RAG pipelines and long conversations resend context on every turn. A prompt that grows quietly doubles the cost of every call it rides on.
Retries multiply silently
Timeouts, malformed outputs and validation failures trigger retries. The user sees one answer; the bill records three attempts.
Tool loops have no natural end
An agent calling tools decides for itself how many steps a task takes. A planning loop that never converges spends until something stops it.
Nobody owns the invoice line
Provider bills arrive as one number per API key. Without attribution, no team owns the overrun — so nobody fixes it.
Two kinds of AI FinOps. You likely need both.
Invoice-side FinOps products read bills after consumption and allocate them — essential for the estate view. Request-path AI FinOps measures and controls each call while it happens. Kosmoy’s claim is the second: the meter and the switch sit on the same wire.
| Question | Invoice-side FinOps | Request-path AI FinOps |
|---|---|---|
| When does it see spend? | After the bill arrives | As each call happens |
| Finest attribution | Account, tag, service | Call: model, app, team, user, agent, tool |
| Can it stop an overrun? | No — it reports one | Yes — hard caps block at the gateway |
| Can it lower unit cost? | Negotiations, commitments | Model routing and prompt-level levers |
| Sees failed and retried calls? | No — invoices aggregate them | Yes — every attempt is a ledger row |
| Best at | Estate-wide allocation and unit economics | AI workload control and containment |
If your cloud team already runs an allocation platform, keep it — it owns the invoice. Add request-path control where the spend is decided: on the AI calls themselves.
The control loop: measure to evaluate, then again.
Cost control is not a dashboard you check — it is a loop you run. Blue stages watch; orange stages act at the gateway.
- 01MeasureTokens, cost, latency and outcome captured on every request.
- 02AllocateSpend attributed to model, app, team, user and agent.
- 03AlertBudget thresholds warn owners while there is time to act.
- 04CapThe gateway blocks calls past the budget — not a report, a stop.
- 05RouteWork that fits a smaller model is routed to one.
- 06EvaluateEvals confirm the cheaper route holds quality — then the loop repeats.
loops back to 01 — every pass tightens attribution, budgets and routing
What request-path control covers.
Attribution on every call
Provider, model, application, team, user or service identity, agent and tool — each request lands in the ledger with its full context, including failed and retried calls.
Budgets with hard caps
Budgets per app, team or use case warn owners as the limit nears — and the gateway stops calls past the cap. A cap is enforcement, not a report.
Model routing
The LLM Router sends work that fits a smaller model to one, and falls back when a provider degrades. Routing is where cost policy becomes savings.
Runaway-agent containment
An agent stuck in a retry loop burns budget until its cap stops it. Agents running in an Action Capsule can also be stopped mid-run with the kill switch.
Chargeback and showback
Per-team and per-tenant spend rolls up for finance — export the ledger, or show each business unit its own consumption against budget.
Cost with quality context
Spend sits next to evaluation scores and user feedback, so a cheaper route is judged on both sides: what it saves and what it changes.
Who runs AI FinOps.
The CIO or Chief AI Officer owns the mandate; the AI platform team operates the gateway and routing policy; FinOps and finance consume attribution, budgets and chargeback. One ledger serves all three — the same per-call records that power engineering dashboards roll up into the numbers finance closes the month on.
In your infrastructure.
Request-path cost data embeds prompts, identities and usage patterns. Kosmoy runs single-tenant in your own Kubernetes — private cloud, on-premises or air-gapped — so the meter, the switch and the data they produce stay inside your boundary.
Module questions, answered straight.
What is AI FinOps?
AI FinOps is the practice of measuring, allocating and controlling what an organization spends on AI — LLM calls, agents and the infrastructure around them. It extends cloud FinOps with AI-specific mechanics: token-based pricing, model choice, retries, context growth and autonomous agents that decide their own consumption. A working AI FinOps capability answers who spent what, on which model, for which use case — and can stop spend it did not authorize.
How is AI FinOps different from cloud FinOps?
Cloud FinOps reads bills after consumption: it ingests invoices, allocates spend and reports unit economics. AI FinOps adds a request-path dimension — because AI spend is decided per call, it can be measured and controlled per call: budgets, hard caps, rate limits and model routing applied while the request happens. Most enterprises need both: invoice analytics for the estate, request-path control for AI workloads.
How can an enterprise track LLM cost by team or use case?
Route calls through a gateway that stamps every request with its context. Kosmoy's gateway attributes each call to provider, model, application, team, user and agent, so cost rolls up by any of those dimensions — including failed and retried calls, which invoice-level data cannot separate.
How do model routers reduce LLM cost?
A router classifies each request and sends it to the cheapest model that meets the quality bar — smaller models for extraction and classification, frontier models where reasoning depth earns its price. Because price differences between tiers are large, routing even part of the traffic down a tier compounds into real savings. Evaluations confirm the cheaper route holds quality before it becomes policy.
Can an AI FinOps platform stop an agent that is looping?
Only if it sits in the request path. A budget cap at the gateway stops the looping agent's calls once its allowance is spent — regardless of what the agent intended. For agents running in a Kosmoy Action Capsule there is a second, stronger lever: Mission Control's kill switch ends the run itself, not just its model calls.
What is the difference between AI cost tracking and AI cost governance?
Tracking is visibility: per-call spend, dashboards, forecasts, chargeback. Governance adds authority: budgets someone approved, caps the gateway enforces, routing policy that encodes the price/quality trade-off. Kosmoy's Cost Tracking module provides the telemetry; the gateway turns it into control.
Can AI FinOps run on-premises or in a private cloud?
Kosmoy runs in your own Kubernetes — private cloud, on-premises or air-gapped — so cost data, prompts and the policy plane never transit a vendor's cloud. That matters for AI FinOps specifically: request-path cost data embeds prompts and identities, which many regulated enterprises cannot send to a multi-tenant SaaS.
Review your AI cost-control architecture.
Bring your current bill and your worst overrun. We will map where a meter and a switch on every call would have caught it.
Or email sales@kosmoy.com.