LLM cost tracking

Why Your LLM Bill Is a Black Box (And How to Fix It)

The invoice from your LLM provider landed last month. $47,000. You knew it was coming because you watched the number climb all month in your provider dashboard. What you don't know is where it went.

Was it your customer-facing chat feature? The internal doc summarizer your team loves? That experimental agent someone spun up last quarter that you forgot to shut down? The dashboard shows total tokens. It shows total cost. What it doesn't show is which of your products, teams, or customers is responsible for which slice.

This is the LLM cost attribution problem, and almost every team running more than one AI feature hits it.

The Gap Between Knowing Your Bill and Understanding It

Total spend visibility is table stakes. Your AWS cost explorer, your OpenAI dashboard, your Anthropic console -- they all show you what you spent. None of them answer the question that actually matters for a growing product: which feature or customer is driving cost, and is that cost justified by the value it creates?

This gets complicated fast when you are running at scale:

A fintech with 10+ models deployed across 3 products couldn't attribute costs to individual customers for billing purposes. They knew their margin was eroding but couldn't pinpoint which customer segment was underwater.
A B2B SaaS team had six different engineers calling the OpenAI API from six different services. Each tracked their own usage locally. Finance was reconciling by hand every month, and it still didn't add up.
A developer tools company realized after eight months that their "free tier" AI features were costing them $0.18 per active user per day. The pricing model was set assuming $0.04. The attribution lag nearly sank the product.

These aren't edge cases. This is what LLM cost management looks like at companies running real products.

Why Direct API Calls Make Attribution Hard

The root of the problem is architectural. When engineers call LLM provider APIs directly from application code, the cost signal lives in the provider's system, not yours. You get aggregate invoices. You get token counts at the account level. You do not get per-feature, per-team, or per-customer breakdowns by default.

Some teams solve this with logging -- add a wrapper, log the request and response, aggregate it yourself. This works until it doesn't:

Logging is added inconsistently across services
Response tokens are harder to log accurately than request tokens (streaming makes this worse)
Logs are useful for debugging but not structured for financial reporting
No single system owns the attribution model, so it diverges over time

The result is a patchwork of cost visibility that requires manual reconciliation and still leaves gaps.

Per-Request Attribution as a First-Class Concern

Real LLM cost tracking requires treating attribution as a first-class concern in your infrastructure, not an afterthought bolted onto logging. That means:

A single ingress point for all LLM traffic. Every request to any LLM provider routes through one layer that can capture, tag, and attribute it.

Structured tagging at the call site. Each request carries metadata -- which product, which customer, which feature, which team. This metadata is attached at the point of the API call, not reconstructed later from logs.

Real-time cost calculation per request. Don't wait for the monthly invoice. Know the cost of each request as it happens, calculated against current provider pricing.

Aggregation that matches your business model. You need to roll up costs by whatever dimensions your business cares about: customer, feature, team, environment.

What Changes When You Have Attribution

The value of per-request cost attribution isn't just knowing your bill. It changes how you operate.

Companies running 10+ models need to know which model to use for which task. When you can see that GPT-4o costs you $0.12 per customer interaction versus $0.02 for Haiku on lower-complexity tasks, you can make routing decisions based on data.

Product managers can set cost budgets per feature and get alerted before they are exceeded. Instead of discovering cost overruns on the monthly invoice, you catch them in real time.

For B2B SaaS, per-customer attribution enables actual cost-based pricing. You can look at your highest-usage customers and determine whether your pricing reflects the cost of serving them.

Finance can produce accurate cost of goods sold for AI features. This matters for board reporting, for pricing reviews, and for understanding product margin.

Fixing the Black Box

The LLM cost attribution problem is solvable, but not by adding more logging to your existing API calls. It requires a proxy layer that sits between your application and your LLM providers -- something that captures every request, attributes it, calculates cost in real time, and makes that data queryable.

That's exactly what we built at SteadIO. Every request that flows through SteadIO gets tagged, costed, and attributed to whatever dimensions matter for your business. You can query spend by customer, by feature, by model, by team, in real time. Budget enforcement lets you set limits per dimension and block or reroute requests that would exceed them.

The black box doesn't have to stay a black box.

Jon Hutchins is the founder of SteadIO, an LLM cost control plane for teams running AI at scale. Reach him at [email protected].

Ready to see your LLM costs with per-request attribution? Visit steadio.ai.