AI Unit Economics: Inference Cost, Pricing and Gross Margin

Why AI unit economics deserve their own operating discipline

In classic software, gross margin is an outcome. In AI, it is a design choice made hundreds of times across the architecture, pricing model and support process. Two companies with identical revenue and identical products can differ by thirty margin points depending on how they route requests, how they price usage and how much human review the product requires.

That gap is the difference between a business that compounds and one that raises capital to subsidise its own customers.

The cost stack, itemised

Model your cost of revenue in five layers rather than one line:

1. Inference compute — the model calls themselves, split by model tier and by task.
2. Retrieval and data infrastructure — embeddings, vector storage, refresh jobs, document processing.
3. Evaluation and monitoring — regression runs, guardrail checks, and the observability required to keep quality claims honest. Small in absolute terms, but it grows with customer count.
4. Human-in-the-loop — review, exception handling, annotation. Frequently the largest hidden cost and the one most often booked outside cost of revenue, which flatters margin misleadingly.
5. Deployment overhead — per-tenant isolation, private deployments, bespoke integrations and the support they generate.

Book all five inside gross margin. Companies that classify review teams as operating expense report software economics they do not have, and are corrected painfully during diligence.

The levers that move inference cost, in order of return

Routing. Send each request to the cheapest model that clears the quality bar for that task. Production traffic is rarely uniform in difficulty, so a tiered cascade — small model first, escalate on low confidence — usually delivers the largest single cost reduction available.

Caching. Exact and semantic caching on repeated queries, plus reuse of stable context. In products with shared corpora, cache hit rates are frequently high enough to change the economics on their own.

Context discipline. Long prompts are a recurring tax paid on every call. Tightening retrieval to fewer, better passages usually improves both cost and quality.

Distillation. Once a task is stable and you have production traces, train a smaller model for it. This trades one-off engineering for permanent per-call savings and lower latency.

Output control. Constrain generation length and format. Verbose outputs cost money twice — once in tokens and once in review time.

Batching and asynchrony. Anything that does not need to be interactive should not be priced as if it were.

Commercial terms. Committed spend, provider diversification and the ability to move workloads are negotiation leverage that only exists if the architecture allows switching.

Pricing architecture as a margin instrument

Pricing is the other half of unit economics, and it should be designed against the cost stack, not against competitor screenshots.

• Anchor on the customer's baseline cost per unit of work, then capture a defensible share. Value-based pricing in AI is unusually tractable because the manual baseline is measurable.
• Use a platform fee plus committed usage to give finance predictability while keeping consumption aligned to cost.
• Set overage rates above marginal cost with headroom — heavy users are the accounts most likely to erode margin quietly.
• Meter the expensive path. If premium reasoning, long-context processing or bulk backfills cost materially more, they should sit in a different price tier rather than being averaged across all customers.
• Give sales a margin floor, not just a discount ceiling. Discount authority without cost visibility is how negative-margin logos get signed.

The metrics an AI board pack should contain

• Gross margin, reported both with and without inference cost.
• Cost per successful task — the true unit metric, superior to cost per call because failed and retried calls hide there.
• Cost trend per task over time; a flat line means no engineering leverage is being created.
• Human review minutes per output unit, by segment.
• Net revenue retention split into seat expansion and usage expansion — usage expansion at healthy margin is the strongest signal in an AI business.
• Concentration: revenue and gross profit by top ten accounts, and by model provider.
• Payback period including implementation cost, which in enterprise AI is often understated by a factor of two.

A practical improvement programme

Quarter one: instrument cost per task per customer and publish it internally. Quarter two: implement routing and caching, and set a target reduction with a quality guardrail. Quarter three: re-price new business against the improved cost base and migrate the worst-margin cohort at renewal. Quarter four: distil the two highest-volume tasks and reduce review workload through better exception design. A disciplined version of this sequence commonly moves gross margin from the forties into the sixties without changing the product proposition.

The takeaway

Treat unit economics as a product roadmap item with an owner, a target and a quarterly review. Route, cache, constrain and distil on the cost side; anchor to baseline value and meter the expensive paths on the price side; and report cost per successful task to the board. AI businesses do not grow into good margins — they engineer them.

Rebuilding the pricing and unit economics of an AI business? Book a free consultation.

---

Kamakshi Wason is Executive Director of TF Global Advisory Partners, which advises enterprise clients on strategy, delivery, marketing and revenue enablement across 500+ international projects and stakeholders from more than 50 countries.