Cost per Token Is Not Cost per Outcome: A Completed-Workflow Cost Model for Enterprise AI

Compare API, self-hosted, and hybrid options in currency per accepted completed workflow at the same quality threshold, demand pattern, and response-time objective. Include retries, exceptions, human review, operating labor, idle capacity, and failure handling. An API can win despite a higher token price when demand is uncertain and operations are expensive. Self-hosting can become credible when a suitable model passes the task, utilization is stable, and ownership costs are measured. Hybrid earns its place only when routing different cases changes the completed-workflow economics enough to justify added complexity.

Cost per token remains an input. It is not the decision unit a business consumes.

Define the outcome before opening the ledger

Use one explicitly synthetic workload for the model: a document-assisted recommendation that retrieves approved context, drafts a structured output, validates required fields, and routes selected cases to a human reviewer. A completion counts only when the output passes the task acceptance rule and reaches the intended workflow state. A response that is fast, fluent, or cheap but rejected does not enter the denominator.

The FinOps Foundation describes unit economics as connecting cost to a unit of business value (FinOps Foundation, Unit Economics). That is the useful translation here. “Accepted completed workflow” is not automatically business value, but it is closer to an accountable operating unit than tokens. The organization can then test whether those completions create value rather than assuming they do.

Write the unit with five conditions: task, acceptance threshold, start state, end state, and time window. For example, the unit may begin when an authorized request enters and end when a validated recommendation reaches its review queue within the response-time objective. Changing any condition creates a different denominator.

The completed-workflow cost model

The core equation is:

Cost per accepted completed workflow = total period cost divided by accepted completed workflows in that period.

Total period cost is the sum of model inference, retrieval and tool services, allocated infrastructure, software and support, operating labor, human-review labor, exception labor, and expected failure or recovery cost. Use one currency and one period, such as currency per month. Record whether taxes, financing, and shared overhead are included.

The denominator needs equal care:

Accepted completed workflows = attempted workflows multiplied by the measured completion-and-acceptance rate.

Retries affect both sides. They add inference, tool, latency, and sometimes labor cost. They may also convert an initial failure into an accepted completion. Record attempts per accepted completion rather than treating a retry as free.

Build the ledger from these variables:

• Demand in attempted workflows per hour, day, and month, including peak concurrency.
• Input and output tokens per attempt, separated by accepted, rejected, retried, and exceptional cases.
• Acceptance rate in accepted workflows per attempted workflow under a versioned evaluation.
• Retry rate in retries per attempted workflow and the reason for each retry class.
• Human review in minutes per workflow, reviewer coverage, and loaded labor cost in currency per hour.
• Exceptions in cases per hundred attempts, handling minutes per exception, and escalation cost.
• End-to-end latency in seconds, queue time in seconds, and the consequence of missing the objective.
• Cache hit rate in eligible requests per hundred requests and the share of input tokens covered by a reusable prefix.
• Capacity in accepted completions per hour at the tested quality and latency threshold.
• Idle capacity as unused provisioned hours per period.
• Operations in engineering and support hours per month, including updates, incidents, evaluation, and capacity work.
• Recovery in events per period, minutes per event, and any separately modeled consequence.

Each input carries four labels: observed, sourced, assumed, or unknown; then owner, source date, and sensitivity range. This claim ledger prevents a borrowed benchmark or a vendor list price from masquerading as local evidence.

Model each architecture without changing the workload

For an API, inference cost is calculated from dated provider prices and measured input and output tokens, with cached-token treatment recorded where applicable. Add retrieval, tools, network or platform charges, support tier, evaluation, integration, monitoring, retries, and human work. Do not assume that the provider operates the workflow around the model.

For self-hosting, allocated infrastructure includes the period cost of owned or leased compute, memory, storage, networking, backup, and facilities as applicable. Add energy in kilowatt-hours multiplied by currency per kilowatt-hour, software and support, model-serving work, patching, observability, evaluation, incident coverage, and capacity management. Divide provisioned cost by accepted completions, not by theoretical maximum tokens.

For hybrid, calculate each routed path separately. Record the routing rule, percentage of attempted workflows on each path, handoff cost, duplicate context, fallback behavior, evaluation differences, and cross-path failure rate. Then combine the path costs using the actual or assumed routing mix. A hybrid average without the routing distribution cannot support a decision.

IMPAKT operates and benchmarks a private, OpenAI-compatible 27-billion-parameter language-model environment with serving extensions, caches, and agent-tool integrations. That gives IMPAKT a direct view of cost categories and measurement dependencies, not an enterprise total-cost result. IMPAKT has not established savings, payback, production utilization, or a comparison winner.

The economic ledger assumes the options have already survived the hard gates in the workload-placement matrix. A cheaper route that violates the workload’s data, capability, control-evidence, or recovery requirement is not an economic winner; it is outside the eligible comparison set.

Test the variables that can reverse the answer

Start with acceptance. If a cheaper model requires more retries or review to reach the same outcome, its token advantage may disappear. Keep the evaluation set and threshold fixed while comparing options. Report severe failures separately because an average acceptance rate may hide a costly class.

Then vary demand and utilization. Model an API from its actual metered charges, commitments, and minimums. Model dedicated capacity as period cost, including idle hours. This comparison defines the economic mechanism without predicting which option is cheaper. The crossover depends on the actual capacity, demand distribution, financing, operations, and fallback needs.

Vary human review and exceptions next. A change of several reviewer minutes per workflow can matter more than a small model-price difference. Measure whether review is universal, sampled, triggered by confidence or rule, or required only for consequential actions. Include reviewer wait time when it affects cycle time or adoption.

Test cache assumptions using the real repetition pattern. In its version 0.5.3.post1 documentation, vLLM explains that key-value cache blocks for a shared prompt prefix can be reused without recomputing that prefix (vLLM prefix-caching documentation). A cost model should distinguish eligible shared-prefix work from variable input and separately measured generation. It should not apply a generic cache discount to all tokens.

Finally, vary operating labor, model change frequency, peak capacity, fallback use, and adoption. Low adoption shrinks the denominator while much of the implementation cost remains. Rapid model or provider change can increase evaluation and migration work. A tight latency objective may require spare capacity that lowers nominal utilization.

Use three scenarios rather than one forecast: a low-demand case, an expected case, and a peak or exception-heavy case. These are assumption sets, not predicted outcomes. The most useful result is often the variable that reverses the recommendation.

Decision rule

Choose the option with the lowest defensible cost per accepted completed workflow only after all candidates meet the same task acceptance, response-time, control, and recovery requirements. Prefer the API when demand uncertainty, capability needs, or operating labor dominate. Consider self-hosting when the model passes, utilization is stable, control requirements are material, and named operators can sustain the stack. Choose hybrid when measurable path differences outweigh routing, integration, evaluation, and recovery overhead.

Require a completed-workflow ledger with every material input labeled observed, sourced, assumed, or unknown. Reject a comparison that changes the workload, quality threshold, review policy, or time window between options. Recalculate when provider prices, hardware allocation, demand shape, acceptance, routing, or operating ownership changes.

The next useful test is to find the three variables with the widest plausible range and run the architecture choice at both ends. If the winner changes, the decision is conditional and the missing measurement is now visible.

What this does not prove

This model does not claim savings, return on investment, payback, labor reduction, or a client outcome. It contains no current provider prices and no unpublished private-lab results. Its outputs are assumption-sensitive and only as credible as the workload definition, evaluation, cost allocation, and demand evidence.

Cost per accepted completion is also not business value. A workflow can complete correctly and still be unused, unnecessary, delayed, or disconnected from an outcome. Before investment, test whether the completion changes cycle time, quality, risk, capacity, or another accountable business variable. Preserve that as a separate hypothesis rather than embedding optimism in the cost model.

Editorial process

This article was developed from IMPAKT's editorial direction with AI-assisted drafting and independent editorial review. Primary sources are linked beside the claims they support and listed below; direct observations, synthesis, recommendations, hypothetical examples, and material boundaries are identified where they appear.

Sources

• FinOps Foundation, Unit Economics — accessed August 27, 2026.
• vLLM automatic prefix-caching documentation — version 0.5.3.post1; accessed August 27, 2026.