Why One-Step Accuracy Does Not Compose Into Reliable AI Workflows

One-step accuracy does not tell a CTO whether a long artificial intelligence (AI) workflow will finish acceptably. A synthetic illustration makes the exposure visible: if 100 necessary steps were independent and each succeeded with probability 0.99, all 100 would succeed with probability about 0.366. Real systems violate those assumptions in both directions. Checkpoints and repair can improve completion; shared state and correlated failures can make it worse. The decision metric is repeated end-to-end accepted completion with intervention, recovery, cost, and severe events visible.

Treat the arithmetic as a warning

For a toy workflow with n necessary steps, assume every step succeeds independently with the same probability q. The probability that all steps succeed is q^n. At q = 0.99, one step succeeds with probability 0.99, while 100 steps succeed together with probability approximately 0.366.

This is authored arithmetic rather than observed agent reliability. Real steps are rarely independent or equally difficult. The illustration has a narrow purpose: repeated necessary steps can expose small component error, so a single-step score needs a system model before it becomes confidence.

Use q^n to expose assumptions and missing evidence, then replace it with observed system results before a deployment decision.

Real systems can perform better than the toy model

A failed intermediate step can become an accepted recovery. Deterministic validators can reject malformed output. Retrieval can be retried against another source. A human can resolve an ambiguity. A checkpoint can preserve valid work. A compensating action can reverse an external effect.

These mechanisms change the state machine. They can convert an attempted failure into an accepted recovery when their own coverage, error rate, time, and cost are measured. The system may also decompose work so that some steps are optional or parallel rather than jointly necessary.

Architecture, verification, and recovery therefore belong inside the reliability measurement for long workflows.

Real systems can also perform worse

Independence is often optimistic. One poisoned instruction can influence many later actions. An incorrect retrieved fact can be copied through a report. Shared hidden state can corrupt several branches. A permission error can expose every tool call. One unavailable dependency can block the complete workflow.

Correlation matters because retries may reproduce the same failure. Asking the same model to reflect can restate the same rationale. A second sample remains dependent when it shares evidence, prompt, evaluator, or tool path.

The tau-bench paper evaluates tool-agent-user interaction in specified domains and highlights the challenge of reliable policy-constrained interaction across multiple turns. OSWorld 2.0 is another long-workflow evaluation record with its own environment and method. Neither benchmark transfers directly to an enterprise workflow, but both support measuring complete trajectories rather than assuming that isolated capability composes.

Define the completion contract first

A reliability test begins with a stable workload contract:

• start state and accepted end state;
• necessary, optional, and prohibited steps;
• ordinary, difficult, edge, invalid, and severe cases;
• identity, resources, operations, and permission limits;
• latency, deadline, cost, and retry budgets;
• human intervention and recourse rules;
• safe stop, rollback, compensation, and recovery objectives.

The accepted end state must include more than output quality. Reject a correct answer produced after an unauthorized read, a valid transaction duplicated during retry, and a task completed without its required evidence trail.

Instrument the whole path

Assign a shared run ID across model interactions, retrieval, tool proposals, tool results, evaluations, approvals, external effects, and recovery. Record the exact configuration: model, prompt, data sources, tools, policies, evaluator, runtime, and owners.

Measure at least:

• attempted and accepted completed workflows;
• severe failures as a separate hard gate;
• per-stage and end-to-end latency;
• tool denials, invalid arguments, and duplicate effects;
• retries, loops, and no-progress terminations;
• human interventions and review minutes;
• state corruption, checkpoint restore, and recovery success;
• cost per accepted completed workflow.

Segment results by case class and workflow length. An average over short easy work can hide a failure concentration in long consequential paths.

Put checkpoints at consequential state changes

A checkpoint is valuable when it protects a meaningful boundary. Place one before an external side effect, a privilege change, an irreversible transition, a high-impact human decision, or a stage whose failure would invalidate expensive downstream work.

The checkpoint can use schema validation, business invariants, authoritative evidence, sandbox execution, an independently calibrated evaluator, or accountable human review. Match the verifier to the failure. Schema checks cover form, while nuanced domain correctness needs an eligible verifier. Human approval becomes enforceable through identity, context, denial, timeout, and recourse.

The OWASP excessive-agency guidance recommends limiting functionality, permissions, and autonomy and adding downstream authorization or user approval for high-impact actions. Its mitigations are design directions, not a sufficiency certificate.

Compare repair with simplification

When a long path fails, adding another reasoning loop is only one option. A deterministic step may replace a probabilistic one. A fixed chain may replace an orchestrator after the path becomes known. A human may retain a rare judgment. A workflow can narrow its scope or stop earlier.

Compare the revised system against the simplest eligible baseline on the same cases and acceptance contract. Count the additional latency, cost, exception burden, and recovery work. A repair is useful only if accepted completion improves enough to justify its operating surface.

Synthetic worked workload — approved invoice posting. Workload AP-POST-01 reads an approved invoice, matches a purchase order, proposes a ledger entry, obtains authenticated approval, and posts one idempotent transaction. External state change requires system authority beyond component accuracy. Place checkpoints after document match, before approval, and before the external write; preserve the valid match so a posting timeout resumes from that state. Measure accepted postings, duplicate effects, interventions, recovery minutes, and cost per accepted posting. Then compare repair with simplification: deterministic matching may replace a model step, and a person may retain the rare exception. Promote only the exact posting field and population that passed. The readiness gate assigns owners to unresolved failures, while the evaluation-oracle article defines regression and rollback evidence. The example remains synthetic; its scope excludes production and payment authority.

Promote only the tested boundary

Evidence authorizes only its tested tool, resource, population, and operating condition. Repeat the complete test after a change to model, prompt, retrieval, tool, permission, evaluator, state representation, retry policy, or recovery mechanism.

Promotion should name the exact authority and its stop conditions. A severe failure, recovery miss, repeated override, missing trace, or material distribution shift returns the system to a lower boundary until evidence is renewed.

Decision rule

Measure repeated end-to-end accepted completion and design checkpoints around consequential state changes. Use q^n only to expose composition risk; replace it with observed system evidence before granting authority or funding scale.

What this does not prove

The synthetic equation assumes equal independent step probabilities and is not an empirical agent forecast. The cited benchmarks use their own tasks, environments, systems, and dates. This article does not establish a universal reliability rate, safe step count, production threshold, or benefit from any particular verifier, retry, or human-review design.

Editorial process

This article was extracted from the IMPAKT LLM Operating Playbook with AI-assisted structure, drafting, editing, and metadata preparation. It underwent an independent critique and substantive revision loop against IMPAKT's publication rubric; primary sources are linked beside supported claims, and synthesis, recommendations, and evidence boundaries remain explicit.

Sources

• Yao et al., tau-bench, submitted June 17, 2024; accessed August 29, 2026.
• Yuan et al., OSWorld 2.0, submitted June 28, 2026 and revised July 13, 2026; accessed August 29, 2026.
• OWASP, LLM06: Excessive Agency, project page; accessed August 29, 2026.