Adequate and Cheap: The AI Threshold We Are Not Watching

{Written using AI tools.}

The AI threshold we are not watching

Every few months the argument restarts. Is this AGI yet? Does the new model actually reason, or is it a very expensive autocomplete? Would it pass a real Turing test, a bar exam, a PhD qualifying exam? The debate is entertaining, occasionally illuminating, and it is tracking the wrong variable.

The consequential moment in AI may not be when a model crosses some philosophical threshold called intelligence. It may be when useful machine cognition becomes cheap and replicable enough that automation, parallelism, verification, and repeated attempts overwhelm much of its remaining unreliability.

Or, sharper: the model does not have to stop making mistakes. Mistakes only have to become cheaper to detect, route around, or retry than to prevent.

The steam-engine question

The economically consequential question about the steam engine was never whether it was stronger than a human in some universal sense. It obviously was in certain configurations and obviously was not in others; a steam engine could not climb a ladder or carry a basket across a field. The question that reorganized production was narrower and more commercial: can this machine perform enough useful work, cheaply enough and consistently enough, that it makes sense to rebuild the process around it?

Notice what that question does not contain. It does not ask whether the engine is impressive. It does not ask whether the engine understands what it is doing. It does not set the bar at matching the best human at their best task. It asks about cost, adequacy, and consistency, and it accepts that the answer arrives one factory and one workflow at a time rather than at a single dramatic moment.

We keep asking the impressive-engine questions about AI. When will it match human experts? Does it really understand? Is the frontier model conscious, creative, general? These are legitimate questions for philosophers and scientists. But they provide surprisingly little operational guidance to anyone trying to anticipate when AI changes the shape of work, security, and the economy. History suggests that production systems can reorganize long before the philosophical questions are settled.

The corresponding AI question is the factory owner’s question: when does a model become sufficiently competent, sufficiently inexpensive to run repeatedly, and sufficiently easy to integrate that it makes sense to rebuild the process around it?

The uncomfortable thing about that question is how much lower the bar may be than the AGI debate implies.

Three thresholds, and the one getting less attention

It helps to separate several ideas that are usually compressed into “is AI good enough yet?”

The first is the capability threshold: can a model do the task at all, under useful conditions? Many such thresholds have already been crossed. Models can produce working code, summarize complex documents, extract structured information, manipulate software through tools, and complete at least some multi-step tasks that would previously have required human effort.

The second is the economic threshold: can the system produce an acceptable result cheaply enough that deploying it beats the alternative? The price of reaching fixed levels of model capability has fallen extraordinarily quickly in recent years. Stanford’s 2025 AI Index, for example, reports that the inference price for a model reaching GPT-3.5-level performance on MMLU fell from $20 per million tokens in November 2022 to $0.07 by October 2024—a reduction of more than 280-fold in roughly eighteen months.^[1] The exact rate varies dramatically by benchmark and should not be extrapolated indefinitely, but the direction is unmistakable.

The third threshold is the one this essay is concerned with: the point at which useful machine work becomes cheap enough to repeat, parallelize, verify, and escalate so that system-level performance matters more than the reliability of any single attempt. Call it the replication threshold.

“Replication” here means replication of attempts or worker instances, not necessarily copying model weights. A company can run a thousand parallel API sessions against a hosted model without possessing the model itself. What matters to the economics is that another attempt is cheap to instantiate.

These thresholds are not perfectly sequential. In some domains, replication and verification are precisely what cause the economic threshold to be crossed. A single attempt may be too unreliable to justify deployment; ten cheap attempts feeding a reliable checker may produce an economically useful system.

There is also a separate question about copying the model itself. AI-risk literature often uses “replication” to mean an autonomous model copying itself onto new machines. That is not required here. The ordinary scenario is simpler: humans deliberately instantiate more machine workers because doing so is useful and inexpensive.

The distinction matters because the mundane version requires no rogue behavior. It can happen through normal procurement decisions, API calls, cloud jobs, local inference servers, and agent orchestration.

The deeper idea is old. In lectures delivered in 1952 and published in 1956, John von Neumann addressed a problem whose title could almost headline this essay: Probabilistic Logics and the Synthesis of Reliable Organisms from Unreliable Components.^[2] Reliability was a fundamental problem in early computing, and von Neumann asked explicitly how reliable systems could be synthesized from unreliable elements.

That is computing’s old trick. Storage systems replicate data or use erasure coding. Transport protocols retransmit lost data. Distributed schedulers replace failed workers. Reliability is often moved out of the component and into the architecture.

AI gives us a new unreliable component.

A model is often evaluated the way a job candidate would be: can this one entity be trusted to perform the task correctly? By that standard, models remain easy to dismiss. Any individual attempt might hallucinate, misunderstand the instructions, or write code that does not work.

A deployed system need not stake everything on one attempt.

The systems question is different: given a component that succeeds some fraction of the time, at some cost per attempt, with some method of checking or escalating its output, what does the compound system achieve?

That question produces very different answers from “is this model smart?”

The arithmetic of cheap attempts

Start with the naive math, because even the naive version rearranges intuitions.

Suppose a model completes some useful task correctly 40 percent of the time. As an employee, that is a firing offence. As a probabilistic component, it is a different story.

If five attempts were genuinely independent, the probability that all five failed would be:

0.6^5 ≈ 7.8%

With ten independent attempts:

0.6^10 ≈ 0.6%

So after ten attempts there would be about a 99.4 percent chance that at least one candidate is correct.

That is not yet a 99.4-percent-reliable system. The system still needs a way to recognize the correct candidate.

And the calculation hides two difficult assumptions.

The first is independence. Ten attempts from the same model are not ten independent draws. Models have systematic blind spots. If the model misunderstands the task, it may misunderstand it in much the same way repeatedly, even with randomized sampling.

Correlation can destroy much of the naive exponential benefit.

The engineering responses form a spectrum. Plain retry is the weakest. Varying prompts or decompositions may create more diversity. Different models may fail differently. Separate agents can pursue genuinely distinct strategies. None of these guarantees independence, but they can make repeated attempts more useful.

The second assumption is more important: successful outputs have to be recognizable.

Retry arithmetic is worthless when there is no economical way to tell whether attempt seven succeeded where attempts one through six failed. Cheap inference does not automatically produce cheap outcomes. The real cost includes generation, verification, orchestration, escalation, and residual human review.

This relationship between producing an answer and checking it is increasingly discussed as generation-verification asymmetry or verification asymmetry in recent AI research and economic writing.^[3] The terminology is not unique to this essay; what matters here is its predictive consequence.

Some domains have a favourable asymmetry: producing a candidate solution is difficult, while checking important properties of the candidate is comparatively cheap.

Code is an unusually favourable case. A compiler provides fast, inexpensive, machine-readable feedback about one class of failure. Tests cover another. Type systems, static analysis, linters, fuzzers, and sandbox execution provide additional signals.

To be precise about what this buys: automated verification checks specified properties, not truth. Code can compile and pass every available test while remaining wrong in a way nobody thought to test.

But partial, cheap verification is still enormous leverage. It lets a system reject many bad attempts automatically and reserve more expensive judgment for a smaller set of survivors.

The same structure appears elsewhere: theorem proving with machine-checkable proofs, optimization problems with constraint validators, database queries whose outputs can be tested, and engineering designs that can be exercised in simulators.

At the other extreme are domains where checking an answer requires roughly the same expertise as producing one: long-horizon strategy, diagnosis without good ground truth, forecasting, and many questions of taste and judgment. Ten unverifiable answers may not be much more useful than one.

There is another limit. More attempts can actually make an imperfect verifier perform worse. Research on Best-of-N inference has shown that, when candidate outputs are selected using an imperfect reward model, increasing the number of samples can eventually optimize for the verifier’s blind spots rather than for genuine quality.^[4] More search helps only when the checking mechanism remains trustworthy under search.

The practical lesson is therefore not “just retry.”

It is:

Cheap attempts become powerful when failures are sufficiently diverse, useful properties can be checked economically, and ambiguous cases can be escalated.

That helps explain why coding has become such a prominent agentic use case. On Anthropic’s platforms, for example, computer and mathematical tasks accounted for 35 percent of Claude.ai conversations in its February 2026 sample, while Claude Code represented a large share of sampled first-party API traffic.^[5] That does not establish that coding leads every AI application globally. It does illustrate how strongly a domain with abundant machine-readable feedback has attracted agentic deployment.

The practical map of AI’s advance may therefore follow the cost of checking work at least as closely as it follows the intelligence of models.

That is a very different map from the one the AGI debate draws.

The loop is the intelligence

There is a second multiplier stacked on top of retries, and it changes what we are measuring when we measure a model.

A raw model can answer a prompt in one shot, without feedback or a second chance. Many familiar model benchmarks isolate capabilities in something close to this form.

Consequential deployments increasingly do not.

A deployed agent can inspect its environment, plan, act, observe the result, and revise. Given a coding task, it need not produce a correct program in one pass. It writes, compiles, reads the error, edits, runs tests, reads the failure, and edits again.

The compiler has become part of the system’s cognition.

So has the test suite. So have the debugger, version control, search, the operating environment, and any other model or tool the agent can consult.

This makes “how smart is the model?” progressively less useful as a deployment question. The economically meaningful object is the loop:

model + tools + environment + verifiers + retry policy + escalation path

A mediocre reasoner inside a strong feedback loop can outperform a stronger reasoner that gets one unaided attempt.

Evaluation is beginning to reflect this shift. METR, for example, measures a frontier agent’s task-completion time horizon: the duration of tasks, measured by how long human experts take, at which an agent is predicted to succeed with a specified probability.^[6] This is explicitly not the literal amount of time an agent runs. It is an attempt to measure the difficulty and length of work the agentic system can complete reliably.

That is much closer to the quantity deployment actually cares about.

The same logic that engineering has long applied to unreliable physical and software components is beginning to apply to machine cognition. Reliability can migrate upward into the architecture.

The sparse-supervision threshold

The loop has one more layer, and economically it may be the most important one: the human at the top of the escalation path.

The cost that matters to a deployer is not the price of inference. It is the total cost of producing a verified, acceptable result:

• model inference;
• retries and parallel attempts;
• tool execution;
• orchestration;
• verification;
• escalation to stronger models;
• residual human review;
• and, where relevant, compliance, liability, and expected failure costs.

Model inference is one component whose price has been falling rapidly. Human expert attention is not following the same curve.

So a critical deployment question becomes:

How sparse can human attention become?

Consider an escalation ladder.

A cheap model handles the routine cases. A retry or second model resolves another share. A stronger model takes the genuinely difficult residue. A human expert sees only the exceptions.

If cheap machine tiers absorb 98 of every 100 cases, the system has not necessarily “replaced the expert.” It has made expert attention unnecessary in 98 percent of cases.

For the rhetoric of automation, that distinction is enormous.

For the economics, it may be much smaller.

Economist Luis Garicano has described a related supervision threshold: a boundary separating work AI can perform with limited oversight from work that still requires sufficiently expert humans to supervise it.^[7] The operational quantity emphasized here is slightly different. Call it the sparse-supervision threshold: the point at which human interventions per completed unit of work become rare enough that one expert can oversee a volume that previously required many people doing the underlying task.

This reframes the labour question.

The debate is often conducted as:

“Can AI do the entire job?”

That invites the response:

“No. Look at these failures.”

Often the failures are real. But the economic progression need not be binary:

human does the task → human works with AI → human supervises multiple AI processes → human handles exceptions

Each transition can be unremarkable by itself. The cumulative effect on how much labour a given volume of work requires can be substantial.

This suggests a useful operational metric:

Human interventions per completed task

A falling intervention rate can change the economics of a workflow even when benchmark scores move only modestly, because much of the improvement can live in better tooling, verification, routing, retries, and escalation rather than in the base model.

Specialists before generalists

There is another reason the relevant bar may be lower than the AGI debate assumes: generality is not required for economic consequence.

A model can be mediocre across a broad range of work and unusually capable in one narrow domain, whether because it has been specialized deliberately or because its training provided unusually rich coverage there.

Legacy-code migration. Vulnerability triage. Contract-clause extraction. SQL optimization. Tax-rule lookup. Specific laboratory workflows.

Each is narrow. Each can be economically meaningful. None requires a universally capable intelligence.

The plausible near future may therefore contain abundant machine expertise before it contains anything everyone agrees to call general machine intelligence: large populations of inexpensive, competent specialists embedded in workflows where their weaknesses are constrained by tools and verification.

That possibility matters because it makes the AGI threshold even less informative as a guide to near-term economic change.

Scale does not care about mediocrity

The same arithmetic runs in directions that are harder to celebrate.

Everything above describes a deployer filtering an unreliable component’s output for value: screening drug candidates, testing software, reviewing literature, migrating code.

The economically relevant comparison is often not one expert against one model instance.

It is one expert against a large volume of machine-work hours feeding a filtering and escalation pipeline.

The uncomfortable symmetry is that nothing in that architecture asks whether the task is benevolent.

For hostile uses, the economics can be favourable for a structural reason: defenders often require high reliability across a large surface, while attackers may require only occasional yield.

Fraud, credential attacks, vulnerability discovery, influence operations, and automated reconnaissance all have real costs beyond inference—access, infrastructure, detection, account reputation, defensive filtering, and operational risk among them. But reducing the cost of generating and adapting attempts still shifts the economics.

The relevant quantity for an attacker is often not reliability per attempt but successful outcomes per unit cost.

That is where the cheap-attempts argument stops being merely a productivity story and becomes a security story.

In the productivity frame, low reliability at scale is a systems-engineering problem.

In the adversarial frame, low reliability at scale may itself be the threat model: not one brilliant attempt, but an industrial volume of adequate ones.

Superintelligence is not required for that.

Adequate and cheap may be enough.

The transition may arrive below the frontier

Public attention tracks the frontier: the largest models, the newest releases, the benchmark records.

That is where new capability is often demonstrated.

It may not be where the largest social effects occur.

A useful distinction is:

Frontier systems discover capability; commodity systems diffuse it.

A capability may first be demonstrated at great cost in a datacenter. Through distillation, quantization, architectural improvements, inference optimization, and hardware progress, an approximation may later become inexpensive enough to deploy widely.

The first event establishes that something can be done.

The second determines who can afford to do it repeatedly.

Computing itself offers a useful, if imperfect, analogy. Early electronic computers such as ENIAC were historically consequential, but they were scarce institutional machines.^[8] Mass-market personal computers changed the economics and social reach of computation by moving substantial computing capability toward individual ownership.^[9] Smartphones expanded that diffusion again.

The analogy is not that frontier machines are unimportant.

It is that availability is itself a capability multiplier.

AI adds another distinction.

A hosted model provides a relatively conditional capability. Its provider can change the service, price it differently, rate-limit it, restrict uses, or discontinue access.

Model weights that can be run on hardware you control provide a more durable capability. They are not merely access to a service; they are an artifact that can be copied, preserved, modified, and used independently of the original provider, subject to the practical limits of hardware, software, licensing, dependencies, and surrounding infrastructure.

Open weights are not required for the replication economics described earlier. A thousand API agents can already provide parallel machine labour.

Open weights matter for a different reason: durability.

Once useful weights are widely mirrored, large-scale recall stops being a realistic governance assumption. The relevant question shifts from:

“Will this provider allow the capability?”

toward:

“Can someone, somewhere, make the capability work?”

Those are different governance problems.

None of this means open weights are a mistake. Open availability can generate substantial scientific, defensive, competitive, and economic value. Durable capability also reduces dependence on a small number of providers.

The point is narrower: diffusion changes the reversibility of capability, and reversibility matters.

What to watch instead

If this argument is right, AGI timelines and benchmark leaderboards are an incomplete dashboard.

The more useful dashboard is mostly operational and economic.

Variable	Why it matters
Cost per verified useful outcome	The outcome metric; the rest feed into it
Cost per attempted task	Determines how much search, retry, and parallelism are affordable
Cost and quality of verification	Determines whether repeated attempts can be filtered economically
Degree of correlated failure	Determines how much multiple attempts actually buy
Human interventions per completed task	Measures how sparse expert attention has become
Escalation rate to stronger models or humans	Shows where inexpensive machine tiers stop being sufficient
Agent task-completion horizon	Measures the difficulty and length of work an agentic system can complete at a stated reliability
Hardware and infrastructure required for adequate performance	Determines how widely the capability can be deployed
Economically deployable parallel instances	Converts individual capability into aggregate scale
Integration and compliance cost	Determines whether technical capability can survive contact with an organization

Cost per token still matters. It is an input cost, sometimes an important one.

But it is not the outcome metric.

A deployer ultimately pays for acceptable completed work.

That cost falls only when the whole loop improves: models, tools, verification, retries, routing, orchestration, and human escalation.

Two further frictions keep this framework honest.

First, economic viability is necessary but not sufficient. Deployment moves at the speed of institutions, regulation, security review, integration, data quality, procurement, organizational politics, and trust. A technically crossed threshold can take years to become an operational one.

Second, high-stakes domains carry costs beyond verification. Liability, compliance, insurance, accountability, and expected harm can keep humans in workflows even when important technical checks have become inexpensive.

With those caveats, the framework compresses into a practical instruction:

Do not rank tasks only by how much intelligence they appear to require. Rank them by whether adequate machine performance is cheap, attempts can scale, useful properties can be verified economically, failures can be routed around, and human expertise can become sparse.

Asked domain by domain, this becomes an empirical question that can actually be measured and compared.

“When is AGI?” offers much less operational guidance.

The transition may be mundane

People often imagine AI transformation as a recognizable event: a model becomes obviously intelligent, the argument about whether it “really understands” finally ends, and the consequences become impossible to ignore. But the changes described in this essay do not require such a moment.

They can emerge gradually from improvements that look unremarkable in isolation. Inference gets cheaper. Verification becomes more reliable. A smaller model becomes sufficient for a particular class of work. Fewer cases need to be escalated to a stronger model or a human expert. Another workflow crosses the point where automation becomes economically sensible. None of these developments needs to be dramatic, and there may be no single date on which the larger transition can reasonably be said to have occurred.

The models involved may remain visibly imperfect throughout. They may hallucinate citations, misunderstand instructions, produce buggy code, or fail in ways that seem embarrassingly simple. Those failures will provide legitimate evidence for anyone arguing that the systems remain unreliable or fall well short of general intelligence. But that criticism and widespread deployment can both be true at the same time.

Organizations deciding whether to use these systems do not need to settle the question of whether they are intelligent in any philosophically satisfying sense. They need to know whether a system can produce useful work at an acceptable cost, whether its failures can be detected or contained, and how much human supervision remains necessary. Once those conditions are met for a particular workflow, the fact that the underlying model still makes conspicuous mistakes may matter much less than it appears to from the outside.

---

A note on terminology and prior work

This essay is a synthesis, not a claim that each component idea is new.

Fault tolerance and reliable systems built from unreliable components have a long history in computing, notably including von Neumann’s 1950s work. Recent AI research has explored test-time compute, Best-of-N sampling, imperfect verifiers, generation-verification asymmetries, agent task horizons, and scalable oversight. Economists and other researchers are also examining how AI changes supervision, apprenticeship, and the organization of expert work.

The argument here is that these pieces point toward a useful practical threshold:

When inexpensive machine work can be repeated, checked, routed, and escalated so effectively that individual model unreliability is no longer the binding constraint, substantial automation can arrive before anything resembling universally accepted AGI.

That is the threshold worth watching.

---

Sources and Further Reading

---

Note: This article explores economic, architectural, and historical perspectives on AI thresholds and capability scaling.