Smart Companies Are 'Dumbing Down' Their AI

The most cutting-edge models are already shockingly smart. But much of enterprise work isn't actually that hard. Claims adjusters, nurses, logistics dispatchers — their job isn't to discover unknown answers. It's to get things right within existing rules and context.

So every workload has an "intelligence threshold": below the line, each incremental gain in model smarts is worth paying for; above it, the real bottlenecks become context and execution, and excess intelligence only adds cost, latency, and complexity.

Today's article, Right-Sizing Your Intelligence Spend, is a new piece by Foundation Capital partner Jaya Gupta, co-authored with two Stanford PhD researchers, Avanika Narayan and Jon Saad-Falcon. We previously translated her article on what she calls "the biggest moat in the AI era," which was about organization. This one is about money: how should enterprises actually spend on "intelligence"?

The article proposes an interesting yardstick: don't look at how much intelligence a company can afford to burn; look at whether the "intelligence consumed per task completed" can keep shrinking.

We need to start accounting for token waste. Amazon researchers estimate that inference systems can generate 7 to 10 times more tokens than actually needed when facing simple tasks. Meanwhile, OpenAI's enterprise customers have seen average inference token consumption rise roughly 320x year over year. Yet in PwC's survey, 56% of CEOs say they haven't seen significant financial returns. The gap between intelligence burned and things actually accomplished is now measurable.

But "AI is too expensive" is only the surface. The deeper question is: what kinds of tasks actually deserve the most expensive intelligence?

The authors' answer is that this line keeps moving. Tasks that require frontier models today may not tomorrow, as open-source models keep getting stronger. More and more ordinary enterprise work will cross that "intelligence threshold" and get gradually migrated off the most expensive models. Frontier models should ultimately be reserved for problems that genuinely need to discover new answers, where one more unit of intelligence still changes the outcome.

So what enterprises will really need in the future may not be "the best model," but a system for allocating intelligence. Every incoming task must first be judged: does it go to a frontier model, or a smaller, more cost-effective (open-source) one? Stay on-premise or go to the cloud? Require model reasoning, or can it already be compressed into a rule?

In other words, models will increasingly resemble a schedulable resource, and enterprises need to keep scheduling authority in their own hands.

This also explains why model companies and enterprises are running completely different books. Model companies want more computation to become useful, selling more tokens; what enterprises actually want is to make a recurring problem cheaper and cheaper over time, eventually not needing to call a model every time. For the former, usage growth is revenue; for the latter, the same usage growth may just be cost.

The two co-authors previously did research on "intelligence per watt" and hybrid inference where "large models decompose tasks, small models execute." The latter has already been validated: letting cloud-based frontier models handle task decomposition while local small models execute can preserve 97.9% accuracy at 1/5.7 the cost.

If you follow this logic to its conclusion, you get a somewhat counterintuitive result: the more mature an enterprise's AI usage, the less it should depend on the most expensive intelligence.

A problem that today requires ten minutes of careful model reasoning may tomorrow need only a smaller model; further down the line, perhaps no model at all. The process of enterprise AI adoption should also be one of continuously precipitating expensive inference into cheaper, more reliable execution.

Of course, this is also a piece with a clear position. Foundation Capital itself is placing bets along the routing layer, execution frameworks, and model improvement layer, and some companies named in the article align with its investment directions. Meanwhile, Anthropic — criticized by the authors as "designed to burn tokens" — has annualized revenue exceeding $3 billion, roughly 80% from enterprise customers.

So I think the most worthwhile thing to read in the full article is precisely this tension: enterprises are frantically buying frontier intelligence, yet may be paying for large amounts of work that doesn't actually need the frontier.

The authors are betting that as enterprises get better at running this calculation, the most expensive models will gradually retreat to a shrinking set of hard problems; issues that have already been solved, that recur repeatedly, will be handed off step by step to cheaper models, routing, and software.

If you happen to have an AI budget in hand, you might start by taking this yardstick to your own bill.

Below is the full translation:

Right-Sizing Your Intelligence Spend

Original title: Right-Sizing Your Intelligence Spend

Authors: Jaya Gupta / Avanika Narayan / Jon Saad-Falcon

Originally published on X

The most cutting-edge models are already shockingly smart.

This May, OpenAI announced that a general-purpose internal model had overturned a core conjecture in the Erdős unit distance problem.

Mathematicians had been puzzling over this problem for nearly eighty years. External mathematicians verified the proof; Tim Gowers commented that if a human had produced this proof, it would be worthy of direct acceptance by the Annals of Mathematics.

Anthropic, meanwhile, reported that Claude Opus 4.6 had identified and verified over 500 high-severity vulnerabilities, some hiding in codebases that had been fuzz-tested for years, accumulating millions of CPU-hours.

These achievements are real, and potentially extremely valuable. Some problems have no known answers yet, and finding them could be worth millions of dollars: a proof, a drug target, a zero-day vulnerability, or a market structure no one has fully grasped. In such cases, more intelligence is unambiguously better; burning more compute, exploring more paths is entirely rational.

But most results on today's model capability leaderboards come from tasks that are easy to define and easy to verify: proofs, patches, exploits, scientific reasoning.

There's only one problem: the rest of the economy doesn't work this way.

A claims adjuster isn't looking for new mathematical truths. She needs to determine which of the company's existing rules apply to this particular accident, this policyholder, this sequence of events.

A nurse doesn't invent medicine either. What she does is take established clinical protocols and judge them against a patient's history, current symptoms, and the hospital's actual conditions.

A logistics dispatcher doesn't discover new algorithms either. He deals with today's inventory, weather, contracts, and delays.

Some work creates value by expanding the boundaries of human knowledge; most work creates value by getting execution right within what the organization already knows.

The Economy Is Not a Research Lab

Frontier research is extremely valuable, but represents a tiny slice of the economy.

U.S. Bureau of Labor Statistics data shows that life sciences, physical sciences, and social sciences occupations combined account for less than 1% of total U.S. employment. There are roughly 2,000 mathematicians in America, about 20,000 physicists, and some 37,000 computer and information research scientists; nurses, managers, administrators, logistics workers, and customer service representatives number in the millions.

Most companies' daily operations don't require discovering unknown answers. What they do all day, every day, is apply what they already know to millions of specific decisions; behind each decision lies customers, policies, inventory, contracts, and history.

Frontier research competes on intelligence; most work in the economy competes on context and execution.

This divide is visible everywhere in the Fortune 500: Walmart coordinates inventory, stores, and suppliers; UnitedHealth coordinates medical services; Amazon operates warehouses, delivery, merchants, and customer interactions at massive scale.

These companies' moats come from proprietary data, processes refined over years, supplier relationships, regulatory licenses, physical networks, and experience accumulated from millions of repeated decisions. They may not "think better" than others, but they know their own world inside out after decades.

Imagine: replacing every employee at American Airlines, Home Depot, or Medtronic with math Olympiad contestants, treating everything as a problem to be discovered from first principles. Costs wouldn't fall; they'd rapidly rise, because every routine decision would need to be rethought from scratch. The entire organization would drown in intelligence it couldn't use or digest.

Software engineering is an important exception: the labor market is large enough, the work itself is digital, and results can often be verified through testing — so here, every incremental gain in model intelligence is especially valuable.

Yet AI frontier labs design their products to make it easy for developers to burn tokens furiously: endless agent loops, recursively nested sub-agents, "don't stop until it's done" modes.

As smaller models and open-weight models can handle more tasks, even programming is beginning to show its own capability threshold.

Every Workload Has an Intelligence Threshold

Every workload has an intelligence threshold.

Fall short, and the model can't do the job. Approach the threshold, and each incremental gain in intelligence may deliver substantial value. Cross it, and the bottleneck shifts: results increasingly depend on whether the system understands company policies, customers, history, tools, and standards, and whether it can deliver reliably, quickly, cheaply.

Models keep getting smarter; most workloads don't. Refund policies, baggage rules, invoice reconciliation processes — none of these have gotten any harder.

The latest frontier models, especially Anthropic's, keep pushing the upper bounds of intelligence: the kind that research mathematicians, physicists, and engineers attacking unsolved problems need. This market is real, but it's small. The trouble is that these same models have become the default option for ordinary enterprise work.

Developers naturally reach for the strongest model at hand, and Anthropic has made this choice frictionless: long-running agents, recursive sub-agents, intensive reasoning loops — designed from the ground up to burn tokens.

The result is frontier-priced intelligence landing on tasks that often don't need it.

Two trends are eroding this premium. On one side, open-source models are driving equivalent capability prices down. On the other, small and medium models keep closing in on frontier models, doing the same things with far less compute.

Qwen 3.8-27B is a case in point for the latter trend: a mere 27 billion parameters, matching or exceeding Opus 4.6 Max on several coding and agent benchmarks.

Our own research on intelligence efficiency points the same direction: in just 16 months, model energy efficiency improved 18x — the compute and energy needed to achieve equivalent task performance has dropped dramatically.

More and more models will cross the threshold for "ordinary enterprise work." Frontier systems only need to be reserved for the remaining handful of tasks where one more unit of intelligence still meaningfully changes outcomes.

In fact, companies already understand this principle when hiring. Have you ever hired someone vastly overqualified for the role? They reopen settled decisions, introduce unnecessary complexity, get bored, and leave. Companies often need someone capable enough who truly understands the customer and the job.

Models can be overqualified too, and the problem is more stubborn: they don't get bored and quit; they endlessly generate extra complexity, variance, and cost.

A password reset doesn't get better because an agent weighed twelve explanations, incidentally launched a security investigation, and wrote a personalized essay. Once customer identity is verified, the job is to reset the password securely and stop.

"Overqualification" is now measurable, no longer just a metaphor.

Researchers studying reasoning models have documented an "overthinking" problem: faced with simple questions, models still spend extra reasoning compute without any accuracy gain. Amazon researchers estimate that inference systems can generate 7 to 10 times more tokens than actually needed on simple tasks.

At sufficiently simple tasks, the marginal value of intelligence even approaches zero: each additional reasoning token brings almost no additional benefit.

When a task fails, is it because the model isn't smart enough, or because it doesn't understand how this company works?

Enterprises that keep defaulting to top-tier intelligence will find themselves paying frontier prices for work that no longer needs frontier intelligence.

Enterprises should assume as a default: open-weight models and specialized systems will eat their way up the stack, with frontier models reserved only for problems that haven't yet crossed the capability threshold.

And that line keeps rising.

Labs and Enterprises Run Different Books

Between labs and enterprises, incentives are in conflict.

For model labs, more useful computation means more business. Their success metrics are token consumption, session length, agent count, reasoning depth, and ability to crack harder problems.

Every additional unit of intelligence sold is a victory. So labs have incentives to set frontier models as default, encouraging longer thinking, more tools, more retries, more agents. When usage rises, their dashboards light up.

Enterprises run a different calculation: once a problem is solved, they'd prefer not to spend as much compute next time. Enterprises care about cost per effective resolution, tokens spent per claim processed, completion time, escalation rates, and customer outcomes.

The day a company figures out how to turn a recurring request into a three-bullet document, a deterministic rule, or no model call at all, the AI model company loses future revenue.

Helping customers avoid calls, routing tickets to cheaper alternatives, compressing requirements into a piece of software: the supplier has no incentive to do any of these.

Once enterprises begin to accumulate experience, this divergence becomes irreconcilable: AI model companies still want to sell more intelligence; enterprises want to use less.

This divergence is already showing up in the data.

OpenAI reports that over the past year, average inference token consumption per enterprise customer rose roughly 320x. PwC surveyed 4,454 CEOs, of whom 56% said they haven't seen significant financial returns from AI.

AI model company metrics are improving; enterprise financial returns aren't keeping up. A dashboard showing 5x token usage growth is good news for an AI model company; in a CFO's eyes, it's an unanswered question: did customer retention improve? Did cost per task completed fall? Or are we just burning more intelligence on the same work?

This isn't even the same game. One side makes money from making fires bigger; the other pays to put them out and keep them from starting again.

Architecture Follows the Economics

If an enterprise's goal is to reduce "intelligence consumed per task completed," it cannot leave every decision about how much intelligence to use and what kind to the model supplier.

This scheduling authority must reside with the enterprise itself, implemented at every layer of the stack.

Three predictions about the future:

First, millions of routers and gateways will emerge.

They'll be embedded in every serious AI harness, at the level of policies, teams, and organizations. Every router must decide for every work unit: does this go to a frontier model, a smaller open-weight model, a local model, a deterministic system, a human, or nowhere at all?

This routing layer may eventually resemble a labor market structure: the vast majority of ordinary decisions handled locally, at low cost; only a small number of genuinely exploratory problems calling on expensive frontier intelligence.

Above the routers, every organization will have a gateway enforcing security and data residency requirements, negotiating prices, setting budgets, and defining which classes of work are allowed to use which models.

Second, hybrid workloads will become the default.

Once applications can allocate intelligence themselves, different tasks within the same workflow can go to different models and systems.

Currently, nearly every request defaults to the cloud, even when data is sensitive, the task is simple, or the answer already lives in the company's own systems. This is expensive, slow, and often unnecessary.

Going forward, simple, repetitive, or privacy-sensitive portions will be handled locally and within enterprise systems as much as possible; only the truly hard, truly novel parts will be sent to cloud-based frontier models.

Our Minions research has already demonstrated one possible architecture: a cloud-based frontier model decomposes complex long-context problems, while local small models execute the resulting subtasks.

This hybrid system preserved 97.9% of frontier model accuracy at 1/5.7 the cost; a more aggressive configuration drove cost down to 1/30.4, with 87.9% accuracy still retained.

In this architecture, expensive intelligence decides what to do; cheaper models handle execution. Cost savings are only one result.

Third, enterprise AI applications will gradually split into two categories.

General proactive assistants like Instinct will let intelligence diffuse into a background capability: they observe work, spot opportunities, and act without waiting for instructions.

Specialized applications like Maximor and PlayerZero will master the context, tools, permissions, workflow states, and definition of "done" for a particular job, reliably executing enterprise intent.

Beneath these harnesses, a dedicated infrastructure layer for model improvement will also emerge.

Platforms like Applied Compute's AC2 let companies train, evaluate, deploy, and swap models running inside existing applications. The harness still handles execution; AC2 makes the intelligence inside it better, faster, and cheaper.

Thus proactive assistants identify needs, application harnesses execute, and infrastructure like AC2 continuously improves models using production experience.

As systems mature, the same tasks keep migrating to cheaper venues: as long as things still get done right, there's no need to keep using more expensive intelligence.

The smartest enterprises won't use frontier models for everything. They'll find ways to minimize expensive intelligence use, turning tasks that yesterday required complex reasoning into tomorrow's cheaper, more reliable execution.

The ultimate measure of enterprise AI progress should be this: the same things get done with less and less intelligence.

If you're also following AI — whether doing hands-on work at a big tech company, conducting research in school, or mulling a startup idea — I'd love to connect. This industry is moving too fast, too much remains unfigured out, and I hope to meet more people inside it who are curious about the future.