The Bitter Lesson
Why does the AI industry believe so strongly in Scaling Law today?
Why Does the AI Industry Believe So Strongly in Scaling Law Today?
👦🏻 Author: Richard Sutton
🥷 Compiled by: NCon
"Crossing" Editor's Note (Koji)
During the National Day holiday, we're dusting off a few classic articles to share with our readers.
To understand why the AI industry believes so strongly in Scaling Law today, I think The Bitter Lesson is an article you can't skip. From large language models to Embodied Artificial Intelligence, practitioners keep coming back to this short piece written in 2019 when debating technical approaches.
Richard Sutton reviews decades of AI research and points out an uncomfortable pattern for researchers: human-crafted domain knowledge tends to lose, in long-term competition, to general methods that can keep leveraging more computation.
This is also one of the important intellectual sources for why many people believe in Scaling today: stretch the timeline, and what truly matters isn't how elegant a method is right now, but whether it can keep improving as compute grows.
So during the holiday, I'd like to recommend reading the original text. It can help us understand where many AI practitioners' confidence in the future comes from, and how this belief influences their choices of problems, system design, and research investment.
Stretching the Timeline
Looking back at 70 years of AI research, the most important lesson is this: in the long run, what works best are usually general methods that can keep leveraging more and more compute.
The reason is Moore's Law. More precisely, it's because computation keeps getting cheaper — the cost per unit of compute has been declining exponentially for the long term.
When AI research projects begin, researchers often implicitly assume that available compute won't change much. Under this assumption, encoding human knowledge into the system seems like one of the few ways to improve performance.
But stretch the timeline even slightly — just a few years beyond a typical research project — and available compute increases dramatically.
To produce results in the short term, researchers naturally turn to domain knowledge they already understand. But in the long run, what truly determines success is whether a method can keep leveraging more and more compute.
In theory, domain knowledge and computation don't conflict; in practice, they often compete for the same time and energy. Time spent on one path can't be invested in another.
And once too much is invested in one path, researchers tend to become psychologically attached to it.
What's worse, encoding large amounts of human knowledge into systems often makes them increasingly complex, making it harder for them to benefit from general methods that could leverage compute.
Examples Everywhere
Examples of this pattern abound throughout AI research history.
The following are especially typical.
In 1997, the computer that defeated world chess champion Garry Kasparov relied on massive, deep search.
At the time, many researchers had been trying to encode human understanding of chess structure into programs. They were deeply disappointed to see deep search win.
What ultimately prevailed was a simpler approach: massive search, paired with specialized hardware and software.
These researchers weren't convinced. They argued that "brute-force" search might have won this time, but it wasn't a general strategy; besides, this wasn't how humans played chess.
They had hoped that methods relying on human knowledge would triumph. The opposite happened.
Computer Go repeated the same story, just about 20 years later. (Translator's note: The "about 20 years" here refers to the roughly 19-year gap between IBM's Deep Blue defeating Kasparov in 1997 and AlphaGo defeating Lee Sedol in 2016.)
Early on, researchers invested enormous effort trying to use human Go knowledge and game features to minimize search as much as possible.
But once massive search truly worked, this carefully designed knowledge not only became irrelevant — sometimes it even held the system back.
Another key to the Go breakthrough was learning value functions through self-play. Learning matters in many games, chess included, though it hadn't played a major role in the 1997 system that beat the world champion.
The "learning" behind self-play, like large-scale "search," can convert massive compute into greater capability.
Search and learning are the two core methods in AI for leveraging large-scale computation.
Whether in chess or Go, researchers initially wanted to use human understanding to reduce search. Truly massive progress only came after they embraced search and learning.
Speech recognition went through the same turning point. In the 1970s, DARPA funded an early competition.
One class of approaches relied heavily on handcrafted knowledge — about words, phonemes, and the human vocal tract. Another class, based on hidden Markov models (HMMs), was more statistical and used more computation.
Once again, statistical methods defeated methods relying on handcrafted knowledge.
How Victory Changed Things
This victory transformed the entire natural language processing field. In the decades that followed, statistics and computation gradually became mainstream.
The recent rise of deep learning in speech recognition is simply the latest step along this path. It reduced dependence on handcrafted knowledge, used more compute, and learned from massive data — ultimately improving speech recognition performance dramatically.
Like with board games, researchers always wanted to design systems according to their understanding of human minds, encoding this understanding directly into machines.
But Moore's Law brought more and more compute, and people found effective ways to use it.
By comparison, forcing human knowledge into systems often backfired in the end, wasting enormous amounts of researchers' time.
Computer vision went down the same detour.
Early methods relied on hand-designed features: finding edges, recognizing generalized cylinders, or describing images with SIFT. Today, most of these methods have fallen out of the mainstream.
Modern deep learning networks preserve only convolution and a few invariance assumptions, yet achieve far better results.
This is a weighty lesson. But the entire AI field still hasn't truly digested it, because we keep repeating the same mistakes.
To avoid falling into the same trap again, we first need to understand: why are these mistakes so tempting?
The bitter part is this: encoding our understanding of "how to think" into systems upfront may work in the short term, but fails in the long run.
History keeps showing the same process:
- AI researchers try to encode their knowledge into agents upfront;
- This does help in the short term, and makes researchers feel good;
- But in the long run, this path stalls and even blocks further breakthroughs;
- True breakthroughs ultimately come from the opposite direction: scaling up computation, letting systems improve themselves through search and learning.
The lesson is bitter because what ultimately wins is often not the human-centered approach that people prefer.
The First Lesson
Take seriously the power of general methods.
Even with today's already massive compute, such methods can keep improving as compute increases.
Currently, the two methods that seem most capable of scaling this way continuously, without showing obvious limits, are search and learning.
The Second, More General Lesson
The content that minds must ultimately accommodate is extremely complex, and this complexity cannot be fully captured by any simple set of rules.
We should stop trying to explain space, objects, multiple agents, or symmetry in advance with a few simple concepts.
These all belong to the external world, and the external world is inherently infinitely varied and complex.
Therefore, we shouldn't encode specific understanding of these things directly into systems. What should truly be built in are "meta-methods" capable of discovering and capturing complexity on their own.
These methods certainly need to find effective approximations, but the work of finding approximations should be left to the methods themselves, not done for them by us.
We want AI agents to discover as we do, not merely to be loaded with answers we've already found.
If we simply encode ready-made discoveries into systems, we end up obscuring how "discovery" itself happens.
Author: Richard Sutton
Original text: https://www.cs.utexas.edu/~eunsol/courses/data/bitter_lesson.pdf