A Turing Award winner is betting $1 billion on AI's next decade (Part 1)

For the first time, he laid out the complete logic.

👦🏻 Author: Shirley

🥷 Editor: Koji

🧑‍🎨 Layout: NCon

While the entire industry is doubling down on LLMs, Turing Award winner Yann LeCun — one of the founding fathers of deep learning — has put roughly $1 billion on the opposite bet: "The large language models we're all-in on fundamentally cannot lead to true intelligence."

Appearing on the science education channel Welch Labs, Yann LeCun explained the full chain of reasoning for the first time: why generative AI, despite appearing powerful, has hit a wall on "understanding the world," and what his alternative bet — the World Model — actually is.

We're unpacking this video because the outcome of this debate will determine where every AI tool we use today is headed. In what follows, we'll follow LeCun's logic step by step, tracing how he arrived at the conclusion that "LLMs are a dead end" starting from the origins of deep learning.

Guest Background

Yann LeCun | Turing Award winner, father of convolutional neural networks (CNNs), former Chief AI Scientist at Meta, widely recognized as one of the founding figures of deep learning.

The convolutional networks he pioneered in the 1980s laid the groundwork for modern computer vision. His "cake analogy" proposed in 2015 accurately predicted the era when self-supervised learning would dominate AI.

Now having left Meta, he has raised approximately $1 billion to found Omni Labs, betting on an entirely new non-generative approach centered on JEPA and World Models. This makes him the most heavyweight and senior voice currently challenging the mainstream LLM trajectory.

Our roadmap: CNN origins → Cake analogy → Curse of fuzziness → Representation collapse → World Models → What he's actually betting on

From CNNs to the Thirst for Labeled Data: Where It All Began

The story starts with LeCun's own field.

Back in the 1980s, Yann LeCun already saw this revolution coming. While the AI mainstream was still building expert systems that "relied on explicit programming rather than learning from data," LeCun had already pioneered convolutional neural networks.

Twenty-five years later, the landmark deep learning model AlexNet turned out to be strikingly similar to the convolutional networks he had proposed in the 1990s.

However, as deep learning surged ahead through the 2010s, LeCun and other researchers grew increasingly concerned about one thing: this approach was too dependent on labeled training data.

AlexNet was trained on the massive, meticulously annotated ImageNet dataset using supervised learning — it was trained to match the labels assigned to each image by human annotators. By contrast, a child can learn a general representation of "dog" from vanishingly few explicitly labeled examples.

So as manual annotation gradually became the bottleneck for supervised learning, researchers began exploring alternative paths:

1. Reinforcement learning: Letting models learn from interaction with the environment rather than labeled data.

2. Self-supervised learning: Deriving labels directly from the data itself.

The Cake Analogy: One Slide That Predicted the LLM Era

LeCun had a slide that later became a meme in machine learning circles:

If intelligence is a cake, the bulk of the cake is self-supervised learning, the icing is supervised learning, and the cherry on top is reinforcement learning.

At the time, the entire field was obsessed with reinforcement learning — DeepMind's breakthroughs on Atari games and Go were its crowning moments — yet LeCun poured cold water on the approach:

This path will never get us anywhere close to human or animal intelligence, because it's too inefficient.

Later events proved that self-supervised learning succeeded in text and language far faster than in more "natural" modalities like vision.

Prediction Fulfilled: From GPT-1 to ChatGPT

OpenAI was founded in 2015, initially focused on reinforcement learning, producing OpenAI Gym and Universe along with impressive performance on complex video games.

But while most people threw themselves at reinforcement learning, Ilya Sutskever, Alec Radford, and others became interested in a new neural network architecture from Google called Transformer, originally designed for machine translation.

In experiments, Radford made an interesting modification: instead of having the Transformer translate blocks of text from one language to another, he switched to a simpler self-supervised approach:

Slice the training text into sequences, feed the model everything except the final token, and train it to predict that last token.

Radford and colleagues pre-trained on an internal OpenAI dataset of roughly 7,000 books, then fine-tuned with standard supervised learning on human annotations. This model became what we now know as GPT-1.

GPT-1 attracted little public attention at the time, yet it represented a massive breakthrough: it freed models from dependence on manually labeled data and opened up unprecedented levels of scale.

The OpenAI team quickly recognized the significance of Radford's work and went all-in, expanding aggressively — GPT-2 in 2019, GPT-3 in 2020, and ChatGPT in 2022.

In 2012, AlexNet trained on roughly one million samples. By 2020, GPT-3 used hundreds of billions.

What's more interesting is that the training paradigm that ultimately emerged was a precise recreation of LeCun's cake:

First large-scale self-supervised pre-training, then supervised fine-tuning, and finally reinforcement learning to shape the raw next-token prediction model into a useful AI assistant.

The Curse of Fuzziness: Why Generative Methods Hit a Wall on Video

This self-supervised generative approach achieved spectacular success in language, but the picture looks completely different for images and video.

Long before GPT-1's success, researchers including LeCun had already tried applying the same self-supervised generative methods to video.

The most direct approach was: have a neural network take in RGB pixel values from a sequence of video frames, then predict the next frame's pixel values — just like GPT predicts the next token in language.

The problem was, when this generative architecture was used to predict the next frame of video, the results came out fuzzy.

Moreover, large language models are autoregressive — ChatGPT generates one token at a time, feeding each freshly produced token back into the input to generate the next. Apply the same autoregression to "next-frame video prediction," and this blurriness compounds catastrophically over longer time horizons. The frame quickly dissolves into formless mush.

Root cause: language is countable, video is not

Language, complex and unpredictable as it is, still pales in comparison to video.

Language models use a fixed-size vocabulary. GPT-2 has 50,257 discrete outputs, each representing a possible next token the model might generate. But this approach of "complete enumeration" simply doesn't translate to video.

A single full-HD video frame, under normal circumstances, has pixels that can each take 256 discrete values. With 1920 × 1080 × 3 color pixels, that means roughly 10^15,000,000 possible next frames — vastly exceeding the number of atoms in the observable universe.

So video prediction models simply cannot, like language models, set up discrete outputs for every possible next frame.

Consequently, many generative video methods of this era simply had the network output raw pixel intensity values directly.

The biggest challenge with this approach: how does the model learn to handle uncertainty?

The bouncing ball thought experiment

Let's contrast an LLM learning to complete "the ball bounced toward…" with a neural network predicting the next frame of a real bouncing ball video:

On the LLM side, because the model has seen examples in its training set of balls bouncing left and bouncing right, and because it has independent outputs for each token, it can update these probabilities independently.

Not so for the video model. If the dataset shows a ball falling along the same trajectory then bouncing in different directions, because the model is forced to predict a single output frame for a given input, the best it can do in the face of this ambiguity is to predict the average of these outcomes.

When you average the pixel values across these videos, what you end up with is a blurry, faded mess.

This is the fundamental barrier the "generative" approach hits when trying to understand the real world: reality's future branches into countless possibilities, yet it is forced to predict only one.

The crucial follow-up: does the model have to be generative at all?

Of course, pixel-by-pixel prediction is just the most naive approach. Over the past two decades, people have tried many interesting image and video prediction strategies with mixed results. But these naturally arising difficulties led LeCun and other researchers to pose an intriguing question:

Does our model really have to be generative?

Returning to the GPT example: after pretraining, the "ability to auto-complete" itself isn't the point. The point is the internal representations it was forced to learn in order to master completion.

But are there other signals and methods that can also learn these powerful internal representations?

Siamese networks: a counterexample that doesn't generate

LeCun recalls that around 2017–2018, researchers began to realize: the best systems for learning image representations were precisely those that lacked generative capabilities.

You take an image, feed it into an encoder, and find ways to force that encoder to extract as much information as possible with certain desirable properties. For instance, you take two photos of the same scene, or take one photo and corrupt or transform it, feed both into the encoder, and tell the system: the representations extracted from these two images should be identical, because they semantically represent the same thing.

This "joint embedding" approach, as we used to call it, is the Siamese network.

The Siamese network LeCun refers to here was invented by him and his Bell Labs colleagues in the early 1990s for detecting forged signatures.

It works by feeding a pair of signature images into two copies of the same neural network. These copies are not trained to generate any data. Instead, they produce a string of numerical values — the embedding vector.

Two types of samples are used:

1. Positive example: a reference signature + a genuine signature from the same person.

2. Negative example: a reference signature + a forged signature.

The network copies are trained to produce embedding vectors with maximum similarity for positive examples, and maximum difference for negative examples.

When a new signature appears, feed it into the network to get its embedding vector, then compare it with the reference signature's embedding vector. If:

• The two embedding vectors are sufficiently similar → classify the new signature as genuine (from the same person as the reference);

• The two embedding vectors are not similar enough (difference exceeds some threshold) → classify as forged.

Through this joint embedding, the Siamese network learned a highly useful internal representation of signature images, all without ever predicting or generating any actual signature image:

Joint embedding = using the same network to encode two signatures into embedding vectors, where the "distance" between these two vectors directly reflects "same person or not" — close means genuine, far means forged.

And "generating images" is precisely what generative methods like GPT do.

Representation collapse: the specter that haunted joint embedding for years

Joint embedding seems to have perfectly sidestepped the "curse of blurriness," but it conceals a fatal flaw that can render the entire approach useless.

Since we're training the network to make embeddings of "original image" and "corrupted image" as similar as possible, the network can find a lazy solution: return the same embedding vector no matter what the input is.

For example, the network could learn to output an "all ones" vector for any input. Then for both the corrupted and uncorrupted versions of the same image, it returns all ones — similarity is maximized, but it hasn't actually learned anything useful. This problem is called representation collapse.

In LeCun's original Siamese network scheme, the team used what is now called contrastive learning to avoid collapse — simultaneously feeding the network both positive and negative examples.

This contrastive approach can also be applied to images and video, training the network to output similar embeddings for "different views of the same original image/video" and dissimilar embeddings for "different images/videos."

These contrastive methods have been successfully implemented for both images and video, but when scaling up, they require substantial computation and negative samples to learn meaningful representations. In the worst case, the number of contrastive samples needed may grow exponentially with the dimensionality of the representation.

By the late 2010s, LeCun and colleagues were acutely aware:

Using generative models to fully reconstruct images and video is not a good strategy for self-supervised learning.

But for representation collapse, there was still no clean solution that would let joint embedding architectures learn the kind of powerful, general-purpose internal representations that large language models enjoy.

Barlow Twins: cracking collapse through "deduplication"

The breakthrough came from a cross-disciplinary insight.

LeCun candidly admits that early methods for training joint embeddings "felt a bit like patchwork tricks," until they encountered Stéphane Deny, a postdoc researcher at Meta, who brought in neuroscientist Horace Barlow's 1961 hypothesis:

Neurons in animal and human visual systems operate by reducing redundant information between neurons.

Applying this idea to network outputs yields Barlow Twins:

Use "cross-correlation matrix approximates the identity matrix" as the loss function: high correlation on the diagonal (corresponding neurons), near-zero on the off-diagonal (different neurons) — thereby avoiding collapse while learning strong representations.

Technical breakdown: deduplication, cross-correlation matrix, and approximating the identity matrix

In the joint embedding architecture we've been discussing, embedding vectors are produced by the artificial neurons in the final layer of the embedding network.

If the embedding vector length is 128, then each network's output layer contains 128 neurons.

We feed a batch of different images into each encoder, and as the images stream in, we plot the output activation of the first neuron. It fires strongly on the first dog image, weakly on the cat image, and so on. Following the joint-embedding logic, the second encoder receives corrupted versions of that same batch.

The core idea of the joint-embedding architecture is to keep the embedding vectors generated from the same original image/video similar — meaning the output of the first neuron in the second encoder should resemble the output of the first neuron in the first encoder.

In a standard joint-embedding setup, you'd simply measure and maximize the similarity between these two vectors. But as we saw earlier, this easily leads to representational collapse — the network can just learn to output the same value for any input.

Now, following the approach Stéphane Deny proposed using the Barlow hypothesis, we should reduce redundancy between different neurons' outputs.

To measure this redundancy, the research team computes the cross-correlation between output vectors: scaling each vector and taking their dot product, yielding a single number — the correlation coefficient (a measure of whether two neurons' outputs "rise and fall together," ranging from -1 to 1; lower redundancy means this coefficient approaches zero).

Arrange the first encoder's neuron outputs vertically and the second encoder's horizontally, and you can compute correlations between all neuron pairs, placing them in a matrix. In this cross-correlation matrix, each row corresponds to an output neuron from the first encoder, each column to one from the second encoder.

Diagonal elements show correlations between corresponding neurons; off-diagonal elements show correlations between different neurons.

Per the Barlow hypothesis, we want corresponding neurons at the same position in both encoders to be highly correlated, while neurons at different positions should have near-zero correlation. So ideally, this cross-correlation matrix should look like an identity matrix.

Deny, LeCun, and other team members therefore designed a new loss function for joint-embedding architectures that measures deviation between the cross-correlation matrix and the identity matrix. They named this approach Barlow Twins.

What Barlow Twins does is force each neuron to mind its own business, instead of all converging to the same thing. This prevents the model from taking shortcuts (no collapse) while still learning genuinely useful representations.

The ImageNet Report Card

Earlier we mentioned that through self-supervised pretraining, GPT-1 could outperform purely supervised models adapted for specific language tasks. In vision, one of the most important benchmarks at the time was accuracy on the ImageNet dataset — AlexNet achieved 59.3% on the ImageNet validation set.

To compare the self-supervised Barlow Twins method against fully supervised models like AlexNet, the team used the common linear probe approach: stack a layer of neurons on top of the trained Barlow Twins encoder output, and train it with supervised learning to classify the image dataset.

The main encoder remains frozen during this training, so the simple linear probe is essentially adapting the representations learned by the Barlow Twins encoder to the ImageNet classification task.

The results were surprising: with just a frozen encoder plus a linear probe, they reached 73.2% accuracy on ImageNet — more than 10 percentage points higher than the fully supervised AlexNet of that era.

However, from AlexNet in 2012 to Barlow Twins in 2021, fully supervised methods themselves had advanced dramatically.

In 2020, Google applied the Transformer architecture to image classification, pushing ImageNet accuracy to a new high of 88.6%.

So by 2021, thanks to the breakthrough of Barlow Twins and other joint-embedding methods, self-supervised learning in vision was advancing rapidly but still lagging fully supervised approaches. The self-supervised generative pretraining approach that had fueled rapid LLM development in language remained out of reach for images and video.

The Relay: From VICReg to DINO V3, Finally Catching Up

Subsequent simplified versions like VICReg, and FAIR Paris's DINO series, carried the baton forward until DINO V3 in August 2025, when self-supervised models first matched weakly supervised/supervised models (88.4%) on image classification.

The quality of representations DINO V3 learns without ever seeing human labels is remarkable.

Take the example below: extract the embedding vector DINO produces from this patch of a human hand, then compare it against embedding vectors from other patches in the image. Using color mapping to visualize similarity between each patch and the hand patch, DINO segments the hand from the background with considerable accuracy.

The same method works for small balls, cats, books, and more.

World Models: A 17-Year-Old and a Billion-Dollar Bet

Following the successes of Barlow Twins, VICReg, and DINO v1, in 2022 LeCun synthesized these threads into a 60-page paper: A Path Towards Autonomous Machine Intelligence.

His opening question cuts deep: "We train on millions of hours of data yet only achieve Level 2 autonomous driving, while a 17-year-old can learn to drive in a few hours of practice — what's the secret?"

His answer: world models — systems that can make predictions about the physical world.

JEPA: Predicting in Embedding Space

After all this buildup, what LeCun really wants to convey is a hierarchical relationship: the joint-embedding architecture is the right foundation for building world models; and the step that actually turns it into a world model is called JEPA.

He gives JEPA's definition:

JEPA, or Joint Embedding Predictive Architecture. You take an observation of the world, take the next observation, feed each into an encoder — that's a joint-embedding architecture; then you add a predictor that uses the state at time t to predict the state at time t+1, and you can condition this prediction on an action. Now you have a world model.

Joint-embedding architecture = foundation. Add a predictor on top (plus action conditioning), and you get JEPA — a world model that can predict the future.

Here's a concrete example: instead of using a generative architecture to predict the pixel values of the next video frame, we can map both "video" and "next frame" into embedding vectors, then train a predictor to predict the "next frame's embedding vector" from the "video's embedding vector."

Under this implementation, the JEPA architecture frees the model from the impossible task of "predicting every pixel," allowing it to theoretically focus solely on predicting those truly salient features in the scene that can pass through the encoder.

LeCun offers a vivid illustration here:

If you train a model to predict what happens next in dashcam footage, a generative approach would burn most of its compute on predicting the random fluttering of roadside leaves — these things have lots of pixels, move constantly, yet are fundamentally unpredictable. JEPA can just discard them outright.

In the V-JEPA 2 paper (which we'll dive into in the next installment), the research team conditions the JEPA model on action signals sent to a robotic arm: the model observes an image sequence containing the arm and its surroundings, and is trained to predict the next video frame's embedding vector while simultaneously receiving control signals sent to the arm.

This allows the predictor to learn how various control signals will change the arm's position in the embedded image.

This learned world model is then used for robot planning and control:

1. Given an image of a target state (say, moving a cup off a platform), feed it into the next-frame encoder to obtain the goal embedding of the robot's target state;
2. Then use a control algorithm to explore the world model's predictions (predicted embeddings) under various hypothetical actions;
3. Ultimately find a set of actions that makes the "model's predicted future state" match the "target state" (predicted embedding = goal embedding).

And this whole approach of "predicting a sequence of actions, then optimizing for the optimal sequence" is, as LeCun puts it, essentially "a new twist on classical optimal control":

You build a model that, given "the state of the world at time t" plus "the action you intend to take (or intervention, or control)," yields "the state of the world at time t+1." With it, you can predict the outcome of a sequence of actions and, through optimization, find the optimal action sequence that reaches a particular result.

This actually dates back to control theory in the late 1950s Soviet Union and early 1960s West.

The only genuinely new elements are two: first, this model is no longer hand-crafted by humans but learned through machine learning; second, what it learns is no longer raw input but abstract state representations of input — that is, prediction happens in embedding space.

And that, precisely, is JEPA.

What Is LeCun Actually Betting On

At the end of the previous installment, LeCun dropped a line that "will offend Silicon Valley":

I cannot understand how you could build an intelligent agent on top of a system that cannot predict the consequences of its own actions. A truly reliable agent must be able to anticipate the outcomes of a sequence of actions, then plan its path and maintain safety boundaries. At that point, reasoning ceases to be mere "autoregressive prediction" and becomes a form of "search."

What he opposes has never been "AI will get stronger." It's the judgment that "the language model path can lead to true intelligence."

What machines truly lack is a world model that can anticipate consequences and plan ahead. This is what he's putting $1 billion on.

But trashing one path is easy; proving another path works is hard.

World models sound compelling, but can they actually outperform today's large models? Can they control a real robot?

To these questions, LeCun gave his answers in the second half of the same conversation.

In the next installment, we'll see how his world model fared when it first met reality.

---

Crossing is looking for independent contributors to write AI product and model reviews.

If you've written articles like: "Hands-on with PixVerse C1", "Hands-on with LibTV", please contact zeo0811@gmail.com. Your email should include: ① a brief bio, ② AI review articles you've written.

We offer competitive rates. Looking forward to observing and documenting the AI era with you 🎪