A 20-Minute Talk by a NVIDIA Scientist: The Endgame for Robots, a 2040 Prophecy

"I'm 95% confident we'll reach the endgame of robotics by 2040."

👩 Author: Shirley

🥷 Editor: Koji

🧑‍🎨 Layout: NCon

In the first few months of 2026, a clear narrative is taking shape in embodied intelligence. This 20-minute talk at Sequoia Capital's AI Ascent event is the clearest industry-wide map we've seen so far.

Jim Fan didn't showcase any product performance curves. Instead, he tied three things together through a single chain of reasoning: the historical arc of the past fourteen years, the current landscape of models and data, and a prediction for the endgame in 2040.

For anyone trying to understand where embodied intelligence is headed, this is a time-stamped mental framework.

Speaker Background

Jim Fan is a senior research scientist at NVIDIA and co-leads the Generalist Embodied Agent Research (GEAR) group. He is widely regarded as one of the leading figures in embodied intelligence.

He was present when Jensen Huang delivered the world's first DGX-1 to the OpenAI team. Today, his work is about converting NVIDIA's compute advantage into systematic breakthroughs in robotics.

Five Core Arguments

1. Great Parallel: Rewriting Robotics Using the LLM Playbook

Jim Fan distilled the success path of LLMs over the past few years into three steps: large-scale pre-training → supervised fine-tuning for alignment → reinforcement learning for reasoning and automated research.

He believes robotics is following the exact same path, and named this replication strategy "Great Parallel": pre-train with world models to predict the next physical state, fine-tune with actions to align with the simulation space that real robots care about, and let reinforcement learning run the last mile.

2. The VLA Era Ends, WAM Is the New Paradigm

For the past three years, the dominant vision-language-action (VLA) models have essentially grafted action heads on top of vision-language models (VLMs). The vast majority of parameters serve language, with vision and action coming last, resulting in models that excel at nouns but are severely deficient in physical laws and verb execution. Jim Fan calls this "top-heavy in the wrong place."

The real breakthrough, counterintuitively, comes from video world models — by merely predicting the next frame's pixels, models spontaneously develop understanding of gravity, buoyancy, and lighting, with no hand-coded physics equations whatsoever.

From this emerges the new paradigm of WAM (World Action Model). NVIDIA's representative WAM research model, DreamZero, jointly decodes the next world state and the next action, achieving zero-shot generalization to tasks never seen during training.

3. Teleop Will Sunset, Sensorized Human Data Is the Future of Data

Teleop (teleoperation data collection) yields roughly 3 hours of effective data per robot per day. It's expensive, invasive, and doesn't scale.

Jim Fan argues that robotic data collection needs to move toward a backend data flywheel model similar to Tesla's FSD, centered on first-person human videos with hand tracking and dense language annotations, scalable to tens of millions of hours. He calls this Sensorized Human Data.

NVIDIA's EgoScale has validated that with 21,000 hours of in-the-wild first-person video for pre-training (zero robot data), plus 50 hours of high-precision motion-capture glove data and 4 hours of Teleop for action fine-tuning, end-to-end control of a 22-DOF robotic hand can be achieved.

The most critical finding: dexterity also has scaling laws — pre-training data volume and validation loss show a clear log-linear relationship.

4. From Real-to-Sim-to-Real to Dream Dojo: Compute = Environment = Data

Frontier LLM labs already have programming environments for reinforcement learning at the million level. Robotics needs training environments at the same scale.

NVIDIA's solution comes in two steps:

Step one is Real-to-Sim-to-Real: film a scene with an iPhone, extract objects through 3D scanning, reconstruct them in a physics simulator, and infinitely generate morphologically similar but randomly attributed variants (digital cousins). The iPhone becomes a pocket world scanner.

Step two is Dream Dojo: rather than relying on traditional physics equations and graphics engines, it shifts to a purely data-driven neural simulator. Input continuous action signals, and it outputs the next-frame RGB image and sensor states in real time — pixels generated live by neural networks, robot mechanics learned automatically from data.

Combined, these two steps form a strategic equation: compute = environment = data.

5. Three-Stage Endgame Prophecy: 2040, the Endgame Arrives

Jim Fan defines the robotics endgame as three gates:

Physical Turing Test (in his estimation, reachable in about 2-3 years — humans cannot distinguish whether a task was performed by a robot or a human).

Physical API (the entire robot fleet can be orchestrated by API, achieving humanless factories akin to an "atomic printer").

Physical Auto-Research (robots autonomously design the next generation of themselves). From AlexNet in 2012 to AI Ascent in 2026 took 14 years. He is 95% confident that the endpoint of this endgame will be reached by 2040.

Full Transcript Below

---

A DGX-1 and a Summer When People Started Believing in Deep Learning

One summer in 2016, right here in this building we're sitting in now, a muscular man in a shiny leather jacket walked in carrying a massive metal tray.

Words were engraved on it: "To Elon and the OpenAI team, for the future of computing and humanity, I present you the world's first DGX-1." That was the first time I saw Jensen Huang.

Like all interns, I quickly got in line to sign my name on it. Can you spot it? My name is right here. Can you recognize another name? Andrej is here too. Looks like we're all heading into the Computer History Museum soon. I feel like an antique.

At that time, I had absolutely no idea what I was part of. But no one could describe what happened next better than Ilya himself: "If you believe in deep learning, deep learning will believe in you."

And deep learning paid us back — hard. Far beyond anyone's expectations. From 2020 to 2026, in just three steps and six years, LLMs completed a journey later called "the race to the endgame":

GPT-3's next-token prediction taught the model the form of language; InstructGPT's supervised fine-tuning aligned this simulation engine to useful work; o1-style reasoning used reinforcement learning to surpass imitation learning, and automated research further accelerated the entire loop beyond human capability.

As Andrej put it, every lab is closing in on the final boss. The people working on LLMs are having the time of their lives, riding a "mythical creature" called Mythos and sprinting toward AGI. Honestly, I'm incredibly jealous.

So why can't robotics get a piece of the action?

Great Parallel: Robotics Must Take the Same Path

As a scientist with self-respect, I copied the homework and gave it a new name. I call it "Great Parallel":

Instead of simulating strings, we simulate the next physical world state; then align through action fine-tuning to that small slice of simulation space that real robots care about; finally, let reinforcement learning run the last mile.

There you have it. Great Parallel. Replicate the LLM success path. If you can't beat them, join them.

So how do you play the endgame? It boils down to two things: model strategy and data strategy. Let's start with models.

The Fundamental Flaw of VLA and the Rise of Video World Models

The past three years have been dominated by VLA (Vision-Language-Action) models — both π0 and GR00T fall into this category. The basic logic of VLA is straightforward: VLM pre-training has become so powerful that you can simply graft an action head on top.

But think about it more carefully. These models should really be called LVA (Language-Vision-Action), because the vast majority of parameters are allocated to language processing. Language is the first priority; vision and action come second.

The result is that VLAs are excellent at encoding knowledge and understanding nouns, but fall far short on physical laws and executing verbs. They're top-heavy in exactly the wrong places.

For example, moving a soda can next to a photo of Taylor Swift — the model had never seen Taylor Swift as a target during robot training, yet it generalizes successfully. But this generalization relies on language concept transfer, not genuine physical understanding.

So what's the second pre-training paradigm? Please — I kept thinking it would be something grand. Turns out, it's what we call "AI video slop." Like security camera footage of a cat playing the banjo. I could watch that all day. Peak internet, basically.

But look closer. No one took this seriously at first — until we realized that these video models are internally learning to simulate the next world state.

Here are some frames generated by Veo 3. You can see the model has picked up on gravity, buoyancy, lighting, reflection, refraction — none of it pre-encoded. Physical laws emerge spontaneously through large-scale next-patch prediction.

Even visual planning emerges. Watch how Veo 3 solves mazes by forward-simulating in pixel space. Pay close attention to the bottom-right example — don't blink. It's genuinely clever. Veo 3 discovered that if you're not looking, geometry is optional. I call this Physics Slop.

DreamZero: The Representative Research Model of the WAM Paradigm

So how do we make these world models useful? We do action fine-tuning, aligning on the superposition of all possible future states, then compressing it down to a thin slice — the part that real robots actually need.

DreamZero — a novel policy model that can simulate several seconds into the future and act accordingly.

Now, motor action signals are high-dimensional continuous signals. Numerically, they look like pixels. We can render them alongside video. So DreamZero can jointly decode the next world state and the next action. Ultimately, it solves zero-shot tasks and verb categories never seen during training.

When the robot executes a task, we can visualize what it's "dreaming" — and the correlation is remarkably tight:

If the video prediction succeeds, the action works; if the video hallucinates, the action fails.

Vision and action become first-class citizens again.

We had a lot of fun with DreamZero — pushing robots around the lab, feeding in random prompts. Of course DreamZero can't robustly complete every task, but it's like GPT-2, grasping at the right morphology of motion in each case.

So DreamZero is our first step toward open-vocabulary prompting in robotics. We call this new class of models World Action Models (WAM).

Let's take a moment of silence for our old friend VLA. You served us well. Rest in peace. WAM forever!

The Data War: From Teleop to Sensorized Human Data

Model strategy covered — now for data strategy.

Here's NVIDIA Chief Scientist Bill Dally doing teleoperation in our lab. Given his salary, this trajectory is probably the most expensive data point in our dataset to date.

The past three years have been the golden age of teleoperation. VR headsets, ultra-low-latency streaming systems, elaborate contraptions that look like medieval torture devices — massive industry investment, deeply painful process.

But teleop has a hard physical ceiling: 24 hours per robot per day. Honestly more like 3 hours, depending on whether the robots are in a good mood — they "throw tantrums" constantly.

So how do we do better? The answer: just strap the robot hand onto your own hand.

This is UMI (Universal Manipulation Interface) — deceptively simple, profoundly deep. Wear the robot end-effector on your hand and collect data directly, while keeping the rest of the robot body out of the control loop.

UMI may be one of the greatest papers in robotics data ever. It directly spawned two unicorn companies — Generalist on the left (improved gripper design), Sunday on the right (built a three-finger data glove).

Last year, we took this further with DexUMI — an exoskeleton designed for 1:1 mapping with five-fingered dexterous robot hands.

Let's see it in action: far left is direct human data collection, fastest; far right is traditional teleoperation, where even the most skilled PhD students need to align very carefully — slow, low success rate; middle is direct exoskeleton wear.

We then trained robot policies on this data. What you're seeing here is a policy trained on zero teleoperation data, running fully autonomously. We broke the 24-hour-per-robot-per-day ceiling. Look how happy these robots are — finally freed from the data collection loop.

So problem solved? Has robot scaling been cracked?

Anyone here drive a Tesla or Waymo? You know that when you drive, you're actually contributing to the largest physical data flywheel. And you're completely unaware — during FSD, data upload runs in the background.

But wearing UMI is still cumbersome, still invasive. Not as seamless as driving to work. We need an FSD equivalent for robotics. Data collection must recede into the background.

So we're going all-in on human egocentric video with hand position tracking and dense language annotation.

EgoScale: Dexterity Pre-training with Zero Robot Data

EgoScale is the first fruit of this bet.

We pre-trained EgoScale on 21,000 hours of in-the-wild egocentric human data. 99.9% of training data is human egocentric video, mapping directly from camera pixels to a 22-DoF high-dexterity robot hand, forming an end-to-end policy — fully autonomous throughout.

During pre-training, we predict hand joints and wrist pose. For the subsequent action fine-tuning stage, we collected only 50 hours of high-precision motion capture glove data, plus 4 hours of teleoperation — less than 0.1% of total training.

This is what lets EgoScale generalize to extremely fine-grained tasks: picking individual playing cards, manipulating syringes to transfer liquid, and learning different shirt-folding techniques from just a single demonstration at test time.

The most fascinating finding in the paper is about neural scaling laws for dexterous manipulation. There's a remarkably clean relationship between the number of pre-training hours and optimal validation loss — essentially a simple log-linear equation.

It's been exactly six years since neural scaling laws were first proposed for language models. If we put all data strategies on one plot, with the X-axis representing alignment with robot hardware and the Y-axis representing scalability:

Teleoperation scales the worst. Wearable data-collection devices can reach hundreds of thousands of hours. And first-person video, if it can spin up the FSD flywheel, will easily hit 10 million hours within a year.

Everything to the left of the dashed line belongs to the new paradigm: Sensorized Human Data.

If I were to make a prediction:

In the next year or two, the proportion of teleoperation will keep dropping to almost negligible levels. Then we'll see a wave of wearable data devices customized for different hardware and use cases. Eventually, the primary data source for robot training will be first-person video.

Let's pour one out for our old friend Teleop. You served us well. But long live Sensorized Human Data!

The Simulation Arms Race: From iPhone to Dream Dojo

Are we done with data strategy? Notice I drew two circles around it — so what's the outer circle?

Every frontier LLM lab is now spending serious budget on millions of coding environments for reinforcement learning. Robotics is no different — we urgently need to scale up our training environments.

Sure, you can do RL directly on real robots. Our lab has pushed certain tasks to near-100% success rates this way, with robots running continuously for hours. But to reach a million environments, the old approach would require a million robots. We need a better solution.

Here's the idea: you take a photo with your iPhone, run it through a 3D world-scanning pipeline to extract all objects, then automatically reconstruct them in a classical physics simulator. Every object becomes interactive after scanning. Then, using what we call "digital cousins," you infinitely augment these objects in simulation. The iPhone becomes a world scanner that fits in your pocket.

We call this process Real-to-Sim-to-Real: digitizing the physical world at scale.

But this still relies on traditional graphics engines. Can we do better? The answer is Dream Dojo.

Dream Dojo upgrades video world models into full neural simulators. It takes continuous action signals and outputs the next-frame RGB and sensor states in real time.

Every pixel you see here is fake. Dream Dojo learns robot kinematics purely through data, with no physics equations or graphics engines involved.

So the new post-training paradigm for robotics is a massively parallel RL system: running on a handful of real robot workstations, with massive GPU graphics cores handling world scanning, and heavy inference compute running world models.

As this equation shows:

Compute = Environments = Data

Together, this is the Great Parallel that robotics will follow. And it's already happening. We're witnessing the beginning of the endgame.

The Three Gates of the Endgame

Has anyone played Civilization? It's my favorite game. I imagine my research as unlocking achievements on the tech tree.

In robotics, there are three achievements left to unlock. Once I unlock them, I'll retire. Honestly, I can hardly wait.

Gate One: The Physical Turing Test

In a broad range of everyday activities, you cannot tell whether a human or a robot is performing the task. The Physical Turing Test measures units of labor output per unit of energy input. Look at this robot's awkward posture — we still have plenty of work to do. Probably 2-3 years.

Gate Two: The Physical API

You own an entire fleet of robots that can be configured via API and command line like any software, eventually orchestrated by some version of Opus 9.0.

With the Physical API, we get lights-out factories — essentially "atom printers" that take designs from Markdown files as input and output fully assembled products, running completely autonomously. Or wet labs that automate scientific discovery in chemistry, biology, and medicine.

Gate Three: Physical Automated Research

When robots begin autonomously designing, improving, and building the next generation of themselves, they will far exceed the boundaries of human capability. You might ask: isn't this too sci-fi? Will we see it in our lifetime?

It took the AI community a full 14 years to go from AlexNet's first forward pass in 2012 — a model that could barely distinguish cats from dogs — to AI Ascent in 2026, where we're already discussing agentic automated research. Now add another 14 years.

2026 sits exactly halfway between 2012 and 2040, and technological progress doesn't advance linearly — it grows exponentially. So:

I am 95% confident that we will reach the endgame, the terminal node of civilization's tech tree, by 2040.

If you believe in robotics, robotics will believe in you.

To everyone here: I think our generation was born at the wrong time — we missed the golden age of exploring Earth, and we won't catch the grand voyage to the stars. Yet we were also born at exactly the right time, at this precise moment for solving robotics.

Crossing is looking for independent writers to produce AI product and model reviews. If you've written pieces like "Hands-On: PixVerse C1" or "Hands-On: LibTV," please reach out to zeo0811@gmail.com. Your email should include: ① a brief bio, ② AI review articles you've written. We offer competitive rates. Looking forward to observing and documenting the AI era together 🎪