Physical Intelligence Co-founder Chelsea Finn: Physical AI Has Reached Its GPT Moment

What Does It Actually Take for Robots to Work in the Real World?

👩 Translated by: Shirley

🧑‍🎨 Layout: NCon

This past May, Crossing published a translated talk by Jim Fan, head of NVIDIA's GEAR lab, from a Sequoia Capital event: The Endgame for Robotics: A 2040 Prediction. Fan distilled robotics' technical trajectory as "copying LLM's homework," complete with a timeline: the physical Turing test would fall within two to three years, and the endgame for robots would arrive in 2040.

Three months later, on August 19, Unitree listed on the STAR Market, becoming the first humanoid robotics company to go public on A-shares. Across the Pacific, Physical Intelligence closed its latest funding round in November 2025 at a $5.6 billion valuation. Capital markets have priced in the promise, but the conditions required to make robots genuinely useful remain contested across different teams.

In this talk at YC Startup School, Physical Intelligence co-founder Chelsea Finn confronts the question directly: reliability, memory, and a general-purpose model that requires no fine-tuning are the three essential elements for robots to truly function in the real world.

And at the close of her talk, she sets a new marker for the field: physical AI has already moved to the right side of the general AI timeline, arriving at its own GPT moment.

Speaker Background

Chelsea Finn | Co-founder of Physical Intelligence, Assistant Professor at Stanford University. In 2017, while pursuing her PhD at UC Berkeley, she proposed MAML (Model-Agnostic Meta-Learning): training a set of initial parameters such that a model could adapt to new tasks with just a few samples and a small number of gradient steps. This turned the idea of "rather than training an expert model for every task, make one model capable of adapting to all tasks" into an optimizable objective.

Full Transcript

Hello everyone! Today I'll be discussing some of Physical Intelligence's recent technical progress.

I started a company two years ago called Physical Intelligence. The question we're genuinely interested in is: how do you get any robot to perform any task in the real world?

I spoke at this same event last year, sharing progress at the time — fairly complex tasks like removing and folding laundry from a dryer, and robots doing useful work in rooms they'd never been in before. In the year since, robots have learned many more things: washing a greasy pot, peeling a carrot, making a grilled cheese sandwich, cutting a zucchini, unlocking doors, and so on.

But today I want to talk not about how many flashy tricks robots can perform, but rather what it actually takes for robots to work in the real world. There are two things here: first, generality — how to develop general-purpose models; and second, how to bring those models into reality so they have impact and are useful to people. Let's start with the latter.

Physical AI Has Less Tolerance for Error Than Any Previous AI

To answer this, it's worth looking at how previous AI systems made their way into the real world.

The first machine learning applications to truly take off were product recommendations and ad ranking. Five years later, deep learning entered those same domains. That step was crucial: deep learning could handle complex input-output problems out of the box, and transfer relatively easily to other applications. Then came ChatGPT in 2022 — the first time a general-purpose model was deployed at massive scale, hitting one million users in five days. Since then, coding agents like Claude Code have become genuinely useful.

Moving rightward along this timeline, general-purpose models are increasingly being used to solve real-world problems. But there's a more notable commonality: in all these genuinely useful and genuinely profitable machine learning applications, the final call is made by the customer themselves — the model essentially just provides recommendations.

This means errors are acceptable. People can usually recognize when something is wrong, or know what to do even if a mistake occurs. So even though these systems are far from perfect, they're still incredibly useful to users, and the external pressure for perfection is relatively low.

Physical AI is fundamentally different. Physical AI that actually operates in the physical world must make decisions that directly impact the physical world. This means its utility increases dramatically only when running fully autonomously; consequently, compared to all machine learning systems deployed to date, physical AI must make far fewer mistakes.

Is there any precedent for this? A year ago, Waymo, the autonomous driving company under Alphabet, surpassed 250,000 autonomous passenger trips in a single week. This demonstrates that a machine learning-based system capable of trustworthy autonomous operation in the physical world is indeed possible to build.

Element One: Hand Iteration Over to the System

To make general-purpose robots useful in reality, we need to figure out how to let them operate autonomously for extended periods, rather than having humans act on the model's predictions.

Suppose we want a robot to make an espresso. The task itself is already difficult: operating the portafilter requires fairly precise and forceful control to lock it correctly into the group head; it must hold a cup of liquid steadily without spilling; and it needs accurate timing — "sense of time" is typically not even a consideration in other areas of machine learning.

And we don't just want it to pull this off once — we want over 90% reliability.

The first step in machine learning is always: collect a dataset, train a model, evaluate how good the model is. Unfortunately, this path rarely works reliably on the first try.

A slightly better approach in practice is to iterate on an existing model: collect more data, improve annotation quality, make annotations more granular, add edge-case data, adjust data mixtures. This does improve reliability, but relying purely on manual human tuning, people eventually get tired, and it's very difficult to maintain reliability at very high levels.

The better approach is to let the AI system itself iterate in the scenarios where you want it to be more reliable — having it actively identify where data is missing and where more supervision is needed. Since the entire process is automated, the number of iterations can be much higher. This may be the path to getting physical AI above 99% reliability.

This thinking itself resembles reinforcement learning quite closely: keep attempting tasks, learn from failures, get better on your own. The question is, how do we build a scalable reinforcement learning system for robots?

On the language model side, there are algorithms like PPO and GRPO (the two most mainstream policy optimization algorithms in language model reinforcement learning). They've already been scaled to large language models and support fairly complex reasoning. But there's a problem when moving them to robots: these algorithms are trained with millions, even tens of millions of attempts, which works because you can just throw compute at them — each attempt is just running a language model in a data center.

Roughly translating this scale to robots: even if we don't need millions or tens of millions, just 1 million one-minute-long robot task trajectories (shorter than the espresso task I just described) would amount to 700 robot-days to get that single task to high reliability. This may not be entirely impossible, but it would be extremely difficult.

The difficulty lies in the fundamentally different nature of this cost. We're not simply burning compute to optimize a use case — we're running real robots in the real world, using real hardware, attempting real tasks. So we need an algorithm with dramatically higher iteration efficiency. And within language model reinforcement learning, there are two major sources of waste.

Waste One: Vast Time Spent on Dead Ends

Here's an example. We want the robot to fold cardboard boxes and stack them neatly. In this trajectory, it accidentally picks up two boxes stuck together. If left to continue, it would just keep trying to fold that box without first separating the two. "Folding two boxes together" isn't effective data for teaching the model to do the task well — the robot is simply wasting time down a wrong path.

Rather than letting it continue to waste time, it's better to have a human intervene and directly demonstrate what to do and how to recover from this situation. In the video, an operator is performing teleoperation (remotely controlling the robot's movements, currently the most direct way to demonstrate tasks to robots), showing "to recover, you need to first separate the two boxes." After the gripper reaches in, we see if the robot can continue on its own; it doesn't, so the human intervenes again to get it back on track. This way, every bit of data collected from the robot is used more efficiently. So: tell the robot how to recover early, don't let time burn on dead ends; if nothing else, end the attempt early.

Waste Two: Resampling the Same Prompt

Algorithms like PPO and GRPO are essentially trying to estimate which responses are better or worse among different answers. So even for just a single prompt, they'll roll out 10 times, 50 times (running the model through a complete pass to generate one answer counts as one rollout) — all to evaluate the value of each attempt, then increase the probability of good outcomes and decrease the probability of bad ones.

But this cost can be amortized: rather than collecting many attempts for a single prompt, spread that cost across different prompts, learn a more general value estimate, and use it to improve autonomous systems.

The specific approach is to train a general value function (a model that judges how far the current situation is from success) on a large corpus of robot experience videos. It learns that accidentally shaking out a shirt while folding clothes is negative progress, while actions that are pushing forward get recognized as such; the same value function can also judge what's advantageous or disadvantageous in a completely different scenario — like retrieving something from a refrigerator. This general value model is essentially predicting the time needed to reach a goal, and can significantly reduce the number of attempts required to learn improvement from experience.

With these two improvements, we have a complete general improvement algorithm: first train a base model on diverse data; use it to collect experience, with human intervention when necessary to avoid dead-end trajectories; then train a general good-or-bad estimator, the value function; and finally use that to improve the model.

13 Hours Nonstop: Real-World Workflow Testing for Reliability

With this improvement pipeline, we can fine-tune the base model to reach higher performance levels.

Let's start with the latte. This is a human-robot collaborative task: the robot handles pulling the espresso shot, while the human steams the milk. The model directly controls the robot's joints, using footage from the robot's cameras as input. It can complete the entire challenging sequence of inserting the portafilter, waiting for the right extraction time, and pouring the steamed milk into the cup. The final step is the hardest — it has to lift a nearly full cup of latte steadily and level, then translate it onto a saucer without spilling. From the robot's first-person perspective, you can roughly appreciate the difficulty of this kind of task.

Back to reliability. We ran this policy, not just once, but continuously for 13 hours. What we're evaluating isn't whether it can make a latte, but whether it's reliable enough to be useful in the real world. The answer is yes — the robot is reliable enough to work continuously for long periods without frequent errors.

The same algorithm naturally does more than just lattes. The Dandelion Chocolate factory is just a few blocks from our office. We took a workflow they normally do by hand — folding boxes, applying labels, stacking — and trained the robot to follow their actual process using the reinforcement learning algorithm described above. The resulting policy proved substantially more reliable at all three tasks.

We also applied it to folding clothes. This time what we're testing isn't the model's ability to complete a single task in a single environment, but cross-environment capability: the clothes are ones the robot has never seen, and the house is one it's never entered. It succeeded, and could work autonomously for extended periods.

Video doesn't necessarily tell the whole story, so we also did quantitative measurements. We care about both reliability and speed — for example, how many boxes can be folded per hour — so here we measure throughput (which combines both success rate and speed). The results show significant improvement in both success rate and throughput from pre-training to the SFT-like (supervised fine-tuning) stage, and again through the RL post-training stage. The RL stage alone delivered roughly 2× throughput. As for the espresso task, looking at success rate alone, we achieved over 90%.

This demonstrates that we can build a scalable, high-reliability solution for complex robotic manipulation tasks. In this case, experience and human intervention bought us 2× throughput; but most importantly, we see how to achieve long-term autonomy in meaningful real-world workflows. I believe this is precisely the key ingredient needed to make robots useful in the real world.

Of course, we've actually only run this algorithm through a few rounds of improvement iterations — running more should yield even greater improvements and higher reliability. And even with these improvements, the robot still makes mistakes and remains slower than humans; there's still plenty of room for improvement.

Ingredient Two: Without Memory, Long Tasks Are Impossible

There's one final ingredient needed to achieve long-horizon autonomy: memory.

You might be surprised to learn that most state-of-the-art robot foundation models currently have no memory or context — they predict actions based solely on current sensor observations and camera readings. Short motor skills and repetitive tasks genuinely don't need memory — none of the videos I showed earlier had context either. But if you're doing a multi-step, sequential long task, memory becomes critical, because you need to track how far you've progressed.

If it's so crucial, why don't these models have it? There are several technical reasons; I'll focus on one: if you simply feed context in mechanically, the math simply doesn't work. Suppose you feed just 10 seconds of video. At the common 50 Hz control frequency in robotics, with four cameras all fed in, and each image roughly 256 tokens, you're stuffing 500,000 tokens into the model. Doing this in real time is currently quite difficult. Even downsampling to one frame per second, you're still feeding 10,000 tokens; at least for now, the cost is prohibitively expensive, and all you get in return is 10 seconds of memory.

I won't go too deep into technical details here. Our solution to this context problem is to give the system memory at multiple timescales. The first layer is short-term video memory, about 10 seconds, but computed far more efficiently than simply stuffing all the video into the model. For longer memory — spanning minutes or even hours — you don't actually need the complete video of that past period, so we represent this portion as text: summarizing what happened in text space, then feeding this compressed textual summary (covering the past 10 to 15 minutes) into the model alongside everything else.

With this multi-timescale memory mechanism, the robot can autonomously execute tasks lasting 10 to 15 minutes, entirely without human intervention. Unlike the earlier demonstrations, this "cleaning the kitchen" task is non-repetitive: wiping the counter with a sponge, drying the counter with a paper towel and throwing it away, putting mustard in the refrigerator, putting dishes away in cabinets, washing dirty dishes in the sink.

The robot isn't repeating the same action over and over — it must remember where it is in each step.

Ingredient Three: Consolidating All Capabilities into One General Model

Those are the ingredients for long-horizon autonomy. Now I want to take one step further and integrate them all into a single general model that can do everything I've shown so far, and more. And to think about how to develop such a general model, it's enormously helpful to place it back in the context of general AI.

Looking back over the past fifteen years, I think the first major milestone was 2012: a deep learning system trained from scratch topped an external benchmark (ImageNet) for the first time. Before this, every method on the leaderboard was purpose-built for image classification; this was the first time a more general algorithm, not specifically designed for image recognition, surpassed those specialized systems.

Two years later, in 2014, we found that "take a model pre-trained on ImageNet and fine-tune it" became standard practice — using BERT or ImageNet pre-trained models did indeed yield better performance.

Further on, roughly starting from GPT-2, we shifted from "pre-train + fine-tune" to general models that work out of the box. Another milestone worth singling out: in 2021, we first saw signs of compositional generalization in these models, with DALL·E as the most typical example.

So what about physical AI?

Even as recently as 2023, for people working in robotics research, it was still very common to collect a bespoke dataset and train from scratch on that dataset. This is analogous to collecting your own ImageNet for every project and training on it — if every project starts from zero, you probably won't make much progress.

A few years ago, we were still at the far left of this timeline. Only recently have we reached that 2014 position: we have some decent pre-trained models, but haven't truly entered that right-hand zone yet.

How do we get to the right side? Specifically, how do we develop a general model that works out of the box and demonstrates compositional generalization? There are two goals here:

Goal one: out-of-the-box capability, analogous to going from BERT to GPT. Currently, if you want the best performance on a task, you always need fine-tuning — the demonstrations at the very beginning of this article (like unlocking) used fine-tuned models, other work we've done on human-to-robot transfer also required fine-tuning, and all the videos with RL post-training mentioned earlier (like making espresso) were also fine-tuned on individual tasks. But if you have to fine-tune for every single task, what you have isn't a general model.

Goal 2: Compositional generalization, inspired by DALL·E in 2021. It mattered because bridging "avocado" and "chair" showed the model had achieved some conceptual understanding of what each thing was — what a chair is, what an avocado is — to the point that it could combine them into something new. Second, it signaled a degree of data efficiency: your data doesn't need to cover every possible concept combination; the dataset doesn't actually need to contain a photo of an avocado chair. Even if it wasn't perfect in 2021, these signs were exciting enough.

With these two goals in mind, there's a well-trodden playbook for building this kind of model: first, get a large and diverse enough dataset; second, train a model with sufficient capacity. On the data side, we used everything we had: very diverse robot demonstration data, including very low-quality demonstration data; policy roll-out data (records of the robot actually running with its current policy); all reinforcement learning training data; human videos; and internet data.

On the model side, sheer capacity isn't enough — it also needs to handle heterogeneous data. We found that the key breakthrough was feeding the model all the context it needs to predict actions through a prompt.

Specifically, we trained the foundation model on inputs including: the aforementioned memory (observation memory), the task instruction to execute, subtask instructions, and metadata (marking data quality and segment length). Additionally, we could optionally provide subgoal images as prompts, meaning "a few seconds from now, you should reach a state that looks like this image."

To actually deploy this, something needs to provide these inputs, so we separately trained two things: a high-level policy to generate subtask instructions, such as what the next step of "clean the kitchen" is; and a world model to generate subgoal images.

And so, using all the diverse data at our disposal, we trained a single model that we call π0.7: it can fold a collared shirt, perform very fine assembly steps like screw insertion and tightening on a robot arm, and replace a trash bag in a garbage can — all from the same model.

Validating Both Goals, and the GPT Moment for Physical AI

Out-of-the-box capability is just the first step. The truly critical question is: how far is this pretrained model from those specialist models we trained specifically for making coffee or folding boxes?

Comparing the single π0.7 against the fine-tuned π*0.6 on throughput and success rate, the pretrained π0.7 matched or exceeded the fine-tuned specialist models developed via RL post-training for downstream tasks on every single item. The same holds for specialist models trained with supervised fine-tuning (SFT). So we do indeed have a single model that can execute many different tasks at a very high performance level right out of the box — Goal 1 achieved.

Now for the second goal, compositional generalization. We ran two tests:

Test 1: Can the robot interact with fairly uncommon appliances like an air fryer? We had it open an air fryer, put a sweet potato inside, and close it. We picked this because we never deliberately collected any training data containing an air fryer. Retrospective analysis found three segments of air fryer-related data in the dataset; but we believe it would likely still work even if we removed those three. Overall, the robot could interact with an appliance that barely appeared in the training set, combining "open/close" interaction skills with it.

Test 2: If we switch to a completely different robot, does compositional generalization still hold? We used this dual-arm UR5e industrial robot on the right side of the screen, and wanted to know if it could successfully fold laundry out-of-the-box without any folding data provided. And it did. The two robots differ not just in size but in link lengths and joint configurations. In the upper left of the screen, you can see subgoal images generated by the model — it's trying to generate images that would move the task forward, and these images are fed back into the model as input; finally it makes a few small corrections to get the clothes flatter.

The quantitative results bear this out too: as we progressed to more advanced models like π0.7, performance on folding towels and shirts on this platform it had never seen improved substantially, even approaching human teleoperation levels.

So the conclusion is that whether it's language-object interaction or task-robot interaction, we observe significant signs of compositional generalization in this model — Goal 2 achieved.

One last experiment, which I think might be the most interesting. I mentioned two ingredients — diverse data and detailed conditioning — how important are they really?

First, data diversity. If you remove the most diverse portion of data from training, the model's performance on held-out tasks (tasks deliberately withheld during training) drops sharply; whereas if you just randomly remove 20% of data (which is less diverse than that most diverse subset), performance only drops slightly. This shows that truly diverse data plays an important role in enabling the model to generalize to new tasks.

Second, metadata prompting. We ran ablation experiments (removing a component to see how much performance drops, to judge how important it really is), comparing "with prompting" versus "without prompting." The most interesting thing is the trajectory of this curve: as you add more and more data, especially more and more low-quality data, without metadata prompting, going from 80% to 100% data actually hurts performance — not too surprising, since you're adding low-quality data to the mix. With metadata prompting, adding that same batch of data actually improves performance.

By the way, all the videos and experiments I've shown evaluate the model in its out-of-the-box state, with no post-training whatsoever.

We've talked about long-horizon autonomy, and shown how to develop this capability in a general model. So returning to that original question: where exactly are robots now?

Going back to that general AI timeline, I believe physical AI is now firmly on the right side of this line, roughly in an era analogous to GPT and DALL·E, and it got here in just a few years.

More importantly, these models are already deployed in real-world scenarios. Two YC companies, Ultra (doing e-commerce packing) and Weave (doing home laundry folding), use PI models for post-training and deploy them to complete tasks like folding clothes and warehouse packing. And the robot embodiments these models can adapt to are remarkably diverse: beyond standard dual-arm platforms, there are drones, quadcopters, surgical robots, even tractors. Going forward, I think we'll see more and more robots not just in demos and research, but actually deployed in the physical world.

Live Q&A

🎙️ Audience Question

How far are we from robotics' ChatGPT moment? What would it look like?

👩 Chelsea Finn

Let me answer the second part first. I'm not sure it will replicate the ChatGPT moment for language models — one million new users in five days. Distribution channels for physical models will be much slower, because you actually need a physical robot there. Waymo's rollout has been pretty remarkable, but deploying things to physical devices still takes time.

So I'm not sure we'll have a moment with distribution volume comparable to ChatGPT; but in terms of model capability, I think we are already in a stage where these models are useful in the real world. Reaching ChatGPT-level capability within the next few years is very possible.

🎙️ Audience Question

When should a small team switch from "making specialist models bigger" to "generalist policies"? What does that transition look like? What are the signals that the timing is right?

👩 Chelsea Finn

I think starting with a generalist policy and fine-tuning it is already extremely effective. There are quite a few strong open-source generalist policies available now, like π0 and π0.5, and we've already seen many people achieve real results from these models.

The only scenario where I wouldn't recommend this is when you're operating in an extremely constrained environment. I've talked to people working on surgical robots — they operate in basement operating rooms with no internet connection and very poor GPUs, so using large models is genuinely difficult there. But even then, you can still run local inference on a workstation.

So taking π0.5 or whatever model you prefer and fine-tuning it is the right path.

🎙️ Audience Question

Given how fast robotics technology is advancing in industry, what are the real advantages and disadvantages of doing a PhD today? Especially for people who want to enter industry afterward.

👩 Chelsea Finn

I actually didn't plan to do a PhD — I wanted to go straight into industry after graduating. Both my parents are engineers who work in industry. My dad even told me he wouldn't hire a PhD. So I thought, maybe I shouldn't do a PhD or I won't find a job. Though he was in a different field, civil engineering.

But I also think a PhD is a very rare opportunity, and I personally loved my PhD experience.

First, a PhD is an excellent opportunity to learn how to deal with uncertainty and how to pick good problems. In research, no one hands you a problem — you have to choose it yourself. And after choosing, you don't know whether this problem will make progress on a six-month, two-year, or ten-year timescale. This ability to "deal with uncertainty" is especially valuable today, because at the AI frontier, in startup environments, no one knows the best path to make models stronger.

Second, it's also an opportunity to do excellent research, with great freedom to work on things that genuinely excite you.

At the same time, industry opportunities are equally rare. Everything I just showed you involves more than just research: robots need to run reliably on an entire software stack, there's a lot happening on the hardware side, plus machine learning infrastructure, data infrastructure. Much of this work doesn't require a PhD; research roles are usually accessible too, and besides, a lot of research today is engineering itself.

So it really depends on the person. Even today, I think I would still do a PhD — to learn to deal with uncertainty, to learn to do research, and because I genuinely enjoy staying at the frontier thinking about challenging problems. But there are great opportunities on both paths.

🎙️ Audience Question

Large language models learn from the internet, but robots don't have an internet-scale dataset of physical experience. What's the robotics equivalent? How do we get it?

👩 Chelsea Finn

Let's start with language models. Data on the web is language data — not all high quality, but some of it is very informative. And this data reflects what you want the model to do: you want it to predict text, complete text, answer questions, and the internet happens to have lots of answered questions and completed text. There's a general rule in machine learning: "train should match test" — what you train the model on should reflect what you'll later ask it to do.

So I think the robotics equivalent is data of robots operating in real-world environments. Our approach is to collect robot experience, to collect data of it doing various tasks. Initial data can be collected through teleoperation, but in the long run, it will also include substantial autonomous experience — records of deployed robots trying things on their own. This is consistent with where language models are now: a lot of time is spent on generating data, which means running the model and letting it think things through.

I think there are other information sources useful for training, like videos of humans doing things, YouTube videos, annotated web images that tell you "this is a kitchen, there's a fridge to the right of the sink." These are helpful for developing a frontier multimodal model that can control robots to perform tasks and reason about how long-horizon tasks should be done.

But robot experience itself is irreplaceable. You can't learn just by watching — I can watch Federer play tennis, that doesn't mean I can play like him. Similarly, robots can't figure out how to do something just by watching humans; they must accumulate experience on their own embodiment to learn effectively. I think we'll need very large datasets. This isn't to say human videos aren't helpful — watching Federer play is useful — but real experience accumulated on robot platforms is the key to building the robotics equivalent dataset.

🎙️ Audience Question

Is it possible for general robot models to become democratized through open source like large language models? Or will the cost of embodied data and hardware keep the best models concentrated in a few well-resourced labs?

👩 Chelsea Finn

I think embodied data and hardware costs will likely make this different, because getting data is harder — even if you want to distill a model, you can't just grab it off the internet.

But we have also seen fairly large datasets being open-sourced, fairly strong models being open-sourced. On the language model side, setting aside truly frontier open-source models, even companies focused on closed-source are open-sourcing heavily — like Gemma, OpenAI's open-source models. These companies are willing to support open source because it helps them build an ecosystem around what they're doing.

So I'm optimistic about this. I think there will be a strong open-source community regardless, but I'm not sure it will follow the exact same path as language models.

🎙️ Audience Question

Does the model output raw motor commands, or does it output target hand positions and let a controller solve for joint angles? Why is this the right level of abstraction to learn?

👩 Chelsea Finn

All the models I showed output target joint positions — what angle this joint should reach, what angle that joint should reach — then a controller (like a PD controller) drives the joint to the target position. The model is also trained to predict target gripper position, meaning where in 3D space the gripper should be, which you can also use to inverse-solve for joint angles. Of course, you could go more low-level and directly output motor torques, voltages, or forces.

Different options have different tradeoffs. We've found that controlling joints and controlling gripper position in 3D space both work well. The advantage of directly outputting things like voltage is that you can also control how stiff or compliant the output is; with a fixed controller, the model can't govern that dimension.

But our current approach (target joint positions + PD controller) seems to work fine — it doesn't appear to be the bottleneck. I generally prefer to focus my energy on what actually seems to be the bottleneck.

🎙️ Audience Question

Do robots need some kind of "imagination" — the ability to anticipate what should happen next — before they can truly become useful?

👩 Chelsea Finn

The π0.7 I showed earlier has something like this: it can imagine what future images should look like, then try to achieve them. On the shirt-folding example, we saw quantitative improvement from using this "imagination" compared to not using it.

We originally considered writing a full technical report just on this capability, but the model without "imagination" was already strong enough. We felt it was important to give more weight in the overall story to "you can get this far without imagination," because what's really powering those strong results is the model's own capabilities, not this "imagination."

So this is more of a design choice — it's hard to say whether "imagination" will become a critical component. But I think "predicting the future" as a training objective, relative to "predicting future actions," is highly relevant and should help with learning from all available data. Empirically it does help currently, just perhaps not as much as you might expect; and even without this imagination, robots can already do quite remarkable things.

Audience Question

Robots seem capable of many impressive tasks now, but they're very slow. What's needed to increase speed?

Chelsea Finn

I'm very excited about speed. Reinforcement learning does bring speed improvements — in RL Tokens we demonstrated speeds faster than human teleoperation.

I think one bottleneck is this: teleoperation is the easiest way to teach robots, and human teleoperation movements are inherently slow. At the end of the day, either you find ways to make the demonstrations themselves faster, or you find ways to make the policy run faster than the demonstrations it learned from. We've already seen evidence of the latter — in RL Tokens, the policy exceeded human teleoperation speed.

Audience Question

What's the most surprising robot task you've seen recently? And what do you most want to see it do next?

Chelsea Finn

The most surprising thing wasn't a specific task. When we were working on π0.7, I trained it to assemble a pinwheel: pick up a pre-cut paper piece, pick up a small pin, and stick the pin through the hole in the paper. We were very careful in controlling the policy — all the data had the right hand holding the pin and the left hand holding the paper, then inserting it.

The robot started doing this, then made a mistake — the paper fell to the right, the pin to the left. And what it did next was: pick up the paper, pick up the pin with the left gripper, and insert the pin with the right hand — it had never seen data of using the left gripper to insert the pin, neither in post-training nor pre-training data. This suggests that the robot essentially learned some kind of equivariance between left and right hands, able to transfer behaviors from one hand to the other. This is an emergent capability I hadn't seen in this type of model before.

As for what I want to see next, I don't really think about specific tasks — it's more about capabilities themselves, and how to extract the next capability from these models. There's still a long way to go in the direction of "making robots do things reliably for extended periods." One thing we've been working on recently is getting robots to cut vegetables with a knife — once you can safely use a knife, there's a lot you can do.

Audience Question

How can someone with a software engineering background transition into robotics?

Chelsea Finn

First, robotics itself has tons of software engineering work — you can try joining a robotics company as a software engineer.

There's another path I've seen people succeed with. A colleague now at Physical Intelligence named Jenny previously worked in algorithmic trading, then on legal-related things at Harvey. She was very interested in robotics, so she bought a cheap robot, tinkered with it in her bedroom, tried fine-tuning an open-source model to make it do something. Then she shared her results and sent me an email saying "I'd love to work in your lab." Her profile looked promising because she actually got her hands dirty, tried things, and genuinely loved it.

So I think getting your hands dirty, taking action, learning from that experience, then sharing it and putting it on your resume — that's a great path. There's a lot of open-source stuff available now, enough to get you started.

Alright, that's the last question — thank you all!

---

Supplement: Three People, Three Paths on Data

At the divergence mentioned at the start of the talk — whether data scale comes from human videos or the robot's own experience — Chelsea Finn takes a clear stance in this piece but doesn't unpack the underlying mechanism: why robot experience is irreplaceable.

Filling in that mechanism is Physical Intelligence's other co-founder, UC Berkeley professor Sergey Levine. In a recent episode of Ryan Peterman's tech podcast, he noted that after language models succeeded by leveraging internet data, a natural inference emerged: pre-train on web videos first, then layer robot data on top. But he argues the effective order is precisely the opposite: first use robot data to establish grounding (aligning the model's representations with real physical actions), and with that foundation in place, the model becomes far better at absorbing knowledge from other sources.

Evidence supporting this claim comes from an analysis by his colleagues and researchers at Georgia Tech. They took Physical Intelligence's own robotics foundation model — trained solely on robot data, never exposed to any human videos — and layered human video data on top, then observed what changed in the model's internal representations. The result: when the amount of robot data was small, human and robot experience remained two disconnected types of representations inside the model; but when robot data was scaled to the full amount, the two types of experience instead organized by task itself, with virtually no distinction remaining between whether an action was performed by a human or a robot.

Sergey Levine also offered three judgments not covered in this piece:

First, the timeline. He believes humanoid robots will become genuinely useful on a "single-digit years" timescale, not more than a decade. As for what counts as "genuinely useful," his criterion is whether the model, placed in a new environment it wasn't trained on, can improve itself through autonomous experience to a practical level — rather than plateauing around 50%.

The deployment path splits into two ends. One end is highly structured scenarios: environments with trained personnel all around, relatively predictable tasks, where safety concerns are lower and deployment could happen now or next year — but the marginal value of each additional data point is also lower, since the environment lacks diversity. The other end is completely unstructured environments like homes, where diversity is abundant from the start, but at the cost of much stricter safety requirements and a longer timeline — though likely still not stretching past ten years. He frames this tradeoff as: take off early but with a shallow slope, or take off late but with a steep slope.

Second, why demos are misleading. Generalization is a property demonstrated across many trials, not something a single trial can prove. A difficult acrobatic performance may look impressive, but it is an entirely different level of difficulty from "doing something mundane with an object you've never seen, in an environment you've never entered."

Third, regarding China's robotics ecosystem. Rather than answering who is ahead, he pivoted to ecosystem health. In his view, beyond models and algorithms, supply chains, manufacturing, and hardware R&D are equally components of this ecosystem; he also explicitly noted that the low-cost, reliable hardware used in robotics research currently comes in large part from China.

This last point happens to fill in another area this piece didn't expand on. Chelsea Finn's talk was entirely about the brain — how to achieve reliability, how to install memory, how one model can cover all tasks. But a ready-to-use general model requires a body that is cheap enough, reliable enough, and standardized enough to attach to — and that capability doesn't live in the lab, it lives in supply chains and manufacturing.