Code Brain | Some Calm Reflections on Figure's New Demo

Combine LLMs with robots, and can humans "manufacture humans"? The moment large models with "emergent capabilities" debuted, they sparked boundless market speculation. Now, everything looks closer than ever.

The first "ChatGPT robot" has arrived. Startup Figure, in partnership with OpenAI, has unveiled Figure 01 — a robot that can speak, see, hand food to humans, pick up trash, and clear dishes. Beyond that, it can reason, independently identify, plan, and execute useful tasks.

Is the Embodied Artificial Intelligence revolution here? What breakthroughs does Figure 01 actually represent? We've drawn on perspectives from the WeChat account "Tools and Will" (Gongju yu Yizhi) and look forward to exploring them with you.

These past few days, Figure and OpenAI's demo video has broken out of tech circles again. It aligns broadly with my prediction from early this year: in 2024, we'll keep seeing exciting developments in robotics. Figure's demo is just one of them, and certainly not the most groundbreaking.

Why this prediction? Because individual technologies across the entire robotics stack are advancing rapidly. Arrange and combine them, and you get some visually striking results.

But a word of caution for anyone getting too excited about the Figure video. Here's the conclusion up front: I don't see any genuinely new technical capabilities in Figure's demo. Its current nature more closely resembles what we called the GPT-wrapper state last year. In other words, reaching this demo state doesn't involve many secrets. Of course, "wrapper" isn't meant negatively here — hardware products need good wrappers. On that front, I have to admit Figure has done decent work.

Let's walk through some details from this demo.

1

On the Overall Architecture

Figure engineer Corey Lynch shared the architecture diagram on X. The entire algorithm breaks into three layers:

• Top layer: OpenAI model. This is what we commonly call the high-level policy, also known as the "brain." It processes language and visual inputs, handles human-robot interaction, and generates action decisions. The essence of these decisions is "selecting the model capable of executing the target action." In Lynch's own words: "deciding which learned, closed-loop behavior to run on the robot to fulfill a given command, loading particular neural network weights onto the GPU and executing a policy." This statement actually implies something: their low-level policy comprises multiple models obtained through learning, but the encapsulation/abstraction level of these sub-models wasn't disclosed (whether it's a fine-grained primitive action library or a coarser task-level library).
• Middle layer: Neural Network Policies (NNP). This is the low-level policy. Lynch emphasized: "All policies are learned, (not teleoperated)." This is crucial — all movements are generated through learning, though he didn't specify which learning method. But Lynch did offer some details: "All behaviors are driven by neural network visuomotor transformer policies, mapping pixels directly to actions." Combined with the movements in the video, we can reasonably guess they're using transformer-based imitation learning. Similar technology appeared in the recently viral Mobile ALOHA (https://mobile-aloha.github.io/), and related papers date back to at least 2020–2021. When Figure 01 places snacks into a basket, the subtle movements of its left hand seem like a telltale sign of imitation learning.
• Bottom layer: Whole Body Controller (WBC). This corresponds to the human cerebellum. Few details were disclosed here. My guess is that NNP outputs whole-body control signals for 24 joints and dexterous hand control signals at 200Hz, while the bottom layer applies safety constraints to correct and interpolate the 24-DoF joint control signals at 1000Hz. Of course, it's also possible that NNP focuses only on manipulation-related control. Watch the clip below: during hand movements, Figure 01's whole-body coordination is quite smooth (for instance, making a slight squat when placing a cup). Though this capability was already visible in Figure's earlier coffee-making video.

2

On the Integration of OpenAI, Large Models, and Robotics

At the high-level policy layer, the perception, reasoning, and conversational abilities demonstrated in the demo are direct manifestations of multimodal large model capabilities — they just hit harder sensorially after being "embodied." These abilities were already emerging last year with PaLM-E (Corey Lynch was also a contributor to PaLM-E), though GPT-4/GPT-4V has pushed perceptual understanding and reasoning a step further.

PaLM-E's downstream controller, RT-1, was also a transformer-based policy model using learning from demonstration. Figure 01 simply employs a more complex dexterous hand with more intricate control, but this doesn't necessarily represent a methodological breakthrough.

What's more interesting is how the interface between high-level policy and NNP works. At the demo's complexity level, these interfaces can be "hand-crafted" (LLMs themselves provide some flexibility). But to truly unlock LLMs' generalization capabilities, I suspect this interface is nontrivial. From recent conversations with people in the field, it seems achievable for some tasks already; extending it to broader manipulation tasks isn't entirely out of reach either.

3

On Mechanical Design

Three keywords: smoothness, cabling, and the big backpack.

Lynch particularly stressed that all videos were recorded at original speed! Original speed! Original speed!

At original speed, Figure 01's movement speed and fluidity are quite impressive — worth praising. This indicates solid performance in motors and reducers, plus well-executed NNP algorithms and subsequent correction/interpolation. It's not inferior to Tesla, which is hardware-strong. The pressure is now on Tesla and various Chinese teams.

However, Figure 01's cabling seems somewhat unrefined, suggesting they haven't adopted hollow routing yet — something that will likely change.

I noticed Figure 01's "big backpack" back when the coffee-making video dropped — probably housing batteries and controllers? On one hand, it's not aesthetically pleasing (personal opinion, to each their own). On the other, it puts the center of mass high and toward the rear.

4

Summary and Some Hopes

Combined with the high-profile announcement from PI, co-founded by heavyweights like Sergey Levine and Chelsea Finn (more luminaries are on the way...), the battle for Embodied Artificial Intelligence has only just begun. Still, I'm genuinely excited to see more resources flowing into this field. Whatever happens, the pace of technological development has accelerated.

Returning to the technology itself, I believe the critical question remains generalization of fundamental capabilities. Chatting with an industry veteran, he noted that generalization in autonomous driving still shows no clear path to resolution. For complex robots with 40–50 joints, true generalization is even harder to achieve. The silver lining: scaling laws are showing effects in this domain too. Generalization capabilities will likely emerge progressively along the sequence of autonomous navigation → gripper grasping → gripper manipulation → dexterous hand manipulation, with a timeline of at least 3–5 years. Those hoping for instant breakthroughs may need some patience.