Sameness and Difference: Future World Lines of Embodied Intelligence (Part 1)

Oasis Capital believes embodied intelligence represents a cross-section of humanity's entry into the intelligent era. In late March in Beijing, we spent an afternoon with several star friends having a thorough discussion on this cross-section. What is the source of Scaling Law? Is simulation useful? Will the future of embodied intelligence fall into an ROI trap? What are the most likely deployment scenarios? Is the robotics industry really all foam? We've selected highlights from this fascinating discussion — we hope you'll join us in digging for sixpence and planting apples. Enjoy.

Guest Introductions

*Listed alphabetically by surname; no particular order

What We Discussed

• What are the areas of consensus and disagreement in embodied intelligence between academia and industry?
• What is the implementation path for embodied intelligence?
• What value can simulation actually provide? Will any company ever build a truly comprehensive simulation software?
• What will general-purpose hardware look like? What role does hardware play in the development of embodied intelligence?
• Does decoupling mean the same "brain" can be applied across different robot form factors?
• How do data from different robot forms and different scenarios contribute to training general embodied intelligence?
• What is the greatest advantage for teams with deep hardware accumulation and understanding in embodied intelligence entrepreneurship?
• Are hardware-background founders constrained by habitual thinking?

Key Insights

• The academic "consensus" is that data is also the source of Scaling Law for robotics, while the "disagreement" lies in where that data comes from
• The implementation path for embodied intelligence, at a macro level, involves different hierarchical levels of tasks. This hierarchical decomposition is consensus, including at Figure, Tesla Optimus, and others. Mid-level task execution planning is the core problem everyone is researching
• When embodied intelligence arrives depends on how well data can be utilized and how cheaply it can be acquired
• Simulation is not a scalable approach — humans cannot write infinite code to make it infinitely approximate reality. The upper limit of knowledge learned from simulators is the length of the code
• The key question for simulation is the degree of abstraction or specificity; before achieving a closed loop from simulation to real-world application, no one even knows how specific the simulation environment should be
• The deployment cycle for software plus hardware is far longer than for software alone; the specific form of deployment is still constrained by the objective laws of hardware
• Suitable scenarios and suitable configurations are strongly correlated; in the near term, a universal form is unlikely
• Intelligence and hardware are decoupled
• Anything involving physical products is related to production costs
• You need to know where the boundaries of current hardware performance lie to find true demand points and the possibility of large-scale deployment
• Hardware-background entrepreneurial path dependency includes both technical reliance and commercial thinking reliance

Oasis Capital: Let's have Professor Gao kick things off — in the direction of achieving embodied intelligence or general-purpose robots, what are the current areas of consensus and disagreement at the academic frontier?

Gao Yang: I believe the consensus is that data is also the source of Scaling Law for robotics, while the disagreement lies in the source of that data. Some advocate using simulation-generated data, others advocate collecting teleoperation data, some believe in harvesting data from the internet, and others argue robots should generate their own data... Different teams hold different views. I personally lean toward internet data and teleoperation-collected data, then letting robots collect data themselves through reinforcement learning. I'm not particularly convinced by data generated through simulation and similar methods.

Oasis Capital: Your VILA (Robotic Vision-Language Planning) work received praise from Figure's CEO, and your new embodied intelligence framework CoPa (Robotic Manipulation through Spatial Constraints of Parts) also demonstrates impressive precision in planning and manipulation capabilities. What do you see as the implementation path and solution for embodied intelligence?

Gao Yang: For the implementation path of embodied intelligence, I think at a macro level, there should be different hierarchical levels of tasks. For example, to make a hamburger, you need to break it into ten steps: take out the bread, make the patty, place the patty, add vegetables... These steps are the first level, called task decomposition. Below that is the specific execution of each task — mid-level task trajectory planning, from natural language commands to concrete robot end-effector trajectories. The bottom level is traditional robot control, executing trajectories. This hierarchical decomposition is consensus, including at Figure, Tesla Optimus, and others. Of these three levels, the biggest problem is the middle one — to date there's no particularly good solution, and it's the core problem everyone is researching.

I personally think the solution should use all available data. The first category is internet data: for instance, human videos. These exist in enormous quantities and contain various scenes humans might encounter in reality. But this data isn't that easy to use — after all, it wasn't filmed to serve robots.

The second data source is teleoperation data specifically collected for robots. This has developed very rapidly recently. For example, Mobile ALOHA, Shuran Song's Universal Manipulation Interface (UMI, which transfers skills directly from human demonstrations to deployable robot policies), and Fei-Fei Li's portable hand motion capture system DexCap — these all fall into the teleoperation category. I think these technologies may be the biggest game changer recently.

Mobile ALOHA is a general-purpose robot developed by a Chinese-led team at Stanford University, led by Dr. Fu

Shuran Song's team UMI — completely unaffected even when lighting conditions change dramatically

Fei-Fei Li's team portable hand motion capture system DexCap

The third data source comes later, when robots already have decent capabilities and perform tasks in real environments with their own data closed loop — this is the most ideal data source. When embodied intelligence arrives depends on how well this data can be utilized and how low the cost of acquiring it can be.

Oasis Capital: Mr. Han, from an industry perspective, what are the areas of consensus and disagreement, and how do you think about these issues?

Han Fengtao: I think the disagreement is "how to get to the end point." From a broad logic perspective, the deployment cycle for software plus hardware is far longer than for pure software. Once you involve hardware and interaction, it gets difficult. I think we can simply draw an analogy to autonomous driving — you have autonomous driving software, driving in a simulation environment, don't crash, stay in the lane, relatively okay. Once you put it in a real car and the physical environment changes, the deployment timeline becomes very long. I think the specific form of deployment is still constrained by the objective laws of hardware.

The problem now is that many data issues have only been partially solved, and we need to continue solving data. If general artificial intelligence is 10 points, it's just barely passing at the chat level, and at the embodied level it's probably around 3-4 points. Hardware, measured against pure general-purpose requirements, might be at a passing level, but everyone's expectation is 10 points, so merely passing hardware is hard-pressed to achieve full generality — we need to segment by scenario. And what exactly those scenarios are, that's the disagreement. Walking, sweeping floors, factories, service? This brings us back to path dependency — those who used to work on legs research legs, those who worked on upper bodies research upper bodies. Those without prior accumulation, well, whatever background team they find, that's what they do (laughs).

I think first, suitable scenarios and suitable configurations are strongly correlated — in the near term, I think it's hard to have a universal form, but it will definitely be possible in the future. Second, we need to enter from scenarios where robots were previously used. The previous problem was simply that robots weren't intelligent enough, so they could be used but the results weren't good. Of course, if an industry has never used robots before, then it's not just a technical problem — there are also cost issues, acceptance of robots, whether the environment is suitable, and a series of other problems.

Liu Qi: Professor Gao, why did you say you don't believe in simulation just now?

Gao Yang: The main problem is that simulation itself is very difficult. For example, simulating the action of unscrewing a bottle cap is extremely troublesome. If you don't set appropriate material stiffness and friction, it might not unscrew or the cap might fly straight into outer space. Simulation is very subtle — many tasks can be done in simulation, but if you specifically build a simulation just to "make it work," the computational cost might be enormous. Doing contact-rich manipulation in simulation environments isn't particularly suitable.

Han Fengtao: Simulation is widely used in traditional robotics because robots need to interact with the physical world. In AI, simulation is actually used less. For example, in autonomous driving and other fields, although some simple scenarios that don't require high modeling precision can be simulated, simulation isn't suitable for scenarios requiring high-precision models and dense physical interaction. From another angle, building a simulation environment for autonomous driving is much simpler than creating a general-purpose robot simulation environment, so why doesn't everyone in autonomous driving use simulation now, choosing instead to run real tests? So simulation is currently just one of several means.

Gao Yang: I also came across an interesting perspective. Everyone talks about Scaling Law — what scales is information. Simulation is human-written code; learning new information from behaviors exhibited by code — can this kind of information scale? Can the information content of human-written code keep growing larger? From this emerges a view: simulation is not a scalable approach, because humans cannot write infinite code to make it infinitely approximate reality. The upper limit of knowledge learned from simulators is the length of the code — this perspective is quite philosophical.

Liu Qi: Let me push back — aren't there cases where a simple equation describes an entire physical process? For example, you describe fluid flow with one equation of motion.

Gao Yang: I agree with that. Reduce it far enough and it's a few lines of quantum mechanics equations.

Leng Zhe: It still depends on what value simulation can actually provide? Or let's look at what's currently using simulation. One of the most commonly used simulations in autonomous driving is vehicle dynamics simulation. Its value lies in scenarios complex enough that you can't determine what effects a certain design will produce, so you use simulation to try it out. But it can't replace real-world experiments, because no matter how good the simulation is, there's still a difference from the real environment. Fluid mechanics has two branches: experimental fluid mechanics and theoretical fluid mechanics. Theoretical fluid mechanics hopes to start from first principles and accurately calculate fluid motion through a set of equations, but in most cases this simply can't be done right now. Hence experimental fluid mechanics, which starts from experimental measurements and builds approximate equations applicable only to specific conditions. These approximate equations sometimes can't theoretically explain the meaning of various parameters or the structure of the equations, but they tell you roughly what something will look like. Because human understanding of the objective world isn't deep enough, we can't simulate complex problems very accurately.

Currently, rigid body or solid body simulation works fairly well. But real-world materials — take a bottle cap, for example — when different forces are applied, its deformation isn't linear but three-dimensional twisting, and considering its internal structure, it might be higher-dimensional variation. Simulating all these variations would be a terrifying amount of work. Software simulation can't avoid some level of abstraction.

My view is that the key question for simulation is the degree of abstraction or specificity. Before achieving a closed loop from simulation to real-world application, no one even knows how specific the simulation environment should be. Will any company ever build a truly comprehensive simulation software? I think it's unrealistic in the near term. Hollywood does hair effects — as long as the hair flows and looks somewhat like reality, that's fine. But in a real environment, hair this long and this thick, moving in the wind — is it really like that? Not necessarily. But to achieve a "looks like" effect for large quantities of objects and materials, Hollywood has already invested enormous resources and published countless papers. Imagine the resources needed to build a comprehensive, bottom-up, highly realistic simulation model — it would necessarily be an even more terrifying number.

Of course, could a startup doing this satisfy investors or the market to some degree? Possibly. If done in limited scenarios — for example, only manipulating certain categories of objects, avoiding things it's not good at — it's possible.

Oasis Capital: Everyone has been discussing data just now. The market also has different voices on hardware — some teams feel it should be sufficiently general-purpose hardware from the start, like humanoid forms. What do you all think about the form of general-purpose hardware?

Han Fengtao: I think intelligence and hardware are decoupled. A healthy person's brain is the same as that of someone with limb disabilities — if you're missing an arm, you just don't use that arm. Similarly on a production line, regardless of gender, height, or build, when it comes to specific tasks everyone uses their hands to grab. The length of hands, strength — these may not affect whether the task ultimately gets completed, only the execution efficiency differs due to different hardware capabilities.

Leng Zhe: Any physical product has cost issues. A sufficiently general-purpose product that can adapt to all scenarios, plus general intelligence — does that make it suitable for all applications? No. Anything involving physical products must consider production costs. Production costs are ultimately paid by the customer. I believe regardless of product form, we must start from specific scenarios and first deploy intelligent robots in a specific scenario with some specialized design — that's the key. Starting with a general-purpose design, I think the result might be that it's not economical no matter where you put it.

Moreover, the total R&D cost of a general-purpose design is certainly higher than a specialized design. Of course, theoretically, a general-purpose design might not need much targeted R&D work in specific application domains, so it could span multiple application domains and amortize R&D costs — something specialized designs can't do. So there's this possibility: when a general-purpose design can truly be applied across multiple domains, the amortized R&D cost might actually be lower than specialized design. At that point, if the general-purpose design's R&D cost advantage can offset its component cost disadvantage, then it might have a total cost advantage. But I think this condition is too ideal — no company can achieve this in the short term.

Han Fengtao: To make an analogy, Windows operating system is very comprehensive, with Professional Edition, Enterprise Edition, etc., but people use computers for different purposes — business? Gaming? Long battery life? Lightweight?

Meng Pengfei: We believe the ultimate form of hardware is definitely humanoid; other forms may not be very meaningful. Whether it can truly achieve generality depends on the ability to continuously secure funding for deployment. Everything must return to business fundamentals — if you make something very high-end but can never deploy it, with no commercial scenarios, that won't work either. Similar things have happened too many times in history.

Liu Qi: Although I don't think any single form can suit all scenarios, I believe humanoid will be a dominant and more general-purpose form. Humanoid robots can better utilize human data — humans can tell the machine how to operate, how to move — and can solve 90% of life scenarios. Hardware isn't expensive now, so why not make it humanoid? You can have different versions, like wheeled or legged. If one form can solve most problems, that's best — no need to分散精力去做各种适配.

Han Fengtao: Everyone is looking at different time horizons when answering this question — there may be near-term and long-term distinctions. For universal needs, perhaps a 1.7-meter humanoid is sufficient. But to move bricks, you definitely need something stronger. To fish at sea, you might need to look more like a fish to be appropriate.

Liu Qi: I think forms are designed to replace humans, not to replace machines — existing mechanical equipment should still exist. What I'm more optimistic about is something like the iPhone model: continuously optimizing one form.

Oasis Capital: What role does hardware play in the development of embodied intelligence?

Han Fengtao: Hardware is the carrier of the brain. If you don't sell the hardware, no one uses the brain, and the data doesn't circulate. Where does data come from? As we discussed, part of it comes from hardware collection — it's just that when collecting data from hardware and converting it into brain skills, this needs to be decoupled from hardware. Just like Tesla initially sold the Model 3, but the collected data could be used for other vehicle models.

Oasis Capital: Does accumulating data on a general-purpose form enable better mapping in the future?

Gao Yang: I think in the vast majority of cases it's decoupled. For example, when a monkey wants to drink water, it also grabs a bottle — although its joints differ from humans, the motion of unscrewing the cap is consistent. Looking at it from an object-centric perspective, it's universal and not very related to the actuator. So I think for objects it's universal, but for a specific hardware form, how to grasp will differ. Even a non-humanoid robot can accumulate enormous amounts of data that covers subsequent fully humanoid forms — there can be some level of abstraction that allows data sharing.

Oasis Capital: For training general embodied intelligence, what help is data from different robot forms and different scenarios?

Gao Yang: Different forms and different scenarios can enrich the dataset for embodied intelligence training, ultimately making the embodied foundation model more generalizable and robust. But which scenarios and forms we can actually obtain data from depends on the sequence of commercial deployment. I believe simpler forms and simpler scenarios will definitely deploy first, so this data will be the first utilized by embodied large models. Later, more complex forms and scenarios will deploy, and then this data will continue to be used to train embodied large models. This will be a gradual process.

Oasis Capital: We've heard Physical Intelligence is also exploring robotics foundation models across different embodied forms. Does decoupling mean the same brain can be applied across different robot forms?

Note: Physical Intelligence was founded in March 2024 by Sergey Levine, with $70 million in first-round investment from OpenAI, Thrive Capital, Sequoia Capital, and others. It is dedicated to developing foundation models and learning algorithms to create a universal robot system.

Gao Yang: This is indeed still being explored academically. But for most scenarios — for example, picking up a water cup — we don't care whether the base is wheeled or bipedal. There may be millions of robot forms, but the core thing to solve is actually similar. The biggest difference might be between two-finger grippers and five-finger grippers, with different grasping methods. Whether two-finger or five-finger, once you've grasped it, it becomes a single rigid body. Currently, the academic research starting point is learning from human data and mapping it to robots. People have found that if you can relatively accurately obtain data from humans, the mapping process isn't that complicated — especially if your downstream is five-finger, you can do very well. The biggest difficulty might just be the difference between five-finger and two-finger.

Oasis Capital: For teams with deep accumulation and understanding of hardware, what is the greatest advantage they can bring to embodied intelligence entrepreneurship?

Han Fengtao: You need to know where the boundaries of current hardware performance lie, so you can find true demand points and the possibility of large-scale deployment. If you haven't done hardware, don't know what hardware is doing, and don't know where the next breakthrough in hardware will be, you can't find where the intersection of hardware and actual demand lies. Even if you make the hardware, it's hard to sell and hard to use well. Either the cost is too high, or there's already a better solution.

Oasis Capital: Could people who work on hardware be negatively affected by certain habitual patterns of thinking when it comes to entrepreneurship?

Han Fengtao: It varies by person. But I think the most likely inertial thinking is path dependency. For example, someone who used to work on arms won't work on legs, or feels robots should be high-end things — build a humanoid robot and it has to sell for 500,000 or 1 million RMB, but robots that can actually scale might only cost 2,000-3,000 RMB. This path dependency includes both technical reliance and commercial thinking dependency. Many people feel they've researched technology for so long, so deeply — asking me to make a robot vacuum, I won't do it. But from a business perspective, perhaps the first scenario where success is possible is robot vacuums.

Oasis Capital: What directions or breakthroughs do you all think would be extremely helpful for the path toward embodied intelligence?

Han Fengtao: Actually, the main bottleneck is still in AI, or general intelligence. The advances in this wave of AI: one is stronger generalization capability, stronger general capability; the other is intelligence level. Previously, every small model achieved decent results in vertical domains; now it's large models that perform quite well across many domains. Robotics is a very typical case of hardware and software iterating separately. The current situation is that robot hardware capability far exceeds software. When software develops to a new stage and discovers hardware functionality is insufficient — such as touch sensing, dexterity, etc. — hardware will iterate further. Both will develop in alternation for a long time to come; finding good integration points is how you succeed.

Leng Zhe: I feel many embodied intelligence companies haven't thought clearly enough about what their deployment scenario should be, and the problem this brings is not knowing what their hardware should look like. Product cost often has a decisive impact on a product's unit economics. Every product's cost, including software, hardware, debugging, data learning, etc. — physical component costs account for a large portion. Accurate assessment of this cost is, I think, a key advantage of hardware-background teams that Han mentioned earlier. The purpose of making robots now is to replace things humans do, so there are naturally two competitors — humans, and specialized machines (non-standard automation equipment). If the unit cost is higher than humans, or higher than specialized machines, then it definitely can't deploy. So cost needs to be considered from the very beginning. The cost of the entire AI system can indeed be amortized, but if the hardware itself (mechanics, circuits, etc.) is very expensive — more expensive than the "competitors" — then no matter how much you amortize later, the cost will never be acceptable.

Han Fengtao: Our team has significant hardware advantages, but objectively speaking this is just a first-mover advantage. To do well, you definitely need strong AI, strong hardware, and strong deployment capability. If a team starts strong in AI without hardware, but through company development recruits an excellent hardware person, that's also possible.

Gao Yang: I'm very focused on data. We discussed various data sources — how to maximally utilize each type of data source, how to prioritize data sources, so that embodied intelligence large models can meet the data demands of Scaling Law. Other questions like: what exactly is the input-output format for embodied intelligence large models? Is it RGB or RGB-D, and how to make good engineering trade-offs that are both cheap and can quickly scale data volume? I've mainly been focusing on these questions recently, and the series of imitation learning and reinforcement learning algorithms needed behind them. I believe the essence of embodied intelligence's Scaling Law is data scaling. So the core of solving this problem is how to make data as scalable as possible. With multiple data sources, each with different properties, technically we need to determine the priority of each data source, then proceed in order to technically achieve good scaling for each type of data.

(To be continued)