AIGC 'Peak Series' | [C] Talk: Robotics X Large Models, How the Inevitable Tech Revolution Will Unfold

Building machines that surpass humanity seems to be an obsession for some. From industrial automation and robotic arms to robot vacuums, service robots, and autonomous driving, robotic applications are increasingly entering mainstream conversation. Yet these applications still fall far short of satisfying that obsession.

On March 6, 2023, Google Brain (now Google DeepMind) published a paper introducing PaLM-E, a 562-billion-parameter multimodal embodied vision-language model (VLM) capable of visual perception and guiding robots without specialized training — a development that electrified industry, academia, and research communities. Researchers called it the largest VLM ever built, capable of executing diverse tasks without retraining. The paper explained how PaLM-E translates instructions into action: when given the prompt "I spilled my drink, can you bring me something to clean it up?", the robot plans a sequence — 1. locate sponge; 2. pick up sponge; 3. bring sponge to user; 4. put down sponge. PaLM-E also demonstrated emergent capabilities such as multimodal chain-of-thought reasoning (analyzing sequential inputs including language and visual information) and multi-image reasoning (using multiple images as inputs for inference or prediction).

Large models have proven capable of complex tasks, and their integration with robotics brings humanity closer to AGI (Artificial General Intelligence). But amid the excitement, numerous peaks remain to be scaled.

Over 40 years ago, Bill Gates proposed: "A computer on every desk and in every home." An outlandish idea at the time, it has since become everyday reality. So, standing at this moment in time, will humans be coexisting with robots 40 years from now?

Where does the path of large models combined with robotics lead? How should entrepreneurs read the terrain before entering the robotics track? What challenges do hardware and software development face? On June 9, Source Code Capital's [C] Talk — focused on AI entrepreneurship and innovation — convened a closed-door session titled "Crossing: Robotics." Amid spirited discussion among Source Code Venture investors and robotics scholars and founders, we heard sharp thinking from across the field.

We don't shy away from risk; we prefer to start from product and demand, exploring opportunity alongside founders through uncertainty. From over four hours of debate, we've distilled twelve statements from industry, academic, and research voices — hoping to spark further discussion through the clash of consensus and non-consensus.

1. Humanity is frantically, excitedly building things to replace itself. If robots eventually become a more efficient form of civilization, whether humans still need to exist in this world is a question worth exploring.

2. Robotics will become the most important interdisciplinary field of the next 10–20 years — robotic forms (arms, hands, wheeled, bipedal, quadruped, drones, soft-bodied, underwater, etc.) and their control technologies, AI, machine vision, human-robot interaction, brain-computer interfaces, sensors, communications, autonomous driving, semiconductors, mechanics, materials, energy, and countless other research directions all converge here. Rich demands and scenarios continuously generate unsolved problems, driving sustained prosperity in robotics research.

3. Large language models (GPT-4) acquire high-dimensional spatial cognition of objects through low-dimensional text learning alone. If such capabilities "emerge" in robotics, the explosive potential would be immense.

4. Just as Android was built for phones, a robot operating system platform will emerge — for development and research, gradually forming an ecosystem. This will happen; only the timing, form, and path remain uncertain.

5. The integration of large models and robotics may prove extraordinarily painful. Entrepreneurs considering this direction must choose with extreme caution and seriousness.

6. A good intelligent agent should be designed with sound mechanisms, then released into the real world. Because the real world itself is the most powerful "compute" — stronger than anything we see today.

7. Will open-source robot hardware exist? Like open-source software and open-source large models, could open-source hardware become industry infrastructure? Open source attracts developers; commercial opportunities may emerge around open-source hardware.

8. Robot applications fall into two categories: those that surpass human capability to achieve what humans cannot, and those that substitute for human capability to solve problems previously handled by people. Sometimes we focus too much on the latter and miss opportunities in the former.

9. The commercialization of general-purpose robots remains highly ambiguous. Today's approaches vary: some pursue endgame mode, building directly toward a general home/personal robot; some lay eggs along the way, investing heavily in R&D and selectively seizing commercial opportunities; others start from demand, integrating existing technologies. Founders should align resources and capabilities, decide their path from day one, maintain strategic discipline.

10. Software people, push software to its limit; hardware people, push hardware to its limit. Then combine the finest software with the finest hardware — perhaps this yields the highest-quality competitiveness in this interdisciplinary field.

11. From experience at major tech companies and startups alike, the central question technology and business must answer is: when. Not just whether the idea works, but whether its technical cost is at this precise moment ready to combine with commercial product. We've seen countless超前 things in labs that, without crossing the critical threshold for commercialization, never ignite.

12. Forty years ago, Bill Gates proposed: "A computer on every desk and in every home." At the time, this seemed nearly impossible. But standing at this moment, we must ask: could every household have its own robot in 40 years? The essence of realizing this vision is "market demand" — if that demand proves durable and strengthens over time, everything else follows naturally.

We welcome such abundant non-consensus colliding in an emerging industry. We know pioneering founders in this field will soon encounter the solitude of "uncharted territory." We hope to explore robotics' unknown world alongside those who master deep technology, obsess over unmet needs, and relentlessly iterate on product.