Liu Changliu, director of the Intelligent Control Lab at Carnegie Mellon University: "It works really well?"

From economic theory to robotics and human-robot interaction safety — today, we're joined by Professor Changliu Liu, head of the Intelligent Control Lab and assistant professor at Carnegie Mellon University's Robotics Institute, to discuss the present and future of robotics. Enjoy.

Oasis Capital: Let's start with your research area. What are your current research directions, and what are your expectations for the future?

Professor Liu: Since my PhD, my research has focused on making robots better serve humans, particularly in manufacturing. This involves two key modes:

First, replacing hazardous tasks. The main goal is to improve workplace safety and protect workers from dangerous and health-damaging tasks. For example, robots can take over weld grinding and other harmful tasks, reducing workers' exposure to dangerous particles.

Second, assisting workers with task management. For tasks that require human involvement but don't pose health risks, robots can assist workers to improve task management and workflow. For instance, introducing robots to help workers with assembly tasks and tool management can boost efficiency.

These research directions not only help improve manufacturing productivity but also address worker health and safety, thereby enhancing overall work quality, productivity, and sustainability. I believe they will have a positive impact on future industrial development.

We're also studying how to achieve efficient human-robot collaboration in shared spaces, making factory footprints smaller and more compact. This includes robots and humans working together in the same space, sharing workspace to reduce facility size.

As for the future of robotics, deep exploration of low-level control has been ongoing for years with significant results. This wave of AI revolution provides an opportunity for us to more deeply explore high-level robot intelligence — fusing multi-dimensional, multi-modal data for understanding and feeding that back into the physical world.

To achieve this, we've pursued multiple avenues. One is designing new sensors. Current sensors are primarily based on vision and sound, but sensor design in areas like tactile sensing remains relatively limited — particularly large-area tactile sensors akin to skin that could play a crucial role in human-robot collaboration and interaction. With this information, robots would undoubtedly become more intelligent.

Another focus is system reliability. With the widespread use of large models, people have recognized that model outputs can hallucinate. Concerns about safety and trustworthiness are growing. The United States recently released a white paper on AI strategy emphasizing "Trustworthiness." For models that have surpassed our level of comprehension, how to provide safety guarantees has become a worth exploring and highly challenging problem — one that our research is committed to addressing.

Oasis Capital: How do you collaborate with colleagues or students on research?

Professor Liu: Robotics is application-driven, making it difficult to specialize in just one direction. Solving application problems typically requires integrating multiple technical approaches. The industry's iteration speed is indeed very fast. We pursue multi-directional research, including end-effector design, low-level control, motion planning, skill learning, and more — all to make robots higher, faster, and stronger.

We also apply some of the latest AI achievements to robotics.

Although AI is developing rapidly, its impact on downstream physical-world applications like robotic task execution is relatively slower. Changes in the digital world require solving a key problem: how to translate digital-world exploration into practical applications. Transforming virtual exploration into real-world application is a noteworthy direction in robotics — somewhat similar to autonomous driving, where significant simulation results have been achieved but real-world deployment cycles remain relatively long.

Oasis Capital: You once spent some time weighing academia versus industry when you graduated. What were you thinking about?

Professor Liu: That's indeed a good question. When I graduated, academia was more suitable for intelligent robotics research. The human-robot interaction work we were doing didn't yet have a clear commercial application scenario. You could say that six years ago, there wasn't the kind of fertile ground for intelligent robots that exists today. Back then, autonomous driving was the spotlight — and while broadly speaking autonomous vehicles are a type of intelligent robot, their core lies in mobility, analogous to human legs. This wave of intelligent robotics development focuses more on manipulation, analogous to human hands. From my perspective, the overall rise of the manipulation-focused intelligent robotics industry is really a phenomenon of the last two years.

Oasis Capital: Intelligent robots have only improved significantly in the past two years. But ten years ago there were already products like the UR5, Google's Firefly — that was also an important milestone. What happened in those ten years?

Professor Liu: To be precise, robot learning has injected new vitality into intelligent control over the past two years, bringing fresh blood that enables robots to acquire more skills at lower time and computational costs. At the same time, declining hardware costs have made owning high-precision robots no longer difficult, so more academic resources have become involved in robotics research.

While there were some important breakthroughs in robotics ten years ago, hardware costs didn't drop significantly. However, in the past two years, especially domestically, costs in the robotics field have clearly declined. The atmosphere in the Yangtze River Delta region has particularly impressed me. Many components have seen significant cost reductions while performance has notably improved, providing strong support for rapid industry development.

Oasis Capital: We previously visited CMU and saw that Pittsburgh also built a factory, deliberately connecting academia and industry. Is that right?

Professor Liu: That's a very good observation. The Robotics Institute is a department at CMU, and under this department there's the National Robotics Engineering Center (NREC), whose members are engineers rather than students or professors. They undertake many industry-related projects.

We have a good mechanism: research-oriented projects are handled by university teams, while engineering-practice-oriented projects are led by professional engineers. This strategic collaboration model works very well in robotics. Because the vast majority of robotics projects, especially those needing to demonstrate results in real-world scenarios, require engineers' technical support. Conversely, achieving outstanding research results also requires breakthrough contributions from research teams.

This collaborative approach undoubtedly greatly benefits the advancement of the entire robotics industry.

Oasis Capital: From this perspective, have you already achieved closed-loop collaboration between your lab and engineers at school?

Professor Liu: That depends on which level you're talking about. At the results level, we have indeed achieved a certain degree of closed-loop. But reaching commercial promotion level, progress hasn't been fast enough.

Oasis Capital: Where does the gap lie between academic research and commercial deployment in robotics?

Professor Liu: I'd say it's the lab environment. People used to talk about "Sim to Real"; now it's "Sim to Lab to Real." The lab environment is something we set up ourselves, intentionally simplifying certain interfering factors. For example, to make perception easier, we might use green screens. Achieving stronger robustness in the real world is quite difficult because there are many uncontrollable factors. Unless we create a manual requiring engineers to work only under very strict conditions — but that would worsen user experience. If the robot's functionality is so limited, what's the point of buying it?

Additionally, academic paper writing focuses on specific research topics. You only need to make contributions in a specialized subfield, without considering coordination of the entire system. But in practical applications, the whole system needs to work together well.

There are many conferences that specifically organize workshops to discuss this issue. I recently attended an IROS workshop called "It works really well?" The question of "works really well" means — does your paper show significant improvement on a specific point? Or does it demonstrate more scenarios? Or did you conduct more robustness testing? These are all questions worth thinking about.

Oasis Capital: From a research perspective, could industry in robotics see explosive development like large language models?

Professor Liu: I think this is indeed a challenge. It depends on which industry robots are applied to. Different industrial domains have different robot requirements and technical approaches. In manufacturing, for example, the main concern is reliability, so we somewhat sacrifice efficiency for safety. However, for robots used in some wild applications, their operation may not threaten human lives, so safety requirements may be relatively lower while efficiency demands may be higher.

I think in industry, some concrete and practical applications will be found with more engineering-oriented solutions. But for fundamental breakthroughs, I believe academia may be the key. Industry currently struggles to find a universal data setup analogous to large language models where massive scale alone produces miracles.

Oasis Capital: It feels like traditional internet companies' AI Labs publish more papers and have louder voices than traditional industrial robotics companies?

Professor Liu: The autonomous driving wave was similar — traditional automakers basically didn't make much noise, while internet upstarts and startups were louder and more ambitious. But over time, problems inevitably emerge, like the shutdown of Argo AI. Meanwhile, some traditional automakers have been quietly adding more automated driving features.

Oasis Capital: Is the development of robotic reliability largely unrelated to large language models, or even divergent from them?

Professor Liu: Logically speaking, large language models actually solve a common problem — further deepening industrial automation. Looking back at the history of industrial automation, it started with power automation, replacing manual labor with electricity. Then automation expanded to material transport, introducing mobile equipment including conveyor belts and robotic arms.

However, many areas remain unautomated. For example, human knowledge — robots still require human operators to be online for supervision and debugging. This knowledge is difficult to encode digitally for automatic debugging. Large language models are different: they contain much human common sense. So we believe they have potential to replace human inspectors and serve as system "babysitters." Even if they can only solve part of the problem, it's enough to significantly reduce labor costs.

Oasis Capital: In certain industrial domains, ensuring accuracy requires high-quality data. However, due to physical sensor instability, data quality may be affected, making it difficult for AI to play a role in these industrial scenarios. Do you think this is a problem?

Professor Liu: I don't know the background of this question. But as long as the scope is relatively narrow, you can still find good enough sensors to obtain good enough data — it's ultimately a matter of price. For example, I once bought a camera that cost more than the robot itself. Its precision was extremely high, and we used this equipment for sub-millimeter assembly. Data collection was effortless.

But if you're trying to solve broader data collection and AI problems, then yes, you would indeed face significant challenges.

Oasis Capital: How do humans complete sub-millimeter assembly?

Professor Liu: Humans determine whether assembly is complete through tactile perception. We wanted to use tactile sensing for robots too, but unfortunately tactile precision wasn't sufficient at the time, so we had to switch to vision. However, data quality issues are indeed something people are increasingly concerned about. CMU previously applied for the Advanced Robotics for Manufacturing Institute, which is now independent from CMU and located nearby. This organization is currently working on the National AI Data Foundry project, building multiple robot production areas to simulate small production lines, installing many sensors, and recording all the data — with data at least fully open to institute members.

Oasis Capital: That sounds somewhat like what ImageNet did back then. This data should be very valuable, with high collection costs.

Professor Liu: The big four [robotics companies] may be doing similar projects. I recall that in 2016, FANUC made a system called Field System, which transmitted operational data from factory robots to the cloud. If they've been continuously doing this for the past seven years, they should have collected massive amounts of data.

Oasis Capital: From a research exploration perspective, where do the difficulties in robotics development lie? What specific schools of thought are there?

Professor Liu: Ensuring safety, reliability, and trustworthiness are the main difficulties. There are two main approaches to ensuring safety and trustworthiness: one is through hardware or controller design to ensure the robot's physical motion meets requirements; the other is through testing and verification methods, continuously stress-testing the system and iterating — where many machine learning methods can be applied.

The design approach splits into two schools: one is Passive Hardware, achieved by adding springs or other energy-absorbing elements, which is common in exoskeletons. The other is the software level that I've worked more on — using force sensors to detect collisions while simultaneously reducing robot stiffness to decrease collision energy, or using cameras to observe what's happening and react to the environment in real-time, such as avoiding collisions.

The testing and verification approaches are even more diverse. For example, one method I'm currently researching uses formal methods to compute all possible outputs of neural networks. For instance, with an obstacle ahead, we verify whether the outputs include options for colliding with the obstacle — if so, it's unsafe, and we need to find the inputs causing the unsafety and retrain. Another approach transforms complex software into symbolic systems that people can understand, helping comprehend system reliability.

Oasis Capital: Everyone is discussing household humanoid robots. From a materials perspective, if metal is used, wouldn't safety risks be quite large?

Professor Liu: Indeed. Many professors at CMU research soft robots — the kind like Baymax.

Oasis Capital: Is soft robot realization even further away? Could there be a difference similar to BERT versus GPT?

Professor Liu: I think there will be similar differences — it mainly depends on the scenario. Soft robots probably won't have much use in industry, but have great potential in home environments and social settings. Industrial environments have trained workers; home environments face elderly and children — mechanical designs would definitely be completely different.

Soft robots are indeed still a research problem at this stage. They haven't solved how to mass-produce; most are handmade with very low efficiency. But for metal processing, industry has accumulated extensive experience.

Oasis Capital: What are the main materials for soft robots?

Professor Liu: There can be various different materials. For example, a professor at Stanford University made a pure balloon surface — like a snake, it inflates to make the robotic arm longer and deflates to make it shorter. A colleague made robots from fabric; we even used fabric as robot skin because sensors can be directly woven in.

Oasis Capital: How do you feel about this wave of humanoid robot popularity?

Professor Liu: My feeling is that people working on more virtual things — because their research iterates faster and it's easier to form unified benchmarks — tend to have louder voices, like those in computer vision. What we're seeing now is many CV scholars wanting to transition from virtual to physical, and proposing concepts through this. Although the integration of virtual and physical has been ongoing in traditional directions, the paths differ. For embodied AI from a traditional technology perspective, it's embodiment first, then adding AI — AI for embodied X or AI for robotics. This has been done for a long time but isn't fully known to outsiders.

Oasis Capital: It seems academia and industry are somewhat similar — marketing capability matters a lot.

Professor Liu: Yes, presentation and marketing are very important. Good wine also fears a deep alley. If you start talking about formulas and details right away, you can easily confuse your audience (laughs).

Oasis Capital: Huawei published a paper in Nature that shook the world's best meteorological prediction institutions — the data barrier requiring large supercomputers seems about to be broken. Could robotics see similar弯道超车 [leapfrogging]?**

Professor Liu: Declining hardware costs are a good opportunity for leapfrogging. Domestic components will rise through this wave, which I think is also a good thing for the industry.

Oasis Capital: From a research perspective, if there were to be a next milestone for robotics, what would it look like?

Professor Liu: There might be milestones in different directions. For example, if fast robot learning could be achieved without the Sim-to-Real gap, reaching human learning efficiency, that would be an enormous milestone. Along this angle, achieving continuous learning would also be a major milestone — robot learning has resets, while human learning is continuous. Another milestone is in my research direction: safety guarantees.

Oasis Capital: Is robotics more worthy of attention than software from a safety perspective?

Professor Liu: Robotic systems are too complex — far more complex than typical software systems, including motors, hardware connections, wires... anything could go wrong. Safety issues easily resonate with people. Actually, adversarial attacks have existed for many years too, but it wasn't until large language models emerged that people suddenly felt urgency.

Oasis Capital: Your research combines economics and engineering. What insights have you gained?

Professor Liu: Economics initially mainly inspired me to start human-robot interaction research. From an engineering perspective, the main research method is giving a differential equation, which is close to the physical world. Differential equations can describe human motion but are far from human cognition and high-level decision-making. To give robots more intelligence, we definitely need to consider more cognitive-level, decision-level content — and economics has already studied this area quite thoroughly.

At the time, I hoped to apply some of economics' models describing human behavior to robot control. Economics assumes people are rational, goal-oriented, and choose their actions based on goals. Combining this framework with solving ODEs in engineering — this gives robots both human-like thinking capabilities and the ability to make correct movements in the physical world.

Oasis Capital: How closely related is the development of agents to robotics research?

Professor Liu: Our research is essentially about wanting agents to control robots. Traditionally speaking about robot intelligence and robot control, it's a master-slave mode: upper level is intelligence, lower level is traditional control. If we directly do intelligent robot control, we call it an agent — this agent has human-like perception and decision-making capabilities.

Oasis Capital: Is your current research direction also your ultimate goal?

Professor Liu: Yes. I always tell my students — don't you think building a robot that looks like you, thinks like you, and acts like you is a really cool thing? (laughs)

Sustaining Vitality

What do you think technological vitality is?

Begins with curiosity, ends with rigor.

— Professor Changliu Liu, Robotics Institute, Carnegie Mellon University