When Embodied AI Is Given a Sense of Touch | VitalCommunity

In early 2023, we believed that embodied intelligence companies powered by general-purpose large models would become the next generation of global general-purpose devices.

Over the past year and a half of exploration, we've been honored to discover and invest in a series of robotics companies. The deeper we go into this field, the more we believe this is a long-term industry that requires a community of believers to explore together.

So we began organizing more industry discussions to explore the future of robotics — for example, a small gathering after the ICRA conference in Japan this past May.

May 2024, post-ICRA gathering in Japan

At this meeting, we discussed many questions about touch. But can touch really be achieved? What would a future with touch look like? So at the end of July, we invited seven senior figures from academia and industry to share their experiences and insights, and to imagine the future development of haptic technology. We've compiled portions of this closed-door salon, hoping these imaginative ideas might spark some inspiration. Enjoy.

July 2024, Oasis Capital haptic technology discussion gathering

*Guest introductions are arranged in order of speaking

Xin Wang, CEO of Yudie Technology

Si'ao Wang, CTO of Jizheng Robotics

Rui Chen, Assistant Researcher, Department of Mechanical Engineering, Tsinghua University

Dawei He, CEO of Tuxin Electronics

Gengchao Xiong, CEO of Saigan Technology

Fred Yang, CEO of UNIX Robotics

What We Discussed

• With vision technology so mature, why do we still need touch?
• How big an opportunity is touch, really?
• What are the obvious limitations of touch? And what does the future of the technology look like?

With Vision Technology So Mature, Why Do We Still Need Touch?

Xin Wang: In my view, humans are essentially an intermediate way of processing signals. I think for human development as a whole, two things matter: first, acquiring more energy; second, improving energy efficiency. Regarding improving energy efficiency, I believe VR is an especially good approach. For instance, it can let people experience attending exhibitions and other activities at extremely low cost, without consuming large amounts of natural resources — that's inherently a very economical thing. At the same time, VR technology lets me experience all kinds of worlds, so I genuinely hope to personally build such a world, one where advanced technology lets me more immersively experience not just mature visual and auditory experiences, but also comprehensive sensations of smell, taste, and touch.

I've always believed touch ranks just after vision and hearing, so from there I committed myself to research and development in this field. In this process, we developed a kind of sensor, and later introduced an entirely new concept — smart clothing. I noticed that many products in our daily lives are becoming intelligent, but clothing rarely is. Smart clothing is actually a very important product, but the biggest obstacle to clothing's intelligence has been the wiring problem: there's no elastic, flexible wiring. Rigid wires like phone charging cables aren't suitable for clothing — they severely impact wearing comfort.

If we can develop elastic wiring, smart clothing will become much more comfortable and practical. Our current AR devices, phones, and computers could be integrated into clothing in the future. Even some medical detection units, which are very important for collecting human body information, could be incorporated. The next step is intelligence — I hope to bring this kind of intelligent service to society as a whole.

Si'ao Wang: I personally believe future artificial intelligence may no longer be limited to making decisions through current digital computing modes. We might adopt a dynamical systems approach. Like this pot of water — if we poke a hole here and keep adding water, when the water level exceeds the hole, water will spray out. Depending on the rate at which you add water and the size of the hole, the spray height and flow rate differ. This itself constitutes a computation. So we believe humans are actually this kind of analog system computation, not a digital system of 0s and 1s.

To achieve this, we started with emotional intelligence. We found that most embodied intelligence or AI on the market today does relatively utilitarian things, whether object recognition or spatial intelligence. I believe the fundamental driver of human evolution is survival fear — that we must stay alive. This desire for life further gives rise to love, curiosity, anxiety, joy, and other emotions.

If we can figure out emotions and intelligence — or perhaps this is the original driving force of biological evolution — then we just need to give the simplest embodied intelligence a set of instructions, and it can evolve on its own, without us needing to study such complex organisms as humans. In their own world, these intelligent agents can explore and develop their own functions. This is what we call "primordial motivation."

Our current entry point is electronic plants — plants that can move, with rich perceptual abilities like smell, vision, and touch. Their function is emotional intelligence, helping humans observe their own emotions. We find that people are often slow to recognize their own emotional reactions. With such an emotional system, it can predict your emotional changes in advance and help you adjust your mental state, thereby maintaining mental health.

Rui Chen: I believe vision, touch, and hearing — whether for humans or machines — are probably the two modalities that acquire the most information.

We ultimately hope to build a machine that can replace humans in performing operations in the real physical world. We expect such robots to complete not just simple tasks, but also handle fine manipulation — packaging, assembly, and even more complex work like polishing and gluing on production lines. Especially for robots, they need interactive data, not just simple data that can be labeled through vision. The data robots need to deliver is very difficult to collect at scale in the real world. We've been committed to achieving data generated from simulation that can ultimately transfer to the real world, so we've done extensive 3D visual simulation and tactile simulation work.

Last year, based on our newly developed tactile sensor platform, we organized the first logical mathematical manipulation challenge, with about 20 teams participating. We plan to continue holding such events, and hope to thereby promote the development of tactile robotics technology, making more and more people aware of the importance of touch in robotics.

Dawei He: Our company's main product is flexible sensors, specifically piezoelectric pressure sensors. In material selection, we paid particular attention to a material called PVDF (polyvinylidene fluoride). The reason for choosing this material was my consideration that if we develop electronic skin in the future, we'll need a material capable of multimodal fusion. Like our skin, it needs to perceive not just pressure, but also lateral and longitudinal pressure, even temperature and other sensations — ideally integrating multiple sensors into electronic skin. This material can monitor not just pressure, vibration, and noise, but also measure temperature, even capture posture, providing possibilities for applications like ultrasound evaluation.

I was initially exposed to first-generation foreign technology, which had relatively low operating temperatures of only 60 degrees and large product sizes unsuitable for fine manipulation. If PVDF material is made into arrays with smaller dimensions, it can achieve local perception, local planning, and local forming. Such designs are not only lower cost but also more suitable for collaboration with other manufacturers, promoting technology commercialization and application.

Gengchao Xiong: Touch itself is a typical multimodal perception, far more than just pressure — it includes sliding, humidity, temperature, vibration, and of course pain. From our company's founding, we've upheld a philosophy: the world is joyful because of perception. We hope to change both large and small user experiences through haptic technology, and further drive overall product iteration and upgrading. Our core technology, nano-interface capacitive sensing, is not just a new technical route and direction, but also a new material system and microstructure system. We have both ionic gel systems and polyionic systems, plus entirely new microstructure designs. Super materials plus super structures yield a series of multimodal sensors with super performance, covering pressure, sliding, temperature, vibration, humidity, and stretching. So our sensors have some advantages in touch applications requiring multimodality: we can achieve almost all tactile perceptions except pain. We're flexible sensors, can do single-point sensors, and are very suitable for multi-point and array sensors. Through compositing and weaving, we can achieve multimodal fusion, realizing multi-sensor integration both longitudinally and laterally.

Fred Yang: What we've always cared about most is how various modalities fuse together. For me, whether for humans or robots, inter-modal fusion actually provides two things: supplementary information and cross-validation.

Our view of touch is that it's fundamentally a baseline issue. If we have demand for relatively precise, dexterous manipulation, then it's a huge opportunity. As for the technology itself, it actually doesn't matter at all, because no single existing technology can satisfy our actual needs. This means no single technology can achieve perception of all vibrations. For example, I have a pen and a piece of paper in my pocket now, and I want to take them out, but I absolutely can't stick my eyes into my pocket to see — this is when humans rely on touch. When we do daily activities like organizing drawers, without touch's help, trying to precisely complete these tasks relying solely on vision would be very difficult, even impossible.

How Big an Opportunity Is This, Really?

Gengchao Xiong: I've never worried about the market, because the rapid development of flexible electronics technology has already brought significant growth. According to projections, the global flexible electronics market has reached an astonishing CAGR of 144.71%, and this growth has continued for years. The sensor industry itself is a massive market, with global scale of roughly several trillion. Among this, pressure sensors alone — the so-called most important part of touch — account for about 20% market share, so the tactile sensor market is no small market. When sensor performance surpasses traditional standards while improving in size, power consumption, accuracy, and sensitivity, this small step forward in technology often brings huge leaps for entire industries.

Take automotive electronics as an example. The technology of "automotive electronic skin" combined with cars' "Sentry Mode" is very useful. Activating Sentry Mode requires surround-view cameras, which need relatively high computing power and power consumption — after enabling this function, vehicle power consumption increases significantly. However, by introducing tactile sensors, it only triggers when people or objects approach and contact the car, both saving energy and improving efficiency. This innovation might transform functions previously seen as inconvenient or optional into industry standard configurations. Going further, it can also be applied to avoiding collisions with children or other objects in blind spots, triggering response through gentle touch rather than strong impact, thereby improving safety.

This is what I mean — a small step in technology, a giant leap in user experience. As these experiences gradually mature, I believe this is exactly what consumer electronics companies, automotive electronics companies, and others are currently seeking in terms of value — no longer blindly competing on so-called raw material prices, shipping prices, or supply chain services at the same price point. Everything returns to customer value creation. This is the big opportunity I see in this market.

What we call general-purpose, in my view, is a higher-level extension of the collaborative concept. They're not specifically for doing past industrial mass production, but through collaboration, through the generality of humanoid robots, to achieve more flexible expansion and extension. Tasks that robots previously struggled with in industrial mass production, like screwing or inserting memory sticks — work that assembly lines couldn't complete — can now be achieved through human labor combined with haptic technology solutions. This will let the entire industry continue moving forward.

Si'ao Wang: If there's demand for relatively precise, dexterous manipulation, then it's a huge opportunity. General-purpose touch will have very high technical difficulty and significant risk — it's a hammer-looking-for-nails situation. We still need to fuse multiple solutions, combining with humanoid robots to jointly develop a new type of touch.

What Are the Obvious Limitations of Touch? And What Does the Future of the Technology Look Like?

Dawei He: Different technical routes each have different limitations, so the future may involve fitting multiple technical routes together.

Rui Chen: Different technical routes will very likely form complementary relationships. Humans have multiple sensing mechanisms, and tactile resolution varies greatly across different body parts. For example, fingertips and tongues have extremely high tactile resolution, while areas like the back have relatively low resolution.

For robots, the situation is similar. In places requiring precise operation, high-resolution sensors can be used to achieve accurate perception. In other areas, low-resolution sensors can be adopted, meeting basic perception needs while reducing bandwidth requirements. Especially for commercial or household robots, safety is paramount, so sensors need to be placed across all body parts, but different resolution sensors can be chosen according to needs.

Overall, for large humanoid robots, different body regions may require sensors with different technical routes and parameters. This approach can meet various task needs while balancing performance and cost.

In 2001, Steven Spielberg's A.I. Artificial Intelligence was released, exploring robots' longing for human emotion. Touch may be just one door opening robots' exploration and perception of the world. Perhaps once robots possess human-like perception, their emotions too will awaken.

Going forward, we will continue hosting robot sharing events on different topics. We will post event notices on our WeChat official account, looking forward to every imaginative idea being fully displayed and exchanged here, sparking creativity together at Oasis.

Appreciating Vitality.