Say Goodbye to Cryptic Code: How to Build a Robot That Works Without Programming | X Lab
Why is it so hard to get a robot to grab something?
Are Industrial Robots Smart Enough Yet?
Robots can do almost anything we want — as long as we tell them precisely what action to perform and how to do it. Every movement depends on meticulous engineering. Throw something different into the mix — say, a tangled bundle of irregularly shaped wires — and they're stumped.
AI has made significant strides in "perception" and "control," but progress in "decision-making" remains limited. Projects that innovate on decision-making algorithms and can assist or replace human labor in complex decision-making systems are still rare.
In Silicon Valley, three young Chinese PhDs have made notable headway on this front. They spent a year and a half at OpenAI, the AI nonprofit co-founded by Elon Musk and other Silicon Valley heavyweights, working alongside top AI researcher Pieter Abbeel. In September, they left OpenAI to found Embodied Intelligence, with the goal of enabling robots to work like humans without programming.
This article traces their journey from leaving OpenAI to building Embodied Intelligence. The intelligence module they've developed can plug into any commercially available robot. Using a VR headset, a person simply demonstrates the task; the robot continuously tracks and learns the hand's movement trajectory, then refines the action through reinforcement learning to make it more efficient.
Whether measured against China's manufacturing needs or the growth trajectory of robotics companies FreeS Fund has previously backed, this type of project holds tremendous application potential in the Chinese market.
This month, Embodied Intelligence announced $7 million in seed funding. The round was led by Amplify Partners, with participation from FreeS Fund, Lux Capital, SV Angels, 11.2 Capital, and A.Capital. FreeS Fund was the sole Chinese investor involved.
Long before founding team members Peter Chen and Rocky Duan began their PhDs, FreeS Fund founding partner Feng Li had discussed startup ideas with them. Beyond familiarity with the team, they fit FreeS Fund's consistent pattern in cross-border investing — Chinese founders, starting in the US to tackle technical challenges and build influence, then leveraging industry resources in China to bring technology and products to the country's massive application scenarios.
In AI, chips, smart sensors, and intelligent manufacturing, FreeS Fund continues to actively seek investment opportunities in the following areas:
- Sensor and chip innovations around smartphones and smart cars — industries with sufficient scale — as well as innovative applications in IoT and 5G communications;
- Algorithms and innovative applications with clear technical advantages and defined use cases in domains where AI can significantly improve efficiency and where data volume and liquidity are high;
- Emerging technical directions, including AI-related heterogeneous computing and edge computing;
- Innovative applications based on decision-making algorithms that can assist or replace human labor in complex decision-making systems.
Former OpenAI Scientists Quit to Found Intelligent Robotics Company. We Sat Down With Their CEO.
Source | Synced (WeChat ID: almosthuman2017)
Written by Tony Peng
Edited by Liu Yan
China is among the most aggressive adopters of industrial robots. By procurement volume, it became the world's largest buyer in 2013, with 27% growth in 2016. Outside China, the rest of the world sold some 300,000 units, with South Korea, Japan, the US, and Germany as other key markets. According to the International Federation of Robotics, industrial robot sales hit a record $13.1 billion in 2016, up 18% year over year.
On the supply side, robotics remains a hotbed for entrepreneurship. Global robotics startups saw 174 funding deals in 2016, up about 18% from 147 in 2015. Roughly 48% of these deals went to enterprise robotics companies focused on heavy industry and manufacturing. Increasingly, robotics startups are looking beyond their home markets toward global opportunities.
In September 2017, Silicon Valley startup Embodied Intelligence launched. The company drew attention when Pieter Abbeel, a prominent figure in reinforcement learning, resigned from OpenAI to co-found it. Beyond its strong technical credentials, we also noted the Chinese background of its founding team and the inclusion of Chinese investor FreeS Fund in its seed round. It's clear that China will be a target market for the company going forward. "We're actively looking for partners and application scenarios in China," said CEO Peter Chen.
▲ The Embodied Intelligence team, from left: Peter Chen (CEO), Pieter Abbeel (President and Chief Scientist), Rocky Duan (CTO), Tianhao Zhang (Research Scientist).
In early 2016, Pieter Abbeel, a professor at UC Berkeley and leading figure in robot learning, brought his two PhD students Peter Chen and Rocky Duan to OpenAI, the nonprofit AI research organization based in San Francisco. Over the past year and a half, the trio contributed to 40% of the organization's published papers, making substantial advances in reinforcement learning and generative models.
Looking back, Chen reflected that his biggest takeaway was "ambition" — then paused and corrected himself: "Actually, perspective. Compared to academia, OpenAI is different. The research there expands your vision, makes you wonder whether you can step outside narrow problems and think about longer-term goals."
Chen gradually clarified what he wanted to achieve and how to get there. In September 2017, Abbeel, Chen, and Duan left OpenAI together, bringing on former Microsoft researcher Tianhao Zhang to found Embodied Intelligence, a startup dedicated to intelligent robot apprentices.
To outsiders, robots might just be assembled hardware repeating simple motions, but Embodied Intelligence aims to give them rapid learning capabilities — acquiring tasks through algorithms and applying automation to factories, warehouses, and even homes worldwide.
"Traditional robot programming is extremely time-consuming and only specialists can do it," said Abbeel, the company's president and chief scientist. "Our intelligence module can plug into any commercially available robot, enabling them to naturally learn new skills without writing obscure code."
▲ With the intelligence module installed, a robotic arm learns from humans to grasp deformable objects — no obscure coding required.
At its founding, Embodied Intelligence was based out of UC Berkeley's robotics lab, where Chen had spent most of his PhD years. Besides other researchers, the lab housed a robot named Bred/Brett. Berkeley students had invested considerable effort in teaching it to pick up objects, assemble things, and manipulate physical items.
Just two months after its founding, Embodied Intelligence raised $7 million in seed funding from Silicon Valley VC Amplify Partners and other investors, including Chinese venture capital firm FreeS Fund.
In early November, the company moved from Berkeley to nearby Emeryville. Synced visited their new office and spoke with CEO Peter Chen about his journey from leaving OpenAI to running a company.
Chen was clearly busy — during the interview he couldn't fully step away from his work, periodically checking code on his laptop and messages on his phone. But he said he hasn't felt a dramatic role shift from Berkeley PhD student to OpenAI researcher to CEO. "We're still working on a research project. It's just that to make it real, you need to build a company."
Why Is It So Hard to Get Robots to Grasp Things?
The pain points of traditional industrial robots are precisely the opportunity for every intelligent robotics startup.
Current industrial robot hardware is beyond reproach. Most robots on the market achieve precision within hundredths of a millimeter, and a mechanical arm priced at three to five hundred thousand yuan will last at least five to ten years. Robots can perform batch, repetitive motions and assembly on production lines, meeting the high-volume demands of rigid manufacturing.
But every action of a traditional robot requires meticulous programming by engineers behind the scenes. From picking up a component to fitting it onto another, programming must account for conditions like distance and angle. Many actions that seem trivial to humans pose major challenges for robots.
Take fitting two gears together: a human can simply twist and adjust the angle to mesh them easily. A robot doesn't work that way. Using traditional mechanical methods, engineers need to precisely measure the angular difference between the two gears, then rotate back and forth to achieve perfect alignment — immediately turning this into a difficult engineering problem.
▲ The high-precision operations of traditional industrial robots rest on meticulous engineering.
"Any behavior that requires looking with your eyes and then adapting with your hands is a challenge for traditional robots," Chen said. Fewer young people want to perform these dull, repetitive tasks in factories or warehouses, yet traditional robots cannot adapt to flexible manufacturing with constantly changing operations. What Chen wants to achieve in the near term is robots that can assemble gears one moment and equip other components or perform different actions the next.
There are many possible approaches to intelligent robotics. This year, robot companies have paid attention to Self-Supervised Imitation proposed by Google, which uses Time-Contrastive Networks to enable robots to learn actions from videos on their own, such as pouring water or standing up.
Chen is pinning his hopes on deep reinforcement learning. This machine learning method, which acts based on environment to maximize reward, has successfully taught computers to play Go and video games over the past two years.
In 2015, Chen's former lab at UC Berkeley had already used deep reinforcement learning to give robots a capability resembling visual motor skills. Humans perform many actions without conscious thought. Before drinking water, for instance, you don't mentally sketch out a plan of picking up the cup — adjusting its angle — pouring water into your mouth; you simply grab and drink by reflex.
The process isn't complicated: deep neural networks, like the human brain, can process visual data to give machines simulated human reflexes. After simulation comes the training method. Using deep reinforcement learning, robots find the "knack" for learning an action through continuous trial and error with reward mechanisms. This learning process doesn't require hand-holding programming.
But the method from two years ago remained theoretical. Deep reinforcement learning for robots requires guided setup rather than aimless trial and error, demanding people with reinforcement learning expertise (mainly PhD students) for upfront programming and debugging. A university lab can have a few PhDs working around the clock, but this clearly doesn't scale to industry.
It took two more years before Chen's team found a deployable solution: imitation learning.
Using VR Devices to Learn Human Motions
The emergence of consumer-grade VR devices has been a pleasant surprise for many AI companies. Embodied Intelligence uses VR to enable robot imitation learning. According to Chen, the core imitation learning research came from co-founder Tianhao Zhang.
Anyone who puts on a VR headset and picks up the controllers is effectively teleoperating a robot in real time. The robot simply tracks the hand's movement trajectory through the VR device in real time; the demonstration data in VR is then used to train deep neural networks. Throughout this process, the robot keeps learning until it signals: "I've got it. I can take over now." The entire process takes just 30 minutes. Even if you multiplied that 30 minutes by 100, Chen said, it would still cost far less than traditional robot training.
This method has excellent transferability or applicability. Even for different actions, the underlying code is identical — the VR code, the demonstration collection code, the training code, and the neural network learning code are all the same. The only difference is the demonstrated action itself.
Currently, the longest action Embodied Intelligence's robots have learned is nearly a minute. There's no strict definition of what can be learned; most actions humans perform by reflex can be taught. It's unclear whether robots could learn unconventional motions like the backflip Boston Dynamics' Atlas pulled off last week.
After completing imitation learning, robots still need reinforcement learning to continue self-improvement. Human motions aren't always optimal — walking in an arc rather than a straight line, or slight tremors while walking. For a robot, reinforcement learning can make actions more efficient.
"Recent breakthroughs in AI have enabled robots to learn to walk, manipulate objects through trial-and-error learning, and acquire new skills from demonstration data collected in VR," said Sunil Dhaliwal, partner at Amplify Partners. "But these advances have been confined to simulation or lab environments. The Embodied Intelligence team has pushed progress forward, and now they're bringing the cutting edge of AI and robotics into concrete application scenarios."
According to Chen, this imitation learning and reinforcement learning-based robot application will enter industry next year. Through this pipeline, robots can easily learn a broad range of skills, especially in areas where traditional solutions falter — manipulating deformable objects like wires, filaments, fabrics, garments, liquid packaging, and food; sorting and organizing items in irregular, cluttered environments; and complex assembly tasks. As material irregularity increases and personalized orders grow, the limitations of traditional hard-coded approaches become especially apparent.
But Embodied Intelligence's vision extends further.
Making Robots as Teachable as Humans
"Imitation learning + reinforcement learning" is the intelligent solution Embodied Intelligence can offer in the near term; meta learning is the company's core focus looking five to ten years ahead. "The key difference between robots and human labor is the cost of 'teaching.' We want to make robots as teachable as humans. Right now, we're just reducing the teaching time."
What is meta learning? Simply put, it's teaching robots a learning strategy. Traditional deep learning obtains results from massive data input, while meta learning derives methods for learning classes of tasks from small amounts of data. In other words, this represents artificial general intelligence embodied in robots — which is why Chen named the company Embodied Intelligence.
Chen has already produced research results in meta learning. This year, he co-authored two meta learning papers on arXiv with his advisor. Meta Learning Shared Hierarchies investigated a meta learning approach for learning hierarchically structured policies, improving sample efficiency on unseen tasks through shared primitives. Meta-Learning with Temporal Convolutions proposed a simple and general class of meta-learner architectures based on temporal convolutions — domain-agnostic, with no specific policies or algorithms encoded.
In Chen's vision, the ultimate intelligent robot Embodied Intelligence aims to build will possess both meta learning and reinforcement learning capabilities. "Reinforcement learning performs well on single tasks; meta learning gives robots the ability to learn quickly. These two can complement each other in the future."
The pressure on Chen in Silicon Valley is considerable. Beyond tech giants like Google pushing into intelligent robotics, numerous startups share this ambition. Just a 20-minute drive from Emeryville, in Union City, another robotics company Vicarious.ai is working tirelessly toward human-level machine intelligence. The company's CTO, Dileep George, told Synced they've chosen a completely different path from deep learning and reinforcement learning — simulating the human visual cortex combined with generative models to enable machines to model object structure and gain understanding of objects themselves. They expect to achieve this around 2040.
No one can be certain which path leads to AI's holy grail. But for now, Embodied Intelligence — armed with the latest research from UC Berkeley and OpenAI — stands at the forefront of robotics research.
(Feel free to share to Moments. Source: Synced. Click "Read Original" to view the original article.)
We Studied Anti-Aging in the Tianzhou-1 Spacecraft | X Lab
What Does Your Commute Have to Do With Data Monetization? | X Lab
