Hyperparameter's Yongsheng Liu: Exploring Evolutionary AI in Virtual Worlds
Build virtual worlds with different scenarios, and let AI learn and evolve within them.
Over the past few years, China's AI industry has made considerable strides, but mostly concentrated in the "perception" phase. As AI advances from "perception" toward "decision-making" and "creation," it becomes increasingly difficult to find standard answers. The industry is looking to an "evolutionary" approach to drive AI development—one that can handle more general tasks and possess genuine self-improvement capabilities.
In early 2019, Yongsheng Liu, former General Manager of Tencent AI Lab and T4-level technical expert, founded Hyperparameter. Gaorong Ventures invested in its Series A round. During his time at Tencent, Liu had led the core team in developing the Go AI "Jueyi" and the Honor of Kings AI "Juewu."
Since its founding, Hyperparameter has focused on the gaming sector. Drawing on leading capabilities in deep learning, reinforcement learning, and large-scale systems engineering, the company has been "using evolution to manifest intelligence" in virtual worlds. It has already delivered AI solutions across diverse game genres including 3D open worlds, 2D casual competitive games, and board and card games, generating commercial value.
Recently, Liu spoke at a live recruiting event co-hosted by Gaorong Ventures and its portfolio companies, titled "Where the Heart Leads, Gaorong Gathers Talent." He shared his understanding of AI's developmental stages and Hyperparameter's progress in exploring evolutionary AI within gaming's virtual worlds.
The following is Yongsheng Liu's presentation (edited for clarity):
"Just Finished the Multiple Choice, Ready for the Essay Questions": The Promise of Evolutionary AI
Over the past year, one question I've been asked with remarkable frequency is: What exactly is AI?
It's a very fundamental question. But in reality, both society and the industry currently disagree not only on how to define AI's developmental stages, but also on what AI actually encompasses.
Regarding developmental stages, much of the media and investor community focuses on strong AI—hoping that AI can make complex decisions like humans. But what's the reality? Right now, AI remains concentrated in the "perception" phase. This disconnect breeds a pattern of short-term optimism paired with long-term pessimism.
Another point of divergence concerns what AI technically covers. Today, AI seems to be everywhere; no matter the niche, teams can claim to possess AI technology. But industry insiders typically understand AI as referring to machine learning algorithms like deep learning or reinforcement learning.
If we try to map out AI's developmental stages, they can be divided into three phases: perception, decision-making, and creation. We've just transitioned from "perception" to "decision-making." It's like taking an exam with multiple-choice questions, essay questions, and composition prompts—we've just finished the multiple choice and are preparing to tackle the essays.
Over the past few years, China's AI industry has achieved a great deal, but largely by scoring points on the easier multiple-choice section. In a sense, AI has been advanced through an "exam-oriented education" approach. By cramming with tutors and practice tests, you can make rapid breakthroughs, but it's hard to draw inferences about other cases from one instance. AI developed this way I call "patchwork" AI—it requires massive amounts of human-labeled data, only works for specific tasks, and cannot grow on its own.
But as we move on to essay questions and compositions, this method falls short. Unlike multiple-choice questions, these have no standard answers. Just as "quality-oriented education" identifies core competencies for human development and creates conditions to cultivate them—without fixating on which specific problems those competencies will solve—we hope to advance AI through an "evolutionary" approach. By creating certain conditions and focusing on a few core capabilities, the main advantage is that we don't need to exhaustively enumerate and individually solve every problem. Instead, we can accomplish general tasks and enable self-improvement.
Of course, evolutionary AI has its drawbacks. As we all know, quality-oriented education actually costs more. Evolutionary AI demands greater computing power and places extremely high demands on infrastructure and algorithms, because it must process large amounts of unstructured data.
Virtual Worlds as the Ideal Proving Ground for Evolutionary AI
Using an "evolutionary" approach to advance AI requires building a simulated environment that can generate rich interactions and data. And virtual worlds, with games as their foremost representative, are precisely the ideal proving ground for AI to achieve evolutionary intelligence.
In recent years, both AI companies and game studios have joined the race in game AI research, and many major advances in artificial intelligence have come through integration with games. These include DeepMind's Atari game AI in 2013; DeepMind's AlphaGo and Tencent's Go AI "Jueyi" in 2016-2017; and more recently, DeepMind's AlphaStar and OpenAI's OpenAI Five.
Hyperparameter was founded in 2019. We are attempting to enable AI to learn and evolve by constructing virtual worlds for different scenarios. We've defined several fundamental capabilities for AI:
First, survival in complex environments. Just as humans need to obtain food and avoid danger to survive in this world.
Second, multi-agent collaboration and competition. The ability to cooperate with others to confront enemies together is also critically important.
Third, imperfect-information games. In real life, people must make many decisions with incomplete information. Previous systems like AlphaGo playing Go operated under perfect information. Imperfect-information games often have no unique solution, with extremely high complexity that is difficult to address through human-labeled data.
Fourth, content creation and comprehension. That is, enabling AI to generate new information and content like humans do.
Fifth, large-scale intelligent ecosystems. How large numbers of intelligent agents form ecosystems and continuously evolve forward. Just as human society progressed from primitive society to slave society, agricultural society, and then industrial society—the wheel of evolution rolls ever forward.
For all five capabilities above, we have already launched collaborations with partners or pursued independent R&D, distributed across games of different genres, stages, and demand scenarios. Specifically:
1. Survival in Complex Environments — 3D Survival Competitive AI "Orion α"
For survival in complex environments, we partnered with Kingsoft Games to create an AI named "Orion α," which launched in the upcoming open-world survival competitive game The Sunken Century.
Some might ask: In shooting games, doesn't AI just "cheat"? The reality is quite different. During gameplay, the information we feed to AI is identical to what human players receive, and the available actions are the same too. AI and human players compete under completely fair conditions. Currently, without relying on any human data and based entirely on reinforcement learning from scratch, "Orion α" has developed capabilities including complex 3D environmental perception, resource searching and usage, multi-weapon combat, and team coordination. It can achieve multiple skill levels and stylistic characteristics. Its in-game abilities are more proficient than most human players, and its movements are remarkably human-like—to the point where even game designers cannot distinguish it from human players.
2. Multi-Agent Collaboration and Competition — Casual Competitive AI
In multi-agent collaboration and competition scenarios, we partnered with Giant Network to train AI with capabilities in team coordination, competitive confrontation, and long-term planning within the game Battle of Balls. Battle of Balls is a quintessential multi-agent collaborative competitive game, with 10 teams competing in each match and 5 players per team, requiring extensive coordination among players. Our AI has achieved relatively high standards in both competitiveness and human-likeness. A portion of users can already experience competing against our AI players.
In smart cities, transportation, engineering, and other domains, multiple groups must collaborate and compete to seek globally optimal solutions. Our multi-agent collaborative competitive AI has considerable application value in many real-world scenarios.
3. Imperfect-Information Games — Tabletop AI Mini-Game
For imperfect-information game capabilities, we are conducting research in a self-developed tabletop AI mini-game called Your Turn: Revelation. The gameplay resembles Werewolf/Mafia: five players survive five nights together, making identity deductions and decisions based on limited information each night. Although the gameplay appears simple, the underlying AI implementation is quite complex. The AI must learn not only to cooperate without direct communication, but also whom to cooperate with and whom to oppose, as well as advanced strategies like deception and disguise, plus complex reasoning abilities.
With large numbers of "virtual players" that behave human-like, match target player skill levels, and possess diverse styles, this game rapidly broke through the cold-start problem after launch, was selected as a WeChat Creative Mini-Game, accumulated millions of users, and enjoys extremely high core player retention.
Beyond these, we've pursued additional explorations, including testing AI's content creation capabilities in text adventure games and experimenting with large-scale intelligent agent ecosystems in self-developed simulation environments. These explorations don't yet have versions available for external experience, but their prospects are well worth anticipating.
From Game AI Commercialization to the Quest for Artificial General Intelligence
Going forward, we hope AI will create value in three dimensions. First, using AI technology to bring efficiency gains and gameplay innovation to games, exploring alongside game studios to generate genuine commercial value.
Second, leveraging gaming and other virtual environments to finally give people the opportunity to train and test various intelligent agent behaviors in sufficiently realistic settings, enabling evolutionary AI development and helping AI advance toward artificial general intelligence.
Finally, once AI reaches sufficient levels of intelligence and generality, we are confident in empowering AI technology across industries in the real world, feeding back into more real-life scenarios and generating greater societal value.
Hyperparameter also looks forward to joining with more like-minded explorers to "use evolution to manifest intelligence, and let intelligence benefit humanity."
Wanli Min's Cunjue Technology raises over $20 million in Series A led by Gaorong Ventures
