AI Agent, a New Form of Life | Agent Insights
Counselor Vitality
Life is a river.
It is neither centralized logical reasoning nor a dramatic leap forward. It is a continuous process of evolution — a river.
The same holds true for Agent. In early 2023, as we delved deeper into the development of LLMs, we came to believe: "The core of this AI revolution lies in Agent. Large language models are merely one piece of infrastructure; ultimately, generalization will happen through Agent." ("The Most Aggressive AI Investor")
But what exactly is Agent? And how can it evolve continuously, like life itself?
With these questions in mind, our team spent the past six months interviewing 22 professors and researchers worldwide working on Agent research — senior professors from top universities at home and abroad, as well as technical experts who have spent years in industry. All these leading scholars formed a network, generating emergent wisdom about the life of Agent.
Today, we invite you to revisit these fragments of wisdom. We hope they inspire you. Enjoy.
Part I
GPT-Driven Agent
We have always believed that AI is not merely a technological revolution, but a profound social transformation. "Technology" is merely a spokesperson — just as in 1800, no one today remembers the model or specifications of the steam engine, yet we live with the consequences of the Industrial Revolution every day. The same is true for AI. Behind these changes lie disruptive transformations in the structure of human society, organizational forms, personal life, culture, values, and more.
Take AI's impact on programming as an example. Globally, roughly 15% of programming tasks can already be handled by AI, and that share is expected to rise to 25–30% by the end of this year. Within three years, it could reach 60–70%. For the roughly 50 to 60 million programmers worldwide, the industry is in for a dramatic reshaping.
As traditional programming tasks are increasingly automated, some programmers will become "super-individuals." In this process, future programmers may no longer be simple coders, but rather appliers and integrators of AI technology. Their work will shift toward requirements analysis, design solutions, and managing the collaboration between AI and humans.
Some researchers have joked that starting next year, there will be no need to hire junior programmers anymore because their work can be replaced by AI. But without junior programmers, where would senior programmers come from? We joked back: does that make this year's graduating class of programmers the last of their kind?
This is not merely a trend in programming—it is a transformation of work patterns across all of society.
For the first time in human history, we possess the ability to infinitely replicate a person's aesthetics and knowledge through learning. Thus, large language models like GPT are no longer just software tools; they are a kind of capability. When combined with memory and guidance, this capability becomes an intelligent agent (Agent) for specialized domains. Agents can learn and absorb knowledge like students, apply it to real-world problems, and eventually evolve into experts that surpass human beings. This new mode of cognition challenges our traditional understanding of software and tools, and compels us to rethink AI's role in human society.
Early Agent systems relied on simple logical rules and preset behavioral patterns. These systems were typically used for automated tasks in specific scenarios—smart home controls based on preset rules, for instance, or simple game agents. At this stage, Agents often lacked adaptive capabilities; when environments changed, they could only adapt through manual rule adjustments. During this developmental phase, Agents primarily served an automation role, executing fixed tasks under specific conditions, but their ability to handle complex scenarios remained limited.
"In earlier years, the changes brought by deep learning in robotics were mainly concentrated in perception or natural language processing. Back then, we considered neural networks a black box. The advantage of generative AI for decision-making, planning, and control is that it does better with complex, high-degree-of-freedom robot planning and control—for example, walking tasks on uneven outdoor terrain, where, combined with reinforcement learning, it continuously explores the environment and learns superior strategies." — Peking University Professor Liu Chang: In Search of Aladdin's Lamp | Agent Insights
Peking University Professor Liu Chang has used generative AI to develop DOZE, an embodied intelligence dataset emphasizing Zero-Shot Object Navigation, better aiding embodied agents in movement training within virtual simulation environments.
Wang Xingyao, author of the 2024 ICLR Agent award-winning paper, believes:
"The essence of an Agent is a foundation built on LLMs, VLMs, or other large multimodal models, upon which multi-turn interactions with humans and the environment take place. I hope to offer the industry a conceptual approach: using code for deployment and tool invocation, directly leveraging existing code packages to save much of the extra work that would otherwise require human developers to predefine different tools." — Oasis Capital and Xingyao on His 2024 ICLR Agent Award-Winning Paper | Agent Insights
Wen Ying, a tenure-track associate professor at Shanghai Jiao Tong University's Institute of Artificial Intelligence and John Hopcroft Center for Computer Science, has built upon pre-LLM Agent research to deeply explore Agent research in the large model era, proposing many important cross-AI-era perspectives. In our interview, he noted:
"An Agent that learned to play Go would find it difficult to then learn StarCraft. This situation persisted until the LLM explosion at the end of 2022, when we discovered that large-scale data pre-training and fine-tuning could effectively improve model generalization capabilities. We then began research on 'generalist' Agents, including training decision (action) large models from scratch on massive decision trajectory data, or building language/multimodal agents by adding tool use, memory modules, and other components on top of pre-trained language or multimodal large models, and using reinforcement learning and other methods to let agents continuously interact with environments, other agents, and humans, learning continuously through various feedback signals." — Shanghai Jiao Tong University Professor Wen Ying: Building "Generalist" Agents | Agent Insights
With the emergence of the Transformer architecture, humanity has fundamentally found a new compression algorithm—one that compresses vast data into the prediction of the next token. In fact, human intelligence itself is a compression algorithm. We find that the more excellent a civilization, the more it tends to live in simple, unadorned ways. The closer a person is to the Dao, the more directly they grasp essence.
Just as a person who has lived a life of prosperity and hardship might, at life's end, leave their family with but one sentence: "Be a kind person."
This is compression. This is wisdom.
Thus, LLM-based Agents are essentially agents built upon this new compression algorithm, vastly expanding the cognitive and intellectual boundaries of what Agents can achieve.
Researcher Mao Hangyu noted in our conversation:
"From deep learning to the Transformer-era NLP Agent, we've seen astonishing changes—ordinary Agents can now accomplish numerous tasks. I felt something akin to the 2015 Deep Q-Network boom. The TPTU architecture gives Agents basic planning and tool-calling capabilities while enhancing their tool retrieval, semantic description, and case learning abilities." — SenseTime Researcher Mao Hangyu: Collaboration and Alignment Between Agents | Agent Insights
In the large model era, the powerful generalization capabilities provided by underlying GPT models enable what might be called world model understanding within scenarios. Beyond this, the extension of new Agent capabilities has continued to advance.
New York University Professor Feng Chen's current research addresses autonomous driving navigation pain points through Agent collaboration:
"At New York University, one new direction we're focusing on is Collaborative Perception—multiple Agents, each with their own sensors, communicating with each other to cooperatively understand scenes, with the expectation that the overall system becomes more efficient and safe. A concrete example: suppose you're driving and a large truck blocks your view, severely affecting your decision-making. If the truck's sensors could share additional information with you, you could make better decisions. This scenario is equivalent to being able to see through objects." — New York University Professor Feng Chen: Multi-Agent Collaborative Perception for Autonomous Driving | Agent Insights
Fudan University researcher Xu Yinghui and his team are developing a flexibly designed, continuously debuggable Agent development interface:
"Traditional LLM-based AI Agent operations platforms, while significantly lowering the barrier to user development, often struggle to meet the needs of professional developers, thereby constraining the flexibility of design, coding, and debugging and hindering the development of more complex AI Agents. AI2Apps, as the first visual integrated development environment (Virtual IDE) for LLM-based AI Agent applications, covers the complete development cycle from prototype design, code writing, Agent debugging, to final packaging and release, helping developers efficiently build AI Agents." — Fudan University AI³ Researcher Xu Yinghui: A Visual Creation Interface for Agents | Agent Insights
Liu Zhiwei, senior scientist at Salesforce AI Research, is closely following the future development of Multi-Agent systems:
"I believe a single Agent has limited capabilities. How to integrate various Agents into a Multi-Agent System to create a more powerful system is a very attractive architecture. Currently, Multi-Agent development is all based on the same LLM. In the future, will there be a platform allowing more people to participate in training their own Agents? The coordination and cooperation involved would be a very worthwhile research direction." — Salesforce AI Research Liu Zhiwei: Thinking Like an Agent | Agent Insights
All at once, machine learning, stagnant for years, seems to have pushed open a door, revealing to humanity the infinite possibilities of Agents as professionals.
"At first it was narrow, barely passable; after walking several dozen paces, suddenly it opened up to broad light."
Part Ⅱ
Re-Understanding Agent: Apprentices Through Observation
Today, many people discuss Agents' hallucination problems and cost issues. At first glance, Agents don't seem like good tools—ready to use out of the box, dissolving upon entry. The answer is obvious: Agents are not tools. Agents are apprentices. The problems Agents exhibit today are the same ones seen in recent graduates—auditory hallucinations, delusions, fantasies. This is precisely where excellent leadership proves its value. Outstanding teachers understand gradual progression, teaching according to aptitude, and setting examples—and Agents too will learn through observation. This is the value of reinforcement learning and SFT.
Thus, foundation models are more like general education, giving each model basic capabilities—we might call this "common sense." Alignment, then, is like establishing "role models" and "guidance" for an Agent's career development based on its general education, allowing it to gradually become an industry expert through professional training.
This is much like child development: subtle childhood experiences imperceptibly shape one's later character and behavior. This is the value of alignment. "Happy people use their childhood to heal a lifetime; unhappy people use a lifetime to heal their childhood"—this applies equally to LLM-based Agents. Therefore, we look forward to deeper understanding and careful attitudes toward large models, viewing GPT and similar AI systems with optimism, and perceiving the infinite possibilities contained in their development.
Harvard University PhD Zhu Kehang has used Agents to model human behavior, simulating roles such as judges, auctioneers, and product testers, demonstrating that large language models can achieve approximate simulation of human society:
"In the field of human behavior modeling, a single human brain contains tens of thousands of trillions of neurons. How these neurons generate neural signals and ultimately determine human emotions and behavior is itself a very complex system. Human societies composed of hundreds of millions of people are currently difficult to model well with limited mathematical or computational methods. The emergence of large language models has broken through this limitation. We use LLM-based Agents as approximate models of humans, proposing a general automated experimental method that can also achieve low-cost, efficient social experimental testing in traditional social science domains." — Harvard University Zhu Kehang: Automated Social Science Agents and Human Behavior Modeling | Agent Insights
The boundary of the apprentice model no longer depends on the number of users, but on teachable "experience." When a person possesses certain knowledge and skills that can be taught to this apprentice, their capabilities can be infinitely extended. In other words, theoretically, there could be thousands of "yourselves" transmitting knowledge in different domains. Ultimately, an Agent's limitations lie not in the number of individuals, but in the depth of cognition and knowledge that a person possesses.
So we believe that discovering and building one's "core capability" is the "right path" in the AI era.
Of course, Agent capability development will not happen overnight. It will undergo continuous growth from "assistant" to "self," eventually achieving "the student surpassing the master" after fully grasping the mentor's core capabilities.
Huazhong University of Science and Technology Professor Chen Wei participated in AI Hospital, bringing a fresh approach to hospital consultation systems—restructuring the consultation process to address the traditional problems of long wait times, cumbersome procedures, and low efficiency. The team also addressed the pain point that current large model deficiencies make interactive evaluation insufficient for medical diagnosis. AI Hospital designed rigorous dispute resolution mechanisms among multiple intern doctors, allowing multiple Agents to compare and discuss, ultimately integrating different diagnostic results from intern doctors, continuously adjusting and optimizing their diagnoses until consensus is reached. He explained:
"In the real world, doctors actively interact with patients through methods like observation, listening, inquiry, and palpation, gathering information, combining examination results, and then making diagnoses. Existing large models often lack this interactive capability, tending instead to provide complete, lengthy answers in one go. To overcome this challenge, we developed AI Hospital, a simulated real-world medical environment interactive evaluation framework. It includes multiple roles—patient, intern doctor, examiner, and attending physician—simulating multi-turn dialogue between doctor and patient, allowing intern doctors to gradually build understanding of conditions through questioning and collecting patient feedback." — Huazhong University of Science and Technology Professor Chen Wei: Grey's Agent | Agent Insights
In fact, humans have significant limitations. For example, the visible light we can see occupies an extremely narrow wavelength range, far less than the entire electromagnetic spectrum. Furthermore, the worlds perceived by flies and dragonflies are completely different from ours. Such perceptual limitations constrain human information within certain bounds.
Moreover, "discrimination" and "attachment" are two inescapable traits that further limit the number of top-tier professionals.
However, as Agent capabilities continue to improve, evolving from assistant to doctor, that doctor's "service" capacity becomes unlimited.
Thus, as apprentices, Agents are assisting humanity in entering an era of "super creators." In this era, every person can become a super creator. Humans should no longer focus merely on compensating for their shortcomings, but rather on excavating and developing their unique core cognition, fully leveraging AI's capabilities to amplify these advantages.
Alex Sheng, who began studying machine learning in high school, was inspired by renowned German computer scientist Jürgen Schmidhuber to develop Agents capable of self-evolution in the coding domain. He expressed hope to see Agents that can assist researchers in their work, making humans "smarter."
— New York University Alex Sheng: The Self-Iterating Path of AI Programmer Devin | Agent Insights
This enhancement of capabilities means that in the future, humans across all domains—language, emotion, philosophy, and industry—will achieve qualitative leaps with AI's assistance. Every cognition worth taking pride in will continue to shine in this new era, further driving societal progress. AI will lead humanity into a new stage, advancing toward a future of human-machine co-creation.
At this stage, we can already see prototype Agent products emerging. Stanford University Department of Psychology research scientist Zhao Xuan developed an AI psychological healing Agent—Sunnie—which possesses multi-turn natural dialogue capabilities and can recommend personalized activities based on large language models. She noted there remains much room for imagination:
"We've recently been looking at how voice can enhance psychology Agents' capabilities. Hand-free interaction methods can currently achieve latency under 500 milliseconds. We design the framework for psychology Agent Sunnie based on our understanding of user psychology." — Stanford University Zhao Xuan: Psychological Healing Agent | Agent Insights
Going further, when each super creator trains their core capabilities into Agents, and these Agents combine into a more complete whole, society will naturally become a Great Harmony society of "all for one, one for all."
Part Ⅲ
Agent Application Exploration and Entrepreneurial Opportunities
Over the past year, the Oasis Capital team has gained insights into current market demands and application scenarios for AI Agents through in-depth conversations with scholars and senior researchers across different fields. These scholars also shared some perspectives that we hope will inspire you.
Globalization
In the past, globalization emerged as a concept precisely because human languages, cultures, and customs divided different regions from one another.
Jindong Wang from Microsoft Research Asia introduced a simple and practical framework for cultural distinction that addresses the pain points of existing approaches to LLM cultural bias. Microsoft Research Asia's Jindong Wang: The Cultural Park of Agent | Agent Insights
The team designed a multi-Agent communication framework called CulturePark for collecting cultural data and generating high-quality, diverse cross-cultural dialogues to fine-tune culture-specific LLMs. CulturePark operates on two levels: the first layer consists of host Agents, typically American Agents with English-language backgrounds, while the second layer comprises participant Agents from multiple national backgrounds. The two layers engage in debate. For how future Chinese large language models can better address cultural conflicts, Wang offered a clear and intriguing solution.
In the future, apprentices will naturally receive a globalized education — able to read books from around the world and understand all human emotions and experiences, including joy and sorrow, reunion and parting. So when discussing AI entrepreneurship today, one point deserves special attention: the apprentice you are cultivating is inherently a globalized apprentice, not one confined to any particular country or region. The only kind of apprentice that future technology pursues is the globalized apprentice.
Personalization
The customization capabilities of Agents will significantly enhance their flexibility and intelligence, enabling them to effectively adapt to evolving needs. Take education as an example: future Agents will employ tailored strategies that teach students according to their aptitude and individual differences, providing each student with a customized learning experience.
Researcher Kaitao Song, during his time at Microsoft Research Asia, launched HuggingGPT — an Agent foundational framework built on the Hugging Face community that enables Agents to solve complex AI tasks across multimodal and multi-domain applications, making customized learning experiences no longer a difficult problem.
"Our HuggingGPT design leverages ChatGPT's language capabilities to decompose user needs into objectives, then uses its own language analysis abilities to understand and select the most suitable models from HuggingFace's model descriptions for executing subtasks. HuggingGPT has application models in many niche domains, such as education, online shopping, and community interaction scenarios." Microsoft Research Asia's Kaitao Song: Embracing Agent | Agent Insights
Moreover, niche personalized needs will also be met.
Wu Zhiyong's team at Shanghai AI Lab introduced the OS-Copilot framework to create FRIDAY — a general-purpose computer Agent with self-improvement capabilities that is more dynamic and vital. Wu candidly noted, "FRIDAY is currently more suitable for niche needs, such as serving as an auxiliary tool for Excel, PowerPoint, and coding through adjusted Agent framework design. Traditional large RPA companies won't dedicate effort and resources to building tools for small demands. OS-Copilot provides a framework that ensures precision — users can directly generate an Agent that understands you and meets your personal needs. Looking at the bigger picture, OS-Copilot will gradually become an open-source OS standard protocol, or a widely accepted framework that helps developers save time on repetitive development." Shanghai AI Lab Researcher Wu Zhiyong: Tomorrow's FRIDAY | Agent Insights
Information Reshaping
In the mobile internet era, image-and-text sharing platforms partially replaced traditional search engines as traffic entry points. But if an Agent can not only help you find the product you need but also complete price comparison, fill in your address, and have it delivered within three days — all with one click — will step-by-step search engines still have a market?
The current reality is that within the e-commerce sector, Agents are beginning to replace supply chain procurement, enabling controllable upstream information comparison in closed information environments and completing multimodal information acquisition and comparison. This foreshadows that Agents will become more intelligent and personalized search entry points in the future.
Agents are no longer confined to specific scenarios; they can end-to-end satisfy users' personalized needs. They can automatically search, compare prices, and complete order generation and delivery. This one-stop, intelligent search experience will greatly enhance user efficiency and satisfaction, ultimately replacing the traditional step-by-step search model.
Professor Yu Su from Ohio State University developed the SeeAct framework. "After developing Mind2Web (a benchmark for evaluating computer Agents' web browsing capabilities), my team and I set out to build SeeAct on top of large multimodal models (LMMs), extending Web Agent capabilities to multimodal tasks such as visual question answering, web page manipulation, and online shopping. This addressed the significant limitations and low accuracy of Web Agents built purely on language models. SeeAct leverages multimodal large models like GPT-4V for web page visual perception, generates plans in text form, and executes operations by mapping text plans to HTML elements through oracle grounding." Ohio State University Professor Yu Su: See! Then Act | Agent Insights
At this moment, humanity stands at a critical point of transformation. We deeply feel that the present is a golden age for exploring AI Agents. We hope that every partner who believes in AI and feels curious about AI Agents can view the Agent as a new form of life — understanding its potential and limitations, pioneering broader application scenarios, and experiencing the new vitality it brings.
We look forward even more to exploring the infinite possibilities of Agents as new life forms, together with all of you.
Because,
Life is a river.
Special Thanks
Listed in chronological order of interview publication
Hao Liu, Assistant Professor and PhD Advisor
Kaitao Song, Senior Researcher
Chang Liu, Assistant Professor, Researcher, and PhD Advisor
Zhongyu Wei, Associate Professor
Tongxin Li, Assistant Professor
Jindong Wang, Principal Researcher
Chen Feng, Associate Professor
Haohan Wang, Assistant Professor
Ying Wen, Tenure-Track Associate Professor
