Salesforce AI Research Zhiwei Liu: Thinking Like an Agent | Agent Insights

A standard protocol for Agents could spare developers a lot of redundant work. AgentLite is one such starting point — focused on unifying development standards for Agent and Multi-Agent systems from a scientific research perspective, so that academia's wild ideas can iterate faster. We asked Zhiwei Liu, Senior Research Scientist at Salesforce AI Research, to tell us how he put this library on a diet. Enjoy.

AgentLite: A Lightweight Library for Building and Advancing Task-Oriented LLM Agent System

Problem it solves: Existing LLM-based Agent libraries lack a unified, hackable underlying architecture. AgentLite provides a platform where researchers can modify code arbitrarily for specialized problems, making it easier to develop Agents for specific application scenarios.

Model architecture: AgentLite has two layers — Manager Agent and Individual Agent. The Manager Agent creates subtasks from a given instruction and distributes them. The Individual Agent generates prompts through the PromptGen module, sends them to the LLM for execution, and outputs the results.

Application scenarios: Agent Library. Developers can build any type of Agent on the AgentLite platform — image understanding, math problem solving, chess, philosopher chat, and more.

Features supported by AgentLite compared to other Agent-building libraries

Test results: AgentLite works with any LLM and demonstrates strong effectiveness in handling complex tasks on the HotPotQA dataset.

AgentLite performance on the HotPotQA dataset

Oasis Capital: What motivated you to develop AgentLite?

Dr. Liu: We formally dove into AI Agent architecture research around June last year. Our first project was BOLAA, focused on evaluation standards for Agent architectures. While working on BOLAA, we noticed that popular open-source frameworks like MetaGPT and LangChain were all geared toward helping developers build better applications — but there was no standard for academic research. Designing new architectures or changing reasoning approaches requires modifying low-level Agent architecture code. With current open-source frameworks, you need to spend enormous amounts of time learning the underlying inference and architecture, which is unnecessary overhead for academic researchers.

To address this pain point, we wanted to build a platform where you could modify code arbitrarily for specialized problems, making it easier to develop Agents for specific application scenarios. That's AgentLite. We want researchers to be able to participate in developing architectures, helping everyone test and iterate faster, and study Agent underlying architecture and reasoning approaches more deeply.

AgentLite is an open-source library. We hope to unify development standards from a scientific research perspective — for instance, combining ReAct with Reflection into a unified Reflection Agent. The end goal is to create a general protocol for academic research Agents, so that people can design their own Agents based on this protocol. This includes Multi-Agent protocols too — for example, what interfaces Multi-Agents should use to communicate and collaborate, and what tasks they ultimately complete together.

Oasis Capital: Could you explain the PromptGen and ThinkAct modules in AgentLite?

Dr. Liu: AgentLite's design is based on a Multi-Agent architecture. The Multi-Agent subcomponents need Individual Agents to complete tasks, and while doing so, summarize the results and return them to the Manager Agent. The Manager Agent breaks large tasks into smaller ones and assigns them to Individual Agents.

Individual Agents only need to focus on their assigned tasks. The Task Package feature breaks down large tasks and wraps them into small, discrete ones — we just need to design the communication mechanism between Agents. Through this approach, you can build larger architectures, stacking layer upon layer. One Manager Agent can also serve as an Individual Agent for another Manager Agent, making the entire system easy to deploy.

Oasis Capital: So there is indeed a hierarchical relationship between them.

Dr. Liu: Yes. In AgentLite's design, when the Manager Agent receives a task, it must wait for execution to complete before generating the next task. Some open-source frameworks do this very well too — for example, MetaGPT can turn conversational interactions into asynchronous Multi-Agent mode, where generating a task doesn't require waiting for execution to finish before starting the next one. It can produce 10 or even 100 Task Packages simultaneously, with 100 different Individual Agents completing them. This enables large-scale parallel task processing.

Oasis Capital: From an academic development perspective, where is Multi-Agent research currently?

Dr. Liu: The key capability of Multi-Agent lies in how to plan, decompose, and distribute tasks. Current open-source frameworks fall into two categories. One type makes a plan after receiving a task, breaking it into different steps, generating a planning list where each item corresponds to completing a different subtask, then mapping those subtasks to different Agents. The other type, after receiving a task, first figures out what to do in step one, immediately starts executing, then distributes to corresponding Agents after completion. The Agents feed back results, and the next task is generated — many of these apply foundational architectures like ReAct and Reflection.

More complex Multi-Agent involves dependencies between different tasks — how to establish correlations between two tasks, and how to handle synchronous versus asynchronous tasks. These represent different design philosophies at the system level.

Oasis Capital: In the future, could Multi-Agent interaction become P2P, completely different from how humans collaborate?

Dr. Liu: That's possible, but P2P means Agents need to know latent information about the entire system — for example, when I communicate with you, I need to know what kind of person you are. AgentLite's hierarchical approach is more convenient because each Agent only focuses on the current task, making it easier to implement large-scale Multi-Agent systems.

In the future, as large models become more capable, we can provide richer contextual information. With stronger ability to understand diverse information, they could automatically find the right counterpart to dialogue with, achieving P2P interaction.

Of course, many problems remain to be solved — for instance, with P2P and very many Agents, how do you design the system? How do you get them to communicate? On what platform? These are all very complex questions.

Oasis Capital: How is the division of labor between Salesforce AI Research and the business teams? Will the next development plan integrate with the overall business direction?

Dr. Liu: Since we're B2B, the whole company wants to build more intelligent general-purpose platforms. We hope to provide Agents to help small companies without resources for independent development deploy solutions, developing Agents tailored to their business. Ultimately, regardless of company size, everyone should be able to converse with Agents to solve practical problems. Customers could even provide APIs for execution.

Salesforce AI Research focuses more on underlying standard protocols, thinking about the most effective reasoning architectures for different scenarios, and evolving new scenarios from there. Multi-Agent architecture is also very new — it involves planning and communication. We also research development standards. Just as PyTorch serves as a platform enabling different deep learning research, we want to become such a platform. We've also trained and open-sourced large language models specifically for Agent scenarios, called xLAM.

Salesforce AI Research expects to stay slightly ahead of product development. Products will follow the most mature architectures, like ReAct or Reflection, to directly develop simple and effective applications that quickly solve user pain points.

Oasis Capital: Does the Salesforce product team directly do secondary development of company products based on AI Research's R&D results?

Dr. Liu: The product team needs to pay more attention to API privacy issues — we don't want users to have any trust concerns about data within large models. This is something Salesforce particularly values. Many trust issues we consider during the model training phase may not cover all scenarios, so this requires collaboration between product and R&D departments.

Oasis Capital: Which applications do you think are deeply integrated with company business?

Dr. Liu: Those deeply integrated with company business lean more toward tool using. For example, we've designed applications for querying weather, movies, and so on based on Q&A tools; and responses based on given information plus external search, similar to RAG. I think whether B2B or B2C, using RAG as an Agent development platform is the most practical approach. RAG enables Agents to call APIs to solve real problems. Large models themselves can't access real-time information — they must obtain real-time context through API calls before generating responses, which also better satisfies personalized needs.

Oasis Capital: What's your view on the Agentic Workflow that Andrew Ng mentioned in his recent talk?

Dr. Liu: My understanding is that Agents are either fully automated, or fixed through a workflow. Open-source Agent platform underlying architectures tend to combine workflow with ReAct — some things are fixed, and after the workflow is designed, it's passed to the Agent. But if users aren't certain about some workflows, you need large model inference to make judgments, then let Agents communicate with each other to decide next actions. That might be a more general expression.

Oasis Capital: Is the recent open-source project DSPy similar to your research direction of wanting to combine with Agents?

Dr. Liu: Yes, projects like DSPy and Guidance hope to control generation results through prompts. We want APIs to output predictable API instructions even after inputting a set of parameters. DSPy can directly output desired generated content from instructions processed by large models. As Agents develop, people are finding DSPy also suitable for generating good Agent instruction controllers.

Users only need key parts of large model-generated content, and at this point need to use certain libraries or code from DSPy. We're more focused on Agent architecture itself. DSPy has some good AI inference approaches, and we'll connect some DSPy controllers under the Agent framework for users to develop Agents, strengthening the output capability of Agent architecture.

Oasis Capital: In this wave of AI development, how do you see the development and differences between academia and industry in AI research?

Dr. Liu: As large models grow increasingly massive, the industry is gradually becoming winner-take-all. Groups with resources and GPUs can build extremely large models, and larger-scale models naturally have much stronger capabilities than 7B-scale models. Although Salesforce AI Research is a corporate research organization, it's not completely tied to products. We also want to do some lightweight product development similar to what academia does, while combining it with the large models we develop to adapt to more scenarios.

Of course, from a product R&D perspective, companies care more about practical functionality and mature, reliable reasoning. For example, OpenAI wants to develop practical large model applications more, focusing on business deployment or scaling — they may neglect some large model underlying technologies. Academia's wild ideas can help industry generate new concepts, like product business models. The two complement each other.

Oasis Capital: What are the next Agent development points you're watching?

Dr. Liu: We're focused on two aspects: First, reasoning approaches and architecture design. ReAct and Reflection both imitate human reasoning logic, but different business scenarios require very different reasoning approaches. As machines, do they really need to think like humans before executing tasks? Are there reasoning approaches more suitable for Agents, rather than making Agents into humans? How to make Agents more intelligent is worth researching.

The other aspect is Multi-Agent. A single Agent has limited capabilities. How to integrate various Agents into a Multi-Agent System to become a more powerful system — I find this a very attractive architecture. Currently, Multi-Agent development is all based on the same LLM. In the future, will there be a platform where more people can participate in training their own Agents? The coordination and collaboration involved would be a very worthwhile research point.

Oasis Capital: By not making Agents more like humans but more like Agents, do you mean stripping away unnecessary processes from human thinking patterns so Agents can better assist humans?

Dr. Liu: In the long run, once large models become Agents, they can proactively interact with the world. World models or things like Sora already have their own understanding of how the world operates, human language, and behavior — do we still need to intervene in their thinking with human thought patterns? I don't think so.

OpenAI co-produced Figure 01, which is an embodied Agent. Once an Agent acts proactively and understands proactively, that means it also proactively possesses reasoning capabilities. I don't think we necessarily need to teach Agents reasoning approaches — they can integrate many thinking steps and rapidly reach conclusions.

Currently, Agents are all virtualized, code-executing entities, but they'll definitely become embodied Agents eventually. That's also a worthwhile research direction. After all, online resources are spaces created by humans. For Agents to serve humans, they need embodiment. We've even discussed whether GPT-5 will be an embodied Agent. GPT-4 already has image generation and tool-using capabilities — will GPT-5 be more generalized, embedded in every aspect of life? I think we can open our minds to that possibility.

Relevant Papers and Code Links

AgentLite paper: https://arxiv.org/abs/2402.15538

AgentLite code: https://github.com/SalesforceAIResearch/AgentLite

xLAM model: https://github.com/SalesforceAIResearch/xLAM