Professor Liu Hao: Traffic Lights in the Agent World | Agent Insights

Last July, Oasis argued that the natives of the AI world would be Agents (The Most Aggressive AI Investor). Building on that, what will serve as the underlying protocol, highway, and interconnection method for Agents? With these questions in mind, Oasis has been exploring continuously.

From the history of Agent development to underlying frameworks, we believe Agents are the real foundation for large model capabilities to take root, and will reshape lifestyles from the ground up of social structures. Going forward, we'll share a series of Agent papers that Oasis has studied, along with insights from conversations with top scholars in the field — exploring Agent innovations together with the industry. Enjoy.

In China, the management of megacities has become an increasingly pressing issue. The concept of smart cities was written into the government work report as early as 2015, followed by successive national policies on smart transportation, urban spatiotemporal big data platforms, and the national integrated transportation network. The 2023 China Intelligent Transportation Conference once again emphasized the deep integration of AI with the transportation industry.

The USAIL team at HKUST (Guangzhou) recently published a comprehensive survey and proposal on urban foundation models. Urban Foundation Models (UFMs) are large-scale models pre-trained on extensive multi-source, multi-granularity, and multi-modal urban data. UFMs can deeply understand and adapt to various types of urban data and tasks. By integrating different types of urban data, they provide comprehensive insights into complex urban environments, reveal intricate dynamic spatiotemporal patterns, and enhance decision-making capabilities across diverse urban contexts — effectively advancing urban general intelligence and leading urban systems toward more integrated, intelligent, and responsive development. The team lead, Hao Liu, is currently an assistant professor and PhD advisor in the AI Thrust at The Hong Kong University of Science and Technology (Guangzhou). He previously served as a senior researcher at Baidu Research, focusing on fundamental research in artificial intelligence and data science, and their interdisciplinary applications in intelligent transportation, urban computing, and recommender systems — work that earned him a spot on the Forbes China 30 Under 30 list.

LLMLight: Large Language Models as Traffic Signal Control Agents

Paper link: https://arxiv.org/abs/2402.01749

Problem addressed: Traditional traffic signals cannot perform intelligent traffic flow prediction; next-generation traffic signal control will effectively improve transportation efficiency

Model architecture: Traffic control can be viewed as a partially observable Markov game under certain conditions. LLMLight combines environmental perception with GPT-4's decision-making capabilities for traffic flow analysis and prediction

Results: Average driver wait time at intersections reduced by 50%, alleviating driver anxiety and optimizing traffic flow

Applications: Intelligent traffic signals, multi-intersection traffic control, smart city planning

To gain deeper insight into the LLMLight Agent, we were fortunate to speak with Professor Liu for his perspective.

Oasis: In the intelligent transportation field, what pain points has reinforcement learning had for traffic planning and signal control? What possibilities does LLM bring for automated Agents in intelligent transportation?

Professor Liu: I focus on smart cities, currently with intelligent transportation as my main area. LLMLight is an Agent for signal control. Signal control has gone through three phases: the first was heuristic algorithms from traditional traffic engineering; the second was machine learning-based approaches; and now we're in the third phase, using LLMs for traffic control. There have been at least nearly a hundred papers on reinforcement learning approaches, with dozens of algorithms already. While these algorithms can perform control and even incorporate real-time data during information input, they still face pain points: they require historical data for training, the process is prone to crashing, and convergence is difficult. Reinforcement learning emphasizes exploration and exploitation, using simulators built from local historical data for exploration and exploitation, but cannot be tested in real traffic environments — so there's a gap between the real and simulated worlds. In this situation, traffic management departments won't adopt it, and service providers won't pay for it. Deep learning itself lacks interpretability, but interpretability is especially important for clients to sign off. Future LLM-based intelligent transportation Agents won't need specialized training for different cities, intersections, or signals, and inherently have natural language interpretability — the generated decisions can be directly reviewed by traffic management experts. With strong general knowledge, LLMs combined with Agent frameworks can also collect new action explorations and update strategies in real time, with capabilities far exceeding reinforcement learning.

Oasis: How is real-world data for intelligent transportation currently obtained?

Professor Liu: Intelligent transportation solutions currently rely mainly on centralized data collection, with each intersection using cameras for recognition. Some intersections have more sophisticated traffic light capabilities and can collect data such as how many vehicles are queued at the intersection. Camera recognition of traffic data is already quite accurate, and future vehicle-road collaborative roadside infrastructure can further enhance situational awareness at intersections.

Oasis: Can multimodal large models like GPT-4V and Gemini promote intelligent transportation Agents?

Professor Liu: An important optimization direction for LLMLight is incorporating multimodal information. The most direct application of GPT-4V is in autonomous driving, which is essentially the same as traffic control. The context understanding problem in autonomous driving also exists in traffic control, except that in traffic control the intersection is stationary. Compared to language models, where the input provider writes prompts to supply information and the large language model decides which information to use and which to ignore for reasoning, multimodal large models can directly perform information extraction from the entire environmental video without expert involvement. The model can itself explore useful visual features, such as queued vehicle counts, obstacles, and so on. In the past, understanding and control in autonomous driving were two separate problems, and visual extraction was also preparation for the policy model. Many reinforcement learning approaches have already been upgraded with GPT-4V capabilities, doing inference and control holistically, and fully vision-based end-to-end approaches are gradually gaining acceptance. Currently visual language models are dominated by GPT-4V, which has better commonsense reasoning effects, and vision grounding is also a good solution for alleviating "hallucination" problems. Therefore, using multimodal large models for traffic control can extract more relevant information while addressing the "hallucination" problem.

Oasis: What similar inspiration can LLMLight's application in traffic signals bring to other Agent application domains?

Professor Liu: Intelligent transportation has three cores at its essence: understanding, prediction, and decision-making. LLMLight is essentially doing decision-making in traffic, and whether it's intelligent transportation, autonomous driving, or intelligent driving, all are based on decision-making at different granularities. Other application spaces such as parking lots, charging stations, and road planning — feeding the information in prompt templates to Agents to get corresponding outputs — can all be reused for different strategies.

Oasis: In five years, what does the AI Agent world look like in your eyes?

Professor Liu: The Agent concept took off around mid-2023. The main current approach to building Agents is still constructing systems, similar to computer architecture doing task decomposition, with each Agent taking on separate tasks, ultimately completing specific tasks and evaluating completion metrics. I think Agent applications won't be particularly prominent in the short term, but will gradually land in five years. In the long run, if AGI can be achieved through large models, Agent is a necessary framework. Beyond decomposing complex tasks, Agents also have capabilities for processing and understanding multimodal data like images, coordinating with external knowledge bases, task-specific optimization, and tool learning. Agent provides a very open framework, and LLM itself has zero-shot learning and generalization capabilities — the two technologies are naturally suited for combination to achieve long-term goals.