Fudan University Professor Zhongyu Wei: Large-Scale Social Media Agent Simulation | Agent Insights

Social media plays a pivotal role in contemporary society, widely regarded as one of the cornerstones of social activity. Professor Zhongyu Wei of Fudan University's School of Data Science has nearly 15 years of research experience in natural language processing and a deep understanding of the underlying mechanisms of social media technology. Under his leadership, his research team successfully built an environment for multi-angle evaluation of social media agents across different scenarios, comparing their performance against real-world data. In this article, we sit down with Professor Wei to discuss his views on agents and his research findings on social media agents. Enjoy.

"Unveiling the Truth and Facilitating Change: Towards Agent-based Large-scale Social Movement Simulation"

Paper link: https://arxiv.org/abs/2402.16333

Problem addressed: Social media is the bedrock of social movements, making it crucial to simulate public reactions and predict potential impacts. The team introduces a hybrid approach to simulating social media users: core users are driven by LLM-based agents, while ordinary users are driven by an ABM (agent-based modeling) framework. They also constructed a Twitter-like social environment to replicate the response dynamics after agents trigger events, observing user interactions and collective attitudes.

Model architecture: Core user agents include a profile module, memory module, and action module. The team also designed the SoMoSiMu-Bench dataset, composed of three real-world datasets — #MeToo, Black Lives Matter, and Roe Overturned — for fine-tuning.

Results: The team successfully built a Twitter-like simulation environment capable of multi-angle evaluation of differences between simulated outcomes and real-world data. The final simulated emotional trajectory closely matched real-world emotional trajectories.

Applications: Social media agents, social science agents

Micro-level evaluation results for social media

Evaluation results: For stance detection, average accuracy between real and simulated user stances exceeded 70% for both the #MeToo and BLM datasets. For behavioral simulation, overall accuracy exceeded 75%.

---

Oasis Capital: Please introduce your research field and your views on agents.

Professor Wei: I've been working in natural language processing throughout my career. During my PhD, I focused more on social media content analysis, including short-text processing for social media, and used related techniques to analyze the connections between online content and real-world events. During my postdoc, I began exploring computational argumentation and multimodal research, hoping to understand richer and more in-depth information in social media. After joining Fudan University's School of Data Science, I established the Data Intelligence and Social Computing Lab, dedicated to using natural language processing and machine learning to solve interdisciplinary problems — involving the design of computational tools and algorithm deployment.

Interdisciplinary research spans many fields, including healthcare, finance, legal affairs, and social media analysis. Before large language models emerged, we would design models for specific tasks, collect scenario-specific data, and complete model training. This research paradigm faced a significant challenge in real-world application: models were difficult to reuse. The advent of large models inspired a new paradigm — foundation models, model fine-tuning, and scenario-specific refinement — which largely alleviated this problem and created new opportunities for advancing research across industries. Our team has applied similar methodologies to launch three domain-specific models: a legal large model, a medical large model, and a financial large model. We are currently preparing to launch a multimodal content analysis large model.

Agents caught our attention mainly because we discovered that agents built on top of general-purpose large models had insufficient capabilities, and even high-quality domain-specific large models couldn't fully address usage needs. So we thought of building agents on top of vertical large models for specific scenarios. For example, medical consultation is a very complex scenario with many different situations in hospitals, and the consultation process contains multiple sub-tasks. We can actually use multi-agent collaboration to construct this scenario, including symptom inquiry, physical examination, symptom analysis, and specialist evaluation — to a certain extent replicating the hospital consultation workflow and addressing the pain point of complex consultations.

Oasis Capital: How do you see the relationship between foundation models, fine-tuning, and agents?

Professor Wei: I think most people don't need to focus on training foundation models themselves. It's enough to build applications in agent form on top of large models for various scenarios, and then optimize models for those agent scenarios. Fine-tuning itself can also be divided into multiple stages. For example, when we built our medical large model, we used two-stage fine-tuning: the first stage used 400,000 data samples, and the second stage used 2,000 meticulously constructed data samples. After fine-tuning with SFT (supervised fine-tuning), the overall model performance improved significantly. So in the fine-tuning process, the impact of refined samples on the model is substantial.

Oasis Capital: What prompted you to research agent environments for social media and to keep deepening your work in this area?

Professor Wei: My research on social media dates back to 2010. When social media first emerged, the social network world was a fresh new object, and some people tried to study the relationship between people's behavior in social media and events happening in the physical world. In the early 2010s, we did a project predicting UK general election results through social media content — essentially doing correlation prediction between events.

Coming back to the present, social media has become fully integrated into human society. Obtaining social media data is far easier than accessing the physical world, so understanding social media data and content can help us better learn about the behavior of specific groups and discover how information spreads within communities. Beyond understanding, we also hope to influence certain social events through social media, providing appropriate information content to moderate and improve the overall online environment. For example, in the future if people are arguing fiercely on social media, we could deliver some jokes to the target to cheer them up — that might also be a positive intervention method (laughs).

Professor Shiping Tang at Fudan University has used ABM for simulation and prediction. ABM can efficiently simulate large-scale user behavior on social media, but its weakness lies in individual modeling capabilities. After large models emerged, people began using multi-agent collaboration for simulation, which turned out to be more difficult than imagined. The Stanford "Smallville" simulation had at most a dozen or so agents, but for large-scale simulation (at the scale of tens of thousands), the accuracy, authenticity, richness, and memory completeness of large model simulation are all extremely resource-intensive.

This challenge inspired our paper — differentiating users when simulating large-scale populations. Member activity in social media communities also follows the 80-20 rule: many KOLs actually determine the vast majority of discourse direction. If we can precisely model these people and use ABM to model the other followers, we can complete large-scale user simulation through this hybrid user approach.

Oasis Capital: How is multi-agent collaboration (Multi-Agent Reinforcement Learning) implemented?

Professor Wei: We used a simple abstraction method. First, we sampled 1,000 users for a single event occurrence, with 300 core users driven by large models and the remaining 700 completed by ABM. Then we constructed a simulated Twitter platform, assuming users post once every six hours. Information spreads first through a public square, then through the remaining users' friend circles. Each ordinary user, having prior information, simulates based on information transmission actions every six hours, completing multi-agent collaboration.

Oasis Capital: What are the selection criteria for ordinary versus core users?

Professor Wei: There are many different methods for selecting core users. For example: users with the most followers within a specific time period, users with the highest posting frequency, and users whose statements were most reposted during specific activities. Based on information from different dimensions, we combine all reference metrics, rank users, and determine that the top 300 agents are core users and the remaining 700 are ordinary users.

In the process, we specifically designed a dataset called SoMoSiMu-Bench, composed of user tweet content related to these three real-world events. We selected these three events because they represent relatively influential online social activities we've seen in the past two to three years, with total discussion text numbering in the tens of millions.

Oasis Capital: Currently, when using large models for social science research, how high is the confidence level of simulation results?

Professor Wei: First, I think evaluating simulation effects is itself very challenging — it's difficult to judge what counts as good simulation and what counts as bad. Our paper's understanding of confidence is whether the results from our simulated user framework align with that user's real-world stance. Current results show that the effects we see in these two coarse-grained simulation scenarios are decent, which can prove that the simulation results are not crude.

Simulated emotional changes vs. real emotional changes

Looking at the emotional comparison between real events and the simulation environment in the figure above, the simulated emotional trajectory is extremely close to the real-world emotional trajectory, so we are relatively optimistic about the simulation results.

Oasis Capital: With your extensive experience in natural language processing research and practical scenario deployment, what substantive problems do you think agents help users solve on top of LLMs?

Professor Wei: Agents are essentially solving the engineering deployment of large models. Many of the domain-specific large models we've launched already perform very well in natural language responses, but what specific problems users actually use them for remains unclear. So we need to help build agent scenarios for deployment contexts, telling users that our domain-specific large models can help you solve an end-to-end problem. Therefore, I strongly agree that agents are essentially realizing the engineering deployment of large models.

Oasis Capital: What are your expectations for the development of foundation model capabilities, and what impact will this have on your research field?

Professor Wei: I believe foundation model capabilities have already reached a certain stage. Under the current technical path, the degree of amazement from subsequent technical iterations won't be as great as before. Future development will focus more on digesting this wave of technological innovation, but it will definitely affect our lives to a tremendous degree.

Additionally, I'd like to add that not every team has the capability to fine-tune large models — fine-tuning is harder than people imagine. The difficulty lies in high-quality data governance. Over the past year, through my interactions with people in the industry, I've found that most people think instruction fine-tuning isn't difficult, believing they can just take open-source tools and data, run them, and achieve results. The reality is that the barrier to training a usable model is not low, and the data construction methods involved carry high technical value.

Oasis Capital: In the future, as more and more agents appear on social media, what impact will this have on people's lives?

Professor Wei: People's cognitive differences are substantial, and people themselves produce inaccurate or false information during information dissemination. When obtaining opinions on social media, whether the people expressing these views are human or machine may not make that much difference to the individual. Of course, the online world already contains a certain amount of false information, and the influx of agents will make this phenomenon more severe, putting considerable pressure on government regulation.

Oasis Capital: How do you consider personal data privacy and security issues in training social media agents?

Professor Wei: Traditional KOL and celebrity simulations face issues in this regard — they may face lawsuits from the individuals themselves. But when doing large-scale population simulation, this issue may not be as serious, because the population simulation process is based on fuzzy modeling of individuals. For example, if we want to simulate women around 30-40 years old with annual incomes between 100,000-200,000 RMB, we only need to simulate based on these few conditions. The potential personal data security issues involved here are relatively small.

Oasis Capital: What are your expectations for the application of this agent framework?

Professor Wei: Using large models for large-scale user simulation is highly practical. Traditional humanities and social science research uses manual methods for surveys, easily involving thousands or tens of thousands of individuals, with complicated processes and high costs. Our method enables social surveys to be completed at extremely low cost. We also look forward to seeing agents in the social media domain applied to increasingly more scenarios in the future, providing valuable contributions to society.