AIGC "Peak Series" | Li Dahai: We Are Entering the Era of Collective Intelligence (Live Q&A)

The ChatGPT boom earlier this year rapidly kicked off a wave of AIGC entrepreneurship filled with imagination. A batch of large model-related startups emerged domestically. Now, half a year later, discussions about large models and AI have entered a different phase: evolving from "talking concepts, discussing the future" to "discussing products, looking at implementation." After the underlying technical logic has been validated and widely accepted, AI entrepreneurs are repeatedly testing the implementation of application products based on large models, while entrepreneurs outside the industry remain on the sidelines, wondering how much these implemented products can actually change specific business operations for industries and enterprises.

Recently, at a "Code Brain Guest · Private Chat" session, Source Code Capital, together with Amazon Web Services, specially invited Li Dahai, co-founder and CEO of ModelBest and partner and CTO of Zhihu, to share his views on AGI. He proposed the development trend from individual intelligence to collective intelligence, pointed out six typical characteristics of AI agents, and exchanged views with numerous entrepreneurs on large model-related entrepreneurship in China.

Li Dahai graduated from the mathematics department of Peking University and was one of the founding members of Google China. He later served as engineering and technology lead at Yunyun and search technology lead at Wandoujia, with over ten years of continuous entrepreneurial experience. In addition to his role as CEO of ModelBest, he currently serves as partner and CTO of Zhihu, leading a team of over 1,500 people, with extensive experience in building top-tier technical systems, strategic planning, technical management, and commercial implementation.

The following is the main content from the live Q&A session:

Question: How do you view OpenAI's recent launch event and the future development of the industry?

Li Dahai: China and the United States are two different stories. About a month or two ago, a group of large model company CEOs gathered to exchange ideas. After everyone shared their views, someone finally asked: "We should figure out whether large language models are a technology or a product."

I hadn't figured out this question at the time, but later I did. The answer differs between China and the United States. In the US, large language models are a product; in China, large language models are a technology. OpenAI's releases can point China in several directions, but they can't completely solve China's problems.

It's simple — few entrepreneurs here are doing enterprise services. Those in enterprise services have had a rough few years. There was an investment peak in enterprise services around 2021-2022. Looking back, very few companies from that wave survived, because China's ToB market is extremely fragmented and inefficient. OpenAI can achieve excellent business growth and strong gross margins through APIs, but this basically doesn't hold up in China. Many domestic large model companies now have their own APIs, but gross margins are basically negative, and it's visibly not going to become a large market. The reasons behind this, I believe, everyone can understand. If this is the case, the business models in China and the US are different.

A truly era-defining product is accompanied by an era-defining technology behind it, plus a very good application scenario and a matching business model — these elements are interdependent. In the early days, Google found search advertising and the pay-per-click model to monetize. Another example is Facebook — its social platform was extremely successful as a product, and its recommendation technology integrated very well with the product, so it went public. But then its stock price went down because its business model changed. Later, they found Facebook's new model on this foundation, and when this model came together, it became excellent. China's current situation is still in the process of finding a model. The business model corresponding to large language models in China isn't like OpenAI's — OpenAI represents one model that can't become the primary model supporting Chinese companies' development. I believe the soil in the two countries is different, so we need our own answers.

Question: I believe at today's stage, large language models still have the intelligence level of a toddler — you can communicate with them, but seriously relying on them isn't realistic. On this basis, I'd really like to hear the industry's view: what level do you think this intelligence will develop to in the future? I'm somewhat pessimistic personally — Moore's Law has reached its end, the existing amount of human knowledge and engineering is what it is, and the scale of training language models is probably about as large as it gets. Is there possibility for further growth, or can intelligence still increase further?

In the ToB market we're working in, if APIs aren't a good model in China, we've already developed some applications for this market. When promoting them, there's a major problem: users find it very difficult to deploy these things. Deploying a model for my application is basically unrealistic — my application costs 200,000, but deploying a model costs 2 million per year. They can't deploy another model just for my application. Do you plan to address this market? If so, what will this market look like in the future?

Li Dahai: On the first question, I'm personally optimistic. I believe AGI is a revolution. The Industrial Revolution lasted nearly a century. If we count the Information Revolution from when the first computer appeared, it's already been several decades. The AI revolution won't be a short-term matter either. At minimum, the first version of ChatGPT is the steam engine of the AI era — there are still many problems to solve, but I'm sufficiently confident.

In recent years, we've witnessed exponential growth in model scale, expanding tenfold annually. This continuous evolution isn't accidental but the inevitable result of emergent technical patterns — what's called the "aesthetics of brute force." After reaching the milestone of basic model usability, we've seen a strategic slowdown in growth rate and cost optimization, which is both feasible and reasonable.

On data: purely from a text perspective, we seem to have mined down to the bottom of internet data. But as a former Google search engine engineer, I deeply understand that what crawlers can access is just a small portion of the data ocean. In the search engine world, we call this the "deep web" — useful data in human society extends far beyond this. Additionally, beyond text, massive amounts of multimodal data remain to be mined; the space for learning is extremely broad. The data horizon is far from being reached. The challenge of computing power has actually inspired new understanding of Moore's Law. As data volume continues to expand, the issue is no longer limited to unit computing power but overall computing capacity improvement. Overall, we maintain an optimistic attitude toward future development.

As for APIs, they represent customers' capabilities and needs — they have the willingness and ability to seek and utilize APIs. But in China, solving the "last mile" problem is crucial. This doesn't necessarily require private deployment. APIs can serve as a tool provided to users, but this isn't the endpoint. We still need extensive interaction in the intermediate stages, with dedicated personnel for system integration. Such integration can facilitate interaction in more economical ways. I believe these two approaches can complement each other.

Question: Regarding communication and dialogue between Agents, many people are using this, including our data. First, is there anything particularly valuable to share here? Second, is it possible that their exchanges and dialogues could spontaneously generate something creative? At this stage, I personally think it's still very difficult to create something truly creative.

Li Dahai: What makes something valuable is that we must approach it with reverence for nature and in accordance with natural principles. Often, we can gain beneficial insights from observing how humans collaborate and how business is structured. Specific methods include how to use tools and reflection techniques to improve quality, and how to communicate from different perspectives — these are our valuable experiences.

On the other hand, you mentioned how Agents can create something creative. Creativity is typically produced through group collaboration. Sometimes human thinking can be seen as obtaining idea fragment A from one person, idea fragment B from another, then after personal reflection adding C, thus forming one's unique idea. In fact, creativity often involves a process of permutation and combination, where large models typically excel. The content they generate isn't directly copied word-for-word from some original, pre-trained document, but rather assembled, broken apart, and reassembled from known information. This process is indeed creative — similar to how humans sometimes produce interesting ideas from false memories.

Question: Agents are actually closely tied to large model capabilities, and we know large models are developing rapidly. A complex point in Agents is task decomposition, which is quite a difficult problem. Task decomposition is closely related to large model capability development — you might do a lot of work, then find that with a large model upgrade, all those capabilities are built in, and many products you developed may no longer be needed. How do you judge which things large models will do, which things OpenAI will do? For example, you mentioned now needing to solve problems of memory, emotion, knowledge attribution, growth, and knowledge bases — which should large models solve, and which should be solved on top of large models?

Li Dahai: First, when it comes to entrepreneurship, the core issue is clarifying what user value is and how to precisely deliver this user value to them. This is something large model companies or OpenAI cannot fully accomplish. Whether you're helping Douyin creators in China or solving specific user pain points in some domain, these problems usually can't be solved by relying solely on large models or just their capabilities. Typically, you need to combine models with various technologies — sometimes models can cover part of it, reducing some workload — but the complete product ultimately delivered to users, models cannot fully replace.

Second, we need to consider the time constraints of technological development. Any technology, such as a model we release, its leading position won't last long because technological evolution is extremely fast. Therefore, beyond solving specific problems, we also need to consider how to build other advantages on top of technological advantages. This question is equally crucial — how to translate technological and product advantages into advantages in other areas.

Question: I have three questions: 1. Regarding Agents themselves, what differences do different base models bring? You just mentioned we also switched from 3.5 to our own model.

Li Dahai: Different base models definitely have capability differences. On the other hand, we need to switch to our own model so that your model can develop a data feedback loop with upper-layer applications. When we first switched, our model was slightly worse than 3.5, but I think this gap is acceptable — there are still many good applications that can be built.

Question: How large is the difference brought by this base model?

Li Dahai: There are many details. For Agents, the model needs to very rigorously follow your required output format — it may not follow closely enough, which degrades performance. And code generation capability — I found our CPM model is better than ChatGPT 3.5 in many aspects, but worse in some specific areas.

Question: The base is connected through APIs. Because many major companies like AutoGPT are also doing this, and we're a startup with base models ourselves. There are many independent startups specifically focused on Agents as their direction. From a development direction perspective, what are the prospects and landscape for these two approaches?

Li Dahai: In this regard, I believe model companies tend to have easier commercial success. From another angle, why develop AI without using ChatGPT's model but choose to develop your own? Because your own model can better adapt to complex tasks. Not everyone needs to develop base models, but for complex applications like Agents, independently developing models can make the model's data loop more efficient. Of course, OpenAI now offers fine-tuning capabilities, allowing you to fine-tune based on its models, but I think from a business perspective, complete reliance on it may carry some risks.

Question: What risks do you mean?

Li Dahai: For example, what if OpenAI releases the exact same framework as yours?

Question: I think the business risk is whether there's still room for startups here. If several major public clouds are already highly concentrated, but multi-cloud management platforms and third-party neutrality still have markets — if we draw an analogy, then for Agents, is it better for startups with their own base models to build them, or how does it work?

Li Dahai: Actually, we should be discussing Agent-based applications — that would be more advantageous. If you position yourself as part of the Agent framework, in the middle, with the LLM layer below and the application layer above, then the value chain in the middle may be very weak. Personally, I don't think this is a good model. But if you combine Agents with applications, having specific application scenarios and building your own advantages through these scenarios, then this approach may be more viable.

Question: The third point I'd like to explore with you is that Agents will definitely bring much new space. For traditional software or this type of industry or track, which will be disrupted and which will gain wings of imagination to fly faster?

Li Dahai: I think future companies can be divided into two types — this is also an investor friend's view that I'd like to share. I believe future companies will basically fall into two categories: "+AI" and "AI+", with the latter committed to disrupting existing patterns. Disruption can be self-disruption or disruption by other companies. We should consider what changes will occur after adding AI — this is more conducive to thinking about future development directions.

Question: I have a small detailed question. Now many large models have emerged. Beyond competing on leaderboards, do you have any relatively specific quantitative metrics internally to evaluate whether a model is good or bad?

Li Dahai: If evaluation isn't driven by pursuit of rankings or other false motives, and you pay attention to avoiding data contamination during the evaluation process, then the data obtained will be more reliable and can be used to judge model performance.

Question: What are the quantitative evaluation standards?

Li Dahai: There are many different evaluation benchmarks. Evaluating large models by application is the most practical approach.

Question: What I mean is whether there's a relatively clear quantitative metric internally, because training goes round by round and you need to know if round two is better than round one — there should be a quantitative metric.

Li Dahai: There are now many open-source evaluation questions available. These can be machine-evaluated, so they have objective metrics. At the same time, we've developed our own increasingly complex subjective questions covering multiple domains that require human scoring — this scoring cost is relatively high. Scoring is usually expressed in points, for example: especially satisfied is 2 points, relatively satisfied is 1 point, unsatisfied is 0 points, then combined into a total score.

Question: I don't know whether you're evaluating SFT potential or direct output potential? Referring to general model base capabilities, knowledge capabilities, logical capabilities, or professional domain SFT capabilities and performance?

Li Dahai: For professional domain SFT, generally speaking, the larger the model, the stronger the capability. This is my practice at Zhihu — when you have 2 million high-quality data points, whether it's 7B, 10B, or 13B, whatever model you use for SFT is almost the same. If your data is good enough, the model doesn't matter.

Question: I'll ask a last-mile question. We're currently in enterprise marketing, helping enterprises with last-mile AI Agent delivery. With single Agents, individual intelligence, we feel that in many scenarios we can help enterprises optimize operations with very obvious comparisons and strong willingness to pay. With frameworks like XAgent, the framework is very cool and we're very excited — we've tried some approaches, essentially moving from simulating an individual to simulating an organization, capable of doing more things. But from actual results, at this stage it's still very preliminary. Enterprises don't care about the process, they directly focus on results, and compared to now there isn't particularly large obvious advantage. From the perspective of specific application scenarios, monetization, and actual output, what kind of development and how much distance is still needed to reach enterprise application-level results, or what are some key bottlenecks?

Li Dahai: To be more specific, multi-agent intelligence communication requires longer memory, while large language models require higher performance. This is a quite important consideration. If work in these areas isn't excellent enough, there may be limitations. For example, if we handle interactions among more than 7 agents, due to LLM context length issues, performance may degrade. Additionally, collective intelligence is a very good model for simulating human social processes. From this perspective, it has potential to play a role in creating new AI processes. I think this is its value — its potential is large. Of course, if other solutions currently have better results, then use the better-performing ones first — don't be too fixated.

Question: I think the imagination space for Agents is enormous. In the most perfect scenario, we'll have a super terminal in the future, phones will become less complex, like AI intelligence — tell it any need, and to achieve this very perfect state, we'll have transitions. Now considering specific scenarios, I hope Agents can operate Meituan APIs and JD.com APIs, book my flight to the destination, and directly reply that everything is arranged, then I just facial-scan to board and check into the hotel, without needing to touch this so-called current internet concept throughout the entire process. This imagination space is indeed enormous and will completely change what you call the human-machine relationship. If we want to do this, in specific operations, how would JD.com or Meituan, which currently occupy market data and users, be willing to give you their data and APIs to let you operate? How does this transition happen?

Second question: actually phone manufacturers have huge advantages because they hold the operating system. Apple has launched such Agents, and most entrepreneurs have very large advantages. To reach this ROA world, what opportunities do entrepreneurs have?

Li Dahai: I think entrepreneurs must always look for specific and down-to-earth scenarios. Entrepreneurs must forget about OpenAI.

Question: From last year to now, OpenAI has actually gone down many times, and Agents have occasionally gone down too, but OpenAI has gone down to the point of being unable to go down further. Building products based on it, whether ToB or ToC, requires SLA. Including when we use products from various domestic API-providing SaaS large model companies, how do we guarantee that my product's SLA is reliable?

Li Dahai: For this kind of problem, there's no good answer — it's service degradation. In normal mature companies, you actually need disaster recovery and should do destructive testing. If large model services become unavailable, your entire service should degrade rather than catastrophically leaving users completely unable to use it. This is an engineering problem.

ModelBest

ModelBest was incubated from the NLP Laboratory of Tsinghua University's Department of Computer Science. The company's founding team consists of experienced industry entrepreneurs, leading AI technologists, renowned NLP scientists and scholars, among others. Core technical team members come from China's top natural language processing research laboratories, all holding doctoral and master's degrees from prestigious universities, having published over a hundred papers in authoritative domestic and international journals and conferences, and obtained multiple patent authorizations — their research and technical capabilities are at the domestic leading level. Additionally, high-end talent with rich industry product and commercialization experience has joined to lead large model scenario and domain application implementation.