Oasis Capital Dialogue with Professor Qi Zhang: Boundary Cognition

The bigger the wave, the louder the noise — how do we seek truth?

Today we're joined by Professor Qi Zhang from the School of Computer Science at Fudan University. Throughout our conversation with him, you can feel his relentless pursuit of "seeking truth" in every moment. Enjoy.

Oasis Capital: Your work on the lightweight NLP toolkit FastNLP and the robustness testing platform TextFlint has won top conference awards. Could you share your latest research focus and progress?

Professor Zhang: FastNLP and TextFlint were projects centered on deep learning models. From 2021 to 2023, most of our work revolved around these two platforms. Starting in 2023, we've shifted our primary focus to large language model tasks. Going forward, we'll continue researching LLM robustness. Personally, I've been particularly focused on a few areas lately: first, how to obtain high-accuracy responses in the LLM domain; second, explaining model capabilities — starting from experimentally observed phenomena to do explanatory work on the differences and connections between large models and earlier deep learning models; and third, model capability evaluation.

Our lab's work spans foundational model pre-training, inference enhancement, multi-agent systems, and other directions.

Oasis Capital: Could you introduce your work and thinking on the evaluation front?

Professor Zhang: Traditional NLP evaluation mostly used benchmarks — releasing a dataset with training, development, and test splits, training on it, then testing and reporting results. But this model doesn't work well for large language models because much of the work can't be reproduced. For one thing, many pre-trained base models aren't publicly available. Beyond the task itself, you also need other SFT (Supervised Fine-Tuning) combined multi-task mixed training approaches. Overall, you can't reproduce the open models. Many trained model architectures aren't open-sourced, making evaluation difficult. In this environment, any open leaderboard with accessible test data gets gamed to high scores, regardless of whether there are actual test results or correct answers.

So we've tackled this problem with two phases of work on how large models should be evaluated. The first phase used multiple evaluation methods: public evaluation by ordinary users, crowdsourced user evaluation, expert lab evaluation, GPT-4 comparative evaluation, GPT-4 itemized evaluation, and so on. The second phase examined what level models have reached from professional domains down to undergraduate-level tasks. This data isn't released to any party before evaluation, and after evaluation, all data is made fully available with no further leaderboard updates, to prevent data gaming. We plan a third phase evaluation in late August or early September, preparing a much larger data volume. Each time we'll only select a portion for single evaluation — after each system is evaluated on this subset, that data gets retired.

Our goal is automated evaluation backed by large-scale data, so that leaderboard distortion is relatively minimized.

Oasis Capital: In our conversations with teams domestically and abroad, Stanford's HELM is widely recognized. What's your take?

Professor Zhang: HELM integrates various NLP tasks and can evaluate a language model's basic knowledge coverage, but it can't evaluate a model's actual problem-solving ability after SFT and reinforcement learning — things like generation fluency, logical coherence, and so on. This evaluation collection has many tasks and uses various metrics, but personally I think it remains quite foundational, quite distant from real-world large model usage.

Oasis Capital: With everyone chasing the LLM hype, there's been lots of noise. From the evaluation perspective itself, how should the market objectively view model capabilities?

Professor Zhang: The market is extremely hot right now. Major companies, including startups, have poured in massive resources, so on the evaluation question, you can't compare it to the scale of pure academic research like a single paper. The impact of evaluation results is multifaceted — beyond the technical layer, you can't cover everything comprehensively.

Our lab's second phase of work started from seeking truth, doing internal research and evaluating how to evaluate, making comparisons between different evaluation methods. After completing that, we used what we considered the most reasonable approach to open-source and publicly release some methods and data, to see what people think.

Evaluation itself promotes research — you can observe and discover many interesting phenomena from it, and these phenomena help researchers better grasp future research directions. So we don't do evaluation to rank models as good or bad, but hoping to push research to the next level.

Oasis Capital: Where does evaluation's impact on NLP development mainly show?

Professor Zhang: Evaluation is a double-edged sword — done well, it promotes research; done poorly, it has negative effects. After papers are published, people judge accuracy improvements based on gains on evaluation sets. But in reality, some models' improvements may be very small, with minimal methodological innovation — just certain tweaks that improve the model somewhat and achieve gains on so-called evaluation sets. If such papers appear in large numbers, they drown out truly innovative research. In the large model domain, this damage from evaluation is even greater.

Overall, public evaluation is just one tool — you still need to test models hands-on to judge quality. When we do projects for companies and select base models, we rely more on non-public evaluation comparisons rather than public evaluations to make decisions.

Oasis Capital: After two phases of evaluation, what insights have you gained for your next stage of research?

Professor Zhang: In February this year we did detailed evaluation of ChatGPT and found some interesting phenomena: you could clearly see its early versions couldn't complete many NLP-specific tasks, like part-of-speech tagging and so on. These aren't questions ordinary users would ask, so early versions performed poorly on such tasks. But one or two months later, the same tasks showed substantial improvement in newer versions. We also found ChatGPT clearly used two different datasets for Chinese and English training. From our previous evaluation results, its capabilities were injected during pre-training or the SFT stage. This only explains how the work was done, the scale of training, and different approaches — not that "emergent" capabilities appeared.

The main problems we discovered through evaluation: first, knowledge accuracy is relatively low, arguably very low. For example, GPT-4's knowledge correctness in the medical domain is only 80%. This has also pushed us to focus on domain-specific models. Compared to GPT-4, domain models' knowledge accuracy could even affect the entire future development of large models. For example, in my professional domain, I can spot major errors in model answers at a glance. But take the current room-temperature superconductor discussion — where exactly is the model's output correct or wrong? Users can't possibly check every knowledge point against search engines one by one. So how to improve correctness is, for me, a crucial question. Though currently, improving overall in general domains is too difficult; doing some work in specific domains is more achievable.

Oasis Capital: Evaluating large models is to judge model capabilities, and we see you've done lots of work on robustness too. How is work in this direction progressing now?

Professor Zhang: Robustness is a major problem in deep learning. Before large models emerged, including through late 2022, many of our papers focused on robustness. The robustness problem allows deep learning models to score very high on standard evaluation sets — on reading comprehension corpora like SQuAD, they can hit 90%, higher than humans. But if you add two commas in the question, or the original position, add a space, add some names, model accuracy drops precipitously. This shows deep learning models still learn surface-level features. We want models to not focus on surface features. Our previous evaluation results found large models still have this problem, though the degree of mitigation compared to deep learning is reduced. For example, in sentiment analysis, robustness transformations that affected deep learning don't impact large models much, and accuracy remains relatively high. But for information extraction NLP tasks, large models' accuracy drops very similarly to deep learning. So we'll continue research in the robustness direction.

Since large models are mostly used for generative tasks, in the past half year we haven't done much work on improving robustness for information extraction and similar tasks in large models, instead putting more energy into generative tasks where large models excel most. Next year we'll likely continue in this direction.

Oasis Capital: There is indeed much controversy between domain-specific models and general models. If general models' reasoning capability is strong enough, wouldn't domain-specific models be unnecessary?

Professor Zhang: Controversy certainly exists, and this is indeed a major topic. Our lab has had many internal debates — can large models solve reasoning problems? If reasoning capability is good enough, then through general training of large models, any task can be done with accuracy meeting requirements, and there'd really be no need for domain-specific small models.

But personally, I don't think large models are suited for reasoning. It's a language model — what it excels at is predicting the next word, solving sentence generation problems, solving what previous natural language generation couldn't do well, while possessing multi-task learning capability. If you force it to do reasoning, even on the GSM8K dataset, models can only solve elementary school word problems, and adding CoT and other things on top yields limited improvement.

Reasoning is accurate, logical — we have excellent reasoning engines, we just can't convert natural language into the standard format of axiomatic theorem engines. Currently I haven't seen large models have good explanations or likelihood of completing such work, so why force them to do work they're unsuited for? I think large models are better suited to completing specific tasks for different industries, different companies, different jobs — like solving certain problems in dialogue systems for a particular company. This kind of model is more practical at this stage.

Industry models might not even be a real proposition. Take finance — what kind of industry model would you build to solve all problems in finance? Finance touches every aspect of life; if you could really build it, it wouldn't differ from a general model.

Oasis Capital: So relative to general models, building a relatively powerful model with sufficient understanding of human language as a central hub to coordinate tasks, with each specific task being a medium-sized model — would that better fit the development direction?

Professor Zhang: There's been much discussion about coordinating models, but I really think that's asking too much of language models (laughs). True coordination is rule-based. If you can get large models to recognize well and convert to structured data, coordination isn't complicated. For humans, writing some rules and small programs handles it — no need for machines to do it. The biggest problem with machine operation is the black box — when something goes wrong, you don't know how to fix it.

For example, in dialogue systems the hardest part is colloquial expressions that can't become structured extracted Slots, or when scaling horizontally adds new Slots that the original model can't extract. Having large models extract Slots, then deciding whether to transfer to human or do other tasks — no need to solve with other models, just write rules. White box is transparent and easy to control.

I think from an application perspective, there's no need to pursue full automation and zero cost — low cost is enough. You don't need to fully rely on large models or AGI to achieve this goal.

Oasis Capital: Large model capability improvements have made industry eager to find landing scenarios, but problems emerge as soon as you try to deploy — and the controversy becomes even greater.

Professor Zhang: Yes. You test a few cases and results look good, but once deployed, completion might only be 80%. But existing systems might achieve 90% with small models — why not use those? The problem now is, if you keep going down the AGI path, desperately trying to get large models to 90 or 95 on all tasks, how far can this road go? If you can truly make revolutionary change, achieving zero cost, then that would be a powerful product.

Oasis Capital: OpenAI probably wants to continue toward AGI — they're passionate about that goal.

Professor Zhang: Yes, with sufficient funding, system products already launched, and Microsoft's support, they can indeed persist on this path. The prerequisite is still having strong resource backing.

Oasis Capital: I believe you also have much exchange with industry. Are there misconceptions in current large model development?

Professor Zhang: I think perhaps it lies in people's understanding of model capability boundaries. As you just mentioned, wanting to make large models into central controllers that solve problems at zero cost. The core issue is how to define this large model. Where are its capability boundaries? What problems can it solve? What can't it solve? I think this can be given analysis and experimental reports from theory or experimental results, letting people see where large models' capability boundaries lie for different tasks. Without knowing, you'll make mistakes.

Oasis Capital: Regarding rumors that GPT-4's capabilities declined after excessive alignment work, have you observed this?

Professor Zhang: We haven't recently done large-scale calling of GPT-4's API on specific tasks, so we haven't specifically tracked changes in its capabilities on certain tasks. From other models' experimental results, the problem of mutual interference between multiple tasks is relatively obvious. Having models complete so many general tasks often means pulling up capability in one area causes another to drop. Models at the hundreds of billions scale face the same problem. So if such rumors exist about GPT-4, it's not surprising. It might adjust model weights based on user input, and in that adjustment process, certain capabilities could decline.

From this angle, it similarly triggers the question of how far the current large model path can go.

Oasis Capital: So large model safety alignment is also a challenging problem?

Professor Zhang: Safety alignment is a very large topic. The alignment capabilities we require may be higher and stricter than overseas standards. Overall, I think there's still insufficient understanding of large model foundational theory across sectors. Without the most basic theory, other developments are hard to judge. Before the Wright brothers built airplanes, there was already a basic prototype of aerodynamics — before building the plane they made dozens of models, even built a small wind tunnel, and after hundreds of tests, finally chose the original wing shape. Though not precise at the time, the principles were clear. For large models, what's most lacking is simple, not necessarily precise, cognitive understanding at the principle level. Without this understanding, the cost of continuing to try is hard to estimate.

Oasis Capital: Is there potential that OpenAI already knows the principles but just hasn't disclosed them?

Professor Zhang: Recently reviewing previous research and papers, we've also seen many phenomena similar to current implementation results. So I think OpenAI has probably summarized some principles, including the MoE architecture.

Integrating many things into one model — many tasks are irreconcilable. For example, adding large amounts of data during the SFT stage reduces the model's generalization ability on Q&A tasks, which is fatal for problem-solving capability. But if you don't add much data during SFT, many NLP tasks are hard to improve. The two tasks are inevitably uncoordinated. With MoE, you can isolate uncoordinated tasks — do heavy SFT data work on Model A, and use small data amounts on Model B to complete knowledge Q&A.

So the MoE architecture should be the right path — at least current experimental results provide good corroboration. But what was the implementation process? Did it learn a dedicated routing layer itself, or was query decomposition and construction done manually from the start? There's no way to know.

Oasis Capital: Do you think there are other problems in large model development?

Professor Zhang: Currently various approaches exist, and domestic universities and companies are in a follow-along stage. How GPT-3.5 was implemented — there's no public reporting, or relevant understanding is scattered across papers without being synthesized, so you can't get foundational understanding. So everyone is trying things out, and the cost may be high.

Oasis Capital: I heard you're also pushing forward in industrial intelligence — could you introduce that?

Professor Zhang: Our goal for industrial intelligence is simple — reduce costs and increase efficiency. What we're using large models for is adding three functions on top of industrial information platforms. First, organizing enterprise internal private knowledge bases; second, upgrading BI platforms to natural language interactive platforms. This addresses the problem of non-computer-background management cadres using platforms; third, using LLM as Copilot, integrated into existing industrial domain development to achieve improvement. Currently in industrial domains, much development uses PLC, ST language, or interface languages, requiring high-level programmers, but obtaining such programmers in the industrial domain is difficult.

Our work can quickly and cost-effectively inject these three capabilities into different companies' existing systems, like modular PLCs, greatly improving efficiency and quickly completing manual work.

Oasis Capital: What's the difference between refined development of large models and private deployment of small models?

Professor Zhang: Models come in large, medium, and small sizes. Depending on task difficulty, you might use several billion, ten billion, or even hundreds of billions in scale. What I mean by refinement and customization, as mentioned in the industrial intelligence work, is custom development based on each enterprise's actual situation. Not so-called industrial large models that all industrial domains can use. We determine what tasks the model needs to complete based on this company's served customers and existing products. Switch to another company — though also industrial — and the tasks may be completely different.

Making a general-purpose product in a short time — I think the possibility is small.

Oasis Capital: What was your original intention and goal in choosing your research direction?

Professor Zhang: Whether large models or deep learning models, they're just algorithms. Our core goal is hoping NLP tasks can achieve low-cost productization. For example, if we want to do information extraction now, we might need to annotate hundreds, thousands, or even tens of thousands of data points. The ultimate goal is that no matter how customers or scenarios change, you only need to annotate dozens of data points. In the future, maybe with just a few samples, the model can learn a capability.

Like humans — you might rarely encounter physics books normally. But if I tell you this is a physics term and give you three to five examples, you can identify physics domain terms with relatively high accuracy. We hope computer systems can ultimately reach this level of capability — learning most NLP tasks given very few samples.

Oasis Capital: Compared to the previous wave, where do you see opportunities in this large model wave?

Professor Zhang: In every wave, the core is integration with industry. If currently you're not an informatization vendor and only have AI technology, rushing in — the results may not last. Take finance — data and informatization were completed long ago. If a company has a financial model, its technology lead might only be six months. For informatization vendors, they have enough time and technical reasons to keep you out. Three months later, a comparable system might appear — maybe your model can score 100, but three months later I might get a 90-point system through other relatively easy channels. So even if your system is perfect, you still can't enter this industry.

If what a company is doing isn't an AGI product, then it can only be done by original informatization vendors, or the company needs to be large enough to acquire original informatization vendors. In the previous AI revolution, the biggest beneficiaries were still those companies that controlled original channels and resources — doing AI upgrades so they wouldn't fall behind.

Sustaining Vitality

What do you think is technological vitality?

Unconstrained imagination, careful verification; courage to face failure, unwavering perseverance.

——Professor Qi Zhang

School of Computer Science, Fudan University

Oasis Capital is a new-generation venture capital firm in China, dedicated to discovering the most vital entrepreneurs of the next decade and growing alongside them to create long-term value. "Sustaining Vitality" is Oasis's vision and mission. This vitality is both the direction of structural transformation in the era and the resilience and evolutionary power of entrepreneurs.

Oasis Capital focuses on early and growth-stage investments, with individual investments ranging from $3 million to $30 million, concentrating on robotics, artificial intelligence, and technology services, empowering China's technology-driven new service upgrade.