Zhilin Yang: Internet R&D Is "Planting Trees," Large Model R&D Is "Contracting a Forest" | BlueRun Ventures Headline

At the 5th Shanghai Innovation and Entrepreneurship Youth Forum held on May 18, Moonshot AI founder Zhilin Yang delivered a speech sharing his original motivations for starting a company, lessons learned along the way, and his views on the development of artificial intelligence technology.

BlueRun Ventures was an early investor in Moonshot AI.

Born in 1993, Yang is the founder of Moonshot AI, a domestic large model company. He earned his bachelor's degree from Tsinghua University's Department of Computer Science and his PhD from Carnegie Mellon University's School of Computer Science. He previously worked at Facebook and Google, among other companies, and has published over 20 papers at top international AI conferences.

Regarding the timing of his entrepreneurial venture, Yang identified two main factors: government support, venture capital backing, and innovative talent as the most important elements of the entrepreneurial environment, and the resurgence of AI coupled with the inspiration of "first principles" as another major factor.

He believes that AI's development owes much to the evolution of the internet, the Transformer architecture, and semiconductor technology — three fortuitous convergences that together created the opportunity for AI to reach consumers. "These three convergences reached their stage at just the right moment. A few years earlier or later wouldn't have been ideal. It just happened to be the end of 2022." He noted, "The greatest value of the internet, in fact, was accumulating more than twenty years of data for AI."

Yang pointed out that breakthroughs in multimodal technology and data bottlenecks are crucial for AGI development, requiring the use of generated data to cultivate and expand datasets. He sees both the increase in computing power and the utilization rate or efficiency of that computing power as the two key factors determining model performance.

Yang described long-context technology as analogous to memory in the computer era — essential for handling complex tasks and delivering personalized services. In March this year, Yang's Kimi intelligent assistant announced support for 2 million characters of lossless context, kicking off the long-text competition among domestic large models.

On AI's impact on worker productivity, Yang predicted that AI will gradually handle more complex tasks and potentially play a more important role in workflows, "possibly from around 1% today to 90% or even 99% in the future."

Yang noted that large model development differs fundamentally from internet development. The internet is like "planting a tree in one spot" — a planned form of development that rarely encounters technical bottlenecks. Large models, by contrast, are more like "buying up the entire forest at once," requiring greater attention to foundational capabilities and allowing the model to emerge with new abilities.

As for entrepreneurship itself, Yang candidly expressed his desire to devote more energy and priority to "climbing stairs" rather than merely "enjoying the view" — addressing the balance between technology R&D and commercial monetization in the entrepreneurial process. "There's still enormous room for model capabilities to improve. We're probably still at the beginning of the industry, having just reached 10^25 floating-point operations. Next we'll likely see 10^26, 27, 28, even 29."

The following is the full text of Zhilin Yang's speech:

Zhilin Yang: Distinguished guests and friends, it's a great honor to be here today to share some of our thoughts and progress.

Artificial intelligence has developed very rapidly over the past decade. Around 2017, the Transformer architecture was proposed. At first, it was still a translation model with relatively limited applications.

In 2018, I began working with several colleagues at Google to train language models based on Transformer, so we were among the earliest to train Transformer language models that outperformed RNNs. In 2019, we chose to return to China to start a company. I think there were two main factors at that time.

The first factor was the environment. We saw tremendous opportunity, including government support, venture capital backing, and the talent cultivated by China's education system over decades, which really made it possible for us to do AI entrepreneurship. The new generation of talent enabled us to pursue AGI. I think this was probably the first important condition.

The second important condition was that we saw AI had enormous opportunity — what we call first principles.

One story that inspired me was that around the 1970s, Intel released its earliest microprocessors. But when they first came out, they were basically useless — you couldn't see much commercial value, and the applications they could run were very limited.

However, Bill Gates and his friend Paul Allen saw something very interesting. Although chips at the time weren't useful for much, they observed that Moore's Law meant computing power doubled every N months. From this they inferred that although it seemed useless now, decades later every household might have a personal computer — which later became reality.

So when we looked at artificial intelligence from the perspective of 2019, we had roughly the same feeling. Your models kept getting bigger, but hadn't reached a state where everyone could use them. But we believed in its first principle — the so-called Scaling Law. Perhaps every N months model compute would increase exponentially, and through this exponential increase in compute, we could achieve improvements in intelligence capabilities. Eventually, whether on phones or new hardware forms, there might be AI that could help everyone.

So based on these two conditions, we chose to start building.

Technology development is often quite magical — in many cases, new technology is built on top of previous technology. It's the development of human civilization's technology tree.

We see why the release of GPT-4 at the end of 2022 was such a huge milestone for AGI — because it achieved effects that were previously impossible. But this effect had a very important precondition, which I think was also a matter of fortuitous timing.

First, the internet of things. The internet developed for more than twenty years and created great value itself. But perhaps in another thirty or forty years, when we look back, we'll find that the greatest value of the internet was actually accumulating more than twenty years of data for AI.

So the fact that we can now access, say, dozens of terabytes of high-quality tokens online is entirely thanks to the development of the internet.

Second, the proposal of the Transformer architecture. This was also a huge variable. Before this, there had never been a network structure that could be scaled.

The third very important point is the development of semiconductors, including the development of the entire software ecosystem, which enabled us to train models with 10^25 floating-point operations.

So these three convergences reached their stage at just the right moment. A few years earlier or later wouldn't have been ideal. It just happened to be the end of 2022.

So for the first time in the world, there was this opportunity for AI to reach consumers. It's somewhat like the development of microprocessors in the 1970s and 1980s — suddenly at a certain moment, you find all the conditions are in place, and it can start becoming a technology oriented toward the majority of C-end users, something ordinary users can use. And as the Scaling Law continues to develop, its capabilities can be further improved.

So in summary, if we look at general artificial intelligence development, its most fundamental level is still first principles. Through predicting the next token, the Scaling Law is essentially doing better lossless compression of data, somewhat like Moore's Law in the computer era, which supported decades of computer development. More and more applications — the underlying law is essentially the Scaling Law.

Then, going one level up, there are two very important technical problems, which I think are probably the technical problems that all AGI companies are most focused on now.

The first level is how to unify. You may have recently seen technologies like GPT-4o — everyone is developing toward multimodality. The essence is wanting to use multimodal technology to further improve intelligence, because text data has to some extent hit a ceiling.

The second level is how to break through the data bottleneck through generating data and cultivation, so that more and more data can join training, and the Scaling Law can continue to develop.

The third level might be some more specific technical problems.

I believe the first level has been discovered and to a large extent validated. Now the core technology in AI is focused on the second level. If the second level can be solved well, it can actually create many new opportunities.

So for Scaling Law, on one hand it's the improvement of computing power itself. But if you only rely on the improvement of computing power itself, it may not be enough. There's another factor — the utilization rate or efficiency of computing power. Ultimately, it's these two factors together that determine what the final model can achieve.

But Scaling doesn't mean simply spending more computing power on training, because sometimes your computing power investment doesn't convert well into intelligence. So it's actually two problems. On one hand, how to continuously invest computing power — this may involve building larger-scale clusters, with chips in each cluster having greater computing capacity themselves. Then a second very important factor is how to maximize the intelligence output per unit of computing power, or how to improve the efficiency of converting computing power into intelligence. For example, you might use active learning approaches, or your architecture itself might be more efficient, which can be stacked on top of the Scaling Law. So ultimately it's the combined effect of these two things.

We've also observed something very important here — long-context technology, analogous to memory in the computer era.

Computer memory also went through decades of development. From around the 1970s, when many microcomputers had only 128K of memory — which seems very small today — and indeed many applications couldn't run. Then a few years later, 512K memory suddenly appeared, and some applications could start running, like the earliest Excel and Word applications, which began appearing around the 1980s. Then after a very long period, perhaps three or four decades of development, now several gigabytes of memory are very common.

So long-context can be understood as the memory of large models — how many tokens you can input into the model at once. And tokens determine how much you can participate in computing. For example, you might want to read a hundred papers at once, or analyze twenty financial reports at once, or process a thousand resumes simultaneously, or you might want an Agent to complete a relatively complex task, like finishing an industry research report, which might require visiting many links, then reasoning and thinking within it, completing a compositional task.

All of these require very long context to allow reasoning within the window, and to model more complex dependency relationships, thereby completing more complex tasks. We've made some small progress in this direction — we can now support 2 million character context window input, providing an important foundation for more application possibilities.

It's essentially a process of entropy averaging. For example, imagine I want to solve a very complex math problem, like proving Goldbach's conjecture. If you only predict one or two tokens, when your length is very short, it's very difficult to solve the problem because its complexity is extremely high.

You need to break down very complex problems into longer sequences — for example, I put it into one or two million tokens to solve. Or say I want to solve a very complex calculation problem — directly predicting the answer is very difficult, but if I break it into many steps, it can better solve the problem. So long text or strong results themselves are actually a process of entropy averaging. It can turn a very complex problem into a simpler problem at each step when extended in length. It can also enable good personalization — for example, each person's interaction history can be recorded, thereby providing you with more personalized service. It can also do video generation, like one-hour or two-hour videos, which can be directly modeled within a very long window, so that dependency relationships between the earliest ten minutes and the final ten minutes can be characterized in the middle. So we believe it's a very important new technology.

We've actually observed that many users employ AI in specific scenarios, including resume analysis, or enabling you to quickly learn a new field, or taking very professional documents or new knowledge as context, so that all subsequent Q&A and learning processes can happen within it, allowing you to quickly become an expert in a new field. This is also a very important application scenario.

We judge that there will be several important directions next, which I think the entire industry will be very concerned about.

First, today if we look at these large model products, in many cases they still remain at relatively simple Q&A. You ask it a question, like who are the authors of Transformer? What is its core idea? These are relatively simple questions. But they may gradually extend to being able to handle some longer-chain complex tasks.

For example, I might not just ask what Transformer did, but ask based on Transformer, help me collect what its latest progress is, what research opportunities might exist on top of it, and can you even directly write these research opportunities into some code, even do experiments, finally analyze the experimental results, and then discuss with me in return? Then it becomes an increasingly complex task, with penetration into your entire workflow possibly rising from around 1% today to 90% or even 99% in the future.

What people are now calling Agents, or "proxies" — an important concept at their core is being able to help you handle many very complex tasks, and not even simply follow your instructions, but possibly even challenge you. For example, you give it an instruction or a goal, and if your instruction or goal itself has problems, it might even discuss with you, acting more like a person working with you, rather than just passively receiving the tasks you want it to do right now.

We believe that from an ecosystem perspective, there are also many new opportunities for hardware and software to combine. Is the current phone the best hardware carrier, or will there be new and better hardware in the future? As AI technology develops, more and more modalities can be integrated — will there be new opportunities? We are actually very, very much looking forward to this.

I think an important point for us in past years of practice, and something I want to discuss with fellow entrepreneurs here, is how startups can better leverage their advantages. Often, a startup's greatest advantage, and possibly its only advantage, is actually its organization. And organizations often need to adapt to production methods. This is very suitable for new technologies, because when you have new technology, your production methods often need to change. At that time, building a new organization from zero to one can have huge advantages.

For example, large models — their development method is likely completely different from internet development methods. Internet development often involves this kind of planned development, where you set a goal of what to develop next. You rarely encounter situations where things can't be developed due to technical bottlenecks.

But large models are now more emergent. For example, you train to 10^25 floating-point operations, but you don't actually know what new capabilities will emerge at that node. You can't optimize case by case, category by category. You probably need to pay more attention to foundational capabilities and let the model emerge. It's somewhat like you're not pointing to a specific place to plant a tree, but directly buying up the entire forest — it has that kind of feeling. So when production methods differ, your organization probably needs to make corresponding adjustments.

Actually, often these new startups can instead leverage advantages in this area. We've also been exploring in this direction, hoping to continuously iterate and develop larger organizations.

At the same time, another important point is that we hope to achieve a balance between climbing stairs and enjoying the view, because model capabilities still have enormous room for improvement. We're probably still at the beginning of the industry, having just reached 10^25 floating-point operations. Next we'll likely see 10^26, 27, 28, even 29 appear. So this is actually a process of climbing stairs. Of course there needs to be balance — perhaps some commercialization and products — but we still hope to devote more energy and priority to climbing stairs, rather than just enjoying the view.

Thank you very much for listening to my sharing today. Thank you.

Host: Please stay on stage, Zhilin. Please remain for a moment. Next, we'll have a brief interaction with Zhilin. Today, as a representative of young entrepreneurs, he just shared a lot from the technical level. Zhilin Yang was born in 1993 and already holds a PhD. Now Zhilin and his partners have brought everyone their entrepreneurial content — Kimi, the large model that is now wildly popular domestically. Yang is the most-cited researcher under 35 in China's NLP (natural language processing) field. But none of this is the point. The point is coming now — this technical guy, this science PhD who has achieved academic success, his childhood dream was to be a rock singer and wandering poet. I'd like to ask Zhilin, why did you want to be a rock singer or wandering poet at that time, and today you're standing here talking extensively about large models and the future of technology? What kind of relationship exists between these two?

Zhilin Yang: We didn't rehearse this question. I think there may be some essential connections — perhaps both are related to innovation. Often, including some rock spirit, it's the same — how can we bring something new, something incrementally different? Because for example, when rock first developed, it was also a completely new genre, representing completely different ideas. In that era's context, there was also a lot of this kind of innovation, or doing new things. This kind of spirit — we do technology to a large extent in the same process. What inspired me most was still rock.

When rock was starting to develop well, it happened to be when computers were also starting to develop. When what was probably the world's largest, highest-market-cap company — IBM, as people described it at the time, said it was the sun, the moon, and the stars, basically encompassing all celestial bodies. But then you didn't expect there would actually be a new company — Microsoft, which at the time had only a few people, but they could completely overturn the market, could do very disruptive innovation. I think this may also be the direction we want to strive for.

Host: Zhilin is a very typical "kid next door" — extremely, extremely excellent. In your growth process, what core factors do you think made you who you are today?

Zhilin Yang: I think what was important for me was probably the cultivation and tolerance of innovative spirit, because often, when you innovate, it means you have to think of some very strange ideas. But I feel fortunate that whether in the Tsinghua environment before, or in the entrepreneurial environment now, I can actually get a lot of tolerance and support. I think this is quite important.

▶ More Exciting Content ◀

Founded in 2005, BlueRun Ventures is a venture capital firm focused on early-stage startups.

Currently, BlueRun Ventures manages over 15 billion RMB in assets under management, making it one of the largest early-stage funds in China. Its investment stage focuses on Pre-A and Series A rounds, covering technology, consumer, and healthcare sectors. It has cumulatively invested in over 200 startup companies, including Li Auto, Waterdrop, QingCloud, Guazi.com, Qudian, Ganji.com, Energy Monster, Gaussian Robotics, Songguo Travel, Yuntu Semiconductor, Machenike, CloudSaint Intelligence, Anxin Network Shield, and BioMap, among others.

BlueRun Ventures has been ranked #1 on Zero2IPO's "China's Top 30 Early-Stage Investment Institutions" and ChinaVenture's "China's Best Early-Stage Venture Capital Institutions TOP30," and was named one of Preqin's Top 10 Global Venture Capital Fund Managers for Sustained High Returns.

Additionally, BlueRun Ventures has received consecutive annual recognition from Forbes China, 36Kr, Cyzone, Caixin Media, CBNweekly, Jiemian, and other media organizations for honors including "China's Best Early-Stage Institution of the Year," "China's Top Venture Capital Institution," "Most Entrepreneur-Friendly Early-Stage Institution of the Year," and "Most Influential Early-Stage Institution of the Year."