Qiming Star | StepFun's Yibo Zhu: In the Agent Era, AI Infrastructure Must Optimize Intelligence, Speed, and Cost | WAIC 2026
In the Agent era, the core objective of AI infrastructure is shifting from merely reducing inference costs to simultaneously optimizing for intelligence, speed, and cost.
Qiming Venture Partners hosted the 2026 World Artificial Intelligence Conference (WAIC) forum "Qiming · Entrepreneurship and Investment Forum: From Compute Origins to Application Deployment" on July 19 at the Shanghai World Expo Center.
In the keynote session, Yibo Zhu, co-founder and CTO of StepFun, shared his latest thinking under the title The Convergence of AI Infrastructure and Model Capabilities. Zhu noted that as we enter the Agent era, the core objective of AI infrastructure is shifting from simply reducing inference costs toward simultaneously optimizing for intelligence, speed, and cost.
Yibo Zhu, Co-founder and CTO of StepFun
He argued that with token demand growing rapidly and model sizes continuing to expand, relying solely on compute scaling is unsustainable. The industry needs to push for co-design across models, systems, and chips to extract more intelligence per unit of compute. For smart terminals, infrastructure must further support edge-cloud collaboration and long-running environments, enabling real-time edge execution to work efficiently with complex cloud-based reasoning — a critical foundation for truly scalable Agent deployment.
The following is a selection from Zhu's speech.
01/
We Have Now Entered the Agent Era
StepFun is a foundation model startup. We're best known for our multimodal capabilities, but we also work on foundation language models and Agents. I'd like to share what we've learned about AI models, infrastructure, and co-design since entering the Agent era.
I like to start with a classic diagram. AI is changing very fast, but if a chart or roadmap remains relevant two years later, that says something about its value. This is the roadmap our CEO drew in early 2024. We divide AI development into three stages:
Stage one: simulating the world. We feed in massive amounts of pre-training data, and large models can chat like humans.
Stage two: exploring and learning from the world, moving into reinforcement learning. This covers late 2024 through 2025, including the reinforcement learning paradigms from OpenAI's o1 and DeepSeek's R1. Reinforcement learning has a classic example in AlphaGo — through RL it eventually surpassed its "teacher model," not just imitating it but innovating new approaches, even outperforming humans. Evolving from mimicking human speech to helping humans get things done enables the various capabilities we now see in smart terminals and Embodied Artificial Intelligence, with coding assistance being the most basic example.
Stage three: inducing the world. We expect this to happen — moving beyond simply rearranging and combining previously accumulated human skills to actually inducing new scientific laws, essentially entering the AI For Science stage. I've already seen some startups discussing this.
The bottom row shows OpenAI's influence during the same period. Their commonly cited five stages of AI development align fairly well — from chatbots to the current Agent era and subsequent innovation.
A bit more on where we are now. Using OpenAI's five-stage framework, we're currently at L3, the Agent era. Marker products include Claude Code and OpenAI's Codex starting in the second half of 2025, which can genuinely replace much routine programming work, followed by more general-purpose Agents like "Xiaolongxia" and Hermes in the first half of this year — more versatile than coding-specific tools. This basically marks the opening and current heated state of the intelligent agent era. At our company, people in HR, legal, and finance — with zero development background — each have an Agent to help with daily work, and it's remarkably efficient.
02/
Finding the Balance Among Intelligence, Cost, and Speed
Here's the iteration of StepFun's models, from Step1 to Step 3.7 Flash. The next version is coming very soon, with some achievements to show. Before reasoning models became prominent, we had Step2, one of China's earlier trillion-parameter models, which topped some leaderboards — though this was before DeepSeek's breakthrough. More recently, our Step 3.5 Flash series was our first Agent model, and it achieved very strong results. But what really stands out is its popularity with users. In March 2026, Step 3.5 Flash reached #1 on OpenRouter's global call volume ranking. Why? I'll explain the model reasons below.
This chart from the National Data Administration shows that from an industry data perspective, token demand growth far outpaces compute growth — tokens have grown by three orders of magnitude while compute has only increased several-fold. So we're still in a very tight situation. If we need to support continued doubling of tokens plus increasingly large model sizes, the infrastructure challenge grows ever greater. On one hand, we certainly hope the semiconductor industry can push further. On the other, model design and software design need to squeeze more intelligence out of each unit of compute to meet demand.
Extracting intelligence from compute is a dimension of cost-effectiveness. But the Agent era adds another dimension — from the dilemma of wanting both high intelligence and low cost, it becomes an impossible triangle, with speed as the third vertex. On the right is the article we published on Zhihu when we launched Step 3.5 Flash, which got over a thousand upvotes from peers. Thanks for the support.
The logic is simple — in the chatbot era, our models only needed to exceed 20 tokens per second, because that's about human reading speed. On a chatbot, if you ask a question and get hundreds of characters back, you can't actually read it all. But now in the Agent era, if I assign it a programming task or other work, I don't watch the model's thinking process. I want the final result delivered as fast as possible — ten seconds rather than a minute. This is why Step 3.5 Flash was so popular: it's fast. So on both the model and software sides, speed becomes a new dimension, and it's discontinuous — unlike before where exceeding 20 tokens didn't make much difference.
Here's a look at our Flash model. It's not the most intelligent model in absolute terms — that basically belongs to OpenAI and Anthropic. But on Artificial Analysis, the evaluation leaderboard people frequently check, when you add the dimensions of speed and cost, we basically rank first or near the top. This is why we achieved such high call volume during that period in the Agent era. Technology keeps evolving; speed advantages get chased down, intelligence needs to iterate with new models step by step. The only constant in this industry is change, and we'll keep pushing forward.
Continuing the topic — I like drawing parallels with the past. The left image shows our Step3 model from July 2025, with what we then considered the most important chart. The right image carries over from the previous slide. The axes show optimization targets for different eras. In the inference era, or chatbot era, I cared about model size; the x-axis was cost, and I wanted good cost-effectiveness. Through model architecture design, software optimization, and adaptation for domestic chips, we showed solid results.
The right chart shows our current optimization target for Step 3.7 Flash: the y-axis is intelligence benchmark scores, the x-axis is model output speed. In some scenarios this creates tremendous value and resonates with users.
One specific technical example. We recently published a paper on "speculative sampling to accelerate model inference" — speculative sampling means that instead of generating one token at a time, the model gambles on ten tokens. If it guesses right, it wins big; if wrong, it falls back to one token. But with some probability of guessing right, it might output many tokens at once, raising average speed. DeepSeek published almost simultaneously on this — great minds think alike. We saw the same critical technical point, with fairly similar approaches, though we tuned more aggressively. Through model-system co-optimization, we achieved up to 9.64x acceleration in relevant tests. This delivers real value for Agent deployment in production, enabling single-card output of over 1,000 tokens per second for individual users under specific conditions. Compared to the previous generation's roughly 30 tokens/s deployment, this brings order-of-magnitude speed improvements for specific Agent tasks.
03/
Edge-Cloud Collaboration Delivers the Best User Experience
A few points of consensus and non-consensus. We've always believed in multimodality — Agents need it too. Some now think coding is enough, but I believe that beyond coding, serving more users requires multimodality, because the most efficient human input is visual, and the most efficient output for me is voice. If instead of speaking today I just handed you a sheet of paper to read, would that be efficient? You can imagine current Agents interacting with users by essentially printing you a paper to read; if you want to talk to it, you have to type. This isn't actually the most efficient human interaction mode, so we've persisted with multimodality. Our distinction is that we focus on visual understanding and real-time voice dialogue, where we've built up a fairly complete model matrix that also supports the terminal strategy I'll discuss.
For example, we recently released a new voice model that has surpassed Gemini 3.1 Pro in ASR speech recognition, approaching its larger variant, with clear leadership in smaller sizes. So we've accumulated substantial capability in voice — you can experience this model in our exhibition hall, though it's still in preview and will improve further.
With these multimodal capabilities in place, we can support the strategy we've pursued since 2024: AI terminals and AI hardware. But this poses considerable infrastructure challenges. At a high level, we believe AI terminals need an edge-cloud collaborative approach to deliver good user experiences. I can't just use a small model on-device that's a hundred times smaller than cloud models — the intelligence gap is too large, and privacy also matters. So we believe pure cloud or pure edge approaches struggle to simultaneously balance intelligence, real-time performance, and privacy; edge-cloud collaboration is more likely to become the mainstream solution for AI terminals.
This does create infrastructure challenges. On one hand, models need to come in full sizes — both cloud and edge versions. But infrastructure itself also needs edge-cloud collaboration. For example, some people recently installed Xiaolongxia and bought related products, paying several dozen yuan monthly for cloud virtual machines, with tokens costing another few dozen yuan monthly. That's not very acceptable for ordinary consumers, so there's substantial infrastructure optimization to do, with additional complexities involved. This includes training such edge-cloud collaborative models. If edge and cloud models need to cooperate on tasks, that requires reinforcement learning too — RL needs to simulate a bunch of phone environments so the model can practice completing edge-cloud collaborative tasks and learn the reward function. Infrastructure like this didn't exist before and isn't mature, so it needs to be built.
At WAIC we also released our edge-side models — actually not one model but a series, based on our understanding of edge hardware. Since we also serve many automakers, we used this understanding to build what we consider the most important models to run on-device: for example, a model controlling your phone screen buttons should run locally — users won't tolerate cloud latency for that; low-latency voice also belongs on-device. Based on this understanding we released edge models that also rank first on benchmarks, and most importantly they'll power our future terminal products.
To summarize: In the Agent era, infrastructure needs to do much more than just serve as compute infrastructure behind models. It's becoming new infrastructure that supports Agents' long-running operation, continuous learning, and entry into the physical world.
PAST HIGHLIGHTS
Qiming Weekly | 10 News Items from Qiming Venture Partners Portfolio Companies Vol.25, 2026.8.11
Founded in 2006, Qiming Venture Partners currently manages 11 USD funds and 7 RMB funds, with total assets under management reaching $9.5 billion. Since its inception, the firm has focused on investing in outstanding early and growth-stage companies in Technology and Healthcare.
To date, Qiming Venture Partners has invested in over 580 high-growth innovative companies, of which more than 210 have listed on the New York Stock Exchange, NASDAQ, Hong Kong Exchanges and Clearing Limited, Shanghai Stock Exchange, and Shenzhen Stock Exchange, or exited through M&A and other means. Over 80 portfolio companies have become recognized unicorns or super-unicorns.
Many Qiming Venture Partners portfolio companies have grown into the most influential companies in their respective fields, including Xiaomi (01810.HK), Meituan (03690.HK), Bilibili (NASDAQ:BILI, 09626.HK), Zhihu (NYSE:ZH, 02390.HK), Roborock (688169.SH), Hesai Technology (NASDAQ:HSAI, 02525.HK), UBTECH (09880.HK), WeRide (NASDAQ:WRD, 00800.HK), HyperStrong (688411.SH), Insta360 (688775.SH), Unisound (09678.HK), Biren Technology (06082.HK), Zhipu AI (02513.HK), Gan & Lee Pharmaceuticals (603087.SH), Tigermed (300347.SZ, 03347.HK), Zai Lab (NASDAQ:ZLAB, 09688.HK), CanSino Biologics (688185.SH, 06185.HK), Schrödinger (NASDAQ:SDGR), MicroPort EP MedTech (688617.SH), Sanyou Medical (688085.SH), Amoy Diagnostics (300685.SZ), SinoCellTech (688520.SH), Insilico Medicine (03696.HK), Hohobio, Yuanxin Technology, MediLink Therapeutics, LaNova Medicines, StepFun, and others.