Faster! Even faster! Fast enough to generate the world in real time | A chat with Shengshu Technology's Jintao Zhang on Vidu S1, inference acceleration, and real-time interactive video

🚥 Nobody likes waiting.

The more powerful the model, the more important inference acceleration becomes.

In this episode, we talk about "inference acceleration."

This week's guest on Crossing is Jintao Zhang from Shengshu Technology. Earlier this month, he led the release of Vidu S1 — a real-time interactive video model: upload a photo, whether of a person, an anime character, or even a pet, and it becomes an AI character that can video chat with you in real time.

Vidu S1 is fast — so fast that its generation speed exceeds playback speed, and it supports unlimited duration. How was this achieved?

On the podcast, Jintao shared a clear technical thread he's followed over the years: from SageAttention (3.5K GitHub stars), TurboDiffusion (3.6K GitHub stars), to TurboServe — a complete system-level solution that provides the underlying infrastructure for S1's real-time interactivity.

Beyond that, Jintao also shared his observations and judgments on the entire AI "inference acceleration" field.

Actually, what's interesting about this episode isn't just the technology.

Jintao is 26 years old this year, still a PhD student at Tsinghua University. What truly drew him into "inference acceleration" was a remarkably simple judgment — when everyone assumed FlashAttention had already pushed Attention operators to their limit, he discovered: there were clearly faster compute units on the GPU that weren't being utilized — "Such obvious gains, why isn't anyone doing this?"

So you can also treat this episode as a "young researcher case study": in the rapidly changing AI field, how do you find your own direction, how do you judge what is "the most correct thing," and how do you carry it all the way to the cutting edge.

🎬 Our video podcast will be simultaneously released on Koji's WeChat Channels, Douyin, Xiaohongshu, Bilibili, YouTube, and other platforms.

📒 The transcript will be published on the Crossing WeChat official account.

🟢 01:40 Lightning round: age, alma mater, MBTI and zodiac sign, Vidu S1, and his responsibilities at Shengshu Technology

🟢 02:48 There were clearly faster compute units on the hardware

"Why isn't anyone doing this? This is such obvious gains."

When people believed FlashAttention was already near optimal, Jintao Zhang discovered that GPUs still had faster compute units not being used for Attention.

Early language models' Attention was more constrained by memory bandwidth, while video generation models are severely constrained by compute speed.

SageAttention had to solve both low-level GPU kernel programming and the precision loss from low-bit computation, ultimately becoming a plug-and-play acceleration solution.

🟢 06:15 From SageAttention to Vidu S1: three layers of acceleration

Operator layer: push Attention, linear layers, and other computations as close to hardware limits as possible.

Model layer: through step distillation and sparse Attention, reduce Diffusion denoising from roughly 50 steps to 4 steps.

Deployment layer: solve multi-GPU parallelism, overlapping communication with computation, and user request scheduling.

SageAttention later became the de facto industry standard; TurboDiffusion advanced model acceleration, while TurboServe handled streaming video cluster deployment.

🟢 11:24 Vidu S1 is more than just fast

"Generation speed needs to exceed playback speed for it to achieve real-time generation."

Ordinary video models make you wait a long time to generate a complete clip, while streaming models must generate frame by frame based on user input.

Vidu S1 achieves 540P at 25–42 FPS, and can run on consumer-grade GPUs.

Harder than speed is unlimited duration: after generating for one or two hours, the image still can't drift or collapse, and must continue responding correctly to the user.

Real-time interactivity is the primary goal; at the current stage, some image quality can be sacrificed, but speed cannot.

🟢 15:21 "In the future, our visual entertainment signals will be flooded with AI-generated content."

Movies and short videos are pre-generated offline content, while chatting, dating, gaming, and daily life are full of real-time visual interaction.

Offline video can be shared during playback; real-time video needs to generate different content for each person.

Jintao Zhang judges that demand for real-time interactive visual entertainment will exceed that for offline content, and this trajectory is already unstoppable.

🟢 17:30 A dog and a gaming buddy

Users have already started uploading pet photos to chat with them, and some have let anime characters watch their gameplay in real time and play games with them.

Vidu S1 can understand input video streams; feed it your computer screen, and it can become a "programmer cheerleader" that understands Coding, WeChat, and Notion.

🟢 20:05 Inference acceleration is more than just operators

Beyond operator acceleration, sparse Attention, MoE, and distillation are needed to reduce the model's own computational load.

Streaming Diffusion introduces KV Cache, and must also handle users joining and leaving at any time, as well as multi-machine multi-GPU cluster scheduling.

TurboServe corresponds precisely to the Serving layer: combining computation graphs, communication, and request scheduling into a truly deployable system.

🟢 24:39 The future of inference acceleration belongs to the lower and higher layers

"The future of inference acceleration still belongs to the lower and higher layers."

The middle operator layer will gradually converge as programming tools improve; the lower layer involves designing chips for general or specific models, while the higher layer uses algorithms to directly reduce computation.

Companies like Taalas that do hardware design for specific models are essentially trading off between generality and ultimate efficiency.

Truly defensible acceleration requires hardware-software co-design: knowing only algorithms makes it difficult to judge whether a solution will actually work on real hardware.

🟢 28:28 Why China leads in video models

After Transformer, model architectures gradually converged, and gaps increasingly came from data volume, data quality, preference alignment, and how learnable the data was.

Low-quality data isn't just useless — it contaminates the model; high-quality explanations let models learn key concepts faster.

China's short video, livestreaming, and e-commerce ecosystem generates both real demand and richer video data; Jintao Zhang believes Chinese video model companies currently lead the United States.

🟢 33:10 World-leading means getting every link right

"Knowing what the most correct, most leading method in the world is, and then implementing it correctly."

Jun Zhu helped him understand that GPT-3's success didn't come from some mysterious trick, but from pushing every link to its extreme.

Two lines of buggy code are enough to make an experiment fail; the person in charge must be able to judge whether the approach is correct and whether the implementation is sound.

Leading a team also means seeing what each person truly wants — salary, credit, or a sense of accomplishment — and coordinating different departments from first principles.

🟢 37:54 One line of code, twice the speed

"Video generation had to be pixel-for-pixel identical to before, but end-to-end inference speed more than doubled."

In a quiet building at Tsinghua, he traced through the underlying issues thread by thread until SageAttention finally worked.

Replacing FlashAttention with SageAttention in Vidu required changing just one line of code; the output remained pixel-for-pixel identical, yet end-to-end speed more than doubled.

He immediately sent the results to Jianfei Chen and Jun Zhu; he later led the team responsible for Vidu inference and S1, describing this experience as "like starting a company."

This happiest period of research also came from the direction, resources, and atmosphere of freedom that Jun Zhu provided.

Welcome to subscribe to Crossing: 🚦 We follow the new industry changes and entrepreneurial opportunities brought by the new wave of AI technology.

🚦 Crossing is a metaphor Steve Jobs used for Apple, describing it as standing at the intersection of technology and the humanities, where great products are born. AI is bringing change to every industry; we seek out, interview, and bring together a new generation of AI entrepreneurs and active participants in the AI era, exploring and embracing new changes and new possibilities together with them.

👦🏻 Host Koji: I founded Crossing, launched AI Hacker House — a community space for a new generation of AI entrepreneurs, and serve as Venture Partner at ZhenFund. I believe technology, especially AI, is the greatest value creation opportunity of our generation. Koji on Jike, Koji's website

View this episode's transcript on Xiaoyuzhou