A Multimodal Symphony: How Can Technology and Products Advance Together? | BlueRun Ventures x Volcano Engine Event Recap

On May 15, BlueRun Ventures | Buming Entrepreneurship Camp and Volcano Engine | V-START Accelerator co-hosted the "AI Frontier, ATRenew — Intelligent Agents and Multimodal Industry Forum," focusing on the technical evolution and entrepreneurial practice of intelligent agents and multimodal AI.

This was a curated industry forum co-created by one of China's largest early-stage funds and its most advanced AI major, bringing together frontline practitioners from multiple ByteDance business lines — including Seed, Lark, Dreamina, Trae, and Volcano Engine — alongside several BlueRun Ventures portfolio companies in the AI space, such as Muyan Zhiyu, Yuaiweiwu, VITURE, and RockFlow. The event also attracted guest speakers from star companies including Zhipu AI and Yingmo Technology. We're pleased to have created this venue for diverse perspectives, where we confronted the industry's most pressing issues at this inflection point of rapid technological evolution through high-quality discussion. The venue, originally set up for 80 seats, was packed to capacity with nearly 140 attendees from AI startups and ByteDance — mostly founders and C-suite executives from startups, plus ByteDance algorithm and product specialists — who engaged in deep discussions during the free talk session after the formal program.

Tao Yeliao, Investment Director at BlueRun Ventures

Following this, Tao Yeliao, Investment Director at BlueRun Ventures, moderated a roundtable on multimodal AI. Mei Yuan, Product Lead for Dreamina Story Agent; Sun Yi, Strategy at Dreamina; Eddy, Product Director for Multimodal Models at Zhipu AI; Zhang Xinhao, Product Lead at a leading AI startup; and Zhang Qixuan, CTO of Yingmo Technology, participated in the discussion. They shared in-depth reflections on key issues including technical evolution, product definition, and scenario deployment.

Mei Yuan: For Dreamina, enhancing the model's perceptual capabilities is the core objective. Only when the model can truly understand complex real-world information can it make decisions that more closely approximate human thinking, achieving more intelligent and natural interactive experiences. This will transform Dreamina from a simple tool into something more — a "friend" or "assistant" with greater personification and companionship, better serving user needs.

Mei Yuan, Product Lead for Dreamina Story Agent

Sun Yi: Setting aside paradigm shifts in technology, and looking purely at model output quality, image generation is currently roughly at the stage between GPT-4 and O1, while video generation remains relatively early — approximately at the GPT-3.5 to 4 level.

Using language models as a reference, model capability evolution can be divided into three stages: initially just "playable" (GPT-3.5 mainly enhanced conversational ability), then capable of completing task "fragments" (GPT-4 began developing practical functions like code generation), and finally able to "deliver complete tasks" (O1 and subsequent models gained end-to-end capability for complex task delivery).

Image generation has followed a similar path: from Midjourney V4 just being able to generate images, to V5/V6 achieving photo-level quality, to current models (with improved reasoning and generation capabilities) being able to deliver complete design solutions such as posters, product images, and marketing visuals.

Video generation currently remains in the "fragment generation" stage. While models like Runway Gen-4 and Veo2 are advancing toward high-quality shot generation, achieving the integrated "understand-generate-orchestrate" delivery capability seen in image design still requires further breakthroughs in model capabilities.

Going forward, video generation will likely depend on a native multimodal model with strong world knowledge and visual understanding capabilities — whether serving as the "brain" for video creation or directly handling generation tasks itself — to truly support end-to-end applications.

Sun Yi, Strategy at Dreamina

Eddy: Different business scenarios actually demand very different capabilities from video models. Film imagery needs to be big-screen ready, with characters that convey emotion and performance, and price sensitivity is relatively low. Short dramas prioritize scale and cost control for high-volume production in short timeframes, while requiring extreme emotional expression. Advertising cares more about refinement and consistent brand logo presentation. Marketing content has particularly high demands for lip-sync accuracy and inference speed. The gaming industry leans toward real-time interactivity, where products require high real-time performance, making autoregressive models more suitable. As for whether the end form becomes a tool, a community, or something else? I believe the first step is to achieve SOTA in at least one domain.

Zhang Qixuan: From a technical perspective, early multimodal AI typically involved single-modality input corresponding to single-modality output. But it was later discovered that single-modality output couldn't satisfy complex needs, requiring more multimodal input for more precise control over output. Even so, previous methods still fell short in ensuring AI generation fully followed instructions. This year, GPT-4o offered a new answer: not just multimodal input, but multimodal output as well, which can significantly improve AI's ability to understand and execute instructions. While GPT-4o didn't show explosive improvement in image generation quality, it made substantial progress in output controllability, allowing non-professional users to obtain more satisfactory images without complex operations. I believe this represents the major trend for multimodal convergence in 2025: enabling more users to escape complex workflows and directly obtain desired results through natural interaction. Currently, language is undoubtedly humanity's most mature and powerful modality, with the largest foundation models, best performance, and broadest audience, so it's natural for everyone to use language as a foundation. But previously, aside from language models primarily using GPT architecture, other modalities like image and video mostly relied on Diffusion or Diffusion Transformer architectures. Next, we'll likely see an important migration trend where various modalities converge toward autoregressive (AR) architecture, including the 3D domain moving in this direction. Looking back at GPT-4o's initial demo video, it already demonstrated image generation capabilities, even 3D generation demos — though at the time people assumed these were composite multi-image effects created with other tools (like Daily 3D). This may actually indicate it already possessed multimodal generation capabilities, just not yet at production-level maturity — a direction we're very much looking forward to.

Zhang Xinhao: In 2024, the industry generally equated multimodal directly with multimodal generation. But over the past six months, as reinforcement learning has become a prominent technical path, multimodal development has begun to converge. I believe a more accurate description is multimodal understanding and generation. This is because the core issue is how we understand multimodal content or the products of multimodal generation. If we simply view multimodal generation as complete correlation between different modalities, then a pure generation route might work fine. But if we acknowledge that causality exists in the multimodal world — whether between frames in video or causal connections between pixels in a single image — then causality becomes crucial for multimodal generation. Understanding inevitably requires language. Because whether it's human understanding of causal relationships between things or causal reasoning about images, some mapping to language logic underlies it all. So as long as we recognize that multimodal content is constituted by causal relationships, and that understanding these causal relationships heavily depends on a language foundation, I believe this will become a very clear trend in 2025.

Mei Yuan: In our daily evaluation of multimodal large models (especially for video generation and creative tasks), we encounter several clear challenges. First is the subjectivity of evaluation. Dimensions like aesthetics and emotional expression inherently lack unified standards and carry strong human bias — different strokes for different folks. Even the quality of literary works is judged completely differently by different people, not necessarily aligning with posterior data distributions. So what we're doing is: trying to quantify these more perceptual dimensions through tools and processes as much as possible. We've found this human-machine collaborative approach effective, for example: left-brain-ifying the "right-brain work" in the creative process; using models to play the "left brain" role — conducting research, reasoning, and planning; while humans provide the inspirational backbone and direction. Second is building an evaluation standard system. For each specific task, we conduct detailed metric breakdowns: splitting a large goal into multiple quantifiable small steps; incorporating expert opinions to summarize some generalizable "patterns"; some content can even be directly used for model training after verification. So when we evaluate whether a model is good, we don't rely on a single standard, but combine task characteristics, expert knowledge, and process breakdown. This evaluation system is also an important direction our team continuously optimizes internally.

Sun Yi: First, evaluation tasks for multimodal models themselves have a "difficulty gradient": for video generation, at the basic level, it's actually relatively easy to evaluate: Is the motion stable? Does it conform to basic physical laws? Is there subject distortion? These dimensions are relatively clear, describable, and more objective to evaluate. But as model quality improves and user expectations rise, many evaluation dimensions start exceeding ordinary people's cognitive boundaries. Here we see a potential solution: introducing high-quality samples as exemplars, rather than setting hard standards. These can serve as aesthetic templates, allowing models to learn aesthetic patterns through RL and optimize through imitating excellence.

Eddy: In our past experience making films and content, we actually had a set of judgment criteria — beyond the semantic dependency and physical plausibility that technical teams care about, we focused more on: whether lighting and subjects are reasonably presented in the frame; whether character performances are authentic and expressive; whether overall image achieves general aesthetic standards in dimensions like color, light and shadow, and composition; and the model's performance capabilities across different genres and scenarios — such as whether it's skilled at expressing emotion, and whether it can maintain coherence in suspense, family drama, and other styles. This set of dimensions abstracted from creative practice can actually effectively guide us in evaluating models more systematically. Therefore, in our internal evaluations: we prioritize quickly judging model performance through this creative-perception dimension system; then supplement with some industry-recognized benchmark evaluation systems to ensure results have generality and reference value; while also continuously monitoring industry developments, such as the evolution pace of open-source models like Pika, Runway, and Open-Sora, to see if there are evaluation methods or directions worth learning from. Overall, we believe: crystallizing industry scenarios and frontline creative experience into structured know-how, then feeding it back into model training, is a more realistic and efficient approach. This "content nourishing model" path helps us move faster and more steadily.

Eddy, Product Director for Multimodal Models at Zhipu AI

Zhang Qixuan: The model evaluation problem far predates the large model era — it was already exposed in traditional recognition and classification tasks. Early on, everyone ran ImageNet rankings and scored. But when model performance gradually surpassed human annotation accuracy on ImageNet, a phenomenon emerged where "scores went up but performance went down." So basically no one seriously runs ImageNet leaderboards anymore. Entering the generative model era, evaluation became even thornier. We felt this acutely working on 3D generation: early tasks were 3D reconstruction, which was relatively easier to evaluate — for example, computing errors between projected results from different viewpoints and original images; but entering true 3D generation, complexity increased: you don't even know the camera intrinsics and extrinsics of the input image, making it impossible to precisely measure generation result plausibility; we initially tried using external models like CLIP for auxiliary evaluation, computing similarity between generated results and input images or text; but soon found this path faltering: in 2023 when we built Clay (a 3D foundation model), we discovered that CLIP and similar proxy models' judgment standards began failing, unable to stably support evaluation logic. So now, our basic approach is: completely shifting to real user feedback as the evaluation basis. The specific method: quietly adding A/B tests in the product, rolling out new models in different groups, and seeing which model gets higher user "confirm rates." Put simply — we no longer believe in any universal metrics, only looking at real user preference performance. This is the current state of evaluation in 3D multimodal: no standard metrics, only user behavior closed loops.

Zhang Qixuan, CTO of Yingmo Technology

Zhang Xinhao: Evaluation metrics themselves shouldn't be the goal; instead, model evaluation logic should be defined in reverse from user scenarios. For example: in certain application scenarios, users may not care about resolution or aesthetics, but rather about controllability; while in other scenarios, users particularly care whether a video can create a specific atmosphere, or whether the model truly understands the content. We don't optimize models to chase certain scores, but rather start from user needs, making evaluation logic scenario-based and task-based.

Mei Yuan: We've indeed observed much technical progress recently, with models basically updating monthly. But my view is that unless there's a particularly major breakthrough, there's no need to chase every minor technical upgrade. Even with current general model capabilities, we're still far from reaching the ceiling in terms of product-side exploration. For example, our work on Agents this year has shown that a new technical framework can further unlock model potential and open new market space in applications. So I believe, first, when building products we need to think: under current product demands, how to maximize existing base model capabilities. Second, we need some judgment about technology development cycles and directions. For instance, even if general models advance quickly, they still can't cover many vertical PMF scenarios. Third, from a product perspective, understanding user needs, exploring AI Native product paradigms, and building new user mental models is more important. We hope to pioneer not just existing markets, but to uncover currently unmet market needs through product-user interaction. This is also what we've accumulated and refined through this process: understanding of users, scenarios, and commercialization pathways, as well as building user stickiness and emotional connection. These aren't entirely constrained by general model capability evolution, but are directions worth investing in early and continuously. AI video creation is a blue ocean market. In the near term, we hope to first build more accessible tools, lowering the barrier for the entire AI creation chain; as tool capabilities strengthen and user numbers grow, we'll have opportunities to discover differentiated content that can only be created with AI. But if we're just making tools, we'll always remain one link in the production chain, replaceable. So from a longer-term perspective, we hope to build an AI community, getting more users involved in the "play" process, further establishing user dependence on the platform and product through their interactions and social engagement.

Sun Yi: Actually, building tools and building communities isn't an either/or question. The core of community is whether you can gather a certain type of people and increase the concentration of a certain type of content. This determines community atmosphere. For example, Douyin is a super large platform, but product managers still use Jike because Jike is full of product managers with highly relevant content, making it a product manager community. So community depends more on your deep insight into certain user needs and content forms, and the density you create. Tools differ — they focus more on your depth of understanding and cognition of a specific creation or production chain. Therefore, I think the key is returning to the entrepreneur yourself: which aspect do you truly have deep insight into? For our team, we're actually exploring both tools and communities in parallel.

Eddy: The key is that current video models are still in rapid development. It's like a six or seven-year-old child — you might call them a genius who answered one math problem very well, but overall they're still in the babbling stage, with considerable distance from the industry's real practical needs. So I lean toward the strategy: wait until it truly demonstrates capability no less than human creators in a specific domain or task type, then find a new product form to land. This form might be: a product path that combines both tool capabilities and inherent entertainment genes.

Zhang Xinhao: I think building products requires abandoning illusions. Many product-level features will eventually be covered by stronger model capabilities. The past one to two years of development has continuously validated this trend. But even so, we shouldn't stop exploring products. I believe that even making some temporary differential advantages, being able to better satisfy certain user needs than current models, is already tremendously valuable. Especially in the large model era, if you too easily fall into endgame-oriented thinking, you'll feel like nothing can be done. But if we let go of this fixation and start from "what problems can I solve for users right now," we can always find product breakthrough points.

Zhang Xinhao, Product Lead at a leading AI startup

Whether it was Manus earlier or Lovart that just came out, they're continuously validating this logic: as long as you find the current gap between models and users, and fill it with product features, that's a great opportunity.

BlueRun Ventures Dialogues with Moonshot AI and Muyan Zhiyu: Standing at the Crossing of the AI Industry | The Road to AGI

BlueRun Ventures Family | VITURE, Which Just Completed a $50 Million Funding Round, Says: "AI and AR Will Eventually Converge into a New Generation of Multimodal Devices"

Multimodal DeepSeek Is Here! Yang Yaodong's Team Collaborates with HKUST to Release Align-DS-V, Supporting Lingchu DS-VLA Alignment Extension | BlueRun Ventures Family Headlines