MiniMax's Hailuo AI Open-Sources VTP for the First Time, Cracking the Scaling Law for Visual Tokenizers

**

Where Many Teams Have Hit a Wall, a New Direction Emerges

👦🏻 Author: Jingshan

🥷 Editor: Koji

🧑‍🎨 Layout: NCon

Over the past year, one statement has generated particularly heated debate:

"Diffusion is simply better than VAE."

It's not hard to see why this caught on. For image generation tasks, diffusion models have delivered visibly superior results — richer detail, greater stability. And as large models and massive datasets became the norm, that advantage only grew.

But look at actual products in the wild, and the picture gets more complicated.

Mainstream models like Stable Diffusion never really "abandoned" VAE at all. They're VAE + Diffusion hybrid systems: the VAE compresses images into a latent space, and the diffusion model handles generation within that space.

This division of labor remains the dominant approach in both academia and industry.

The same pattern shows up in world-model research too.

Whether it's Google's Genie-3, Tencent's HunYuan-GameCraft, or Mirage AI, their technical architectures are all understood to rely on VAE structures.

Against this backdrop, VAE retains considerable stature in research and industry circles, with substantial resources still flowing into it.

🚥

On December 17, a technical project was open-sourced:

The Hailuo AI video team at MiniMax "open-sourced for the first time" its VTP (Visual Tokenizer Pre-training) project.

They simultaneously released a notably rigorous paper. Its most intriguing contributions boil down to three points:

🚥

[1] "Better reconstruction can mean worse generation" — the conventional VAE intuition is wrong

[2] What truly drives generation is "understanding capability," and Visual Tokenizer is highly useful for this.

[3] VTP is the first to establish a Scaling Law for Visual Tokenizer (this is the most important).

In short: The default scaling path for visual generation that everyone had assumed was valid may itself be flawed, and they've proposed an alternative route.

Has the Traditional VAE Broken Down?

To understand VTP's innovation, we need to examine the conventional VAE first.

In two-stage generation frameworks like LDM and DiT, the first stage's visual tokenizer (typically a VAE or AutoEncoder) compresses images into latent representations, while the second-stage diffusion model learns generation on that latent space.

Under the traditional paradigm, one assumption has gone largely unchallenged:

The better the first-stage compression (the more closely reconstructed images match originals), the better the second-stage generation should be.

However, through extensive experiments, the Hailuo team presents a counterintuitive finding in their paper: This positive correlation not only fails to hold, it may actually be negative.

The conventional approach: tokenizer pre-training using only reconstruction loss. The paper identifies this as the pre-training scaling problem.

Put simply, if you throw more compute and data at tokenizer training for reconstruction, reconstruction metrics keep improving (images look more like originals), but this does nothing for downstream diffusion model generation metrics — it may even make them worse.

Here's an analogy: It's like a student who memorizes a text so perfectly they can recite every punctuation mark and stroke — excellent reconstruction — but completely fails to comprehend what the passage actually means — poor semantics — and can only regurgitate without true understanding.

The root cause, they argue: reconstruction objectives are heavily biased toward low-level details (texture, edges, pixel-level error), while generation quality fundamentally requires more "compact" high-level semantic structures in the latent space.

See Figure 4 in the paper for the visualization of this phenomenon: as training FLOPs increase, rFID (reconstruction) improves, but gFID (generation) degrades.

The one-sentence takeaway: a tokenizer shouldn't be treated as a "compressor," but rather as a "representation learning model."

The VTP Framework

So if pure reconstruction pursuit is a dead end, what's the way out?

The Hailuo team's answer is elegantly simple: Understanding capability is what truly drives generation.

To achieve this, they propose the VTP (Visual Tokenizer Pre-training) framework. This isn't merely a new model architecture — it's a new conceptual approach.

The related project has been open-sourced on GitHub: https://github.com/MiniMax-AI/VTP

The arXiv paper link: https://arxiv.org/pdf/2512.13687

In essence, VTP aims to evolve the Visual Tokenizer — not just to see images clearly, but to read and comprehend them.

The Hailuo team lays out the full pipeline clearly:

For architecture, VTP makes a comprehensive shift to ViT (Vision Transformer).

This is a bold choice, since ViT tends to be less stable than CNN for pixel-level reconstruction tasks. To put it bluntly: when GAN Loss (adversarial loss) combines with ViT, numerous problems emerge, notably training instability.

To address this, VTP employs a two-stage training strategy:

[1] Pre-training Stage

Joint optimization of all objectives, but without GAN Loss — focusing on L1 Loss and Perceptual Loss.

[2] Fine-tuning Stage

Freezing the tokenizer backbone, training only the decoder (Pixel Decoder), with GAN objectives introduced to enhance image quality.

This design leverages Transformers' stronger representation learning capabilities while sidestepping their instability in detail generation.

To summarize, VTP's two-stage training deliberately separates two distinct goals:

Stage One: Teach the tokenizer "how to see the world"

Stage Two: Then teach it "how to paint the world beautifully."

Consider a child learning to draw:

Under traditional methods, the first step has the child directly copy artwork, mastering details and shadows through imitation. But at the second step, when asked to draw "a cat" or "a dog" from imagination, they're completely lost.

VTP's two-stage approach: first expose the child to many images to recognize and understand what cats and dogs look like; only then gradually develop their rendering skills.

And this possibility of "gradual improvement" is the core innovation.

VTP's centerpiece is its loss function, which no longer relies solely on reconstruction loss but introduces three-dimensional joint optimization objectives.

Composed of three components:

REC: "Don't forget all the details"

SSL: "Spatial structure and local semantics"

CLIP: "Global semantics + cross-modal alignment"

VTP draws on CLIP's approach, aligning visual features from the tokenizer with text features — effectively telling the model "this is a cat" versus "this is a dog."

Here's an engineering detail worth highlighting.

These three tasks actually have conflicting batch size requirements during training. CLIP demands enormous batches (e.g., 16k) to ensure sufficient negative samples, while reconstruction tasks need only smaller batches (e.g., 2k).

So VTP designs a sampling strategy: within one large batch, all samples participate in CLIP training, while random subsets are drawn for SSL and reconstruction training.

This engineering solution makes multi-objective joint training feasible.

Establishing the Scaling Law for Visual Tokenizer

What the paper most wants to prove isn't "this method edges out another by a small margin" — it's that they've finally produced scaling curves for tokenizer pre-training analogous to those of large models.

This is also what most excites the technical community about VTP: it establishes the Scaling Law for Visual Tokenizer.

In the deep learning era, if a method can't scale, its potential is inherently limited. Traditional VAE was stuck here — more compute, no improvement.

VTP's experimental data demonstrates that once "semantic understanding" is introduced, the Scaling Law becomes attainable.

This part gets somewhat technical.

With reconstruction-only training, scaling up becomes useless or even counterproductive. As shown in the table below, rFID improves from 2.0 → 0.5, but gFID worsens from 55.04 → 58.56.

Let's revisit this figure:

Here's the critical finding.

Add "understanding tasks (CLIP+AE / SSL+AE)," and understanding and generation improve together.

Whether CLIP or SSL, injecting semantic representations causes generation vs. understanding to move in the same direction, with continuous improvement as FLOPs scale.

This curve is arguably among the most aesthetically satisfying in the entire paper. It demonstrates that understanding capability (Linear Probe Accuracy) and generation capability (gFID) are positively correlated.

Moreover, under equivalent compute budgets, CLIP+SSL+AE achieves the best ceiling, with strongest generation and understanding metrics.

This may mean that going forward, it's no longer a matter of luck. As long as tokenizer semantic understanding can be improved, downstream generation quality will likely rise accordingly.

I also spotted a particularly crucial figure: VTP has a "new curve" that traditional AE lacks.

As the Encoder scales larger, VTP's gFID steadily decreases (improves), while AE remains flat. With more data, VTP improves markedly, while AE shows virtually no benefit.

Even decoder scale yields additional improvements:

The paper provides one striking number: without modifying standard DiT training configurations, simply scaling up tokenizer pre-training FLOPs, VTP delivers 65.8% downstream generation improvement, while traditional AE saturates at merely 1/10 of those FLOPs.

What does this indicate?

It means we can now directly trade increased compute during pre-training for improved generative model performance.

The Scaling Law has arrived.

Beyond theory, what are VTP's actual results? See the figure below:

On standard ImageNet benchmarks, VTP achieves SOTA-level performance:

[1] Zero-shot accuracy: 78.2%

This demonstrates strong image understanding without any fine-tuning.

[2] Reconstruction quality (rFID): 0.36

Showing that introducing semantic understanding doesn't sacrifice reconstruction precision — rather, the two reinforce each other.

[3] Compared to distillation-based methods (e.g., VA-VAE), VTP achieves 4.1× faster convergence during generative training.

This translates to substantially reduced time and monetary costs for training generative models.

Overall, VTP demonstrates strength across understanding (zero-shot/linear probe) + reconstruction (rFID) + generation (FID).

The industry has previously attempted to imbue VAE with semantics, for instance by distilling features from pre-trained models.

One ICLR 2026 submission on OpenReview notes this approach:

But the Hailuo team argues that this "imitation" approach has a ceiling.

Their paper shows that VTP's ground-up joint pre-training achieves substantially higher performance limits than distillation methods.

Another alternative is directly using off-the-shelf DINO or CLIP features for generation.

But VTP's comparative experiments reveal that lacking joint optimization on the decoder side, this approach tends to produce color shifts and texture loss in reconstructed images. VTP maintains significant advantages in color accuracy and fine texture preservation.

🚥

Taken as a whole, MiniMax's Hailuo AI video team open-sourcing this work matters beyond any specific codebase.

At the academic level, the paper demonstrates that Visual Tokenizer should focus not merely on pixels but on understanding the world.

At the engineering level, it has found the Scaling Law for Visual Tokenizer. For teams pulling their hair out over stagnant DiT performance, VTP may be the overlooked "critical lever."

With the Scaling Law proven viable and actionable, the capabilities of next-generation video generation models may rise again.

This time, models will see more clearly — and understand more deeply.

VTP's code and paper are now fully open-sourced on HuggingFace and GitHub.

For anyone looking to learn the foundations of AI visual generation, it's well worth exploring — substantial substance awaits.