Open-source models need a new table of their own

"Take a seat at the table!" Open-source models have reached their most dangerous moment.

After the frenzy of coding capability improvements in Q3, domestic model labs have spent the last two months scraping together whatever they can.

Chinese labs have pulled out all the stops to squeeze out a few extra points on the Arena.

This has produced a rather comical phenomenon: the higher a Chinese model scores on the Arena, the more likely it is to actually be unusable.

Because these labs have all resorted to "thunderous deep thinking" to game their scores — the higher the score, the longer the reasoning process, the slower the task completion.

Benchmark scores and real-world user experience have diverged dramatically. In fact, sky-high benchmark scores have become a contrarian indicator.

And you'll notice that the Arena, much like OpenRouter call volume — which was all the rage in late 2025 — has been thoroughly gamed by Chinese models.

Look at the Arena ranking shifts over the past few months, and you'll see that aside from MiniMax, there's hardly a Chinese model that couldn't climb the ranks.

First GLM, Moonshot AI, and Qwen traded the #1 spot back and forth. Then Mimo broke through too, straight to #1 among open-source models.

When Mimo topped the Arena, Qiu Mu asked me, puzzled, why Mimo's promotional materials were taking veiled shots at competitors for distilling while implying they were the earnest, diligent ones — when they'd clearly pulled off a benchmark Great Leap Forward that had nothing to do with earnest diligence.

I told Qiu Mu that this was because Fuli Luo is truly great, and under her leadership, Xiaomi's Mimo would surely become the fastest-improving, most cost-effective open-source model.

Though I did have one small quibble: I'm very much pro-distillation. Distillation is a righteous act.

To borrow the Biden inequality: you can't support distillation only when you're distilling human knowledge.

Distillation is a tool against closed-source lock-in. You distill all of humanity; I'll distill you. Perfectly reasonable. Without distillation, wouldn't those two closed-source labs have a monopoly on the entire world? Outrageous.

Qiu Mu still wasn't satisfied. He said Mimo topping the Chinese model rankings on the Arena proved the same thing as MiniMax's OpenRouter surge back in the day — the metric itself had lost all meaning.

As a Xiaomi fan, I have no comment on that.

But back to the point.

Mimo wasn't the only one doing this. StepFun, which had been relatively quiet, also turned in a solid exam-room performance. Step 5 Preview briefly shot up to top three among open-source models on the Arena.

And it gets worse.

Two weeks ago, Ant Group's Ling-3.1 Flash — part of its Bailing foundation model family — briefly gamed its way to #5 among Chinese models on the Arena, ahead of K3 and even surpassing Claude Haiku 5.5.

I can only think of two explanations for Bailing's sudden benchmark breakthrough:

One: Bailing's real performance is genuinely on par with K3, Qwen 3.8 Max, and GLM 5.3, and it crushes models like Seed 2.1 Pro. Two: the Arena is completely gamed, pure comedy.

Vote in the comments, fam ❤️

All I can say is, Ant Group really can't catch a break. Even being K3-tier, nobody knows they exist. Pure air.

Based on all this chaos, I've reached a serious conclusion after sober reflection: the Arena is America's Star Wars program — a tool to disrupt Chinese open-source model training.

American firms' muscle memory is to set standards, using the moral high ground to establish standards bodies and industry gatekeeping. As a follower, you have to exhaust yourself, drain your resources, reshape yourself in their image just to earn their approval.

This effectively forecloses any possibility of open-source models catching up.

Because large models are an engineering problem. Algorithm architecture sets the floor; performance depends mainly on data and compute. Whoever has higher-quality training data and larger-scale compute will almost certainly train the better model.

We have to face reality: Chinese models are followers. High-quality data supply and compute are far behind American peers. Over these past two months, the gap between domestic models and OpenAI and Anthropic seems to have widened further.

If we compete with identical strategies, Chinese models have zero chance of winning.

So Chinese labs have championed open source.

Open source is the most promising path forward because it reframes competition away from China vs. US narrative to open vs. closed narrative — seizing the moral high ground. And it concentrates the efforts of so many labs to accelerate progress.

But open source alone isn't enough.

If all open-source models just game the Arena, if their highest goal is boosting a few famous benchmarks, the motivation is still to extract gold from primary and secondary market old-timers — because these old-timers are mostly compradors who can only read the most famous leaderboards and then value you at 1/10 of Anthropic's price, with zero agency of their own.

And if everyone's gaming the same leaderboard, still competing head-on in the same lane — wasn't the whole point of open source to avoid this? It accomplishes nothing. OAI and Anthropic can still brute-force their way to massive leads with resources. Still zero chance of winning.

So open source isn't enough. Chinese models must set up their own table, establish their own standards. You fight your way; I'll fight mine. Chinese models need agency.

So how?

First, stop gaming the Arena. It's thoroughly gamed. Topping it only proves you're in the same league as Mimo and Bailing. Pure waste of precious compute.

Second, we need numerous benchmarks grounded in real tasks and real environments to evaluate model capabilities.

We have to accept a reality: no single leaderboard can fully and comprehensively reflect model performance.

Comprehensive leaderboards are doomed to meaninglessness. Large models have effectively bifurcated into Flash and Pro tiers, and will further segment by work domain. Cross-tier comparisons are pointless.

It's like trying to rank cars by 0-60 acceleration or range alone. If someone compares a Yangwang U8 to a Model Y, they're an idiot.

We can only evaluate large models by specific domain.

The good news: after this round of Chinese model Great Leap Forward, mainstream models have far surpassed the usability threshold.

DeepSeek 4.1 Flash is fast and capable. MiniMax 3.1 Flash is fast and nearly capable. In short, no mainstream lab's model performs worse than Qwen 3.8 27B — except Baidu's ERNIE. Out of respect, Qiu Mu considers Baidu a model lab too.

So the future direction emerges naturally: every work type will have its own SOTA model.

What Chinese models should do isn't chase a single comprehensive leaderboard that's doomed to irrelevance. It's find their own lane and become #1 in that lane.

Simply put: reclaim the right to define model performance. Don't let American good ol' boys lead you by the nose.

Wenfeng Liang is genuinely godlike on this front.

A month ago I didn't get his approach. DeepSeek had never been a ceiling-chaser this past year — they'd focused entirely on inference efficiency, effectively going all-in on Flash.

Then the domestic model GPU shortage worsened. 2T-parameter models became agonizingly slow. Meanwhile Liang's DeepSeek 4.1 Flash absolutely rips — I get 150 TPS in daily use.

Turns out Liang's route is innovative too: pure vendetta against closed-source. You drop a GPT-6; I drop a faster version at 1/10th the price, with solid gross margins.

Pure revenge arc.

Another positive example I've been citing for three straight months: K3.

Yutong Zhang also has magical tactics. K3 specifically optimized training data for mini-games and web pages, becoming #1 in front-end development. They successfully hacked the basic fact that people everywhere test models by generating mini-games in one shot — breaking through user mindshare, making everyone genuinely feel Kimi is incredible.

Of course, Kimi's path dependency is also a problem. After GPT-6 pioneered the big-model 3D capability hype cycle, K3.1 tried to follow suit, hyping its own 3D chops and again pricing at a fraction of American rates.

This backfired. The reason is simple: Kimi's one-shot mini-game generation was innovation, and innovation gets rewarded. Hyping 3D capability was following, playing the budget alternative game — mostly scraping the bottom of the barrel, nobody respects that. Problems exposed during follow-on get magnified; you're instantly flamed for copying.

I'm sure Kimi fam have deeply felt the burn these past two weeks 😭

With this in mind, Zang AI has decided to build an open-source benchmark community. We'll evaluate large model capabilities across domains using questions drawn from real tasks and real environments.

Several benchmarks we're collaborating on will be released open-source. We also welcome fam to collaborate on developing more killer benchmarks. Hope to stand tall with Chinese model labs and build agency together!

⬇️

Subscribe to our Substack: funeralai.substack.com