Baidu Open-Sources 10 Models in One Go | A Deep Dive into Its Open-Source Technical Report
From "Open Models" to "Open Systems"
**
From "Open Model" to "Open System"
👦🏻 Author: Jingshan
🥷 Editor: Kavana
🧑🎨 Layout: NCon
On June 30, Baidu officially open-sourced the ERNIE 4.5 series, including ten models, pre-trained weights, inference code, and accompanying tools and case studies.
Baidu's open-source strategy had once been a topic of debate. So this move drew considerable attention in tech circles, with a wave of positive reactions on X and Reddit.
When asked about this, a source at Baidu told Crossing:
"Open-source and closed-source are just different paths. What matters in the end is whether the applications work well and the ecosystem thrives."
Driven by curiosity about this apparent reversal, the Crossing team dug into the technical report and found several highlights worth noting.
What Stood Out in the Technical Report
1) The Biggest Draw: Major Leap in Multimodal Understanding
The ERNIE 4.5 series released ten open-source models in total.
This time, the product lineup is more clearly structured, divided into three tiers: two 0.3B models, four A3B models, and four A47B models — covering small, medium, and large sizes.
The text-only models are optimized for general language understanding and generation, improving performance on everyday conversation and creative writing tasks. The multimodal models support both "thinking mode" and "non-thinking mode" operation.
We compiled a list and visualized it:
These models achieved SOTA results on multiple text and multimodal benchmarks. The technical report, for instance, details the performance of the ERNIE-4.5-Base pre-trained model, which showed strong results on general, reasoning, math, knowledge, and coding benchmarks:
Near the end of this 68-page technical report, we found two interesting test cases the team ran. They fed a video to ERNIE-4.5-VL to test its visual understanding and perception capabilities.
[1] They asked ERNIE-4.5-VL to "watch a video and write captions," breaking the video down into key events with concise descriptions:
Prompt: Based on the video, can you help me create some engaging subtitles to place below it?
Model response: Explore the vibrant streets of Chongqing at dusk, where tradition and modernity blend together. Catch a glimpse of adorable giant pandas in their natural habitat — a true symbol of China. ...
[2] They asked ERNIE-4.5-VL to pinpoint specific video details, testing whether the model could accurately locate and identify particular events:
Prompt: At what second does "CHONGQING" appear in the video?
Model response: At the 5-second mark, "CHONGQING" appears in red text on a large white screen.
From these test cases, it's clear that ERNIE-4.5-VL performs impressively on image and video understanding. Its deep grasp of visual knowledge and perceptual abilities line up well with its benchmark scores.
For example, on video understanding and visual perception benchmarks like LongVideoBench, ERNIE-4.5-VL shows a notable lead:
In the official technical blog, ERNIE-4.5 outperformed even the closed-source OpenAI o1 on multiple benchmarks covering visual perception, visual knowledge, and document/chart comprehension, such as RealWorldQA and the bilingual MMBench:
ERNIE-4.5-VL's visual capabilities largely stem from its innovative "multimodal heterogeneous Mixture-of-Experts (MoE)" architecture, which merges text and image processing into a single model.
In simple terms, it puts text-processing and image-processing "experts" inside the same model, where they can collaborate without interfering with each other.
To prevent text and image training from colliding, ERNIE 4.5 designed a "heterogeneous MoE" structure: shared components allow different information types to communicate, while dedicated "experts" handle specific content.
ERNIE 4.5 architecture, supporting image, video, and text input with text output.
The biggest advantage of this architecture: the AI model learns from both text and images simultaneously. Like a person learning to read and look at pictures at the same time, this produces an AI more attuned to subtle distinctions between different information types. As a result, it performs better at understanding text, generating text, comprehending images, and reasoning across both modalities.
Both textual and visual information get adequate "room to express themselves."
Expert usage across Transformer layers. Left: text modality expert group; Right: visual modality expert group.
When we previously covered Agent products, we judged that "multimodal fusion" would grow increasingly important — now we're seeing it manifest in foundation models themselves.
2) Lowering the Barrier for Large Models
While reading through the technical report, we found that ERNIE 4.5 has made further strides in resource efficiency.
From a framework perspective, ERNIE 4.5 runs on Baidu's homegrown PaddlePaddle framework. Notably, as large language models have surged ahead, deep learning frameworks have become a must-have for foundation model "heavy hitters," with nearly everyone going all-in.
PaddlePaddle 3.0 launched a few months ago, reducing distributed core code development for Llama pre-training by 80% and doubling single-machine deployment throughput for the full DeepSeek-R1.
Now we're seeing PaddlePaddle show up in this technical report as well.
Through PaddlePaddle, the ERNIE 4.5 model achieved 47% Model FLOPs Utilization (MFU) on 2,016 NVIDIA H800 GPUs with RoCE interconnect. These technologies and configurations enable ERNIE 4.5 to process extremely large-scale data.
From a model architecture standpoint, the team has pursued efficiency gains on nearly every front.
We visualized all these complex techniques, with examples for each to aid understanding. For instance, this "heterogeneous hybrid parallelism" + "multi-level load balancing" strategy is a key framework for boosting pre-training throughput:
To implement these efficiency strategies, four specific techniques are used: intra-node expert parallelism, memory-friendly pipeline scheduling, FP8 mixed-precision training, and fine-grained recomputation.
Each one drills down on performance gains:
On the inference side, the team proposed two optimization methods — multi-expert parallel collaborative quantization and convolutional encoding quantization. Both share the same goal: making trained AI models respond faster in real-world applications, cutting user wait times:
This enables near-lossless 4-bit and even 2-bit quantization:
Additionally, two more interesting optimizations help squeeze fuller value from available resources:
Overall, even the largest ERNIE-4.5-A47B model can be deployed on a single node.
Specifically, it requires just four 80GB A800 or H800 GPUs (4-bit precision), or a single 141GB H20 GPU (2-bit precision). Beyond NVIDIA GPUs, ERNIE-4.5 can also be deployed on various other hardware platforms through PaddlePaddle.
This means users have more hardware options and greater deployment flexibility.
Most importantly, such a complex AI model can now run on relatively affordable hardware, lowering the barrier for startups and enterprise customers looking to deploy large models.
3) Specialized Training for Different Scenarios
We've also observed that multimodal tasks are growing increasingly important. In different scenarios, an AI model's ability to process images and videos often determines how well an application performs.
A pattern has even emerged where the "multimodal fusion" at the foundation directly determines how tall the "application building" above it can rise. To reach broader application scenarios, foundation models need strong multimodal capabilities.
In response to this trend, the technical report shows that ERNIE 4.5 opts for "customized post-training" tailored to different task requirements:
ERNIE-4.5 post-training flow chart
After repeated close reading, we found that to improve multimodal performance, the team employs multi-stage optimization. Every model goes through "multi-stage post-training" — like graduating from school and then attending vocational training to learn more practical, fine-grained skills.
Three main methods are used here: SFT, DPO, and UPO:
The techniques themselves are too complex, so here's a plain-language breakdown:
[1] SFT gives the model more "correct answers" to learn from, improving accuracy.
[2] DPO teaches the model to "read the room," selecting response styles that better match user preferences.
[3] UPO can be understood as a "comprehensive ensemble" — it combines multiple preference optimization techniques to more thoroughly align the model with human preferences and needs.
From the team's perspective, multimodal capabilities clearly rank high in priority. They directly determine how many scenarios a foundation model can genuinely help teams building applications on LLMs.
From "Open Model" to "Open System"
After covering the model technology, the latter half of the report deliberately highlights developer-friendly toolkits.
Alongside the ERNIE 4.5 series itself (ten models spanning MoE/dense and multimodal/text-only), Baidu open-sourced two training & deployment toolchains plus APIs, and with the PaddlePaddle community, has for the first time created a complete "download-to-deployment" system.
All ERNIE 4.5 models (including pre-training, fine-tuning, and inference code) and projects are now live on Hugging Face, GitHub, and the PaddlePaddle Starlight Community.
Beyond Baidu's official ERNIE Bot, the PaddlePaddle community also offers direct chat testing.
Community interface screenshot
For example, selecting the ERNIE-4.5-300B-A47B model allows direct testing in the PaddlePaddle Playground, with various parameter adjustments:
[1] ERNIEKit helps users fully customize and efficiently fine-tune models (achieving better results with fewer resources).
[2] Once trained, models can be deployed directly via FastDeploy.
The entire flow is seamless — developers no longer need to hunt around for tools to piece together.
ERNIEKit and FastDeploy significantly boost development and deployment efficiency. All code can be copied with one click from the corresponding model pages.
For example, fine-tuning with ERNIEKit is simple. Download the model locally, just copy one line of code:
huggingface-cli download baidu/ERNIE-4.5-300B-A47B-Paddle --local-dir baidu/ERNIE-4.5-300B-A47B-Paddle
Run the Erniekit tool for SFT (supervised fine-tuning):
erniekit train examples/configs/ERNIE-4.5-300B-A47B/sft/run_sft_wint8mix_lora_8k.yaml
After supervised fine-tuning, further align with DPO (preference optimization):
erniekit train examples/configs/ERNIE-4.5-300B-A47B/dpo/run_dpo_wint8mix_lora_8k.yaml
With FastDeploy, you can deploy the model as an HTTP API service with one click, ready for frontends or applications to call directly.
For example, here's a basic deployment command to start a FastDeploy inference service with an OpenAI-compatible API:
python -m fastdeploy.entrypoints.openai.api_server --model baidu/ERNIE-4.5-300B-A47B-Paddle --port8180 --metrics-port8181 --quantization wint4 --tensor-parallel-size8 --engine-worker-queue-port8182 --max-model-len32768 --max-num-seqs32
The PaddlePaddle community also hosts many application projects and hands-on tutorials built on foundation models.
For instance, while browsing the flagship multimodal ERNIE-4.5-VL-424B-A47B model, we found the tutorial series "Building Intelligent Industry Applications from Scratch: ERNIE+RAG+Agent Practical Guide" updated to the July 1, 2025 version, with the base model already switched to ERNIE-4.5:
Many similar applications and projects exist. You can find them using the visual card we made below:
When complex AI capabilities become "one-click development" callable in a few lines of code, combined with this "bundle-style" developer toolkit, this transformation gives more people the opportunity to participate in creating AI applications.
More ideas can be realized. More possibilities are being unleashed.
We look forward to more breakthroughs in the foundation model space this year. Crossing will continue to follow and interpret these "highlight-worthy" technical reports.
