Multimodal DeepSeek is here! Yang Yaodong's team collaborates with HKUST to release Align-DS-V, powering Lingchu's DS-VLA alignment and expansion | BlueRun Ventures Family Headlines
Based on Align-DS-V, the Peking University-Lingchu Joint Lab has already begun deeper exploration in the VLA (Vision Language Action Model) space.
Amid the rapid evolution of AI, "strong reasoning with slow thinking" has emerged as one of the dominant development trajectories, profoundly shaping R&D directions and investment decisions. Extending this paradigm to more modalities — and eventually all modalities — while ensuring alignment with human values and intentions has become a highly forward-looking and critical challenge.
Previously, DeepSeek's in-house Janus-Pro-7B did not incorporate reasoning capabilities. Now, a domestic research team has achieved this first — based on their self-developed full-modality framework Align-Anything, a Peking University and HKUST joint team has launched the multimodal version of DeepSeek-R1, Align-DS-V, which surpasses GPT-4o on certain visual understanding benchmarks. Align-Anything and Align-DS-V were developed by Peking University in collaboration with The Hong Kong University of Science and Technology. Both the Align-Anything framework and the multimodal version of DeepSeek-R1, Align-DS-V, are now open-sourced, with the team committed to long-term maintenance.
The Peking University alignment team within the joint research group focuses on safe interaction and value alignment of AI systems. The team is advised by Yaodong Yang, Assistant Professor at the Institute for Artificial Intelligence, Peking University. The Hong Kong Generative AI R&D Center (HKGAI) within the joint research group is led by Council Vice-President and Academician Yike Guo of The Hong Kong University of Science and Technology as Center Director.
Building on Align-DS-V, the PKU-Lingchu Joint Laboratory has already begun deeper exploration in the VLA (Vision Language Action Model) domain. The VLA model under development at Lingchu uses a multimodal large model for alignment and fine-tuning at the "brain" end, outputting action tokens to the "cerebellar" controller; the cerebellar controller then processes these tokens alongside other modality information to generate specific robot control commands. Both stages require post-training and fine-tuning techniques for multimodal large models. The PKU-Lingchu Joint Laboratory states that the multimodal strong reasoning capability of Align-DS-V is core to the brain end of the VLA model, and the upcoming research and training plan leverages the cross-modal penetration capability of multimodal reasoning models to achieve action penetration, ultimately realizing a truly efficient VLA model. The same post-training techniques can also be applied to fine-tune the cerebellar controller, achieving higher success rates, generalization, and robustness.
BlueRun Ventures is the lead investor in Lingchu Intelligent. We congratulate Professor Yaodong Yang's team on this important progress and look forward to the PKU-Lingchu Joint Laboratory continuing to push the boundaries of technology.
Align-Anything framework repository:
https://github.com/PKU-Alignment/align-anything
DeepSeek-R1 multimodal version Align-DS-V open-source repository:
https://huggingface.co/PKU-Alignment/Align-DS-V
Thanks to its powerful reasoning and long-context thinking capabilities, DeepSeek R1 has garnered significant attention since its open-source release. DeepSeek R1 achieved 79.8% on AIME2024, slightly outperforming OpenAI-o1-1217. On MATH-500, it attained a remarkable 97.3%, matching OpenAI-o1-1217 and clearly surpassing other models. On coding tasks, DeepSeek-R1 demonstrated expert-level performance in code competitions, earning a 2029 Elo rating on Codeforces and outperforming 96.3% of human participants.
The emergence of DeepSeek R1-Zero and R1 once again demonstrates the potential of reinforcement learning. R1-Zero was built from a base model using pure reinforcement learning (RL), without supervised fine-tuning (SFT) from human expert annotations. During training, as steps accumulated, the model gradually developed long-text reasoning and long-chain-of-thought capabilities. As reasoning paths progressively lengthened, the model also exhibited self-healing abilities, discovering and correcting its own earlier mistakes.
The outstanding performance of DeepSeek R1-Zero and R1 in pure text modalities is impressive, which naturally raises the question: what would DeepSeek R1's deep reasoning model look like when augmented with multimodal scenarios?
• Modality penetration and modality linkage are expected to further enhance strong reasoning capabilities. The information humans receive in daily life is typically full-modality, with different sensory channels complementing each other to help us more comprehensively understand and express complex concepts. This full-modality information flow is equally critical for the paradigm shift of large models toward artificial general intelligence. Researchers have begun attempting to extend large language models to additional modalities, obtaining full-modality models capable of not only processing language but also understanding and generating images, audio, video, and other information — such as GPT-4o and Chameleon.
• Full-modality extension will become DeepSeek R1's next major breakthrough. First, it enables the construction of a closed-loop cognitive system of "perception-understanding-inference" in complex decision-making scenarios, expanding the boundaries of intelligence across multiple contexts. For example, through cross-modal alignment techniques, the model can establish semantic associations between grayscale features in CT scans and professional terminology in pathology reports, synchronously analyzing X-ray shadow distributions and patient-reported symptoms in medical diagnosis. Furthermore, this spatiotemporal associative reasoning capability allows autonomous driving systems to simultaneously parse vehicle trajectories in road-condition video, the flickering frequency of traffic signals, and abnormal sounds in the surrounding environment, enabling more precise multi-dimensional risk prediction.
• Extending strong reasoning capabilities to full-modality scenarios faces numerous challenges. In text modality scenarios, many complex reasoning tasks can use rule-based rewards to provide supervisory signals as carriers of human intent and preferences. When extending from text modality to multimodal and even full-modality scenarios, many issues emerge: as the number of modalities increases, can traditional binary preferences or rule-based rewards capture diverse or hierarchical preferences in human intent? When multimodality expands to full-modality space and modality interactions become more complex, what improvements are needed for RL methods? Across different modalities, how can modality-specific and modality-shared information be unified in reward signal modeling?
Following the release of DeepSeek R1, the Peking University and HKUST joint team rapidly extended the DeepSeek R1 series to image-text modality within one week based on their self-developed framework align-anything, achieving superior visual understanding performance. Below are partial evaluation results:
| Align-DS-V-8B | GPT-4o | |
|---|---|---|
| mathvision | 27.0 | 30.4 |
| llava-bench-coco | 105.3 | 104.9 |
| mathvista | 62.2 | 63.8 |
| A-OKVQA | 83.7 | 87.3 |
More importantly, the team also discovered the enhancement effect of modality penetration on the model's text-modality reasoning capabilities.
Specifically, in their full-modality experiments with DeepSeek R1, the team found that after multimodal training, the model's performance on text-modality tasks improved, with gains in scientific tasks, complex reasoning, and mathematical coding.
| DeepSeek-R1-Distill-Llama-8B | Single-Modality | Multi-Modality |
|---|---|---|
| ARC (5-shot) | 32.7 | 34.2 |
| ARC-Challenge (5-shot) | 21.4 | 40.5 |
| BigBench-Hard (3-shot) | 72.2 | 73.4 |
The team believes that current multimodal large models already possess strong cross-modal penetration and fusion perception capabilities, enabling efficient reasoning and collaborative output across multiple modalities (such as images, text, audio, video, etc.) by combining world knowledge and in-context learning capabilities.
And based on continuous self-evolution through slow-thinking strong reasoning capabilities, breaking through the limitations of single modalities, cross-modal penetration depth is significantly enhanced. Through deep fusion of world knowledge, the reasoning boundaries in text modality are expanded.
To validate the powerful capabilities of full-modality reasoning large models in vertical domain applications, the R&D team conducted localized alignment of Align-DS-V with Hong Kong regional values, enabling Align-DS-V to adapt to mixed Cantonese/English/Mandarin language inputs and deeply integrate Hong Kong local life scenarios including MTR dynamics, typhoon warnings, and Octopus card payments.
When asked via image-text query which Vitasoy drink (a popular beverage in Hong Kong) is more suitable for weight loss, Align-DS-V accurately selected the low-sugar original soy milk, while also noting that the original soy milk is equally suitable for weight loss, providing convenience for daily dietary choices in Hong Kong.
When facing image-text math problems containing traditional Chinese characters, Align-DS-V can accurately link image and text modality information, demonstrating rigorous step-by-step mathematical derivation in its solution process, showing credible prospects for application in education and other industries.
The multimodalization of large language models is an unstoppable trend, and full-modality large models supporting arbitrary modality inputs and outputs will become a future milestone. How to align full-modality large models with human intent has become a highly forward-looking and critical challenge. However, as modalities increase, the distribution of input-output spaces becomes more extensive, and hallucination phenomena increase, making full-modality alignment more complex. In fact, to further promote community research on multimodal alignment, the research team has contributed open-source efforts across four dimensions: datasets, algorithms, evaluation, and code libraries:
• Data: A 200k dataset containing human language feedback and binary preferences, covering full modalities including images, text, video, and speech.
• Algorithms: A synthetic data paradigm for learning from language feedback, substantially improving the performance of RLHF post-training methods.
• Evaluation: Modality linkage and modality selection evaluation for full-modality models.
• Code Library: A code framework supporting full-modality training and evaluation for images, text, video, and speech.
The align-anything framework is dedicated to aligning full-modality large models with human intent and values, where full-modality includes arbitrary-to-arbitrary input and output modalities such as text-to-text, text-to-image, text-image-to-text, and text-to-video. Overall, the framework designs an alignment training framework with high modularity, extensibility, and ease of use, supporting arbitrary modality model alignment fine-tuning derived from the four basic modalities of text, images, video, and audio, and verifying the implementation correctness of the framework's alignment algorithms. The framework features:
• High Modularity: Abstraction and carefully designed APIs for different algorithm types, allowing users to modify and customize code for different tasks, as well as customized model and dataset registration and other advanced extension usages;
• Supports Fine-Tuning Across Arbitrary Modality Models: Includes fine-tuning capabilities for large models spanning multiple modality generation and understanding, such as LLaMA3.2, LLaVA, Chameleon, Qwen2-VL, Qwen2-Audio, Diffusion, etc.;
• Supports Different Alignment Methods: Supports multiple alignment algorithms on arbitrary modalities, including both classic algorithms such as SFT, DPO, PPO and newer algorithms such as ORPO, SimPO, and KTO;
• Supports Multiple Open and Closed-Source Alignment Evaluations: Supports over 30 multimodal evaluation benchmarks, including multimodal understanding evaluations such as MMBench, VideoMME, and multimodal generation evaluations such as FID, HPSv2;
Meanwhile, to promote further community development of full-modality alignment models, the research team has released the first full-modality human preference dataset align-anything. Unlike existing preference datasets that focus on single modalities with uneven quality, align-anything provides high-quality data including any modality in inputs and outputs, aiming to provide detailed human preference annotations and fine-grained language feedback for critique and improvement, enabling comprehensive cross-modal evaluation and improvement.
Drawing on LLaVA's training approach, the team extended R1's visual modality by training a Projector to map the Vision Encoder's outputs to the language representation space. In the Align-Anything library, the team open-sourced the complete training pipeline. First, based on the DeepSeek R1 series models, they constructed a "text + image -> text" architecture, as shown in the following script:
python projects/any_to_text/build_llm_vision.py \
--language_model_path deepseek-ai/DeepSeek-R1-Distill-Llama-8B \
--vision_tower_path openai/clip-vit-large-patch14-336 \
--save_path outputs/Align-DS-V-Init
In the new multimodal model, input images are processed through the Vision Encoder to extract features, generating intermediate representations, which are then mapped through the Projector to obtain visual representations. Meanwhile, language instructions are processed to generate language representations. These visual and language features are jointly fed into the language model, which combines both types of information for reasoning and ultimately generates text responses.
After constructing the modality-extended R1 architecture, specific training proceeds in two steps:
Step 1: Freeze all model parameters except the Projector, and pre-train the Projector so that it can map visual representations from the Vision Encoder to the language representation space.
deepspeed --module align_anything.trainers.text_image_to_text.sft \
--model_name_or_path outputs/Align-DS-V-Init \
--train_datasets ${TRAIN_DATASETS} \
--train_template ${TRAIN_TEMPLATE} \
--train_split ${TRAIN_SPLIT} \
--output_dir outputs/Align-DS-V-Step1 \
--freeze_vision_tower True \
--freeze_mm_proj False \
--freeze_language_model True \
--epochs 1 \
--ds_cfgs ds_z2_config.json \
--learning_rate 1.e-3
Step 2: Fine-tune both the Projector and the large language model simultaneously to activate the language model's multimodal reasoning capabilities.
deepspeed --module align_anything.trainers.text_image_to_text.sft \
--model_name_or_path outputs/Align-DS-V-Step1 \
--train_datasets ${TRAIN_DATASETS} \
--train_template ${TRAIN_TEMPLATE} \
--train_split ${TRAIN_SPLIT} \
--output_dir outputs/Align-DS-V-Step2 \
--freeze_vision_tower True \
--freeze_mm_proj False \
--freeze_language_model False \
--epochs 3 \
--ds_cfgs ds_z3_config.json \
--learning_rate 2.e-5
Chen Gang of Maomaoguo: 12 Years in Dali, Raising a Group of Happy Children | Chuangye You He Lan
