Clipped Importance Sampling Policy Optimization

CISPO (Clipped Importance Sampling Policy Optimization) is a reinforcement learning algorithm developed by MiniMax as a key innovation in its RL training pipeline . The method was designed to improve on traditional approaches like DPO or GRPO by preserving update paths for all tokens rather than clipping tokens and losing contextual information, which significantly boosted convergence speed and quality .

The algorithm operates in deep coupling with MiniMax's "Lightning Attention" architecture and was central to training MiniMax-M1, enabling the model to handle up to 1 million input tokens and 80,000 output tokens while completing RLHF training on 512 H800 GPUs in just three weeks—cutting costs by an order of magnitude versus comparable industry efforts . Meta later validated CISPO independently, adopting the loss function across more than 400,000 GPU hours of experiments and describing it as "the most suitable scheme for large-scale training" in a published paper that cited MiniMax's work as representative of recent RL breakthroughs .