BlueRun Ventures Headlines | Moonshot AI Partners with Tsinghua University and Other Institutions to Open-Source Mooncake, a Large Model Inference Architecture

In the era of large language models, more data, bigger models, and longer context windows bring greater intelligence — but they also place heavier demands on inference system efficiency. How to handle high inference loads, reduce costs, and cut response latency has become a shared challenge across the industry. In June 2024, Moonshot AI and Tsinghua University's MADSys Lab jointly released the Mooncake inference system design that underpins Kimi. Built on a KVCache-centric disaggregated prefill-decode (PD) architecture and a storage-for-compute approach, the system dramatically improved inference throughput, drawing widespread attention from the industry. Recently, to further accelerate adoption of this technical framework, Moonshot AI and Tsinghua University's MADSys Lab joined forces with 9#AISoft, Alibaba Cloud, Huawei storage, ModelBest, Qujinger Technology, and other industry-academia-research partners to launch the open-source project Mooncake, collaborating to build a KVCache-centric inference architecture for large models. On November 28, the Mooncake technical framework was officially released as open source. https://github.com/kvcache-ai/Mooncake BlueRun Ventures was an early investor in Moonshot AI and has continued to back the company in subsequent rounds.

Mooncake Inference System Architecture Diagram Extending from the research paper, the Mooncake open-source project centers on a large-scale KV Cache pool, using the innovative storage-for-compute concept to substantially reduce compute overhead and significantly boost inference throughput. The open-source release will proceed in stages, gradually open-sourcing the high-performance, multi-tier KV Cache caching implementation Mooncake Store, while ensuring compatibility with various inference engines and underlying storage/transfer resources. The transfer engine component, Transfer Engine, is already available as open source on GitHub worldwide. The ultimate goal of the Mooncake open-source project is to establish a new standard interface for high-performance memory-semantic storage in the large model era, along with a reference implementation.

Xu Xinran, VP of Engineering at Moonshot AI, said: "Through close collaboration with Tsinghua University's MADSys Lab, we jointly developed the disaggregated large model inference architecture Mooncake, achieving extreme optimization of inference resources. Mooncake not only enhances the Kimi user experience and reduces costs, but also provides effective solutions for handling long-context and high-concurrency demands. We believe that through open-source collaboration with industry, academia, and research institutions, we can drive the entire industry toward more efficient inference platforms." The project welcomes more enterprises and research institutions to join in building Mooncake, collectively exploring more efficient and advanced innovations in model inference system architecture, so that AI assistants and other products built on large model technology can continue to benefit broader populations.

BlueRun Ventures Family Headlines | Math Benchmarks Against o1 Series, Search Evolves Again — Kimi's New Reasoning Model Expands the Boundaries of Intelligence With You BlueRun Ventures Leads $42 Million Series A for AllinkBio | BlueRun Family

BlueRun Dialogues with Moonshot AI, Muyan Zhiyu: Standing at the AI Industry's Crossing | The Road to AGI