Luchen Tech Closes Series A Financing of Hundreds of Millions of RMB to Build Distributed AI Development and Deployment Platform | BlueRun Ventures

Recently, Luchen Tech announced the completion of a Series A funding round worth several hundred million yuan. This marks the company's third round of financing within just 18 months of its founding. The fresh capital will primarily go toward team expansion and business development.

BlueRun Ventures led Luchen Tech's angel round.

As is widely understood, the "emergent" capabilities of generative AI today stem from the maturation of underlying large foundation models. Yet the compute, networking, and data costs required to train such models are beyond what ordinary companies can bear. Luchen Tech aims to offer a solution to this very problem.

Founded in 2021, Luchen Tech's core business is building a distributed AI development and deployment platform that helps enterprises reduce the cost of bringing large models into production while improving training and inference efficiency. Founder Yang You previously conducted research in distributed computing, machine learning, and high-performance computing at both UC Berkeley and National University of Singapore, where he set world records for ImageNet and BERT training speeds. Around 2021, he became increasingly convinced of the large model trend and founded Luchen Tech that same year with the goal of lowering the barrier to large model adoption.

Luchen Tech's current product lineup includes the open-source efficient deep learning system Colossal-AI and its corresponding enterprise PaaS platform. The platform consists primarily of a heterogeneous memory management system, an efficient N-dimensional parallelism system, and a low-latency inference system — all designed to help customers minimize model deployment costs and maximize compute efficiency.

On memory management, Yang explained that as model parameters and layers grow, so does computational demand. GPT-3's 175 billion parameters might occupy 800GB of memory. Add in the gradients and optimizer states needed during neural network training, and "GPT-3 consumes 3,200GB of memory before it even does anything," Yang illustrated. Given the inherent scarcity of memory resources, scientific management becomes exceptionally critical in large model training scenarios. When GPU memory can't hold all this data, portions must be migrated to CPU or NVMe storage.

Yang noted that managing GPU, CPU, and NVMe together is called heterogeneous management. Traditionally, this followed a static approach, estimating upfront the resources needed for parameters, gradients, optimizers, and so forth. In Yang's view, this rigid method can't adapt to actual training dynamics, likely leading to resource waste. Luchen's dynamic approach, by contrast, enables more flexible resource balancing. "We want data in the GPU whenever possible. If the GPU is full, it goes to CPU; if CPU is full, to NVMe. But critically, we need to minimize data movement between CPU, GPU, and NVMe," Yang said, adding that Luchen's heterogeneous memory management system achieves this objective.

On another front, enterprises training large models today often rely on hundreds or thousands of GPU cards. In theory, more cards means less training time and more efficient large model deployment. In practice, however, more cards means more machines handling computation, and the communication required to aggregate results across machines introduces new efficiency losses.

To address this pain point, Luchen developed its efficient N-dimensional parallelism system. Yang explained that the system employs high-dimensional tensor parallelism to boost efficiency. The underlying principle involves designing tensor parallelism in two dimensions. Tensor parallelism allows decomposed computational tasks to run simultaneously. The two-dimensional slicing approach means each machine only needs to communicate with machines in its same row or column, rather than with all machines. "If we need 10,000 machines computing, traditional one-dimensional methods require each machine to communicate with 9,999 others. We only need to talk to 99," he said.

Third is the low-latency inference system, which reduces the delay caused by slow model inference. Yang noted that solving this requires attention to both overall deployment architecture and model-level optimization. On the optimization side, Luchen's memory management, tensor parallelism techniques, and pruning and distillation solutions all contribute.

As can be seen, the heterogeneous memory management system and efficient N-dimensional parallelism system primarily take effect during training, while the low-latency inference system accelerates the inference phase. More granularly, the memory system especially helps customers save on resource costs, while the parallelism system especially boosts computational speed. Yang said all three systems are currently integrated into the company's PaaS platform, and the open-source Colossal-AI has garnered approximately 30,000 GitHub stars. In terms of service delivery, customers can train models directly through Luchen's PaaS platform, or Luchen can assist with model training. The company's solutions have reportedly been deployed across autonomous driving, cloud computing, retail, pharmaceuticals, semiconductors, and financial services.

Regarding 2023 plans, Yang said the company's business volume has surged this year alongside growing large model training demand across industries, with revenue expected to grow 3-5x compared to last year. Following this funding round, Luchen will accelerate expansion and seeks to attract more top talent in MLOps, AI large models, and AI frameworks to better serve customers.

👇 For funding or partnership inquiries, long-press the QR code below 👇 Add BlueRun's "Xiaochi" on WeChat to connect directly with us.

▶ More Content ◀

Originating in Silicon Valley, BlueRun Ventures was established in 2005 as a venture capital firm focused on early-stage startups.

BlueRun Ventures currently manages multiple USD and RMB dual-currency funds in China, with assets under management exceeding RMB 15 billion, making it one of the largest early-stage funds domestically. The firm invests primarily at Pre-A and Series A stages across hard tech and innovative interaction, enterprise technology, new consumer, and healthcare sectors. It has backed more than 150 startups, including Li Auto, Waterdrop, QingCloud, Guazi.com, Qudian, Songguo Mobility, Ganji.com, Energy Monster, Yuntu Semiconductor, Machenike, Yunheng Intelligence, Anxin Wangdun, and BioMap.

BlueRun Ventures has been ranked #1 on Zero2IPO's "China Top 30 Early-Stage Investment Institutions" and ChinaVenture's "China Best Early-Stage Venture Capital Firms TOP30," and was named among Preqin's Top 10 Global VC Fund Managers for Sustained High Returns.

Additionally, BlueRun Ventures has received consecutive recognition from Forbes China, 36Kr, Cyzone, Caixin Media, CBNweekly, Jiemian, and other media outlets as "China Early-Stage Firm of the Year," "China Top Venture Capital Firm," "Most Entrepreneur-Friendly Early-Stage Firm," and "Most Influential Early-Stage Firm."