Oasis Capital Dialogue with Professor Liu Zhuotao: The Circulation Code of Private Domain Data

Data is the fuel that feeds large language models. As LLMs move toward domain-specific deployment, how should private data interact with them?

We spoke with Professor Zhuotao Liu, Assistant Professor and PhD advisor at Tsinghua University, recipient of the National Natural Science Foundation of China's Excellent Young Scholars (Overseas) award, and former tech lead at Google Cloud data centers, to understand the opportunities and challenges in LLM development from a data perspective. Enjoy.

Oasis Capital: Where does your research group enter the LLM field, and what directions are you pursuing?

Professor Liu: Large models have three core pillars: compute, algorithms, and data. The compute issue boils down to chips — China faces a bottleneck there. On algorithms, our talent density is actually quite strong, but the lack of high-performance chips somewhat constrains development speed. On data, I believe China has distinctive advantages and opportunities, and our group has always centered its research around data.

Whether it's ChatGPT or GPT-4, most of their training data comes from publicly available internet sources — excellent for building general-purpose or foundation models. But when it comes to enterprise-specific models tailored to business needs, fine-tuning on "private domain data" becomes essential. Google's latest I/O releases, Sec-PaLM and Med-PaLM 2, for instance, are fine-tuned versions of PaLM 2 on security and medical domain knowledge respectively. Of course, the emergent capabilities and in-context learning abilities of foundation models may reduce fine-tuning costs to some degree.

Beyond that, LLM inference involving private data is also worth attention. Several European countries previously banned ChatGPT, with preventing privacy leaks being one of the motivations.

How should private domain data interact with large models? This is what our group is currently thinking through and working to solve. I see two broad directions:

First, private deployment: Similar to private cloud, organizations or enterprises deploy their own large models to ensure data never leaves their domain. The challenge here is elevating enterprise R&D capabilities. Many companies are currently building businesses around private deployment — much like how, over a decade ago when public cloud emerged, many companies rushed to build vertical clouds.

Second, private inference and fine-tuning: If using models from providers like OpenAI or other specialized LLM vendors, private data inevitably becomes involved — whether fine-tuning models on private data or providing private prompts for inference. Technically, these can be viewed as extensions of privacy-preserving computation. I've noticed growing efforts from both academia and industry in this direction.

Oasis Capital: How do these research directions translate into actual implementation?

Professor Liu: With "magic" (laughs). The core technology for private inference and fine-tuning is privacy-preserving computation, and more specifically, applied cryptography. Cryptography itself is somewhat "counter-intuitive." Take zero-knowledge proofs — you want to prove something without revealing its actual content. Similarly, private inference and fine-tuning aim to use data without ever obtaining it in plaintext. Let me briefly introduce the current SOTA (State-of-the-Art) techniques and what we should do next:

People started working in this area a few years ago. The earliest efforts focused on private inference — completing inference without giving plaintext data to the model. These works centered on designing neural network forward-propagation protocols based on two fundamental cryptographic primitives: secure multi-party computation (MPC) and homomorphic encryption. The current academic SOTA is probably Alibaba's "Cheetah" published in 2023, which achieves ten-second-level private inference on ResNet-32 in computer vision.

But large models are vastly more complex. We generally characterize model complexity by model size and the complexity of nonlinear functions. For example, ResNet uses ReLU (Rectified Linear Unit), while Transformers in large models typically use GELU (Gaussian Error Linear Unit), which is more computationally intensive than ReLU. Nonlinear layer computations are the most time-consuming part of ciphertext-domain calculation and represent the system's bottleneck. Current SOTA remains far from practical use. For a BERT-scale model (several hundred million parameters), one private inference takes over half an hour with 80GB of communication. Private training is even slower than private inference.

For both privacy-preserving computation companies and applied cryptography researchers, the challenge is how to apply cryptography, how to design scenarios, and how to customize domain-specific hardware. Just as zero-knowledge proofs represent an engineering innovation, large models themselves are an engineering innovation.

My group recently completed work on further improving LLM private inference performance. Overall, we built an LLM private inference system, identified its performance bottlenecks, then performed corresponding model architecture replacement and fine-tuning. The results are quite promising, and we'll be open-sourcing this work soon.

Oasis Capital: This is a promising research direction. ZK has hardware acceleration solutions, and MPC or homomorphic encryption could also have hardware acceleration. Do you think practical implementation is achievable in the near term?

Professor Liu: Hardware acceleration is a direction that can't be ignored. From my current understanding, it's difficult to substantially improve LLM private inference and training performance without hardware assistance. For example, previous work designed trusted hardware to generate multiplication triples needed for secure multi-party computation, which could significantly reduce the overhead of ciphertext computation.

From an industry deployment perspective, this should align with the broader adoption pace of privacy-preserving computation. Currently deployed privacy-preserving computation scenarios involve far simpler computation than LLM inference and fine-tuning. However, due to enterprise compliance requirements, privacy-preserving computation becomes a necessary path to explore and accept for proper use of private domain data — somewhat like a "technology tax," you could say.

Another possible approach avoids cryptographic methods entirely and instead uses DP (Differential Privacy). Computationally, DP is close to plaintext computation, so efficiency isn't an issue. But DP does affect functionality to some degree — for instance, model quality degrades as more noise is added. In the near term, having enterprises apply DP-based noise addition to allow data to leave their domain (China's Personal Information Protection Law requires raw data cannot leave the domain) may be the fastest solution to deploy.

Oasis Capital: Recent services from Amazon, and Google's Vertex AI, let you run various open-source models in clusters with one click. Will this affect the privacy-preserving computation scenarios you described?

Professor Liu: Most public cloud strategies are actually hybrid cloud — they need to interact with users' on-premises private deployments. LLMs will likely follow a similar pattern, which can satisfy certain enterprise scenarios where data must not leave the domain. But a large number of scenarios will still require privacy-preserving computation, because some institutions need data that doesn't reside within the institution itself. For example, when a bank wants to check a user's credit rating, it needs carrier data. Regardless of private deployment, the bank's data is involved. How to handle that bank data requires privacy-preserving computation assistance.

Oasis Capital: What opportunities have large models created for privacy-preserving computation or data itself?

Professor Liu: I believe future large model development will depend on private domain data. For more compliant use of such data, there are two broad approaches:

First, privacy-preserving computation: If "absolutely private" data is used in LLMs, then private inference and fine-tuning of LLMs becomes necessary — as I detailed earlier.

Second, data spaces: This is a recently emerging area with considerable momentum in both Europe and China. Simply put, data spaces enable data circulation and transactions between institutions. Data providers can control how their data is used, by whom, at what price, and how to assign accountability for violations. In a nutshell, a data space is a secure and controllable cross-domain data interaction platform. Relatively speaking, most data in data spaces doesn't have privacy requirements as stringent as those in privacy-preserving computation scenarios, so the technology stacks differ accordingly. Meanwhile, data spaces typically support larger-scale cross-domain data interaction and computation, nicely complementing where privacy-preserving computation falls short.

Of course, privacy-preserving computation and data spaces aren't mutually exclusive — they complement each other.

Oasis Capital: What's the distinction between privacy-preserving computation/data spaces and AI Safety?

Professor Liu: Both fall under the umbrella of "safety," but they address different layers of the problem. Privacy-preserving computation and data spaces concern how to use private domain data. AI Safety focuses on whether the final model is toxic, whether its robustness is sufficient, whether it's fair, whether it has ethical issues, and so on.

Compared to AI Safety, discussions around data privacy may currently receive less attention — at least academic papers on the topic don't yet match the volume on AI Safety. This actually follows a pattern common in computer systems development: performance comes first, then security, and finally privacy. After the internet was born, the initial focus was on increasing bandwidth. Around 2010, attention gradually shifted to internet architecture security. It wasn't until 2018 that Europe introduced GDPR (General Data Protection Regulation), and China introduced relevant privacy protection laws in 2021 — meaning data had been flowing in plaintext across the internet for over two decades before privacy became a major concern.

However, large models were born in an era when data privacy protection is widely recognized and accepted. In China, the digital economy backdrop has made data itself an important corporate asset. Consequently, data circulation and transactions become focal points. Entrepreneurs and scholars alike need to adapt to regulations, so I believe discussions around privacy will only grow.

Oasis Capital: Can you summarize the impact of this wave on your research field and its future development?

Professor Liu: The biggest impact of large models on my research field is how to help them use private domain data more compliantly. Data spaces and privacy-preserving computation will play important roles in this. Against the backdrop of China's vigorous development of the digital economy, this represents a significant opportunity. At the same time, it faces numerous technical and non-technical challenges.

---

Championing Vitality

What do you think constitutes technological vitality?

Technological vitality lies in continuous innovation and transformation, and in consistently empowering human progress and betterment throughout this process. And the source of all this is people themselves.

— Professor Zhuotao Liu, Assistant Professor and PhD advisor, Tsinghua University

Oasis Capital is a new-generation venture capital firm in China, dedicated to discovering the most vital entrepreneurs of the next decade and growing alongside them to create long-term value. "Championing Vitality" is Oasis's vision and mission. This vitality is both the direction of structural transformation in our era and the resilience and evolutionary force of entrepreneurs.

Oasis Capital focuses on early and growth-stage investments, with individual checks ranging from $3 million to $30 million, concentrating on robotics, artificial intelligence, and technology services to support China's technology-driven new service upgrade.