Oasis Capital in Conversation with Professor Yu Zhou: Technology Is a Service, and So Is Entrepreneurship

The upheaval in AI is an opportunity for all sectors to reassess their core purpose and strengths amid this wave, and to chart their future direction.

Today, we share our conversation with Professor Zhou Yu, assistant professor in Columbia University's Computer Science department, a Forbes "30 Under 30" scientist, and founder of Articulate.AI. Enjoy.

Oasis Capital: How much of an impact have open-source models had?

Professor Yu: I'd say they're generating excitement across the board. In education, for instance, many companies are looking to build AIGC-based learning products. Every traditional industry is exploring potential AIGC applications. Our own company, for example, has been looking into corporate training.

Oasis Capital: In U.S. education, from K-12 through higher education, how do you see technology driving change?

Professor Yu: It's definitely an opportunity for education. AI can help teachers prepare course materials and assist students with Q&A and practice. At the same time, there are legitimate concerns — will students simply use AI tools to complete assignments without actually learning?

Oasis Capital: LLMs have a tendency to "hallucinate." Does this pose an even greater challenge for educational applications?

Professor Yu: Yes, especially when LLMs are deployed in high-stakes industries where output controllability is critical. K-12 education, for example, cannot afford to mislead children. Currently, most focus has been on STEM education — logic, math, physics — where you're dealing with facts and reasoning rather than opinions. Those areas are less problematic. From a "facts" perspective, various models have employed different adjustment methods, though none can claim to be completely accurate. It really depends on how constrained your domain is. Narrower domains offer stronger controllability — you can use calculator APIs to address computation errors, or apply domain-specific error correction. But if your domain is broad, encompassing even after-class Q&A, you'll exhaust yourself through trial and error alone, and still need RLHF (reinforcement learning from human feedback) to improve the model. So when first deploying applications, I'd recommend starting with more specialized domains — middle school math and physics, for instance — which are more forgiving.

Oasis Capital: Have you seen changes in demand for professional skill development or retraining?

Professor Yu: Demand is growing, absolutely. People are evaluating whether their current work suits them. In the United States particularly, there was a wave of resignations when remote work became possible and unemployment benefits were available. Having that time at home with benefits gave people space to think about their careers. Many changed industries or enrolled in courses — UI design, programming, and so on. The worker upskilling market has been expanding; Coursera and Udemy saw significant growth during the pandemic.

China certainly has demand too, but since remote work never became as entrenched, people didn't have extended periods to contemplate career goals. And for most Chinese, the priority is earning money to support their families and pay mortgages — with these pressures, it's difficult to find time for career transitions or self-improvement.

American employees tend to be more individualistic; everyone has their own ideas, and the employment relationship differs somewhat. Workers view their relationship with companies as a partnership — if they're dissatisfied, they switch jobs. From the company perspective, especially during the pandemic, retaining employees required offering benefits, increasing training budgets, and providing career development planning. Otherwise, employees would leave.

Oasis Capital: Regarding healthcare and education, how does LLM impact differ between China and the U.S.?

Professor Yu: Different incentive structures drive different choices. From Sam Altman's perspective, it's about "changing the world, creating positive impact." To maximize positive impact, healthcare and education are the channels.

Healthcare and education most need "positive impact." This isn't just a China-U.S. difference — it's global. New technologies always begin with those most receptive to them, in more commercializable industries, before finally reaching education and healthcare, which resist commercialization. Education typically lags because there's less money and more regulations. Healthcare is quite marketized in the U.S., so there are some differences, but most countries are broadly similar. The Bay Area's distinct values do play a role — there's greater desire to create "positive impact" rather than simply make money.

Most people need to make money. Those with ideas and ambition don't prioritize money first. We chose education because we believed it would have "positive impact." If we wanted the easiest money, we'd do entertainment — companion robots, virtual boyfriends or girlfriends.

Oasis Capital: What are the current state-of-the-art systems or products in multimodal dialogue systems?

Professor Yu: There are basically no mature products yet; it's still primarily research groups. Our company is also building products. "Multimodal" encompasses several types — audio only, or video plus audio? Does your robot need a fixed persona? A photo? A virtual human? A more realistic AI digital human? Different content entails different degrees of product maturity. Digital humans are more developed in China — I've heard JD.com has livestreaming digital humans, though such livestreams offer minimal interactivity, mainly product promotion.

What's more common in the market are digital humans, focused on making the persona's appearance, voice, and gestures more human-like. This differs significantly from our focus. Our approach uses multimodal information to understand a person's state — are they happy? Confused? This is crucial information for task processing. We track mental states and adjust the entire dialogue system based on feedback. This is work we've done extensively at the research level — interview practice, social conversation, gently gauging preferences for certain topics through non-intrusive methods. We're gradually productizing this research: negotiation training, communication training, using multimodal information to understand people and enable machines to know them better, achieving human-computer interaction.

Oasis Capital: From a user perspective, how important is an agent's persona? In the sci-fi film Her, there's no physical form, yet she's exceptionally intelligent with a pleasant, appealing voice — isn't that sufficient for users?

Professor Yu: It depends on the scenario. In professional contexts, persona may not matter. In companion scenarios — teaching math to children, for instance — kids find concrete personas more conducive to interaction and demonstration.

Oasis Capital: From your research perspective, what are the biggest challenges in multimodal dialogue systems currently?

Professor Yu: First, data collection. Second, how much multimodality actually improves products in specific scenarios. Could you build a good product without these multimodal signals? So multimodality and scenario are strongly coupled.

Oasis Capital: Is the lack of clear downstream demand causing insufficient upstream data, creating research difficulties?

Professor Yu: It's varied. Take interview scenarios — understanding and adapting to users is important. Humans gather more information to improve communication quality; similarly, can machines improve their communication with additional information? But everything has costs — recording video, processing video, computation. Which scenarios justify these costs? Research isn't something professors decide arbitrarily; it requires funding. Which institutions will fund what? If functionality is unclear, industry rarely invests. Governments are similar — is it essential for defense, for intelligent scenarios? Academia has market-driven research, future-oriented research, and purely curiosity-driven projects.

Oasis Capital: In recent years, has your actual research been more cutting-edge than industry, closely connected, or is industry more implementation-focused?

Professor Yu: Academic and industry research are inherently different. In AI, it depends on what each lab's PI chooses to pursue. AI originated in universities — speech recognition, for example. Only in the last 5-10 years has industry begun hiring AI researchers, after discovering application scenarios. Google has done much influential work — Transformer, for instance — which profoundly impacted academia; we now all use Transformer as our foundation model. But many academics disagree, believing academia and industry should differentiate. Competing with industry is unreasonable given their greater resources and direct implementation scenarios.

Academia should pursue more frontier research — assuming when immature technologies will mature, synthesizing what others lack imagination to combine, and starting research early. I began working on multimodal dialogue systems ten years ago, when visual unsupervised emotion detection was just becoming somewhat reliable. Industry would consider the technology too immature for products for another four or five years. High-risk, long-duration projects don't easily gain industry support.

Oasis Capital: Have multimodal dialogue systems achieved large-scale industry deployment?

Professor Yu: Not really — it ultimately comes down to scenarios. Industry has many constraints and policies that are hard to anticipate. A more important issue here is privacy. How do you structure user agreements for multimodal systems? American users increasingly value their privacy; European users even more so. From the user perspective, privacy protection concerns — will you record my video, how will my video be handled — require careful consideration. From the company perspective, what's the exchange value in promoting this product, how do you gain user approval? If recording video means better user experience, will users choose it? This isn't a simple yes-or-no proposition; it involves many legal dimensions. Privacy is one reason implementation is difficult. Beyond privacy, there's computational cost — is the product experience improvement worth the increased cost?

Oasis Capital: How do you view the claim that "many scenarios don't need large models, small and medium models are sufficient"?

Professor Yu: Whether unit economics work is one matter. From the perspective of human progress, the more information machines can receive, the more they understand their surroundings and conversational partners, enabling relatively better responses and greater intelligence.

Oasis Capital: OpenAI's LLM suddenly swept the globe. Alongside its benefits, are there drawbacks?

Professor Yu: Privacy and security concerns are fairly prominent. Some European countries banned OpenAI due to privacy issues, because OpenAI didn't provide reasonable, trustworthy handling methods to European authorities. And of course models still "hallucinate" — generated content that's half-true, half-false can mislead users.

Oasis Capital: Does your research address security through pre-training or fine-tuning? Or what are the respective approaches for research versus implementation?

Professor Yu: These differ substantially. Academia does much work on bias and fairness. LLM training data is scraped from the internet, and the sources themselves contain biases. Researchers evaluate what categories of output bias stem from source bias, and whether models amplify these biases. Many labs focus on testing bias; others approach it from the embedding angle. Our previous collaboration with NYU researchers worked on "debiasing embeddings," addressing some bias upstream so downstream applications improve significantly. You can also work from the data side — if you detect known biases in certain data, you can study whether regenerating data can mitigate bias and retrain models.

Machine learning offers various ways to handle this. But LLMs are different — with such massive data volumes, solving data problems at the root is unrealistic. However, from a machine learning perspective, you can address bias from the upstream data, to the second-layer embeddings, to downstream applications, using various methods.

Beyond bias and fairness, there's toxic language — hate speech, for instance — which is another distinct research area. We certainly do data cleaning, but inevitably miss some odd content. Major companies build their own filtering APIs, like Perspective, though these don't work particularly well. Toxic language evolves constantly; much racial bias requires context to detect, not to mention coded language that can't be easily identified through simple keyword matching.

Model security is another research direction. Once deployed, can malicious actors attack your model to extract original training data, potentially leaking user privacy in the process? One security solution involves adding noise during model training, so the resulting model itself has security guarantees — external attacks can't retrieve original data. Of course, such models trade off security for usability; you must balance these. My group also works on differential privacy for NLP.

Given enormous demand, academia has invested heavily here. If major companies want to use AI for practical tasks, they must address security, bias, ethics, and so on. Fairness, for instance — hiring systems must ensure AI used for resume screening is inclusive across backgrounds; bank loans, AI-influenced decisions — these all require authenticity and fairness, otherwise legal disputes arise in practice.

Returning to my own research, our domains are less broad, based on specific scenarios — negotiation, training, communication — each with fixed scenarios and roles. Limited degrees of freedom let machines detect when they stray from domain; you can tell the model what task we're doing and that it must stay on task. We certainly do precise filtering to ensure generated content is safe. When users say inappropriate things to the machine, it can respond: "Discussing this topic makes me uncomfortable. Let's return to [X] topic."

Oasis Capital: Why does commercial product controllability depend on product characteristics themselves?

Professor Yu: Because specialized domains mean people won't circumvent various things to attack you. OpenAI has many people finding convoluted ways to make models say inappropriate things. If our machine clearly knows it's meant for one specific task, and everything else is outside its scope, we can avoid some problems.

Oasis Capital: So companies building foundation models need to pay more attention to these issues?

Professor Yu: Yes, but specialized domains also need attention — psychological counseling is high-risk, education is high-risk. Answer accuracy matters; you can't mislead students.

Oasis Capital: Do you think OpenAI has addressed this well?

Professor Yu: Not well (laughs). I asked ChatGPT to introduce Professor Zhou Yu from Columbia. It fabricated a lot of untrue content. Especially the half-true, half-false stuff — for those who don't know better, it's very easy to be misled.

Oasis Capital: With so many cases to address, shouldn't common scenarios be prioritized?

Professor Yu: That's the industry view. From academia, we design various algorithms to discover better solutions, rather than writing if-then statements or using more training data to improve performance. RLHF is also a way to reduce factual inaccuracies. Academia has done much work to reduce "fabrication," but so far there's no highly effective method, making this a core research direction.

Oasis Capital: You were a pioneer in open-domain social dialogue systems, among the earliest to focus on chatbots. I believe that original passion remains. Is this still a lifelong pursuit for you?

Professor Yu: Yes. When I first started working on dialogue systems, most applications handled fixed tasks — booking flights, reserving hotels, with simple information flows. As a robot, I knew what hotels existed; as a human, you knew what kind of hotel you wanted. The process was simple information exchange with an obvious task for the machine to complete. But I found mere information exchange insufficient. In many scenarios completing complex tasks, user experience matters enormously — it's not accomplished through simple information exchange.

Interaction has two dimensions: task engagement and social engagement. Remove task engagement, focus only on social engagement, and you can see whether the system demonstrates natural interactive states. For flight and hotel booking, even poor user experience is tolerable if the booking ultimately succeeds — people endure and continue using. But complex scenarios are completely inadequate.

Chatbots that engage in "chitchat" without fixed tasks represent another extreme — without concrete tasks, task engagement disappears, leaving only social engagement to study. We've done extensive work on chatbots: how to communicate proactively with users, how to measure interaction quality. We've also worked on multimodality because understanding users — where they look, their expressions — matters significantly. This isn't about making machines identical to humans, but making human-machine interaction feel as natural as human-human interaction. A robot capable of natural human interaction is our ultimate goal. (We are aiming to build agents that can elicit natural human behaviors.)

When cognitive load between human and machine is minimal, indistinguishable from chatting with a real person, user experience becomes optimal. Human-human interaction sometimes involves tasks, sometimes doesn't, sometimes alternates — it's quite complex. To achieve ultimate natural interaction, machines can't merely complete tasks; they need social capabilities.

Once "chitchat" reached a certain level, we began mixing "chitchat" and "tasks," creating social bonds between humans and machines: I need to persuade you to exercise, give you emotional support, improve your mood, train you in negotiation, teach you negotiation techniques... Dialogue isn't a single task; it requires planning. This complexity gives machines higher-level capabilities.

Oasis Capital: After AutoGPT emerged, autonomous agents grew rapidly. From the social attributes perspective, what's your view?

Professor Yu: I think AutoGPT and similar are overhyped. We've tested them — the simplest test, checking weather for the past 7 days, and it gets stuck on some API. More often, if you give it a large task, it can plan, write scripts, and step-by-step call specialized models for execution. The concept is completely valid, and something people have long advocated — decomposing diverse tasks into modular tasks, each handled by a specialized model.

Linguistics calls this "compositionality," which parallels how language itself forms. First learn nouns, verbs, then combine nouns, verbs, prepositions to generate meaning. Tasks work similarly — any complex task can be decomposed into smaller tasks completed by specialized models. Whether current models can autonomously do this depends on where they get stuck, how they unlock themselves — seek human help, know their own limitations, independently find relevant training data online to self-update?

Research directions are broadly similar: one, how to involve humans in improving machine intelligence; two, whether machines can independently find suitable new data online for training and parameter updates.

Oasis Capital: Are there corresponding metrics for defining multimodal dialogue system quality?

Professor Yu: Different metrics apply depending on your task. For customer service calls, the most important metric is whether the task was completed, whether user needs were met. Also dialogue efficiency — was it an hour or 10 minutes to resolve? And whether users had positive experiences and are willing to reuse the system. These are customer service metrics.

Education's evaluation criteria relate closely to whether students learned the material. Entertainment's most important metric is user retention. Different scenarios have different evaluation standards.

Oasis Capital: Do multimodal dialogue systems require far greater computing power and data volume than unimodal systems?

Professor Yu: Yes. Processing video alone demands high computing power, plus various technical implementations — video input, storage, processing — all have costs. What's more common in the market currently is language plus images; audio is basically absent, primarily because data volume requirements are too high. Finding different parallel data is itself difficult.

Second, insufficient application demand. Vision and images have clear quantitative foundations — academic models like CLIP, later Flamingo vision-language models. GPT-4 now has image input, all because demand exists. Text-to-image, or language-image combined understanding, has substantial demand. Going forward, people may want text-to-video; many are working on this. Video itself contains audio, which can be converted to text via ASR (Automatic Speech Recognition), plus vision, and finally video itself can serve as input information.

Oasis Capital: What's distinctive about speech recognition training?

Professor Yu: Commercial speech recognition models are already quite good. What's distinctive is, for example, strong background noise requires speech enhancement, with前置的信号处理; also significant variation across populations in speech recognition — children have immature vocal organs, differing from adult pronunciation; or elderly speech that's unclear, making recognition difficult; and accents, since no standard speech exists. Amazon invested heavily in collecting accents to build corpora and retrain models — what we call resource-rich languages — achieving very good speech recognition. Text is similar; due to language diversity, GPT performs poorly on low-resource languages.

Oasis Capital: The AIGC wave has prompted many to start companies. Some say that by combining tools like AutoGPT and ChatGPT, companies that previously required 100 people might now be built by one or two. Is this reality or exaggeration of model capabilities?

Professor Yu: One or two people is definitely impossible — you need front-end, back-end for a product. However, with new tools, some engineering tasks can achieve 70-80% completion by wrapping APIs. The crucial point in entrepreneurship is that you're not building a product but a service — are your end users satisfied? How you use technology to package your service is itself an art.

Oasis Capital: From your dual background in linguistics and computer science, how do you view LLM's future impact and trends for the world?

Professor Yu: It is indeed transformative — not in baseline algorithmic changes, but in making models more powerful through scale. This showed the world possibilities, making it accessible for everyone to adapt models to their ideas, allowing traditional industries to gradually understand AI. AI isn't something experts build and promote; it requires domain insiders to understand and co-develop. The current moment is an excellent opportunity.

Of course this boom has some bubble characteristics, but overall it provides an enlightening education about AI. What are models actually like? What can they do? What can't they do? Can your needs be addressed through models? OpenAI and ChatGPT's demonstration showed many possibilities. AI experts, or those with technical experience, can consider how to interface with traditional industries and co-develop implementable products.

Oasis Capital: Do you think LLMs already have a bubble?

Professor Yu: I think too many people want to do the same thing, which structurally creates excessive competition. And many previously outside this domain suddenly decide they want on board. The grassroots may be overheated; if teams aren't suitable, they'll quickly fail. Entrepreneurship still requires caution — whether founders are suited for this, whether they have appropriate teams. It's not that because there's a boom and large market, jumping in guarantees success. You must have advantages over others, advantages that can convert to real returns, while thoroughly understanding your own strengths.

Vitality

What do you think is technological vitality?

Innovation is the foundation of technological vitality, but where lies the direction of innovation? I believe technological innovation must effectively integrate with social welfare, driving positive social transformation, to endure.

— Professor Zhou Yu Columbia University Computer Science Department

Oasis Capital is a new-generation venture capital firm in China, dedicated to discovering the most vital entrepreneurs of the next decade and growing alongside them to create long-term value. "Vitality" is both Oasis Capital's vision and mission. This vitality represents both the direction of structural transformation in the era and the resilience and evolutionary power of entrepreneurs.

Oasis Capital focuses on early and growth-stage investments, with individual investments ranging from $3 million to $30 million, concentrating on robotics, artificial intelligence, and technology services to support China's new service upgrade driven by technology.