Oasis Capital in Conversation with Professor Shang Jingbo: One Keystroke

What is the ultimate form of automation? How can extremely weak supervision be achieved? How can humans and AI complement each other's strengths? During WAIC, we had a delightful conversation with Professor Jingbo Shang from the Department of Computer Science and Data Science at the University of California San Diego. Professor Shang embodies the principle of "unity of knowledge and action" in the truest sense. Enjoy.

Oasis Capital: What was your original motivation for studying computer science?

Professor Shang: My original motivation for studying computer science was the idea that you could press Enter and have everything automated. When I started my PhD, I realized the scenarios where a single keystroke could solve everything were extremely limited. In real life, automation typically requires massive amounts of manual annotation as a prerequisite. So my research shifted to: how do you automate the entire process of building automation itself, minimizing the need for human data labeling?

There are several approaches to reducing manual annotation:

The first is distant supervision. A common distant supervision method uses Wikipedia or existing knowledge bases to guide a given task. Typically, these knowledge bases weren't designed specifically for the task at hand, so the supervision obtained this way is "distant" — hence the name. We can use distant supervision for many tasks, such as finding entity names in text, identifying relations between entities, and so on. Wikipedia contains abundant internal hyperlinks that already mark lots of entity and relation information; we extract the portions relevant to our specific task, train models on them, then use these trained models to extract more knowledge and information from new text data. This way, distant supervision can address many knowledge extraction or information extraction tasks.

The fatal flaw of distant supervision methods is that they're terrible at expanding knowledge boundaries. The trained models easily get trapped within the scope of the knowledge base. Infrequent, long-tail, and emerging information is almost entirely missed.

The second approach is unsupervised learning. You feed in data, and the model learns whatever you give it — no supervision whatsoever. The output is entirely determined by the data. The problem here is that the same input data might serve multiple different needs, which unsupervised methods can't accommodate. For instance, if I want to categorize news articles, the same article could be classified by topic, location, or event type — each requiring completely different criteria.

Even with data in hand, humans still need to tell the model what they want. This "telling" is supervision. If you explain in great detail, walking through many examples, that's the familiar fully or semi-supervised approach. If you explain simply, that's weak supervision. And if you explain as simply as humanly possible — that's extremely weak supervision, the third approach that our lab (Shang Data Lab, SDLab) has been pushing in recent years.

This is what I want. Every task has its own version of extremely weak supervision: harder tasks need a bit more explanation; simpler ones need less. The scope of supervision varies too. "Full-scope" means informing the model about all possible outputs. For text classification, the outputs are the class names. In a full-scope scenario, you'd tell the model that every news article must belong to one of five categories — sports, politics, economy, science, culture — and every new article must fit into one of these. This closed-ended multiple-choice format is full-scope.

The downside is you need to know everything before doing the task. But even domain experts often can't achieve full-scope completeness. We're computer scientists ourselves, and we'd struggle to confidently list all subareas of the field.

To address this, my vision is to move toward knowledge discovery — not closed-world extraction, but open-world discovery. Discovery still needs human guidance: tell the model what you already know that's most common. For example: "These are news articles. I want to classify them by headline. I only know about sports news and political news; for the rest, I don't know, you figure it out." This is partial-scope guidance because it covers only a small fraction of possible outputs. Though this instruction is extremely brief, if the "model" were an educated human, they'd have nearly everything needed — what type of text is the input? What's the classification criterion? What's the granularity? So this should theoretically be feasible. When all necessary information is present, can computers complete the task? What's missing is high-quality representation. Our goal is to find efficient methods that organize data into a hidden space according to human guidance, ultimately completing the task with 80-90% accuracy — that would be excellent.

After this step, humans can provide feedback, and the data flywheel starts spinning. Traditional active learning, semi-supervised learning, and so on all become applicable. From a product perspective, users need a great first experience to keep using. Getting 80-90% accurate information from just a few sentences creates such a good experience that users are motivated to continue, adjust, modify, and give feedback. This starting point matters. In research too, we want our open-source software and tools to achieve plug-and-play effectiveness across different vertical datasets — extremely weakly supervised knowledge discovery. I used classification as an example earlier; our group also works on multi-word phrase mining. Phrase plus classification gives you entities. Entity plus phrase plus classification gives you relations. Working through this systematically, the classic problems and algorithms across the entire information extraction field can be reshaped.

Ultimately, many things come down to classification, and finding specific spans and candidates. Currently our group has invested heavily in the classification step, seeing how far we can push it. Since large language models emerged, especially as OpenAI's latest GPT models no longer release model weights, we've begun exploring new work based on LUI-only (language user interface-only) LLMs. LLMs excel at brainstorming — a capability particularly suited to proposing valuable candidates during discovery. So user guidance can become even more general. In classification, for instance, users don't need to provide any class names at all — just give broad directions for classification (plus perhaps indicate whether a few articles should group together). Our latest methods, GoalEx and ClusterLLM, can present both classification criteria and results directly to users, even offering several tiers of granularity to choose from.

Oasis Capital: What have you learned from researching and working with language models?

Professor Shang: In terms of my journey, with early language models, we could access model parameters and own their embedding outputs. From the earliest ELMo, BERT, and BART to more recent models like T5, you could fine-tune them — get the data, fine-tune it, or adjust the embedding space to develop new methods and models. The biggest challenge now is that the largest, best models (like GPT-4) only offer a language user interface; you can't access embedding data. And they're essentially limited to commonsense knowledge, because fine-tuned model inference costs multiply several-fold. So personally, I don't think the fine-tuning path is sustainable, or at least its applicable scenarios will be very limited. Even if hardware and algorithms advance, building the best models will always require greater scale; and once scale increases, ordinary people simply can't afford to run these models themselves. Packaging them as services — selling subscriptions, selling APIs — this model will definitely persist for the next 5-10 years. And there will only be 2-3 companies doing this at the top level, just as there's virtually no market for off-brand CPUs or GPUs.

Our team basically doesn't do training LLMs from scratch — the barriers in compute and capital are too high, and others are already doing it well, so there's no need to compete. Our original technical route has always been building applications on top of language models, using extremely weak supervision to quickly adapt across different verticals, languages, and tasks. GPT-4 actually gave us a solid baseline. Our latest papers basically all use GPT-4 for comparison — our methods ultimately apply GPT-4 combined with human supervision and domain data to achieve better accuracy, or even directly train a relatively smaller but more effective language model.

I believe "intelligence = knowledge + reasoning." For LLMs, they essentially have only two things: knowledge (Commonsense Knowledge) and logical reasoning ability (Reasoning). The former is intuitively demonstrated by how GPT-4 can basically memorize 40+ TB of pure text data. This is something humans can't even approach. But GPT-4's reasoning is still quite weak — weak enough that it only ranks in the bottom 5% in Codeforces programming competitions. I think reasoning can be trained to the level of an average middle or high school student; probably not college level, since that requires specialized knowledge. But the level of an average middle or high school student is already quite meaningful.

The first time I used ChatGPT, I told my PhD students and friends: don't overestimate its capabilities. It's great, but don't hype it to death. Because ChatGPT from GPT-3.5 onward is supervised — the first few questions that pop into your head have very likely already been seen and tuned during model training. In the deep learning era, zero training error is completely unremarkable. Current LLMs have some generalization ability, but still fall short of high-standard "flexible application." Actually, the human brain is the same — think back to classmates who were excellent at exams, incredibly fast at problem-solving, but completely stumped when questions got creative. From the "intelligence = knowledge + reasoning" perspective, it's massive knowledge reserves allowing simple problems to be solved through direct "lookup," but lacking reasoning and analogical ability.

The intelligent model we truly expect is one where knowledge doesn't need to be large (new knowledge can even be directly inserted), with strong reasoning and generalization, capable of completing tasks it hasn't learned — given a few new axioms, can the model derive all theorems that can be evolved from them? This is what current LLMs haven't solved. And this capability is the core ability truly needed to use LLMs for domain-specific tasks.

An important direction our lab is currently pushing is how to use large models with extremely weak supervision provided by humans to help train (small) models. The stronger large models get, the happier I am, because they can provide us with cheap supervision. For example, in a recent project called ClusterLLM, on average just 60 cents of OpenAI API cost per dataset can make a relatively small model outperform the large model (GPT-4). Our success mainly borrowed the large model's reasoning ability to pull the small model's originally kindergarten or elementary school-level reasoning up, then combined with domain-specific data for fine-tuning — small model fine-tuning is extremely efficient and cheap compared to large models; the best and biggest large models currently can't be casually fine-tuned by everyone. Small models trained this way are simply better than general-purpose large models themselves on specific verticals and specific tasks.

Separating knowledge and reasoning: knowledge learned from domain-specific data, reasoning learned from large models, with just a bit of human guidance — this is my current research direction — in the future everyone can train one or more small models for themselves, each performing well in the verticals or specific tasks they need, accepting human feedback, fine-tunable, and efficiently iterative. In the end, supervised learning will be greater than or equal to unsupervised learning. And the core of my research is how to make this first step better and more stable.

Oasis Capital: Will AI replace humans?

Professor Shang: LLMs are a foundation; there are dedicated teams working on that. That's not our ambition. The future needs many small models complementing large model capabilities, using general-purpose large models as CPU-like engines to call upon, combined with vertical data to solve problems within verticals, reaching 60, 70, 80, 90-point states. I think 100 points is impossible — don't even think about it, 95 is very hard. That final portion of improvement will definitely come through interaction between AI and humans.

There's also liability. Often AI can already do better than human average — autonomous driving in some scenarios can already match human average. But people's expectations of AI are very high; even 99.99 isn't enough. Why? Fundamentally it's a responsibility problem. When something goes wrong, who's responsible? Sentence the AI to prison? I don't think AI can ever replace humans in making decisions. Unless laws and ethics catch up, everyone accepts that crashes don't require anyone to take responsibility, or someone or some insurance company is willing to proactively assume liability — only in that kind of environment could it possibly happen.

The reason I'm personally bullish on LLM applications is that they're not making decisions for people, but assisting people in making decisions. The term AI — if I were to explain it, I'd say Augmented Intelligence. It's fundamentally not something that replaces or imitates humans, but expands the boundaries of (human) intelligence.

From an evolutionary perspective, from plants evolving to animals, the first two abilities to emerge were perception (Perception), such as vision, and mobility (Mobility). The simultaneous appearance of "seeing" and "walking" is no coincidence: "seeing" is for better "walking," "walking" is for better "seeing." "Walking" is still quite difficult for Embodied AI currently. Why is walking so simple for humans? Look carefully at people around you — it's easy to notice everyone's gait is slightly different. From a machine learning perspective, the model (human joints, muscles, soft tissue, etc.) is large, parameters (how to control joints, muscles, etc.) are numerous, so solutions naturally abound, and humans easily find feasible ones. Everyone's feasible solution differs somewhat, but all work. What's hard about robots walking? Mechanics limit their "solution" space.

I use the "walking" example mainly to illustrate that AI and humans excel at different things. LLMs as AI can effortlessly memorize 40+ TB of pure text — humans can't; conversely, we humans only need to study relatively small books to master complex reasoning, which current LLMs can't do.

Since it's not good at it, why force it? Many problems can indeed gradually improve and be solved with time and effort from a long-term perspective, but why not take an "AI + human" path? AI isn't good at walking, isn't good at reasoning; humans aren't good at memorizing massive amounts of data — so we put AI on people's heads: humans serve as AI's legs, AI provides relevant knowledge for humans, humans use this knowledge to reason, each contributing their strengths. Back and forth, this augments the boundaries of intelligence. Robots for specific scenarios should be built, but don't need humanoid form; the goal isn't replacing humans, but doing what humans don't want to do or what's dangerous. Since AI and humans are good at different things, there's no need to make them the same.

This is my vision for the future. In this process, a large portion of ownership actually lies with the Agent Creator. This person contributes much data, spends much time tuning the AI. So where can we as researchers contribute? This actually circles back to my research theme of Extremely Weak Supervision: first, lowering barriers so people who can't program can create intelligent agents with data. This has always been a dream of mine — hoping to lower the barrier to training and creating AI so every ordinary person can participate. Once this barrier is successfully lowered, creating AI will become as simple as using smartphones, changing our lives in every aspect. Otherwise business can't scale — it's forever just people who already could program getting better tools.

AI won't replace people. People who use AI might replace people who don't; people who use AI well might replace people who use AI poorly. So I strongly recommend everyone try it, embrace new technology, at least experience ChatGPT-like products, write some complex prompts, and feel for yourself where ChatGPT's capabilities currently stand.

Oasis Capital: What's the ideal state of "AI + human"?

Professor Shang: I think in the future humans and intelligent agents will exist equally in a social network. When a specific problem needs solving, you can assemble a team of relevant (multiple) intelligent agents and (multiple) people, create a group chat, and discuss and solve problems together. In this process, intelligent agents learn human reasoning, humans learn knowledge provided by intelligent agents. If such an ecosystem can truly be realized, I think it would be ideal.

The difficulty here is whether LLM reasoning ability is sufficient to support different verticals, whether it can reach the level of an excellent high school graduate or even college student. As more people work on LLMs, we believe this capability will strengthen. I think "vertical" = "major in university," mainly providing new data for people/AI with sufficient reasoning ability. With new data and reasoning ability, only extremely weak supervision is needed to complete the initial creation of intelligent agents.

Oasis Capital: LLM capabilities will keep improving; the debate in the middle is whether as LLM water levels rise, they will erode the survival space of vertical applications or small models?

Professor Shang: As we mentioned earlier, the core of vertical intelligent agents is Data + Reasoning. The amount of data currently used to train LLMs is already enormous, basically exhausting high-quality data. From this perspective, LLM knowledge reserves are hard to improve further. However, LLM reasoning ability will definitely get better and better. For vertical applications, much data is often private. Without data, great reasoning is useless. This data closedness means general-purpose large models, even if they eventually become smarter than all humans combined, still only have ordinary knowledge of ordinary people — they can't learn all verticals. So I believe vertical applications and vertical small models definitely have survival space.

Oasis Capital: Let's explore this point — is there a possibility that when foundation model reasoning becomes strong enough, for instance some great scientist predicted that you could throw Euclid's three laws into a dialog box and the LLM could derive all of Euclidean geometry?

Professor Shang: It would be fantastic if we could eventually reach that level. The "three laws" here can essentially be understood as private data within the vertical of Euclidean geometry. I'm personally quite interested in education — this is the ultimate goal of education. My favorite kind of student is the one who, given only axioms, can derive all the theorems without coming to class, just skimming the book a week before the exam and acing it. That goal is perfect, but there's a long road ahead.

I have another interesting perspective on LLMs. The highest award in computer science is the Turing Award. What achievement would guarantee you a Turing Award today? One path would be proving "P=NP." In layman's terms, NP problems refer to a class of problems where finding an answer is very hard, but verifying an answer is very easy. To prove "P=NP" essentially means finding an extremely efficient way to find valid answers for NP problems (like SAT problems). What's fascinating to me about LLMs is that they can propose an answer from a completely different angle.

(Oasis Capital: This is what Terence Tao meant when he said GPT gives him inspiration for solving math problems.)

Exactly. If LLMs could be trained in that direction — finding an NP problem that machines can easily verify (like SAT problems) — perhaps LLMs could quickly generate many candidate solutions for an NP problem, and among them there might very likely be a feasible one. If that could actually be achieved, it would at least represent a leap in productivity within the computer world. But this currently remains at the stage of a bold hypothesis of mine; it hasn't been put into practice yet.

Oasis Capital: So overall, you believe LLMs represent a vast ocean of possibilities. From a practical standpoint, vertical private data combined with foundational LLM reasoning capabilities performs better in specific scenarios — is that right?

Professor Shang: Yes. Beyond that, specific combinations of public data, paired with LLMs' reasoning capabilities, can also yield some innovations. All of these innovations need to attract the masses to participate together, because everyone's ideas and application scenarios are extremely unique. Only when everyone gets involved and the data flywheel starts spinning can things get better and better. In the end, supervised learning will always be greater than or equal to unsupervised learning.

Oasis Capital: The majority in both industry and academia hold this view, and the concern it raises is that this wave of AI will end up looking very similar to the last one in terms of industrial deployment. The previous round also had very high expectations for CV, but in the end that last 5% gap couldn't be closed — hardware issues with cameras, for instance. A model adapted for one device might not work on another. This is somewhat like public data versus private data. Will this same problem persist in this round?

Professor Shang: You're right about the problems with that CV wave, and it's a common ailment of AI model transfer. But from my research perspective, it's not about taking a model trained on a vertical task like news classification and using it for medical case analysis — that definitely won't work. Our goal is to start from any private data, combine extremely weak supervision with LLMs' reasoning capabilities, and quickly obtain a vertical task-specific model that scores 80 to 90 points. Looking back at our earlier news classification example, we only need news data, no labels whatsoever, and we can achieve around 90% accuracy. In my research, the common component across different verticals is the LLM's reasoning capability; everything else is private to each vertical. That's why there will be many small models. These small models operate without labels — say you have two people, one with lots of medical cases and one with lots of news, both doing classification. In the end you'll have two different agents, each bringing their own data to me: one says, "I have news, classify by topic — politics, sports," and we can get 80-plus points; the other says, "I have EHRs, classify by disease type," and we can also get 80-plus points. There's no transfer in this process. What we have is the same algorithm, combining extremely weak supervision with LLMs' reasoning capability, creating two different agents.

Oasis Capital: If going from 80 to 95 points, does your method become difficult to transfer?

Professor Shang: That's when the data flywheel kicks in. If someone continuously uses the same agent, discovers problems, and teaches the agent — a few examples in, and the results can improve dramatically. Going from 80 to 95 points isn't that hard; the hard part is how to reach 80 points in the first place.

Oasis Capital: So the difference between the two waves is: previously, the foundation contributed 20% and domain-specific work did 80%, making costs very high for each domain, with high transfer and scale costs. But this wave follows the 80-20 principle — the foundation provides 80%, the same method accounts for 20%, and the same method can be applied across multiple domains, spinning on its own without needing labels, so costs come down. With that said, do foundation model companies have significant advantages?

Professor Shang: Exactly. My research is dedicated to making the foundation provide that 80%. Having LLM reasoning capabilities and private data alone isn't enough. As we mentioned earlier, the same dataset can give rise to completely different needs from different people. So this process definitely also requires extremely weak supervision methods to truly understand what the user wants to do with their data.

Oasis Capital: Can we talk about issues of safety, controllability, and trustworthiness?

Professor Shang: On safety and security, our group also has initiatives. Recently DeepMind did a program optimization, and later someone said they prompted ChatGPT and got the same code. Has anyone considered whether GPT or LLMs have a universal prompt capability (analogous to neural networks' universal approximation capability)? For instance, if I specify an output (like that optimized program), does there necessarily exist a prompt that can produce the exact same output, or is there an extremely high probability that this output can be generated? Is this proposition true?

Our lab recently built a tool that might help us answer this question.

This is fundamentally a deep neural networks (DNNs) problem: how to understand the input-output mapping relationship of a trained DNN.

Take image recognition. Suppose we've trained a model (ResNet, CNN, or MLP) to distinguish whether an input image is a handwritten 0 or 1. All input images are 28-by-28 pixel square grayscale images. Since grayscale is typically an integer from 0 to 255 in computers, the possible inputs amount to 256 raised to the power of 28 times 28. This is an enormous input space, with a vast number of different possible input images. And since it's a classification problem, the output is actually just a number between 0 and 1 representing the probability that the image is the handwritten digit 1.

What input-output mapping relationship are we interested in? Specifically, how many inputs (like the grayscale images mentioned above) can lead to a particular output (like "probability of being handwritten digit 1 equals 0.9")? This is essentially trying to obtain an output histogram — with the X-axis being output values and the Y-axis being the number of inputs corresponding to each output value.

To solve this problem, our latest work (ICML'23) can already provide a relatively efficient, non-brute-force-enumeration, sampling-based estimate of the output histogram. At the same time, we can also tell you what representative inputs look like for each output value. We mainly work on image models because their output is a single number.

Returning to LLMs: what is the model's output? It's a piece of generated text. What is the input? It's a prompt. If given a prompt, through LLM computation, we can easily obtain the probability that this prompt produces a certain piece of text. Analogous to the image recognition example, if we treat "the probability that this prompt produces a fixed piece of text" as the output (this too is a number between 0 and 1), we can use similar methods to obtain an output histogram characterizing how many prompts (of fixed length) (Y-axis) have what probability of generating this fixed piece of text (X-axis).

This output histogram actually has strong guiding significance for security and safety. If for a given application we have a blacklist of content that must not be generated, we can use this output histogram to examine whether many prompts exist that can generate this blacklisted text with high probability. If there's a high peak in the Y-axis where X is close to 1, then the model is unsafe. If there's only a single peak where X is very close to 0, then the model is very safe. With this tool, we can gain a more thorough understanding of LLM generation.

Oasis Capital: This is somewhat like penetration testing back in the day — if the system is a black box, you probe through constant trial and error looking at outputs, essentially requiring a certain amount of reverse engineering.

Professor Shang: Exactly. This task is extremely difficult because we're dealing with arbitrary inputs. The input space is enormous. We need an efficient sampler to directly estimate what this output histogram looks like. We've already published a paper on image inputs at ICML'23, mainly drawing on sampling techniques from condensed matter physics. Currently, we're pushing forward with applications on LLMs.

Oasis Capital: What about trustworthiness issues in the medical domain?

Professor Shang: My view on triage is that the final output given to patients must still be rule-based, still following existing protocols. This ties back to the liability issue I mentioned earlier. Any attempt to use AI to replace human decision-making while bearing responsibility is unrealistic, unless there's legal framework or insurance companies willing to take it on.

How to translate plain language that ordinary patients use into highly professional, domain-specific language that can ultimately match the rules mentioned in protocols? This is where LLMs can help.

Oasis Capital: What does this wave of AI mean for the education system in the United States?

Professor Shang: The moment ChatGPT came out, our school immediately formed a task force to study how ChatGPT could be used in university teaching — which use cases should be encouraged, which shouldn't, and whether it would affect academic integrity. They eventually produced a guideline. From my perspective, all university students should be using ChatGPT. In my own teaching, all my assignments and exams are take-home, with at least 48 hours to complete them. Even before ChatGPT, students were allowed to search the open internet. Being able to find answers online was itself a skill. Anyone who can use ChatGPT to extract answers to these questions — I consider them competent enough to deserve full marks.

What AI excels at is eliminating repetitive labor. As I mentioned earlier, people who use AI will outperform those who don't; people who know how to use AI will outperform those who merely use it. Of course, this "outperformance" doesn't mean AI replaces humans — it means people who master advanced tools outperform those who don't. This has been true throughout history with every wave of productivity growth.

We also discussed earlier: "Intelligence = knowledge base + reasoning ability." Knowledge evolves continuously with the times and with personal development — new knowledge keeps getting added. So I believe the core of education lies in cultivating reasoning ability. How do we cultivate reasoning ability? By using specific knowledge points to teach and practice it. It's like how we still learn arithmetic and do hand calculations even though we have calculators. Why? Because this knowledge is an essential building block for developing reasoning ability. ChatGPT's current reasoning ability is actually quite weak. I think once students have mastered the necessary foundational knowledge, introducing ChatGPT in education — especially at the university level — is an inevitable trend.

Oasis Capital: Do you have an ultimate goal or major milestone from a research direction perspective?

Professor Shang: I have to make good on my boasts (laughs). That one-line goal I mentioned at the start: solving everything with a single keystroke.

Oasis Capital: How is this goal different from AGI?

Professor Shang: My direction is more data-driven and more domain-specific. Our goal is to develop a methodology that can incubate different solutions for each specific vertical. From this angle, it's slightly different from AGI.

Our lab is essentially applying data mining methodologies to text-related data mining. Although we publish many papers at top NLP conferences, our approaches are all data-driven. I personally find it a bit regrettable that LLMs weren't created by data mining researchers, because they aren't linguistic at all — they're purely data-driven, which aligns perfectly with the philosophy of data mining.

Oasis Capital: What are your predictions for the future? At any time scale.

Professor Shang: First, better multimodal large models will emerge. Without better training methods, it will certainly come down to scaling up both data volume and model size together. One piece of work I'm very interested in is ImageBind, recently released by Meta AI. It uses many datasets, with each dataset basically containing images, image-text pairs, image-IMU sensor data, and image plus other modalities. Images serve as an anchor point, with everything embedded into the same space. This may represent the future trend for large models: integrating multimodal datasets, using common modalities as anchors, and ultimately stitching everything together into one system.

I'm working with Rajesh Gupta at UCSD on several PhD students. We're mainly focused on Smart IoT — AI plus IoT. The challenge here is that IoT data is extremely difficult for humans to read, and the volume is massive — a single sensor can generate 100 data points per second, and there are typically many sensors operating simultaneously. So how do you label IoT data? Currently, you bring people into a lab, have them wear specified devices, follow a fixed set of instructions to perform actions, and record all the timestamps — that gives you labeled training data. But this lab-collected data is very different from real-world data. So we have another direction: "make the human unreadable data readable" — anything humans can't read, we find ways to "translate" into images or text, making it interpretable and identifying the minimal semantic unit. This is where LLMs can help. What we do is ask the LLM: here's a person wearing sensors in these locations, and I want to determine if they're playing basketball — which sensors and what kinds of signals would be useful? The LLM can give you a detailed list. There will be redundancies and irrelevant points, but most of it is correct. We can use this as initialization, then do unsupervised alignment and correction, and it may genuinely become possible to automatically discover and generate minimal semantic units, like words in natural language. This is a direction we're recently exploring.

The major future trend, I believe, is large models and multimodality. Bigger models, more types of data fed into them, followed by iterations of various foundation models. In this process, our extremely weak supervision methods will become increasingly efficient, requiring less and less human intervention — eventually no data labeling at all. If brain-computer interfaces mature, people might just think about something and the work gets done. That's the symbol of a leap in productivity: the less human labor required, the higher the productivity. Finally, it will come down to whether inference costs can be reduced enough to achieve one model per person, five per person, even a hundred per person. Why did mobile internet become so big? Because phone costs dropped enough for everyone to have one or two phones. The problem with large models now is that the barrier is too high and too expensive. Even if APIs are cheap, actually using them effectively requires a fairly high technical threshold. For true commercialization, you need to lower the barrier and control costs.

On Vitality

What do you think constitutes technological vitality?

Time is the most expensive thing. "An inch of gold cannot buy an inch of time" is absolutely correct. So the value of technology lies in how to make time more valuable — how to use minimum human efforts to complete a task or achieve something? This minimum human effort, viewed from another angle, is maximized productivity. Different paths, same destination. This has been my consistent research philosophy.

——Professor Jingbo Shang

UC San Diego Department of Computer Science and Halıcıoğlu Data Science Institute

Oasis Capital is a new-generation venture capital firm in China, dedicated to discovering the country's most vital entrepreneurs over the next decade and growing alongside them to create lasting value. "Championing Vitality" is Oasis's vision and mission. This vitality represents both the direction of structural transformation in the era and the resilience and evolutionary force of entrepreneurs.

Oasis Capital focuses on early and growth-stage investments, with individual ticket sizes ranging from $3 million to $30 million. It concentrates on robotics, artificial intelligence, and technology services, supporting the upgrade of China's new service economy driven by technology.