Why We Shouldn't Deify ChatGPT | BlueRun Ventures

With ChatGPT's continued dominance in the headlines, you've probably been wondering about this very question too.

After several days of reflection, we'd like to share a measured, objective take. This piece comes from Dr. Xiaoyi Qian of Beiming Laboratory. A scientist and entrepreneur who has spent a decade advancing the symbolic AI tradition, he argues that ChatGPT has indeed pushed statistical algorithms' ability to "master patterns in samples" to an extraordinary level — and in doing so, it seems to possess a certain logical and problem-solving capability.

But this is still an illusion. A pure pre-trained model cannot possess human creativity, deep logical reasoning, or the ability to solve complex tasks. Fundamentally, ChatGPT's impressive performance is an "emergence" effect caused by model parameters and data scale crossing certain critical thresholds. To expect it to possess general intelligence akin to humans based on this alone is not rigorous scientific thinking.

This is a hardcore technical analysis, but Dr. Qian writes with remarkable clarity. If you're interested in AI, we think you'll find it well worth your time. Enjoy.

---

Over the past decade, connectionists have led the AI race, powered by various deep learning models and riding the tailwinds of big data and abundant compute.

Yet every time a new large deep learning model is released — like the recently viral ChatGPT — the initial wave of awe at its capabilities is followed by heated debate about the research methodology itself, and the model's flaws and limitations surface.

Recently, Dr. Xiaoyi Qian from Beiming Laboratory, a scientist and entrepreneur who has spent ten years in the symbolic tradition, published a notably calm and objective assessment of the ChatGPT model.

• Overall, we consider ChatGPT a milestone event.
• Pre-trained models began showing powerful effects a year ago; this time they've reached a new height, drawing wider attention. After this milestone, many work patterns related to human natural language will begin to change, with substantial replacement by machines.
• No technology is achieved overnight. Rather than dwelling on its shortcomings, a scientist should be more sensitive to its potential.

The Boundary Between Symbolism and Connectionism

Our team paid special attention to ChatGPT this time not because of the stunning effects the public saw — many of those we can still understand at the technical level.

What truly struck us was that in certain tasks it broke through the boundary between the symbolic and neural traditions — logical capability. In tasks like self-coding and code evaluation, ChatGPT seemed to demonstrate this ability.

We have long believed that the symbolic tradition excels at reproducing the strong logical intelligence seen in humans: how to solve a problem, analyze its causes, create a tool, and so on.

Connectionism, by contrast, is essentially a statistical algorithm for discovering smooth patterns from samples: finding the pattern for what to say next through sufficient human dialogue; finding the pattern for recognizing and generating corresponding images from descriptive text.

We can understand how these capabilities can become exceptional through larger models, higher-quality data, and reinforcement learning loops.

We believe humans embody characteristics of both the symbolic and neural paths. All reflectively accessible cognitive processes, knowledge acquisition and application, numerous reflectively accessible patterns of thought, behavior, and expression, and reflectively accessible motivations and emotions can be readily explained and reproduced in symbol-based systems.

When you've seen enough foreign faces, you gain the ability to recognize them without being able to explain why.

After watching one TV drama, you can naturally imitate how the male lead speaks.

After enough conversations, you can chat without thinking. These are neural characteristics.

We might analogize strong logical capability to growing bones, and "non-logical pattern mastery" to growing flesh.

Using symbolic "bone-growing" ability to "grow flesh" is difficult; likewise, using neural "flesh-growing" ability to "grow bones" is arduous.

As we've seen in building companion AI, symbolic systems excel at grasping specific dimensions of a conversational partner's information, analyzing underlying intentions, inferring related events, and giving precise advice — but they struggle to create smooth, natural dialogue.

We've also seen that GPT-style dialogue generation models, while capable of producing smooth conversation, struggle to use long-term memory for coherent companionship, generate reasonable emotional motivations, or perform sufficiently deep logical reasoning to provide analytical advice.

The "largeness" of large models is not an advantage but the price statistical algorithms pay to grasp certain underlying strongly-logic-driven patterns from surface-level data. It embodies the boundary between the symbolic and neural.

After gaining deeper understanding of ChatGPT's principles, we found that it merely treats relatively simple logical operations as a pattern to be trained and generated, without breaking out of the original statistical algorithm category — meaning system consumption still grows geometrically with the depth of logical tasks.

So how does ChatGPT break through the limits of ordinary large models?

How ChatGPT Breaks Through the Technical Limits of Ordinary Large Models

Let us explain, in non-technical language, the principle behind how ChatGPT surpasses other large models.

GPT-3 demonstrated superior experience compared to other large models from its debut. This relates to self-supervision — that is, self-labeling of data.

Still using dialogue generation as an example: a large model trained on massive data masters the pattern for 60-turn dialogue and next-sentence expression.

Why is so much data needed? Why can humans imitate a TV drama's male lead after watching just one series?

Because humans don't use previous dialogue turns as input to master the pattern for what to say next. Rather, in subjective dialogue they form an understanding of context: the speaker's personality traits, current emotions, motivations, associated knowledge, plus previous dialogue turns, to master the pattern for what to say next.

We can imagine that if a large model first identifies contextual elements in dialogue, then uses these to generate the pattern for the next sentence, it could greatly reduce data requirements compared to using raw dialogue. Thus how well self-supervision is done is an important factor in a large model's "model efficiency."

Whether a large model service performed self-labeling of certain contextual information during training can be examined by testing whether its dialogue generation shows sensitivity to such contextual information (whether generated dialogue reflects consideration of this contextual information).

Human-written desired outputs are the second contributing factor.

ChatGPT uses human-written outputs for several task types to fine-tune the GPT-3.5 model, which has already learned general dialogue generation patterns.

This is the spirit of pre-trained models — the patterns of dialogue in a closed scenario may be 99% or more general patterns of human dialogue generation, with scenario-specific patterns accounting for less than 1%. Thus a model trained on general human dialogue patterns plus additional training of a small model for the closed scenario can achieve the effect, with very small samples needed to train scenario-specific patterns.

The next contributing mechanism is that ChatGPT incorporates reinforcement learning. The entire process works roughly as follows:

Preparation: A pre-trained model (GPT-3.5), a group of trained labelers, and a series of prompts (instructions or questions, collected from extensive user interactions and labeler design).

Step 1: Randomly sample a large number of prompts. Data personnel (labelers) provide standardized responses based on the prompts. Labelers can input prompts into GPT-3.5 and reference the model's output to assist in providing standardized answers.

This collects data in the form <prompt, answer>, with large amounts of data forming a dataset.

The GPT-3.5 model is fine-tuned through supervised learning on this dataset. The resulting model is temporarily called GPT-3.X.

Step 2: Randomly sample some prompts (mostly sampled in Step 1). For each prompt, generate K answers through GPT-3.X (K >= 2).

Labelers rank the K answers. Large amounts of ranked comparison data form a dataset, from which a scoring model can be trained.

Step 3: Use the reinforcement learning strategy PPO to iteratively update GPT-3.X and the scoring model, ultimately obtaining a policy model. Initialize the policy model's parameters from GPT-3.X, sample some prompts not sampled in Steps 1 and 2, generate output through the policy model, and score the output with the scoring model.

Update the policy model's parameters based on the policy gradient generated from the scores to obtain a stronger policy model.

Have the stronger policy model participate in Step 2, obtain new datasets through labeler ranking and annotation, and update to a more reasonable scoring model.

The updated scoring model participates in Step 3, yielding an updated policy model. Iterating Steps 2 and 3, the final policy model is ChatGPT.

If the above is unfamiliar, here's an accessible analogy: It's like having ChatGPT learn martial arts. Human responses are the master's forms; GPT-3.5 is a martial arts enthusiast's forms; the scoring neural network is an evaluator telling ChatGPT who performed better in each match.

Thus ChatGPT can, in its first observation of the comparison between the human master and GPT-3.5, improve slightly from GPT-3.5 toward the human master. Then, with this evolved ChatGPT participating as the enthusiast in comparison with the human master, the scoring neural network again tells it where the gap lies, allowing further improvement.

How does this differ from traditional neural networks?

Traditional neural networks directly have a neural network imitate the human master. This new mode has the neural network grasp the difference between a decent enthusiast and the master, allowing it to make subtle adjustments from its existing foundation toward the human master, constantly refining.

From the principles above, we can see that such generated large models take human-labeled samples as their performance ceiling.

That is, they maximally master human-labeled samples' response patterns but lack the ability to create new response patterns. Second, as a statistical algorithm, sample quality affects model output accuracy — a fatal flaw when ChatGPT is applied to search and consultation scenarios.

Needs like health consultation are rigorous and unsuitable for independent completion by such models.

ChatGPT's demonstrated code ability and code evaluation ability come from massive amounts of code, code description annotations, and modification records on GitHub — still within the reach of statistical algorithms.

A valuable signal from ChatGPT is that we can indeed use approaches like "human highlighting" and "reinforcement learning" to improve "model efficiency."

"Largeness" is no longer the only metric tied to model capability — for example, InstructGPT with 1.3 billion parameters outperforms GPT-3 with 175 billion parameters.

Nevertheless, because training's computational resource consumption is only one barrier to large models, with high-quality large-scale data being another, we believe the early commercial landscape will still be: large tech companies provide large model infrastructure, smaller companies build super-applications on top of this, and smaller companies that become giants then train their own large models.

The Integration of Symbolic and Neural

We believe the potential of symbolic-neural integration lies in two points: training "flesh" on "bones," and using "flesh" on "bones."

If underlying patterns in surface samples contain strongly logical threads (bones) — as in the dialogue training example earlier, where contextual elements are the bones — then purely training from surface samples to obtain patterns containing bones is costly, manifesting in sample requirements and higher model training costs, that is, the "largeness" of large models.

If we use a symbolic system to generate context as neural network sample input, this is equivalent to finding patterns against a background of strong logical recognition — training "flesh" on "bones."

If a large model is trained this way, its output will be sensitive to strong logical conditions.

For example, in dialogue generation tasks, if we add to the input both parties' current emotions, motivations, associated knowledge, and related events, the large model's generated dialogue can with some probability reflect responses to these contextual information. This is using "flesh" on strong logical "bones."

Previously in developing companion-level AI, we encountered the problem that symbolic systems couldn't create smooth dialogue. If users aren't willing to talk to the AI, all the logic and emotional capabilities behind it have no chance to manifest, nor conditions to optimize and iterate. We solved dialogue smoothness through similar integration with pre-trained models as described above.

From the large model perspective, pure large model-created AI lacks wholeness and three-dimensionality.

"Wholeness" mainly manifests in whether dialogue generation considers context-related long-term memory.

For example, in a previous day's chat, the AI and user discussed the user having a cold, visiting the hospital, various symptoms, how long it lasted. The next day the user suddenly says "my throat really hurts."

In a pure large model, the AI responds based on context content, expressing "why does your throat hurt," "did you go to the hospital" — expressions that immediately contradict long-term memory, demonstrating long-term memory inconsistency.

Through integration with symbolic systems, the AI can associate from "user's throat still hurts on day two" to "user had a cold yesterday" to "user visited the hospital," "user's other symptoms" — placing this information in context, thereby using the large model's context consistency capability to demonstrate long-term memory consistency.

"Three-dimensionality" manifests in whether the AI has obsessions.

Whether it persists in its emotions, motivations, and concepts like humans do. A pure large model-created AI might randomly remind someone to drink less at social occasions. Combined with a symbolic system, knowing from long-term memory that the user has liver problems, combined with the common knowledge that liver problems mean no drinking, it generates strong, persistent advice against drinking, follows up after social occasions on whether the user drank, and affects its emotions due to the user's lack of self-discipline, thereby affecting subsequent dialogue. This is three-dimensionality.

Are Large Models General Artificial Intelligence?

From the mechanism of pre-trained model implementation, it has not broken through the capability boundary of statistical algorithms "mastering patterns in samples." It merely leverages the carrier advantages of computers to push this capability to a very high level, even creating the illusion that it possesses certain logical and problem-solving capabilities.

Pure pre-trained models cannot possess human creativity, deep logical reasoning, or the ability to solve complex tasks.

Thus pre-trained models have certain generality due to low-cost migration to specific scenarios, but they lack the general intelligence that humans possess — "generalizing from limited underlying intelligent mechanisms to produce endlessly varied intelligent manifestations at the upper level."

Next we must discuss "emergence." In large model research, researchers find that when model parameter scale and data scale cross certain critical thresholds, some capability metrics rapidly strengthen, showing emergence effects.

In fact, any system with abstract learning capability will display "emergence."

This relates to the essence of abstract computation — "not being attached to the correctness of individual samples or hypotheses, but grounding in the statistical correctness of overall samples or hypotheses."

Thus with sufficient samples, and models capable of supporting discovery of fine-grained patterns, certain capabilities suddenly form. In symbolic-oriented mind engineering, we've seen symbolic AI in language learning also exhibit "emergence" like human children's language acquisition — after reaching a certain level of listening and reading, comprehension and speaking ability advance by leaps and bounds.

In summary, treating emergence as a phenomenon is fine. But explaining all system functional mutations with unclear mechanisms as emergence, and expecting a pure algorithm at sufficient scale to emerge with human overall intelligence — this is not rigorous scientific thinking.

General Artificial Intelligence

The concept of artificial intelligence emerged almost alongside computers. At that time it was a simple idea: transplant human intelligence into computers. This was the starting point of AI, and the earliest concept of AI referred to "general artificial intelligence."

Human intelligence pattern is general intelligence, and transplanting this intelligence pattern into computers is general artificial intelligence.

Subsequently various schools emerged attempting to reproduce human intelligence mechanisms, but none produced outstanding results. This led Rich Sutton, the distinguished DeepMind scientist and founding figure of reinforcement learning, to strongly express a view:

• The biggest lesson that can be drawn from 70 years of AI research is that: seeking short-term results, researchers tend to utilize human experience and knowledge of the field (imitating human mechanisms), but in the long run, utilizing scalable general computational methods is ultimately effective.

Today's outstanding achievements of large models corroborate his advocacy for "algorithmism," but this doesn't mean the path of "imitating the Creator in making humans" to create intelligent agents is necessarily wrong.

So why did previous schools imitating humans fail one after another? This relates to the wholeness of human intelligence's core. Simply put, human language, cognition, emotional decision-making, and learning ability form subsystems that are mutually supportive in implementing most tasks — no single subsystem can run independently.

As a highly integrated system, an upper-level manifestation comes from the cooperation of numerous underlying mechanisms; any defect in one affects the manifestation of this surface effect.

Like the human body, also a highly complex system: a healthy person and a sick person may differ subtly, but this subtle pathological difference suppresses function across dimensions.

Similarly for general artificial intelligence, the effects of the first 99 steps may be very limited. Only when we complete the final piece of the puzzle do the functions of the previous 99 steps manifest.

Previous schools each saw part of human intelligence's whole from their own perspective, and achieved certain results in imitating humans. But compared to the energy the overall system could release, this was a fraction of a fraction.

Process Intelligence and Human Civilization

Every local human intelligence has been or is being far surpassed by computers. But even when all local intelligences are surpassed by computers, we can still assert that only humans can create civilization; computers are merely tools.

Why?

Because behind civilization creation is the process of human intelligent activities — that is, human civilization comes from "process intelligence." This is a severely neglected direction currently.

"Cognitive process" is not a task; it is the organization of many tasks in a process.

For example, AI curing a patient's manifested symptoms is a "goal-solving" task.

First it must convert to causal attribution solving, a cognitive task. After finding possible causes it becomes a "specific event occurrence solving" task of judging whether possible diseases occurred. This task continues decomposing and transferring to other tasks. If knowledge is lacking during the process, it becomes a "knowledge solving" task.

It can obtain existing knowledge through inquiry, search, and reading, or through "statistical cognition." After statistical cognition discovers correlations, it can further insight into underlying causal chains for better intervention. At this step it often again turns to knowledge solving due to lacking knowledge. To verify hypotheses requires designing experiments for specific event occurrence solving...

With causal chains, it can again attempt to achieve the goal, performing causal chain intervention, converting the original goal into creating, terminating, preventing, or maintaining events in the causal chain. This returns to a type of "goal-solving" process...

From this perspective, technologies like ChatGPT are for implementing tasks; the symbolic-oriented general artificial intelligence framework organizes these local task capabilities to support human-like intelligent activity processes.

General artificial intelligence is the "person" itself — it can utilize internalized capabilities and externalized tools to complete tasks, and organize these tasks to support intelligent activity processes.

Humans have strong herd effects. A school in high-output period attracts the vast majority of researchers.

Few independently reflect on a technical path's natural capability boundaries, or independently seek more valuable research directions at the macro level.

We can imagine: if we could reproduce human overall intelligence on computers, enabling machines to support independent exploration of cognition, tool creation, and problem-solving to achieve goals, leveraging the carrier advantages of computers to amplify human overall intelligence and process intelligence as before, we could truly release AI's energy and support human civilization to new heights.

---

About the Author

Dr. Xiaoyi Qian is a symbolic AI scientist, senior engineer, Hangzhou high-level recognized talent, early explorer of the logic-biomimetic framework, and creator of the first version of the M-language symbolic system. Founder, CEO, and Chairman of Beiming Xingmou. Ph.D. in Applied Economics from Shanghai Jiao Tong University, M.S. in Financial Engineering from Claremont Graduate University's Drucker School of Business, dual B.S. in Mathematics and Finance from Zhejiang University's Chu Kochen Honors College and Yau Mathematical Sciences Center's Mathematics Talent Program. 11 years of research in general AI, leading team engineering practice for 7 years.

More Content

Originating in Silicon Valley, BlueRun Ventures was established in 2005 as a venture capital firm focused on early-stage startups.

Currently, BlueRun Ventures manages multiple USD and RMB dual-currency funds in China, with assets under management exceeding 15 billion RMB, making it one of the largest early-stage funds domestically. Investment stages concentrate on Pre-A and Series A, covering hard tech and innovative interaction, enterprise technology, new consumption, and healthcare. It has cumulatively invested in over 150 startups, including Li Auto, Waterdrop, QingCloud, Guazi.com, Qudian, Songguo Mobility, Ganji.com, Energy Monster, Yuntu Semiconductor, Machenike, Yunsheng Intelligence, Anxin Wangdun, and BioMap.

BlueRun Ventures has been ranked #1 on Zero2IPO's "China's Top 30 Early-Stage Investment Institutions" and ChinaVenture's "China's Best Early-Stage Venture Capital Institutions TOP30," and was named to Preqin's Top 10 Global Venture Capital Fund Managers with Sustained High Return Performance.

Additionally, BlueRun Ventures has consecutively received honors from Forbes China, 36Kr, Cyzone, Caixin Media, CBNweekly, Jiemian, and other media institutions, including "China's Best Early-Stage Institution of the Year," "China's Top Venture Capital Firm," "Most Entrepreneur-Friendly Early-Stage Institution of the Year," and "Most Influential Early-Stage Institution of the Year."