Code Brain | Starting With What GPT-5 Is

Recently, OpenAI CEO Sam Altman offered valuable insights on GPT-5 at the Davos forum. He pointed out numerous deficiencies in GPT-4's performance, noting that GPT-5 will show significant improvements in cognitive ability and generality compared to the current GPT-4. For example, if GPT-4 can solve 10% of human tasks, he estimates GPT-5 might handle 15% to 20%. These advances matter not just for AI's application to specific problems, but for its overall general-purpose capabilities.

Altman also discussed how access to proprietary data and making AI more relevant to actual work will be areas of major progress this year — including faster processing speeds, real-time responsiveness, improved accuracy on longer and more complex problems, and expanded capabilities. Of course, Altman emphasized that AI's most important potential lies in dramatically accelerating the pace of scientific discovery and making it increasingly automated.

Additionally, as models become more powerful and better at reasoning, less training data will be needed. He also believes compute infrastructure remains insufficiently prepared for large-scale AI.

Altman stressed that humans have poor intuition for exponential growth. In early 2023, he stated that AI would be an exponentially growing technology: flat before the inflection point, then shooting vertically upward. Now he reiterates that if future versions like GPT-5 and GPT-6 achieve order-of-magnitude leaps over their predecessors, the consequences will be profound.

Though detailed technical specifications haven't been publicly released, Altman's confidence at Davos and his vision for AI's future have raised expectations for GPT-5. "Nobody knows what happens next, and it's very important to stay humble about the future." We must approach powerful technological transformations like AI with both caution and anticipation, considering how they will shape our future.

This article is from "Information Parity" (WeChat: xinxipingquan202309)

Sam Altman participated in 4-5 interviews at Davos, and people are underestimating the information density — it may be far more important than Zuckerberg's mention of 600,000 GPUs. I've extracted Sam's comments on GPT-5, and the picture is already quite clear:

"Current GPT-4 has too many flaws. It's much worse than the version we'll have this year, and vastly worse than what we'll have next year."

"If GPT-4 currently solves 10% of human tasks, GPT-5 should be at 15% or 20%."

"The most important thing isn't the specific problems it solves, but that broad generality is increasing."

"More powerful models and better use of existing models are two multiplicative factors, but clearly more powerful models matter more."

"Access to proprietary data, making AI more relevant to actual work — we'll make big progress on these this year. People's current complaints about speed, about not being real-time, these will get better this year. More precise performance on longer, more complex problems, the ability to do more — these capabilities will improve too."

"I think the most important thing about AI is dramatically accelerating scientific discovery, making new discoveries increasingly automated. Of course this isn't a near-term thing, but once it happens, it's a very big deal."

"As models become smarter and better at reasoning, we need less training data. Nobody needs to read 2,000 biology textbooks — you just need a small amount of extremely high-quality data, and to think deeply and chew on it. The model will work harder thinking through a small set of known high-quality data."

"Compute infrastructure prepared for large-scale AI is not enough."

"GPT-4 is best viewed as a preview — its limitations are obvious. Humans are naturally bad at intuiting exponential growth. If GPT-5 improves over GPT-4 as much as GPT-4 did over GPT-3, and GPT-6 over GPT-5, what does that mean? If we stay on this trajectory, what does that mean?"

"As AI becomes more powerful, possibly discovering new scientific knowledge, even automating AI research itself, the pace of world development will exceed our imagination. I often tell people: nobody knows what happens next. Stay humble about the future. You can predict a few steps, but don't make too many predictions."

"When cognitive costs drop a thousandfold or a millionfold, and capabilities are massively enhanced, what impact will this have on the world? If everyone in the world owned a company of ten thousand extremely capable virtual AI employees, experts in every field, who never tire and keep getting smarter — what would the world look like? The timing is unpredictable, but it will stay on an exponential curve. How much time do we have to prepare?"

"I don't think smartphones will disappear, just as smartphones didn't replace PCs. But on the other hand, I don't think AI is just a simple computing device like a phone plus a bunch of software. I think it might be something of much greater significance."

My own takeaway: moderate short-term expectations, but raise long-term expectations. But a core assumption is whether the "exponential growth" paradigm holds. Where does Sam's belief in continued exponential growth come from? Because broadly speaking, human society has never seen anything that can grow exponentially forever, let alone an industry where the leading company, with 90% market share at the very beginning, is already worth $1.4 trillion. Looking back at the internet, smartphones, electrification, etc., defining what stage we're at becomes crucial. Linear extrapolation at the beginning leads to missing out; linear extrapolation at the tail end leads to losing money. This reminds me of this chart.

The essential point is that the biggest difference between AI and the internet is that the internet became practically useful once it crossed zero, while AI has zero value below 60%. Because the internet replaced things that barely existed before, or were extremely costly (transcontinental data communication before undersea cables), so it could deploy easily. But AI replaces humans, or existing software — and the cost-efficiency balance of these has already been optimized to the extreme by today's world. Therefore AI's value inflection point is fundamentally about crossing the threshold of societal intelligence costs; once crossed, AI's value does rise non-linearly.

Because there's this possibility, also the biggest difference between AI and the smartphone/internet era: the iPhone's form basically stabilized after the iPhone 4, with only incremental changes ever since, no more qualitative leaps (camera, touch interaction, various sensors — these key structures fixed). Current AI clearly hasn't reached its iPhone 4 moment yet, but the key question is: once AI crosses its iPhone 4 moment (perhaps GPT-5 or 6), qualitative change might not stop, the exponential curve might not stop — this would be the biggest difference from the internet era. It's as if, back then, phones kept undergoing qualitative leaps after iPhone 4, each generation a major version update. If history had been like this, would the internet giants' landscape be as stable as today? Wouldn't Apple, or even chip manufacturers, command a higher value share? So for AI, might "compute" value exceed "application" value for a considerable time?

Returning to the question of the societal intelligence cost inflection point — how significant could this be? The internet merely digitized the physical world and drove the marginal cost of information distribution to zero, generating trillions of dollars in annual commercial value. If AI drives societal intelligence costs to zero, how much value could that create? The world's largest commercial value is probably societal intelligence itself. Once intelligence can be replicated at zero cost in bulk, massive human capital value would partially convert to AI capital value. Elon Musk has said: "The economy is productive entities times productivity — labor population times per-capita productivity. If population can expand infinitely, where's the economic ceiling?" Second, most goods' cost structures can ultimately be decomposed into labor costs (mental and physical). When these are dramatically reduced, referencing the "Model T," previously expensive goods and services will rapidly democratize — bespoke personal financial and legal advisors, customized software for everyone, personalized content and entertainment formats, spawning numerous new industries. Third, as Bridgewater's research notes, cost reduction creates surplus social wealth, increasing disposable consumption potential and enabling new consumption categories. Finally, letting imagination fly: if one day we achieve AGI or even superhuman intelligence, "high-level thinking" or "genius" is no longer scarce — brains like Musk, Steve Jobs, Geoffrey Hinton/Ilya Sutskever, Jeff Bezos could be mass-produced (mass-producing Albert Einstein...), which is what Sam meant at Davos by "everyone can have 10,000 brilliant brains serving them." What social and commercial forms would this create?

What I want to express is: from the perspective of motivation and potential returns, the weight of AGI means any rational, capable commercial organization or nation will persist desperately, because no one would abandon the massive lottery ticket or option of "building god." In 1847, British railway investment at its peak reached 7% of GDP; in the five years after the 1996 Telecommunications Act, telecom companies invested over $500 billion in fiber optics, switches, and wireless networks (over $1 trillion in today's value). Why? Faced with a productivity revolution full of unknowns and infinite possibilities, an entire generation of humanity collectively FOMOs. And currently, NVIDIA's revenue / global GDP is what, 0.1%? (Not suggesting NVIDIA revenue can be linearly extrapolated, just one reference indicator for whether total AI investment is bubbly.)

But returning to today, before reaching the inflection point, the reality is: industries need to cross the passing threshold one by one:

Current GPT-4's level only reaches "solving a specific task," not yet "replacing a specific job." Because any human job comprises many "task items"; solving one task cannot sustain an entire occupation. But as shown above, human job types are layered. As AI capabilities climb step by step, it's a continuous replacement of individual tasks, eventually entire job categories. Slow AI progress means slow replacement; rapid AI progress with leaps (such as reaching AGI) means sudden acceleration. This may be the narrative for the next five years. Not seeing commercial value in the short term doesn't mean it's permanently 0 or 1 — human tasks crossing the passing threshold accumulate from quantitative to qualitative change, until AGI breaks through the quadrant one day...

So Bill Gates put it clearly: in 18 months (June 2025), we'll see substantive, widespread AI penetration across domains. UBS and Morgan Stanley both surveyed CIOs (Chief Information Officers or Technology Officers) at North American Fortune 500 companies, and results similarly show AI transformation of enterprise processes is still in POC validation phases. H2 2024 will see more prototypes validated and entering actual production workflows; large-scale production deployment may only come in 2025. Why 2025 for everyone? Because they're waiting for two things: 1) GPT-5 (or whatever it's called) release — a step up in model capabilities, solving hallucination problems, robustness and consistency issues, complex reasoning ability problems; 2) compute costs dropping to 1/10 of current levels. At present, compute costs are declining by 10x every 12-18 months; in 18 months, many cost-constrained application scenarios can finally deploy.

This leads to the key questions: 1) How capable will next-generation models be; 2) Will AI subsequently hit bottlenecks and "hit a wall"?

The magnitude of GPT-5's (or whatever name) capability improvement is largely already determined. First, it's definitely still far from AGI — Sam Altman clearly stated this at Christmas, and after reading the above you understand what AGI's weight implies, you'll feel even more awe, even hoping this thing stays science fiction and arrives later. Second, GPT-5's capability floor should at least exceed Gemini Ultra. Just looking at paper results, we can roughly infer: multimodal with added video generation capability (3D unknown), longer sequence input windows thereby significantly improving generality, noticeably improved complex reasoning compared to GPT-4, possibly beginning to possess stronger planning capabilities. To quantify further: Sam's Davos analogy "if GPT-4 completes 10% of human work, GPT-5 should be 15% or 20%." Additionally, by common sense, the compute cost to solve the same problems may drop an order of magnitude compared to GPT-4.

Will GPT-5 hit a wall afterwards? GPT-4's use of MoE sparked questions about whether OpenAI's single-model capabilities had hit a wall, but MoE's greater value lies in reducing inference costs — it's more "optimization" than "moonshot." Model bottleneck 1 is debate about the transformer architecture. There is indeed possibility for micro-innovation; in pre-"incident" interviews, Ilya mentioned existing attention mechanisms' excessive compute consumption, but also noted solutions were in sight. But you ask whether new architectures like RWKV, Mamba could replace transformer? Citing one friend's view: leading companies have voted with their feet. This is an ecosystem, resources, and talent self-reinforcing process. New architectures rising is somewhat difficult — at least according to leading figures like Ilya, Anthropic's Dario, transformer potential still has much room to mine. Moreover, from LSTM to transformer's emergence was 20+ years; even accelerated, architectural innovation rhythm is on the decade scale.

Model bottleneck 2: Is high-quality compressed world knowledge data exhausted? Following the scaling law curve, reaching an AI "capable of writing papers and doing independent research" requires five orders of magnitude more data than currently available. Where to find it... Video and other multimodal data's value lies more in grounding text-provided knowledge to the real world, but video/images themselves compress world knowledge far less efficiently than text — just as a few-hundred-KB book's knowledge converted to video might be several terabytes. What to do?

Per Sam's Davos comments, future models won't need as much data; quality matters more, training efficiency is improving (extracting more cognition from less data), and we can set more epochs for repeated "chewing." And previous speculation about Q-star, plus comments from Jim Fan, Musk and others, suggest OpenAI has likely already achieved effective use of synthetic data. Even Anthropic's Dario mentioned in a podcast: "Data is probably not the constraining factor; for various reasons I shouldn't elaborate, but there are many data sources in the world, and many methods to generate data." This synthetic data bootstrapping can be analogized to human evolution: our primate ancestors, before developing language, couldn't summarize, refine, apply, or accumulate cognition and experience. But once humans developed language, genetic/cultural co-evolution emerged — very similar to LLMs' synthetic data/self-play loop. Additionally, a lighter analogy: we've read thousands of books, traveled thousands of miles, seen thousands of worlds — like the film The Man from Earth, someone who's lived a million years, practically a walking world knowledge base. When he learns something new, does he need so much input? We say someone with high "aptitude" gets it with a hint — why? Because past high-quality training established underlying "correlations" of how the world operates. This may be what current model training is doing (no wonder OpenAI internally says they're building god).

Therefore, we might as well be optimistic about scaling law's continuation. This time we may really be in the early stage of an exponential curve. Like Moore's Law, this is an empirical regularity — does it necessarily need rigorous theoretical explanation? Maybe not. It wasn't until a century after the steam engine's invention that humans fully understood thermodynamics. Technology's history often features invention preceding theory; perhaps this time with AI is similar. Just as, no physical law dictates Moore's Law must continue; bottlenecks always emerge prompting cries that Moore's Law is dead. But great companies and leading figures at TSMC, Intel, AMD, Apple — driven by industry, commerce, even humanity's deepest motivations — kept this empirical law going for decades.

So returning to this year's GPT-5, expecting its specific capabilities may not matter that much. What matters more is continuously observing whether we stay on this trajectory exponential curve.

Take mobile agents, for example. If 2024 is the prototype, or the "year one" of mobile agents, don't set expectations too high. But as frontier model capabilities climb, agents will gradually take over personal life, work, and application tasks, reflected in agents' subscription ASP rising year by year. It's neither like hardware terminal bundled pricing, nor membership fees, nor quite like SaaS subscriptions — more like hiring a butler whose capabilities grow from intern to CEO, who can do increasingly complex things for you, from assisting your work to creating incremental value for you, who knows you better and better, with increasing stickiness, and whose salary you keep raising (of course this is just from the demand side; pricing ceiling depends on supply, scarcity, etc.).

Citing an overseas blogger article Sicong sent me a few days ago — this blogger compared internet and AI paid user stages, perhaps more persuasively. Internet paid user penetration looked like this:

The blogger also roughly estimated AI user paid penetration:

So from paid user penetration perspective (closer to real commercial value), AI today is merely equivalent to the internet in 1996:

This author also calculated current AI CAPEX ROI, roughly for reference:

And AI layer profit margins and investment returns — compute & network has the highest value content and profit margins:

This is interesting. Combined with earlier comparison of AI versus smartphone/internet eras, I personally believe that as this exponential curve rises, underlying compute will remain in dramatic flux — meaning the "iPhone 4" moment, even when it arrives, is just a starting point, not an endpoint; qualitative change will continue advancing. If underlying compute paradigms are in dramatic flux, innovation, unable to stabilize, how can upper-layer applications build stable ecosystems? Therefore, I believe compute & networking's share of total AI value will remain elevated for a considerable time — starkly different from the internet era. Of course, the story's ending is always "whoever's closest to the C-end captures maximum value," but the time to reach that ending stable state can be completely different. From this perspective, how should we view NVIDIA?

This also reminds me of a recent small discussion topic: who can eat NVIDIA from fish head to fish tail — hedge funds, long-only, proprietary capital, or industrial capital? More specifically, semiconductor analysts, software analysts, internet analysts, or early-stage VC investors? (Open question, no answer.)

High-speed railway trains are impractical because passengers would suffocate and die, unable to breathe at such speeds. — Dr. Dionysus Lardner (1793-1859), Professor of Natural Philosophy and Astronomy, London Messing with alternating current is a waste of time. People will never use it. — Thomas Edison, 1889 "Horses aren't going obsolete, and automobiles are just a passing novelty." — President of Michigan Savings Bank, 1906 The world will probably need... about five computers. — IBM, 1943 After the first six months, television won't have any market. People will quickly tire of staring at a plywood box every evening. — Darryl Zanuck, 20th Century Fox executive, 1946 There is no reason anyone would want a computer in their home. — Ken Olsen, President of Digital Equipment Corporation, 1977 Mobile phones won't replace landlines. — Marty Cooper, 1981 I predict the Internet will soon go spectacular supernova and in 1996 catastrophically collapse. — Robert Metcalfe, 1995 No 3G support, expensive, and it can't even pass basic drop tests — unlikely to pose a threat to Nokia. — Nokia engineer's assessment of first-generation iPhone, 2007

(End)