Code Brain | Top 10 AI Predictions for 2024
The year 2023 opened with ChatGPT, released by OpenAI, kicking off a full year of AI frenzy. Now, no one doubts that a new era is coming — but it may not unfold in the ways we imagined. With 2024 here, what developments lie ahead for this rapidly evolving industry? Feel free to share your thoughts in the comments.
Before that, let's look at Forbes' top 10 AI predictions for 2024:
【1】Nvidia will make aggressive moves to become a cloud services provider
【2】Stability AI will go under
【3】The terms "large language model" and "LLM" will become less common
【4】State-of-the-art closed models will continue to meaningfully outperform state-of-the-art open models
【5】Some Fortune 500 companies will create a new C-suite role: Chief AI Officer
【6】An alternative architecture to the transformer will gain meaningful adoption
【7】Cloud providers' strategic investments in AI startups, and the associated accounting implications, will face regulatory challenge
【8】The Microsoft/OpenAI relationship will begin to fray
【9】Some of the hype and herd mentality that shifted from crypto to AI in 2023 will shift back to crypto in 2024
【10】At least one U.S. court will rule that generative AI models trained on the internet constitute copyright infringement, and the issue will begin to climb toward the U.S. Supreme Court
01
Nvidia Will Attempt to Launch a GPU Cloud Service
Most organizations don't buy GPUs directly from Nvidia. Instead, they access them through cloud providers like Amazon Web Services, Microsoft Azure, and Google Cloud Platform, which purchase chips from Nvidia in bulk. But Amazon, Microsoft, and Google — Nvidia's largest customers — are now becoming its competitors. Given how much value in AI currently resides in silicon (Nvidia's stock price is clear evidence), these major cloud providers are all actively developing their own AI chips, which will directly compete with Nvidia's GPUs.
Cloud providers are trying to capture more value by moving deeper into the technology stack, specifically into silicon. Meanwhile, Nvidia is moving in the opposite direction — offering its own cloud services and operating data centers — aiming to reduce its dependence on traditional cloud companies as distribution channels.
Nvidia has already begun experimenting with this strategy, launching a new cloud service called DGX Cloud earlier this year. We predict Nvidia will significantly ramp up this strategy next year.
This could include Nvidia building its own data centers (currently DGX Cloud still relies on other cloud providers' physical infrastructure), and might even involve acquiring emerging cloud providers like CoreWeave. CoreWeave already has close ties with Nvidia, and such an acquisition would represent Nvidia's attempt at vertical integration. In short, as we enter 2024, expect the relationship between Nvidia and major cloud providers to become more complicated.
02
Stability AI Is on the Brink of Collapse
In AI circles, it's an open secret: Stability AI, once a celebrated startup, spent most of 2023 like a slow-motion train wreck. Stability has been bleeding talent. Recent high-profile departures include the company's COO, chief people officer, VP of engineering, VP of product, VP of applied machine learning, VP of communications, head of research, head of audio, and general counsel.
Last year, Coatue and Lightspeed led a $100 million funding round for Stability. But in recent months, reportedly due to disagreements with Stability CEO Emad Mostaque, both firms have stepped off the company's board. Earlier this year, the company attempted to raise additional funding at a $4 billion valuation but ultimately failed.
We predict this troubled company will finally crumble and shut down next year under mounting pressure.
Facing investor pressure, Stability has begun searching for potential acquirers, but so far there appears to be little interest.
One recent positive development for Stability: the company raised $50 million from Intel, which will help extend its runway. For Intel, this investment is clearly driven by a desperate desire to land a marquee customer for its new AI chips as it tries to gain footing against Nvidia.
However, Stability is notorious for its high burn rate: at the time of Intel's investment in October, the company was reportedly spending $8 million per month, with revenue representing only a small fraction. At that burn rate, the $50 million investment likely won't last through the end of 2024.
03
"Large Language Model" and "LLM"
Will Gradually Fade from Use
In today's AI field, the term "large language model" (and its abbreviation LLM) is often used loosely to mean "any advanced AI model." This phenomenon arose mainly because early generative AI models like GPT-3 were text-based.
However, as AI model varieties proliferate and AI technology evolves toward multimodality, this term is becoming increasingly inaccurate and impractical. In 2023, the rise of multimodal AI became a defining theme. Many leading generative AI models now encompass text, images, 3D, audio, video, music, and even physical actions — far more than traditional language models.
For example, some AI models are specifically trained on amino acid sequences and molecular structures of proteins to create novel protein-based drugs. While architecturally descended from GPT-3, simply categorizing such models as large language models seems inappropriate.
Consider also foundation models in robotics — large generative models that combine visual and language inputs with broad internet knowledge to execute actions in the physical world, such as through robotic arms. For these models, "language model" is clearly too narrow a label. Researchers have already begun using new terms like "vision-language-action" (VLA) models.
DeepMind's FunSearch model is another example: though its authors call it an LLM, it deals more with mathematical problems than natural language processing.
By 2024, as our models become increasingly multidimensional, the terminology used to describe them will grow richer and more diverse as well.
04
State-of-the-Art Closed Proprietary Models
Will Continue to Significantly Outpace State-of-the-Art Open Models
In current AI discourse, the debate between open-source and closed-source models has become a major topic. While cutting-edge AI model developers like OpenAI, Google DeepMind, Anthropic, and Cohere keep proprietary rights to their most advanced models, a few companies like Meta and emerging startup Mistral have chosen to publicly release their latest model weights.
Currently, high-performance foundation models like OpenAI's GPT-4 remain closed-source. However, many open-source advocates believe the performance gap between closed and open models is gradually narrowing, and some even predict open models will soon surpass closed models in performance — possibly as early as next year. (A recent chart generated significant attention.)
Caption: We can see that the rise of open-source local models is catching up to cloud-based large-scale (expensive) closed models
However, we hold a different view. We predict that through 2024 and beyond, the most advanced closed models will continue to significantly outperform the best open models.
In the rapidly evolving field of foundation model performance, Mistral recently claimed it would open-source a GPT-4-class model sometime in 2024 — a statement that generated tremendous enthusiasm in the open-source community. But note: OpenAI already released GPT-4 in early 2023. By the time Mistral launches this new model, it will likely be more than a year behind the industry curve. By then, OpenAI may have already released GPT-4.5 or even GPT-5, ushering in an entirely new performance era. (Rumors suggest GPT-4.5 might launch before the end of 2023.)
In many fields, following a pioneer and quickly reaching the frontier is far easier than being the first to break new ground. For instance, OpenAI's decision to develop GPT-4 using a mixture-of-experts architecture — before its viability was proven — was undoubtedly riskier, more challenging, and more costly. Mistral's follow-up months later, developing its own model with a similar architecture, was considerably simpler.
In 2024, it won't be easy for open models to surpass closed models in performance. The investment required to push the state of the art with new models is enormous, and costs will only increase as model capabilities advance. OpenAI's estimated spending to develop GPT-5 is around $2 billion.
As a public company, Meta must answer to shareholders. The company doesn't appear to expect direct profits from its open-source models. Meta reportedly spent about $20 million building Llama 2; even without direct revenue increases, such investment makes sense given strategic advantages. But would Meta really invest nearly $2 billion to develop an industry-leading AI model, then open-source it for free with no clear return on investment?
This is also a dilemma for emerging companies like Mistral. There is no clear profit model for open-source foundation models, as Stability AI's experience demonstrates. For example, providing hosting services for open-source models ultimately devolves into fierce price competition, as we've recently seen with Mistral's Mixtral model release. Even if Mistral had billions of dollars to develop a model surpassing OpenAI's, would it really choose to share that achievement for free?
We have a hunch that as companies like Mistral invest more capital in developing more powerful AI models, they may gradually abandon their open-source stance and instead charge for their most advanced models as proprietary products.
(To be clear: this is not an argument against the merits of open-source AI. Nor is it saying open-source AI won't matter in the future of artificial intelligence. On the contrary, we believe open-source models will play a key role in AI adoption in the coming years. However, we foresee that the most advanced systems pushing the boundaries of AI will remain proprietary.)
05
Many Fortune 500 Companies
Will Create New Chief AI Officer Positions
This year, artificial intelligence rapidly became a top priority for Fortune 500 companies, with boards and management teams across industries scrambling to figure out what this powerful new technology means for their businesses. We predict that one strategy large enterprises will widely adopt next year is appointing a "Chief AI Officer" to lead their AI initiatives.
A decade ago, we saw a similar trend with the rise of cloud computing, as many companies began hiring "Chief Cloud Officers" to help them understand the strategic implications of cloud technology.
Given a parallel trend underway in government, this movement will gain additional momentum in the corporate world. President Biden's recent AI executive order requires U.S. federal agencies to appoint chief AI officers, meaning the U.S. government will add more than 400 chief AI officers in the coming months.
Creating chief AI officer positions will become a popular way for companies to outwardly signal their seriousness about AI. But whether these roles will prove valuable in the long term is another question worth exploring. (How many chief cloud officers still exist today?)
06
Emerging Technologies as Alternatives to Transformer Architecture
Will Gain Recognition and Growing Importance
The Transformer architecture, originally introduced by Google in a landmark 2017 paper, is now the core paradigm of today's AI technology. From ChatGPT and Midjourney to GitHub Copilot, every mainstream generative AI model and product is built on the Transformer architecture. But technological dominance doesn't last forever!
On the fringes of AI research, some teams are working to develop entirely new, next-generation AI architectures that surpass existing Transformers in certain respects.
Among these, Stanford's Chris Ré lab is a major center of this research area. Ré and his students work on developing novel model architectures whose computational requirements grow sub-quadratically with sequence length — compared to the quadratic growth of current Transformers. Such models would (1) require less computation, and (2) perform better on long sequences. Notable sub-quadratic model architectures from Ré's lab in recent years include S4, Monarch Mixer, and Hyena.
The latest and perhaps most promising sub-quadratic architecture is Mamba. Just last month, two of Ré's students released the Mamba research, generating tremendous excitement in the AI research community, with some commentators even suggesting it signals "the end of the Transformer era."
In the search for Transformer alternatives, MIT has developed liquid neural networks, and Sakana AI, a new company led by one of the co-inventors of the Transformer, was also mentioned in a recent report.
We predict that next year, one or more of these new architectures will achieve breakthroughs and begin seeing widespread adoption, transitioning from mere curiosities of novel research to reliable AI alternatives in production.
We don't believe Transformers will exit the stage in 2024. They are foundational technology, and the world's most important AI systems are built upon them. However, we predict that in 2024, advanced Transformer alternatives will be integrated into real-world AI applications.
07
Regulators Will Closely Scrutinize
Cloud Providers' Strategic Investments in AI Startups
and the Accounting Issues They Create
This year, investment capital flooded from big tech companies to AI startups. In January, Microsoft invested $10 billion in OpenAI, then in June led a $1.3 billion round in Inflection. That fall, Amazon announced it would invest up to $4 billion in Anthropic. Shortly after, Alphabet announced plans to invest up to $2 billion in the same company. Additionally, Nvidia — the world's most active AI investor this year — made investments in numerous AI startups using its GPUs, including Cohere, Inflection, Hugging Face, Mistral, CoreWeave, Inceptive, AI21 Labs, and Imbue.
It's obvious that part of the motivation for these investments is to lock these fast-growing AI startups in as long-term compute customers.
Such investments touch on a gray area in accounting rules. While this may sound like an esoteric topic, it will have major implications for the future competitive landscape in AI.
For example, suppose a cloud provider invests $100 million in an AI startup on the condition that the startup uses that money to buy its cloud services. In essence, this isn't truly independent revenue for the cloud provider; rather, the vendor is converting cash on its balance sheet into revenue through the investment.
This transaction pattern, known as "round-tripping" (money invested that quickly flows back), attracted considerable attention among Silicon Valley leaders this year, with venture capitalist Bill Gurley weighing in.
As the industry saying goes, "the devil is in the details." Not all mentioned transactions are truly "round-tripping." For instance, whether investments come with explicit requirements for startups to use the funds to purchase the investor's products, or merely facilitate broad strategic partnerships, are critical factors. The contracts between Microsoft and OpenAI, and Amazon and Anthropic, are not public, so we cannot know their exact structure.
But in at least some cases, cloud providers may be recognizing revenue through these investments that should not be recognized.
So far, these transactions have faced virtually no regulatory scrutiny. But that will change in 2024. Expect the U.S. Securities and Exchange Commission (SEC) to more closely examine "round-tripping" in AI investments next year, potentially leading to a significant decrease in the number and scale of such deals.
Given that cloud providers have been a primary source of capital driving the generative AI boom, this could have major implications for the overall AI fundraising environment in 2024.
08
The Microsoft/OpenAI Relationship Will Face Challenges
Microsoft and OpenAI are closely partnered. To date, Microsoft has invested over $10 billion in OpenAI. OpenAI's models power key Microsoft products such as Bing, GitHub Copilot, and Office 365 Copilot. Last month, when OpenAI CEO Sam Altman was unexpectedly fired by the board, Microsoft CEO Satya Nadella played an important role in his reinstatement.
Nevertheless, Microsoft and OpenAI are two separate organizations with different goals and different visions for AI's future. While this partnership has been mutually beneficial so far, it is more a strategic alignment. The two organizations are not fully aligned in many respects.
We predict that problems may emerge in this partnership between the two tech giants next year. In fact, signs of potential future friction are already beginning to show.
As OpenAI aggressively expands its enterprise business, it will increasingly compete directly with Microsoft for customers. For its part, Microsoft has multiple reasons to seek advanced AI model providers beyond OpenAI. For example, Microsoft recently announced a partnership with Cohere, an OpenAI competitor. Faced with the high costs of operating OpenAI models, Microsoft is also internally investing in AI research on small language models like Phi-2.
Looking further ahead, as AI technology grows more powerful, issues around AI safety, risk, regulation, and public accountability will become hot topics. The stakes are high. Given the two companies' different cultures, values, and histories, they are likely to diverge in their philosophies and approaches to these issues.
As the world's second-largest company, Microsoft carries a market cap of $2.7 trillion. Yet OpenAI, under Sam Altman's leadership, is ambitious and far-reaching in its goals. Currently, the two organizations coexist to mutual benefit. But this state of affairs is unlikely to last forever.
09
The Hype and Herd Mentality That Shifted
from Crypto to AI in 2023
Will Flow Back to Crypto in 2024
Currently, it's hard to imagine venture capitalists and tech leaders being more excited about anything other than artificial intelligence. But a year is a long time, and VCs' enthusiasm and direction can shift quickly.
The cryptocurrency industry is a cyclical domain. While its current lull may seem subdued, we shouldn't overlook that another major bull cycle is inevitable, as we've seen in 2013, 2017, and 2021. Notably, Bitcoin's price has risen sharply in recent months from under $17,000 at the start of the year, climbing from $25,000 in September to over $40,000 today. We may be in the early stages of a significant Bitcoin uptrend, and if this materializes, it will drive substantial cryptocurrency activity and hype.
Many prominent venture capitalists, entrepreneurs, and technologists now fully committed to AI were deeply involved in cryptocurrency during the 2021-2022 crypto bull market. If crypto asset prices rally strongly next year, we can expect some of them to return to the cryptocurrency space, just as they pivoted to AI this year.
(Frankly, seeing excessive AI hype shift to other areas next year would be a welcome change.)
10
At Least One U.S. Court Will Rule That Generative AI Models
Trained on the Internet Infringe Copyright,
and the Issue Will Gradually Escalate to the U.S. Supreme Court
A major and underappreciated legal risk facing generative AI today: leading generative AI models are trained on vast amounts of copyrighted content. This fact could trigger enormous liability and even transform the industry's economic model.
Whether it's poems composed by GPT-4 or Claude 2, images created by DALL-E 3 or Midjourney, or videos produced by Pika or Runway — these generative AI models can produce astonishingly sophisticated works because they are trained on much of the world's digital data. Typically, AI companies obtain this data from the internet at no cost and use it freely for model development.
But do the original creators of this intellectual property — authors of books, poets, photographers, painters, video makers — have any say in how AI practitioners use their works? Do they have any right to share in the value created by AI models?
The answers to these questions will depend on how courts interpret a key legal concept: "fair use." Fair use is a long-standing legal principle, but its application to the emerging field of generative AI raises complex theoretical questions with no clear answers.
"People in machine learning may not know the details of fair use well, and courts have ruled that some high-profile real-world cases don't qualify for fair use — cases very similar to AI-generated works," notes Stanford researcher Peter Henderson. "Currently, litigation outcomes in this area are highly uncertain."
How will these issues be resolved? The answer lies in case-by-case adjudication and court rulings.
Applying fair use principles to generative AI is a complex task requiring innovative thinking and subjective judgment. Both sides have reasonable arguments and points.
So don't be surprised if next year at least one U.S. court rules that generative AI models like GPT-4 and Midjourney do indeed violate copyright law, and that companies developing these models are liable to the owners of the intellectual property on which the models were trained.
This won't resolve the fundamental issue. In other jurisdictions, facing different case facts, other U.S. courts may well reach the exact opposite conclusion: that generative AI models are protected by fair use principles.
The issue will gradually climb toward the U.S. Supreme Court, where it will ultimately receive a definitive legal ruling. (The path to the nation's highest court is long and winding — don't expect a Supreme Court ruling on this issue next year.)
In the meantime, there will be a flood of litigation, much settlement negotiation, and lawyers worldwide will busy themselves navigating various case law complexities. Billions of dollars in capital flows will hinge on the outcomes of these lawsuits.
Original: 10 AI Predictions For 2024 Author: Rob Toews
