Happy first birthday to ChatGPT-3.5! What new lessons have AI and we learned over the past year? | Yunqi Tech Talk

On November 30, 2022, ChatGPT 3.5 was released. In just five days, ChatGPT hit one million registered users — an internet history record. Two months later, in January 2023, monthly active users reached 100 million.

This rocket-ship growth foreshadowed the explosive development of generative AI this year. When a16z founder Marc Andreessen declared "Software is eating the world" in 2011, he likely didn't imagine that twelve years later, the phrase would be updated to "AI is eating the world."

As ChatGPT marks its first anniversary, this edition of "Yunqi Kepu" takes a look at what AI — and we — have learned over the past year.

Reprinted from "Silicon Rabbit Race"

Authors | Eric Gu, Zuri Wang

Nov. 30, 2022

OpenAI releases ChatGPT, powered by the GPT-3.5 model.

Even before this, Jasper — an AI marketing content generator — and Stability AI — an AI image generator — had each raised new rounds at valuations of $1.5 billion and $1 billion respectively. These stood out in a sluggish primary market during the second half of 2022, igniting American investor enthusiasm for generative AI.

Dec. 2022

OpenAI's investment appeal shoots straight up. Veteran tech journalist Eric Newcomer wrote that "some well-known investors in Silicon Valley have been considering buying shares from OpenAI shareholders. Meanwhile, according to people familiar with the matter, Microsoft has been in talks with OpenAI about a direct investment."

Jan. 2023

Microsoft's continued investment in OpenAI is confirmed — an additional $10 billion, bringing its stake to 49% and OpenAI's valuation to $29 billion. More deal details emerge: if OpenAI can generate $10.5 billion in returns for Microsoft and $15 billion for other investors, OpenAI will have the right to buy back its own shares.

Feb. 2023

The Google-Microsoft rivalry begins to take shape.

Microsoft launches Bing Chat, a chatbot-powered search engine backed by GPT-4. Caught off guard, Google hastily announces it will soon launch Bard. But Bard makes an error during its demo, sending the stock down 9% and wiping out $100 billion in market cap. For a long time afterward, debate raged over whether Google Search could maintain its position in the generative AI era.

Meanwhile, Google reaches out to Anthropic — a large language model company founded by former Google and OpenAI researchers. Google not only invests $400 million exclusively but also forms a partnership, implicitly challenging the Microsoft-OpenAI alliance.

Mar. 2023

Tech giants and startups rush in.

Bloomberg releases BloombergGPT, a 50-billion-parameter model. At NVIDIA's product launch, Jensen Huang declares that "the iPhone moment of AI has arrived." Bill Gates publishes a blog post calling for recognition that "the Age of AI has begun," describing it as "revolutionary and disruptive — as significant as the advent of smartphones and the internet."

Adept AI — an AI agent company founded by authors of the foundational generative AI paper "Attention Is All You Need" — and AI social company Character AI each close new funding rounds, achieving unicorn status. Elon Musk quietly establishes xAI, a large model company.

AI's demonstrated capabilities spark panic.

Led by Musk, more than 1,000 Silicon Valley entrepreneurs and scientists sign a joint letter warning "Danger! Immediately pause all giant AI research!"

Apr. 2023

The foundation model race enters white-hot territory.

To counter Microsoft's aggressive moves, Google merges Google Brain and DeepMind into Google DeepMind, presenting a united front. Amazon joins the large model wave, launching Bedrock — a managed service for third-party models — along with its own large language model, Titan FM.

The "chip shortage" problem gradually emerges.

Musk stockpiles roughly 10,000 GPUs. CoreWeave, which started in blockchain mining, pivots to providing AI compute resources and successfully closes a new funding round at a $2 billion valuation.

Foundation models lower the barrier for application development, and generative AI apps begin to explode.

Among 268 projects in Y Combinator's winter batch, 50% are AI-related. AutoGPT, the new AI sensation, takes GitHub by storm with nearly 80,000 stars — surpassing PyTorch (65,000). The AI Agent direction that AutoGPT represents draws widespread attention.

Also, because it's so sought-after by investors, Rewind AI founder Dan Siroker — whose personal history search engine/memory assistant and its straight-line ARR growth (reaching $700K in just a few months) has investors going wild — records a video and opens up fundraising. With 170 term sheets to choose from, NEA ultimately wins the ticket, leading a new round at a $350 million valuation.

Undercurrents swirl beneath AI's surface: privacy, data security… numerous problems persist. Italy's privacy regulator temporarily bans ChatGPT, citing improper methods of collecting and storing information.

May 2023

Google and Microsoft developer conferences both land this month, thick with rivalry.

At Google I/O, AI is the absolute center. PaLM 2 language model, Vertex AI platform, one-click photo editing Magic Editor, office assistant Duet AI, and an upgraded Bard all debut. At Microsoft Build, Windows Copilot is released, embedding AI as infrastructure into Windows' core system.

The chip shortage drives NVIDIA's earnings well beyond expectations. Its stock surges to $380 after hours, surpassing its all-time high and pushing NVIDIA's market cap past $1 trillion — making it the fifth company after Apple, Google, Amazon, and Microsoft to cross that threshold. Microsoft, Google, Amazon and other cloud providers race to deploy AI chips, planning to increase supply in the second half.

Companies offering foundation models and AI services to enterprises draw attention. Adept AI's CSO & CTO (authors of the Transformer paper) depart to found Essential AI, helping enterprises better use large models — announcing a raise at a $50 million valuation. Another "Attention Is All You Need" author, Aidan Gomez's enterprise model company Cohere becomes a unicorn, backed by strategic investments from NVIDIA, Oracle, and Salesforce.

AI concerns further intensify. An AI-generated image of "an explosion near the Pentagon" goes viral, briefly sending the S&P 500 lower. Sam Altman appears at a congressional hearing to discuss AI risks and regulation with lawmakers.

Jun. 2023

Large models continue to show extraordinary fundraising power.

Chips become the oil of the AI era.

Former GitHub CEO Nat Friedman and prominent angel investor Daniel Gross purchase 2,512 H100 chips to build a server cluster worth nearly $100 million, prioritizing compute access for startups backed by their fund, AI Grant.

It's revealed that the US Commerce Department is preparing to update AI chip export controls implemented last October — NVIDIA and AMD will be affected. According to WSJ, the Biden administration is planning to restrict Chinese companies' use of US cloud computing services, potentially impacting providers including AWS and Microsoft.

The AI battle spreads to cloud data platforms. Databricks and Snowflake are at each other's throats, choosing the same week for their annual conferences — shifting from a frenemy relationship to major competitors on the AI battlefield. Against this backdrop, Databricks acquires MosaicML (ML model deployment), founded just a year and a half prior, at a sky-high $1.3 billion.

Jul. 2023

Large model companies reveal their distinct priorities.

OpenAI is assembling a new team led by chief scientist Ilya Sutskever to develop methods for steering and controlling "superintelligent" AI systems. The team has access to 20% of OpenAI's compute resources.

Anthropic launches the latest version of its AI chatbot Claude 2, making it freely available to a broader user base. Claude 2 now supports file uploads for document analysis and data organization tasks.

Musk leads a 12-person team to found xAI.

Hugging Face raises $200 million at a $4 billion valuation.

Meta continues pushing open-source large model ecosystems, releasing Llama 2 for free commercial and research use, with 7B, 13B, and 70B parameter variants. The model was trained on 2 trillion tokens and fine-tuned with 1 million human-labeled data points.

Apple, silent in the generative AI era until now, finally makes noise — reportedly building an LLM framework called "Ajax" and developing an "AppleGPT." Insiders say AppleGPT is a replica of Bard, ChatGPT, and Bing AI.

Google DeepMind pushes into embodied intelligence, releasing robot foundation model RT-2 after seven months of work. It features symbolic understanding, reasoning, and human recognition capabilities, combining PaLI-X, PaLI, and PaLM-E.

Aug. 2023

Despite ChatGPT's traffic declining for three consecutive months, new players and directions continue emerging from foundation models to applications.

"Attention Is All You Need" author Llion Jones leaves Google, teaming up with former Stability AI research head to found Sakana AI, planning to build next-generation artificial intelligence.

A serious Midjourney competitor emerges: text-to-image company Ideogram AI closes a $16.5 million round led by a16z and Index Ventures. The founding team includes authors of landmark papers such as Diffusion and Imagen.

OpenAI also seems actively searching for solutions to its traffic decline — announcing its first-ever acquisition: Global Illumination, developer of a popular online role-playing game. This move invites multiple interpretations: strengthening visual capabilities, expanding into gaming, or training AI agents.

In contrast to this uncertainty, OpenAI's push into enterprise is crystal clear. According to The Information, OpenAI's ARR exceeds $1 billion — roughly half from ChatGPT subscriptions, half from B2B customers. OpenAI announces ChatGPT Enterprise, focused on serving large corporations with generative AI services while guaranteeing data privacy and security. ChatGPT Business for SMBs will launch soon.

Sep. 2023

This month's keywords: open source, compute.

LLaMA and Falcon solidify their open-source standing. Meta announces it's developing an open-source large model to rival GPT-4, with parameters several times larger than LLaMA 2, expected to begin training in early 2024. Falcon-180B, a free-for-commercial-use open-source large model from TII, Abu Dhabi's leading global tech center, beats Llama 2 on benchmarks and approaches GPT-4 in performance — shooting straight to the top of the Hugging Face leaderboard upon release.

The compute crunch continues to cause headaches. According to an exclusive The Information report, competition for compute resources has caused rifts between Meta's internal AI teams. Of the 14 authors on the original LLaMA paper, more than half have already left. According to Reuters, OpenAI is exploring building its own AI chips and has even evaluated potential acquisition targets.

OpenAI continues leading large model development, reaching a $90 billion valuation. ChatGPT gets a multimodal upgrade with voice and image capabilities; DALL·E 3 generates images with photorealistic detail, dramatically lowering the barrier for users.

Google's large models are still playing catch-up. Google has opened an early version of Gemini to a small group of companies, aiming to compete with GPT-4. The model uses 5x the training compute of GPT-4, with significantly enhanced multimodal capabilities. Bard updates to connect with the Google ecosystem, pulling information from Gmail, Google Docs, Google Maps, YouTube and other apps. Bard also launches a model self-checking feature.

Oct. 2023

Year-end approaches; it's nearly time for everyone to hand in their report cards.

In September, ChatGPT's global iOS and Android app downloads reached 15.6 million, with total revenue near $4.6 million — 20% growth, down from the previous month.

GitHub paid users surpass 1 million.

Microsoft's cloud business grows beyond expectations. As of September, Azure cloud growth hit 29%. By contrast, Google's cloud growth fell short of expectations.

AI search unicorn Perplexity hits $3 million ARR with roughly 15,000 paid users, planning to raise at a *50x multiple, $500 million valuation.

Embodied intelligence and AI Agents make important progress.

Google DeepMind open-sources the Open X-Embodiment dataset, containing 500+ skills and 150,000 tasks demonstrated across over 1 million scenarios. It promises to become a cornerstone of embodied intelligence.

Adept.ai open-sources a small version of the model powering its product, Fuyu-8B, on HuggingFace. Compared to other multimodal models, Fuyu-8B is easier to understand, scale, and deploy — purpose-built for AI Agents.

Imbue, one of the few companies with sufficient capital to develop foundation models, aiming to create a dedicated large model for AI Agents, closes a $12 million round with participation from the Amazon Alexa Fund.

AI regulation sees important updates.

The US issues its first new executive order on artificial intelligence, requiring new safety assessments, fairness and civil rights guidance, and research on AI's impact on the labor market.

Nov. 2023

2023 is the year of "AI." "AI" is named 2023's most notable word by dictionary publisher Collins. Over the past year, usage of the term quadrupled. 2022's word of the year was "NFT."

OpenAI stages a palace coup. In the first half of the month, it holds its first developer conference — ChatGPT App Store, GPT-4.5, Assistant API… a series of major launches putting AI entrepreneurs on edge. In the second half, Sam Altman is "stabbed in the back," 90% of employees threaten to quit, and OpenAI nearly gets "zero-priced" by Microsoft. Finally, Sam Altman stages a "real return," coming back as CEO and defusing the crisis.

Musk's xAI bears fruit. xAI releases its first large model product, Grok. At 33 billion parameters and 8K context window, Grok-1 — trained over two months — surpasses ChatGPT-3.5, Inflection-1 and others on benchmarks.

Will chip supply ease? NVIDIA launches its latest H200 chip, featuring 141GB of next-generation "HBM3" memory with 2x the performance of H100. The B100 chip based on Blackwell architecture is expected in 2024. Microsoft introduces two chips: AI compute chip Maia 100 and general-purpose compute chip Cobalt 100 Arm. OpenAI has already used GPT-3.5 Turbo to refine and test Maia.