GPT-3

GPT-3 is a large language model developed by OpenAI, released in June 2020 as the third generation of the company's Generative Pre-trained Transformer series . It features 175 billion parameters and was trained on roughly 500 billion tokens drawn from sources including Common Crawl, WebText2, books, and Wikipedia . At the time, it was the largest publicly disclosed monolithic model and demonstrated that a single model could handle both language and logic tasks through carefully designed text prompts .

The model's significance lay in proving the scalability of the transformer architecture—OpenAI used a top-tier supercomputer system with over 285,000 CPU cores and 10,000 GPUs for training, at an estimated cost of $12 million . However, GPT-3 could generate unreliable or harmful outputs and didn't always align with user intent, which led OpenAI to develop InstructGPT and later ChatGPT through supervised fine-tuning and reinforcement learning from human feedback . In the view of 源码资本, GPT-3 marked the moment OpenAI began prioritizing how to turn a base model into a usable product, setting the stage for the interface innovations that followed .