Sam Altman on OpenAI: The Hard but Right Path | Bolt Recommends

线性资本·November 20, 2024

Opportunities and Breakthrough Strategies for Startups in the AI Wave

Y Combinator (YC) is a startup accelerator and venture capital firm founded in 2005, headquartered in Mountain View, California. It focuses on seed-stage startups, providing funding, mentorship, and a three-month entrepreneurship training program to help these companies grow and develop.

Recently, YC president Garry Tan invited Sam Altman to discuss the rapid evolution of technology, particularly the explosive growth of AI and its potential implications. As CEO of OpenAI, Altman reflected on OpenAI's journey and shared his perspectives on conviction, the future of AI, and other topics.

We've organized and translated selected portions. The original video can be accessed via the "read more" link.

Photo: YC interview video description

📝 Summary

  1. The best time for AI startups: The world currently underestimates AI's potential. For tech startups, this is an exceptional moment to scale.

  2. Key strategies for success: In rapidly evolving technology fields, extreme conviction in a single bet combined with data-driven iterative adjustment is critical for startup success.

  3. The value of peer companionship: In early-stage venture capital, having strong team partners helps cultivate conviction, spark inspiration, and provide mutual support.

  4. Startup advantages: Compared to larger, slower-moving organizations, startups have significant advantages in speed, focus, and rapid adaptation to technological advances.

  5. The importance of business fundamentals: Despite AI's enormous opportunities, core business principles — building strong products and establishing competitive advantages — remain essential for long-term success.

  6. AI's rapid development: Major breakthroughs, including AGI, may arrive sooner than most people expect.

Part.01

Building the OpenAI Team

1. Garry Tan: What was the catalyst for assembling the OpenAI team?

Sam Altman: Looking back, we stated clearly from the beginning that our goal was to pursue AGI, even though the industry widely considered this nearly impossible and somewhat irresponsible. This ambition attracted many young, talented people, but also invited ridicule. I felt this polarization was a good sign — it represented our courage to challenge conventions. We were a small team without rigid hierarchy. I was 30 and already the oldest person on the team. Many people thought we were young and clueless, talking nonsense. But it was precisely those willing to take risks and innovate who formed the core supporting our mission. We met people one by one, bounced ideas around, and gradually assembled the team over nine months.

In December 2015, we announced OpenAI's launch, but didn't actually begin until January 2016. We returned to Greg's apartment, a dozen of us sitting together, as if we'd accomplished something monumental, then immediately fell into confusion: "What do we do now?"

This reminds me of many startup founders' experiences — after raising funds, they feel they've achieved something significant, but only when the real work begins do they realize it's just the start of new challenges. At OpenAI, we spent a long time figuring out what to do next. Despite many twists and turns, our initial broad direction was always correct. We wrote these goals on a roadmap, then tried many things — some succeeded, some failed — but ultimately achieved our systematic objectives. Looking back, we took many detours, but never strayed far from our original direction.

2. Garry Tan: Did the initial roadmap mention deep learning?

Sam Altman: Yes, our goals were to build a large unsupervised learning model and solve reinforcement learning (RL). These objectives were set during an early team offsite.

I remember three goals from that session:

  1. Find methods for unsupervised learning
  2. Solve reinforcement learning (RL)
  3. Keep the team under 120 people

We didn't achieve the last one, but the first two proved directionally very correct.

Part.02

Unwavering Conviction Despite Skepticism

3. Garry Tan: Why was the idea of scaling deep learning considered heretical?

Sam Altman: Our core belief from the start was: deep learning works, and it gets better as it scales. I think both ideas were somewhat heretical. We weren't certain how much better deep learning would get with scale. But what people knew was that if you made neural networks larger, they improved. We were very confident about this.

Many argued that deep learning wasn't real learning or real reasoning — it was like a magic trick. These views came from industry leaders who not only rejected the idea but considered it dangerous, potentially triggering an "AI winter." But we saw experimental results continuously improving. Later, seeing the effects of scaling, we decided to keep pushing forward, convinced it would produce the expected results.

We began sensing that deep learning was ready to take center stage. Though we didn't fully understand all operational details (and still don't), we recognized something fundamentally transformative was happening. It felt like discovering a new element on the periodic table. We resolved to push this direction forward, despite having far fewer resources than DeepMind. We knew competitors would try many different strategies, while we chose to focus on one direction and concentrate resources — this is how we could win. This strategy works very well for startups.

We told ourselves: we don't know what we don't know. But we knew this direction worked, so we committed fully. Other teams might try to outsmart themselves with too many strategic pivots; we chose to focus on what was in front of us and keep pushing forward.

Scaling has always interested me — for startups, for deep learning models, for many things. I think scaling is an underrated property. When you're not confident, think about what produces better results when you do more of it, make it bigger — then I think you should do more, make it bigger, scale.

People always want "less is more," but we firmly believed "more is more" and committed to pushing that philosophy. One thing about OpenAI that perhaps isn't widely understood: we had an exceptionally talented research team even when we were unknown.

If you have the smartest people in the world, you can drive something forward.

Many industry veterans criticized us for wasting resources, even accusing us of potentially causing an "AI winter." They believed resources should be distributed, not concentrated on one project. Most people still don't understand the value of betting with extreme conviction on a single project. At the time, the common approach was to spread resources across multiple directions; we chose focus. This was a more optimistic approach. I think this conviction is reflected in many successful YC startups.

4. Garry Tan: Can this be summarized as: when the world gives you opposition that doesn't make sense to you, you should persist?

Sam Altman: Exactly right. This is actually something I keep seeing in the venture industry. I used to firmly believe there were always "adults in the room" who had everything figured out, who knew the truth, who knew what to do. When someone opposed you, you'd assume they knew better. But after going through YC, I gradually understood: you can try to do anything, and no one will give you the answers. There are no "adults" who can magically tell you exactly what to do. All you can do is iterate quickly and find your own path. Understanding this was a major breakthrough in my life.

However, conviction doesn't mean blind persistence. If you're wrong and don't adjust, don't seek truth — that kind of conviction isn't effective. What we tried to do was trust what the results indicated and truly focus on the task at hand. There were issues we held strong convictions about, but many times we were wrong. When we realized we were wrong, we went all-in to correct course. Conviction matters, but once you have data feedback, you must adjust based on data. Many people stick to their views even after getting data.

But it's an iterative process. There is indeed a phase where you must persist with conviction without data — at that point, conviction is all you have to push forward.

5. Garry Tan: Right, focus is a crucial factor. Often people need to make choices about what to focus on, and hopefully make the right choices, because you can't always have unlimited options. So making choices, prioritizing — these are important skills to practice, to increase your probability of success.

Sam Altman: We tried many different directions at OpenAI. While these attempts helped us accumulate scientific understanding, we didn't take shortcuts. If we'd known then what we know today, the entire development process could have been accelerated. But that's not how things work — you can't guess right every time. So we started with many hypotheses about technical directions, what the company should look like, how AGI would develop. We took hits, made mistakes, but our advantage was getting back up and continuing forward.

Which scientific path to choose, how the world works, what product form should take — we didn't fully know, or at least I didn't. Maybe Alec Radford knew. But we all started with experimentation. We didn't know what large language models could achieve today; we also researched robotics and video game agents. But a few years later, GPT-3 completely changed our trajectory. None of what happened later was visible to us at the time.

6. Garry Tan: Sounds like you gained key insights from GPT-1.

Sam Altman: Actually, even before GPT-1. I remember the paper was called The Unsupervised Sentiment Neuron, done independently by Alec. He's an extraordinary person who did this amazing work and discovered a neuron that varied based on sentiment when processing generated Amazon reviews. Others may have hypothesized and researched this too, but regardless, it was Alec — the main contributor to GPT-1 — and later others scaled it into GPT-2. Everything stemmed from this discovery, ultimately leading to the GPT series.

Part.03

GPT-4's Breakthrough and Validation

7. Garry Tan: Jake Heller of Case Text believes GPT-4 was a major inflection point. GPT-3.5, while powerful, still produced significant misinformation in legal contexts. GPT-4 reached a new level where users could break inputs into smaller workflows to get precise results. He built extensive test cases on this technology and eventually sold his company for $650 million. What's your take?

Sam Altman: I remember discussing GPT-4 with him and realizing we truly had something exceptional. When we first tried selling GPT-3 to founders, people thought it was cool, could demonstrate amazing effects, but beyond copywriting, it didn't create major business value. With GPT-3.5's release, especially as some YC startups began using it, the dynamic shifted. People started wanting to buy our product — it was no longer us pushing, but real market demand. Then with GPT-4, we began getting extensive user feedback asking "how many GPUs can you provide?" We quickly realized GPT-4 was genuinely an outstanding product.

We ran numerous tests with excellent results. The model could perform tasks we hadn't anticipated. When we experienced it ourselves — seeing it write rhyming poems, tell slightly humorous jokes, produce all sorts of unexpected effects — everyone was amazed. But only after putting the product in users' hands did we truly know if it would succeed. We were proud of our work, but ultimate validation came from users. So before real product testing, we had some anxiety. Only after getting user feedback could we confirm the final results.

Part.04

Sam's Entrepreneurial Journey

8. Garry Tan: I think you, Elon Musk, Jeff Bezos, and many others started as founders from different starting points, then gradually leveled up — whether Looped, Zip2, or other software projects. From today's perspective, what path advice would you give entrepreneurs? For example, some might want to dive into the most radical technological innovation. Should they go all-in directly, or first make some money and then deeply participate in such major innovations through investment?

Sam Altman: This is an interesting question. Initially, having the ability to provide early funding support for OpenAI was indeed very helpful. Convincing others to do this at that stage might have been difficult. Later Musk contributed more resources, which I greatly admired and appreciated, and others gradually joined in support. Beyond that, I'm glad I could support some other projects, because getting others to take on these responsibilities might be equally difficult. I learned many valuable lessons from this, which was definitely worthwhile. I do feel my work at Looped was somewhat of a waste of time. But I absolutely don't regret it, because these experiences form the puzzle pieces of life and taught me more.

9. Garry Tan: If you could go back, would you make different choices? Or what would you tell your 19-year-old self?

Sam Altman: This is a bit difficult, because AI has always been what I wanted to do. At the AI lab where I worked, there was clear consensus: absolutely do not research neural networks. I experimented too, with poor results — but that was long ago. I think if I went back, there might have been better choices than Looped. I don't actually know what that better choice would have been, but looking at outcomes, things turned out fine. So it's okay.

Throughout history, many people have dedicated themselves to improving human life through research and inventing new technologies. I often think about this, and those unknown people who made computers. Many of them have probably retired, but I'm grateful to them, because looking at history, it's many people's hard work that brought technological leaps. I encountered computers at a very young age, which changed my life. So I feel being able to add our own brick to this long path of technological progress is truly a beautiful thing.

Part.05

Reflecting on OpenAI's Dramatic Year

10. Garry Tan: It's been an extraordinary year for OpenAI, though not without drama. What did you learn from last year's personnel changes? How do you view some team members' departures? Teams do evolve and change, but how did you personally feel?

Sam Altman: Tired but good. We seem to have rapidly gone through in two years what a medium or even large tech company typically experiences over a decade. ChatGPT launched just two years ago. This rapid development comes with many grueling experiences. Any company going through expansion will replace management at some pace.

People good at the zero-to-one phase aren't necessarily good at one-to-ten or ten-to-hundred. We're also changing our goals, making many mistakes along the way, and getting some things right. All this comes with much change. I think a company's goal — whether AGI or whatever you want to call it — is to make the best possible decisions at each stage. But this does lead to much change. We're entering a relatively calmer period now, but I believe there will definitely be other moments ahead when things change again.

11. Garry Tan: So, how does OpenAI operate now? I think compared to many other companies, OpenAI's quality and speed are already quite good.

Sam Altman: This is actually the first time I feel we truly know what we need to do. From now until achieving AGI still requires enormous effort. While there are unknowns, we broadly have clear direction for action. This will be a long and difficult journey, but also incredibly exciting.

On the product side, though details remain to explore, we roughly know the goals and optimization directions. With this kind of clarity, you can move fast. As long as you're willing to focus on a few key things and strive to do them extremely well, you can advance rapidly with clear research paths, infrastructure plans, and increasingly clear product development.

For a long time, we didn't have this capability — we really were just a research lab. Even as conditions gradually improved, we struggled to walk this path resolutely because there were many other things we wanted to do. But the degree to which you can get everyone aligned and focused in the same direction is a critical determinant of your speed of execution.

📮 Further Reading

Linear Bolt Bolt is Linear Capital's dedicated investment program for early-stage, globally-oriented AI applications. It upholds Linear's investment philosophy, focusing on projects driven by technological transformation. Bolt aims to help founders find the shortest path to their goals — whether in execution speed or investment approach, Bolt's commitment is lighter, faster, and more flexible. In the first half of 2024, Bolt invested in seven AI application projects including Final Round, Xinguang, Cathoven, Xbuddy, and Midreal.