Li Mu: "Starting a company is about defying fate — what exit strategy are you even thinking about?"
One year in AI is three years on Earth.
Author: Mu Li, Co-founder of BosonAI
Original title: One AI Year, Three Human Years, first published on Zhihu.[1]
A progress report for friends on my first year of LLM entrepreneurship, with all the dilemmas and reflections.
I started thinking about starting a company around my fifth year at Amazon, but the pandemic got in the way. By year seven and a half, the itch became unbearable, so I handed in my resignation. Looking back, if there's something you want to try at least once in this lifetime, do it early. Because once you actually start, you'll discover there's so much new to learn — you'll constantly wonder why you didn't begin sooner.
The Name: Where BosonAI Comes From
Before founding the company, I worked on a series of projects named Gluon. In quantum physics, a gluon is a boson that binds quarks together — symbolizing how the project began as a joint effort between Amazon and Microsoft. The project manager came up with the name on a whim, but naming things is hard for programmers; we agonize over filenames and variable names every day. So for the new company, we simply went with Boson. We hoped people would smile at the reference to "bosons and fermions making up the world." What we didn't anticipate was how many people would read it as Boston.
"I'm in Boston, let's grab coffee?" "Huh? But I'm in the Bay Area 😅"
Fundraising: Lead Investor Bails the Day Before Signing
At the end of 2022, I had two ideas for using large language models (LLMs) as productivity tools. I happened to run into Yiming Zhang and asked his advice. After our discussion, he turned the question back on me: why not build the LLM itself? My instinctive reaction was to shrink back — our team at Amazon had worked on this for years, it would take tens of thousands of GPUs, and on and on with the difficulties. Yiming simply smiled: these are all short-term problems; you need to take the long view.
My strength is that I take advice. So I actually went and did it. I assembled a founding team with leads covering data, pre-training, post-training, and architecture, and went out to raise money. Got lucky — secured seed funding pretty quickly. But it wasn't enough to buy GPUs, so we needed a second round. The lead for this round was a very large institution; we spent months on due diligence, negotiating terms. Then the day before signing, the lead pulled out, which directly caused several co-investors to drop out as well. I'm deeply grateful to the remaining investors who stuck with us and let us close the round — we got our ticket to build LLMs.
Reflecting now, the capital markets were still hot then; we could have kept fundraising. Maybe we'd be sitting on a billion in cash like some of our peers. At the time, I worried that raising too much would make exit difficult, or put us on too high a pedestal. Looking back, entrepreneurship is about defying fate — why was I thinking about exit strategies?
Hardware: Early Adopters of the Painful Kind
With money in hand, we went to buy GPUs. Every supplier gave the same answer: H100 delivery in a year. On a whim, I emailed Jensen directly. He replied instantly saying he'd look into it. An hour later, the Supermicro CEO called. We paid a premium to cut the line, and had machines in 20 days. Proud to have been an early crab-eater.
We ate crab until we questioned our life choices, encountering all manner of bizarre bugs. GPU power supply instability, later patched by Supermicro engineers modifying BIOS code. Fiber optic cleave angles being slightly off, causing communication instability. Nvidia's recommended network topology being suboptimal — we designed our own solution, which Nvidia later adopted themselves. To this day I don't understand: we bought fewer than a thousand GPUs, a small buyer by any measure. Yet the problems we hit — didn't the big buyers encounter them too? Why did it take us to debug them?
We also rented an equivalent number of H100s. Same story — bugs galore, GPUs failing daily. We even wondered if we were the only ones on that cloud eating this particular crab. Later, reading the Llama 3 technical report, where they mentioned switching to H100s and having training runs interrupted hundreds of times — I felt deep solidarity with the pain between those lines.
Comparing build-versus-rent: renting for three years costs about the same as building. Renting saves headaches. Building has two advantages. First, if Nvidia's technology remains far ahead in three years, they can control pricing to keep GPUs holding value 😭. Second, data storage costs are lower when you build. Storage needs to be close to GPUs, and whether big clouds or small GPU clouds, storage pricing is high. But one model training run can use several TB for checkpoints, and training data storage starts at 10 PB. AWS S3 for 10 PB runs $2 million a year. That same money, self-built, gets you 100 PB.
Business: Grateful for Customers, Break-Even in Year One
Very fortunately, our revenue and expenses balanced in year one. Our spending is mainly on people and compute — thanks to OpenAI's deep pockets and Nvidia's "yaoyao" lead, both are substantial 😭. Our revenue comes from building custom models for large enterprise customers. Companies that adopted LLMs early mostly did so because their CEOs had real decisiveness — they weren't deterred by high compute and personnel costs, but pushed their internal teams to experiment with new technology. Immensely grateful to our customers for giving us breathing room; otherwise these past few months I'd be running between investors again.
Going forward, more companies should start experimenting with LLMs, whether for product upgrades or cost reduction. On one hand, technology costs are falling; on the other, industry leaders (like our customers) will release LLM-based products that raise the competitive temperature.
We're also watching LLM consumer applications. The last wave of top apps like c.ai and Perplexity are still searching for business models, but there are also a dozen or so LLM-native applications with decent revenue. We provided models to a character-roleplay startup targeting hardcore players — they balanced revenue and expenses, which is impressive. Model capabilities keep evolving, more modalities (voice, music, images, video) are converging, and I believe even more imaginative applications will emerge.
Overall, the industry and capital remain impatient. This year, several companies that raised over a billion after just a year chose to exit. From technology to product is a long process — two or three years is normal. Factor in user demand emerging, and it may take even longer. We focus on finding our way through the fog, staying optimistic about the future.
Technology: Four Stages of LLM Understanding
My understanding of LLMs has gone through four stages.
Stage one was BERT to GPT-3. The feeling was: new architecture, big data, this can work. At Amazon, we were among the first to do large-scale training and product deployment.
Stage two was when GPT-4 dropped, right as I started the company. Massively impressed. Largely because the technology went closed-source. Based on rumors, I estimated one training run at $100 million, data labeling at tens of millions. Many investors asked me what it would cost to replicate GPT-4; I said maybe $300-400 million. Later, one of them actually wrote a check for several hundred million in one go.
Stage three was the first half year of the startup. We couldn't tackle GPT-4 head-on, so we thought: start from concrete problems. We began finding customers — gaming, education, sales, finance, insurance. Training models for specific needs. At first there were no good open-source models, so we trained from scratch; later many excellent models emerged, lowering our costs. Then we designed evaluation methods for business scenarios, labeled data, identified where models fell short, and improved them targeted.
By end of 2023, we were pleasantly surprised to find our Photon series (a type of boson) beating GPT-4 on customer applications. The benefit of custom models: inference cost is 1/10 of API calls. Though APIs are much cheaper now, our own technology has advanced in parallel — still 1/10 the cost. Plus better control over QPS, latency, and so on. The insight from this stage: for specific applications, we can beat the best models on the market.
Stage four was the second half year. Though customers got the models contracted for, they weren't their ideal — because GPT-4 itself is far from enough. Early this year, we found that training for a single application, the model was hard to leapfrog again. Thinking it through: if AGI is average human level, customers want professional-level. Gaming wants professional designers and actors; education wants gold-medal teachers; sales wants top salespeople; finance and insurance want senior analysts. This is AGI plus industry expertise. Though we held deep reverence for AGI, it felt unavoidable.
Early this year we designed the Higgs (the God particle, another boson) series. The positioning: general capabilities tracking the best models, but with standout strength in one area. We chose roleplay: playing virtual characters, playing teachers, playing salespeople, playing analysts, etc. By mid-2024 we iterated to generation two[2] — on Arena-Hard and AlpacaEval 2.0 testing general capabilities, V2 trades blows with the best models; on MMLU-Pro testing knowledge, not far behind either.
Higgs-V2 is based on Llama 3 base, with complete post-training. We can't spend Meta-level money on data labeling, so V2 outperforming Llama 3 Instruct likely comes mainly from algorithmic innovation.
We also built a roleplay evaluation benchmark[3], covering in-character roleplay and scenario-based roleplay. A bit embarrassing — our own model took first on our own leaderboard. But the model never touched the evaluation data during training. We built this benchmark for internal use, wanting authentic reflection of model capabilities, so we deliberately avoided overfitting. But the evaluation team wanted to write a technical report, so we released it. Interestingly, the in-character roleplay test samples came from c.ai, yet their model ranked dead last.
The stage four insight: good vertical models can't be weak on general capabilities either — reasoning, instruction following, these are needed vertically too. Long-term, both general and vertical models must head toward AGI. Vertical models can just be slightly lopsided: ace the specialized courses, decent on the general ones, so R&D costs are somewhat lower and approaches differ somewhat.
What about stage five? Still in progress — hope to share soon.
Vision: Human Companionship
Embarrassingly, we buried ourselves in technology, built custom models for customers, and only slowly thought about what vision we ourselves pursued. We looked at what customers want, what we want, what the future might need. Personally, years ago I dreamed of a robot nanny to help raise my kids, keep them company — because I found it hard, and didn't quite understand their current cognition and thoughts. I wanted a brilliant virtual assistant at work to invent new things with me. When I'm old, I want interesting robots keeping me company too. My prediction for the future: production tools become ever more advanced, one person accomplishes what previously took a team, leading to greater individual independence, everyone busy pursuing their own things, and thus more loneliness.
Combined together, we set our vision as "intelligent agents for human companionship." An agent with high EQ and solid IQ. Translated to real people, it would be a professional team. Want it to play with you? It's a professional designer plus actor. Exercise with you? A motivator plus professional coach. Study with you? It can explain what you don't understand. The advantage of models: they can provide long-term companionship, truly know you. And can be "genuinely for you."
But current technology is far from this vision. For now, it can just chat. And not even that well in many scenarios — thin content, IQ and EQ sometimes both offline. All problems to solve now. If anyone is building overseas applications in this space, welcome to reach out.
Team: Hard Things Require Teams
Only after starting a company did I truly appreciate the importance of team. At big companies, I felt like a screw, team members were screws, even the team itself was a screw. But a startup team is a car. Small, but it runs, it carries load, turns nimbly, can go anywhere. Early in the company, Cai from miHoYo dropped by, saw everyone in one room, and sighed: small teams are wonderful.
Of course there are inconveniences — constantly checking if there's gas, being careful on rough roads not to shake the car apart. Every member matters, no redundancy; one person underperforming is like a flat tire. People are precious — losing one means losing a tire.
Before, I'd choose projects I could lead development on. But that also meant the problem wasn't very challenging. For the startup, I chose a huge problem, and had to rely entirely on the team. Don't let all the "I"s in this article fool you — the work was all done by the team.
Without the team, I'd probably have to pivot to selling courses.
Personal Pursuit: Fame or Fortune?
So far I've followed my inner voice for decisions — going back for a PhD after working, making videos, starting a company. Entrepreneurship requires strong motivation to overcome endless difficulties. This demands deeper analysis of one's motivations.
Motivation comes either from desire or from fear. Ten years ago I probably craved fame and fortune more, but at my current age, the marginal utility of money has declined, and the emotional value of reputation has shrunk too. My deeper motivation comes from fear that life may be meaningless. Setting aside the vastness of the universe, in the long river of human history, one person is just a grain of sand. Accidents come, swift disappearance. A hundred billion people have lived on Earth; the vast majority leave no trace in history. The names on my family genealogy — I barely recognize any.
So what is the meaning of a person's existence? As a child I once fell into depression over this question. So subconsciously, I want to create value, to give existence meaning. I choose "self-improvement" to enhance my ability to create value; choose making long videos and writing textbooks to create educational value; choose writing summaries of PhD, work, and entrepreneurship experiences, describing the dilemmas and difficulties, to create value as example; choose entrepreneurship to unite many people's strength to create greater value.
Postscript Last year, walking with Hua Su at Stanford University, he put his hand on my shoulder: "Tell me the truth — why do you want to start a company?" I brushed it off at the time:
"Just wanted to do something different."
Then Hua Su smiled.
Now I understand — because he's been through the full flavor of entrepreneurship. If asked again today, I'd say:
"I was just out of my mind."
But I'm also glad I didn't realize how hard it would be, so I dove right in. Otherwise, you might be reading "Reflections After Ten Years of Work." I think today's story is more interesting.
"To all entrepreneurs."
Finally, a plug for our job openings[4] (Bay Area + Vancouver), and overseas application builders please reach out: api@boson.ai
References [1]Originally published on Zhihu: https://zhuanlan.zhihu.com/p/714533901
[2]Iterated to second generation by mid-2024: https://link.zhihu.com/?target=https%3A//boson.ai/higgs-v2
[3]Roleplay evaluation benchmark: https://link.zhihu.com/?target=https%3A//boson.ai/rpbench-blog
[4]Job openings: https://jobs.lever.co/bosonai
