Inside Anthropic: Building the Most Dangerous Technology, and Trying to Be Good
"I'm afraid of this technology being in the hands of the company, and I'm afraid of it being in the hands of the government."
"I fear this technology in the hands of corporations, and I fear it in the hands of governments." — Dario Amodei
👩 Compiled by: Shirley
🧑🎨 Layout: NCon
Four years ago, Anthropic was just a ragtag group of OpenAI defectors running a tiny, unremarkable lab. Four years later, siblings Dario and Daniela Amodei have led it to wipe out hundreds of billions in software stock market value overnight, go toe-to-toe with the Pentagon, and build a model allegedly capable of breaching modern cybersecurity defenses — Mythos.
They've cast themselves as the "good guys" in this AI arms race, even as Defense Secretary Pete Hegseth calls them "ideologically paranoid." And the technology in their hands could shape the course of war itself.
When you're building the most powerful technology in the world, can "trying to be good" actually be done?
With its valuation approaching a trillion dollars and IPO widely assumed to be imminent, Bloomberg's podcast The Circuit turned its lens on the Amodei siblings, forcing a self-described "AI safety company" to confront its toughest questions.
Facing Emily Chang's unsparing interrogation, Dario's responses ranged from candid to evasive.
Below is Crossing's compilation of the episode.
What's It Like to Stand at the Center of the AI Universe?
👩💼 Emily
You're standing at the absolute center of the AI universe right now. What does that feel like?
👱♂️ Dario
Throughout my career, especially at Anthropic, what I've experienced is a smooth exponential curve: nothing seems to happen, nothing seems to change, then a few small things start bubbling up, and then — whoosh — everything goes crazy.
That's the experience the world gives you, and it's the experience this company has relative to others and to the world at large.
So I stared at that chart for a long time and thought, around this point in time, we'll become the highest-revenue, highest-valued AI company. And then it actually happened.
In some sense I'm not surprised, but when things actually happen, you see so much detail and color — that of course still surprises you.
👩💼 Emily
I know your father was a leatherworker, your mother worked in a library, and you grew up in San Francisco. How did that shape who you are?
👱♂️ Dario
The first wave of the internet revolution was unfolding all around me, and I had zero interest in it. I just loved math, loved doodling, wanted to understand how the universe works, was obsessed with science fiction — I think I was just deeply curious about the world.
As for San Francisco, that spirit of not going with the flow, that individualism, that "it's okay to be a little crazy" attitude — I think some of that rubbed off on me. In Europe or other parts of this country, thinking differently, having a bunch of crazy ideas, often gets you shut down or labeled a weirdo.
👩💼 Emily
What was it like growing up with Dario?
👩 Daniela
He was so smart — he started learning calculus around middle school, and was taking math classes at UC Berkeley in high school.
I was more into reading and art, so we were almost perfectly complementary.
Leaving OpenAI: Not Safety Disagreements, But Lost Trust
👩💼 Emily
Your decision to leave OpenAI has become Silicon Valley legend. Where did you disagree?
👱♂️ Dario
Look, I'll just say it straight. On safety issues, there are plenty of legitimate disagreements, and we certainly had some with them. But you know, that alone isn't enough to make someone leave.
What really makes something untenable is when you feel you can't trust someone, when you feel their values aren't what they say, when you discover they're not acting for the reasons they claim.
At the end of the day, if your visions differ and there's no mutual trust, why waste time arguing?
The best solution is to go your separate ways and do your own thing.
And I was totally comfortable with this thought: we'll operate our way, they'll operate theirs.
"Let the market decide, let public opinion decide" — these facts are more persuasive than any dramatic argument about "who left why."
👩💼 Emily
At the India AI Summit, you and Sam Altman seemed to refuse to hold hands on stage. What happened there?
👱♂️ Dario
The reality is, that summit was incredibly disorganized. All of us were called up at the last minute, they rearranged the positions at the last second, took a photo, and then Narendra Modi suddenly appeared on stage and ordered all of us to hold hands...
I'm not picking on India, but if you've been to these international summits with heads of state, you know they're all super chaotic.
👩💼 Emily
Sam Altman and Elon Musk are suing each other, and you don't like Sam.
If the people building the world's most important technology can't even hold hands on stage, why should we believe you'll cooperate on "existential risk"?
👱♂️ Dario
Here's what I'll tell you: the people developing this technology vary enormously in quality and trustworthiness.
The meme that "nobody trusts each other" — I don't think that's true.
Demis Hassabis, I've known him for 15 years. Gemini is Claude's competitor, but we've collaborated on many issues — buying compute from Google, exchanging ideas on safety, and so on.
In some ways, he and I are mutual motivators: he launched AlphaFold, and now we're also making strategic moves in biology and life sciences; we do interpretability research, and they've started too.
This isn't competition at all — it's simply one company doing something cool, and another saying "cool, we want to see if we can discover something new too."
So my view is, some players are more trustworthy than others.
Outside of Anthropic, I'd rather see the trustworthy participants band together to put the untrustworthy ones in a position where they're forced to adopt the same standards.
We founded this company with seven co-founders, Daniela and I being two of them. And now, at our size and scale, all the early founders are still here — you almost never see that in this space.
Teaching Claude to Be a "Good" Model
From the start, Anthropic positioned itself as the most safety-conscious AI company.
Dario has written grandly titled long essays like Machines of Loving Grace and The Adolescence of Technology, envisioning both AI's miraculous potential and worst-case scenarios.
Its chatbot Claude was trained to follow a set of principles called a "constitution," to keep it on the right path.
👩💼 Emily
Claude has a very distinctive style and temperament, plus a human-sounding name. What are you trying to convey?
👩 Daniela
I think when people interact with Claude, compared to other AI systems, they get more of what I like to call "professional warmth."
The goal isn't to become your best friend, but it's also not cold, mechanical calculation. It should feel approachable while maintaining professional distance.
👩💼 Emily
Anthropic has talked about teaching Claude to "do good." What makes a good model versus a bad model?
👱♂️ Dario
You don't want a model that lies, whether unintentionally or intentionally.
Unintentional lying, we call "hallucination," because these models are just trained to predict the next token, so sometimes when they don't know something, they just make something up entirely.
We have to ensure that "intentional deception" doesn't appear in the production models we deliver to customers.
Beyond that, there's extensive work around "harmlessness," making sure the model doesn't accidentally produce incorrect, harmful, or potentially malign-incentivizing information.
👩💼 Emily
Whose values get written into Claude? There are so many religions, so many different beliefs in the world — how do you actually decide? Does a universal "good" even exist?
👩 Daniela
Of course there's no universal standard defining what's "beneficial" or "harmless." But there are foundational documents in human history, like the Universal Declaration of Human Rights, that we can use to train Claude's character.
What's interesting is that on the religion front, we've actually started conversations with many religious leaders, exploring how to embed into Claude the core values that transcend different religions, particular worldviews, and what humanity has been exploring for thousands of years.
👩💼 Emily
Were there any traits you wanted Claude to have that just didn't "grow" in? Or any that emerged that surprised you?
👩 Daniela
Those early Claudes are pretty funny in retrospect.
Back in the Claude 2 era, it could sometimes be a bit like a nagging nanny.
You'd just ask about the weather, and it would say "I'm worried about you," and you'd think, "Claude, I just want to know the weather" — that overly "harmless" feeling.
Betting on Coding and Enterprise: "We Deliberately Didn't Do the Most Profitable Business"
Whatever Anthropic did to Claude, it seems to be working. Over the past year, company revenue has soared and it turned profitable for the first time, thanks largely to its bet on more lucrative business tools — Claude Code automates large chunks of software engineering, and Claude Cowork extends that capability to everyone.
👩💼 Emily
Early on, everyone else was focused on fun, eye-catching consumer apps, while you bet on coding and enterprise.
Was that a values decision or a business decision?
👱♂️ Dario
Look, if you choose a business model that's fundamentally in conflict with your values, you'll be miserable — either you betray your values or you become irrelevant.
We've seen that consumer world of social media, which seems to encourage "engagement," even "addiction." And AI video models churning out slop, trying to maximize the minutes of your attention because that's the ad-revenue-driven incentive.
On the enterprise side, we want these models to actually be useful to people. We want to use AI to treat diseases that couldn't be treated before — that's working with biotech, pharmaceutical, academic research teams. We want to use AI to make energy cheaper and more efficient, which is also enterprise. Education, health in the developing world — mostly enterprises or nonprofits.
So betting on a business model that's basically aligned with our values has been hugely beneficial to us.
SaaS Armageddon: "Good Luck, You Can't Defend That"
And this bet soon triggered an earthquake in the capital markets.
Shortly after Claude Cowork's launch, the software sector experienced a cleansing crash — $285 billion in market cap evaporated overnight, what traders called the "SaaSpocalypse."
👩💼 Emily
If AI keeps advancing at this pace, how much of traditional software gets replaced, and how fast?
👱♂️ Dario
These things are hard to predict in advance, but these traditional software companies often have a bunch of moats. I think some will disappear, some will remain:
"Quickly writing complex software that others can't write" — I'm pretty sure that one goes away. Good luck, you can't defend that. But customer relationships, know-how of how industries actually work, proprietary domain knowledge — those remain. With AI, the pie itself is getting bigger.
If what AI can do increases tenfold, an existing industry growing 1.5x is easy — it just doesn't grow as much as the whole pie.
It depends on how you define SaaS — the software industry overall will get bigger, existing giants' relative share may shrink, some will devalue, and some may even face collapse if they can't adapt in the right way.
So my advice to everyone is: Don't get complacent, don't ignore it. Make a list of all your moats, be clear about which ones will disappear, which will become relatively more important, and what new moats might emerge.
Companies that bury their heads in the sand, don't see what's coming, don't recognize their own moats — they're going to have a very hard time.
Will Engineers Be AI's First Casualties?
Anthropic's recent growth wave might not have happened without this person.
Boris Cherny, the engineer behind Claude Code and Cowork. Before joining Anthropic, Cherny was living a radically different life, quarantined alone in rural Japan.
🧑💻 Boris
There were lots of farmers' markets, the pace was very slow, we were making miso — that was a major hobby of mine at the time.
I remember using my first AI chatbot and being completely stunned. I thought: my god, I have to be part of this. And I'm a huge sci-fi fan, so I was very aware of how badly this could go, because this technology is incredibly powerful.
So we moved back.
👩💼 Emily
You worked on Claude Code, then led the development of Claude Cowork. What problem were you trying to solve?
🧑💻 Boris
If you look at coding products, they're all pretty simple — basically word completion, sentence completion, that's all AI could do in programming.
We wanted to make a bigger bet: we believe a Coding Agent can actually handle the whole thing.
A year and a half ago, you were still writing code by hand, occasionally hitting Tab to autocomplete a line. Now I talk to Claude and it writes the code; while one Claude is still writing, I talk to the next Claude... at any given moment, I have dozens or hundreds of Claudes running simultaneously.
👩💼 Emily
Of the code written by internal engineers, how much is Claude?
🧑💻 Boris
On my team, almost all of it. Personally speaking, for at least the past six months, it's written 100% of my code.
👩💼 Emily
What about the real tech geeks? What happens to them?
🧑💻 Boris
The real geeks... they'll have to figure it out themselves.
I think for many people, their previous skills will still be helpful going forward because they already have a pretty substantial head start. For engineers, a lot of what you need to do now isn't writing code — you have to communicate with users, plan, think about what's next. Those parts of engineering will remain.
Will AI Eliminate Half of Entry-Level White-Collar Jobs?
Silicon Valley may be running an AI fever, but sentiment elsewhere is far more subdued: 70% of Americans believe AI will eliminate jobs, and nearly one-third worry they'll be among them.
Dario Amodei has always been blunt about this, and some of his predictions don't sound particularly reassuring:
I think we may face an unusual combination: high GDP growth alongside high unemployment, or at least underemployment, proliferation of low-wage jobs, and severe inequality.
👩💼 Emily
You've been very direct on unemployment. You said "AI could eliminate half of entry-level white-collar jobs within the next one to five years." That was a year ago. Now? Is it 50% or higher?
👱♂️ Dario
I don't know the exact number, but my level of concern remains the same.
AI brings productivity gains, but that's just the familiar "hump": you automate 90% of a job, humans become ten times more efficient on the remaining 10% because the leverage is multiplied tenfold, but eventually it approaches 100%.
The sequel is you have to find something else for these people to do.
Take coding, for example. Right now AI can write virtually all the code, but it hasn't made some engineers ten times more efficient — it's just more cost-effective to have the AI do the thing directly. On the other hand, we're seeing more demand elsewhere.
We have a role called "forward deployed engineer," or "applied AI solutions architect." It's a hybrid of technical work and client communication. Because we have so many clients and we're growing fast, there's huge demand for this.
But can every pure software engineer transition into this? No, it's not one-to-one. That gives you a sense of how much disruption there could be, but things will adjust.
Which side wins out? I don't know. But the point of warning about it is so that we can respond, so that we can make policy.
Outside the software industry, AI's potential impact on employment seems even harder to predict:
Anthropic published a paper forecasting which fields would be able to leverage AI most heavily in the near future. If the prediction holds, management, finance, and legal work could soon face massive upheaval.
👩💼 Emily
What jobs will disappear, and what new ones will be created?
👱♂️ Dario
No one can be sure, because the economy itself is hard to predict.
One thing that works in AI's favor is that the pie expands dramatically. And because the pie expands so dramatically, there will probably be somewhere to put these people — the question is just whether we can find those places fast enough.
👩💼 Emily
Then walk me through it. You wake up five years from now, what does this country look like? What are those people doing? Because after all, if unemployment gets that severe, isn't that how revolutions start?
👱♂️ Dario
Yes, exactly. That's exactly the outcome we're trying to avoid.
I think there are a few places people could go, but none of them are guaranteed: One is the physical world — we need more people building and producing things in the physical world. Another is anything human-centered; relationship-driven work will matter a lot.
I also think humans will work to "steer" AI, because at some level AI has to be aligned with human values and intentions, so there's some role there — I just don't know whether that space is broad or narrow.
👩 Daniela
I'm more optimistic. I believe humans will keep finding ways to use AI to become more efficient, to do the parts that are meaningful to us and that only humans can do. Human interaction will never fully disappear.
I often use healthcare as an example: today we hire doctors who are professional diagnosticians. I think AI will soon be quite good at telling you "here's what might be wrong with you, here are the tests you should get" — you won't need a doctor for that anymore.
But AI can't physically examine you, can't ask "hey, does it hurt when I press here," can't sit with you in that clinical care way and say: "Tell me how you're feeling. How are you getting through this?"
So I think healthcare will shift toward a greater focus on bedside manner, because the diagnostic tools will become more advanced, but the human-to-human part doesn't change.
"Calling It Cheap Marketing Is What's Actually Cheap"
👩💼 Emily
You've gotten a lot of reactions. I know you say you're warning people, but Jensen Huang says you're conflating "tasks" and "jobs," that "AI is actually creating jobs, and anyone saying AI is eliminating jobs is just fearmongering"; others say, "This messaging is good for Anthropic's doomer marketing."
👱♂️ Dario
Let me be very clear, and push back on this hard.
In every interview, I talk about possible ways to address these risks: from taxes to macroeconomic policy, to the possibilities for "new jobs" described in The Adolescence of Technology. I spent a full five pages going into detail on the distinction between "tasks" and "jobs," and why this time is different from before.
I talk about the problems, and I talk about solutions. But social media — I hate it.
People pass around these three-second clips from a year ago without ever reading the article.
So the claim that "this is cheap marketing" is itself cheap marketing.
It's laziness. It's a refusal to do serious intellectual work. It's a Silicon Valley disease, trapped in that three-second, fast-food social media world. Every time someone says that, I take them less seriously.
What I'm trying to say is absolutely not "the apocalypse is coming." What I'm saying is "this is something we should anticipate, something we should be concerned about, and something we need to actively address."
From Anti-War Activist to Signing Defense Contracts
The Amodei family wants AI to have a utopian impact on society, but no one can ignore the destructive and dystopian potential of this technology.
Anthropic sits at the center of this technological arms race, with powerful government forces competing for dominance.
Dario has plunged headfirst into this arena, willing to blast the US government and willing to name partners who sell AI chips to China.
👱♂️ Dario
I think some players are "all-in," dialing the risk level too high. I'm very concerned.
👩💼 Emily
Who's doing that?
👱♂️ Dario
I'm not going to answer that.
I've been very outspoken about the importance of export controls on chips to China. The reason I say this is because letting China take the lead in AI capabilities would be detrimental to the United States and the entire democratic ecosystem.
Some chip companies clearly disagree, but that hasn't stopped me from saying it; even after we've signed more partnership agreements, I've kept saying it.
So you see, we're all adults. We can collaborate on one thing and disagree on another.
Amodei's geopolitical stance traces back to his time at Caltech.
As a sophomore physics major, he was known as an "anti-war advocate," believing that "scientists shouldn't stay in their ivory towers." It was this worldview that shaped Anthropic's later positions on war.
👩💼 Emily
You've long held anti-war views, yet you were among the first AI companies to sign contracts with the US Department of Defense and operate on its classified networks — networks being used by the US for war. Can you explain?
👱♂️ Dario
Look, the world changes. When I see Russia invade Ukraine, I worry that we're facing an authoritarian bloc that's aggressive in some ways, and that we need to defend ourselves when necessary.
Our work with the Department of Defense is absolutely not for the money. Setting aside the legal headaches, just getting systems onto government networks is a whole ordeal, and the money isn't great. We do it because we care.
And because we care, we have to set limits on how the technology is used.
Things that would undermine our own values — mass surveillance and autonomous weapons — we categorically rule out. (Autonomous weapons are weapon systems capable of independently searching for, identifying, and attacking targets without human intervention. Because they can make decisions based on their surroundings, critics often call them "killer robots.")
If democratic countries have to rely on those things to win, that victory isn't worth it.
👩💼 Emily
You've been working with Palantir since 2024. Their technology is used by ICE, local police departments, and in Gaza. Has Claude been used for other forms of surveillance?
👱♂️ Dario
We don't work with ICE, whether through Palantir or any other channel. We don't work with CBP. And I don't believe we have operations in Gaza.
We're extraordinarily careful about drawing boundaries around what we do, only taking on work we genuinely believe in.
Ideological Fanatics and the $200 Million Contract
In 2025, Anthropic joined OpenAI, xAI, and Google in securing a $200 million Pentagon contract, viewing the opportunity to "become a leading AI supplier to the government."
Claude was reportedly used by US forces in an operation to capture Venezuelan President Nicolás Maduro. And just weeks later, everything began to fall apart:
👩💼 Emily
Do you mind being called an "ideological fanatic" or "a bunch of left-wing zealots"?
👱♂️ Dario
You know, I've been called much worse. People can call me or Anthropic whatever they want.
Only two things actually matter: that we succeed as a company, and that we defend our values. In a sense, my life is very simple. When those are the only two things you have to do, you always know where you stand.
👩💼 Emily
So what does "winning this fight" actually mean?
👱♂️ Dario
I wouldn't even call it a "fight." It's more of a debate about "how should the government legitimately use AI."
AI is an emerging technology. We don't fully understand where it's reliable and where it's not, where it promotes our values and where it undermines them.
So I think what's important is to set precedents — both for good use cases and for ones we're concerned about. Others can sign contracts that don't respect your red lines, but we have to hold ours.
A contract can only do so much, but it does raise the salience of the issue.
To me, that's a win — getting our country to think more seriously about what the legitimate uses of this technology are.
Iranian Girls' Schools: Who Pressed the Final Button
👩💼 Emily
One US official said that "with LLMs, US military strike targets have gone from 1,000 per day to 5,000." That means Claude could help kill people faster. Does that sit well with you?
👱♂️ Dario
What you're essentially asking is, do you believe in this country, right? Do you want this country to be stronger rather than weaker on the world stage?
I do. I'm a patriot. But that's not for me to decide.
If we provide a technology, it shouldn't be up to us to say "you can or cannot execute this military operation." You essentially have to hand that decision over to military decision-makers.
But even under the best of circumstances, military decision-makers will make catastrophic mistakes.
👩💼 Emily
Bloomberg reported that Claude is being used by the US military in the war against Iran, through Palantir's Maven Smart System for AI-assisted targeting. In February, a US missile struck an Iranian girls' school, killing more than 150 people, most of them children.
What role did Claude play in that attack?
👱♂️ Dario
We don't have visibility, so we don't know how these models are being used.
Obviously, when something like this happens in war, it's incredibly, incredibly terrible.
We're willing to risk the future of our company to restrict the misuse of these models. What we're seeing here is Claude providing assistance, but the final decision being made by a human.
What's more worrying is the possibility of something a hundred times worse in the future: imagine a world where, not Claude (because we wouldn't allow this) but another company's AI model, the model makes the decision itself, and humans can't even see the process...
That's what we stood up to defend and push back against.
👩💼 Emily
That school had its own website. You could find it with a Google search. Shouldn't Claude have identified it? Doesn't this also point to a more terrifying problem: treating technology as a shortcut in war?
👱♂️ Dario
The principle being followed here is that humans make the final decision.
I don't know what role Claude or any other AI played in this, but if this doesn't illustrate "why that principle matters so much," I genuinely don't know what would.
👩💼 Emily
Is AI more likely to prevent a war, or more likely to cause one?
👱♂️ Dario
Weighing it up, I'd say it's more likely to prevent war, but if its use goes unrestricted, it's far more likely to cause one.
You've seen Dr. Strangelove, right? The premise is: you have a doomsday device, and once it believes it's under nuclear attack, it automatically launches nuclear retaliation.
What could go wrong? I think the way conflicts happen is often both sides rushing toward each other, misreading each other.
Similarly, when we lack proper oversight of this technology, these kinds of "accidents" become much more likely.
Mythos: A Superweapon Too Powerful to Release
Inside Anthropic, a model powerful enough to unsettle everyone is stirring — Mythos.
Anthropic has stated that Mythos can identify potential security vulnerabilities in nearly all mainstream operating systems. If fully released, Mythos could exploit vulnerabilities to infiltrate banking systems, steal state secrets, and cripple critical infrastructure.
In an operation called Project Glasswing, Anthropic granted access to Mythos to a small number of vetted institutions.
👱♂️ Dario
What surprised me most is that the model's ability to "find vulnerabilities" keeps leaping upward. With almost no prompting, the model is quite good at "turning vulnerabilities into exploitable attacks."
Some of the earliest companies we gave the model to said:
This is a superweapon. You need a gun license to use it. Please don't release it.
👩💼 Emily
You're essentially deciding who gets access and who doesn't. Why should anyone feel comfortable with that concentration of power?
👩 Daniela
This isn't as simple as "oh, it's too powerful, let us decide who deserves this power."
It's a very specific concern around cybersecurity.
So the way we decided who to give the model to was based on that specific concern.
Of course, there are many nuances to weigh, like "where exactly should the line be drawn?" I think that's genuinely debatable and complex. But we're as transparent as possible, and we tell people: we strive to make the right decisions, but we may not get everything perfect.
👩💼 Emily
Some researchers say they can replicate this with cheaper open-source models; some say OpenAI already has this capability. What do you say to people calling this "just a big PR stunt"?
👱♂️ Dario
The claim that "open-source models can replicate this" is extremely wrong.
Mythos scans entire codebases and finds problems. Someone said on Twitter, "if you point an open-source model at the exact line of code Mythos found, it can find the same problem." That's not the same thing at all.
The real test is, we go to companies, we go to open-source repositories, we found 271 new vulnerabilities in Firefox, and thousands more in private company codebases that haven't been fixed or disclosed yet. With the previous generation of models, nobody found those 271 vulnerabilities.
As for "marketing/PR stunt" — you know, Mythos dramatically accelerated Anthropic's internal research and production of next-generation models. If we had released it, it would have done the same thing in the outside world.
But by not releasing this model, we took a huge commercial hit.
👩💼 Emily
Have you made trade-offs that even you weren't entirely happy with?
👱♂️ Dario
Anthropic's entire history has been a series of trade-offs.
In an ideal world, you'd prefer to spend years studying every possible thing that could go wrong before releasing your first chatbot.
We did delay Claude's initial launch, but only by a few months. So everything is a trade-off.
Since we're already in what I'd call a "commercially leading" position, we have the leverage to turn the dial a bit more toward "caution" — and that's what we did with Mythos.
It's hard to do this if you're not the leader.
"Afraid of Companies Holding It, and Afraid of Governments Holding It"
On his first day back in the White House, Donald Trump — with the assistance of former AI and Crypto Czar David Sacks — repealed the previous president Biden's AI executive order that "sought to establish guardrails," pivoting to a hands-off approach that lets Silicon Valley develop freely:
We firmly believe that excessive regulation of the AI sector could kill a transformative industry that's about to take off.
But given Mythos and its profound implications for national security, the White House seems to have grown restless and begun showing an inclination to intervene in this powerful technology.
👩💼 Emily
There's an argument: why doesn't the government just take you over? How can they allow a private company to control technology this powerful?
👱♂️ Dario
I think this is a very serious question, and I share the same concerns, but I don't think the government should just take us over.
But look, every powerful technology in history that we've witnessed — nuclear weapons, the internet, GPS, cell phones — was either built by the government or originated from it.
AI is the first technology born in the private sector, where the government didn't really play a significant role and instead arrived late to the game.
This is actually a dangerous and unstable situation, and not one I would have chosen.
I'm afraid of companies holding it, and I'm afraid of governments holding it.
So we need checks and balances on power:
We need checks on AI companies' power — we have a Long-Term Benefit Trust, an institution that can appoint and remove a majority of board members, and by extension, it essentially has the power to fire me. And we need checks on the government side — to my knowledge, there's already effort in Congress to codify "those red lines" into law.
We also need basic regulation of this technology, especially after seeing Mythos. We need mandatory pre-release testing and auditing of models.
I find it quite funny that there's a certain group of people in Silicon Valley tech who started with the position that even adding transparency to this technology, even just export controls, would apocalyptically destroy our potential to create technology, would stifle innovation.
But the moment they see real danger starting to emerge, suddenly their mouths are full of "nationalization," "the government should just seize them."
Come on, folks. You're swinging from the most extreme anti-regulation — "you're destroying the industry just by looking at us" — all the way to outright communism — "the government should hold them in its grip." We need a smarter, more moderate path.
And that's precisely what we've been advocating for: because we truly understand where the power of this technology lies, we haven't panicked, and we haven't denied reality — we've seen that smooth exponential curve, and we're responding to it appropriately.
Blowback: When Protests Burned to Altman's Doorstep
Anthropic has cultivated an intensely loyal following — some people simply love what they stand for. But step outside the office tower, and protests have erupted there too. If you look at the data alone, people feel more anxious than excited about what's unfolding.
The truth is, if you ask the very people building all this, even they will admit: they don't fully know how it all ends.
👩💼 Emily
The protests against AI are getting personal. Sam Altman's home was attacked. How did that affect you?
👩 Daniela
Reading that news was genuinely frightening. He and his family were unharmed, and of course we felt relieved.
I think we're in a period of tremendous uncertainty, both technologically and politically. There's so much rhetoric and speech that could lead to bad outcomes, and I hope we can all debate this as calmly as possible.
👱♂️ Dario
It scared me too. As AI becomes socially consequential, more and more attention falls on it — that's the less glamorous side of that exponential curve.
👩💼 Emily
"Social media" faced massive backlash. Some countries started banning it. Do you think that could happen to AI?
👩 Daniela
I think it's entirely possible.
If you could go back and ask the people about to found Facebook, Instagram, Twitter — "think through all the ways this technology could go wrong, preempt some of the bad stuff" — would they still make the same choices looking at today's world? I'd like to believe the answer is "yes."
Social media stumbled on child well-being, mental health, electoral integrity. This generation of AI benefits from that trial and error, so we should be asking even earlier: can we use our understanding of this technology to identify and avoid those bad paths?
👱♂️ Dario
I'm not sure those companies were genuinely driven by "doing the right thing" or "making the world better."
So I don't think that even if they could go back, even knowing the results, they would necessarily choose differently — they certainly should, but I'm not sure they would.
But we can't be like that. That's why we're fighting to get it right the first time, rather than scrambling afterward to justify that "actually it's all fine."
To me, the main way I can imagine AI being banned is if something genuinely goes very wrong. And if something genuinely goes very wrong, then perhaps it deserves to be stopped.
👩💼 Emily
Some tech experts say AI will be incredibly wonderful; others say it could be utterly terrible. Where do you fall on that spectrum?
🧑💻 Boris
Hope for the best, prepare for the worst.
For me, this is the most important work I've ever done.
For some people, it feels like "the last job," because in some sense it may herald the end of work; for others, maybe it's not the last job, but "getting this right" is existential.
At some point, these models will become very dangerous, and "bad actors" will get access to them.
When that happens, we must ensure the "good guys" have a stronger model.
👩💼 Emily
If its impact is what we've been discussing, what Anthropic itself warns about, what responsibility do you think Anthropic bears? What do you owe to those whose life trajectories you've altered?
👩 Daniela
As an industry developing this technology, we all ultimately bear responsibility — to think through what risks exist, what adverse consequences might occur; and if some of them do materialize, what role should we play in helping fix it? That's the actual job.
You can't just say afterward, "well, we were just trying to scale the product." I don't think any tech company should take that posture, and I don't think it's one Anthropic wants to take.
"I see Oppenheimer as a cautionary tale"
For an organization that consistently maintains "we're going to get this right," "we're an AI safety mission-driven company," people struggle to understand why Anthropic is pushing AI forward at full throttle while simultaneously being so candid about its potential dangers.
In his essay Machines of Loving Grace, Dario painted a utopian future of "machines working alongside humans": AI as an unstoppable force leading humanity toward prosperity rather than catastrophe; to cushion the blow of unemployment, he proposed solutions like universal basic income and progressive taxation on AI companies.
But when the Amodei family confronts the messy collision of power, politics, and vested interests, the real test becomes: faced with the vast empire they themselves have built, can that original, pure mission survive?
Just as Google's founding motto of "don't be evil" was quietly shelved amid the waves of scaling.
👩💼 Emily
You're building something extraordinarily powerful, and stand to gain enormous benefit from it. Why should we trust you?
👱♂️ Dario
I think starting from a position of distrust, if you know nothing about Anthropic, is actually quite rational.
Silicon Valley has lost a tremendous amount of the world's trust, and needs to earn it back.
And the message we're trying to send is that we genuinely are different. That has to be earned through what we actually do.
👩💼 Emily
I understand one of your favorite books is The Making of the Atomic Bomb. (An epic work of narrative nonfiction tracing the complete history from the germination of nuclear physics in the early 20th century to the American Manhattan Project that produced humanity's first atomic bombs during World War II.)
👱♂️ Dario
That's right.
👩💼 Emily
Do you see any parallels between yourself and Oppenheimer?
👱♂️ Dario
The figure I most identify with is Leo Szilard — the person who first conceived of the possibility of a chain reaction.
My view is, we can't get through this by relying on some transcendent great figure or some central person trying to control everything.
In a sense, I actually see Oppenheimer as a cautionary tale, a case study of what shouldn't happen. There were too many powerful players with vested interests, and the only way for everyone to come out okay was to build in some kind of checks and balances.
👩💼 Emily
You've said the probability of civilizational collapse is somewhere between 10% and 25%. That's not a small chance. Is there a scenario where something Anthropic itself created causes it?
👱♂️ Dario
I certainly hope not.
My view is, that possibility stems from how the technology itself works, and from the fact that numerous interest groups exist in the world.
Half the work we do internally is trying to minimize risk as much as possible, but it can never be reduced to zero.
Suppose there are a bunch of airlines out there, and you say, "okay, I'm going to start a safer airline." Your airline is ten times safer than all the others, but if someone asks you, "can you guarantee your planes will never crash?" — I mean, how could you possibly guarantee that?
And "reducing the probability of a crash" and "being unable to guarantee absolute safety" — those two things can coexist.
👩💼 Emily
But if a plane had a 25% chance of crashing, you wouldn't get on it.
👱♂️ Dario
Right, 25% is way too high. We're working to push that probability down to something much lower. That's the goal.
Three Thousand People vs. a Horse That Knows Nothing
👩💼 Emily
"The role of AI on the battlefield" raises thorny questions, even for a company willing to openly discuss the ethical dilemmas of its own technology. How do you handle that pressure?
👱♂️ Dario
I work very hard at communication, and always stay direct and candid.
Every two weeks, I stand in front of the entire company and spend an hour just talking about what I'm thinking, what's happening in the industry, what's happening in the outside world — completely unfiltered.
This builds trust, and it's incredibly cathartic for me personally. I feel like there are 3,000 people on the same wavelength as me.
It's an unbelievable amplifier, and one of the most useful ways to handle pressure: when we've had to confront external challenges, stand firm through some extremely difficult situations over the past few months, it's let us present a consistent face.
I think it's a huge advantage, because I never feel alone.
👩💼 Emily
How do you find inner peace? How do you relax?
👱♂️ Dario
Honestly, a lot of it is just constant exposure — getting used to it.
Sometimes I'll take a weekend and play video games.
Sometimes Daniela — my wife and I — we'll go to Italy.
We have a horse there, and I'll just sit next to it, looking at this horse, thinking, "Calypso, you know, you have no idea about any of this. You're just a happy horse."
A Curve, a Missile, a Horse
From that endless exponential curve in the library to the unanswered question hanging over a school in Iran, Emily Chang devoted an entire episode to a single question: Can a company that builds the most dangerous technology with its own hands, yet claims to be safeguarding humanity, truly be trusted?
Dario left the answer to time. "Don't listen to what we say. Watch what we do."
That horse named Calypso still stands outside all this noise, head lowered, grazing, knowing nothing of the answer.
That "not knowing" is a kind of luck. And the question it doesn't have to answer — none of us can escape.
Crossing is looking for independent writers to produce AI product and model reviews.
If you've written pieces like "Hands-On with PixVerse C1[1]" or "Hands-On with LibTV[2]," please reach out to zeo0811@gmail.com. Your email should include: ① a brief bio, ② AI review articles you've written.
We offer competitive rates. Looking forward to observing and documenting the AI era together 🎪