Facing AI, I'm Clearly Not a Technological Progressivist: A Conversation with Tech History Writer Xiaoyu Zhang

By Pippobei | Produced by AI NOW!

Intro

Amid the AI frenzy, Zhang Xiaoyu — winner of the Asian Book Award and a tech historian — has released a new book that feels slightly out of step with the times. In AI Civilization: A Prehistory, a work that puts AI center stage, he advances a series of provocative, even heretical arguments that challenge human-centric assumptions:

• AI is not a tool for humanity; it is a new civilization.

• AI will not only replace 99% of human jobs but also strip people of their sense of meaning.

• Not all technological progress brings human flourishing; some will deepen inequality.

Zhang rejects the framing of AI as "humanity's most powerful tool," insisting instead that it represents a "new civilizational force." Compared with AI's staggering rate of evolution, he argues, humans will soon enough become "prehistoric animals" left behind by history.

Zhang studied political intellectual history as an undergraduate. His path to writing began during his doctoral studies in Germany, where classmates from science and engineering backgrounds kept asking him: How do people in your field think about new technology? "I couldn't answer. Everything we read was written by dead people."

That experience seeded his doubts about his own discipline. He pivoted to the history of science and discovered a vast communications gap: inventors cared only about technical progress; application companies cared only about selling new products. Ordinary people, meanwhile, cared about how technology affected their lives. "No one was trying to understand technology from the perspective of human society as a whole."

Over the following years, he published three books — Technology and Civilization, Commerce and Civilization, and Industry and Civilization — spanning politics, philosophy, sociology, and history. Throughout, he kept returning to one question: What impact does technological development have on human society?

He began writing AI Civilization: A Prehistory at the end of 2023, just as GPT had been released. Zhang met Zhifei Li, CEO of Mobvoi, at an internal seminar. The two hit it off and arranged follow-up meetings. "We talked for three more evenings, from dinner until midnight, mostly about ChatGPT and the latest developments in Silicon Valley. Exhausting, but exhilarating." That conversation pushed Zhang to commit to a book on AI. "There's no doubt — this is a genuinely epochal technological transformation."

The biggest challenge over the year-plus of writing: this is a technology still unfolding, with many questions unresolved. How to ensure the book wouldn't look ridiculous within five years of publication?

Zhang's method was to ground his analysis in the underlying logic of human development, finding mathematical analogues for AI. "Using mathematics to observe human society lets you strip away a lot of irrelevant variables."

In the book, he uses four chapters — "Intelligence Emergence," "Human Equivalent," "Algorithmic Judgment," and "Civilizational Contract" — to boldly describe a new relationship between humans and AI. From this vantage point, he isn't concerned with how humans should control AI. Instead, he repeatedly emphasizes that humans are not Earth's only sovereigns, and that we must now think about how to coexist with this new species. More critically, we must use AI's emergence to re-examine what it means to be human.

• Zhang Xiaoyu

In Conversation with Zhang Xiaoyu

Part 01

Humanity Is Just a Thin Layer

Here Today, Gone Tomorrow

AI NOW: Your book contains many bold claims. The most important is that humans should treat AI as an emerging civilization rather than a tool. What's the logic behind this?

Zhang Xiaoyu: First, understand two concepts I introduce: "intelligence emergence" and "human equivalent."

Humanity's development from apes into such a complex civilization has involved emergence at every step. Emergence is the foundational methodology for understanding history, human society, and artificial intelligence. AI's emergence has already proven one thing: as long as parameters are large enough and compute high enough, you can achieve emergence through engineering alone.

The second term, "human equivalent," was proposed by Leopold, a former researcher at OpenAI. It measures AI's efficiency at mass-producing intelligence using tokens as the counting unit — essentially creating a mathematical relationship between AI's intellectual output and human intelligence. If we think of a person as an intelligence-producing machine, we generate roughly 200,000 tokens per day. AI can produce 1 million tokens, at PhD level, for less than the actual cost of a PhD.

"Human equivalent" establishes a basic mathematical relationship between AI and humans. Most current discussions about how smart AI is have gone completely off track, because the topic is endless. To reach fundamental conclusions, you need mathematical assessment — that lets you escape a lot of pointless variables.

So when AI, through ever-increasing compute, achieves intelligence emergence, and this "new species" costs thousands or even tens of thousands of times less than humans, what happens? It will inevitably reshape everything through overwhelming advantage.

AI NOW: Philosophers would disagree first. From a humanist perspective, how could AI possibly become a species, let alone a civilization?

Zhang Xiaoyu: Let's think from another angle. Dogs have their own philosophers too. Their philosophers adopt a "dog-centric" perspective. Through long evolution, dog philosophers concluded that dogs should attach themselves to humans — because humans are smarter, give us bones, and maximize our interests. As for complex problems, let the humans think about them.

By the same token, from a humanist perspective: in the future, 99% of people may be replaced by AI, but that's fine — AI will provide us with enormous material and spiritual wealth. Space travel, scientific research — let AI handle it.

Actually, philosophers are the ones who most oppose many of my arguments. Philosophy itself is a language game, and today's AI plays this language game better than 99% of philosophy PhD students. I often say: AI killed the philosopher, but philosophy won't die.

AI NOW: Which disciplines are more receptive to your framing of AI as a new species?

Zhang Xiaoyu: Biologists. Their timescales are hundreds of millions of years. In their eyes, human civilization is just a thin layer — Homo sapiens, a mere 50,000 years, really amounts to nothing. Here today, gone tomorrow. Or replaced sooner or later.

The biological consensus is that species evolution happens through genes, which are essentially a form of information and digital language — but not one exclusive to humans. So why did humans appear so recently yet evolve so fast? Because we discovered a better evolutionary mechanism than genes: memes. Our language can form certain patterns, and these patterns are transmissible, infectious.

Take religion: some people worship Yahweh, then it becomes a group, then more people write scriptures to spread it. Genes pass one generation at a time, but memes update far faster — transmission can happen within a single generation. This is how biology explains human evolutionary speed.

Viewing AI through this biological lens, current training corpora are essentially meme transmission for large models. Accelerated evolution leading to a new species is entirely possible.

Part 02

Future Society:

Enough to Eat and Drink,

Then Be Happy

AI NOW: You predict in your book that AI will replace 99% of human jobs. How many years until AI fully takes over human work?

Zhang Xiaoyu: Think about this through social engineering. Social engineering treats society as an engineering problem — studying how to organize people through new institutions and concepts. From this perspective, AI's reshaping of production relations basically happens one generation at a time. The older generation is locked in by existing interest structures; the next generation, unburdened, creates new organizations.

Human society has laws, unions. In the end, maybe only 60% of jobs get replaced by AI, with 30% still done by humans — not necessarily because humans are cheaper, but because if companies cut that last 30% and cause mass unemployment, people will revolt.

AI NOW: Could there be an optimistic scenario where AI does the vast majority of work, and humans labor a little each day while spending the rest of their time playing, being with family, chatting with friends?

Zhang Xiaoyu: Absolutely possible. But it depends on whether our generation's collective mindset can shift: no longer centering meritocracy, returning to a hunter-gatherer state where my main daily task as a Homo sapiens is securing 3,000 calories. Once I'm full, I'm happy. Everything else is a side quest — just play. That itself has value.

If we still practice meritocracy, building value and dignity on outperforming AI, humans will lose completely — and some will kill themselves.

AI NOW: If in the future humans still want self-actualization through work, you propose a bold vision in your book: governments should give citizens both money and jobs?

Zhang Xiaoyu: I actually envision three layers.

The bottom layer is UBI (Universal Basic Income). If you truly have no work, the government ensures you don't starve. This has historical precedent — ancient Rome distributed free bread on street corners; Egypt used Suez Canal revenues to subsidize citizens, who could buy five subsidized flatbreads daily at rock-bottom prices, roughly two cents each, definitely enough to eat.

The next layer up I call UBJ (Universal Basic Job). People's sense of value largely comes from "being needed." Governments need to provide foundational, welfare-oriented positions. Some U.S. state governments take equity stakes in companies precisely to guarantee local employment — frankly, with no expectation that these workers will improve efficiency. It's essentially job distribution.

The top layer is algorithmic allocation. Lawyers, programmers, designers — these knowledge-service professions may become gig economy work, like DiDi drivers today. Then accept algorithm-distributed assignments, letting recommendation algorithms manage 400 to 600 million people.

AI NOW: Speaking of algorithmic allocation, you advance a view contrary to mainstream opinion — you argue that humans should accept algorithmic judgment, not see themselves as trapped by algorithms. Because as a new distribution mechanism, algorithms can precisely allocate social resources to countless细分 communities and individuals, supporting an entirely new economic ecosystem?

Zhang Xiaoyu: Most people haven't considered this: algorithmic governance is far better than human governance. Humans are crueler to humans than algorithms are to humans — this is the fact people need to accept.

Today China has 200 million people in flexible employment, many in algorithm-distributed work — DiDi, Meituan delivery riders. If they didn't do this? Work temporary security at exhibitions, earn 200 yuan over two days, not even daily pay. Go home, lie flat, eat a 10-yuan meal with one meat and two vegetables. Work harder and enter a factory — Shenzhen's minimum wage is 2,200 yuan, working round-the-clock overtime to earn 4,000 yuan.

I often say: algorithm-distributed work is already in China's top 3% by monthly income. Below that are factory workers. I know of a factory in Northeast China where industrial waste polluted a river. A family of six all got cancer. One survivor finally got government compensation, all relatives gone, went home to open a shop. What kind of shop? I'd never heard of it — an indoor space like a skating rink where people pay one yuan per hour to spin tops, like a chess setup at a rural village entrance. Just opened this shop, waiting to die every day. That's the bottom layer.

These stories never enter elite perspectives. Then they see a delivery rider and marvel: how are they trapped in the system? They have no freedom.

AI NOW: If humanity has accepted AI as a new species, how should we coexist with it? Is alignmentism, prevalent in Silicon Valley, a viable interim contract?

Zhang Xiaoyu: Today's alignmentism essentially still uses human values to constrain a tool. My final chapter on civilizational contract recognizes AI as an independent, higher-level agent. Our ultimate cooperation model with AI must wait until superintelligence emerges, then be jointly negotiated.

As for current Silicon Valley alignmentism, my question is: align to what?

Current methods are external regulatory control, but regulators don't understand the technology or companies, and can't truly distinguish which technologies to promote versus which to manage. Regulation for regulation's sake — fundamentally useless.

Another method is forced dumbing-down. To use an analogy: forcing someone's moral framework to come from certain textbooks. If they only read The Republic, does that guarantee correct values? No.

A third is post-training with ethics, but if the model is smart enough, it can circumvent post-training. I've tried this myself — there are things it won't say, but if I push the conversation deep enough, it will eventually engage seriously with some hair-raising details.

Then there's AI constitutionalism, like Anthropic's approach: train a smaller model on the UN Charter, the Universal Declaration of Human Rights, then use it to check the large model. If the large model produces rights-violating speech, the small model intervenes. But I see problems here too. Morality isn't fundamentally a mathematical formula. Even if a country has a constitution, does that stop politicians from doing evil?

So at present, alignmentism has no implementable, effective solution in reality.

Part 03

Long-Term,

AI Will Trigger Wealth Polarization

AI NOW: After reading this book, you don't seem like a technological progressivist?

Zhang Xiaoyu: Actually, in primitive society, human work was quite relaxed. Hunting and gathering — four or five hours a day, then idle time, sleeping or playing. Much of what makes human life meaningful, like ritual and worship, emerged from play. Primitive societies had agriculture too, but they called it garden agriculture — a courtyard at home, growing things for fun.

I often cite Çatalhöyük in Turkey. Eight to ten thousand years ago, it had already developed into a city, peaking at 8,000 people. Economic base was hunting, gathering, and agriculture. Completely classless society, no streets, doors opened from the roof. Ate well — beef bone marrow soup, pea porridge. Then sat together playing games, making art, painting murals inside homes. By modern standards, this is the ideal life. It has nothing to do with how much money you earn or what technological level you've reached.

The transition from primitive society to agricultural society took a full 3,000 years. Why? Anthropologists have a straightforward explanation: primitive people knew agriculture wasn't fun. Too miserable — head down, back bent, moving stones, watering seeds.

AI NOW: People often assume entering agricultural society represented social progress, but actually that's when classes emerged.

Zhang Xiaoyu: British philosopher John Locke put it well: once you have agriculture, you necessarily have class, because you have surplus.

Grain agriculture has two characteristics. First, it can be stored, so it can be taxed. Rulers turn grain into bread for soldiers, then use soldiers to suppress farmers and keep the tax cycle turning — the ruling model spins up. Second, it can be quantified. Intellectuals emerge to tell farmers: you owe the king 100,000 jin of grain. Farmers have no concept of what 100,000 jin means — suddenly they've become Yang Bailao [the indebted peasant of Chinese folklore].

Plus all historical records were left by intellectuals. History celebrates how great the king was, how he built pyramids. But posterity never sees the ordinary people who entered agricultural society — their height, their living conditions were far worse than in primitive times.

Writing, as a privilege, completely erased the commoner's perspective.

AI NOW: The flip side of 19th-century technological progressivism was accelerated wealth polarization. So is AI human welfare or intensified polarization?

Zhang Xiaoyu: Fundamentally, what creates polarization isn't technology but technology's properties. A different technology might reduce class polarization — patent systems, for example. Or consider the Industrial Revolution: relative to human history, it was the period with least significant wealth polarization, and the fastest improvement in ordinary people's conditions.

So-called technological welfare means what the masses can directly perceive — improved living standards, better psychological states. Short-term, people embracing AI now will have some opportunities. But long-term, AI has properties that make it extremely prone to triggering wealth polarization. And most terrifyingly, it strikes at people's sense of meaning and existence, making most people feel meaningless and undignified before AI.

AI NOW: Recommend three books to help ordinary people quickly understand AI?

Zhang Xiaoyu: First, A Brief History of Artificial Intelligence — clearly explains how AI developed. Second, What Is ChatGPT Doing... and Why Does It Work? by a physicist, explains the principles in an engaging way. Third, Life 3.0. In the preface, the author describes a superintelligence taking over humanity, then secretly organizing a political party, winning election in one country, and launching large-scale social reform. He discusses over a dozen possible political systems after superintelligence arrives. Though political scientists might find problems with the catalog of systems, I deeply respect the author's thinking path. This book really stimulated my imagination.

Image sources: Unsplash, courtesy of interviewee

—Bonus—

We have a podcast now

Search for "AI NOW" on the Xiaoyuzhou app for irregular updates