Ted Chiang: ChatGPT Is a Lossy Compression Tool

The past month has been Oasis Capital's busiest start to a year since our founding: on one hand, we've been gathering as much AIGC information as possible from domestic and international sources to sharpen our understanding; on the other, we've met with numerous top scholars at Chinese universities to get at the facts. Amid all this excitement, we'll be sharing our perspectives and insights in the coming weeks so we can all meet the AIGC moment together — and we look forward to hearing what information and perspectives you have to share with us.

The following piece comes from Ted Chiang, one of the finest contemporary Chinese-American science fiction writers in the United States. He offers one lens for thinking about AIGC, and we've translated it here with help from DeepL. We hope you find it useful. Enjoy.

In 2013, workers at a German construction company noticed something strange happening with their Xerox photocopier: when they made copies of a house's floor plan, the copies differed from the original in subtle but critical ways. In the original plan, three rooms each had a rectangle indicating their area: 14.13, 21.11, and 17.42 square meters. Yet on the copies, all three rooms were labeled as 14.13 square meters. The company contacted computer scientist David Kriesel to investigate this bizarre situation.

They needed a computer scientist because modern Xerox copiers don't use the physical xerographic process popular in the 1960s. Instead, they scan documents digitally and print image files — a technology in which nearly every digital image file is compressed to save storage space. And so the mystery of the distorted copies began to unravel.

Image generated by Tiamat

Specifically, compressing a file involves two steps: encoding, in which the file is converted to a more compact format; and decoding, which reverses the encoding process. If the recovered file is identical to the original, the compression is described as lossless — no information has been lost. Conversely, if the recovered file is only an approximation of the original, the compression is described as lossy. Lossless compression is typically used for text files and computer programs, where even a single incorrect character could have catastrophic consequences. Lossy compression is typically used for photos, audio, and video in cases where absolute precision doesn't matter — because we won't notice whether a picture, song, or film has been perfectly reproduced. Only when files are compressed very tightly does the loss of fidelity become more obvious. In these cases, we notice what's called compression artifacts — the blurriness of heavily compressed JPEG and MPEG images, or the poor quality of low-bitrate MP3s.

The Xerox copier used a lossy compression format called jbig2, designed specifically for black-and-white images. To save space, the copier would identify regions of the image that looked similar and store a single copy of those regions; when the file was decompressed, it would reuse that copy to reconstruct the image. The result: the copier judged the labels indicating each room's area to be so similar that it only needed to store one of them — the 14.13 square meter room — and when printing the floor plan, it reused that same label for all three rooms.

That the Xerox copier used a lossy rather than lossless compression format wasn't itself a problem. But the copier compressed the image in such a subtle way that the compression artifacts weren't immediately recognizable (Xerox released a patch to fix this in 2014).

I believe this incident with the Xerox copier is worth keeping in mind as we examine OpenAI's ChatGPT and other similar programs — what AI researchers call "large language models." Copiers and large language models may seem worlds apart, but consider this scenario: suppose you will permanently lose internet access and, to prepare, you plan to create a compressed copy of all text on the internet so you can store it on a private server. Unfortunately, your private server only has 1% of the space you need; if you want everything to be accurate, you can't use a lossless compression algorithm. But you could write a lossy algorithm that identifies statistical patterns in the text and stores them in a specialized file format. Since you have virtually unlimited computing power for this task, your algorithm can identify very subtle statistical patterns, allowing you to achieve the required 100:1 compression ratio.

Losing internet access no longer seems so scary, because you've stored all the information on the web in your server. The only problem is that because the text is so heavily compressed, you can't search for information using exact keywords — you'll never get an exact match, because what's stored isn't words. To solve this, you create an interface that accepts queries in the form of questions, finds the relevant answers on your server, and communicates the response.

Doesn't this process I just described sound a lot like ChatGPT, or most large language models? Try thinking of ChatGPT as a lossy compressed JPEG of all text on the internet. It retains most of the information on the web, just as a JPEG retains most of the information of a high-resolution image. But you can't find the exact sequence of bits; you only get an approximation. Because this approximation is presented in the form of grammatically correct text, and ChatGPT is particularly good at generating such text, the results it displays are usually acceptable. What you get is a blurry JPEG, but because it's not "blurry" in the traditional sense, you don't feel like you're looking at an unclear picture.

Image generated by Tiamat

This analogy with lossy compression not only helps us understand that ChatGPT is repackaging information gathered from the internet, it also offers a way to understand why people experience "hallucinations" or produce nonsensical answers to factual questions. Large language models like ChatGPT are particularly prone to this — these hallucinations are artifacts of compression. But like the erroneous labels produced by the Xerox copier, they are so plausible that identifying them requires comparison with the original. That is, to determine whether content generated by a large language model is accurate, you must either compare it against information on the internet or against your personal knowledge of the world. Viewed this way, hallucinations become entirely comprehensible. If a compression algorithm is designed to discard 99% of the original text before reconstructing it, we should expect that much of what it generates will be entirely fabricated.

A common technique used by lossy compression algorithms is interpolation — that is, estimating what's missing by looking at what's on either side of the gap. When an image program displays a photo, it must reconstruct pixels lost during compression; it looks at nearby pixels and calculates an average. This is what ChatGPT does when people ask it to describe socks lost in a dryer in the style of the Declaration of Independence: it takes two points in "word space" and generates text that occupies a position between them ("When in the course of human events, it becomes necessary for one to separate his garments from their fellows, that they may be cleansed and ordered..."). ChatGPT is remarkably good at this, and people have discovered a tool for "blurring" paragraphs rather than photos — and they're having great fun with it.

Given that large language models like ChatGPT are often touted as the cutting edge of artificial intelligence, describing them as "lossy text compression algorithms" may seem like a letdown, even deflating. But I do think this can correct the trend toward anthropomorphizing large language models. The compression analogy has another angle worth considering, though. Since 2006, an AI researcher named Marcus Hutter has offered a cash prize — known as the "Compression of Human Knowledge Prize" or the "Hutter Prize" — to anyone who can losslessly compress a specific 1GB snapshot of Wikipedia by more than the previous winner. The requirements are simple: compress the data smaller than the last winner. If you've ever used zip files, zip can compress Hutter's 1GB file to around 300 megabytes; recent winners have managed to reduce it to 115 megabytes. This prize isn't just an exercise in refinement. Hutter believes that better text compression would help create human-level AI, in part because maximum compression can only be achieved by understanding the principles behind the text.

To grasp the proposed relationship between compression and understanding, imagine you have a text file containing a million arithmetic problems and their solutions. While any compression algorithm could reduce this file's size, the way to achieve maximum compression would be to derive the principles of arithmetic and write code for a calculator program based on them. Such a calculator could not only perfectly reconstruct the million problems in the original file, but also any other arithmetic problems you might encounter in the future. The same logic applies to compressing Wikipedia excerpts: if a compression program understands that force equals mass times acceleration, it can discard much content when compressing physics pages because it can reconstruct them. Similarly, the more a program understands supply and demand, the more it can discard when compressing economics pages, and so on.

Large language models can identify statistical regularities in text. Analysis of web text reveals that phrases like "in short supply" frequently appear adjacent to phrases like "prices rise." When asked a question about the effects of supply shortages, a chatbot containing this correlation might answer with something about rising prices. If a large language model compiled extensive correlations between economic terms to the point where it could provide reasonable responses to various questions, could we say it understands economic theory? Models like ChatGPT are ineligible for the Hutter Prize for many reasons, one being that they don't precisely reconstruct the original text — that is, they don't perform lossless compression. But could their lossy compression still embody the kind of "genuine understanding" that AI researchers are interested in?

Let's return to the arithmetic example. If you ask GPT-3 (the large language model underlying ChatGPT) to perform addition and subtraction, it almost always answers correctly when the numbers have only two digits. But as the numbers get larger, its accuracy drops; by five digits, its accuracy falls to ten percent. Most of GPT-3's correct answers can't be found on the web — there aren't many web pages containing text like "245+821," so it's not simply memorizing. Yet despite ingesting vast amounts of information, it hasn't been able to derive the principles of arithmetic. Examining GPT-3's incorrect answers reveals that it doesn't carry the "1" when doing arithmetic. There are certainly explanations of carrying the "1" on the web, but GPT-3 hasn't incorporated this understanding. GPT-3's statistical analysis of arithmetic examples allows it to produce something that approximates the appearance of real arithmetic, but nothing more.

Since GPT-3 fails at an elementary school subject, how do we explain its sometimes impressive performance on college-level essays? Despite large language models' frequent hallucinations, they sometimes sound as if they genuinely understand subjects like economic theory. Perhaps arithmetic is a special case, and large language models haven't yet cracked the logic behind it. Is it possible that in domains beyond addition and subtraction, statistical patterns extracted from text content could correspond to real-world knowledge?

I think there's a simpler explanation. Imagine what would happen if ChatGPT were a lossless algorithm. It should answer questions by providing verbatim quotes from relevant web pages, and we would think of the software as merely an upgraded version of a traditional search engine — we wouldn't be nearly so impressed. In fact, ChatGPT's fresh reformulation of material from the web, rather than quoting it verbatim, makes it seem like a student expressing ideas in their own words rather than simply parroting what they've read; it creates the illusion that ChatGPT understands the material. Among human students, rote memorization isn't considered an indicator of learning, so ChatGPT's inability to accurately quote web content actually makes us think it has learned something. When we process sequences of words, lossy compression looks smarter than lossless compression.

People have proposed many uses for large language models, and comparing them to blurry JPEGs is one way to evaluate what they may or may not be suitable for. Let's consider several scenarios:

Can large language models replace traditional search engines? For us to trust them, we'd need to know they haven't been fed propaganda and conspiracy theories — we need to know the JPEG is capturing the right parts of the web. But even if a large language model contained only the information we wanted, the problem of blurriness would remain. There's an acceptable kind of blurriness: rephrasing information in different words. There's also an unacceptable kind of blurriness: fabrication, which is intolerable when we're seeking facts. It's unclear whether it's technically possible to preserve acceptable blurriness while eliminating unacceptable blurriness, but I hope we can achieve this in the near future.

Even if it's possible to limit large language models' blurriness, should we use them to generate web content? This only makes sense if our goal is to repackage information already on the web. Some companies exist precisely for this purpose — we usually call them "content mills." Perhaps the blurriness of large language models is useful to them, helping them avoid copyright infringement. But I'd argue that what's good for content mills isn't good for people searching for information. The rise of this kind of content processing has already made it harder to find what we actually need online; the more text generated by large language models gets published on the web, the more the web becomes a "blurry" version of itself.

Although there isn't yet enough public information about GPT-4, the next generation of ChatGPT that OpenAI is preparing to release, I'll make a prediction: in collecting the massive amounts of content used to train GPT-4, OpenAI will do everything possible to exclude material generated by ChatGPT or any other large language model. If this proves true, the measure will inadvertently confirm that the analogy between large language models and lossy compression is meaningful. Repeatedly resaving a JPEG produces more compression artifacts, because more information is lost each time — just as when we used to make copies of copies, image quality only got worse.

In fact, one useful measure of a large language model's quality might be whether a company is willing to use the text it generates as training data for a new model. If ChatGPT's output isn't good enough for GPT-4, we can take that as a benchmark indicating its quality isn't good enough either. Conversely, if a model generates text excellent enough to be used to train new models, then we can have confidence in the quality of its output (I suspect such results would require major breakthroughs in the technology underlying these models). If we begin to see model outputs that are as good as their inputs, then the lossy compression analogy will no longer apply.

Image generated by Tiamat

Can large language models help humans with original creation?

To answer this, we need to clarify what originality means. There's an art movement known as "Xerox art" or "copy art," in which artists use the copier's characteristics as a creative tool; using ChatGPT's copying abilities could certainly serve these purposes. In this sense, large language models can help with originality. But I don't think anyone would claim copiers are essential tools in artistic creation; the vast majority of artists don't use copiers in their creative process, and no one thinks this limits their originality.

Now let's assume we're not talking about "Xerox art"-type originality, but rather new written creation. Can text generated by large language models serve as a useful starting point for writers, whether of fiction or nonfiction? Can having large language models handle the boilerplate allow writers to focus on the truly creative parts?

Obviously, no one can speak for all writers, but here's my view: starting from a blurry copy of unoriginal work is not a good method for achieving originality. If you're a writer, before you produce original work, you write a great deal of unoriginal work. And the time and effort spent on this unoriginal work is not wasted; on the contrary, I believe this is precisely what enables you to eventually create original work. The time spent choosing the right words and arranging sentences to better connect is what teaches you how prose conveys meaning. Having students write essays isn't just a way to test their mastery of material; it gives them experience expressing their own ideas. If students never write the kinds of essays we all know, they'll never acquire the skills and techniques needed to create something genuinely new.

And it's not as if once you're no longer a student, you can safely use templates provided by large language models. The struggles and tangles of expressing ideas don't disappear after graduation; they can occur when you begin drafting new work, sometimes only in the act of writing itself — it's in those moments that you discover your original ideas. Some might say that large language model output looks no different from a human writer's first draft, but I maintain this is only a surface similarity. Your first draft isn't a clearly expressed unoriginal idea; it's a poorly expressed original idea, accompanied by your amorphous dissatisfaction, your awareness of the distance between what it says and what you want it to mean. This is what teaches you during the rewriting process, and it's precisely what's most lacking when you begin with AI-generated text.

There's nothing magical or mysterious about writing, but it involves more than placing existing documents on an unreliable copier and pressing the copy button. In the future, we may build an AI that can write good prose based on its own experience of the world. The day we achieve this will certainly be momentous, but it's far beyond anything we can currently predict.

In the meantime, we have reason to ask: what use is a tool for repackaging web content? If we truly could never access the internet and had to store a copy on a space-limited private server, then a large language model like ChatGPT might be an excellent solution, provided we could ensure it doesn't fabricate content. But since we haven't encountered such an extreme situation, how useful is a blurry, lossily compressed JPEG when you still have the original?

Oasis Capital is a new-generation venture capital firm in China, dedicated to discovering the most vital entrepreneurs of the next decade and growing alongside them to create long-term value. "Championing Vitality" is Oasis's vision and mission. This vitality is both the direction of structural transformation in our era and the resilience and evolutionary power of entrepreneurs.

Oasis Capital focuses on early and growth-stage investments, with individual investments ranging from $3 million to $30 million, concentrating on healthcare, biotechnology, enterprise services, and other technology-enabled service sectors, supporting China's new service upgrade driven by technology.