Travel Between The Pages

Over the last month or so I have seen quite a few news articles and blog posts about how Anthropic, creator of the large language model (LLM) Claude, set out in 2024 to “destructively scan all the books in the world,” as an internal document put it. The company called its initiative Project Panama, a “soft codename” chosen “because we don’t want it to be known that we are working on this.”

Bookdealer Scott Brown recently published an article on Substack about his experience with the voracious AI bookeaters. Here’s excerpt and a link to the fascinating full story Scott Brown’s Dispatches from the Rare Book Trade on Substack.

I’ve sold four books to AI companies which added $439 to my gross sales before online commissions were subtracted. Three of the books were titles I wouldn’t ordinarily stock; they arrived with books I actually wanted and seemed too valuable to put in my free pile: Analytical Characterization of Aluminum, Steel, and Superalloys (2006); Austronesian and Other Languages of the Pacific and South-East Asia: An Annotated Catalogue of Theses and Dissertations (1981); and Languages of Irian Jaya Checklist: Preliminary Classification, Language Maps, Wordlist (1975). The fourth title gave me more pause. It was a nice first edition in a dust jacket of Al Perkins’s Tubby and the Poo-Bah, a title in Dr. Seuss’s Beginner Books series. For some reason, it’s a slightly scarce book and my $150 first edition was the least expensive copy on Abebooks.

I don’t positively know what happened after I mailed the books to massive logistics warehouses in the middle of the country, but I can make a pretty good guess. They were de-spined by an electric guillotine, fed into a high-speed scanner, and tossed into a Dumpster. A server farm deconstructed the digitized text, reduced it to a series of probabilities, and fed it to one of the large language models behind an AI chatbot.

Along with my colleagues in the used and antiquarian book trade, I find myself at the center of a maelstrom of public outrage over the training of AI models.

Used books don’t normally get this much attention. The stereotype of used and rare booksellers is that we sit around dressed in tweed reading books all day. Suddenly the goods we sell are indispensable for training AI systems. Around the world, thousands of small, independent shops have received a tsunami of anonymous orders for books.

There are real questions whether scanning books is even legal under copyright law. The AI companies are not seeking permission, and to cement their generally dismissive attitude toward the human creators whose work is so essential to their software, they’ve gone about getting books in the shittiest ways possible.

In the first lawsuit to settle – of more than one hundred currently underway against AI companies – Anthropic, the maker of the Claude chatbot, has agreed to pay the authors and publishers of 500,000 books a total of $1.5 billion. The suit had two components, one focusing on pirated ebooks and the other on scanned physical books. It will take years and probably the Supreme Court to settle the copyright issues.

The Anthropic settlement put hundreds of internal company documents into the public record. In those documents were the unpleasant details about Anthropic’s Project Panama, “our effort to destructively scan all the books in the world”.

Anthropic didn’t have to annihilate books; they didn’t have to put booksellers in the position of selling books to be destroyed. In fact, the man they hired to run Project Panama, Tom Turvey, had been one of the leaders of the largest non-destructive scanning operation ever attempted, Google Books’s program to digitize tens of millions of library books without harming the originals.

With that legacy, he could have gone down in history as one of the greatest preservers of books in history. Instead, he is the public face of one of the largest libricides on record. Internal documents explain that “books will be cut to provide cost effective conversion.” A company spending billions of dollars for every data center opted not to spend a few extra million on non-destructive scanning.

The only consolation for book lovers is that the mission to destroy all the books in the world was an example of corporate grandiosity. Anthropic’s lawyers repeatedly insisted that the word book meant “written works that have been issued a 10- or 13-digit ISBN”. So they didn’t intend to buy and destroy one copy of every book ever published, just most of the books issued since the ISBN was introduced about 1970.

Anthropic decided to focus their scanning on used books, which had lower prices. Most of Anthropic’s books came from mega-sellers like Better World Books, World of Books in the UK, and Thrift Books.

Project Panama is now just the tip of the iceberg. Many other AI companies around the world are following Anthropic’s plan, acquiring vast quantities of books in secret for destructive scanning.

Beginning in the second half of last year, the insane AI demand for books overflowed the capacity of sources like Better World Books and Thrift Books. The first I heard about it was on the blog of Chuck Roberts, the owner of Wonder Book in Washington, DC. He asked in one post, “Will we move a million books? What a challenge. That would be close to twenty miles of books.” In another, he referred to his massive orders as “The Book Rush of ’26. This century’s Gold Rush for printed matter.”

The sales put booksellers in an odd spot. AI companies have enlisted us in an effort to destroy books, pitting our belief in freedom of expression against our mission to preserve books. Lots of booksellers support Banned Book Week, a movement to draw attention to efforts to suppress books. Booksellers have a fundamental belief in people’s right to books, even when we personally disagree with the content of the books or the beliefs of the customer. We have an abiding optimism that good ideas will win out over bad, at least in the long run. Now we have to decide if these principled stands still apply when the readers are machines, not people. The answer is not cut-and-dried.

I don’t know anyone who likes the idea of destroying books. People have a visceral response to it. Even people who don’t read are often reluctant to discard a book. More than any other medium, books store the collected wisdom, knowledge, history, and beliefs of humankind. Books have individual authors but they always reflect the spirit of their time, which is a shared creation. We all contribute to the language we speak and the culture we inhabit. The idea of a machine chewing through books strikes people as violence against humanity itself.

I did an informal survey of booksellers about how they felt about selling books to AI companies. The overriding sense of the comments was one of distaste. “I’d hate to be sending good books to be destroyed” is fairly typical of the responses. Another bookseller said the AI purchases were “ultimately an unsatisfying set of transactions.”

The sales, however, have been good for the book trade. The used book business is notorious for its slow rate of inventory turn. Normal retailers cull items that haven’t sold after a few months on the shelf. Booksellers regularly keep books for decades. One finally found vindication in that approach with the sales to AI companies.

We have always had a lot of slow-to-sell stock so it’s been somewhat of a redemption for our 50 years of (arguably) irresponsible buying habits. Eighty percent of books we’ve sold to the bulk buyers were books we’ve had online for 15 or 20 years, and were almost certainly headed to the Dumpster or donation at some point…

We’ve tried to be selective in what we sell. For example, we’ve cancelled orders for several books that are too rare/important/cool to potentially be destroyed….

I’m not sure what the online bookselling world will look like when it’s over, but it is nice to finally get to cash in a little on a bubble rather than get screwed yet again.

Another bookseller, in South Africa, wrote me and described the books going to AI companies.

The vast majority of the books had been shelf-sitters for many years, even decades. Had these orders not come along, I would almost certainly have had to dump volumes of Afrikaans poetry, apartheid history, obscure local histories, and heaps of cookery and military history titles. For many of us who live in isolated spots, with no walk-in trade, the sudden interest in our inventory has been more than welcome.

A few booksellers have refused to send any books to the AI guillotines. One told me, “I cancelled these orders and took a massive hit to my fulfillment rating. This isn’t bookselling.”

Most booksellers seem to be aiming for a middle ground, withholding books where their value as artifacts exceeds AI’s need for the text. The problem gets trickier with books where acquiring an artifact, the first edition or a copy signed by the author, is the only way to obtain the text, because it has not been reprinted or is scarce on the market.

One bookseller expressed concern about sending an uncommon work on Chicana anarchism to the AI companies for destructive scanning. The large language models behind AI chat bots already have an inherent bias toward content they find repeatedly in training materials. Withholding viewpoints outside of the mainstream only makes AI’s bias towards the mainstream even more pronounced. Should a bookseller save a Chicana anarchist work for a future human engaged in radical politics, or should she sacrifice it so that at least one AI model will have read a book on the subject? It’s hard to know the right answer.

I’m glad my inventory has evolved toward first editions and collectible books that the AI robots aren’t much interested in. I would be very conflicted about sending many of my books to be destroyed.

Like all gold rushes, this one will probably end soon. Eventually the big AI companies will have ingested everything they need, and I hope the quiet used and antiquarian book world will go back to normal. Many people are worried that AI will eliminate jobs. It won’t replace the idiosyncratic work flows of booksellers, at least not right away. It may well replace the jobs of the professionals who are the primary customers for our books. We may not be training the robots that will replace us, but we might be training the robots that replace our customers.