Now that we’ve all had experience with large language models, their limitations are all too visible. Yes, they can write. But their prose doesn’t explode in the mind like the words of Jennifer Egan, Emily St. John Mandel, or David Foster Wallace do. Yes they can make music. But Taylor Swift and Kendrick Lamar are sleeping very well at night. And they sure can summarize history speedily and neatly, but not with the perspicacity of Barbara Tuchman or Ron Chernow. LLMs are just getting started, though. They’re bound to get better.
But how much better? We’ve seen multiple instances where AI has conquered regions thought to be impenetrable by robots, from chess to the LSAT. Could it do the same in the creative arts? Lately I’ve been obsessed with that question. Can something generated with AI be truly brilliant?
Given that, I was easy pickings for a PR release I received last week. The subject line: “AI book reviewers?” The press release touted “the first book to ever be reviewed by AI avatars.” I’ve written book reviews, and my books have been reviewed, and I know that the best examples of that art are penetrating and illuminating. Had someone cracked the code to do that via AI?
So I followed up, and I found that the touted “book reviews” weren’t quite what I had in mind. It turns out that a PR firm was promoting a novel called The Cloud, one of those dystopian scenarios where a heroic human fights back against AI overlords, and the firm had come up with some AI gimmickry. It created a set of AI-generated characters—a rogue android, a time traveler, etc.—and had those characters offer assessments of the book.
Sounds great, until you hear the details. The two- or three-sentence, ChatGPT-generated “book reviews” only drew on the jacket copy and some Amazon reviews. The avatars did not get access to the actual text of the book, “for copyright reasons.” Since most reviewers prefer to actually put eyes on the pages of the book they’re judging, these AI-generated comments would seem to not qualify as actual reviews. Indeed, they come off like promotional blurbs dashed off by a friend of a friend who hadn’t bothered to read the book or even ask what it was about. One typical example: “As a survivor of the post-apocalyptic empire, I can tell you that The Cloud by Robert Rivenbark is an enthralling read. With its gripping storytelling and memorable characters, I highly recommend this book.” Not one of these comments invokes the name of a single character, the theme of the book, the quality of the prose, or a glimmer that a mind was at work. Henry Roi, one of the PR execs in charge of the promotion, concedes that most authors do want actual humans to review their work. “But afterwards they want more content,” he adds, presumably for social media posts on TikTok and elsewhere. AI-generated content fits that bill.
So much for the avatars. Still, the experience got me thinking about whether it might be possible in the future for AI to create real book reviews. To get a baseline on the state of the art, I asked ChatGPT to review my own first book, Hackers. Since the book has been around for a while, there’s plenty of material for a large language model to chew on. Maybe ChatGPT could compensate for that lousy review Hackers got in the Sunday New York Times Book Review! (The Washington Post review was better.) ChatGPT prefaced its work with a disclaimer that, as an AI model, it didn’t have opinions or emotions. Nonetheless, it dove into the breach. The first paragraph read:
“Hackers: Heroes of the Computer Revolution” by Steven Levy is a classic exploration of the early days of the computer industry and the pioneering individuals who shaped its development. Originally published in 1984, the book provides an insightful and engaging look into the world of computer enthusiasts, tinkerers, and programmers who were instrumental in laying the foundation for modern computing and internet culture.
Not bad. But some people have told me they loved Hackers. Those fans might think that review was insufficiently enthusiastic. So for those loyal readers—not for me, you understand—I prompted ChatGPT to be more positive. It complied within seconds.
Source
