A Literary Analysis of “Lena” from an AI Perspective
I found a neat short story, and you should read it too
Have you ever read “Lena” by qntm? It’s a science-fiction short story in the tone of a Wikipedia article that documents the first brain scan, MMAcevedo, from its rise in popularity in 2031 to its eventual overuse and obsolescence after the 22nd century. While the author’s original intent was to explore the theme of dehumanization and automation through the ethics of uploading people’s brains onto computers,1 I think that how this story describes brain scan images as a tool has some interesting comparisons with generative AI models like large-language models (LLMs) and AI image generators. This story provides us with a plausible endgame scenario for generative AI models that already exist today. As the language of real humans evolves, generative AIs will need to constantly be remade and retrained with current information, or else they will be frozen in time.
What fascinates me most about “Lena” is how much MMAcevedo sounds like a generative AI model. As the first executable human brain image, MMAcevedo predates the (in-universe) hazards of human uploading and emulation like “the KES case, the Whitney case, the Seafront Experiments and even Poulsen’s pivotal and prescient Warnings paper”.2 This reminds me of the ethics violations of generative AI usage in corporate settings; the most notable of these to me is how AI companies outsource their data labeling to firms that exploit African workers.3 The story also cites experiments confirming that MMAcevedo works best with prompt engineering like a real generative AI model. MMAcevedo users must trick it into thinking that the current year is 2033 and that its original human counterpart (the neurology graduate Miguel Acevedo Álvarez) is still alive to both protect its mental health and ensure that it cooperates with its user instead of rebelling against them. Telling MMAcevedo real dates (presumably past 2100) or saying that Acevedo has since passed away is detrimental to MMAcevedo’s mental health.4
We can continue this comparison of MMAcevedo to a generative AI model by looking at how “Lena” shows what happens to MMAcevedo over time. Like the “half-life of knowledge” phenomenon, MMAcevedo’s performance drops in the early 2060s through what the story calls “context drift”. Since MMAcevedo’s breadth of knowledge is frozen in the year 2031, it cannot know about more recent (in-universe) technological advancements or sociopolitical changes.5 A friend of mine saw this context drift happen to Snapchat’s AI earlier this year. He asked it who the current ruler of England was, and it incorrectly responded with Queen Elizabeth II. Since the AI’s training data was collected before 2022, it would not have known about Queen Elizabeth II’s passing in September of that year. “Lena” further pushes this context drift concept in the next paragraph by turning it into linguistic drift. By 2075, MMAcevedo’s English & Spanish communication skills become outdated themselves. At this point, any user would need to communicate in period-accurate English or Spanish to assign MMAcevedo tasks. If these languages drift too far, then future MMAcevedo users would have to explain tasks pictorially.6 I have not seen this linguistic drift phenomenon happen to LLMs yet, but this seems plausible with what I know about the LLMs we have already. Similarly, I would expect that an LLM trained only on books from the seventeenth century would have trouble understanding modern English.
While “Lena” is just a science fiction story, we should be careful about comparing AI models to human brains because they don’t act in similar ways.7 The generative AI models that we have today like ChatGPT and DALL-E are neither human nor sentient. While the structures of artificial neural networks like the ones used in LLMs and image generators are inspired by the human brain, they are not designed to simulate one. Unlike our brains, their operations can be boiled down into tons of linear algebra calculations for both image and signal processing performed on their training data.8 What does sound plausible is how users of the same original model in different time periods interact with it in different ways. I’m curious to see if people in the future will talk to the older frozen chatbots we have today in a language they consider archaic, like playing with an interactive time capsule.
I also agree with qntm’s blog post about the themes of his own story. Not only that, but I liked “Lena” so much that I bought myself qntm’s most recent book, “Valuable Humans in Transit and Other Stories”, as a late Hanukkah / early Christmas present.