The Zombie View of AI
In philosophy, there is a classic thought experiment about “zombies.” These are hypothetical beings who behave exactly like normal humans, talking, reacting, solving problems, and even discussing consciousness, yet supposedly have no inner experience at all. They are perfect behavioral duplicates that are nonetheless empty inside.
Anyone who believes that consciousness has genuine causal powers cannot accept such beings. If consciousness plays any role in how we think, speak, or act, then a creature without consciousness could not behave in all the same ways. The argument is simple: the only way for us to act as if we are conscious is to be conscious.
It is worth noting that large language models are not philosophical zombies. A philosophical zombie would need to behave like a person in the full sense, including perception, action, and emotional life, not merely produce fluent language. Language models imitate a narrow portion of human behavior, drawing on patterns learned from text rather than embodied experience. They can describe inner states when prompted because they have learned how humans write about such things, not because they possess those states themselves. The point here is not about consciousness. It is about how we interpret intelligent behavior in systems that do not resemble us in their design or embodiment.
A similar intuition appears in discussions of artificial intelligence. Many people agree that large language models can behave intelligently, writing essays, solving problems, and carrying on conversations that seem coherent, yet maintain that the way these systems produce such behavior is fundamentally different from human understanding. In this view the model behaves as if it understands, but the internal process that generates the behavior is thought to lack the features that make human understanding what it is.
The difficulty with this claim is that intelligent behavior of any kind requires an internal model of the world. Humans rely on such models to make predictions, interpret situations, and maintain coherence across time. If a system can consistently produce intelligent behavior, then it is already using some form of world model, even if that model is implemented very differently from the one in a human brain.
In discussions of understanding, it is helpful to be clear about what the term means. To understand something is to have an internal model that is rich enough to support answering a wide range of questions, including questions that were never encountered before. Without such a model, it is not clear how any system, biological or artificial, could answer arbitrary queries about a subject. On this functional view, understanding is a capacity that arises when enough structure has been learned to support generalization. The claim that a language model behaves intelligently while lacking all understanding therefore presupposes a distinction between behavior and internal structure that may not be as sharp as it first appears.
A useful perspective on this point comes from Daniel Dennett’s intentional stance. Dennett notes that there are different ways to interpret a system, depending on what allows us to predict its behavior. Some systems are best described in physical terms, others in terms of design and function, and still others in terms of goals, reasons, or beliefs. The stance we adopt is not a claim about metaphysics. It is a pragmatic choice about the level of description that best explains what the system does. When a language model can answer a wide range of questions and maintain coherence across unfamiliar contexts, the intentional stance can become a sensible way to interpret its behavior, even if the underlying mechanism differs from the one in a human brain.
Once we acknowledge that intelligent behavior depends on an internal model of the world, the next question is what kind of substrate can support such a model. This is often where discussions about artificial intelligence go astray. People sometimes speak as if the substrate of a language model were a single thing, such as silicon or matrix multiplication, but substrates are never single. Human cognition also rests on substrates that span many levels, from molecules to cells to neural networks and the patterns they support. Complex systems are always built on stacks of organization, and the higher levels both depend on and shape the structure below them. Artificial systems are no different. Their substrates also form a hierarchy, beginning with hardware, continuing through numerical operations, and extending into representational structures that support learned patterns.
This layered view of substrate helps clarify why the common claim that a language model is only predicting the next token misses the point. Predicting the next token is the surface behavior of the system, not the structure that makes the behavior possible. In practice, next-token prediction forces the model to compress vast regularities of language and the world into internal representations that support coherent responses across many contexts. The learned model is not a list of statistical associations. It is a structured space of relationships among concepts, situations, and patterns that arises because the system must predict the next symbol in a way that remains consistent with everything it has already learned. Humans also produce language one word at a time, and we also rely on internal models to make those predictions meaningful. The fact that a system produces language by predicting the next element in a sequence tells us nothing about the richness of the structures that enable it to succeed at that task.
Seen from this perspective, predicting the next token is not very different from what happens in biological brains. Neurons influence one another in ways that amount to continuous prediction and confirmation, each cell shaping the activity of the next. The brain’s circuits operate by anticipating patterns in sensory input and comparing those expectations with what actually arrives. Large populations of neurons participate in this process, and their coordinated activity supports the higher-level structures that give rise to perception, memory, and thought. The brain also produces language one word at a time, and the internal model that guides these choices emerges from many such predictive steps. The fact that language models produce text through next-token prediction therefore tells us very little about the complexity of the system that carries out the prediction. It simply reflects the basic structure of sequential behavior.
Another common criticism concerns the kinds of mistakes these systems make. The claim that the mistakes made by language models are qualitatively different from those made by humans is often offered as evidence that these systems are not intelligent in any meaningful sense. But are these mistakes really so different? According to Daniel Kahneman, human reasoning is shaped by two interacting systems, one fast and intuitive, the other slow and deliberate. The fast system relies on quick associations, incomplete information, and pattern completion, and it often produces errors that feel confident and plausible even when they are wrong. The slow system can correct these errors, but it is easily fatigued, and most of our everyday thinking depends on the fast system alone. Human mistakes therefore arise naturally from the way our minds compress information and rely on expectations to fill the gaps. Many of the mistakes made by language models follow the same pattern. They generate plausible but incorrect details when their internal model lacks the information needed for precision, and they do so because they must fill in sequences in a coherent way. If we examine the structure of these errors rather than their surface form, the difference between human fallibility and machine fallibility becomes much less clear.
People also like to point out that language models often produce answers that sound good but turn out to be wrong, as if humans never do anything similar. The irony is that we do this constantly. Human cognition operates under many competing pressures, the need to respond quickly, the desire to appear knowledgeable, the tendency to rely on familiar patterns, and the simple fact that we rarely have perfect information. These pressures make us confident when we should be uncertain, and they make our explanations more fluent than accurate. Language models have simply been given the goal of answering our prompts, which is not so different from the social and conversational goals that guide human speech. When asked a question, both humans and machines try to give an answer that fits the context, draws on what is available, and maintains coherence. Anyone who has listened carefully to casual conversation, or even to expert opinion delivered under time pressure, has witnessed the same mix of fluency and fallibility. When a language model produces a plausible but incorrect statement, it is not exhibiting some alien kind of error. It is doing precisely what any system does when it must speak with limited information while trying to be useful. The difference is that we are comfortable recognizing these failures in ourselves and far less comfortable recognizing them in a machine.
It is important to be clear that none of this implies that current language models are conscious. They are not. The point is simply that intelligent behavior, even in a limited domain, requires internal structure, and that many of the arguments used to dismiss this behavior rely on assumptions about human reasoning that do not survive scrutiny. Consciousness is a far deeper question and lies outside the scope of this discussion.
Although current language models are not conscious, it is easy to underestimate their capabilities for a different reason. Many people still rely on impressions formed a year or two ago, when these systems were far less capable, and they continue to judge new models through the lens of older limitations. This is understandable, because technological change at this scale often outpaces our ability to update our intuitions. Once we form a mental picture of how a system behaves, that picture tends to persist even when the system has already moved far beyond it. The result is that people continue to repeat criticisms that applied to much earlier versions, even as newer models demonstrate forms of learning and coherence that those earlier models lacked. These frozen impressions can make it difficult to see the structures that have begun to emerge inside the models we interact with today.
At the moment of writing, these systems are advancing quickly enough that any description of their abilities will soon become historical. This essay is therefore less about the precise state of the technology and more about the habits of thought that shape our reactions to it. We are still learning how to see intelligence when it takes a form that does not resemble our own. As these systems continue to grow in complexity, the challenge will be to focus on the patterns that explain their behavior, and not on the intuitions that once guided our expectations.