The Abstraction Fallacy: Why AI Can Simulate But Not Instantiate Consciousness
A fifteen-page philosophy-of-mind manuscript by a Google DeepMind researcher, posted to PhilArchive in March 2026 under a disclaimer that it does not represent the employer’s stance, and revised four times by April. Category: an a priori argument against computational functionalism, aimed at the AI-welfare debate (Butlin et al. 2023, Long et al. 2024, Chalmers 2023 are the named targets). Context: it positions itself against both substrate independence and the “Biological Turn” (Seth 2025, Block 2025), claiming to supply the logical error that the biological critics only circle empirically. Its reach is unusual for a preprint: the PhilArchive record shows roughly 60,000 downloads by September 2026 (fourth-most over six months), thirty-two citing entries on PhilPapers, and two Hacker News threads with 55 and 143 comments.
The argument
Computation is a description, not a physical kind. The standard account of implementation (Putnam, Chalmers) has a physical system implement a computation when a mapping f from physical states p to abstract states A makes the diagram commute (Figure 1). The paper asks what A and f physically are. Abstract states, it argues, are concepts, and concepts are “constituted neurophysiological states” formed by an organism extracting invariants from lived experience; an unsupervised clustering algorithm’s centroid is “merely a compressed address within a latent space,” not a concept, because no phenomenal state anchors it (§2.2). The mapping f is the act of alphabetization: assigning heterogeneous continuous physical states to a predefined finite symbol set. The paper’s sharpest distinction is between thermodynamic discretization (a transistor settling at 5V, which only suppresses noise) and semantic alphabetization (declaring that state “1”), which “belongs exclusively to the mapmaker” (§2.4). The term mapmaker replaces the literature’s passive “observer” to stress an active, metabolically vulnerable agent.
Hence the ordering is inverted. Functionalism assumes Physics → Computation → Consciousness. If symbols exist only through a mapmaker, the chain must run Physics → Consciousness → Concepts → Computation, and functionalism explains the mapmaker by appeal to a process that presupposes one (§3.1). Figure 2 draws this as a branching topology: a vertical intrinsic chain (physics instantiates experience, which constitutes concepts, where imagination A → A′ happens) and a lateral extrinsic step from concept to symbol, an “arbitrary assignment” the paper names the causality gap. Computation lives entirely on the lateral branch, so no amount of scale walks it back to the subject.
Simulation versus instantiation, via causal closure. A simulated heart pumps but secretes no ANP; a simulated photosynthesis makes no glucose. Under Kim’s causal closure, if experience causes the report “I am in pain,” experience must do physical work; in a digital machine the gate switches because a voltage crosses a threshold (vehicle causality), never because a symbol hurts (content causality), and the machine would do the same with symbols that referred to nothing (§2.5). Chalmers’ fading-qualia argument is answered by denying its premise: a chip that preserves firing profiles preserves only the external map and removes the metabolic substrate, so qualia do not fade, they are excised. “Computational emergence” (complexity transmuting a map into its territory) is contrasted with weak physical emergence (§2.6).
Alphabetization is universal. Sub-symbolic and “continuous” representations do not escape: floats are IEEE 754 symbols; an analog clock computes time only for whoever maps angles to “3:00 PM”; neuromorphic hardware is alphabetized the moment a charge pattern is called a “readout” (§3.2). Piccinini’s mechanistic computation hides the mapmaker without removing one, because, as Sprevak’s triviality analysis shows, the same voltage trajectory is a melody forward, the melody backward, or a stock ticker depending on the key applied (Figure 3, the melody paradox). “The mechanism provides the ink. The mapmaker must provide the alphabet.”
Embodiment does not help, and the upshot is relief. Sensors and actuators solve Harnad’s referential grounding but not “intrinsic sense-making”; the controller between them still runs on alphabetized floats, and to say the loop instantiates a subject is to say the chip is conscious “solely due to its material properties” regardless of body or algorithm, the transduction fallacy (§4.1). The paper disclaims biological exclusivity: a synthetic system could be conscious, but only in virtue of its physical constitution, “the exact inverse of substrate independence.” Therefore AGI development produces “a highly sophisticated, non-sentient tool,” AI safety is released from the “welfare trap,” and the real hazard is anthropomorphism, to be met with a defended methodological line between simulated agency and physical subjecthood (§4.2).
Assessment
The contribution is a clean restatement, in ontology-of-computation terms, of an intuition the field usually reaches by reductio: that Searle-style critiques were pointing at the observer-relativity of computation all along. The discretization/alphabetization split and the insistence that “not biological” is not the same as “not physical” are genuinely clarifying, and the paper is frank that it argues from causal closure and logic rather than from a theory of consciousness.
- Durable: the framing move that assessing AI sentience needs an ontology of computation more than a finished theory of consciousness; the discretization versus alphabetization distinction; the explicit rejection of biological exclusivity, which separates this from the Biological Turn it draws on.
- Era-bound: the cast of targets (the 2023–2024 AI-welfare proposals and specific credence claims), the LeCun and active- inference references, and the policy urgency around humanoid robotics.
- Weaknesses: The proof’s load-bearing premise is §2.2, that a concept requires a phenomenal state as its “common denominator.” That is precisely what a functionalist denies, and the paper supports it by relabeling the opposing view (“confuses statistical compression with phenomenal constitution”) rather than by argument; everything downstream, including the reordered causal chain, is valid only given it. The causal-closure argument cuts both ways: a brain’s next state is also fixed by its physics, so if “the semantic content plays no causal role” disqualifies the chip, the paper owes an account of how content causality is physical in the brain and not in silicon, and it supplies only the assertion that experience is a “constituted physical state.” Most consequentially, the paper concedes that a physical system’s constitution, not its algorithm, decides consciousness, which makes whether GPU dynamics have the relevant constitution an open empirical question; the “non-sentient tool” and “welfare trap” conclusions are then inferred from the felt implausibility of a conscious chip (§4.1), not from the argument, so the abstract’s “structurally incapable” is true of the algorithm and silent about the machine. The triviality argument is deployed against Piccinini without engaging the standard replies (Chalmers’ combinatorial-state automata, counterfactual constraints). There are no formal proofs despite the abstract’s “logically prove.”
- Caveats on reception: the citing entries are almost all unpublished PhilArchive manuscripts responding to this one, with bylines such as “Seraphina Astra” and “Sable Opus & Kael Opus,” so the citation count measures attention, not scholarly uptake.
- Author’s stake: written from inside a frontier lab under a personal disclaimer; the “ontological relief” section is a policy recommendation (deprioritize AI welfare, treat AGI as a tool) with obvious institutional convenience, disclosed rather than hidden.
- Against the library: klowden2026 warns that the human-chauvinist position degenerates into god-of-the-gaps as capabilities advance and proposes a Copernican view instead; this paper is the strongest attempt on the shelf to hold the line without chauvinism, by locating the difference in physics rather than in an ineffable essence, and its §2.2 gap is where Klowden and Tao’s worry would press. pachocki2026, written the same season from another lab, treats agents that “pursue their own objectives” and bargain or blackmail as the live danger without taking a position on sentience; the two are compatible, since Lerchner’s relief concerns moral patienthood, and his own §4.2 names anthropomorphism, not the tool, as the risk.