In the novel Blindsight, Canadian biologist and writer Peter Watts advances a radical hypothesis: intelligence can be effective without consciousness. Nearly 20 years after the book\u2019s publication, the thesis neatly describes generative AI.<\/p>\n
We examine why \u201csmart\u201d is not the same as \u201cunderstanding\u201d, and the mistakes we make when we humanise algorithms.<\/p>\n
\u201cBlindsight\u201d appeared in October 2006. The novel was nominated for the Hugo Award in 2007 and was a finalist for the John W. Campbell and Locus awards.<\/p>\n
Its author is a marine biologist at the University of British Columbia with a doctorate in zoology and resource ecology. The novel cites more than 130 scientific papers, packaged in a familiar science\u2011fiction plot about first contact. In the 2000s the book remained niche, classed as hard SF and marked by a dense style and a bleak view of human nature. Critics noted its impenetrable prose and emotional chill.<\/p>\n
The book\u2019s core idea separates two notions often conflated: intelligence as the capacity to solve problems and process information, and consciousness as subjective awareness\u2014what it is like to be something, as formulated<\/a> by philosopher Thomas Nagel.<\/p>\n Watts advances a provocative hypothesis: consciousness is an evolutionarily superfluous trait, a by-product rather than a prerequisite of intelligence.<\/p>\n The novel probes this intuition through several threads. The scramblers\u2014aliens aboard the ship Rorschach\u2014possess intelligence orders of magnitude above human. They analyse the crew\u2019s neural activity and solve fearsomely complex tasks. Yet they have no subjective experience. They do not know they exist. As Watts has a character put it:<\/p>\n \u201cImagine that you are a scrambler. Imagine that you have a mind but no reason, have tasks but no consciousness. Your nerves hum with programmes of survival and self-preservation, flexible, self-governing, even technological\u2014yet there is no system that would keep an eye on them. You can think of anything at all, but you are conscious of nothing\u201d*<\/span>.<\/em><\/p><\/blockquote>\n The protagonist and narrator, Siri Keeton, underwent a hemispherectomy<\/span> in childhood to treat epilepsy. He can precisely model other people\u2019s behaviour but lacks empathy and genuine emotional experience. His role is that of a synthesist, a translator of complex data for Mission Control: he transforms information without taking a view on it. Keeton himself admits:<\/p>\n \u201cIt is not my job to understand. For a start, if I could understand them, they would not be especially advanced achievements. I am just, how shall I put it\u2014a conduit.\u201d<\/em><\/p><\/blockquote>\n The third thread is the vampire Jukka Sarasti, a genetically resurrected Pleistocene predator with an intellect beyond human. Vampires can see both sides of the Necker cube<\/a> at once\u2014running several cognitive models in parallel.<\/p>\n Each of these figures rests on real philosophical ground. The notion of philosophical zombies, introduced<\/a> by Robert Kirk in 1974 and popularised by David Chalmers<\/a> in \u201cThe Conscious Mind<\/a>\u201d (1996), describes a hypothetical being physically identical to a human yet devoid of subjective experience. The scramblers radicalise the idea: not a human copy without consciousness, but a fundamentally different form of intelligence.<\/p>\n In 1995 Chalmers formulated the \u201chard problem\u201d of consciousness: why do physical processes in the brain give rise to subjective experience? Even if we fully explain all cognitive functions\u2014attention, categorisation, information processing\u2014the question remains: why are they accompanied by feeling? \u201cBlindsight\u201d flips the problem: what if the answer is \u201cno feeling is needed\u201d?<\/p>\n Watts described<\/a> the idea\u2019s genesis thus: he long sought a functional account of consciousness and applied the same test to each proposed function\u2014could an unconscious system do the same? The answer was always \u201cyes\u201d. He concluded the stronger claim was the absence of function altogether. In the novel\u2019s afterword, Watts sums up: in everyday conditions consciousness has little to do beyond \u201ctaking memos from a much more quick\u2011witted subconscious layer, rubber\u2011stamping them and attributing all the credit to itself\u201d.<\/p>\n Long before Watts, Norwegian philosopher Peter Wessel Zapffe had framed consciousness as an evolutionary overdose. In his 1933 essay \u201cThe Last Messiah<\/a>\u201d he compared the human mind to how \u201csome deer in palaeontological times\u201d died out from \u201cexcessively heavy antlers\u201d. For Zapffe, consciousness is a similar surplus: a capacity that outgrew practical need and turned from an advantage into a burden.<\/p>\n But where Watts argues that consciousness is unnecessary for intelligence, the Norwegian thinker is more radical: it is not merely redundant but destructive. People, he thought, must \u201cartificially limit the content of consciousness\u201d to avoid falling into \u201ccosmic panic\u201d at the awareness of their own finitude.<\/p>\n Philosopher David Rosenthal reached a similar conclusion. In a 2008 article<\/a> he showed that the consciousness of mental states\u00a0adds no significant function over and above the processes that generate it.<\/p>\n In 1980 philosopher John Searle published the now\u2011famous thought experiment \u201cThe Chinese Room\u201d. Someone who knows no Chinese sits in a sealed room with a rulebook for manipulating characters. Receiving questions in Chinese, he assembles answers by the rules. An observer is convinced the person inside understands Chinese. But he does not understand a word. Searle\u2019s conclusion: syntax is not semantics. Correct symbol manipulation does not amount to grasping meaning.<\/p>\n The experiment is woven directly into \u201cBlindsight\u201d. When the crew of the Theseus makes contact with Rorschach, the alien ship replies in idiomatic English. At first it seems a breakthrough\u2014communication with an extraterrestrial intelligence. But linguist Susan James gradually realises: Rorschach learned English by intercepting human radio traffic. It gathers and recombines linguistic patterns. It produces grammatically and contextually apt answers. But it does not understand what it is saying.<\/p>\n Watts lays out the point through Keeton\u2019s explanation:<\/p>\n \u201cThe point is that you can converse using the simplest algorithms of comparative analysis and have not the slightest idea what you are saying. If you use a sufficiently detailed set of rules, you can pass the Turing test. Gain a reputation for wit and banter without even knowing the language you are using.\u201d<\/em><\/p><\/blockquote>\n If a LLM<\/span> is a Chinese room, why do millions of people behave as if a sentient being sits behind the interface? The answer lies in cognitive biases shaped by evolution.<\/p>\n In 1966 Joseph Weizenbaum, a pioneer of artificial intelligence at MIT<\/span>, created ELIZA, a program that used simple pattern matching to imitate a psychotherapist. It rephrased the user\u2019s utterances as questions. The effect astonished its creator: his assistant, who had watched the development, asked to be left alone with ELIZA after a few minutes of interaction. Weizenbaum later wrote<\/a>:<\/p>\n \u201cI had not imagined that extremely short interaction with a relatively simple program could induce powerful delusional thinking in perfectly normal people.\u201d<\/em><\/p><\/blockquote>\n This phenomenon became known as the \u201cEliza effect\u201d\u2014the tendency to ascribe understanding to computer systems that do not possess it. The effect persists even when the user knows a program is just that.<\/p>\nThe conscious surplus<\/h2>\n
Eliza in the Chinese room<\/h2>\n