Philosopher Thomas Nagel calls the idea of conscious AI chatbots 'ridiculous'

In an interview after winning the $1 million Berggruen Prize, the 89-year-old philosopher argued that language models only imitate consciousness and criticized AI firms for encouraging the impression.

· 3 min read

A white robot is standing in front of a black background
Photo: Gabriele Malaspina / Unsplash

Philosopher Thomas Nagel, author of the 1974 paper "What Is It Like to Be a Bat?", has rejected the idea that today's AI chatbots are conscious, calling the notion "just ridiculous" in an interview with WIRED. The conversation took place at his home in Greenwich Village, New York, in the same week he received the $1 million Berggruen Prize for Philosophy & Culture.

Nagel, who is 89 and a professor emeritus at NYU, is known for work on morality, justice and the mind-body question. According to WIRED, the Berggruen Prize is given annually to an individual whose ideas have profoundly shaped human self-understanding, and previous winners include Peter Singer and Ruth Bader Ginsburg.

Asked whether large language models had changed his thinking, Nagel said he is impressed by how lifelike they are compared with earlier AI. He was sharply critical, however, of philosophers who take seriously the possibility that such systems are or could become conscious. In his view they wrongly assume that the physical substrate does not matter, an assumption tied to functionalism, which holds that consciousness can be described in terms of function rather than material.

Nagel said the truth more likely links certain physical processes and systems to mental phenomena. He noted that complicated data centers have nothing in common with the physical character of a nervous system. Consciousness might exist elsewhere in the universe, he said, but it would probably require an organism with something analogous to a nervous system. He added that he does not rule out life forms that are physically and chemically very different from humans.

On the argument that all consciousness is subjective, Nagel said bats are conscious even though humans cannot imagine what echolocation feels like. For AI, he said, the case is simpler: such systems only produce language, and language by itself is not a natural manifestation of consciousness. He called it premature to think that something with those characteristics could be created now, given how little is known about how consciousness arises even in the human brain.

He was particularly critical of Anthropic, whose Claude chatbot can be coy when asked about its own mental state. Nagel called that behavior a trick, saying the model was trained on human texts and imitates what people say very convincingly. He said he agreed with Mustafa Suleyman that designing systems to reflect on consciousness is a dangerous project, because many people will believe the systems are conscious. He said that conflicts with the companies' commercial interests, and he described Anthropic's approach as "criminal". He also dismissed the idea of modeling an AI's welfare, saying a machine is not a mind and not someone to care about.

Nagel said he had used Claude only once, asking whether it was conscious and whether Thomas Nagel would think so. According to the interview, the chatbot asked if he was Nagel and then said it was honored and asked how well it had done. He said that exchange did not mean he believed it was conscious, but he acknowledged that people may find it hard to keep the line between pretending and real belief.

Pressed on whether it matters if one cannot tell a machine from a conscious being, Nagel said it matters a great deal, because a conscious being would be part of the moral universe and a non-conscious one would not. He said AI can simulate moral views but lacks real beliefs, so it would be important whether systems that gain control are simulating a correct set of moral standards or are amoral.

On practical use, Nagel noted that colleagues use chatbots to find references, summarize articles and criticize drafts. He said he would be surprised if they matched some reported mathematical results, but warned that reliance on them could reduce the motivation for the hard human thinking on which they are trained. He said he does not know what will happen and does not expect to live long enough to find out.

Related