'I propose to consider the question, "Can machines think?"' With that question, Alan Turing began Computing Machinery and Intelligence. The article from 1950 is likely the first to address the subject of Artificial Intelligence, or AI, in such a way that machines were viewed in a completely new light. How? With his Imitation Game, or the Turing Test.
The original Turing Test is a text-based interaction between a human evaluator, a human, and a machine. If the evaluator cannot reliably determine which of the two is the machine, then the test is considered 'passed'.
The Turing Test became a milestone in the history of AI. The general view of machines was shattered, and the question of whether computers can think was never the same again.
The Turing Test in a New Light
With the rise of large language models (LLMs) like ChatGPT, Claude, Gemini, and Mistral, the Turing Test has suddenly become relevant again. These AI systems are capable of having conversations that, especially on the surface, are hardly distinguishable from human communication. They answer questions, increasingly recognize and make jokes, understand context, and can even come across as empathetic. Thus, they seem to pass the classic Turing Test with flying colors. In fact, earlier this year, a LLM convincingly passed the test.
This creates a new dilemma. Because if AI comes across as so human-like, while it still does not truly 'understand' what it is saying, what does that say about the Turing Test? Concerns about 'pseudo-intelligence' - systems that seem smart but lack consciousness or understanding - are widely shared among AI researchers. Instead of actually defining whether you are talking to a machine, the Turing Test now primarily measures how convincingly a model can imitate human language behavior.
Moreover, many conversations with LLMs are no longer comparable to the original test setup. Where Turing envisioned a strictly defined setting with multiple participants and a clear timeframe, AI chats are often conducted one-on-one, and the evaluator, for example through prompt bias, influences the answers. The context has changed, and so has the value of the outcome.
Turing Test vs. AI: How (In)Suitable Is It?
The core criticism of the Turing Test is that it has become too successful. Or rather: too easy to manipulate. AI systems today are trained on vast amounts of human language, allowing them to effortlessly reproduce patterns, formulations, and interaction styles. This leads to convincing output, at first glance. In longer, more intense, and 'personal' interrogations, it becomes increasingly clear that you are talking to an AI model.
Currently, instead of the Turing Test, other benchmarks for AI are being used, such as:
- Winograd Schema Challenge, which also addresses where the Turing Test falls short. This challenge tests whether an AI can correctly interpret sentences with subtle semantic nuances.
- ARC (Abstraction and Reasoning Corpus), which focuses on 'fluid intelligence' by giving AI tasks that require little to no prior knowledge from humans.
- Theory of Mind evaluations have long been used in psychology to assess how much someone can empathize with others. For AI, this is still challenging.
These alternatives look at AI from a more human perspective and focus more on less obvious interaction points. Where a bell might ring for humans, this does not necessarily resonate with AI.
A Moral and Philosophical Compass
All of this does not mean that the Turing Test is outdated. Just ask yourself: is it morally acceptable for an AI model to appear so human-like that it cannot be discovered to be a machine within a certain timeframe? Yes, there are options to determine whether AI is AI, but these options should not become too difficult in themselves.
For modern AI systems, the most important test is not whether they appear human, but whether they are reliable, explainable, and safe. In that sense, the Turing Test has given way to more robust evaluation frameworks. But the original philosophical value remains: we must always, now more than ever, continue to ask ourselves: 'Can machines think?'
