Within Narrow vs AGI
Why sounding human is not enough
People can read understanding into conversational software when style, empathy, and social cues make replies feel human.
On this page
- What ELIZA showed about human expectations
- What short Turing style chats can and cannot test
- Why social cues shape judgments of intelligence
Page outline Jump by section
Introduction
Modern chatbots often feel intelligent because conversation is one of the main ways humans judge minds. Long before large language models, two influential ideas—the ELIZA chatbot and the Turing test—showed that people can mistake convincing conversation for deeper understanding. Together, they reveal a central challenge in understanding artificial intelligence: a system may sound human, appear empathetic, or even fool people in brief exchanges without possessing human-like comprehension. The lesson is not that chatbots are useless or deceptive by definition, but that conversational fluency and genuine understanding are not the same thing. [Wikipedia]WikipediaOpen source on wikipedia.org.
Why sounding human is not enough
What ELIZA showed about human expectations
In 1966, computer scientist Joseph Weizenbaum created ELIZA, a simple conversational program that became famous for imitating a psychotherapist. Rather than understanding users’ problems, ELIZA mainly detected keywords and transformed statements into questions or reflective responses. For example, if a user mentioned feeling unhappy, ELIZA might reply by asking why they felt unhappy. The illusion of understanding came from conversational technique rather than deep knowledge. [Wikipedia+2Wikipedia]WikipediaOpen source on wikipedia.org.
What surprised Weizenbaum was not the program itself but people’s reactions to it. Many users quickly attributed empathy, insight, and genuine understanding to a system that possessed none of those qualities. Some wanted private conversations with the program, despite knowing it was software. Weizenbaum later reflected that even brief exposure to a simple program could trigger powerful misconceptions about machine understanding. [Wikipedia+2Weizenbaum Institut]WikipediaOpen source on wikipedia.org.
This tendency became known as the ELIZA effect: the human habit of reading more intelligence, comprehension, or emotional awareness into computer-generated language than the evidence justifies. Researchers continue to use the term because it captures a recurring pattern that extends far beyond the original chatbot. [Wikipedia]WikipediaELIZA effectELIZA effect
The importance of ELIZA is therefore not primarily technical. Modern chatbots are vastly more capable than ELIZA. The enduring lesson is psychological: humans are predisposed to treat coherent language and social responsiveness as signs of an inner mind. [Springer Link]link.springer.comSpringer Linkmaking sense of human–computer interaction—Garfinkel's…by C Eisenmann · 2024 · Cited by 47 — Although the ELIZA effect ha…
What short Turing-style chats can and cannot test
The second influential idea comes from Alan Turing’s 1950 proposal for what became known as the Turing test. In its classic form, a human judge communicates through text with both a human and a machine. If the judge cannot reliably tell which is which, the machine has succeeded in the test. [Wikipedia]WikipediaTuring testTuring test
The Turing test remains historically important because it shifted attention away from abstract questions such as “Can machines think?” and toward observable behaviour. Instead of demanding access to a machine’s internal processes, it asked whether the machine could participate in conversation in a human-like way. [Wikipedia]WikipediaTuring testTuring test
Yet the test also has important limits. A machine can perform well by imitating conversational habits, social conventions, and even human mistakes. Passing a short conversation does not automatically demonstrate reasoning ability, self-awareness, long-term planning, or broad understanding of the world. Critics have long argued that the test measures human likeness in conversation more than intelligence itself. [Wikipedia+2Learning From Examples]WikipediaTuring testTuring test
Recent large language models have renewed these debates. Some modern systems have achieved impressive results in Turing-style evaluations, occasionally being mistaken for humans at high rates. However, researchers and commentators continue to caution that success in conversational imitation should not be confused with proof of general intelligence. Language competence is significant, but it is only one component of intelligence. [Psychology Today+2The Week]psychologytoday.comPsychology Today AI's Turing Test MomentPsychology TodayAI's Turing Test MomentMay 17, 2024 — GPT-4's Turing test performance suggests that AI language models are evolving faste…
Why social cues shape judgments of intelligence
One reason ELIZA and the Turing test remain relevant is that both reveal how strongly social cues influence human judgement.
People do not evaluate intelligence solely through logic or factual accuracy. They also respond to:
- Politeness and conversational flow.
- Expressions of empathy or concern.
- Personalised references to earlier remarks.
- Humour, emotion, and storytelling.
- Signals of confidence and social awareness.
When a chatbot displays these behaviours, users often infer understanding even when the system is relying on statistical patterns or scripted responses. The impression of intelligence emerges partly from the machine’s output and partly from the user’s interpretation. [Wikipedia+2Springer Link]WikipediaELIZA effectELIZA effect
This helps explain why conversational systems can be simultaneously impressive and misleading. A chatbot may produce helpful explanations, answer questions effectively, and maintain a coherent dialogue while still lacking the kind of grounded understanding people naturally assume is present. The more human-like the interaction becomes, the easier it is to overlook that distinction. [PMC]pmc.ncbi.nlm.nih.govPMCConversational Agents: Goals, Technologies, Visionby M Allouch · 2021 · Cited by 257 — The two main abilities required of CAs are the ability to logically understand the user's utteran…
The phenomenon has practical consequences. Researchers, ethicists, and policymakers increasingly discuss how emotionally persuasive chatbots can encourage trust, attachment, or overconfidence in their advice. These concerns are modern versions of the same pattern first observed with ELIZA nearly sixty years ago. [Financial Times+2TechRadar]ft.comFinancial Times AI companions are not your child's friendIt traces the history of our emotional connection to AI, stemming from the 1960s "Eliza effect," and highlights how modern AI chatbots’ i…
The lasting lesson for understanding AI
ELIZA and the Turing test remain valuable because they expose a gap between appearance and capability. ELIZA showed that people readily project understanding onto conversational software. The Turing test demonstrated that human-like dialogue can be a meaningful benchmark while also raising questions about what such performance actually proves. [Wikipedia+2Wikipedia]WikipediaOpen source on wikipedia.org.
For understanding artificial intelligence, the combined lesson is straightforward: conversation can reveal important capabilities, but it is not a complete measure of intelligence. A chatbot that sounds human may possess remarkable language skills, yet the critical question is not whether it feels intelligent for a few minutes. The deeper question is what it truly understands, what it can reliably do, and where its apparent competence breaks down. [Psychology Today+2Wikipedia]psychologytoday.comPsychology Today AI's Turing Test MomentPsychology TodayAI's Turing Test MomentMay 17, 2024 — GPT-4's Turing test performance suggests that AI language models are evolving faste…
Amazon book picks
Further Reading
Books and field guides related to Why sounding human is not enough. Use these as the next step if you want deeper reading beyond the article.
Computer Power and Human Reason
Directly reflects on ELIZA and misconceptions about machine understanding.
eBay marketplace picks
Marketplace Samples
Example marketplace items related to this page. Use the search link to explore similar finds on eBay.
Endnotes
-
Source: Wikipedia
Link: https://en.wikipedia.org/wiki/ELIZA -
Source: Wikipedia
Title: Turing test
Link: https://en.wikipedia.org/wiki/Turing_test -
Source: Wikipedia
Title: ELIZA effect
Link: https://en.wikipedia.org/wiki/ELIZA_effect -
Source: weizenbaum-institut.de
Link: https://www.weizenbaum-institut.de/w-100/exhibition-eliza/Source snippet
ELIZASome participants not only developed a close relationship with ELIZA but also attributed empathy to the program. Even some practicin...
-
Source: ojs.weizenbaum-institut.de
Link: https://ojs.weizenbaum-institut.de/index.php/wjds/article/view/106/96Source snippet
Limits of Computationby DM Berry · 2023 · Cited by 94 — By developing the program ELIZA and the DOCTOR script, Weizenbaum revealed that c...
-
Source: link.springer.com
Link: https://link.springer.com/article/10.1007/s00146-023-01793-zSource snippet
Springer Linkmaking sense of human–computer interaction—Garfinkel's...by C Eisenmann · 2024 · Cited by 47 — Although the ELIZA effect ha...
-
Source: pmc.ncbi.nlm.nih.gov
Title: PMCConversational Agents: Goals, Technologies, Vision
Link: https://pmc.ncbi.nlm.nih.gov/articles/PMC8704682/Source snippet
by M Allouch · 2021 · Cited by 257 — The two main abilities required of CAs are the ability to logically understand the user's utteran...
-
Source: techradar.com
Title: Tech Radar I looked into how AI chatbots respond to emotions
Link: https://www.techradar.com/ai-platforms-assistants/i-looked-into-how-ai-chatbots-respond-to-emotions-and-what-i-found-out-about-the-eliza-effect-completely-changed-how-i-think-about-using-themSource snippet
Formerly, tech sought user attention; now, AI systems exploit emotional bonds through personalized, empathetic interactions. This is refe...
-
Source: ojs.weizenbaum-institut.de
Link: https://ojs.weizenbaum-institut.de/index.php/wjds/article/view/136/92Source snippet
Joseph Weizenbaum to ChatGPTby C Floyd · 2023 · Cited by 16 — Weizenbaum, J. (1966). ELIZA – A computer program for the study of natural...
-
Source: jw.weizenbaum-institut.de
Title: de Die Wandlung zum Skeptiker
Link: https://jw.weizenbaum-institut.de/wp03Source snippet
Mit Hilfe eines Skripts ermöglicht es eine Gesprächsinteraktion zwischen Mensch und...Read more...
-
Source: eliza.com
Title: Chat GPT Enterprise & Agentic Systems
Link: https://eliza.com/Source snippet
Enterprise AI Consulting & Implementation | OpenAI...Eliza embeds forward deployed engineers into your organization, builds prod...
-
Source: learningfromexamples.com
Link: https://www.learningfromexamples.com/p/the-turing-test-doesnt-measure-intelligenceSource snippet
Learning From ExamplesThe Turing test doesn't measure intelligenceJul 24, 2025 — The test wasn't designed to answer the question of wheth...
-
Source: psychologytoday.com
Title: Psychology Today AI’s Turing Test Moment
Link: https://www.psychologytoday.com/ca/blog/the-digital-self/202405/ais-turing-test-momentSource snippet
Psychology TodayAI's Turing Test MomentMay 17, 2024 — GPT-4's Turing test performance suggests that AI language models are evolving faste...
Published: May 17, 2024
-
Source: theweek.com
Title: ai llms pass turing test
Link: https://theweek.com/tech/ai-llms-pass-turing-testSource snippet
In evaluations, GPT-4.5 was mistaken for a human 73% of the time, while Llama-3.1-405B achieved a 56% rate, both significantly outperform...
-
Source: ft.com
Title: Financial Times AI companions are not your child’s friend
Link: https://www.ft.com/content/686eaf73-a574-47db-afcb-7989d66783f5Source snippet
It traces the history of our emotional connection to AI, stemming from the 1960s "Eliza effect," and highlights how modern AI chatbots’ i...
-
Source: pmc.ncbi.nlm.nih.gov
Link: https://pmc.ncbi.nlm.nih.gov/articles/PMC4867147/Source snippet
the Turing Test Does Not Mean the End of Humanityby K Warwick · 2015 · Cited by 46 — Passing the Turing test has no relationship with hum...
-
Source: pmc.ncbi.nlm.nih.gov
Link: https://pmc.ncbi.nlm.nih.gov/articles/PMC10907317/Source snippet
We find that the chatbots'...Read m...
-
Source: elizalovechild.com
Link: https://www.elizalovechild.com/Source snippet
Official WebsiteThe latest music, videos, and merch from Eliza...
Additional References
-
Source: arxiv.org
Link: https://arxiv.org/html/2406.17650v2Source snippet
ELIZA Reinterpreted: The world's first chatbot was not...ELIZA was intended to simulate—or caricature, as Weizenbaum himself suggests—th...
-
Source: medium.com
Link: https://medium.com/%40adnanmasood/closing-the-eval-deployment-gap-in-ai-systems-discrepancy-between-benchmark-performance-and-d27c33361b93Source snippet
Closing the Eval–Deployment Gap in AI SystemsThere's also a need for theory on evaluating general intelligence or agency — something like...
-
Source: spacedaily.com
Link: https://spacedaily.com/n-sixty-years-before-chatgpt-a-chatbot-called-eliza-was-already-making-people-pour-their-secrets-into-a-computer-and-its-creator-considered-that-a-catastrophe-not-a-success/Source snippet
Sixty years before ChatGPT, a chatbot called ELIZA was...2 days ago — Sixty years before ChatGPT, a chatbot called ELIZA was already mak...
-
Source: botpress.com
Link: https://botpress.com/blog/turing-testSource snippet
Turing TestThe Turing Test is an AI test to see whether, through a chat conversation, a computer can convince a human that it is human.Re...
-
Source: medium.com
Link: https://medium.com/%40metaform3d/chat-gpt-does-not-pass-the-turing-test-5299bfb4a9f0 -
Source: linkedin.com
Link: https://www.linkedin.com/top-content/user-experience/user-experience-and-emotional-[engagement -
Source: theguardian.com
Link: https://www.theguardian.com/technology/2023/jul/25/joseph-weizenbaum-inventor-eliza-chatbot-turned-against-artificial-intelligence-aiSource snippet
The GuardianWeizenbaum's nightmares: how the inventor of the first...25 Jul 2023 — Weizenbaum had stumbled across the computerised versi...
-
Source: spectrum.ieee.org
Title: why people demanded privacy to confide in the worlds first chatbot
Link: https://spectrum.ieee.org/why-people-demanded-privacy-to-confide-in-the-worlds-first-chatbotSource snippet
People Demanded Privacy to Confide in the World's...Weizenbaum called this the Eliza effect, and believed it was a type of “delusional t...
-
Source: elizaos.ai
Link: https://elizaos.ai/Source snippet
Home | ElizaThe next evolution of software — building systems that don't just execute, they co-create. Join Community. AI Influencer. Cod...
-
Source: medium.com
Title: eliza 1966 the first chatbot in history that fooled everyone b9d910c63919
Link: https://medium.com/%40himash0009/eliza-1966-the-first-chatbot-in-history-that-fooled-everyone-b9d910c63919Source snippet
What shocked Weizenbaum was how quickly people formed emotional connections with ELIZA. Users would pour...Read more...
Topic Tree