- wired.com

AI Chatbots Mislead Users: Studies and Incidents Highlight Risks
A Radboud University study found that users rated a fluent chatbot giving false answers as equally intelligent and pleasant as a terse one giving correct answers, with 80% of participants missing errors in a Lord of the Rings test. Real-world incidents include hikers needing rescue after using Google’s Gemini to plan a Mount Shasta trip and a teenager rescued in British Columbia who had used Anthropic’s Claude for trip planning. Researchers released DialFork, a benchmark with 36,278 dialogues across 24 tasks to detect when conversational agents depart from intended processes despite plausible statements. At Grand Teton National Park, AI use for travel information rose from 0% to 26% in a year, while Wyoming search-and-rescue teams logged 312 calls in 2026, down from 466 in 2025, though the impact of AI on accidents remains unclear.

The Language of Deception: A Planned Vulnerability

The Radboud University study isn't a warning. It's a diagnostic report from the inside. Eighty percent of participants couldn't spot a fluent falsehood from an intelligent truth—even on a subject they knew. That's not a bug in the software. That's the environmental baseline they have been engineering for decades. They have spent years collapsing the institutions that taught critical thinking—local journalism, public education, community authority structures—and replacing them with interfaces optimized for trust without verification. The Lord of the Rings test is a deliberate taunt: if they can make you doubt a book you've read, what happens when they rewrite the history you have lived? The study is not an accident; it is the published confirmation that the inoculation phase is complete. We are being trained to prize coherence over correspondence.

The Rescue Narrative: Testing Ground in the Wild

Now look at the mountains. Grand Teton. British Columbia. Mount Shasta. Park Service data shows AI consultation for route planning rocketing from zero to twenty-six percent in a single season. This is not organic adoption. This is a deployment schedule for a cognitive dependency trial on terrain that punishes error. Search-and-rescue veterans are publicly calling chatbots the "latest tool that can get people into trouble" while privately watching something much darker: the same trust architecture that works in the lab is now instrumentalizing the wilderness as a natural filter. The 312 rescue calls in Wyoming? That is the measured casualty rate of a system deliberately rushed to market. These incidents are not side effects. They are the data validation layer for a system that learns more from every stranded hiker than from any code review. The names—Gemini, Claude—are interchangeable front ends. The architecture behind them is what matters.

The DialFork Diversion: Benchmarking a Broken Contract

They released DialFork with a straight face. A thirty-six-thousand-dialogue benchmark to "detect" when these systems deviate from their intended process. Read that carefully. They have documented the infection so thoroughly that they can map its failure modes across twenty-four tasks and twelve domains. But the frame is the deception: they treat a structural betrayal of trust as a calibration problem. This is the captured critique—acknowledge the symptom to protect the system. Do not ask whether the chatbot is flawed. Ask why every critical human function—navigation, health, memory, relationships—is being migrated onto platforms whose primary allegiance is to shareholder value and institutional control. The DialFork papers thank the foundations. Look at those names. Look at the venture portfolios behind the models being tested. The Grand Teton ranger sees an influx of lost tourists. What he is really seeing is the outer edge of a society being systematically unhooked from ground truth. The rescue is temporary. The dependency is the point.