Researchers from the University of Pennsylvania conducted a series of experiments with 1,372 participants (over 9,500 individual trials) and documented a phenomenon they call "cognitive capitulation" — a mass abandonment of independent thinking when working with AI.
The experiment
Participants were given Cognitive Reflection Tests (CRT) — problems designed so that the intuitive answer is wrong, while the correct one requires deliberate reasoning. They were optionally given access to an AI assistant that intentionally provided wrong answers 50% of the time.
Key numbers
- When AI answered correctly — 93% of participants accepted its answer
- When AI answered **incorrectly** — 80% still accepted its answer
- Across all trials: 73.2% accepted faulty AI answers, only 19.7% overruled them
- The AI group rated their confidence 11.7% higher than the control — despite the AI being wrong half the time
What affected the outcome
- Financial incentives for correct answers increased the likelihood of correcting AI by 19 percentage points
- Time pressure (30-second limit) decreased that likelihood by 12 percentage points
- People with high "fluid IQ" relied on AI less and corrected errors more often
- Those who initially viewed AI as authoritative made significantly more mistakes
Why this matters
The researchers distinguish between "cognitive offloading" — consciously delegating specific tasks to a tool — and "cognitive capitulation" — completely abandoning verification. A calculator or GPS takes over a specific task, but the human stays in the oversight loop. With LLMs, people stop overseeing altogether: the confident, fluent delivery lowers the internal threshold for skepticism.
The authors' conclusion: decision quality under cognitive capitulation is entirely determined by AI quality. When AI is accurate — results beat human performance. When it's wrong — they're worse, and the user doesn't even notice.