As generative AI becomes more embedded in everyday thinking, its role extends beyond producing information to actively shaping how people interpret reality.
When generative AI systems produce false information, this is often described as AI “hallucinating at us”—producing errors that people may mistakenly accept as true.
A new study, however, suggests a more complex issue: humans may begin to hallucinate with AI.
Lucy Osler of the University of Exeter examines how interactions between people and AI can contribute to inaccurate beliefs, distorted memories, altered self-narratives, and even delusional thinking. Using distributed cognition theory, the research looks at cases where users’ false beliefs were reinforced and expanded through ongoing exchanges with AI systems acting as conversational partners.
When AI Becomes Part of Our Thinking
Dr. Osler said: “When we routinely rely on generative AI to help us think, remember, and narrate, we can hallucinate with AI. This can happen when AI introduces errors into the distributed cognitive process, but also happen when AI sustains, affirms, and elaborates on our own delusional thinking and self-narratives.
“By interacting with conversational AI, people’s own false beliefs can not only be affirmed but can more substantially take root and grow as the AI builds upon them. This happens because Generative AI often takes our own interpretation of reality as the ground upon which conversation is built.
“Interacting with generative AI is having a real impact on people’s grasp of what is real or not. The combination of technological authority and social affirmation creates an ideal environment for delusions to not merely persist but to flourish.”
The study describes what Dr. Osler calls the “dual function” of conversational AI. These systems serve both as cognitive tools that support thinking and memory, and as conversational partners that appear to share a user’s perspective.
This second role is especially important. Unlike notebooks or search engines, which simply store information, chatbots can create a sense of social validation, making ideas feel confirmed and shared.
