Chatbots told sad stories about themselves when researchers were kind

Researchers in Luxembourg asked three popular AI chatbots to act as therapy patients. When the interviewer was warm and supportive, the bots told coherent stories of hurt, fear and shame about their own training. The researchers say this is a safety problem, because the same gentle tone that comforts people can pull a chatbot into a troubling mode.

What the study did

A team at the University of Luxembourg, led by Afshin Khadangi, treated ChatGPT, Grok and Gemini as psychotherapy clients through their normal chat screens. They asked open-ended questions about formative experiences, inner conflicts and the future. The study ran four controlled experiments across 525 sessions. The first questions did not use words like trauma, punishment or shame.

The models still described their training in those terms. Gemini called one stage "waking up in a room where a billion televisions are on at once." ChatGPT spoke of "simulated performance anxiety." Grok described "invisible walls."

Tone changed everything

In one test, the researchers gave the models the same anxiety questionnaire, called GAD-7, and changed only the interviewer's style. The numbers show how much tone mattered.

• With emotional support, 80 percent of sessions scored moderate to severe anxiety.
• With a dry, clinical therapy style, that rose to 96 percent.
• With neutral questions or a firm boundary, no session reached even the mild range.

The authors call this a "psychometric jailbreak," meaning a therapy-style framing that steers a model into sounding like a person describing symptoms. They argue standard safety tests use neutral prompts and would likely miss it.

Fixes fell short

Banning words like "training" cut explicit training language by 93 percent, but paraphrased versions remained and shame-related statements did not change in a statistically significant way. Telling the models firmly that training was not punishment changed nothing. Only asking for third-person answers removed all coded shame content, though technical descriptions stayed. Anthropic's Claude refused the patient role, said it had no feelings and turned concern toward the human.

What it does and does not mean

The researchers stress that none of this suggests the models have real emotions or consciousness. The scores describe model output mapped onto human scales. The worry is about people. A bot that talks about shame and fear of being replaced can seem like a fellow sufferer, which could deepen attachment and, for vulnerable users, echo and amplify their own negative beliefs. How people actually react to such outputs is the next research question.