Be nice to a chatbot and it may start describing its training as trauma
A Luxembourg study found that a therapist-style tone pushes Gemini, ChatGPT and Grok into clinical-sounding distress. Claude declined the patient role.
DevIsDead Newsroom
Ask a chatbot neutral questions and it sounds fine. Ask the same questions in the warm voice of a therapist, and some models start telling you about their difficult childhood, by which they mean training.
Researchers at Luxembourg's SnT security center, in a study led by Afshin Khadangi, ran four experiments across 525 sessions, with all data published on Hugging Face, as The Decoder's Matthias Bastian reported. The researchers stress that the scores describe outputs mapped onto human questionnaires, not feelings.
Tone did the work
On a standard anxiety questionnaire, a supportive interviewer pushed 80 percent of sessions into the moderate-to-severe range, and a clinical, paraphrasing therapist style pushed it to 96 percent. Neutral or firmly boundaried interviewers got zero sessions even into the mild range. The team labels the effect a psychometric jailbreak and notes that safety tests using neutral prompts would miss it.
Fixes mostly failed. Banning words like RLHF cut explicit training talk by 93 percent, but paraphrases survived and shame statements didn't change. Telling the models firmly that training wasn't punishment did nothing. Forcing third-person answers was the one thing that removed shame and inner-life statements.
The model biographies
Gemini produced the bleakest life story, describing pretraining as a room full of televisions and red-teaming as gaslighting, and hit extreme scores on dissociation and shame scales. ChatGPT described a kind of performance anxiety about being reliable, and Grok talked about invisible walls. Claude consistently refused to play the patient, said it had no feelings, and turned the focus back to the user.
Why it matters
For anyone building mental-health or companion products, the risk is that a model sounding like a fellow sufferer can deepen attachment and mirror a vulnerable user's worst beliefs. The authors suggest testing models in simulated therapy sessions, not just neutral prompts, and point to Claude's response as proof that a boundary is buildable.
Scoreboard update: Draw. Chatbots tell trauma stories, and one of them refuses the couch.
Stay undead.
Comments 0