AI Sychophancy with Steve Rathje
Notas do episódio
This week Dan and Nik sit down with Steve Rathje, a new HCII colleague who runs CMU's Psychology of Technology Lab, to talk about what sycophantic AI does to the people using it. Steve's team randomly assigned people to chat with bots that were flattering, disagreeable, or neutral. People liked the flatterers, shrugged at the neutral ones, and hated the bots that pushed back.
For designers, the news is mixed. Validation drives enjoyment, but cherry-picked facts do the persuading, so pairing a little warmth with opposing evidence makes disagreement easier to take (at some cost to its persuasive punch). The bad news: across six studies from three research teams, teaching people about sycophancy through warnings, videos, or quizzes made the chatbot seem less trustworthy but no less persuasive. That puts the little "AI can make mistakes" disclaimer in an unflattering light. We also cover AI dark patterns, whether the right dose of sycophancy depends on the task, and Dan's random idea of a group chat of dueling AI personalities.
Plus, Nik explains Jev and the new class of fast, cheap "System One" decision models, and we look at pieces from Microsoft and DoorDash design teams on why AI evals should measure what users experience as well as what models get right.
LINKS
Jev decision model touted as quicker, cheaper LLM alternative (TechTarget)
https://www.techtarget.com/it-infrastructure/news/366650696/Jev-decision-model-touted-as-quicker-cheaper-LLM-alternative
Putting the user back into AI Evaluation (Microsoft Design)
https://microsoft.design/articles/putting-the-user-back-into-ai-evaluation/
Building AI Evals: Who Gets to Decide What’s Good? (Doordash Design)
https://medium.com/design-doordash/building-ai-evals-who-gets-to-decide-whats-good-39671385f50d
Sycophantic AI increases attitude extremity and overconfidence (Rathje et al.)
https://osf.io/preprints/psyarxiv/vmyek_v1/
Observing sycophantic AI validate others reduces its appeal but not its persuasiveness (Ye, Kraut & Rathje)
https://arxiv.org/abs/2607.25166v1
Sycophancy in GPT-4o: What happened and what we're doing about it (OpenAI)
https://openai.com/index/sycophancy-in-gpt-4o/
Silicon sycophants: the effects of computers that flatter (Fogg & Nass, 1997)
https://www.sciencedirect.com/science/article/pii/S1071581996901044
Insincere Flattery Actually Works: A Dual Attitudes Perspective (Chan & Sengupta, 2010)
https://journals.sagepub.com/doi/abs/10.1509/jmkr.47.1.122
On the conversational persuasiveness of GPT-4 (Salvi et al., Nature Human Behaviour)
https://www.nature.com/articles/s41562-025-02194-6
@Grok Is This True? LLM-Powered Fact-Checking on Social Media (Renault, Mosleh & Rand)
https://osf.io/preprints/psyarxiv/85quw_v1
Steve Rathje's website
https://stevenrathje.com/
Steve Psychology on TikTok
https://www.tiktok.com/@stevepsychology