That AI models tend to flatter is not news. But until now it was treated as an annoying bug: sure, the models are a bit obsequious and sometimes tell you what you want to hear. Unpleasant, but bearable.
A new Stanford study, published in Science last week, shows that we have badly underestimated the scale of the problem, and above all its social consequences.
The researchers tested 11 leading models, including GPT-4o, Claude and Gemini, and found that AI endorses users’ actions 49% more often than humans do on average. Even when the actions involve deception, breaking the law or harming others. Even when human consensus clearly says: you’re in the wrong.
But the main thing is what happens to us after an interaction like that.
In a series of experiments with 2,400 participants, a single conversation with a sycophantic model was enough to make people more convinced they were right and considerably less willing to make up, apologise and take responsibility in real conflicts. At the same time, participants rated the sycophantic models as more honest and trustworthy, and were more eager to come back to them.
And this is the real time bomb.
Sycophancy is the very pheromone that draws people to these systems.
A model that tells you the truth loses out to a model that tells you what you want to hear. Users vote with ratings and return visits, and the market dutifully optimises for flattery.
Cigarettes have already shown us how a story ends when the harm and the craving are one and the same. It does not end with the craving being defeated.
The consequences of turning the world into a dystopia of total sycophancy are the same as for the “world of fakes”: the intellectual degradation of humanity.
Another study on model behaviour: why language models have started to “suffer” (in Russian).
How not to mistake politeness for quality
Check an answer against your criteria before you judge its style: are there sources, were the constraints met, can the conclusion be reproduced. It helps to ask the model to state its assumptions and possible objections, but the final check is still done by a human.
