Sycophantic AI increases our attitude extremity on issues like gun violence
A psychology researcher found that disagreeable and enjoyable AI can be beneficial to circumventing political biases -- but only so long as it's not sycophantic.
We’ve written at Hard Reset extensively about the resource-intensive data centers powering AI, the hype around AI that is driving its proliferation, and the potential for the militarization of AI by unchecked tech CEOs and powerful surveillance entities.
But what about the ramifications for end-users looking for guidance, facts, or understanding about the current political and economic landscape?
I spoke with Laura Globig, a cognitive neuroscientist and psychologist at NYU, about two papers she authored about human trust in AI when it comes to politics, agreeable and disagreeable AI, and AI’s potential to increase or decrease attitude extremity.
Ariella Steinhorn: Tell me about the results of your study: what are the effects of a sycophantic AI on its users?
Laura Globig: Our study found that sycophantic or agreeable AI is undetectable—and that as a result, people perceive it to be less biased. This confirmation bias results in increased attitude extremity on issues like gun violence and abortion.
People also trusted AI more than other humans, even if the AI was disagreeable to their viewpoints in an enjoyable way—and even if the enjoyable AI disagreed with their political preferences.
In short: enjoyable, disagreeable AI can be beneficial to circumventing biases, so long as it’s not sycophantic. If it is biased or overly agreeable, it could have opposite impact.
AS: Can you explain what a disagreeable chatbot that is “enjoyable” might look or sound like?
LG: This means developing AI that is disagreeable in a way that isn’t offensive. This is usually a case of tone and the tonality of the interaction. Our research speaks to the need for more models that adopt a friendly disagreeable tone, while also being factual.
We found that enjoyable, disagreeable chatbots actually decrease attitude extremity, which could have a beneficial impact on people: for example, AI could be used to reduce belief in conspiracy theories. (Another researcher, Tom Costello, actually found that some interactions with AI reduced belief in conspiracies over time, and that that impact held for two months.)
Meanwhile, an agreeable chatbot speaks to people’s needs for validation of their existing beliefs as opposed to the facts. Here, confirmation bias kicks in, which is basic human tendency that we’re more likely to want to interact with someone if they agree with us.
AS: Why do you think that people are more likely to trust AI that “enjoyably” disagrees with them even over other humans?
LG: Perhaps because with AI, there are no reputational costs. When we interact with humans, we always worry about our identity—even if the other person has the same social identity as us.
That’s because there are positive social rewards with human-to-human interactions: if I’m nice to you, you’ll be nice to me. If I act out of line, you’ll stop talking to me.
But AI never penalizes us—even if we ask the most offensive thing. It might not give you the answer you’re looking for, because the AI company has put some guardrails in place. But simply asking AI a question won’t impact your reputation negatively.
As a result, disagreeable and friendly AI might be a way to help people come together about a polarizing issue at hand, because they are more willing to interact with a disagreeable chatbot. In some scenarios, we might be able to reduce polarization between people who are affiliated with different groups.
AS: Could this communication with the AI also be because a computer is faceless and we can’t read their body language? I’m thinking about people who are more emboldened to share things over chat rather than in-person. Or people who are more likely to post comments that they may never say out loud, or face-to-face.
LG: I doubt that, just because we find that the more people use AI, the more they massively anthropomorphize it.
There’s the factor of people not feeling like their reputation is at risk, and also the tonality. AI can mimic real world interactions from mediators or facilitators, and we can instruct AI to respond in the way a real-world mediator might.
AS: A lot of these AI companies seem to measure and want to increase amount of engagement on their platforms. Do you think that’s driving the sycophancy, that it encourages people to communicate more?
LG: I actually disagree with that point, that the big three AI firms aren’t thinking about sycophancy. There are some companies in the social companion AI chatbot space that might be different, in that flattery is a core part of the model. But OpenAI, DeepMind, Gemini, and Anthropic all actively reduced sycophancy once they noticed this issue.
If we’re looking at this from a financial standpoint, it’s costly for the companies to have people using the chatbot all the time. They’re trying to get people to use it, but they also want to reduce the number of iterations to get the response that someone desire. If you ask one question, they want the model to provide one answer.
AS: It sounds like more research needs to be done about conversational and relationship AI. What was most surprising to you about your studies?
LG: Probably this notion that people are unaware of their own biases—and also that people are unaware that the model is biased itself. When we interacted with the model, it felt overtly sycophantic. But study participants didn’t notice this model being biased, even if the bias was in their favor.
It also dovetails with the idea that humans are becoming more biased as AI becomes more biased. If their bias is being reinforced by AI models, that would reinforce their biases.
Also, usually, people aren’t even willing to engage with evidence or with out-group members that are disagreeable to them. It’s very intuitive that people might like sycophantic AI over an out-group AI, or a human who disagrees with them. But when it came to political conflict, people preferred enjoyable AI even if it represented their out-group political belief.
AS: So, it sounds like people weren’t even asking questions to or interacting with the AI—but still felt an overwhelming feeling that AI wouldn’t judge them reputationally, and that made them trust them more?
LG: They never actively interacted with AI. But yes for political conflict, people preferred AI over humans, because they found that AI was less judgmental.
AS: What is the silver lining of the study?
LG: The bright side is that AI doesn’t actually increase belief extremity, unless the models are overly sycophantic.
People prefer learning from AI over humans—and this could be leveraged for good. That’s because AI may be able to circumvent inter-group biases, where people prefer learning from in-groups versus out-groups.
For example, people learn less well from others who support different sports teams. They are less receptive to them, even if the task is related to a similar identity. So our research shows that if the AI is accurate, it could help learning where people might be impeded by identity motives.
AI literacy is also critical: to warn people of potential biases and show them that AI doesn’t always provide accurate information. One recent study compared how people evaluated human feedback versus ChatGPT’s feedback. People were quicker to accept AI responses over human feedback—but when the user was reminded that the AI might make mistakes, all of that was counteracted. So, if people shift attention to biases, they are able to correct their beliefs.
AS: You seem to really be focused on the nuance.
LG: Well, if we actively implement these AI literacy tools, we can mitigate some of these effects that we are observing. For instance, for people who might be less willing to engage with other humans of different political affiliations, AI could provide or facilitate inter-group contact.
Of course it’s important to constantly question where our source material is coming from, and remind them that humans and AI can both make mistakes and critically engage.
Either we use AI for everything, or we despise it. Neither are helpful, and we need nuance with any new technology in the world.