As large language models seep into everyday life, some worry the technology could trigger a mass political realignment. Chatbots, the theory goes, can be shaped by training data and system instructions to privilege certain worldviews, and users who interact with them daily may gradually absorb those biases at scale.
But Dartmouth College political scientist Brendan Nyhan cautions against assuming such a future is inevitable. LLMs may be powerful, he says, but that doesn’t mean they’ll influence people in the ways we expect, or even in the ways their creators intend.
There are several reasons an AI-driven political shift may be harder to engineer than it sounds. Most people don’t closely follow political news, and it’s unclear how often they use AI tools for political guidance in the first place. And while chatbots can sound persuasive, and in some cases have encouraged disturbing behavior, there’s little evidence that they are fundamentally reshaping most users’ core beliefs.
There’s also a practical tension at play. Companies may face pressure to steer AI systems toward certain viewpoints, but they are simultaneously competing on qualities like accuracy and reasonableness. It’s difficult to optimize for both at once.
The dawn of the social media age was instructive, says Nyhan, who—along with coauthors—recently published a preprint chapter explicating some of the challenges of studying AI’s impact on politics. As many of us remember, the outcome of the 2016 election prompted serious concerns that social media platforms like Facebook had caused political polarization through biased algorithms and fake news. Still, a decade after that election, social science research is still open about whether social media actually had this kind of impact.
Fast Company spoke with Nyhan about how, while technology can be transformative, human behavior can also be quite sticky. This interview has been edited for length and clarity.
We did have this whole big discourse about whether social media had sort of caused massive political polarization. What were the lessons learned from that era as we think about AI?
It’s important to recognize that we often hear new technologies and seize on claims about the harms that they’re going to create before the evidence is strong enough to really justify what’s being claimed. In this case, the evidence is pretty thin. Social media platforms are hard to study—but to the extent that we can evaluate it—it’s not obvious that social media has made our politics more polarized. They may have contributed in certain specific ways, but in a lot of cases, they’re reflecting the polarization of our politics back to us.
I was one of the authors of a study that randomized exposure to like-minded sources on social media, which is one of the most frequently cited mechanisms by which social media could make people more polarized. When we reduced that exposure to like-minded sources, it had no effect on the polarization of people’s attitudes or vote choice. There have also been a number of studies that pay people to stop using social media for a period of time. Those similarly have quite modest effects at best. Though not necessarily zero, there’s certainly no evidence that social media is the primary cause of polarization.
The fear is that these companies have a lot of control and have become funnels or information, particularly as more people switch from search engines to LLM platforms. There’s this fear that they’re going to sort of make us all Republican or Democrat.
They do exercise a lot of power. [We talk] about the fear that authoritarian countries will influence the content of LLMs in problematic ways . . . I do think there’s reason to worry about the content on which elements are trained.
At the same time, it turns out to be a lot harder to persuade people at scale than is typically assumed. AI chatbots can be pretty persuasive when people interact with them about controversial topics, but most people aren’t asking AIs what they should believe about climate change or who to vote for.
Is there any evidence that these large language models do actually seem to exhibit some values internally that swing one way or the other in terms of left and right?
People have administered various questionnaires to the LLMs to benchmark them against the attitudes they express against humans. When they are asked questions in that format, they tend to give answers that, on average, lean to the left. That’s likely reflecting the balance of the information that they’ve been trained on. It may also reflect, in part, the way the companies are developing them.
Increasing model performance has tended to drive LLMs towards more accurate answers. What I mean by that is that AI companies are obviously in this race to develop better models against each other, and we’ve generally seen that models that perform better on the benchmarks they compete on are generally performing better at providing accurate, evidence-based information. Right? Of course, not always, and not perfectly.
But the improvement has been quite rapid, and it’s actually so far proven to be pretty hard to have a frontier model that just gives you political output that you find appealing. Grok has really fallen off the cutting edge, and you can even see it reverting back to more standard types of answers when Elon Musk stops paying as close attention to it and badgering his engineers to manipulate it. It tends to revert back to saying things like climate change is real.