Misleading Survey Results Can Amplify Animosity and Fear

Survey results can be surprising — sometimes even shocking. 

For example, in 2022, a Pew Research survey found that 12% of adults aged 18-to-30 were licensed to operate a class SSGN nuclear submarine. 

Obviously that result isn’t accurate. The actual number is effectively 0%. Pew conducted that survey to show how distorted the results of online opt-in surveys can be — especially when surveying young people. 

Our nation’s youth operating nuclear subs might be a funny (or scary?) idea — but there’s a serious message here. People often believe faulty survey results — especially when those results align with their existing beliefs. When we believe and share false information related to our divides, we can ramp up our own fears and animosity — and other people’s. Believing bad information can make us unintentional amplifiers of toxic polarization.  

We should have a healthy skepticism about survey results and other information we encounter. When you see a survey result that riles you up, ask yourself, “Is it possible this is wrong?” You should take the time to look into it further — just as you should avoid taking headlines at face value. 

An inaccurate survey on Holocaust denial

Let’s take another example: a recent Economist/YouGov survey found that 20% of 18-to-30 year-olds believe “the Holocaust is a myth.” That result was understandably quite concerning; it received a lot of attention.

Like the submarine survey, this was an online, opt-in format. When Pew tried to replicate this result with a mailed survey, which is more reliable than opt-in, they got 3% instead of 20% — a percentage consistent across all ages. Pew’s work led to some news outlets adding a clarification to their coverage of the first survey (for example, see The Economist’s note).   

Holocaust denial is a serious, concerning issue at any level. But an exaggerated view of that problem can lead to unnecessary fear and anxiety, amplifying tensions. As Pew Research put it: “…reporting on complex and sensitive matters such as these requires the use of rigorous survey methods to avoid inadvertently misleading the public, particularly when studying the attitudes of young people.”

Distorted views on support for political violence

In a recent piece of ours, we examined surveys that dramatically exaggerate Americans’ support for political violence. Some estimates for such support have been as high as 40%! 

In a 2024 report by Sean Westwood, director of the Polarization Research Lab, and his team, they wrote, “Fewer than 4% of Americans support violent crimes like assault or arson against political opponents, with minimal difference between Democrats (3.5%) and Republicans (3.8%).” Via email correspondence with Westwood, he told us, “Democrats and Republicans think support for violence is about 20 times larger than it is. It’s hard not to see panic-narratives driving these incorrect views.”

Survey bias and inaccuracy can result from leading language. For example, a 2023 PRRI survey asked respondents if they agreed that “because things have gotten so far off track, true American patriots may have to resort to violence in order to save our country.” 

The results found that substantially more Republicans (33%) than Democrats (13%) agreed with that statement. But one can see how the “patriot” language may have skewed responses, with conservatives being more likely to respond to that framing. We asked Westwood for his thoughts and he said, “I agree that ‘patriots’ may have biased the results.” 

Again, we can be concerned about political violence (3% is still a lot of Americans!) while also seeing the importance of not overstating that problem. 

Many factors can lead to distorted results

In his 2024 piece “Why polling is so often wrong,” James Rosen examined multiple reasons for inaccurate poll results. (His focus was on election-prediction polls but the factors apply to surveys in general.) One factor is the drive to get clicks: 

The internet, with its voracious appetite and greatly expanded space for new information no matter how incremental, has made some political journalists less discriminating and fueled more questionable polling. The financial pressures on news organizations have increased the need for digital readers, which has led to election articles that are little more than click-bait.

Also, our societal volatility can make it hard to get an accurate view of what people think: 

Especially in our volatile political climate, pollsters no longer can use past voting habits to predict future turnout. Barack Obama rode to victory in 2008 largely thanks to new voters casting ballots. Similarly, eight years later, Trump surprised pollsters by persuading many who had sat on the sidelines to support him.

Toxic polarization creates a turbulent system. New, polarizing issues that arise quickly (for example, COVID-19) can shift people’s views in rapid, unpredictable ways.

Polling as a form of partisan venting

“Expressive responding” refers to survey takers venting their emotions in ways that don’t reflect their true beliefs. The toxicity of our political divides lead to many people who have a desire to express their animosity toward their political opponents. 

Let’s imagine that some percentage of survey-takers who said that “the 2016 election results were illegitimate” or “the 2020 election results were illegitimate” did so not because they were certain in those beliefs, but partly as a way to express their high distrust of the “other side.” Those survey results would lead to people on the “other side” being more anxious and upset, which would lead them to say more polarizing things, and so on. Survey results can be one of the many factors involved in the feedback cycle of toxic polarization. 

We must aim for accuracy and avoid fanning the flames

To reduce the toxicity of our divides, we must embrace a healthy skepticism of survey results — especially those directly related to divisive issues. When you see people discussing survey results that are unusually shocking or scary, make it a habit to find the actual survey and dig deeper. You can look at the specific questions asked and consider how you would have answered them. You can think about what factors might have influenced people to answer one way or the other. Then, consider whether the headline that led you to the survey accurately describes its results.

You can also educate others when you think they’re believing or sharing false information. Remember that people may see criticism of their information as a criticism of their concerns, and this may make them angry. Be persuasive and empathetic by making it clear you’re not dismissing their concerns. Point out that basing their concerns on faulty information can be self-defeating, reducing their credibility in a boy-who-cried-wolf type of way. 

Want to stay in the loop on overcoming our toxic divides? Sign up for our newsletter.