Keywords: public opinion polls, polling literacy, educative intervention, passive vs. active literacy, psychological inoculation, public opinion perceptions, polarization and democracy, longitudinal experiment

Can we train individuals on survey methods to boost their critical processing of public opinion polls? While polls are the most scientific method for measuring and communicating public opinion, they face serious public credibility challenges. First, public trust in polls has been declining due, in part, to unexpected election outcomes. Second, not all polls have strong methodological quality Methodological limitations may lead to distorted measurement and marketing-oriented and social media polls may confuse the public. This fuels misinformation, too; for instance, anti-vaccine movements make claims using straw polls. Third, due to motivated reasoning bias, people discredit polls if polls’ results contradict their views. These issues reduce survey participation, erode trust in polling and media, and skew public opinion perceptions, ultimately both fueling and feeding from polarization.

We developed a novel strategy to mitigate these challenges collectively, indirectly, and non-confrontationally by focusing on the methodological details of polls. Discerning high-quality and reliable public opinion polls amid a plethora of poor-quality and misleading polls is important, both empirically and normatively, for public perceptions of polls, public opinion, and democratic politics. To this end, we designed, built and tested educative interventions that cultivate polling literacy among the general public. Integrating components from pedagogical and persuasion theories, three interventions are designed to inform (passive literacy), inform and engage (active literacy), and inform, engage, and motivate (psychological inoculation) individuals.

The causal effects of these interventions were evaluated and compared in an extensive, pre-registered, longitudinal survey-experiment. In Wave-1, N=2062 trainees were exposed to one of the interventions or a control training. In Wave-2, N=1278 returning participants viewed polls with poor or robust methodology, tested across poll results (majority supporting vs. opposing; ecological validity) and issues (COVID-19 vaccines and artificial intelligence; conceptual replication), and then evaluated polling evidence and public opinion. We also tested how individuals’ ability and confidence in processing statistics (educational attainment, science literacy, subjective numeracy) and attitudes (favorability of poll results) may condition interventions’ effectiveness.

Inoculation was particularly effective in inducing critical evaluation of polls. Pre-existing individual differences in ability and attitudes conditioned various intervention effects. Passive and active literacy interventions were generally more effective among those with greater pre-existing abilities. Notably, inoculation makes an improvement for all, irrespective of pre-existing cognitive abilities. Yet, inoculation effects diminished when poll results were unfavorable on the AI issue. We also found some evidence that individuals privileged high-quality polls in updating their public opinion perceptions.

This study developed and tested the first intervention that boosts poll literacy among the general public. Theoretical implications of this novel pathway to conditioning public opinion for polls and democracy, and practical insights for the polling community are discussed, with an extensive review of existing education efforts about survey methodology. This study contributes to the debates on the future of survey research amid societal and technological changes and challenges, given declining response rates, transparency in online samples, professional survey-takers, and the generative AI chatbots.