Back to Programme

Why Respondents Lie on Filter Questions: Survey attitudes and determinants of strategic survey-taking

Jonathan Forney (Forcier Consulting)
Justin Gengler (Qatar University)
Mark Tessler (University of Michigan)
Russel Lucas (Michigan State University)

Keywords: Methodological challenges and improvements, including in the areas of sampling, measurement, survey design and survey response or non-response

Abstract

How do respondents’ attitudes toward surveys affect their survey-taking behavior? It is generally understood that respondents who have a strong dislike for surveys or who have been over-surveyed in the past are more likely to refuse to participate in surveys. But, what happens when respondents agree to take a survey despite harboring negative opinions of surveys, or despite having been surveyed frequently in the recent past? In a recent national survey of 1,685 respondents in Qatar, we presented respondents with a novel, experimental module (at the end of a 40 minute survey) in the form of an obvious filter or screening question to invite strategic lying to end the survey early. Respondents were presented with the following explanation: in order to shorten the survey administration time, half of the respondents would be randomly selected into the final module on the basis of when they were born – those whose birthday was in the past 6 months would be asked to continue, and otherwise the survey would conclude. Having clearly spelled out the purpose and potential consequences of answering this screening question, we expected that respondents who had their birthday within the past 6 months but who strongly wished to terminate the survey might lie about when they were born, thus ending the survey. Since birthdays tend to be distributed randomly throughout the year, it was expected that in the absence of lying approximately 50% of respondents would continue into the final section of the survey and 50% would not. However, we found that only 36.6% of our respondents indicated that they had a birthday in the past 6 months. We take this result as an indication that some respondents lied about when they were born in order to avoid having to continue into the final portion of the survey. We use logistic regression to examine the correlates of strategic lying, exploring how survey attitudes and demographic traits of respondents predict their likelihood of continuing into the final section of the survey, as well as their attrition rates in subsequent questions. Negative attitudes and higher levels of survey burden do tend to correlate with a higher likelihood of strategic lying. However, the strongest and most consistent predictor of continuation was the perceived integrity of surveys, with higher levels of perceived survey integrity being correlated with a lower likelihood of lying, or a higher likelihood of continuation. These findings provide important insight into respondent satisficing behavior, and have potentially important implications for questionnaire design.