Keywords: Methodological challenges and improvements, including in the areas of sampling, measurement, survey design and survey response or non-response

Background and Methodology

In this research we examine passive nonresponse bias in a statewide population-based survey (MO, USA) in three broad ways and with attention to differences in respondents that were reached via cell phone versus those that were reached via land lines (phone type). We utilize data from the Missouri Crime Victimization Survey (MCVS), which was fielded January – March, 2016 (n = 2,008), in order to assess 1) crime victimization and responses, 2) opinions on and experiences with policing, and 3) perceptions of community and social issues among adult residents. Both cell phone and landline users were selected via random digit dialing.

First, we investigate the issue of saliency by examining responses to crime and victimization issues, assuming those with more interest in the survey topic were more likely to complete the survey. Here we consider three variables: 1) whether the respondent was a victim of a crime in the past 12 months, 2) how confident or not confident the respondent is in local law enforcement, measured on a five point scale, and 3) the extent to which the respondent is worried about crime and victimization. We will test to see how these items vary across phone type, urban-rural residence, age, and gender. Additionally, phone type will be assessed within each demographic group.

Second, we evaluate whether early and late survey respondents (based on number of times contacted) differ on phone type and the key outcomes above. If late respondents are similar to nonrespondents, then this information will give us a sense of how key data points may have been affected by nonresponses. We compare early and late survey responses across phone type and the demographic variables described above. Third, we compare the demographic distribution of the unweighted sample to the state population and then see if the underrepresented groups differ systematically from overrepresented groups across phone type, demographics, and the three key indicator variables above.

Preliminary Results

About 75 percent of respondents were reached via a cell phone while 25 percent were reached via a landline. Although it is not a focus of this work, we tested whether time to complete the survey in minutes varied across phone type and found it did not.

With regard to our first research question, we found that there was a significant difference in 1) victimization in the previous 12 months (χ2=24.75, p<.000), 2) confidence in local law enforcement (χ2=10.21, p<.05), and 3) worry about crime (χ2=34.13, p<.000), across phone type. Second, the number of attempts did not significantly vary across phone type nor did it correlate significantly with confidence in law enforcement or worry about crime. Third, we found the statewide weight variable significantly differed across phone type (T= -7.93, p<.000). We explore our research questions in more depth in multivariate analyses.