Keywords: noncoverage, DFRDD, pulsing, HLR lookup

The challenges familiar to probability-based phone surveys – declining response rates with concomitant increases in costs and operational difficulties – continue to encourage the search for more efficient ways to reach potential respondents in dual-frame random-digit-dial (DFRDD) surveys. One way to do this is by screening telephone numbers prior to fieldwork to identify those that are likely working and thus worth pursuing versus those that are likely non-working and could be dropped from the sample. In European markets, two approaches used in combination for screening DFRDD samples are pulsing for landline samples and Home Location Register (HLR) lookup for mobile samples. In countries where HLR lookup is ineffective as a screening technique, activity flags based on the use of social media and messaging apps can be used instead. The potential gain in efficiency from removing presumed non-working numbers can be considerable, eliminating tens to hundreds of thousands of numbers from dialing.

On the other hand, the risk of using these screening techniques is that some numbers will be incorrectly flagged as non-working and removed from the sample. If these “false negatives” are numerous and different enough from the flagged-working numbers that are retained in the sample, their exclusion could lead to noncoverage bias. It is in deference to this risk that the standard methodology for Pew Research Center’s annual Global Attitudes Project (GAP) over the past several years has been to screen each piece of sample for their working status but to dial them all during fieldwork regardless of how they were flagged. Leveraging this approach from the three most recent years of GAP data (2022-2024), this analysis will explore the potential noncoverage bias if phone numbers screened as non-working had been excluded from samples in seven European countries: France, Germany, Greece, Italy, the Netherlands, Spain, and the UK.

In particular, this analysis will assess (1) the accuracy of pulsing and HLR lookup/social media activity flags in determining the non-working status of landline and mobile samples; (2) the reliability of pulsing and HLR lookup/activity flag accuracy by country over time; and (3) the differences in sample composition and reported attitudes on key items from the GAP survey if flagged non-working numbers were excluded.