Keywords: item nonresponse, factorial survey experiment, vignette study, implausibility

Factorial survey experiments (FSEs) are a widely used tool to identify judgment principles and subgroup differences, using their experimental design to draw causal inferences. Systematic item nonresponse can undermine this advantage, arising from diverse and frequently unobserved causes before (questionnaire design) or during (respondent characteristics) data collection. The purpose of this paper is to provide a guide for identifying potential threats prior to data collection, detecting systematic item nonresponse afterwards, and assessing its impact. These techniques are demonstrated in a case study of an FSE investigating entrepreneurs’ likelihood of purchasing voluntary unemployment insurance in Germany. The study reveals substantial item nonresponse, likely driven by the implausibility of certain dimension levels. We show that the impact can be mitigated by excluding the problematic level dimension without losing level balance and independence of all dimension levels. The findings highlight the importance of thorough pre-collection preparation, as well as identifying and addressing potential causes of item nonresponse to ensure high data quality.