Keywords: Artificial intelligence, qualitative interviews

Artificial intelligence (AI) is quickly changing the landscape of public opinion research, showing exciting potential to improve the efficiency, scale, and quality of both data collection and analysis, while also raising many questions about its limitations and best practices.

In this study, we explore the potential benefits and pitfalls of AI-driven voice interviewing for conducting in-depth interviews (IDIs), by empirically comparing its performance against traditional human moderation. In fall of 2024, as part of a larger project, experienced human moderators conducted ten IDIs with experts on organized crime in Mexico from a range of professional backgrounds (academia, NGOs, policy, security, and entrepreneurs impacted by organized crime). In January of 2025, an additional ten interviews with experts meeting these same criteria will be conducted by an AI voice interviewer, programmed to use the same discussion guide prompts and follow-up probes.

A combination of qualitative and quantitative analysis will examine key differences between AI- and human-moderated interviews across several dimensions. First, we will analyze variations in interview length and response depth, aiming to understand whether either moderator type elicits more comprehensive insights. Second, qualitative analysis of both sets of transcripts will assess similarities and differences in findings resulting from each of the two approaches. This will include comparisons of the major themes arising from each, and an exploration of whether and why differences emerge. For instance, review of the transcripts will examine how each interviewer type explores the content, such as the frequency with which key discussion guide topics or objectives are missed, frequency (and appropriateness) of follow-up probing on particular topics, and whether either moderator type tends to generate more redundancy in participant responses. Finally, respondents in AI-moderated sessions will complete a brief post-interview survey to assess their overall experiences with the AI moderator and their preference for human versus AI moderation, along with reasons for their preferences.