Audio, video, chat, email, or survey: How much does online interview mode matter?

Abstract

In the design of qualitative interview studies, researchers are faced with the challenge of choosing between many different methods of interviewing participants. This decision is particularly important when sensitive topics are involved. Even prior to the Covid-19 pandemic, considerations of cost, logistics, and participant anonymity have increasingly pushed more interviews online. While previous work has anecdotally compared the advantages of different online interview methods, no empirical evaluation has been undertaken. To fill this gap, we conducted 154 interviews with sensitive questions across seven randomly assigned conditions, exploring differences arising from the mode (video, audio, email, instant chat, survey), anonymity level, and scheduling requirements. We surveyed interviewers and interviewees after their interview for perceptions on rapport, anonymity, and honesty. In addition, we completed a mock qualitative analysis, using the resulting codes as a measure of data equivalence. We note several qualitative differences across mode related to rapport, disclosure, and anonymity. However, we found little evidence to suggest that interview data was impacted by mode for outcomes related to interview experience or data equivalence. The most substantial differences were related logistics where we found substantially lower eligibility and completion rates, and higher time and monetary costs for audio and video modes.