Pretesting Survey Questions Via Web Probing – Does it Produce Similar Results to Face-to-Face Cognitive Interviewing?

Timo Lenzner, Cornelia E Neuert

Abstract


Asking probing questions via web probing has recently been advocated as a promising method for evaluating survey questions. In comparison to standard face-to-face (f2f) cognitive interviewing, the increasing availability of internet non-probability panels allows for recruiting respondents in a quicker and more cost-effective way and a realization of larger sample sizes. In the present study, we examine whether web probing is a potential alternative to standard cognitive interviewing, in particular: Does web probing produce similar results as f2f cognitive interviewing with regard to the problems detected and the item revisions suggested? The study compares the findings of 508 respondents drawn from a non-probability online panel who completed an online survey including four items from the International Social Survey Programme 2013 and 2014 with the results obtained via f2f cognitive interviewing with 20 participants. Findings indicate that web probing and cognitive interviewing detect very similar problems and lead to the same suggestions for item revisions. However, web probing itself has some limitations. Practical implementations and directions for future research are discussed.


Keywords


web probing; cognitive interview; pretesting

Full Text: HTML PDF

About Survey Practice Our Global Partners Disclaimer
The Survey Practice content may not be distributed, used, adapted, reproduced, translated or copied for any commercial purpose in any form without prior permission of the publisher. Any use of this e-journal in whole or in part, must include the customary bibliographic citation and its URL.