Validation of Metrics—A Comparative Analysis of Predictive- and Criterion-Based Validation Tests in a Qualitative Study

Erin Fordyce, Michael J Stern, Sabrina Avripas Bauroth, Catherine Vladutlu


A primary concern for researchers, when collecting self-reported data from respondents, is supporting the accuracy and reliability of the data collected. Measurement error can be introduced by questions asking for specific, factual information that may be difficult for respondents to recall. It can further increase due to the underreporting of sensitive information (Blattman et al., 2014) often attributed to social desirability. Researchers continue to struggle with finding cost-efficient and burden-free approaches to validate self-reported information. The National Survey of Children’s Health (NSCH) collects data on the physical and emotional health of children including current and pre-existing conditions (or disabilities). As part of the NSCH Redesign study, 64 cognitive interviews were conducted with predictive and criterion-based validation tests to validate household screener items as well as items related to medical diagnosis and health insurance status. To assess predictive validity, a test-retest approach was used, whereby a subset of respondents were re-administered items from the household screener and main questionnaire two weeks subsequent to their initial interview. A separate, criterion-based validity test was conducted by asking the remaining respondents to provide documentation to validate the household screener items as well as items related to medical diagnosis and health insurance status. Respondents who did not provide documentation were asked permission to contact the child’s primary care provider. This paper addresses important issues surrounding instances where a researcher is asking respondents to provide documentation to validate reported information. For instance, we discuss the impact of requesting various types of documentation on respondent burden and the advantages and disadvantages of requesting documentation versus other means of validating information. In addition, we discuss the effectiveness of conducting re-tests in identifying potential measurement error.


validity, cognitive interviews, health

Full Text: PDF HTML

About Survey Practice Our Global Partners Disclaimer
The Survey Practice content may not be distributed, used, adapted, reproduced, translated or copied for any commercial purpose in any form without prior permission of the publisher. Any use of this e-journal in whole or in part, must include the customary bibliographic citation and its URL.