Home Is Where the Cooperation Is

The Association between Interview Location and Cooperation among Cellphone Users

Christopher Ward NORC at the University of Chicago, 55 East Monroe Street, 30th Floor, Chicago, IL 60603, USA

Becky Reimer NORC at the University of Chicago, 55 East Monroe Street, 30th Floor, Chicago, IL 60603, USA

Meena Khare National Center for Health Statistics, CDC, 3311 Toledo Road, Hyattsville, MD 20782, USA

Carla Black National Center for Immunization and Respiratory Diseases, CDC, 1600 Clifton Road, Mailstop A-19, Atlanta, GA 30329, USA

Abstract

Interviewing respondents on cellphones poses challenges to survey researchers. Using data from the National Immunization Survey, a national, dual-frame random-digit dial survey, we examined whether respondents’ level of cooperation varies by their telephone status and location at the time of the interview. Specifically, we used regression models to compare the cooperation rates for respondents who are contacted on landlines while at home, those who are contacted on cellphones while at home (“cell-at-home”), and those who are contacted on cellphones while away from home (“cell-away”). Our models included a number of respondent characteristics that may be related to the likelihood of cooperation versus breakoff during the survey. Results indicated that observed differences in cooperation between landline and cellphone-while-away respondents are primarily due to cell-away respondents being less likely to respond. Our results also suggest that time of interview is a significant predictor of likelihood to provide permission to access children’s health care records. This research provides insight into the behavior of cellphone respondents and the conditions under which they may be most likely to respond.

Introduction

Random-digit dial (RDD), computer-assisted telephone interviewing (CATI) surveys have been integral to interviewer-assisted survey research for decades, providing a comparatively low-cost means of conducting large-scale studies across wide geographical regions. The rapid adoption of the cellphone, however, has proven to be a challenge in implementing CATI surveys. Interviewing respondents on cellphones is associated with response rates that are lower than (or at best, on par with) those in equivalent landline studies (Brick et al. 2006; Curtin et al. 2005). These lower response rates, in tandem with legal requirements prohibiting the auto-dialing of cellphones and dictating that cell samples be manually dialed by interviewers, are associated with increased costs in comparison to landline-based surveys (Brick et al. 2007). Risk of undercoverage is likewise a problem with declining landlines in households, especially among the young and less affluent (Blumberg and Luke 2013).

Another important difference between landline telephone respondents and cellphone respondents is that the latter group may be mobile. Whereas a landline interview must be conducted at a fixed place (e.g., in a residential, population-based study, the telephone interview is ordinarily conducted in the respondent’s home), a cellphone respondent can participate in an interview wherever there is a cellular signal. A growing body of research suggests that the circumstances of cellphone interviews conducted away from home pose unique challenges to data collection in comparison to landline interviews or cellphone interviews conducted at home (Link et al. 2007). In particular, interviewing a respondent who is away from home risks distraction, greater cognitive burden, and increased respondent concerns about privacy. These challenges can render the interview more difficult for the interviewer to control. Much of the increased cognitive burden may arise from the challenges of multitasking while using a cellphone (Hyman et al. 2009). Lavrakas et al. (2010) have suggested that further research should investigate the consequences of multitasking on the burden for cell respondents who are away from home. Link et al. (2009) have called for extending research into the effect of burdens on respondents who are in an area of “high distraction” (e.g., away from home) that may threaten their “cognitive engagement” at the time of the interview.

In light of the rapid adoption of cellphones and the challenges associated with conducting CATI surveys on cellphone frames, we pose the following research question: Does cooperation vary by the respondent’s location? A key corollary of the question arises: Is the behavior of cell-at-home respondents different from that of landline respondents, and how can survey design address observed differences in behavior? We used data from the National Immunization Survey (NIS, http://www.cdc.gov/vaccines/nis) to investigate these questions.

Research Data

Analyses were performed on data from the 2012 NIS, a large, national- and state-level, dual-frame RDD telephone surveillance survey sponsored by the Centers for Disease Control and Prevention. To collect data about childhood vaccinations, the NIS screened for households with at least one child between 19 and 35 months of age and employed a take-all household approach (i.e., the study did not screen to determine the extent to which a respondent owned or used one or more cellphones or landline telephones). To identify the location of cell respondents at the time of the interview, the NIS added the additional question: “Would you mind telling me if I reached you today away from home or at home?” The structure of the survey is outlined in Table 1.

Table 1 Structure of the National Immunization Survey, 2012.

Section title Topic
Section S Screener
Section B Child(ren)’s vaccination history
Section C* Demographic and socioeconomic information
For respondents in the cellphone frame, this section included the question about whether the respondent is at or away from home at the time of interview.
Section D Contact information for vaccination providers
This section included a question asking the respondent to grant consent for NIS researchers to contact the child’s health care provider(s) to obtain vaccination records.
HIM Health Insurance Module

*For tracking completed interviews during data collection and for this analysis, an interview is designated as complete after a respondent completes Section C. However, even if an interview is designated as complete, data collection continues with Section D, which collects provider data, and the Health Insurance Module.

The location of cellphone respondents (whether at home – “cell-home,” or away from home – “cell-away”) was collected in the demographics section of the survey (Section C), whereas landline respondents (including both users of traditional landlines as well as related telephone types, such as internet telephones) were presumed to be at home at the time of interview. Several correlates of survey response were also collected in this section, including the respondent’s relationship to the child, the mother’s race and ethnicity, total household income, the mother’s level of education, and whether the mother lived in the same place as at the time of the child’s birth. For cellphone respondents, the demographics section included questions about whether they use cellphones exclusively or as their main type of phone communication.

The key dependent variable in this study was whether the respondent granted consent to make his or her child’s vaccination records available to researchers. Granting consent is a key measure of cooperation, allowing NIS research staff to collect vaccination histories from the health care providers nominated in Section D.

Analytic Subsample

Since the respondent location question is located deep within the instrument in the demographics section, we defined the analytic subsample to include only those respondents – cellphone and landline – who screened and completed the interview within the same call. This criterion associated the respondent’s behavior during the interview (that is, from screening to completion) with his or her sample frame and location. To ensure comparability across households, the subsample was limited to the first child named in multiple-child households. Twenty-six interviews where cellphone respondents reported “don’t know” or “refused” to the location question at the time of interview were also excluded. From these criteria, a subsample of 15,067 cases (approximately 59 percent of the total 2012 sample of all 25,334 cell and landline cases) was identified.

Methods

A logistic regression was estimated to examine the relationship between respondent location (landline, cell-home, and cell-away) and the probability of granting consent to contact children’s health care providers. The regression is expressed as follows:

\(\ln \left( {{p \over {1 - p}}} \right) = {\beta _0} + {\beta _1}{x_1} + \ldots + {\beta _n}{x_n},\)

where p is the probability of the outcome variable, x1xn are the independent variables, and β0βn are the model coefficients. The model incorporated two groups of independent variables. The first group included variables that are ordinarily controllable through survey design features such as screening, sampling, or other mechanisms: the respondent’s location, the time of interview, whether the respondent is the child’s mother, and whether the respondent is cell-only/-mainly (that is, cell-only users and dual users who report using a cellphone more extensively than a landline telephone). The second group included socio-demographic and other related covariates commonly associated with survey response, including the mother’s education, having a record of the child’s vaccination history, housing tenure, having multiple eligible children in the household, household income, mother’s race and ethnicity, and age. The dependent variable was whether the respondent granted consent for researchers to contact vaccination providers to obtain records.

The model incorporated weight, stratum, and cluster variables to account for the complex sample design of the NIS (Long and Freese 1997) and used imputed versions of variables on the public-use file where available. Backward- and forward-selection stepwise regressions were conducted on the model to confirm that appropriate variables were specified for the model.

Results

Table 2 illustrates the key performance indicators among all interviewed households in 2012 and contrasts statistics between landline and cellphone respondents. In 2012, the Council of American Survey Research Organizations (CASRO) Type II response rate was 64.54 percent for landline respondents and 30.64 percent for cellphone respondents. The American Association for Public Opinion Research Response Rate 3 (AAPOR RR3) was equivalent to the CASRO rate (Barron et al. 2008). The overall rate of consent to contact providers was 74.67 percent for landline respondents and 71.30 percent for cellphone respondents. Table 3 further compares the rates of consent to contact providers across all three location types among households in the analytic subsample. Because the analytic subsample included only interviews completed on one call, these rates do not account for refusal conversion attempts on subsequent calls and are specific to the analytic subsample. They are therefore not comparable to the NIS provider consent rates for the overall sample.

Table 2 Key data collection statistics (2012 NIS).

Telephone type AAPOR RR3/CASRO type II response rate Age-eligible children with completed household interviews Number of completed interviews with consent Rate of consent to contact provider(s)
Landline 64.54% 12,325 9,203 74.67%
Cellphone 30.64% 13,009 9,276 71.30%

Table 3 Provider consent rates by respondent location.

Phone-location type Subsample size Number with consent Consent rate
Cell-away 2,050 1,376 67.1%
Cell-home 5,308 3,872 72.9%
Landline 7,709 5,773 74.9%
Overall 15,067 11,021 73.1%

Analytic subsample: 2012 NIS.

Model for Consent Rate

Table 4 presents the results of the logistic regression. Controlling for other variables, there was no statistically significant difference in the likelihood of granting consent between cell-home and landline respondents. However, cell-away respondents were significantly less likely to grant consent than landline respondents (OR=0.730).

Table 4 Logistic regression model of rate of respondent consent to contact vaccination providers (2012 NIS).

Parameter Odds ratio 95% CI
Intercept 2.696*** (1.963, 3.703)
Cell-home 0.987 (0.838, 1.161)
Cell-away 0.730** (0.594, 0.897)
Reference: landline
Daytime (before 5 p.m.) 1.269** (1.084, 1.485)
Nighttime (after 7 p.m.) 1.125 (0.957, 1.322)
Reference: evening (5–7 p.m.)
Whether mother has college degree 0.923 (0.792, 1.078)
Whether R is mother 1.591*** (1.382, 1.831)
Whether R is cell-only/-mainly 0.935 (0.804, 1.088)
Whether R owns dwelling 1.032 (0.887, 1.200)
Whether R has shot card for child 1.107 (0.966, 1.268)
Number of NIS-eligible children in household 0.692*** (0.581, 0.825)
Mother’s race/ethnicity: Hispanic 0.951 (0.753, 1.203)
Mother’s race/ethnicity: non-Hispanic black only 0.914 (0.718, 1.077)
Mother’s race/ethnicity: other 1.049 (0.782, 1.407)
Reference: Mother’s race/ethnicity: non-Hispanic white only
Poverty status: above threshold, >$75,000 0.957 (0.814, 1.124)
Poverty status: below threshold 1.397** (1.157, 1.688)
Poverty status: unknown 0.472*** (0.344, 0.648)
Reference: Poverty status: at or above threshold, <$75,000
Mother’s age ≤29 0.950 (0.815, 1.108)

*p<0.05; **p<0.01; ***p<0.001.

Several characteristics of the interview and of the respondent were also associated with a greater likelihood of granting consent. Controlling for other variables in the model, respondents who were contacted before 5 p.m. had greater odds of granting consent (OR=1.269) than those contacted from 5 p.m. to 7 p.m., but no significant difference between evening (5 p.m. to 7 p.m.) and nighttime (after 7 p.m.) calling was observed. Mothers were also more likely to grant consent than non-mother respondents, such as fathers and grandparents (OR=1.591). Respondents in households below the poverty level were also more likely to grant consent than those at or above the poverty level (OR=1.397). Likewise, some characteristics were associated with a lower likelihood of response, such as having multiple NIS-eligible children (OR=0.692) and refusing to report any household income data (OR=0.472).

Limitations

The analytic subsample may be biased towards more cooperative respondents, as it included only those who screened and completed the interview in one call. This criterion ensured that respondent behavior could be associated with his or her location, but it also caused the selection of a subsample of an inherently more cooperative group of respondents. Indeed, this study likely underestimates the true impact of being away from home on the likelihood of cooperation. It is possible that cell-away respondents were more likely to break off early in the interview, but this study cannot observe their behavior due to the position of the cell location question in the instrument.

Conclusions and Further Research

Being away from home during the interview was a powerfully negative predictor of cooperation. Moreover, in the context of one definition of cooperation – granting consent to contact vaccination providers – this study found minimal difference in cooperation between landline and cell-at-home respondents after controlling for other variables found in previous research to be associated with response.

The behavior of cell-away respondents appeared to account for much of the overall differences between landline and cellphone respondents in the model. Although not tested directly in this study, previous research suggests that the extra burden, distractions, and privacy concerns faced by cell-away respondents may account for these observed differences in behavior (Link et al. 2007). On the one hand, the study’s findings are consistent with previous research: cell respondents who are away from home at the time of interview appear to be less likely to cooperate than are respondents – whether on landline or cell telephones – who are at home. On the other hand, respondent location is but one variable associated with the likelihood of response, so we cannot assert that it is the definitive factor in response.

These burdens have implications for CATI survey operations, including the development of rules for scheduling calls, methods of screening respondents, and means of averting refusal or breakoff. To improve cooperation, survey interviewers could increase calling attempts during times that are more amenable to respondents, screen for cellphone respondents who are at home, or use the CATI instrument to alert interviewers when respondents are away from home to avert breakoff at key points of the interview. However, many of these options could impose additional respondent burdens or introduce greater costs to survey design.

Our findings suggest that modifying these features to minimize the potentially adverse effects of interviewing cellphone respondents while they are away from home may improve respondent cooperation. The challenges of interviewing cellphone respondents increase as the proportion of cell-only and cell-mainly households grows, thereby magnifying the effect of cell respondents who are away from home at the time of interview. Future research should examine the efficacy and feasibility of methods to mitigate the effect of additional burden introduced by interviewing cellphone respondents who are away from home.

Author Note

The findings and conclusions in this paper are those of the authors and do not necessarily represent the views of the National Center for Health Statistics, Centers for Disease Control and Prevention, or NORC at the University of Chicago.

References

Barron et al. 2008
Barron, M., M. Khare and Z. Zhao. 2008. Cell telephone response rates. In: Joint statistical meetings proceedings, survey research methods section. American Statistical Association, Alexandria VA.
Blumberg and Luke 2013
Blumberg, S.J. and J.V. Luke. 2013. Wireless substitution: early release of estimates from the National Health Interview Survey, January–June 2013. National Center for Health Statistics, Hyattsville, MD. Available at: http://www.cdc.gov/nchs/data/nhis/earlyrelease/wireless201312.pdf.
Brick et al. 2006
Brick, J.M., S. Dipko, S. Presser, C. Tucker and Y. Yuan. 2006. Nonresponse bias in a Ddal frame sample of cell and landline numbers. Public Opinion Quarterly 70: 780–793.
Brick et al. 2007
Brick, J.M., P. Brick, S. Dipko, S. Presser, C. Tucker and Y. Yuan. 2007. Cell phone survey feasibility in the U.S.: sampling and calling cell numbers versus landline numbers. Public Opinion Quarterly 71: 23–39.
Curtin et al. 2005
Curtin, R., S. Presser and E. Singer. 2005. Changes in telephone survey nonresponse over the past quarter century. Public Opinion Quarterly 69: 87–98.
Hyman et al. 2009
Hyman, I.E., S.M. Boss, B.M. Wise, K.E. McKenzie and J.M. Caggiano. 2009. Did you see the unicycling clown? Inattentional blindness while walking and talking on a cell phone. Applied Cognitive Psychology 24(5): 597–607.
Lavrakas et al. (2010)
Lavrakas, P.J., T.N. Tompson and R. Benford. 2010. Investigating data quality in cell phone surveying. Paper presented at the 65th annual conference of the American Association for Public Opinion Research, Chicago, IL.
Link et al. 2007
Link, M.W., M.P. Battaglia, M.R. Frankel, L. Osborn and A.H. Mokdad. 2007. Reaching the U.S. cell phone generation: comparison of cell phone survey results with an ongoing landline telephone survey. Public Opinion Quarterly 71: 814–839.
Link et al. (2009)
Link, M.W., C. Daily, C.D. Shuttles, C. Bourquin and L.T. Yancey. 2009. Addressing the cell phone-only problem: phone sampling versus address based sampling. Survey Practice, February 2009. Available at: http://www.surveypractice.org/index.php/SurveyPractice/article/view/162/html.
Long and Freese 1997
Long, J.S. and J. Freese. 1997. Regression models for categorical and limited dependent variables. Stata Press, College Station, TX.

Comments on this article

View all comments


About Survey Practice Our Global Partners Disclaimer
The Survey Practice content may not be distributed, used, adapted, reproduced, translated or copied for any commercial purpose in any form without prior permission of the publisher. Any use of this e-journal in whole or in part, must include the customary bibliographic citation and its URL.