Attrition When Dropping CAPI from a CATI/CAPI Panel Survey

Oliver Lipps Swiss Centre of Expertise in the Social Sciences

Florence Lebert Swiss Centre of Expertise in the Social Sciences

Abstract

In this paper, we study attrition in a household panel survey, where in the first wave those with a matched landline number were surveyed by telephone, while those without received a face-to-face visit. In the second wave, the face-to-face mode was dropped. We find among the first wave face-to-face households a high likelihood to attrite due to “no contact” rather than due to “cannot be tracked” or “refusal”. Socio-demographic characteristics have the expected effects. For example households with young children, with a short-term residence permit, or one-person households cannot be tracked, while those with a face-to-face visit in the first wave, or foreigners with a mother tongue that is not offered in the survey refuse more often. More first wave calls and contacts are associated with all reasons to attrite, in particular with refusal. Based on the findings, we give recommendations to tailor fieldwork to decrease attrition.

Context of this Research

There are a number of reasons to drop (or to add!) a survey mode in a panel survey (Lynn 2013). In the case of the Swiss Household Panel (SHP) 2013 refreshment sample (SHP III), the aim of the first wave was to reach as many as possible sample households and convince them to complete the biographical paper and pencil questionnaire. The Swiss Federal Statistical Office (SFSO) drew the SHP III at random from the Swiss population register, matched the names and addresses with its own telephone register, and delivered the registered numbers to the firm which conducts the survey. Note that the matching algorithm does not match all numbers and the SFSO does not provide unregistered numbers. To complete the household grid questionnaire, households with delivered telephone numbers received a telephone call, while the rest received a face-to-face visit (“computer-assisted personal interviewing [CAPI]-households”). In addition to the household composition, the grid questionnaire asks for (not matched) landline and mobile telephone numbers, in particular from the CAPI-households. From the second wave (2014), the SHP III sample conducts the annual questionnaire using computer-assisted telephone interviewing (CATI). The main reason to drop the face-to-face mode was high costs.

This context gives rise to a number of questions, such as:

  1. How many of the CAPI-households provided a landline telephone number (which could not be matched) or a mobile phone number?
  2. To what degree does registered landline match and telephone availability (and disclosure) as well as socio-demographic characteristics have an effect on participation status in the second wave?

Data and Descriptive Statistics

For the analysis, we use the 4,001 households from the SHP III who answered the grid questionnaire in 2013. As depicted in Table 1, 79.7 percent of these households had a matched landline in 2013. The 79.7 percent are larger than the proportion of the households sampled in 2013 that were matched by the SFSO (67.7 percent; see Lipps 2016). This shows that the group of matched households is more likely to be tracked, and/or easier to be contacted, and/or participate more (Sala and Lillini 2014). The complete picture of landline availability from the participating households in 2013 is depicted in Table 1. We list the CAPI households separately and include mobile telephone availability.

Table 1 Landline and mobile telephone availability in 2013.

% All households
(n=4,001)
CAPI households
(n=753)
Matched landline (by SFSO) 79.7 0.4a
Self-reported landline 10.6 51.5
No landline 9.7 48.1
At least one mobile telephone number reported 93.6 94.6

Data: SHP III refreshment sample (Grid completed in 2013).

a These three households were first contacted by telephone and received a face-to-face in a refusal conversion phase.

More than half of the CAPI households reported a landline number, which the SFSO could not match or did not deliver. The proportion of (at least one) reported mobile from CAPI households does not differ from the overall report of mobiles.

Next, we analyze if households distinguished by telephone availability exhibit a different retention behavior in 2014 (see Table 2).

Table 2 Grid completion in 2014 by telephone availability in 2013.

% Mean (std error)
All households (n=4,001) 80.5 (0.006)
With matched landline (by SFSO) (n=3,187) 83.1 (0.007)
With self-reported landline (n=426) 74.2 (0.021)
With mobile telephone only (n=388) 66.0 (0.024)

Data: SHP III refreshment sample (Grid completed in 2013).

The sample with a matched landline participated significantly more than the sample with a self-reported landline. The latter, in turn, are more likely to participate than those with only a mobile phone available.

Multivariate Model

In this section, we analyze the final participation status in 2014, using a multinomial model. The categories of the dependent variable are “participation” (reference category; 79.9 percent of the sample households), “not tracked” (4.3 percent), “no contact” (3.4 percent), and “refused” (12.5 percent).

The main independent variable is the survey mode (telephone vs. face-to-face) and telephone disclosure (self-reported landline and/or mobile) in 2013. In addition, we use characteristics of the households and the household reference persons from 2013 which have been shown to have effects on participation behavior (Groves and Couper 1998; Lipps 2016; Stoop 2005). Finally, we use call data information from 2013:

  1. CAPI vs. CATI (reference category)
  2. (self-reported) landline in the household that was not matched in 2013
  3. Reported availability of at least one mobile phone in the household
  4. Presence of at least one child in the household, and age of youngest child if children present: no child, 0–2 years, 3–6 years, 7–17 years
  5. Reference person is a foreigner from another than a neighboring country (and thus not sharing one of the survey languages)1
  6. Reference person is a foreigner with a short to midterm residence permit (up to B-permit)
  7. Reference person is a foreigner with a permanent residence permit (C-permit)
  8. Reference person is single and never married
  9. Single-person household
  10. Reference person is retired
  11. Living in a big city (more than 100,000 inhabitants)
  12. A deprivation index (number of market goods – max. 7 – not present because the household is too poor to purchase them)
  13. Number of calls in 2013
  14. Number of contacts in 2013

We tested interactions between the survey mode in 2013 and the other independent variables but did not find any significant effects. This is evidence of a uniform effect from the survey mode on attrition for all sample members.

In Table 3, we list the predicted marginal effects. We give a reading example for the row CAPI 2013, and the column “No contact”: the probability of not being contacted in 2014 is ceteris paribus 16.9 percent points higher if the sample household was surveyed via CAPI instead of CATI in 2013. In the interpretation, we take both significance and effect sizes into account.

Table 3 Results of multinomial logit model of final response status in 2014 (predicted marginal effects).

Not tracked (n=172) Tracked, no contact (n=135) Contacted, no cooperation (n=499)
CAPI 2013 0.052 0.169* 0.076
Self-reported landline -0.010 -0.039** -0.052
Mobile telephone -0.007 -0.033** -0.035
Age youngest child 0–2 0.053* 0.013 -0.034
Age youngest child 3–6 0.031 0.002 -0.022
Age youngest child 7–17 0.030* -0.001 -0.021
Foreigner from non-neighboring country 0.003 0.015 0.085*
Short-term permit 0.055* -0.006 -0.014
Permanent residence permit 0.012 -0.018* 0.000
Single 0.009 0.012 -0.003
One-person household 0.037** 0.005 0.003
Retired 0.014 -0.017** 0.032*
Living in big city 0.023* 0.011 -0.014
Deprivation 0.008 0.011** -0.012
Number calls 2013 0.0004** 0.0005** 0.0009**
Number contacts 2013 0.0020 0.0004 0.0069*

Data: SHP III refreshment sample (Grid completed in 2013. n=4,001). *(**) p<0.05 (0.01).

McFadden Pseudo R2=0.059.

As for effects from the mode in 2013 and telephone availability, we find that the CAPI households exhibit an increased likelihood to drop out for all reasons (significantly so only for no contact). These households are likely to be different from the others in respects related to participation (integration, income, socioeconomic conditions, etc.). Those with a self-reported landline or a mobile drop out to a smaller extent due to noncontact or noncooperation. While a mobile phone certainly improves contractibility, we suspect that both groups show trust, are easier to be contacted and cooperate more.

Regarding socio-demographic variables, we find an increased likelihood of an unsuccessful tracking for households with children and in particular those with a small child, or those with a short-term permit as well as those living alone. The likelihood of making contact is not particularly different across the socio-demographic groups, including the deprivation index. Foreigners from a non-neighbor country and to some extent retired people cooperate less.

Concerning fieldwork related variables, those with a higher number of calls are less likely to be tracked, to be contacted, and especially to cooperate. Noncooperation is also high for those with many contacts.

Fieldwork Recommendations

Based on these findings, we give some recommendations to lift the retention rate: Households that switched the survey mode from face-to-face to telephone, those with small children, foreigners with a short-term residence permit, and one-person households need special care to be tracked. These household types are known to move more often than others (Couper and Ofstedal 2009). More tracking effort is likely to be effective and in particular gathering more reliable contact information in the first wave to be able to find these household in the next wave. Again mode switchers, foreigners with a nationality from a country not sharing one of the survey languages and to some extent retired people may be motivated to stay in the panel with an extra incentive. Foreigners could also be approached by bilingual interviewers (Kappelhof 2015). Mode switchers need most care to be contacted. For them and for those with many calls, optimizing calling times (Lipps 2012) could be a good idea.

References

Couper and Ofstedal 2009
Couper, M.P. and M.B. Ofstedal. 2009. Keeping in contact with mobile sample members. In: (P. Lynn, ed.) Methodology of longitudinal surveys. Wiley, New York.
Groves and Couper 1998
Groves, R. and M. Couper. 1998. Nonresponse in household interview surveys. Wiley, New York.
Kappelhof 2015
Kappelhof, J.W.S. 2015. Surveying ethnic minorities: the impact of survey design on data quality. PhD thesis, University of Utrecht, The Netherlands.
Lipps 2012
Lipps, O. 2012. A note on improving contact times in panel surveys. Field Methods 24(1): 95–111.
Lipps 2016
Lipps, O. 2016. Non-observation bias in an address-register-based CATI/CAPI mixed mode survey. Methods-Data-Analysis 10(1): 5–24.
Lynn 2013
Lynn, P. 2013. Alternative sequential mixed-mode designs: effects on attrition rates, attrition bias, and costs. Journal of Survey Statistics and Methodology 1(2): 183–205.
Sala and Lillini 2014
Sala, E. and R. Lillini. 2014. The impact of unlisted and no-landline respondents on non-coverage bias. The Italian case (No. 2014-16). Institute for Social and Economic Research, University of Essex.
Stoop 2005
Stoop, I. 2005. The hunt for the last respondent: survey nonresponse in sample surveys. Social and Cultural Planning Office, The Hague.
Footnote
1 Foreigners from a neighboring country show no differences to native Swiss.


About Survey Practice Our Global Partners Disclaimer
The Survey Practice content may not be distributed, used, adapted, reproduced, translated or copied for any commercial purpose in any form without prior permission of the publisher. Any use of this e-journal in whole or in part, must include the customary bibliographic citation and its URL.