Recruiting Hard-to-Reach Populations: The Utility of Facebook for Recruiting Qualitative In-Depth Interviewees

Introduction

Social networking facilitates new ways of contacting hard-to-reach populations. Facebook – now ground zero for social media – provides previously unimaginable cost-effective approaches to participant recruitment. Indeed, aware that “social scientists have sought to utilize it for participant recruitment and data collection” (Rife et al. 2016), we turned to Facebook to solve our puzzle-at-hand: Given limited resources and the need for microtargeting, how could we efficiently recruit in-depth interviewees from a hard-to-reach population?

More specifically, for qualitative research investigating the policy implications of gray divorce (divorce at or over 50 years old), we were challenged to recruit a pool of qualified respondents from which to secure 80 interview subjects of geographic diversity divided evenly by sex. Although gray divorce is increasingly common – approximately one in four of all domestic divorces now occurs within this age group – the incidence rate of individuals in the general population meeting these criteria renders traditional recruitment methods cost-prohibitive and/or too slow; indeed, the common theme behind recruitment through advertising in magazines, posted notices, or telephone or mail contacting, is that they are all either too expensive and/or too time-consuming.

With limited resources and the need for microtargeting, we turned to Facebook, where, within 13 days spread out over four waves, we recruited 178 qualified respondents from which we successfully selected 80 interviewees while preserving geographic diversity. We specified the characteristics of Facebook users who would be exposed to our solicitation: geography (United States), age group (50–65+), gender, placement of the advertisement on a mobile device (news feed) and placement on a desktop (news feed or right column of the screen).

Once a respondent clicked on our Facebook ad, she or he was redirected to an online screener we administered directly. Overall, we enrolled 178 qualified respondents at a mean cost of $1.18 per respondent for total recruitment costs of $210.04. Compared to other approaches, the cost-efficiency is exceptional. This article details this fast, cost-efficient methodology for using social media to recruit a low-incidence, hard-to-reach population for qualitative in-depth interviews.

Literature

On Hard-to-Reach Populations

In the late 1940s, Lundberg and Larsen (1949) investigated whether the personal characteristics and views of survey nonrespondents differed from those of respondents. In so doing, they concluded that the problem of nonresponse is widespread, since some individuals are difficult or impossible to reach. Similarly, and beyond probability sampling, in using respondent-drive sampling (RDS) to recruit prostitutes in Manchester, England for interviews about drug use, Faugier and Sargeant (1997) highlighted the benefits of a nonrandom sampling approach when working with hard-to-reach populations. Such a method, they advised, works in the absence of an adequate sampling frame or when respondents are prone to hide their involvement because of a topic’s sensitivity.

Managing cost-effective data collection or recruitment from low-incidence, hard-to-reach populations is a challenge for which instructive lessons can be drawn from literature on other sampling techniques. Discussing RDS under conditions reasonably similar to ours, Teitler, Reichman, and Sprachman (2003) stressed the need to allocate data collection resources efficiently when searching for the hard-to-reach; Link and Burks (2013) demonstrated that tailoring data collection designs when contacting the hard-to-reach can help “increase participation and reduce costs – or at least allow for more efficient use of a fixed pool of financial resources.”

On Recruiting Through Social Networking

With the rise of social networking, researchers are realizing new ways of contacting the hard-to-reach. Facebook’s potential as a medium to locate the hard-to-reach was demonstrated in a 2015 Survey Practice article, “Facebook as a tool for respondent tracing” (Schneider, Burke-Garcia, and Thomas 2015). Remarkably, the authors noted that at the time of their article, over “160 million people in the United States ha[d] Facebook accounts,” and unlike a person’s home address, telephone number or email, those profiles are substantially stable over time (Schneider, Burke-Garcia, and Thomas 2015).

Using Facebook’s huge membership pool, those authors detailed the process of contacting former foster care youth who had been randomly selected for a longitudinal evaluation of a prior intervention. A major challenge of the survey was locating the intended respondent, especially those who had “run away” from home as their 18th birthday approached. After combing public records failed, the researchers searched through Facebook by examining “individual profiles for clues to the Facebook users’ identities,” such as birth date, high school, and physical location. Eventually, the researchers were able to locate and interview 20 of 33 (slightly over 60%) of the respondents searched through Facebook (Schneider, Burke-Garcia, and Thomas 2015).

From simple follow-up field interviews to respondent-driven sampling and differential incentives, methods of contacting the hard-to-reach have evolved and become more efficient. Today, given its enormous following, Facebook offers researchers further opportunities to continue and expand this positive trend by providing previously unimaginable and cost-effective ways of connecting with hard-to-reach populations, whether for direct data collection or, as in our case, recruitment of in-depth interviewees.

Recruitment Protocol and Outcomes

For our qualitative project, we needed to locate and recruit 80 interview respondents, divided evenly between men and women with nationwide geographic dispersion, who experienced divorce at or over the age of 50. Our main concern was that while currently approximately 25% of all divorces in America are gray, the incidence rate is still nonetheless so low in the general national population as to stymie traditional in-depth interviewing recruitment efforts.

Our qualifying criteria for interviewees were as follows:

1. 50 years of age or older;
2. divorced, with the divorce (whether a first, second, or later divorce) having taken place at or after the age of 50 or older; and
3. English-speaking.

These restrictions meant that our targeted respondents were hard-to-identify and hard-to-sample. Two standard methods of reaching this type of population include screening surveys and recruitment advertising. The first method involves using telephone or mail surveys to locate potential respondents. Screener questionnaires are then employed to identify persons hidden within the general population, but who qualify for the study (Teitler, Reichman, and Sprachman 2003; Tourangeau 2014). As previously noted, due to the low incidence of this group in the general population, surveys of any type were simply cost-prohibitive.

We then investigated the possibility of advertising, for example, in The AARP Magazine, but this option was both extremely expensive and would limit exposure to those organizational members. We also researched using public posters to recruit participants at locations across the country; that approach, however, was rendered unmanageable under cost-constrained circumstances by the fairly extensive coordination efforts required to properly locate and maintain those notices. Once again, these measures proved impractical.

We therefore sought out alternative forms of respondent recruitment, and more specifically, began to examine social networking sites as vehicles for identifying our target sample (Bhutta 2012; Rife et al. 2016; Schneider, Burke-Garcia, and Thomas 2015). Facebook, the obvious first choice, appeared promising in that we could display our recruitment notice with relative efficiency to those users in our desired geographic regions who met the age requirement for having experienced a gray divorce.

Indeed, the outcome well-exceeded our expectations: Over four waves that together totaled 13 days, we were contacted by 178 presumptively qualified respondents at an average cost of $1.18 per respondent, for a total recruitment cost of $210.04. Broken down in further detail, this total cost reflects the number of times the study advertisement is “clicked” multiplied by the “cost per click.” The “cost per click” varies based on the selection criteria we used to locate our sample. In this study, it ranged from $0.21 per click for the first wave of data collection from both sexes, to $0.09 per click for the third wave of men only. Summary statistics indicate that over the course of this study, our advertisement was presented to 25,183 Facebook users who clicked a total of 1,325 times at a mean cost of $0.15852 per click (resulting in the total cost of $210.04).

Apparently in order to maintain a particular aesthetic, Facebook places strict guidelines on the use of images in all advertisements (Figure 1). The key restriction is that any image may only be 20% text, and Facebook provides a grid tool to test potential advertisements for the text-image proportions. For academic institutions or well-known products, this can be a limitation because of the brand equity of the institutional or commercial logo. We therefore developed two alternatives, one that is a “tight shot” of the informal Rutgers University seal (Figure 2), and another that is a picture of a Rutgers University monument sign on the flagship New Brunswick Campus (Figure 3). In both instances, these two images allowed us to represent Rutgers’ identity without violating Facebook’s rules.

Figure 1   Facebook text rules.

Figure 2   Recruitment advertisement: informal university seal.

Figure 3   Recruitment advertisement: university monument sign.

The institutional seal-as-visual study advertisement was simple; the title indicated the principal investigator’s institutional affiliation (Edward J. Bloustein School of Planning and Public Policy, Rutgers University), and showed the Bloustein School logo and date. The ad displayed the Rutgers University shield as the central visual element reading, above the visual, “Rutgers Professor seeks volunteers for study on divorce experienced at age 50 or older” and below the visual, “If you had a divorce at or after 50 years old, and would like to participate in this interesting academic study, please click the link to learn more about the project” (Figure 2). Similarly, the monument sign study advertisement contained the exact same narrative, only with a classic Rutgers University monument sign substituted as the central visual element (Figure 3).

If a potential respondent clicked through the advertisement, she or he saw the following text on the first page of the online screener questionnaire:

• My name is Jocelyn Elise Crowley, and I am a Professor of Public Policy at the Edward J. Bloustein School of Planning and Public Policy at Rutgers, The State University of New Jersey. Thank you for your interest in participating in this study. I am studying the experiences of women and men who divorced at or after the age of 50. To do so, I am asking for volunteers, who divorced at or after the age of 50, to participate with me in a one-hour telephone interview. This project is fully confidential, and has been approved by the Rutgers University Institutional Review Board. Confidential means you will not be identified in any resulting publications or reports that come from my research and that no one will ever know that you participated. However, to schedule your interview at a convenient time for you, I will need to contact you. So that I can do so, kindly provide the following information.

To screen for study qualification, potential respondents were then asked, “Did you divorce when you were 50 years of age or older?” Those who answered negatively were thanked for their interest, advised they did not meet the criteria for participation, and exited from the process. From those answering affirmatively, we collected their self-reported first and last names, sex, preferred email address, telephone numbers, and preferences regarding time and day of the week for the one-hour interview. They were then advised that the researcher would be contacting them over the next few weeks to schedule an interview.

After performing a second, confirmatory screening review, and taking into account dropouts and/or other failures to qualify, 167 of the original 178 presumptively qualified respondents constituted the final recruited sample. We then interviewed 80 of these qualified respondents.

Between July 23, 2014, and February 9, 2015, we collected four waves of data; we used waves for two primary reasons: First, we could constrain the number of “fresh” contacts at any given time so that the primary investigator could contact and interview respondents in a timely fashion. Second, we initially believed that we only needed 60 interviews, 30 with men and 30 with women. However, after that number of interviews, we had not reached data saturation, i.e., we were still learning new content from our respondents about their divorce experiences (Hennink, Hutter, and Bailey 2011). We therefore conducted two more waves of data collection, of which wave 3 was men only, and wave 4 women only, bringing us to a total of 80 interviews (40 of each sex).

Table 1 describes the disposition of each recruitment wave. Moreover, since Facebook reaches all across the United States (if not the globe), we were able to reach qualifying respondents across a wide variety of locations (Figure 4).

Figure 4   Geographic distribution of qualified Facebook recruited respondents.

Limitations and Extensions

This protocol is highly adaptable to a wide variety of recruitment circumstances beyond in-depth interviewees, including focus group recruitment, message and/or question pretesting, and nonprobability surveys. While the implications for researchers seeking hard-to-reach/hard-to-contact populations are profound, the approach is not without limitations.

One potential limitation is that Facebook allows respondents to engage with our advertisement content by sharing the recruitment notice with “Facebook friends” and connections online. While this may be favorable, resulting a second-level de facto form of respondent-driven sampling, it may also be undesirable in that some intra-Facebook-user communications may unfavorable to, or may undermine the purpose of, the advertisement. While it does not appear that this limitation affected our effort, it is worth noting.

Another, more specific limitation is raised by the “basic question about the effect of recruitment process on … representativeness” (Belson 1960). While our qualitative study made no probability-sample-backed effort to generalize to the national population of gray divorcees, we were concerned with the potential for recruitment bias. Our simple-but-effective effort to address recruitment bias centered on gathering more than twice the required number of interviews and, within the larger set of recruited potential respondents, balancing the interviewed sample, as best as possible, on our selection criteria. Still, and with Belson’s caution in mind, recruitment bias is the most prominent limitation of this approach, taking two forms. The first is akin to sampling population undercoverage, i.e., that non-Facebook users cannot be contacted through this method; the second, endemic in virtually all sampling approaches, is self-selection bias: to be sure, “self-selection bias is possible even if the research has used random sampling techniques” (Costigan and Cox 2001).

To illustrate the undercoverage issue, out of the general U.S. population as of January 2017 there were 32.3 million Facebook users in the 45–54 year old age range; 14.1 in the 55–64 range, and 20.2 in the 65 and over range, aggregating to 66.6 million Facebook users in the 45 and over group (Statista: The Statistics Portal 2017). The Census Bureau projected that group, at July 2017, to include 142.7 million persons^[1] (U.S. Census Bureau, Population Division 2014); the implication is that approximately 53% of that age group could not have been reached through Facebook-based recruiting alone.

Another potential limitation is self-selection bias, which is typically a function of a relationship between the subject matter of a study (as reflected in the content of the recruitment approach) and the interests and motivations of a potential respondent. It is a core principle of sampling and bias analysis that if “selection is correlated with respondent characteristics in variables important to the study, it can undermine the representative nature and bias the sample on which estimates are based” (Shapiro-Luft and Cappella 2013). Moreover, “the sample selection process under nonprobability sampling is subjective, which does not guarantee the elimination of selection biases” (Lee 2009).

Given the research design and core questions for our study, we were not concerned about this potential bias, largely because the subject matter was not related to the propensity to be a Facebook user (Dennis 2010). However, other qualitative studies, particularly those involving Internet and social media use, sociability, communication propensity, and the like, are advised to explore the possibility of self-selection bias at the time they design their recruitment protocol.

One suggestion to reduce Facebook sampling self-selection bias, which would not have been practical in our hard-to-reach and hard-to-contact circumstances, is to omit the selection criteria from the recruitment notice (Springer et al. 2016). Still, we note that prior research studying divorcing couples, albeit not based on Internet recruiting, found that men who participate “are more invested in their fathering role” (Costigan and Cox (2001) citing Boyd (1985); for a quantitative perspective, see “Assessing and compensating for self-selection bias (non-representativeness) of the family research sample,” Braver and Bay (1992)). Applying this observation to our study suggests that those with more extreme experiences – both positive and negative – may have had a greater propensity to respond to the recruitment notice. This concern was addressed in our project’s substantive analysis by balancing the use of interviews from respondents with different gray divorce experiences (Crowley 2018).

This recruitment protocol need not be a replacement alternative to traditional sampling approaches but could be deployed as part of a dual frame to reduce the costs of traditional methods, such as telephone recruitment; moreover, traditional methods may still be needed to ensure a representative sample on the variables of interest.^[2] Such efforts may take many forms, including combining recruitment with outreach to community groups, with print advertising in media of interest to the hard-to-reach/hard-to-contact population, as well as with telephone recruitment based on zip codes in which the target hard-to-reach/hard-to-contact population is concentrated (Willis 2015).

Qualitative researchers would be well advised, particularly in nonprobability sampling, to study the impact of the limitations we have noted. Useful additional research could include comparisons of probability-based to nonprobability sampling methods within the same research protocol, coding assessment of substantive outcomes based on different recruitment methods for the same research question, and similar comparative techniques that vary recruitment approaches while controlling for other aspects of the study.

Conclusion

We have detailed an easy and cost-efficient approach to recruiting for qualitative in-depth interviewees from a hard-to-reach and hard-to-contact population, which easily permits stratification by geography and gender. While this recruitment method is not immune to the potential limitations inherent in all sampling approaches, social media recruiting in general – and Facebook in particular – is applicable to a wide variety of research contexts and represents nothing less than a change in ease, speed, and economic efficiency in this practice area.

Acknowledgements

This work and the underlying qualitative research project were supported by Rutgers University’s Edward J. Bloustein School of Planning and Public Policy’s Michael J. and Susan Angelides Public Policy Research Fund. M.B. Crowley, M. Patrick Simon, and K. T. G. Weiner-Simon provided ongoing support and encouragement.