In an effort to boost response rates, survey researchers frequently offer monetary incentives to respondents or potential sample members. Historically, these incentives have been paid via cash or check, and then distributed through direct mail or in-person as a prepaid or postpaid incentive (Singer and Ye 2013). As online surveys have risen in popularity (Daikeler, Bošnjak, and Lozar Manfreda 2020), however, the administration of survey incentives presents a new set of challenges. Mailed incentives can no longer be distributed with the same ease to members of online samples, whose mailing addresses are typically unavailable. Meanwhile, the electronically-deliverable incentives that have emerged, such as e-gift cards, are non-monetary and thus imperfect substitutes for traditional cash incentives (Singer and Ye 2013). Despite this, survey researchers have been slow to employ the use digital monetary incentives, particularly those paid via mobile pay applications, such as Venmo or Zelle. According to one estimate, half of American consumers made digital payments in 2017, up from 40% in 2015 (Greene and Stavins 2018), suggesting that their popularity is expected to rise. Furthermore, unlike mailed incentives, digital incentives can be distributed without mailing addresses, making them uniquely accessible across survey mode.

Despite these trends, there is relatively little consensus around best practices for incentives distributed to online samples (Birnholtz et al. 2004). For surveys with email recruitment and online questionnaires, researchers have primarily relied on e-gift cards (Singer and Ye 2013) and, much less commonly, digital cash incentives, with inconclusive results as to the relative efficacy of each. Lottery incentives (Göritz 2006) are also frequently used, though their prizes can comprise cash, gift cards, and non-monetary gifts. There are a number of conditions under which all of these digital incentives can be administered (Dykema et al. 2013), ranging from the value of the incentive (DeCamp and Manierre 2016), to its timing (i.e., pre- or post-survey, though lottery prizes are always postpaid) (Robbins and Hawes-Dawson 2020), to the store at which the non-monetary incentive is redeemable (Lederer 2019), all of which can shape findings and complicate comparisons across experiments.

According to two studies, mailed cash incentives produce higher response rates than e-gift cards from Target (Brown et al. 2016) and Amazon (Birnholtz et al. 2004) when the cash and e-gift cards are both $5 in value, though Birnholtz et al. (2004) compare cash with e-gift card codes presented in the text of mailed recruitment letters. Still, these results are relatively unsurprising given consistent evidence of respondent preferences for monetary incentives (Helion and Gilovich 2014; Ryu, Couper, and Marans 2006; Singer and Ye 2013). In a comparison of mailed versus e-gift cards, Robbins and Hawes-Dawson (2020) report higher response rates for e-gift cards.[1] In contrast, Birnholtz et al. (2004) find no difference in response rates to mailed versus e-gift cards. Beyond their potential impact on response rates, e-gift cards are also more cost-effective than mailed gift cards (Robbins and Hawes-Dawson 2020) and mailed cash incentives, the cost of which “includes cost of incentive, labor for assembly of incentive mailings, first-class postage and envelope for delivery” according to Brown et al. (2016), who contend that the cost of e-gift cards “includes cost of incentive and labor associated with e-mailing of incentive.”

The effectiveness of lottery incentives is harder to evaluate given the range of prizes used across surveys; lotteries can also vary along a number of other features, such as the timing at which prize winners are notified (Tuten, Galesic, and Bosnjak 2004). To briefly summarize, research on lotteries in web-based surveys has produced mixed results, with some scholars finding no benefit (Porter and Whitcomb 2003) to small benefits of lotteries when compared to no incentive (Heerwegh 2006; Laguilles, Williams, and Saunders 2011; Tuten, Galesic, and Bosnjak 2004). In a comparison of e-gift card and lottery incentives redeemable at an online book and CD store, Deutskens et al. (2004) find no differences in response rates. Finally, the efficacy of electronically-delivered cash has received very little attention in the literature. One experiment found no significant differences in response or completion rates to cash lotteries versus electronic cash incentives (Bosnjak and Tuten 2003), though the electronic cash incentive was worth $2 via PayPal, an important detail considering the sample’s demographic profile.[2] Researchers have yet to examine differences between mailed cash and electronic cash incentives, leaving several questions unanswered.

How do electronically-delivered cash incentives stack up against mailed cash incentives, then, in the eyes of survey respondents? According to our exploratory study on survey incentive payment preferences, results suggest a greater preference for digital transactions. We conducted a small, web-based Qualtrics survey using a sample of interviewers at the University of Wisconsin Survey Center (n=52).[3] After completing the survey, respondents were taken to a separate form and asked to select their preferred payment method for receiving a $5 incentive (see Figure 1). Among the five options included—four different mobile pay applications and cash (direct mail)—over 90% of respondents selected a mobile application (see Figure 2). Venmo, in particular, was the most popular mobile application, preferred by almost 85% of respondents.

Figure 1
Figure 1.Qualtrics survey item asking for preferred payment method
Figure 2
Figure 2.Respondents’ (n = 52) preferred payment methods for post-survey incentives

Given these preliminary findings, as well as the rising popularity of digital transactions more broadly, we encourage researchers to consider the use of digital payments via mobile pay applications for incentives in online surveys. Beyond the popularity of such payment methods, there are several other reasons we advocate for their use. First is cost-reduction: Digital transactions are less costly—mobile pay applications typically do not charge any added fees—and take considerably less time to administer than mailed incentives (Brown et al. 2016; Robbins and Hawes-Dawson 2020). Second, digital payments can be distributed to members of online samples with ease (Birnholtz et al. 2004). There are also limitations to using mobile pay applications to deliver incentives. Respondents must have and use one of the offered applications, and researchers may incur costs related to learning how to use the application and distribute payments. While our findings are compelling, they are far from conclusive, highlighting the need for further research on the efficacy of digital payment methods for the distribution of survey participation incentives.

Author Contact Information

Tiffany S. Neman (lead author)
tneman@wisc.edu

Acknowledgements

The collection of survey data for the Voices Heard Survey was funded by NIMHD grant P60MD003428 (PD: A. Adams). Project: Increasing Participation of Underrepresented Minorities in Biomarker Research (PI: D. Farrar Edwards). This study is based upon work supported by the National Science Foundation (grant number SES-1853094 to J. Dykema and D. Garbarski]. Project: Effects of Interviewers, Respondents, and Questions on Survey Measurement. Additional support was provided by the University of Wisconsin-Madison Office of the Vice Chancellor for Research and Graduate Education with funding from the Wisconsin Alumni Research Foundation, the Charles Cannell Fund in Survey Methodology at the University of Michigan, the University of Wisconsin Survey Center (UWSC), which receives support from the College of Letters and Science at the University of Wisconsin-Madison, and the facilities of the Social Science Computing Cooperative and the Center for Demography and Ecology (NICHD core grant P2C HD047873).

  1. Robbins and Hawes-Dawson (2020) specifically compare Amazon e-gift cards with mailed Target gift cards, a distinction that research by Lederer (2019) suggests is meaningful.

  2. The authors use an online sample of real estate agents and brokers to compare $25–$50 lotteries with a postpaid $2 digital incentive, an amount they contend may be “too low to have an effect… given the nature of this sample, which is trained to think in economic exchange dimensions” (p. 216). Indeed, DeCamp and Manierre (2016) find that only a $5—and not $2—digital postincentive in the form of university credit provides a benefit in participation rates over no incentive at all.

  3. In February 2021, 216 current and former interviewers were invited via email to participate in the survey, which was on a topic unrelated to preferred payment methods for survey incentives, generating a 25% response rate. Respondents fell between the ages of 19 and 36, with a median age of 21.