Introduction
New technologies continue to push the boundaries of collaboration, bringing together geographically dispersed people into a single, virtual space. Combined with budget pressures, the availability of high-speed Internet and online communication platforms encourage opportunities to use virtual focus groups. Virtual, or online, focus groups may be either asynchronous or synchronous (Mann and Stewart 2001). In asynchronous focus groups, participants access a site to answer moderator-posed questions or respond to other participants’ comments. Participants in synchronous focus groups interact with the moderator and each other “live.”
Despite conditions favoring their use, comparatively little literature exists on using online focus groups for social science research, particularly webinar-type synchronous focus groups. Mayer, Jeruss, and Parsons (2006) conducted five such groups with caregivers and providers of pediatric patients and found that participants in the chat-based groups were able to share their experiences and express emotions by using emotions available in the conferencing system. Participants’ comfort with the messaging technology varied, and some required assistance with technical problems such as logging on or using the software. Underhill and Olmsted (2003) compared transcripts from face-to-face focus groups and chat-based, online focus groups and found no significant differences in participation rates, the number of unique ideas generated, the total number of relevant comments, or participant satisfaction. However, more off-topic comments were generated in the online groups.
What literature exists on this topic originated nearly a decade ago, and technological innovations affecting this data collection method evolve rapidly. In this paper, we draw from our experiences on two recent studies to describe considerations for using online, synchronous focus groups as well as lessons learned from implementing them.
Methods
We conducted three online focus groups across two studies, both involving hard-to-reach populations: U.S.-based health care professionals and internationally-based information technology professionals. Eight individuals participated across the groups (five were in one group, two in the second, and, after other individuals were no-shows, only one participated in the third). The populations’ geographic dispersion made the traditional, in-person format impractical and opened the door for using online focus groups as a viable alternative. Each group used webinar-style features including audio and presentation slides. The focus group with the internationally-based professionals was part of a larger study that involved in-person focus groups with similar domestic populations; the online and in-person groups used an identical protocol. The protocol for the health care professionals was developed specifically for online use. Experienced moderators conducted all the groups.
Findings
Considerations for conducting online, synchronous focus groups
We considered four major factors when deciding whether to implement online focus groups in place of the in-person format: (1) discussion topics; (2) population characteristics, including respondent location; (3) available technology; and (4) costs incurred (Mann and Stewart 2001). All of these considerations pointed toward online focus groups for our studies. These considerations can be applied more generally when deciding whether an online, synchronous focus group is an appropriate data collection method. The considerations include:
Topic. Sensitive topics can be particularly well-suited for online focus groups. Sensitive topics may include issues related to health or sexual behavior; emotionally charged topics like politics; or illegal activities related to drug use, gangs, or the underground economy. Online focus groups can help facilitate a more honest, open discussion of such topics – and therefore improve data quality – because participants cannot see each other and therefore some anonymity is maintained. For example, we found that the internationally-based information technology professionals were willing to share their experiences with obtaining employment in the United States and abroad during the focus group session, which could be a sensitive topic for individuals known to each other working within the same industry.
Population. The online focus group format is well-suited for a variety of populations, especially those who are “hard-to-recruit” or geographically dispersed. Some of these respondents (for example, parents of young children or busy professionals) are tied up by substantial time commitments which make it difficult to devote time to travel to a facility and participate in a discussion. The online format gives such individuals fewer barriers to participation by eliminating the need for travel.
Similarly, groups with low-incidence characteristics, such as rare diseases or unusual occupations, may not be in close geographic proximity. These groups are therefore easier to bring together online, and doing so increases the likelihood of achieving sample representativeness because recruiting broadly across many locales may yield a sample that is more diverse than one recruited from one city. On the other hand, the in-person format is better suited for populations with less comfort using technology or who have disabilities that make the technology impractical to use.
Technology. Online focus groups are feasible when the moderator and participants have access to and understanding of the appropriate technology. In an increasingly technological world, access to and comfort with technology is becoming a lesser concern. However, researchers must carefully evaluate if their populations of interest will be able to successfully engage with and manage technology. The study team must have access to reliable technology that will meet all of the requirements necessary to successfully engage with a group remotely. Likewise, participants must have reliable access to phones, computers, and the Internet. It is not enough to be able to access the Internet. Both moderators and participants must to be able to navigate the software programs, interact with polling or chat features, and be able to troubleshoot if they experience difficulties.
Costs. We are unaware of any studies comparing costs of online to in-person focus groups, though we hypothesize that the former may be less expensive by eliminating costs for travel to a site, facility fees, and refreshments. Because respondents are likewise not asked to travel to an online focus group, incentives may be smaller.
Best practices for planning and implementing online, synchronous focus groups
A number of best practices ensuring both efficiency and success of online focus groups emerged from our experiences. These best practices can be grouped into three main stages of data collection: preparation for the groups (before), administration of the group on the day of data collection (during), and follow-up procedures after the completion of the groups (after). Figure 1 depicts the key best practices, which are described in more detail below.
Preparation (Before)
A number of best practices in the preparation stage can help ensure that online focus groups proceed as smoothly and efficiently as possible. These preparatory best practices relate to four main areas: (1) recruitment, (2) scheduling, (3) equipment testing, and (4) detailed communication.
Recruitment. As with in-person focus groups, recruiting a sufficient number of participants is key to ensuring that enough attendees contribute to an in-depth discussion that explores the full range of respondent experiences. No-show rates for our online focus groups differed between projects; one study had fewer no-shows, and the other had more relative to our experience with in-person focus groups. Over-recruiting by two or three participants allows for the likely possibility that several attendees will cancel due to unforeseen circumstances. However, keeping the online groups smaller than in-person groups is critical in keeping the discussion manageable; Sweet (2001) recommends six to eight participants. During recruitment, the researcher should stress the importance of the potential participant’s input, that she cannot be easily replaced, and that it is crucial that she is able to attend at the scheduled time. From the onset, researchers should explain the technological requirements for participation and identify any potential barriers (technological or otherwise) to participation.
Scheduling. As with in-person focus groups, aim for a time that works best for the target population. Additionally, it is critical to choose a time that works best across all participating time zones and is also feasible for the moderator.
Testing. Moderators and study team members should test the equipment and software in advance of the focus group, ideally with participants who actually plan to attend the focus group from the location in which they plan to attend. Just as a moderator may test the voice recorder before going into the field, online moderators must test all aspects of the equipment well in advance of the group. Be sure to test phone connection quality, whether the addition of headsets or other audio tools improves the connection quality, and whether the participant is able to log into the focus group session and view the accompanying presentation.
Detailed communication. Finally, providing detailed communication in advance of the group helps make the day of the focus group run smoothly. In contrast to in-person groups, communication for online groups includes details beyond time and place. Include detailed instructions about technology and consent process information. Researchers should send detailed emails confirming the date and time of the groups in the participant’s local time and instructing participants on how to access the webinar, including software or web browser needs, and setting up a user account that protects the confidentiality of the participant. We found that calendar invitations were particularly effective forms of confirmation emails, as they tend not to get lost in an email box and have built-in reminder functions. In addition to confirmations, the study team should still send multiple reminders and ask individuals to confirm receipt.
As part of the detailed communication with study participants, researchers should have a system in place for obtaining consent prior to the group if possible. In our studies, we emailed the consent and honorarium forms to the U.S.-based participants three days prior to the focus group for review and signature. Participants used a secure fax to return signed copies to project staff. The pre-consent process was more complicated for our internationally-based participants due to the cost associated with faxing the form. In this case, we emailed the consent forms in advance of the groups for review, then obtained verbal consent at the start of the session.
Administration (During)
As described earlier, our two studies utilized webinar technology to administer the focus groups. While there are a myriad of web-based software available to use, we found that GoToWebinar and WebEx met our needs. For the U.S.-based health care professionals, we used GoToWebinar, which allowed us to poll and chat with respondents and view individuals’ names in a participant panel. We used WebEx with the internationally-based information technology professionals because its callback feature allowed participants to call the conference line at no cost to them. This software also includes a chat function to communicate with participants on-screen, and a participant panel to view names. Discussing the merits of using these software systems overothers is beyond the scope of this paper and requires further exploration. However, three key best practices in online focus group administration emerged: using slides, monitoring participation, and employing a round-robin discussion technique.
Use of slides. Slides are an effective means of reinforcing the topic of conversation visually throughout the discussion. Researchers can use slides to support talking points for each topic, including introductory materials, consent, and questions that are a part of the protocol. Even if the participants provided written consent in advance of the group, presenting consent materials can remind participants of their human subjects protections. If consent is to be obtained verbally, it is critical to visually display the consent text word-for-word as it appears on the consent form. At the conclusion of the consent statement, each participant is called on by first name only and asked to indicate they either consent to participate, or drop off of the call. The moderator should have a plan in place to deal with latecomers, who arrive during or after the consent process. Consider re-starting the consent process for late participants or asking them to leave the group if the discussion is well underway. After consent is obtained from all participants, the group can proceed to the protocol questions.
In displaying questions on the slides, keep slides simple, straightforward, and display questions the same way in which the moderator would pose it for discussion. The slide text should not include probes or conditional follow-up questions.
The moderator may navigate forward to the next question only after discussion related to the displayed question is complete. Displaying questions in this format aids respondents to keep on topic and keeps them engaged in the conversation.
Monitoring participation. Most webinar software displays an attendee list in moderator’s view of the screen. This participant list can be used to identify whether or not expected participants have logged on or whether or not any unexpected participants have accessed the webinar. Having a list of attendee names can help the moderator to manage speakers during the course of the presentation. The list of attendee names, while convenient, is a way for the other participants to see who else is participating. It is very important that participants are instructed to only enter their first names (or pseudonyms) during the registration portion to ensure anonymity remains among the participants.
Many webinar platforms may also allow the moderator to determine if participants are potentially multitasking by indicating whether the webinar window is “active.” This information may prompt the moderator to take corrective action.
Round-robin questioning format. Participants do not have the same nonverbal cues available as they would during an in-person discussion in order to interpret signals from other participants that they are about to speak. To avoid participants talking over each other and ensure that all individuals have the opportunity to participate, a round-robin approach can help facilitate a smooth conversation. Calling on respondents by name can discourage respondent multitasking. Moderators can let participants know that they will be called on individually. Respondents can then either answer when called upon, or say “pass” if they do not wish to answer. The downside to a round-robin approach is that participants have less of an opportunity to engage in discussion with each other about issues others have raised. To mitigate this issue, at the conclusion of each question, provide an opportunity for participants to share additional comments based on the information yielded during discussion.
Follow-up (After)
The moderator for an in-person focus group ensures that participants are paid and that the recording of the audio is saved to a secure location. The online focus group is similar in this regard, though the remote nature of the group creates additional considerations.
Incentives. Both studies compensated participants for their participation. The U.S.-based professionals were mailed a check shortly after completion of the focus group; because participants were based domestically, there were no barriers to mailing payments. In contrast, checks and traditional gift cards were not feasible for the internationally-based group. We instead emailed electronic gift cards with a “read receipt” to ensure delivery. This enabled the group to receive their incentive quickly.
An advantage of electronic incentives is that they can be delivered rapidly by email and avoid international shipping costs. They were also well-received by our participants. If necessary, double-check that the gift card will come from a vendor at which out-of-country participants are able to redeem the gift card.
Recordings. Ensuring the recording of the discussion is stored securely is a critical and urgent step immediately following the focus group. Many webinar-style software platforms have the ability to record, which removes the necessity of using an audio recorder in addition to the software. However, it is important to check where the recording is saved, whether its level of security meets the project requirements, and whether it can be downloaded and saved to a secure location. Many software packages save the audio recording to the cloud, necessitating an immediate download of the file after completion of the focus group to a secure server.
Conclusions
Our experiences conducting online, synchronous focus groups with two different populations demonstrate its promise as a data collection method. Although virtual groups differ in key ways from in-person focus groups, such as the reliance on nonverbal cues and group management techniques, we found other strategies to be effective for ensuring participants were engaged in the discussion. We recommend scheduling comparatively small groups of participation, asking round-robin questions, and using presentation slides or other visual elements.
Although more research is needed to compare the cost and quality of data collected from in-person and virtual focus groups, survey researchers may find the latter an effective tool for many of the same purposes as the former. These include investigating a phenomenon in detail as part of instrument development, soliciting feedback on newly drafted questions, or supplementing quantitative data with rich, descriptive information as part of mixed-methods study.
Acknowledgements
The authors thank Tessa Kieffer and Nyna Williams for their contributions to this work.