Strategies for Increasing Asian American, Native Hawaiian, and Pacific Islander Representation in Survey Research

Introduction

The Asian American, Native Hawaiian, and Pacific Islander (AANHPI) population is growing rapidly in the United States (Budiman and Ruiz 2021; Ramakrishnan and Ahmad 2014). Between 2000 and 2019, this population grew by 81% and is estimated to reach more than 46 million by 2060 (Budiman and Ruiz 2021; Paulose-Ram et al. 2017). Given this trend, it is time for researchers to intentionally seek to understand the perspectives of this group and to fully include them in national opinion surveys.

Despite the tremendous growth of the AANHPI population in the United States, however, detailed public opinion data on the perspectives and experiences of this diverse group continues to be lacking. Many national surveys such as the General Social Survey^[1] and the Medicare Current Beneficiary Survey^[2] do not currently report AANHPI response rates—either as a whole or by the numerous and diverse communities within AANHPI—and the publicly available data are not sufficient to generate estimates. Thus, it is difficult to fully understand whether AANHPI participation rates in major national surveys is on par with or lags behind that of other racial and ethnic groups. Raising awareness among researchers of the importance of obtaining and reporting better data for AANHPI populations and developing best practices are foundational to improving AANHPI representation in national level surveys.

Including AANHPI populations in research is difficult given the cultural and linguistic diversity within this population, which is about 6.3% of the total adults in the United States (U.S. Census Bureau 2022). Much of the available survey data on this population is drawn primarily from the English-proficient AANHPI population, given that many public opinion surveys either do not field in languages other than English or Spanish or use frames or panels that have been recruited in English and Spanish only. Beyond limited English proficiency (LEP), addressing cultural differences related to familiarity and willingness to participate in research are key to successfully including AANHPI populations in surveys. Indeed, the broad diversity within the AANHPI population will make assessing the issues associated with research participation and developing methods to address them quite challenging.

There is an urgent need for the research community to identify and incorporate evidence-tested methods that will increase the representation of AANHPI populations in research. Culturally responsive methods such as community-based participatory research, engaging community leadership, tailoring study messages, keeping communities informed about the research process to build trust, and providing culturally responsive translations in diverse languages, can all have significant impacts on representation, data quality, and overall understanding of the many communities that make up the AANHPI population. This article presents findings from a joint research project by AARP and NORC to explore best approaches for engaging AANHPI populations in research. We present the key themes that emerged from a review of the literature and interviews with experts on addressing barriers and motivators to survey participation with members of the AANHPI community.

Methods

NORC conducted a review of the literature and a series of expert interviews focused on barriers to survey participation and strategies for overcoming barriers (Lee et al. 2020, 2021). The results from these research activities form the basis for our findings and recommendations on best practices for increasing AANHPI survey participation. The 15 experts we interviewed included survey researchers, representatives from national advocacy organizations, and academic researchers whose advocacy work or research included East Asian American and South Asian American subgroups as well as Native Hawaiian and other Pacific Islanders (NHPI).

Barriers and Challenges to Survey Participation for the AANHPI Community

The research highlighted several barriers and challenges to enlisting the participation of AANHPI populations in surveys The three main themes were the following: lack of familiarity with surveys and their importance, lack of trust in those conducting surveys, and issues of language and translation.

Lack of familiarity with surveys and their importance. Across AANHPI communities, the widespread lack of familiarity with surveys—and their importance—may be a barrier to AANHPI survey participation. Those with lower levels of education, which is associated with lower levels of English language proficiency and being foreign born, may have lower levels of familiarity with surveys and lower levels of participation (Evans et al. 2019). Focus group research on Census participation revealed that lack of knowledge about the Census, such as the purpose and content of the survey and process of participation, was a barrier for several groups, including for Chinese, NHPI, and Vietnamese (Evans et al. 2019). Barriers are particularly prominent for Asian immigrants who completed any education outside of the U.S., which appears to decrease civic participation, such as voting or volunteering (Chaudhary and Mai 2021). The impact was greater for some Asian American immigrant groups (Korean and Indian) and less apparent for others (Chinese and Vietnamese). This variability underscores the importance of accounting for differences across Asian American groups in understanding the potential impact of educational place or immigrant status on research participation.

Unwillingness or fear of participation may be tied to the lack of understanding of how survey data would be used and by whom. This lack of understanding has been worsened by the frequent practices of researchers who conduct studies in these communities but who never return to share the results or communicate the impact. This approach has left members of these communities with little evidence of the potential value of research to their community, leaving them less likely to participate in future research.

Lack of trust. Members of AANHPI communities with a general distrust of government may have heightened concerns about participating in a survey due to concerns about how the data will be used, by whom, and risks of harm (Evans et al. 2019; McGeeney et al. 2019; Park and Yelei 2009; Sha et al. 2018). Perceptions of the degree of potential harm can be higher for those whose families may have more recently immigrated, with lower levels of education, or lower levels of English proficiency (Evans et al. 2019; McGeeney et al. 2019). This risk of harm is something many may have experienced in their home countries or through recent and historic anti-immigrant policies in the United States (e.g., actions taken by U.S. Immigration and Customs Enforcement, discriminatory treatment of AANHPI), and inflammatory discussions around asking about citizenship on the 2020 Census have led to the fears of many that government data collection efforts are not safe or trustworthy, resulting in lower participation and willingness to identify as AANHPI (Evans et al. 2019; McGeeney et al. 2019).

Issues of language and translation. The diversity of AANHPI languages and the prevalence of limited English proficiency among some AANHPI groups contributes to the difficulty of participating in research. Survey translations are frequently limited to only the most common AANHPI languages, if any are provided at all. Formal translations are expensive and not easily adapted to colloquial speech or differences in language use across groups, nor a range of literacy levels (Pan and Landreth 2009; Park and Yelei 2009; Sha et al. 2018). While in-community interpreters, including family members, can be an option, these approaches are challenging to administer, and different modes have varying levels of interviewer control or assistance. Even in the best scenario, informal interpretation by community or family members results in each respondent having a different translation.

Beyond the diversity of language, as well as dialects within languages, different AANHPI subgroups have unique historical, political, or economic circumstances driving their immigrant experience. These cultural influences result in varied survey taking experiences across the communities. Pacific Islander communities are one example of this—an NHPI expert noted that there are more than 20 languages spoken across these communities, each originating from a place with a different relationship to the immigrant experience and the U.S.

Findings on Strategies to Increase AANHPI Survey Participation

The research revealed several strategies to engage with the community that may increase the participation of AANHPI respondents in surveys, particularly for the smaller, harder to reach AANHPI subgroups. These strategies include (1) working together with leaders, experts, and organizations at the local community or national level, depending on the study focus; (2) tailoring messaging about a study; and (3) providing more culturally responsive translations.

Involve recognized and trusted AANHPI community leaders, experts, and organizations. The trust of the community is required to conduct a successful survey within AANHPI communities. Establishing trust in the community and providing information about the research to address their questions and concerns are key to gaining survey participation. The experts made several recommendations for increasing AAPI community involvement and trust using community-based participatory research (CBPR) approaches with AANHPI populations. For example:

• Partner with reputable community-based organizations (CBOs) on the design, outreach, administration, analysis, and the reporting and dissemination of study findings.

• Have the research reviewed by trusted members or groups within a community and a community Institutional Review Board (IRB) that includes AANHPI researchers.

• Use in-community individuals for face-to-face data collection efforts.

• Give back to the community by volunteering time to support other research and evaluations that address community needs.

While there are several benefits to engaging AANHPI stakeholders, it can be challenging to identify and recruit the right stakeholders to partner with on a survey. At the local level, it requires understanding the infrastructure of a community to identify who can facilitate access to members of the community and for many local AANHPI communities, there may not be a single organization or community leader that is known and trusted by an entire community. Further, when conducting research on a national level, it may not be feasible to build relationships with local organizations at the geographic scale needed for full population coverage. Yet, as discussed below, it is imperative that the research plan should include approaches to tailoring messages to address the community’s questions and concerns and for showing the value and intended outcomes of the study. Researchers must implement culturally responsive approaches to translation of questionnaires and other materials and to take steps to address the unique questions and concerns that AANHPI communities may have about the research. Ideally, AANHPI experts and community leaders should be consulted early in the research process to advise on these approaches.

The Native Hawaiian and Pacific Islander National Health Interview Survey (NHPI NHIS) is an example of a federal survey that successfully integrated community engagement with survey methods typically used on national surveys (Galinsky et al. 2019). Galinsky et al. partnered with community leaders and community members along with experts in the survey planning to design survey materials and interviewer trainings that reflected learnings from the community. They also worked together with trusted community leaders in developing methods for survey outreach and disseminating survey findings to the community.

Tailored messaging about the purpose and value of the study. Central to encouraging survey participation is tailored messaging about the purpose and benefit of the survey or research as it relates to the AANHPI communities (e.g., Bates and Pan 2009; Paulose-Ram et al. 2017; Sha et al. 2018; Sykes et al. 2010). The messaging should convey what the study is about, what a survey is, and how the data would be used. Focusing on the tangible and proximate benefits to the community and clear messaging about how the data will be useful for the community are important. Tailored messaging shifts the narrative from “I’m here for research” to “I’m here to help.” Also, one expert noted that for some groups, it may be important to acknowledge historical events and to assure them that participation will not result in any harm to themselves or their communities. Particularly for immigrant communities who are not usually included, highlighting the opportunity to represent their community by sharing their voice may help to encourage their participation. For example, use messaging that helps people understand that study findings are often reported for only three groups (white, black, and Hispanic) and that participating in a survey is a way for their groups to also be represented in study findings. Knowing the outcomes of their participation can be a motivating factor for some respondents. For example, the Census uses messaging emphasizing that participation matters because it shapes our resources (e.g., how money is distributed to schools).

In a survey sample, it is often not known with certainty which households may include AANHPI individuals, and whether the potential survey respondent may face barriers to participating in the study. Yet, study materials may still be designed to include the information that AANHPI sample members could find helpful. Indeed, additional information on what survey participation entails, the uses of the data and options for contacting the study for further information will likely help to better inform all survey participants beyond those who are AANHPI.

Suggestions for ways to develop better tailoring of messaging about a study include:

• Conduct qualitative research when planning the research study. For example, hold focus groups with local community members or national experts to understand how to address AANHPI concerns in the study messaging.

• Let respondents know that surveys provide a sense of representation for their community and that their voice matters. Participation helps ensure that the opinions and experiences of AANHPI communities are included in study findings which can be used to meet the specific needs of these communities.

• Ensure that critical information that AANHPI sample members need is available in the study materials. Let people know what the study is about, why AANHPI participation matters, and how to get more information about the study. Collaborating with trusted community partners to help disseminate the information can help ensure it reaches more people and is viewed as legitimate and trustworthy when it does.

For a survey conducted within local communities, such as the NHPI NCHS (Galinsky et al. 2019), qualitative research can help the researcher identify community needs and resources, determine ways to give back, and provide tailored information about study benefits to participants. Benefits that could be provided to a local community include programs or interventions and educational materials. Researchers should not engage with a community only to conduct a study and then leave without offering benefits of the research back to the community. The NHPI NCHS (Galinsky et al. 2019), while part of a major federal survey, demonstrated the value in conducting research in partnership with a community. Their methods produced high quality publicly available data that provided detail data for NHPI subgroups typically not represented in surveys. However, for many national surveys, close collaboration at the community level to plan and carry out the research and to provide benefits at the local level may not be feasible. Still, at a minimum, all researchers should identify a plan to partner with community stakeholders, such as national level representatives of the community, to the extent possible and to disseminate the findings so that AANHPI communities can learn what the study results were and the potential uses of the data.

Culturally responsive approaches to translation. A central element for conveying messages about a survey and fielding it successfully is taking culturally responsive approaches to developing professionally translated survey materials. A multistep, participatory, and culturally responsive approach to survey translation is needed to ensure that the translations effectively and appropriately convey the intended information (Pan and Landreth 2009; Sha et al. 2018). The quality of translations will impact both recruitment of respondents with limited English proficiency and the data quality from these respondents. Although survey questionnaires and supporting materials should be offered in the languages needed by AANHPI community members, the experts did not come to a consensus on how many and which languages the translations should cover. Approximately 80% of the Asian American population is represented by the “big six” Asian ethnicities: Chinese, Indian, Filipino, Korean, Vietnamese, and Japanese. Offering languages for these largest Asian American ethnic groups in addition to English (Chinese - Mandarin and Cantonese, Tagalog, Vietnamese, Korean, Japanese, and Hindi) is best practice for a national population sample, though English-only materials may suffice for the Japanese, Filipino, and Indian populations (Lee et al. 2021). The ideal number of translations for any survey will depend on project goals and research objectives and the tradeoffs with practical considerations, such as cost, project schedule, and the extent to which the study population has limited English proficiency.

Ways of enhancing the cultural responsiveness of methods for developing translations include the following:

• Work with translators from the linguistic community itself or experts with strong ties to the community.

• Translate the survey in the languages required by the study and have the translations reviewed by community members and practitioners with expertise with the population. Compare translations to the original survey language to confirm that the translations are correct.

• When translations are provided by professional translation companies, conduct further review and testing of the translations with members of each linguistic community. Include community members with different levels of literacy.

• If necessary, include interviewer notes or definitions for the respondent to explain what potentially unfamiliar terms mean.

Although many surveys do not offer respondents the opportunity to complete the survey in an Asian language, there are some studies that do successfully include Asian language options. The California Health Interview Survey (CHIS) fields in multiple Asian languages, including Chinese (Mandarin and Cantonese), Vietnamese, Korean, and Tagalog. Translations of the questionnaires into these languages entailed a multistep process that included iterative review of the translations by multiple translators and an adjudication process to resolve discrepancies (California Health Interview Survey 2021). AAPI Data fielded a recent survey of voters that offered the option of completing in Cantonese, Mandarin, Korean, or Vietnamese as well as English, and collaborated on a worker survey that includes these languages as well as Hmong and Cambodian languages (AAPI Data 2020; Public Religion Research Institute 2019). Although translations into Asian languages are not commonly available in surveys, the research participation of Asian American respondents who are not proficient in English would likely be improved if it became the standard to provide such translations.

Barriers and Strategies to Survey Participation among AANHPI and Other Populations

An important point that emerged from the expert interviews is that some barriers or challenges to survey participation may not be unique to the AANHPI population. Survey research in general faces falling response rates (Lavrakas et al. 2017). Likewise, while the experts detailed strategies that may increase AANHPI survey participation, several noted that specific strategies may not be needed to encourage survey participation in this population, since many may be eager to be heard. Rather, the focus should be on the methods to identify and recruit AANHPI populations and to make it possible for them to participate (for example, by providing translations). Identification and recruitment of AANHPI populations extends beyond geographic targeting and seeking sufficient sample size. Instead, the key focus should be on meaningful and culturally responsive representation. For example, conducting research in English-only formats but achieving the population estimate for overall representation excludes whole communities within the diverse AANHPI population who cannot be captured this way. Therefore, identification must first be actual representation, not statistical representation, of the whole population of interest—and recruitment efforts must speak to the unique characteristics of every included subpopulation. Indeed, lack of awareness and use of culturally responsive methods, rather than a true lack of AANHPI participation, may be the central issue.

Conclusion

This article presents findings from a literature review and series of expert interviews on the best practices for AANHPI survey participation and strategies to improve representation. Although much evidence is already available on the impact these best practices can have, an important next step is for the broader research community to implement these suggested strategies in collaboration with members of the AANHPI community. As more researchers adopt these strategies and disseminate findings on what works, we will build a body of evidence that will further clarify the most effective, rigorous, and culturally responsive methodologies for increasing representation of all AANHPI populations and communities in research.