A Review of the Cross-Cultural Survey Guidelines Website

By Beth-Ellen Pennell, Kirsten Alcser and Janet Harkness Reviewed by Tim Johnson, Director, Survey Research Laboratory, University of Illinois at Chicago

Earlier this year, a set of best practices for the conduct of cross-national and cross-cultural survey research were announced during the Berlin International Conference on Survey Methods in Multinational, Multi-regional, and Multicultural Contexts (http://www.3mc2008.de/). Available at: http://ccsg.isr.umich.edu/, these recommendations are intended to provide guidance to researchers who might be developing plans for cross-national, and other forms of cross-cultural, survey research.

The conduct of cross-cultural survey research has rapidly grown in importance over the past several decades, and there are an increasing number of ongoing multi-national collaborative survey studies now being fielded on a regular basis by governmental, business and academic consortiums. With this increased activity, practical knowledge and experience are also being accumulated, although there have been few attempts (although see Harkness et al. 2003) to systematically organize this information and make it available to the larger survey research community, until now.

This project was no small undertaking. The efforts of working groups who contributed to these guidelines had been ongoing for several years. They grew out of a CSDI (Comparative Survey Design and Implementation) initiative spearheaded by Beth-Ellen Pennell (University of Michigan) and Janet Harkness (University of Nebraska & ZUMA, Mannheim, Germany). Thirteen additional authors from multiple institutions are also credited with having contributed to these guidelines, which have been further vetted by an impressive list of reviewers.

The guidelines are organized into a series of modules that cover the various steps necessary when developing, conducting and processing data from a cross-cultural survey. Some of these modules deal with topics that will be very familiar to survey practitioners (e.g., sample design, questionnaire design, pretesting), albeit tailored to address cross-cultural considerations. Other modules, in contrast, focus directly on the additional methodological and operational challenges that must be confronted when conducting survey research across cultures (e.g., translation, adaptation, harmonization of survey and statistical data).

Although different in format, these various modules can be thought of as independent book chapters (although structured very differently) that enable readers to quickly locate the specific information of interest. Thus, while anyone contemplating a cross-cultural research project would greatly benefit from a thorough familiarity with each of the recommendations contained in this document, those needing advice on very narrow matters should be able to find what they are looking for very quickly.

The integrative nature and complexity of cross-cultural survey work is clearly recognized here. There is something in these guidelines for all collaborators in cross-cultural surveys. The needs of survey methodologists, survey technicians, and survey administrators are all addressed. While most of the available survey literature tends to focus on methodological topics, and there are some good technical manuals available, there are relatively few easily accessible documents that directly consider the administrative aspects of survey research practice. And, as with all other aspects of cross-cultural survey research, their administration is also likely to be far more complicated and challenging than a typical mono-cultural project. It would have been very easy to overlook this all-too-often-ignored topic when constructing these guidelines. The organizers are to be commended for acknowledging its obvious importance to successful cross-cultural collaborations.

Each module is organized in a practical format that directly confronts relevant issues. Following a brief introduction, guidelines are immediately presented. A clear rationale is formally presented for each, followed by the specific procedural steps necessary for its implementation. Each guideline concludes with a brief section on “lessons learned.” Although the specific format varies some across the modules, these lessons typically present brief case study examples and/or tips from the experts regarding potential obstacles that researchers may encounter when following each recommendation. By themselves, these lessons learned make for excellent reading. Within the context of each module, they highlight the importance of each guideline and help the document avoid the common pitfall of reading like a dry checklist of required activities.

The guidelines are also carefully documented, as each module includes appropriate references and a glossary of relevant terminology. Some modules, such as the one on Survey Instrument Design, also contain one or more appendices. Others contain additional lists of recommended reading on specific topics. The authors have provided a valuable service, all-the-more-impressive in that it is freely accessible to all on the web, rather than being available only as an expensive text.

Some might challenge the need for, or appropriateness of, cross-cultural guidelines, citing the principle that none have the right to dictate standards of conduct to researchers working in other cultural contexts (other critics might go even further and argue that survey research itself is an inherently Western methodology that is in some instances systematically imposed on less-quantitative cultures – this equally important issue can be addressed some other day). I would like to briefly defend them, however, by first noting that the authors of these guidelines do continually acknowledge that various elements of the survey process may well need to be adapted, albeit in a manner that preserves the scientific rigor of the cross-cultural elements of the survey, to local cultural conditions. As Thomas Kuhn (1962) has observed “men whose research is based on shared paradigms are committed to the same rules and standards for scientific practice.” These guidelines adhere to the spirit of Kuhn’s basic principle while also respecting and valuing the cultural diversity that drives this form of research.

In summary, the guidelines provide a valuable service to the profession, well above-and-beyond what is currently available in the published literature. Even experts in the field of survey methodology and/or cross-cultural survey research practitioners will find new and useful information here. For persons looking for a basic introduction to the practice of cross-cultural survey research, this is also a great place to start.

This document is an ongoing project, and a few sections remain incomplete as of October 2008. It currently exceeds 300 pages in length and will hopefully continue to be updated as new knowledge accumulates. Some may view the length as a barrier to use, but as suggested earlier, the various modules are stand-alone pieces and each is actually quite reasonable in length (none currently exceeds 20 pages in length). I would strongly urge readers to take advantage of this valuable resource, which concentrates in one place, arguably for the first time, a set of recommended practices for collecting cross-cultural survey data.

References

Harkness et al. 2003
Harkness, J., F.J.R. van de Vijver and P. Ph Mohler. 2003. Cross-cultural survey methods. Wiley, New York.
Kuhn 1962
Kuhn, T. 1962. The structure of scientific revolution. University of Chicago Press, Chicago.


About Survey Practice Our Global Partners Disclaimer
The Survey Practice content may not be distributed, used, adapted, reproduced, translated or copied for any commercial purpose in any form without prior permission of the publisher. Any use of this e-journal in whole or in part, must include the customary bibliographic citation and its URL.