1) Rory, thank you for joining us today and for sharing your insights on the evolution and impact of the European Social Survey over the past two decades, as well as your vision for its future. Could you start by describing the origins of the ESS and how it has evolved over the past 20 years?

In the 1990s, European scholars, led by German political scientist Professor Max Kaase, made the case for a new high quality, comparative general social survey through the European Science Foundation. It was argued that a new survey was needed because existing data were not considered truly comparable across countries. A blueprint for the survey was developed, and funding was obtained for central coordination from the European Commission and over 20 national funders. Led by a strong Central Coordinating Team to ensure an input harmonised survey, the first round of fieldwork took place in 2002. In 2003, data were released publicly and freely with equal access to all potential users for the first time.

Over time, the ESS has evolved into a European infrastructure with over 230,000 registered users as of October 2024 and almost 7,000 publications having analysed our data. In 2013, the ESS became an independent legal entity (ESS ERIC). In 2024, 28 nations are members of the ESS – the highest of any ERIC across scientific disciplines – and they provide stable funding for its central operations.

2) What are some of the key lessons learned from designing and implementing cross-national surveys like the ESS? How can these lessons be applied to improve survey practice?

Input harmonisation has been the key concept shaping ESS data collection, ensuring that the same or at least equivalent methods and survey questions are used when collecting the data in different countries. The founder director of the ESS, Roger Jowell, referred to it as ‘one survey in over 30 countries’. This approach helps the end data user by ensuring transparency and reducing the chances that methodological artifacts distort the data or that post harmonisation routines mask the impact of different data collection approaches. For example, in the early days of the ESS, modes other than face-to-face interviews were not permitted, as mixing modes within or between participating countries was known to introduce mode effects.

Other areas where the ESS has provided important innovations are the use of equivalent sampling designs based on effective sample sizes (approved by a central team before fieldwork to prevent deviations); a conceptually organised questionnaire design procedure (informed by multiple methods to ensure a cross-national and cross-cultural framing); a committee approach to translation that avoids the pitfalls of back translation; shared planning and communication between the central and national teams to aid compliance with specifications; detailed recording of the contact and response process, which facilitates comparative analysis; and clear data processing protocols that lead to free and immediate access to the data for all, supported by clear, transparent documentation.

Collectively these innovations have underpinned high quality data collection and dissemination to support comparative analysis. Whilst many of these methods are focused on cross-national surveys, they also serve as a reminder that many national surveys are cross-cultural. Ensuring equivalence is important in these surveys too.

If you have limited resources for a cross-national project, you need to consider where the biggest risk to comparability might come from and focus efforts there. Getting the translations to be equivalent can be especially important, so try to follow best practice here whenever possible.

3) What have been some of the major challenges the ESS has faced during its operation, and how have you addressed them?

In the early days of the ESS, ensuring funding at the international and national level was a real struggle, requiring lots of grant writing by the Core Scientific Team (CST) and the ESS national coordinators. In 2013, the ESS was successful in its application to become a European Research Infrastructure Consortium (ERIC). This structure ensures the core ESS central activities are funded without new competitive applications. This has allowed the central team to focus on other tasks and helped to create a more strategic approach to operating the ESS. Unfortunately, many national ESS teams still have to apply for funding, but we are working with the members of the ESS ERIC General Assembly to try to provide more stability in future here too.

As an ERIC based in the UK, Brexit posed challenges with the potential that the ESS might have to leave its UK home. Lots of contingency planning was needed, but with the UK now associated with Horizon Europe, the ESS can remain in the UK for the foreseeable future.

The biggest challenge to the ESS has undoubtedly come from the decline in face-to-face interviewer capacity in Europe. Prior to the COVID pandemic, we were becoming increasingly concerned about falling response rates driven by increases in refusals and high non-contacts due to poor coverage of addresses. The pandemic led to ESS experimenting with new methods, which have transformed our future. More on this later in the interview!

4) What are some effective strategies for designing questionnaires that are culturally sensitive and comparable across different national contexts?

Diana Zavala Rojas at UPF and I wrote a book chapter about the ESS questionnaire design process ("A Model for Cross-National Questionnaire Design and Pretesting" , 2020). The approach used has been carefully designed to ensure it is conceptually driven whilst allowing appropriate cross-cultural input. Whilst the simultaneous development of questionnaires in all the languages in which the survey will be fielded might be optimal, it is rarely practical. We therefore start by designing a single source questionnaire in British English, working with a cross-national questionnaire design team consisting of substantive experts and questionnaire design experts. We also consult our national teams in every country throughout the process and conduct quantitative and qualitative testing at key design points. Advance translation and translation question ‘annotations’ also assist in developing an instrument that can be well translated. This combination of activities strengthens the ability of the instrument to capture cross-national differences.

5) What are some of the innovative methods the ESS has adopted for data collection in recent years?

A key innovation has been the development of the ESS ‘web-first’ panel known as CRONOS. Now entering its third cycle, panelists are recruited at the end of the ESS interview. Those taking part in the panel agree that their original ESS responses can be linked with data from further short surveys. Most data collection is online, but a paper self-completion questionnaire is offered to the offline population. The panel now includes countries from across Europe: Austria, Belgium, Czechia, Finland, France, Hungary, Iceland, Poland, Portugal, Slovenia, and the UK. Recruitment and subsequent response rates have exceeded our expectations. The ESS aims to further expand the panel and embed it permanently within the ESS infrastructure in the future.

6) How does the ESS balance introducing innovative methods with maintaining existing approaches that help maintain the comparability of the data over time?

This is always a difficult trade-off, especially as the ESS aims to allow change over time to be measured but also seeks to develop and use the best methods for cross-national survey research, which, of course, evolve over time. In terms of questionnaire wording, the ESS has tried to avoid changes unless they are essential, as we know that small changes can have a big impact on estimates. In other areas where we introduce innovation, we aim to ensure that changes are as transparent as possible. The ESS has therefore invested heavily in documentation, with openness and transparency being key ESS values.

7) How does the ESS collaborate with other international research projects or institutions?

Here at the European Social Survey we love collaboration! The ESS has signed several formal agreements with survey projects outside of Europe, including those in Australia, China, Japan, South Africa, South Korea, and the USA, as well as with the East Asian Social Survey. These collaborations either support ESS questions being fielded on other surveys and/or facilitate methodological collaboration. A new UKRI-ESRC funded project is supporting a scoping study to look at the possibility of adding countries outside of Europe to the ESS web-first panel, CRONOS, in the future. Exciting!

In Europe, the ESS is coordinating a major new initiative to harness existing and new social science data to address the Next Generation EU policy agenda. Project partners include some of the other major social science infrastructures that have data covering younger Europeans, namely the European Values Study, the Gender and Generations Programme, and the CESSDA data archive. From 2025, we will also be working with SHARE ERIC and other research infrastructures in a new project to address the priorities of the Green New Deal (So Green). The ESS is using its CRONOS ‘web-first’ panel in both of these projects to address policy-relevant topics.

Finally, we signed an Memorandum of Understanding with the European Values Study (EVS) in 2024 to encourage collaboration between the two projects. This involves fielding some EVS questions on the ESS in the future, as well as efforts to facilitate more networking between data collectors in Europe from large-scale infrastructures.

My advice to those looking to set up their own cross-national proposals is to start early to identify partners, allow plenty of time for discussion, and take care to spell things out to avoid misunderstandings

8) As we look to the future, what new directions do you foresee for the ESS? Are there any upcoming initiatives or changes that we should be aware of?

The ESS is about to undergo the most significant transition in its history, moving away from face-to-face, in-person data collection to self-completion (web and paper). The change is driven by the declining capacity for face-to-face interviewing in Europe and deteriorating quality. At the same time, there is evidence that the quality of self-completion fieldwork is good and continues to improve.

We have developed an approach that uses either a ‘postal-first’ or ‘fieldworker-first’ design for making contact, depending on whether it is possible to use national sample frames and postal services to invite potential respondents via a postal letter. A choice is then made to use either a concurrent or sequential approach for offering the paper questionnaire. The contact protocol has been built on the late, great Don Dillman’s approach to surveys without field interviewers but adapted for cross-national implementation.

The ESS has built a single tool for translations, which links to the Computer-Assisted Web Interviewing instrument and even allows for the creation of paper questionnaires in every language. This ensures data are collected in a harmonised way and stored centrally.

To help navigate the transition, ESS Round 12 will see half of the completed interviews in each country conducted using face-to-face interviews and half in a self-completion mode, with both web and paper options offered. This will allow us to assess the impact of the mode change at both the country and variable levels. We will then provide guidance to data users to help them navigate any impacts on the time series.

Existing evidence from testing and piloting suggest that most countries will achieve response rates between 30% and 40%, with sample composition on key demographic variables similar to our earlier face-to-face surveys. Many countries will also see fieldwork costs fall, but this will be uneven with some seeing costs rise compared with face-to-face (especially where labour costs are low).

The ESS is perhaps the only example of a cross-national, input harmonised, general social survey that is making a mode transition in such a planned way, with all countries making the mode change at the same time. This approach fits with the ESS emphasis on input harmonisation to maximise cross-national comparability and helps to ensure that data users get a single data structure when accessing our data. The change presents many opportunities as well as challenges, and is requiring great efforts from all ESS stakeholders, including all our wonderful national teams.

Lead author contact information

Rory Fitzgerald
r.fitzgerald@city.ac.uk