One way the “voice of the ordinary European” currently reaches EU policy makers is via opinion polling. The European Commission’s Eurobarometer surveys measure what Europeans know and how they feel about many topics. Sample sizes are large. Respondents are chosen randomly. Theoretically, this means that responses are representative of all Europeans. But, is this really true?
What surveys actually measure
What surveys do best is put opinions and knowledge into perspective. They tell us how many people know this or which kinds of people support that. They can be quite valuable for tracking changes in opinion over time and comparing opinions of one group vs. another. But, they don’t necessarily tell us how people themselves define the issues, how well they actually understand those issues, or the complex and often contradictory feelings and thoughts that form opinions. This is because respondents can only choose from a pre-selected list of answers to a question. They can’t add different answers (or questions). They can’t ask for clarification of questions they don’t understand. So, whoever writes the questions and answers effectively determines the range of public opinion that is measured.
Who writes the questions (and answers)?
In the case of most Eurobarometer surveys, both the questions and answers are written by Commission bureaucrats and reviewed by research experts with no input from regular people. There is rarely preliminary qualitative research to determine how ordinary citizens might frame issues or define the range of possible answers. There isn’t even pre-testing of questions to see if they’re understandable. Questions that may be embarrassing to the EU or a member state simply are omitted.
How they’re used
Eurobarometer surveys can become quite controversial once you consider how they’re used. On one hand, they can give decision makers the political courage to tackle issues that are genuinely important to citizens but politically sensitive, like climate change. On the other hand, they can be misused by politicians to promote agendas that informed citizens might not actually support, like the notorious use of a biased Eurobarometer survey to justify EU funding for Galileo.
Ideally, Eurobarometer surveys would be used early in the policy development process to help set agendas that reflect citizens’ desires. They would also be used to plan communications strategies to best explain policies and programs. In reality, they often take place too late in both of these processes to be used this way. Instead, they’re frequently used to fulfill statutory obligations to measure the impact of policies on citizens or to provide “news value” when promoting EU policies and programs.
Transparency
Unlike most opinion polling by national governments, the results of Eurobarometer surveys are made available to the public. This is a good thing. But, transparency clearly has its limits.
Results of Eurobarometer surveys that support Commission goals tend to be released with great fanfare. Other studies are released quietly, without even a press release.
Reports summarizing the Commission’s interpretation of the data are easily available on its website. Raw results (that could lead to different analyses and conclusions) are also made available to the public. However, they’re difficult to access and require expensive statistical analysis software to use.
Fortunately, the Commission is experimenting with new ways to introduce more true “randomness” into their public opinion research. That will be discussed in part II.
This article originally appeared in Random Europeans on March 2, 2008 with the subtitle “Thoughts from The Centre’s Polls Apart Roundtable (20 February 2008) – Part I”
Photo: James Cridland