Does It Really Make a Fracking Difference? The Conditional Effects of Question Wording on Support for Hydraulic Fracturing

Mileah Kromer Goucher College

Kirby Goidel Texas A&M University

Michael Climek Louisiana State University

Abstract

We utilize statewide surveys conducted in Maryland and Louisiana to investigate whether including the phrase “fracking or hydraulic fracturing” in a survey question influences self-reported awareness, perceptions of safety, or support for the drilling practice. We find that question wording effects are only present in the context of state support for additional drilling and only among Louisiana residents. Additional analysis reveals that the effects of question wording are contingent on partisan affiliation. Republicans are more inclined to support drilling but become less certain when the phrase “fracking or hydraulic fracturing” is used.

Introduction

A persistent challenge to survey research involves the development of opinions toward complex policy issues. How do people, who often lack basic political information, form opinions on issues that are technical in nature? Our research considers this question in the context of hydraulic fracturing, an important contemporary energy policy issue.

To date, most research analyzing opinion toward hydraulic fracturing has been done using national level surveys. For example, work by Boudet et al. (2014) found that while the majority of Americans had heard nothing about fracking, those who were aware are split in their support or opposition to its use. Clarke et al. (2015) suggests that the wordfracking” elicited more negative top-of-mind and environmental associations than did the phrase “shale oil or gas development.” Finally, incorporating surveys from New York and from counties in Pennsylvania within the Marcellus Shale region with national level samples, Evensen et al. 2014 concluded that the word “fracking” carries a negative, or even lewd, connotation.

Departing from the use of national-level surveys, our research analyzes how the phrase “fracking or hydraulic fracturing” influences public perceptions toward awareness, safety, and the economic/environmental trade-offs of the drilling practice in differing statewide contexts. Utilizing surveys from Maryland and Louisiana, we consider how state of residence might alter the framing effects. Our findings suggest that the phrase “fracking or hydraulic fracturing” has little effect on awareness or perceptions of safety, yet it has a significant effect on whether the public believes state government should encourage new drilling; however, this effect is only present among Louisiana residents. Thus, results suggest that statewide context matters when explaining the differences in opinion toward hydraulic fracturing resulting from framing or question wording effects.

Question Wording Effects

The big impact of small changes in question wording on poll results is well established in the public opinion literature (Bishop et al. 1978; Borrelli and Lockerbie 2008; Lockerbie and Borrelli 1990; Schuman and Presser 1996; Smith 1987; Smith and Squire 1990). Within an environmental context, Schuldt et al. (2011) find that Democrats respond in the same way to climate change as global warming, but Republicans responded more favorably to climate change. Akerlof and Maibach (2011) find that public preferences for the terms climate change, global warming, or global climate change are related both to partisan affiliation and whether the respondent believes climate change is occurring.

Question wording effects also vary across a variety of individual level factors. For example, better educated respondents are less susceptible to question wording shifts (Narayan and Krosnick 1996; Schuman and Presser 1981) while less educated respondents are more susceptible to questions that provide additional contextual information (Smith and Squire 1990). Within the context of energy policy, Cacciatore et al. (2012) demonstrate that the public evaluates biofuels more positively than ethanol and this effect is particularly pronounced among Democrats.

Given the previous research, we expect the use of terms such as fracking or hydraulic fracturing to increase awareness and concerns about safety while decreasing support for drilling. The logic here is relatively straightforward. The terms hydraulic fracturing or fracking, compared to a generic description of the drilling practice, should be more familiar to respondents and elicit a stronger response given its established negative connotations. As noted above, opinion toward energy and environmental policy is highly partisan. Thus, we expect that opinions of Republicans, more so than their Democratic counterparts, to be more sensitive to question wording.

State context should also matter. Louisiana ranks 4th in total energy production while Maryland ranks 39th (U.S. Energy Information Administration 2012). These differences and their ancillary effects lead us to expect that Louisiana and Maryland residents hold fundamentally different views in regard to energy issues. Question wording should have a greater influence in Louisiana where the economy is strongly tied energy production.

Data and Methods

This research utilizes question experiments from statewide public opinion polls in Louisiana and Maryland to consider whether the inclusion if the phrase “hydraulic fracturing or fracking” influences public perceptions toward drilling practice. The data come from three separate statewide surveys in Louisiana (2012 and 2013) and Maryland (2013). See Table 1 for a summary of the surveys included in the analysis. All interviews were conducted by phone by trained interviewers at the Public Policy Research Lab at Louisiana State University and the Sarah T. Hughes Field Politics Center at Goucher College. All surveys were dual-frame with landline and cell phone respondents selected via random digit dialing. Final data are weighted to match the adult population demographics of their respective states.

Table 1 Summary of surveys included in analysis.

Survey Completes Cell phone Landline Field dates
2012 Louisiana Survey 731 29% (214) 71% (517) 2/7–2/29
2013 Louisiana Survey 930 38% (356) 62% (574) 2/8–3/17
2013 Goucher Poll 791 35% (275) 65% (515) 3/3–3/7

For full Louisiana Survey methodology, see: http://pprllsu.com/wp-content/uploads/2015/12/LA-Survey-2012.pdf (2012); http://pprllsu.com/wp-content/uploads/2015/12/LA-Survey-2013.pdf (2013). For full Goucher Poll methodology, see: http://www.goucher.edu/Documents/Poli_Sci/hughes/March%202013%20Goucher%20Poll%20Results%20%28Friday%29.pdf (2013).

Instrument Design and Experiment

Respondents were randomly assigned to one of two groups. One group received the battery of questions with the phrase “hydraulic fracturing or fracking” included while the other group received the battery of questions without “hydraulic fracturing or fracking.” All respondents received their questions in the same order – (1) awareness, (2) safety, and (3) state encouragement of drilling. Specific question wordings are provided below.

  1. Awareness
    1. Fracking: How much have you heard or read about hydraulic fracturing or fracking – a process that uses the high-pressure injection of water, sand, and chemicals to remove natural gas from rocks deep in the earth’s surface – a lot, some, not much or nothing so far?
    2. Non-Fracking: How much have you heard or read about a process that uses the high-pressure injection of water, sand, and chemicals to remove natural gas from rocks deep in the earth’s surface – a lot, some, not much or nothing so far?
  2. Safety
    1. Fracking: As far as you know, do you think hydraulic fracturing or fracking is a very safe method to extract natural gas from the ground, somewhat safe, not very safe, or not at all safe?
    2. Non-Fracking: As far as you know, do you think this process is a very safe method to extract natural gas from the ground, somewhat safe, not very safe, or not at all safe?
  3. State encouragement
    1. Fracking: Some people say the state should encourage hydraulic fracturing or fracking because of the economic benefits, while others say the state should not encourage hydraulic fracturing or fracking because of potential environmental impact. Which comes closer to your view?
    2. Non-Fracking: Some people say the state should encourage drilling for natural gas by this process because of the economic benefits. Others say the state should not encourage drilling by this process because of the potential environmental impact. Which comes closer to your view?

Analysis

Table 2 presents a comparison of awareness, perceptions of safety, and whether the state should encourage by question wording. As Table 2 reveals, question wording matters very little in terms of self-reported awareness or perceptions of the safety. In Maryland, using the term “fracking” does appear to increase self-reported awareness but the difference is not statistically significant.

Table 2 Awareness, perceptions of safety, and support for state encouragement of drilling by question wording.

Heard some or a lot about hydraulic fracturing (% a lot) Hydraulic fracturing is somewhat safe or very safe (% very) State should encourage drilling
Louisiana 2013
Fracking 40.3 (15.2) 44.8 (11.0) 38.4*
Non-fracking 42.6 (21.6) 47.3 (15.4) 51.6
Louisiana 2012
Fracking 36.7 (18.1) 34.5 (12.3) 39.0*
Non-fracking 38.2 (19.8) 42.5 (12.6) 52.0
Goucher 2013
Fracking 48.1 (16.4) 34.8 (5.9) 25.0
Non-fracking 39.6 (16.3) 34.5 (6.2) 29.2

*p<0.05.

We do find significant differences for whether the state should encourage drilling, but only in Louisiana. Specifically, respondents in Louisiana were significantly less likely to say that the state should encourage drilling when the terms “hydraulic fracturing or fracking” were used.

So what explains this question wording effect, and especially its appearance in Louisiana but not Maryland? We argue the effect of question wording is contingent upon the larger economic and political context. First, overall support for drilling is generally lower in Maryland regardless of question wording. Considering the prominent role played by the oil and gas industry in Louisiana, greater statewide support for drilling is hardly surprising. Further, Maryland was considering a ban on fracking during the fielding period for the Goucher Poll. Second, according the Cook Partisan Voting Index, Louisiana is 12-points more Republican than the national average while Maryland is 10 points more Democratic. These partisan differences may help to explain not only baseline differences in support for drilling across states; they may also help to explain why the use of the term “fracking” has a larger impact in Louisiana than Maryland. Louisiana residents may be more likely than Maryland residents to adopt a standing decision to support additional drilling.

However, the use of the phrase “hydraulic fracturing or fracking,” primarily appears to give Louisiana respondents pause to reassess their support for drilling. We can see some evidence of this in the distribution of “don’t know” responses by question wording. When the phrase “hydraulic fracturing or fracking” is used, the percent of Louisiana respondents who said they “didn’t know” if the state should encourage drilling increased from 13 percent to 26 percent in 2012 and from 13 percent to 22 percent in 2013. We would expect this shift from support to uncertainty would be particular pronounced for those residents most inclined to support drilling absent additional information, i.e., Louisiana’s Republican partisans.

To test this hypothesis, Table 3 includes ordinal regressions predicting whether a respondent said the state should encourage drilling.1 Responses are coded 1 for respondents who say the state should encourage drilling, 0 for respondents said they did not know, and −1 for respondents who say the state should not encourage drilling. We include a dummy variable indicating whether the respondent received the question with the words “hydraulic fracturing or fracking,” a standard 7-point scale for partisan affiliation with strong Democrats coded as 1 and strong Republicans coded as 7, and interaction between these two terms. In addition, we include sex, race, education, and age as control variables2 as well as dummy variables to identify state and year. Finally, to provide a more stringent test of the hypothesis, we include a final model incorporating individual level awareness of fracking and perceptions of the safety as predictors. Awareness of fracking is coded as 4-point a scale ranging from 0 for respondents who reported have heard nothing so to 3 for respondents reporting have heard a lot. Perceptions of safety are coded on a 5-point scale ranging from 0 for respondents who thought fracking was not at all safe to 4 for respondents who thought fracking was very safe.

Table 3 Ordinal regressions of state support of drilling on demographics, partisan affiliation, and fracking awareness and safety.

Additive model Interactive model Interactive model w/fracking controls
Sex (Female) −0.74 (0.08)** −0.74 (0.08)** −0.48 (0.10)**
Race (White) 0.07 (0.10) 0.07 (0.10) 0.03 (0.11)
Age category (6 category) 0.18 (0.03)** 0.19 (0.03)** 0.16 (0.03)**
Education (4 category) 0.02 (0.04) 0.02 (0.04) −0.02 (0.05)
Party ID (7 category) 0.24 (0.02)** 0.29 (0.03)** 0.22 (0.03)**
Louisiana 2012 0.37 (0.10)** 0.38 (0.10)** 0.31 (0.12)**
Louisiana 2013 0.42 (0.11)** 0.41 (0.11)** 0.36 (0.12)**
Fracking awareness −0.08 (0.05)
Fracking safety 1.14 (0.05)**
Fracking question −0.35 (0.08)** −0.08 (0.15) −0.06 (0.17)
Fracking × party Id −0.08 (0.04)* −0.08 (0.04)*
Constant 1 0.76 (0.21) 0.91 (0.22) 2.78 (0.26)
Constant 2 1.71 (0.21) 1.86 (0.22) 3.9 (0.27)
n 2,342 2,342 2,322
Pseudo R-squared 0.07 0.08 0.21

*p<0.05; **p<0.01.

When the question includes the terms hydraulic fracturing or fracking, respondents were less likely to say the state should encourage drilling. When we include the interaction term in the model (column 2), we find that Republicans are less likely to say the state should encourage drilling only when the terms hydraulic fracturing or fracking are used in the question wording. Finally, this finding is relatively robust holding even after including measures for fracking awareness and safety.

In Figure 1, we illustrate the nature of the interaction. As the figure reveals, question wording has a more pronounced effect among Strong Republicans who are less likely to endorse hydraulic fracturing when the question wording specifically uses the term “fracking.”

Figure 1  Probability respondent said state should encourage drilling by question wording.


SP-Vol-10_January_Kromer_fig1.jpg


Conclusions

Survey researchers have long understood the importance of language to question wording and ultimately to our understanding of public opinion. In this paper, we add to this understanding by demonstrating the effect of the use of the terms “hydraulic fracturing or fracking” on public support for additional drilling. The use of language in this context matters, but its effect is contingent on local context and previously held political orientations. Specifically, question wording matters more in Louisiana where respondents are typically more inclined to support drilling. Notably, Louisiana residents do not express greater opposition when the question wording shifts to include fracking but instead grow more uncertain.

References

Akerlof and Maibach 2011
Akerlof, K. and E.W. Maibach. 2011. A rose by any other name...?: What members of the general public prefer to call “climate change.” Climatic Change 106(4): 699–710.
American Council for an Energy Efficient Economy. 2013
American Council for an Energy Efficient Economy. 2013. The 2013 state energy efficiency scorecard. Available at http://aceee.org/research-report/e13k.
Bishop et al. 1978
Bishop, G.F., R.W. Oldendick and A.J. Tuchfarber. 1978. Effects of question wording and format on political attitude consistency. Public Opinion Quarterly 42(1): 81–92.
Borrelli and Lockerbie 2008
Borrelli, S.A. and B. Lockerbie. 2008. Framing effects on public opinion during prewar and major combat phases of the US wars with Iraq. Social Science Quarterly 89(2): 502–522.
Boudet et al. 2014
Boudet, H., C. Clarke, D. Bugden, E. Maibach, C. Roser-Renouf and A. Leiserowitz. 2014. “Fracking” controversy and communication: using national survey data to understand public perceptions of hydraulic fracturing, Energy Policy 65(C): 57–67.
Cacciatore et al. 2012
Cacciatore, M.A., D.A. Scheufele and B.R. Shaw. 2012. Labeling renewable energies: how the language surrounding biofuels can influence its public acceptance. Energy Policy 51(C): 673–682.
Clarke et al. 2015
Clarke, C.E., P.S. Hart, J.P. Schuldt, D.T. Evensen, H.S. Boudet, J.B. Jacquet and R.C. Stedman. 2015. Public opinion on energy development: the interplay of issue framing, top-of-mind associations, and political ideology. Energy Policy 81: 131–140.
Evensen et al. 2014
Evensen, D., J.B. Jacquet, C.E. Clarke and R.C. Stedman. 2014. What’s the ‘fracking’ problem? One word can’t say it all. The Extractive Industries and Society 1(2): 130–136.
Lockerbie and Borrelli 1990
Lockerbie, B. and S.A. Borrelli. 1990. Question wording and public support for contra aid, 1983–1986. Public Opinion Quarterly 54(2): 195–208.
Narayan and Krosnick 1996
Narayan, S. and J.A. Krosnick. 1996. Education moderates some response effects in attitude measurement. Public Opinion Quarterly 60(1): 58–88.
Nelson, T. and Z. Oxley. 1999
Nelson, T. and Z. Oxley. 1999. Issue framing effects on belief importance and opinion. Journal of Politics 61(4): 1040–1067.
Schuldt et al. 2011
Schuldt, J.P., S.H. Konrath and N. Schwarz. 2011. Global warming or climate change? Whether the planet is warming depends on question wording. Public Opinion Quarterly 75(1): 115–124.
Schuman and Presser 1996
Schuman, H. and Presser, S. 1996. Questions and answers in attitude surveys: experiments on question form, wording, and context. Sage.
Smith and Squire 1990
Smith, E.R. and P. Squire. 1990. The effects of prestige names in question wording. Public Opinion Quarterly 54(1): 97–116.
Smith 1987
Smith, T.W. 1987. That which we call welfare by any other name would smell sweeter an analysis of the impact of question wording on response patterns. Public Opinion Quarterly 51(1): 75–83.
U.S. Energy Information Administration 2012
U.S. Energy Information Agency, U.S. States. 2012. Rankings: total energy production, 2012 (trillion Btu). Available at http://www.eia.gov/state/rankings/?sid=US#/series/101.
Footnote
1 We ran similar models for fracking awareness and safety but found no significant effects for question wording or for the interaction between partisan affiliation and question wording.
2 Sex is coded 1 for female respondents, 0 otherwise. Race is coded 1 for White, 0 otherwise. Education is measured on a 4-point scale with 1 indicating respondents with less than a high school education, 2 indicating a high school education, 3 some college and 4 a college degree. Age is measured on a 6-point scale with 1 indicating ages 18–24, 2 25–34; 3 35–44, 4 45–54, 5 55–64, and 6 65 and older.


About Survey Practice Our Global Partners Disclaimer
The Survey Practice content may not be distributed, used, adapted, reproduced, translated or copied for any commercial purpose in any form without prior permission of the publisher. Any use of this e-journal in whole or in part, must include the customary bibliographic citation and its URL.