Measuring public preferences for government spending under constraints: a conjoint-analytic approach

ABSTRACT

Governmental budgets and the priorities they reflect are the subjects of recurring political debate. Research on political representation commonly focuses on relative spending preferences, mostly in isolated domains that are unconstrained, and so provides only limited information about people’s preferences. Recent survey work considers the effects of asking about absolute spending levels in different substantive areas and in the face of revenue constraints. No studies do all three, though two studies get close and provide more fine-grained measures of preferences for spending change. We follow their lead but in a more general way, offering budget profiles that include increases as well as decreases in spending levels, embedded in a conjoint experiment in Sweden. Our results reveal that people appear to hold preferences on specific magnitudes of spending change, budgetary constraints matter, and the effects of increases and decreases in spending are not symmetrical. Although the implied preferences for spending are similar in direction to expressed relative preferences that are unconstrained, the levels of support across domains are different. The findings have implications for assessments of opinion representation, as inferences about the responsiveness of policy to preferences – and the congruence between them – differ depending on measurement of the latter.

Introduction

The question of how much governments should spend in different policy areas is the subject of much scholarly and political debate. Previous research on public opinion about government spending relies on people’s relative preferences, based on responses to questions asking citizens about the extent to which they want “more” spending in particular domains. Such surveys tend to show that most citizens want more spending in most social domains, leading scholars to infer a pervasive democratic deficit (Bartels 2015). Importantly, research documents fundamental problems with interpreting relative preference responses in this way (Wlezien 2017).

The literature increasingly recognizes that government decisions involve changing specific spending amounts in multiple domains all at once, usually in the context of budgetary constraint (Jones et al. 2009). Previous scholarship has shown that multidimensionality and budgetary constraints can make a difference to people’s preferences (Hansen 1998). The implication is that expressed, unconstrained spending preferences provide only partial information to policymakers and to scholars interested in opinion-policy congruence. Not surprisingly, recent work has included trade-offs into survey questions (e.g. Bansak, Bechtel, and Margalit 2021; Bonica 2015; Bremer and Burgisser 2022a, 2022b; Busemeyer et al. 2018; Busemeyer and Garritzmann 2017; Hausermann, Kurer, and Traber 2019; Tuxhorn, D'Attoma, and Steinmo 2021). Some of this research also elicits preferences for changes in amounts of spending in different areas, usually in terms of percentages (see Bansak, Bechtel, and Margalit 2021; Tuxhorn, D'Attoma, and Steinmo 2021). However, that research only allows for spending decreases in the face of austerity, and so provides limited information about people’s preferences.

In this note, we build on and extend the previous research that uses conjoint analysis as a tool to overcome challenges confronting the elicitation of spending preferences (Bansak, Bechtel, and Margalit 2021; Bremer and Burgisser 2022a, 2022b). Following Bansak, Bechtel, and Margalit’s (2021) lead, we propose a design that addresses three problems we identify in current work: (1) the reliance on relative spending preferences, by including specific (and benchmarked) amounts of spending; (2) the reliance on isolated spending domains, by presenting respondents with a bundle of spending domains; (3) the neglect of revenue constraints, by including information about how each alternative budget might affect the average citizen’s tax contribution, i.e. revenue.

In our research, we ask respondents to rate spending profiles for a set of salient policy areas given the existing budget constraint. By including specific amounts that would increase, decrease or maintain current spending levels, the approach more closely reflects the choices governments face when making spending decisions, by comparison with existing approaches. The results of our analysis in Sweden strongly suggest that people hold preferences on specific amounts of spending change under budgetary constraints. Additionally, these preferences appear to be similar to but still different from unconstrained preferences: they are similar in direction but weaker in intensity and different also in terms of the ordering of preferences for change across domains. These results imply that unconstrained spending preferences misrepresent – to some degree – public support for policy change. This has implications for our assessment of opinion representation, as it distorts the correspondence between what appears to be true preferences and what policymakers do. Even though previous research shows that policy change follows the level of unconstrained support for spending change, our results indicate that responsiveness may be even stronger.

Existing work

Previous research provides important insights into citizen preferences for governmental spending: the coherence of mass spending preferences (e.g. Jacoby 1994); the micro-level determinants of spending preferences (e.g. Eismeier 1982; Rudolph and Evans 2005); the causes and consequences of specific welfare spending preferences (e.g. Gingrich and Ansell 2012; Hausermann, Kurer, and Traber 2019; Rehm 2011); response to spending (e.g. Wlezien 1995; Soroka and Wlezien 2010); trade-offs between spending, deficits, and taxes (e.g. Hansen 1998; Busemeyer and Garritzmann 2017; Hausermann, Kurer, and Traber 2019; Tuxhorn, D'Attoma, and Steinmo 2021; Bremer and Burgisser 2022a) and between individual spending domains (e.g. Busemeyer et al. 2018; Bremer and Burgisser 2022b).

Three caveats are common to most previous work. First, scholars typically rely on measures capturing basic relative preferences for spending, i.e. assessments of whether citizens want “more” spending in a specific domain (Problem 1). The underlying assumption – among those who administered such surveys – is that citizens only can express vague spending preferences relative to the (unspecified) status quo, that is, only about the direction of change. This assumption presumably is based in part on research on innumeracy, which refers to the difficulty people have understanding specific quantities (Peters 2012). Some new research allows people to express preferences for magnitudes of spending change but almost always only in percentage increments, i.e. measuring specific change relative to the (unspecified) status quo (Bansak, Bechtel, and Margalit 2021; Tuxhorn, D'Attoma, and Steinmo 2021; but see Bonica 2015).

Second, much previous work relies on spending preferences registered in isolated domains (Problem 2). The underlying assumption seemingly is that survey questions with more spending dimensions and under budgetary constraints suffer from “excessive complexity” (Groves et al. 2011, 228). For instance, Krosnick and Presser (2010, 264) recommend asking about one thing at a time, and survey organizations have been following this advice when estimating spending preferences. This obviously limits research. But, beginning with Hansen’s (1998) examination of “domestic” and “defence” spending and continuing to the present, research considers a range of programs and documents interdependence of preferences (e.g. Tuxhorn, D'Attoma, and Steinmo 2021; Bremer and Burgisser 2022a, 2022b).

Third, and finally, many studies use preferences that do not reflect revenue constraints (Problem 3). Since revenue and spending are connected, budget balance might matter for people’s preferences for the latter. This also has been well-known for some time (Hansen 1998), but has recently received more attention (Ballard-Rosa, Martin, and Scheve 2017; Busemeyer and Garritzmann 2017; Tuxhorn, D'Attoma, and Steinmo 2021; Bremer and Burgisser 2022a, 2022b).

While research addresses each of the “problems,” only two studies consider all three – Tuxhorn, D'Attoma, and Steinmo (2021) and Bansak, Bechtel, and Margalit (2021; see Appendix 1 for a literature overview). The former uses interactive budgeting tools to let respondents increase or decrease spending in different areas. The latter employs conjoint analysis and asks respondents to choose between budget profiles that offer decreases in different areas to address austerity. This allows estimation of causal effects, which is important, but it does not assess preferences for spending increases. Although this makes sense when analyzing response(s) to austerity, it does preclude a more complete assessment of people’s preferences.

In this paper, we follow the lead of Bansak, Bechtel, and Margalit (2021) and propose conjoint analysis to address the first two problems – relative preferences and isolated domains – simultaneously, while also considering the third. Conjoint experiments (Hainmueller, Hopkins, and Yamamoto 2014) are an increasingly popular approach for survey-based analyses of aggregate preferences with multiple determinants, in which respondents are asked to state a preference for two profiles. Bansak, Bechtel, and Margalit’s (2021) results illustrate the potential of conjoint experiments for estimating budgetary preferences. Their results from the specific case of support for different austerity packages are encouraging for an application to a more general case of support for a governmental budget including a publicly salient set of policy areas.

Research design

Conjoint experiments enable us to estimate preferences for a budget across a set of domains treated collectively given a particular budget constraint and real-world spending levels. We conceptualize budget constraints as the amount available for spending. Based on people’s budgetary preferences, we can infer the magnitude of change citizens prefer given the constraint.

Respondents were sampled from a standing opt-in panel online in December 2016 (net sample = 3,122) maintained by the Laboratory of Opinion Research at the University of Gothenburg. The sample is representative of Swedish adults for the characteristics of age, gender, and education. For more information about the data, see Appendix 2; the full survey text is in Appendix 3.

In the survey, respondents compared two pairs of spending profiles displayed on their computer screens and indicated their preference for one (dependent variable: Profile Choice). Since all respondents made two of these comparisons, evaluating two spending profiles each time, the data cover more than 12,000 evaluated spending profiles, i.e. 3,122 times 2 times 2. We focused on four salient spending domains to minimize the risks of satisficing.

Each profile includes randomly-varied spending levels across four randomly-ordered domains: education, healthcare, defence, and labor market. These domains were selected based on their importance in surveys at the time and previous research (see Eichenberg and Stoll 2003; Oscarsson and Bergström 2016). Of the 27 budget domains in the government’s proposal, our four account for 29% of the government’s proposed expenditure for 2016.

The five spending levels used in each domain were chosen based on the 2016 budget proposal of the Swedish government, where the mid-point reflects the proposed amount.¹ We informed respondents about this to highlight the status quo. Although citizens rarely know the amounts of spending in particular categories, there is reason to suppose that they know whether they want 5 or 10 billion more than is being spent in different areas, especially salient ones. This was validated in our pre-test, where respondents stated that they often had difficulties answering the question because they did not know the status quo spending amounts.²

Respondents received two more pieces of information. We operationalize the limits imposed by a budget constraint by telling respondents how much the budget would change total spending and how much the profile would change the tax burden on Swedish citizens to approximate revenue effects. These features as well as the information on the status quo are part of the experimental treatments.

In sum, with our survey design we address Problem 1 by including specific and benchmarked amounts of spending and Problem 2 by presenting respondents with spending amounts in multiple spending domains. Our design also addresses Problem 3 by including information about how each alternative budget might affect the average citizen’s tax contribution, i.e. revenue. At the same time, we recognize the limitations of the latter, as a proper assessment would entail addressing tax preferences as well (see e.g. Ballard-Rosa, Martin, and Scheve 2017).

We estimate causal effects from the conjoint experiment using the cjoint package in R (see Hainmueller, Hopkins, and Yamamoto 2014; version 2.1.0) with clustered standard errors by respondent. We estimate the Average Marginal Component Effects (AMCE) for the effects of each spending level in each of the domains on people’s approval for the entire budget. For diagnostic and robustness tests, see Appendix 4. These estimates reveal the levels of spending people prefer on average across the different domains under budgetary constraints.

Results

We begin by estimating the AMCE for each spending level in the different domains. This provides information on the causal effect of each spending level on the probability of respondents preferring a budget profile. Figure 1 shows the results alongside the 95-percent confidence intervals for each spending level.

Figure 1. Effects of spending levels on approval of spending profiles. (N = 12,376 observations, 3,122 respondents). Note: The baselines represent the current (2016) level of government spending in each domain. All spending levels are in billions of Swedish Kronor.

The depicted AMCEs show the average change in the probability that a profile is chosen when a given spending level is included by comparison when the baseline is included, here, the status quo. For example, the results demonstrate that support for a budget decreases as spending on healthcare, education, and defence drops below the status quo, that is, since those coefficients are increasingly negative. Specifically, a spending profile with the lowest spending level on healthcare of 60 billion SEK instead of the baseline amount of 70 billion SEK decreases the probability that the spending profile is chosen by 0.21 (SE = 0.01), or 21 percentage points. This is the strongest estimated effect for any of the treatments in any of the domains. Budgets involving decreases in healthcare are rejected and increasingly so the larger the decrease, and the reverse is true for increases. This symmetry implies strength and coherence to public preferences for healthcare spending.

The pattern is quite different for each of the other spending areas. Here, citizens generally are not more supportive of spending increases.³ Indeed, for labor market spending, support decreases significantly for profiles including the highest spending levels (90 billion SEK). That said, the public is not supportive of decreases to spending in any of the areas, particularly education and defence.

Taking spending in the different domains together, as a package, only a budget that is 10 billion SEK larger and imposes an additional small tax burden receives public support, and all budgets smaller than the current one and reducing taxes are rejected (see Appendix 6). Overall, the results of the multidimensional survey format reveal that the public prefers a lot more spending on healthcare, possibly wants a little more spending on defence and education but seemingly not less and appears pretty satisfied with labor market spending.

How do these results compare to those from traditional unconstrained spending questions? To provide initial answers, we also included the traditional questions in our survey and randomly assigned respondents to receive either the traditional questions first and then the conjoint, or the reverse.⁴ Do note that responses to the two question formats are not perfectly comparable. Most importantly, since the conjoint approach offers choice between two spending profiles, which about 75% of the time contain increases and decreases in different domains, the baseline level of net support will tend to be closer to 0 than that using the traditional wording, where respondents can favor increases in every domain.⁵ The format may thus dampen support for spending change and complicate direct comparisons of net support for spending change. That said, it has little consequence for our ability to compare preferences for spending change across domains; indeed, the elicitation of these “priorities” may be what most recommends the conjoint approach.

We calculated aggregate-level preferences for each format. For the traditional items, this means subtracting the percentage saying they want less spending from the percentage saying they want more, i.e. “net support” for spending (Wlezien 1995). For data from the conjoint experiment, we produce similar measures by using individuals’ preferred budgets and the resulting distribution of preferred spending levels over both choice tasks.⁶

Table 1 summarizes these measures of aggregate net support across domains. Results suggest that imposing budget constraints and real numbers does not change the direction of results: net support is positive for healthcare, defence, and education and negative for labor market. These findings validate our approach and also are good news for studies using the relative spending questions for inferences about people’s preferences for the direction of spending change.

Table 1. Comparison of spending preferences obtained from isolated relative preference questions and multidimensional preference questions.

	Isolated preferences			Multidimensional preferences
	% preferring “less”	% preferring “more”	Difference (net support)	% preferring “less”	% preferring “more”	Difference (net support)
Defence	25.4	53.9	28.5	36.8	42.7	5.9
Education	9.9	72.4	62.5	35.4	43.1	7.7
Healthcare	24.8	52.9	28.1	29.0	49.5	20.5
Labor	32.6	24.3	−8.3	39.9	39.3	−0.6

Note: Both isolated and multidimensional preferences were generated from the same survey. Respondents were randomly assigned to receiving questions on isolated preferences either before or after the multidimensional ones.

However, Table 1 does reveal differences regarding the magnitude of support. This is in part as we expect based on the question format. But the differences in preferences vary dramatically across domains, from nearly 55 points for education to only 7.6 for healthcare. The pattern indicates that imposing budgetary constraints and real numbers dampens the intensity of citizens’ preferences for spending change in some domains more than others. This has implications for inferences about the ordering of preferences across domains. Results using isolated preferences suggest that the public favors, in descending order, increasing spending on education, defence, and healthcare spending, and decreasing labor market spending. By contrast, those based on multidimensional preferences reveal that respondents most favor increasing spending on healthcare and to a much lesser degree increasing spending on education and defence.⁷ Finally, the results indicate that spending on labor market is about right – almost exactly so – whereas isolated preferences suggest a (slight) preference to decrease spending. These are not simple artifacts of format and demonstrate that the conjoint approach including a budget constraint and using real spending numbers matter for preferences.⁸

This finding is important for policymakers and researchers because studies of policy responsiveness regularly show that governments are sensitive to the direction and magnitude of public support. While our results imply that a responsive policymaker would produce the same policy in terms of the direction of spending change using both sets of results, the magnitude of the response seemingly would differ across domains.

Discussion

In this paper, we illustrate how conjoint experiments produce preferences that differ from those based on responses to traditional survey questions. The differences are not trivial and have potentially important implications for studies using the traditional spending questions for substantive analyses, e.g. of opinion-policy congruence. Many scholars take expressed relative preferences at face value and assume that responses to questions about policies registered in isolation reveal exact preferences (e.g. Monroe 1998; Bartels 2015). Our analysis suggests that the characterizations may be correct when it comes to the direction of change, but not the magnitude, which can imply a different preference ordering across domains.

While our approach is revealing about preferences, there are limitations. First, our design may understate the true effects of conjoint experiments owing to possible priming effects that a split-sample design would eschew (but see Appendix 3). Second, we cannot evaluate the individual impact of the different features in our experiment, as we were unable to isolate these effects given our research design. Third, we focus on a limited set of salient domains and do not assess differences across sub-groups of respondents.

Even accepting our results, we recognize that they are based on but a single study in a single country, limiting generalizability. Even were we to find similar patterns elsewhere, we do not recommend replacing the traditional spending questions with conjoint experiments, for two reasons. First, the traditional questions do appear to effectively capture the public’s preferred direction of policy change, which is what scholars often are wanting to capture. Second, dropping those questions would undermine important and now long-standing time series. It also is important to remember that conjoint experiments require a lot of respondents and/or survey space and are not inexpensive. The approach thus may be best thought of as a supplement to existing surveys when researchers want to evaluate the level of public support for spending change, which matters to policymakers (Wlezien 2017).

Acknowledgements

Previous versions of this article were presented at the 2016 EPOP conference in Canterbury and the 3rd Barcelona-Gothenburg-Bergen Workshop on Experimental Political Science in 2017. We would like to thank the participants as well as the reviewers and editor of JEPOP for their very valuable comments, which improved the manuscript.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by Forskningsrådet om Hälsa, Arbetsliv och Välfärd [grant number 2013-2692].

Notes

1 See Appendix 3 for more information on the considerations relating to the spending levels used in the profiles.

2 For more information on the pre-test results and the survey design, see Appendix 3. Including a status quo might produce a status-quo bias, potentially making our approach a (more) relative one as well, but with specific amounts. While we cannot test this given our research design, it implies that our analysis might provide conservative estimates of the effects of the conjoint approach.

3 Only the effect of including 55 billion SEK for defence is (just) statistically significant (p < .05).

4 See question 2a-d in Appendix 2, which also includes a 5-point response scale where the mid-point reflects the status quo. While this design may induce some bias due to priming, we find little evidence of such effects – see Appendix 3, Tables A1 and A2.

5 Approximately 60% of the profiles include the status quo for at least one domain.

6 See Appendix 5 for an example. Appendix 4 shows that the results remain substantively the same when net support is calculated using the distribution only from the first choice task.

7 Recall that these differences are not a methodological artifact, i.e., they cannot be explained by the greater balance implicit in the conjoint approach.

8 Some issues (healthcare) seem less sensitive in their direction and magnitude to changes in the question format than others. It might indicate that respondents have a “relative preference importance” (Hausermann, Kurer, and Traber 2019, 1067) for healthcare spending.