A framework for policy mix analysis: assessing energy poverty policies

ABSTRACT

Under the topics of climate change and sustainable transitions, the importance of policy mix understanding and energy poverty is simultaneously discussed. Both concepts do not have universal definitions, and literature focuses on building the different fragments of each one to design new ways to understand, analyze and develop policies. Energy poverty is complex and has a multitude of drivers, such as income, energy prices, and buildings/energy efficiency are examples of how different policies are required to erase this problem. Understanding how those policies work together and should be evaluated challenges new perspectives between different fields. Framed in this subject matter, and after an overview of its state of the art, a flexible and systemic framework for policy mix analysis is proposed considering five steps: definition of objectives, instrument selection, single instrument analysis, instruments interaction analysis, and evaluation. The major contribution of the proposed framework is a clear yet adaptable criterion for instruments interaction analysis. Energy poverty literature is reviewed in the optic of how policy mix can help develop and analyze policies for its erasing, and specific criteria for its instruments analysis are suggested.

KEYWORDS:

• Policy mix
• policy analysis
• energy poverty
• sustainable transitions

1. Introduction

In the last decade, governments have been designing new policies facing complex problems which involve different levels of governance and the integration of various studies fields and actors. These different policies are especially being developed under sustainable transitions and climate change mitigation at various policy levels (Rogge, Kern, and Howlett 2017).

The use of policy mix in general policy sciences started during the late 1980s and early 1990s to explore how multiple policies and instruments could answer desired inputs and outputs (Flanagan, Uyarra, and Laranja 2011; Rosenow et al. 2016). In the first decade of the current millennium, policymakers and analysts understood that innovation, sustainable transitions, and research and development policies were facing the use of multiple goals, instruments, and actors (OECD 2007; 2012; Magro and Wilson 2019; Kern, Kivimaa, and Martiskainen 2017). Following Tinbergen’s rule of economic policy, which defends multi-aspects problems should have a policy instrument per target to avoid redundancies (Tinbergen 1952), the current discourse present in the European Union (EU) and Organisation For Economic Co-Operation and Development (OECD) about sustainable transitions faces the need to understand how to combine multilevel, multi-instruments and the mix of goals within policies to answer the complexity of its problems (Diercks 2019; Kurowska-Pysz, Castanho, and Loures 2018; Solorio 2011; Edmondson, Kern, and Rogge 2019).

Paris agreement, Clean Energy for all Europeans Package, United Nations (UN) Sustainable Development Goals (SDGs), and Renovation Wave are examples of how a path is being developed to define and achieve climate target goals (European Commission 2015; European Union 2019; United Nations 2017; European Commission 2020b). In this context, different countries and local entities are also contributing to changing how different policies can be articulated to achieve these goals. Understanding how to analyze and evaluate these policies is an important step for designing better ones and redesigning those already implemented.

Although policy mix and energy poverty (EP) do not have a consensus definition, both are currently being addressed in the importance of sustainable transition policies at the EU level. Policy mix focuses on understanding how multiple and complex policies can work together and must be analyzed or evaluated (Lindberg, Markard, and Andersen 2019b). Furthermore, policy mix analysis requires the discussion of different blocks and evaluation criteria which are not only found in its own topic but, for instance, also in the policy coherence, integration, and coordination debate (Cejudo and Michel 2017).

On the other hand, EP is an example of a current societal issue requiring a mix of complex and multi aspects policies (Primc and Slabe-Erker 2020; Rosenow et al. 2016). Different dimensions of the problem interact within themselves, affecting outcomes of each other. The problem combines the lack of income, energy prices, energy access, and efficiency of housing or technologies used and has consequences for the health and wellbeing of the targeted population (Gouveia and Palma 2021; Thomson and Bouzarovski 2019). Therefore, fighting for no poverty (SDG 1), good health and wellbeing (SDG3), gender equality (SDG 5), affordable and clean energy (SDG 7), and reduced inequalities (SDG 10) are examples of SDGs that EP interacts with (Primc and Slabe-Erker 2020; OECD 2019).

This paper will present a framework proposal for Policy Mix analysis after depicting a theoretical background of policy mix and EP concepts. The aim is to develop a framework that can be used in different policy fields to understand the policy mix by evaluating the instruments individually and visualizing their interactions. In this way, a clear process will help structure a systematic and iterative method that combines the policy mix blocks and the policy coherence studies to comprehend the complex issues currently being contemplated at the different levels of the policy studies. Then, it is suggested how to apply the framework in EP cases considering the policy guiding of Energy Poverty Observatory (EPOV), a European Commission initiative aiming at EP which has now been continued by the Energy Poverty Advisory Hub (EPAH) (EPAH 2022a).

2. Literature review

2.1. Policy mix

In the first decade of the current millennium, policymakers and analysts understood that innovation and sustainable transitions policies were facing the use of multiple goals, instruments, and actors (OECD 2007; 2012; Magro and Wilson 2019). Policy mix analysis takes special attention to the study of interactions between its elements, differencing itself from traditional analysis of single instruments, which aims to compare the quality of instruments.

Due to the complexity of policy mix, it has been difficult to trace and define a universal model for its analysis. Although there is no common model to assess policy mix, different and experimental methods are being taken to further test and propose new analysis methods, making policy mix analysis seen as a framework rather than a model (Magro and Wilson 2019; Bouma et al. 2019).

The concept of policy mix emerged during the 1960s as an analysis of the way fiscal and monetary policies could be related to economic policy: ‘monetary and fiscal policy can be used as independent instruments to attain the two objectives if capital flows are responsive to interest rate differentials, but it is concluded that it is a matter of extreme importance how the policies are paired with the objectives’ (Mundell 1962). The quote shows how the objectives should be related to attaining a successful articulation between policies. However, when it comes to different policies and the evolution of the concept, some questions appear: (1) are the objectives the only way policy mixes can articulate?; (2) how are policy mixes currently being applied?; (3) what type of problems can appear with this way of using policies? To analyze these questions, we need to understand how policy mix is currently being used and how authors agree with its definition.

There is still no standard definition of policy mix, and three types of literature discuss different aspects of the subject. First, authors define the elements in consideration when approaching policy mix (Edmondson, Kern, and Rogge 2019; Lindberg, Markard, and Andersen 2019a; Rogge and Reichardt 2016). In this type of literature, there is a recognition of the evolution of the concept, starting by focusing on the mix of instruments and ending involving elements, policy process, characteristics, dimensions, and its interactions and dynamics. Elements ‘comprise the (i) policy strategy with its objectives and principal plans for achieving them and (ii) the instrument mix with its interacting policy instruments’ (Rogge and Reichardt 2016). Policy processes determine the strategy, instruments, and characteristics and change over time. Characteristics are the key features that make it possible to analyze and comprehend how the policy mix occurs. Second, literature justifies the concept used in different fields (Howlett and Del Rio 2015). Third, studies and discussions of different approaches to analyzing policy mix (Costantini, Crespi, and Palma 2017; Bouma et al. 2019; Mavrot, Hadorn, and Sager 2018). Overall, the policy mix analysis starts understanding the complexity of the subject, and defining the elements that will be assessed. Then, an iterative process is applied using criteria to evaluate the policy mix.

2.1.1. Policy mix analysis

Policy Mix Analysis takes special attention to the study of interactions between its elements, differencing itself from traditional analysis of single instruments, which aims to compare the quality of instruments (Ring and Schlaack 2011).

Although there is no common model to assess Policy Mix, different and experimental methods are being taken to further test and propose new strategies for analysis (Magro and Wilson 2019; Bouma et al. 2019). Hence, Policy Mix Analysis can be seen as a framework rather than a model.

Rogge and Reichardt (2016) identify two critical challenges for analysts beyond the taxonomy of policy mix that has already been defined as a step to select the framework. First, there is a need to set boundaries: (1) understand the complexity of the subject, (2) define the unit of scales, for instance, deciding which actors take place in the analysis, and (3) understand boundaries are iterative and may change during the process. Second, after delineating the policy mix study, the analyst may capture the instruments, policy strategy, policy process, and characteristics.

Mavrot, Hadorn, and Sager (2018) start completing the Rogge and Reichardt (2016) policy mix analysis framework by adding settings and target groups to their model. The authors argue that policy mix analysis should focus on the form of policy instruments and their context of implementation. Settings are identified as the ‘specific context in which policy instruments are implemented’ and target groups how the instruments interact. Moreover, special attention is given at the micro-level, where interaction occurs between target groups and policy instruments. In summary, the adopted policy mix by the authors is based on four aspects of the Rogge and Reichardt (2016) blocks of policy mix (Mavrot, Hadorn, and Sager 2018; Rogge and Reichardt 2016).

Bouma et al. (2019) identify three steps to evaluate policy instrument mixes: (1) disentangle the policy mix, mainly focusing on the policy objectives and means, (2) understanding if the mix occurs on the objectives or instruments and (3) selection of the existent methodologies to define a framework. The authors discuss that policy objectives mix is a political question and policy instrument mix depend on the relevant market, governance and behavioural failures. Typical methods that are based on multi-criteria, cost-effectiveness, and cost–benefit need to set boundaries for their application. Since Policy Mix does not have a clear point or baseline to distinguish their policies and, in summation to the complexity of the instruments’ interactions, these methods are difficult to apply for Policy Mix analysis. (Bouma et al. 2019).

Lindberg, Markard, and Andersen (2019b) studied energy policy mix in the EU, focusing on its policies and actors. They start by collecting and selecting the data and explain the process to the final elements considered. Next, they present the coding scheme used to categorize actors and policies. Finally, to conclude their policy analysis framework, there is an explanation of the operationalization of elements and the criteria used to weigh them.

Edmondson, Kern, and Rogge (2019) ‘develop a novel conceptual framework for analyzing the co-evolution of policy mixes and socio-technical systems in processes of sustainability transitions’(Edmondson, Kern, and Rogge 2019). Then, it is taken an empirical illustration of the framework of the policy mix on the United Kingdom zero carbon homes. First, they start by collecting and analyzing ‘policy documents, industry journals, secondary literature, government consultations, select committee publications, inquiries, and debates in the House of Commons and House of Lords over the period 2006–2016’. Then, following a top-down approach, they identify the policy mix by considering the targets and instruments.

Rosenow et al. (2016) analyze the policy mixes in 14 EU countries’ energy efficiency policy. They start by collecting and separating policy instruments based on European Energy Directive (EED) and then characterize them by function, the underlying theory of change and behaviour type in a two-entry table, which helps identify how different policy types can deliver the same intervention. Thus, there were 11 policy types specified which would result in 160 combinations if three policy types were considered, it is explained how this would not be useful for the report, and therefore the study only focuses on bipartite policy mixes combination and the criterion used to judge it is the effectiveness of the combination on saving energy when compared to the single-use of the policies. The study also assumes that the policy instruments occur in the same sector and time.

Policy mix analysis is a step to understanding how the optimal policy mix would address an issue. Policy design can be summarized in three steps: ‘(1) a primary selection of the specific instruments most suitable among the wide range of different possible instruments; (2) the concrete design and/or ‘customization’ of the instruments for the context in which they are supposed to operate; and (3) the design of an instrument mix, or set of different and complementary policy instruments, to address the problems identified’(Rosenow et al. 2016). It is important to mention that this is a brief summary of policy design and that it is much more complex to define because of its political characteristic.

In general, the studies discussed have a similar approach when it comes to policy analysis. First, the authors start to understand the concept of policy mix and the field where it is being studied. Here, occurs typically the literature revision where the most important concepts of the study are given a definition. Second, the instruments and/or information for analysis are collected, and it is defined which dimension will be the focus. Some studies focus on the instrument mixes, others on the actor and their level mixes, and other combinations of elements of policy mixes, previously described by the authors, are taken into consideration to analyze the specific field. Because of the number of possible combinations of the elements, their dynamic and multi-aspects, the need to limit the analysis and, therefore, the third stage appears to define the mixes that will be evaluated. Fourth, the framework of the approach is designed to analyze the collected instruments, the type of evaluation used, and the criteria. Currently, there is no universal method to approach the complex, interchangeable and dynamic features of policy mix. Thereupon, experimental and mix of methods are used for policy mix analysis. Finally, in the five stages, authors analyze the results, identify the limitation of their studies and give recommendations to further investigate the subject they have developed. Moreover, the complexity and dynamic nature of policy mix never stop during the analysis, making it an iterative and learning process.

2.2. Energy poverty

In 2009, European Commission addressed for the first time EP with the Directives 2009/72/EC and 2009/73/EC, which ordered the development of national action plans or other frameworks to deal with the EP issue (Gouveia, Palma, and Simoes 2019; Kyprianou et al. 2019). Thus, when considering the key junctures, EP is mentioned further back in time under other concepts such as fuel poverty, energy efficiency, and energy justice. Kyprianou et al. (2019) studies EP policies and measures in 5 EU countries and, after assessing the complex nature of EP and the difficulties to identify EP consumers, defined a timeline of main policies in the EU on the topic of EP.

On the EPAH website, indicators, data to approach, and measures studied during the EPOV and revised in EPAH can be consulted (EPOV 2020b; Gouveia et al. 2022). On its Guidance for Policymaking (EPOV 2020a), four points are considered essential to address EP in policies: (1) Measurement; (2) Definition; (3) Policy Type; and (4) Financing and Funding. While working closely with relevant stakeholders, such as Covenant of Mayors (Covenant of Mayors 2021), and through the information and data collected, EPOV addressed the key gaps and made it easier to develop research and urban policies under the topic (Bouzarovski and Thomson 2020), while in Gouveia et al. (2022) each EP indicator was assessed in detail for a better understanding and to get key insights for their proper use and interpretation.

When it comes to EP definition, EPOV defined it as ‘a distinct form of poverty associated with a range of adverse consequences for people’s health and wellbeing – with respiratory and cardiac illnesses, and mental health, exacerbated due to low temperatures and stress associated with unaffordable energy bills’ but there is no standard definition on EP and authors have studied the evolution of the concept and different ways to approach the issue (Siksnelyte-Butkiene et al. 2021; European Commission 2020a). However, it can be noted that the EPOV definition of EP primarily focused on the health effects and wellbeing, while others focus more on the expenditure dimensions such as the capacity of the energy poor cover their energy bills.

Although some policymakers view EP as a phenomenon in low-income households, the problem goes further than that, finding mid-to-high income households are considered energy poor (Yearwood and Pye 2016). Different authors (Pye et al. 2015; Kyprianou et al. 2019) analyze how European countries define those vulnerable consumers. Kyprianou et al. (2019) consider four criteria (individual characteristics, specific circumstances, social welfare system and related to energy consumption with contract limitations), and Pye et al. (2015) consider five categories (energy affordability, receipt of social welfare, disability/health, range of socio-economic groups and not available) of vulnerable consumers definitions.

When comparing the criteria and categories used while analyzing the definition of vulnerable consumers, it is possible to comprehend that the last case only differs on adding the situation where the concept is not mentioned- not available category. This is justified since Kyprianou et al. (2019) only analyzed five countries, all defining vulnerable consumers, and Pye et al. (2015) analyzed the 28 EU’s member states at the time. The income is the most focused driver considered in the definition in both studies, being used in total by 14 EU member states. However, as previously mentioned, energy-poor consumers are not only a consequence of low income since other drivers take an important action on defining these consumers.

In sum, socio-economic group, housing situation, energy use and demographic situation are the general parameters used to identify the targets on EP policies. Additionally, to define the vulnerable consumers as a step of definition of EP and ability to choose indicators, it is vital to distinguish vulnerable consumers and EP and understand how these are not synonyms and are being currently used for different objectives in the literature.

2.2.1. Energy poverty drivers and indicators

Several authors (Primc and Slabe-Erker 2020; Gouveia, Palma, and Simoes 2019; Yearwood and Pye 2016) state the importance of collecting the drivers to understand EP and selecting indicators for policy-making. Yearwood and Pye (2016) consider six types of drivers: (1) structural, (2) markets, (3) natural systems, (4) macroeconomy, (5) economic, and (6) policy. The main drivers identified by Gouveia, Palma, and Simoes (2019) are energy-inefficient housing stock, elderly household occupants, low household income, and high energy prices, falling especially under the economic driver identified by Yearwood and Pye (2016). However, the former authors note that EP is also an issue of health, energy, social services, and housing, and this is an important concern for policymakers to consider.

Policies and plans are essential factors on EP definition, measuring, monitoring, and a step to battle the issue. Yearwood and Pye (2016) propose categorizing policy interventions as (1) short-term financial interventions, (2) additional consumer protections targeted explicitly at vulnerable consumers, (3) energy efficiency measures targeting structural EP problems, and (4) improved consumer awareness and information. Kyprianou et al. (2019) follow the same type of boundaries, identifying four types of measures used in Europe to tackle EP.

Besides the already mentioned studies, other authors highlight the importance of selecting EP metrology to assess the EP problem (Sareen et al. 2020; Palma, Gouveia, and Simoes 2019; Antepara et al. 2020). Sareen et al. (2020) propose an iterative framework with five dimensions (historical trajectories, data flattening, contextualized identification, new representation, and policy uptake) to analyze EP metrology and discuss policy uptake's importance in domestic energy deprivation. Palma (2017) develops the mapping of thermal comfort in residential dwellings needs in Portugal by assessing the gap between bottom-up buildings typology approach and a top-down energy use statistics-based approach. Kyprianou et al. 2019 analyze the EP policies and measures in 5 countries, presenting six indicators (Arrears on utility bills, Hidden EP (HEP), Inability to keep home adequately warm, High share of energy cost in income (2M), Home uncomfortably hot in summer and Presence of leak, damp or rot).

Gouveia, Palma, and Simoes (2019) propose an Energy Poverty Vulnerability Index (EPVI), which focuses ‘on space heating and cooling, to map energy-poor regions and identify hotspots for local action’ and tests the method in 3092 civil parishes in Portugal. The EPVI combines socio-economic indicators with climate variables, energy consumption levels, calculated energy demand for space heating and cooling, climatization technologies, and construction characteristics of several building typologies. The authors also discuss three categories of EP measures taking into consideration the indicators based on (1) expenditure (e.g. LIHC), (2) consensual approaches (e.g. EU-SILC), and (3) energy needs calculation (e.g. DECC) (Gouveia, Palma, and Simoes 2019).

In conclusion, different policies are being used to access EP, such as social and energy policies, and there is a scientific recognition of how these policies affect the situations analyzed. Understanding the different elements of EP - measurement, definition, policy types considered, and indicators – shows how different kinds of policies, primarily related to sustainable transitions, requires understanding how policies work together (Siksnelyte-Butkiene et al. 2021). indicators are being developed and analyzed to assess EP. The selection of the indicators is an important step in designing policies since it reflects the contextualization of the problem. Moreover, indicators also reflect the different dimensions considered to define EP: income, energy efficiency, energy consumption and expenditures. Therefore, EP combines the complex problematic nature and mix of different fields, which are common points addressed in the policy mix studies.

3. Framework for policy mix analysis

Since policy mix and its analysis do not have a universal definition or methodology, all the framework was developed considering different information, patterns and convergences, summarized from different topics. For this reason, the goals were defined based on literature and organized into different themes (Table 1).

Table 1. Goals that justify the adopted methodology to their respective literature theme.

Theme		Goal	Source
Policy mix evaluation framework	1	Maximizing the effectiveness or efficiency of the complementarity of the instruments involved.	(Schröter-Schlaack et al. 2013)
	2	Appropriate mix of instruments and actors.
	3	Consider target groups.
Interaction analysis	1	Focus on existing or proposed instruments, analyze two or more instruments, and finally aims to identify possible conflicts or synergies between these instruments	(Ring and Schlaack 2011)
	2	Comparing the scope of the instruments, the nature of the objectives, the timetable of the instruments, the operation of the instruments, and the process of implementation
Coordination	1	Develop reinforcement mechanisms; address possible negative interactions.
Policy mix analysis	1	policy mix analysis does not primarily ask whether one instrument is more effective or efficient than another, assuming only the more effective instrument should be used. The interesting question for policy mix analysis is on the interaction between instruments.
Keywords for the analysis process	1	Policy coherence	(OECD 2007; Ring and Schlaack 2011)
	2	Positive/negative interactions
	3	Policy context

In summary, the framework presented herein aims to answer the following topics:

• Policy mix evaluation framework: what are the different goals which should be considered when defining the methodology of the analysis?

• Policy mix analysis: what should be the intrinsic objective, id est, what is the focus of the analysis and its contribution to the policy mix topic?

• Keywords for the analysis process, interaction analysis and coordination: the various criteria and support for the framework adopted in this work.

The framework is essentially divided into five steps (Figure 1): (1) definition of objectives, (2) instrument selection, (3) single instrument analysis, (4) instruments interaction analysis and (5) evaluation. Figure1 illustrates the iterative process and the policy mix analysis framework, presenting its previously explained five steps and the secondary steps (contextualization and pathway).

Figure 1. Framework for policy mix analysis.

3.1. Instruments selection

The instruments, a term used herein to aggregate policies, programmes, initiatives, and other elements, should be selected based on the topic of assessment. The evaluators should consider the context of the analysis to define the goals and key factors to apply the framework which will affect the instruments selected for analysis. In this situation, it is important to define criteria to limit the number, the type of documents, and the elements of the policy mix that will or not be covered.

There are different pathways for policy mix analysis, and they are not discrete or continuous. It is important to understand the specific context of policy mix to select which pathway will be followed and if it is necessary to combine and adapt different phases (Figure 2) (Mulligan, Lenihan, and Doran 2017; Samset and Christensen 2017).

Figure 2. Policy phases and its pathway for analysis adapted from (Samset and Christensen 2017).

In the ex-ante case, a single or various new instruments are applied to the already existent background. Henceforth, the performance of the single instrument and its additional value or conflict are analyzed in the idea and decision phase and at the start of the implementation phase (Samset and Christensen 2017). It should be noted that the policy mix does not necessarily need to be already recognized, but the elements which are already being developed or implemented can be analyzed to understand how they do or do not contribute to a new topic contribution to manage and correct mistakes. This is especially important to recognize since politics are rapidly changing, and more and more instruments are being developed to answer complex issues requiring coordination and integration between different policy fields.

The ex-post case aims to analyze the existent policy mix and evaluate the selected instruments and their interactions using criteria for single instruments analysis and designing policy mixes. Moreover, an ex-post analysis will especially consider the outcomes of the policy, considering the final implementation and the operational phases.

The instruments used for ex-post analysis, already require information on the outcomes of the policy mix. Then, this is the more significant difference between the ex-ante and ex-post analysis. And the instruments selected can define the pathway, which, consequently, will relate to the objectives of analysis and influence the selection itself. Hence, the choice of instrument is an iterative process that considers the context of its application and requires the capacity to learn about the subject. At the same time, the goals, pathways, and criteria are selected and adapted in the framework.

All things considered, the instrument's selection should contemplate and influence the objectives of the analysis. This iterative process requires understanding the dimensions and interactions which will be considered and how the instruments selected can provide sufficient information to support the application of the criteria for evaluation. The policy field contextualization is what generates the structure to choose the pathway to consider.

3.2. Single instrument analysis

The single instrument analysis will depend on the pathway, as has already been explained for the ex-ante and ex-post cases selected for study.

Considering the pathway for analysis and the context of the policy mix, the evaluator should define the information to collect and the criteria to evaluate these instruments individually. In this framework, the focus of analysis is not the individual instruments but their interactions since their mix and, consequently, interactions distinguish the single policy analysis from the policy mix analysis. Therefore, this step should aim to collect information to support the analysis of their interactions. Two hypothetical examples are defined to clarify how pathways and context can change the information to collect and how to organize it:

• Hypothesis 1: the analysis aims to understand how different instruments interact between various dimensions, goals and measures and contribute to a broad policy mix. In this case, the single instruments analysis will collect and categorize the goals, measures, and other design features to follow the instruments’ interactions and final evaluation. Since we are at the start of the implementation phase, it is impossible to collect the policy mix's impacts and, consequently, only the official documents will be considered.

• Hypothesis 2: the analysis aims to understand how the existent policy mix contributed to achieving the policies’ goals and how measures were the drivers for achieving those goals. In this case, different information assessing the indicators change and other reports will be organized and related to the measures. Finally, the interaction analysis will identify where the instruments contributed positively or negatively to their impacts. The evaluation will focus on the effectiveness and efficiency of the policy mix.

Overall, different organization methods and information can be used and collected. This aims to give the evaluator an understanding of the instruments, which will help identify the policy mix's patterns, loopholes, and design features. Moreover, it is also important to note that different instruments can be selected for analysis and, therefore, this iterative process will require the capacity to adapt the data assessed.

3.3. Instruments interaction analysis

After the single instrument analysis, the information collected should support the categories of instruments interactions. Interactions aim to visualize how the different instruments are related and identify the counterproductive or complementary aspects of their relations.

The proposed framework for policy mix analysis gives special attention to this step. The complexity of policy mix is to understand how instruments affect each other and, in an ex-post scenario, the consequences of their implementation can be evaluated to further application of the policy mix.

A homogenous categorization of the instruments’ interactions permits to directly compare their relations and identify the aspects which are more relevant or require special attention (Flanagan, Uyarra, and Laranja 2011; Milhorance, Bursztyn, and Sabourin 2020). Therefore, four different elements are proposed for categorization: (1) types, (2) dimensions, (3) forms, and (4) combinations (Table 2).

Table 2. Types, dimensions, forms an,d combinations of interactions.

Types	Direct interaction	Indirect interaction	Operational interaction	Sequencing interaction
Dimensions	Policy space	Governance	Geography	Time
Forms	Different instruments, same actors or group	Other instruments, different actors or groups	Different instruments, different processes in broader ‘systems.'	Within the same instrument
Combination	Inherently complementary	Inherently counterproductive	Path dependent	Context-specific

First, four types of interaction are defined as:

• Direct interaction: there is a clear influence between instruments, and the change in one affects the other one.

• Indirect interaction: there is an influence between instruments, but a change in one does not necessarily affect the other.

• Operational interaction: the instruments operate together, but a change in one does not necessarily affect another.

• Sequencing interaction: the instrument is followed by another one in time.

However, another definition of different interaction types can also help understand how to distinguish these interactions. Milhorance, Bursztyn, and Sabourin (2020) identify four types of interaction: (1) enabling, (2) facilitating, (3) consistency and (4) synergy.

Second, the dimensions and forms of interaction were based on different studies (Flanagan, Uyarra, and Laranja 2011; Rogge and Reichardt 2016; Sorrell 2003). The dimensions aim to assess how the policy mix presents interactions within the policy space, governance, different geographic spaces and in the time of development and/ or implementation. In the case of forms of interaction, it is possible to distinguish if the mix occurs in the instruments, actors, our groups and/or within the instrument.

Lastly, a combination of instruments was divided into the four categories. Those categories are the combination of Ring and Schlaack (2011) and Gunningham, Grabosky, and Sinclair (1998) definitions and their adaptation to EP instruments:

• Inherently complementary: their interaction positively influences one another.

• Inherently counterproductive: their interaction negatively influences one another.

• Path dependent: their interactions can be positively influenced by introducing a new instrument or reforming existent instruments.

• Context-specific: the instruments can positively influence each other depending on the context of their interaction.

3.4. Evaluation: criteria selection

As a result of the collected information and analysis, the final step is to evaluate the policy mix based on selected criteria. The criteria suggested were based on different authors’ collections (Rogge and Reichardt 2016; Mavrot, Hadorn, and Sager 2018; Samset and Christensen 2017; Mulligan, Lenihan, and Doran 2017) and should be selected based on the context and objectives of the policy mix analyzed.

Consistency, coherence, credibility and comprehensiveness are the four major criteria considered for analysis and are defined by the following questions by Mavrot, Hadorn, and Sager (2018)

• Consistency: Are the policy mix elements aligned and work towards the same goal?

• Coherence: is the policy mix synergic and systematic?

• Credibility: is the policy mix believable and reliable?

• Comprehensiveness: is the policy mix extensive and exhaustive?

However, the context of the policy mix will limit the type of criteria used. For instance, credibility requires understanding how actors or other groups respond to the policy mix, which consequently means the instruments selected need to support this conclusion. Furthermore, other criteria can also be used depending on the objectives, context, pathway selected and single instrument analysis. The evaluators should test and adapt these criteria to their needs (Mavrot, Hadorn, and Sager 2018; Samset and Christensen 2017). The following criteria have special contributions to the ex-post scenario and the evaluation of the policy process:

• Stability: how did the policy mix changed in the long term, and did it affect its stability?

• Adequacy: is there a concordance between the ex-ante and ex-post scenario?

• Effectiveness: were the goals achieved?

• Efficiency: how the resources were used to achieve the goals?

Finally, it is also possible to evaluate the type of policy mix (Table 3) present in the analysis effectuated. In this case, it is proposed to assess the policy mix based on the previous work of Bouma et al. (2019) and Howlett and Del Rio (2015), which consider the number (single or multiple) of instruments, objectives, and government. Moreover, this typology could clarify if the instruments selected are a policy mix situation or instrument mix. Based on Howlett and Del Rio studies, the policy mix occurs when multiple governments are implicated.

Table 3. Typology of policy mix (S- single and M- multiple) adapted from Howlett and Del Rio (2015).

Policy mix category			Typology
Instruments	Objective	Government
S	S	S	Simple single-level instrument mix
S	M	S	Complex single-level instrument mix
S	S	M	Simple single-level policy mix
S	M	M	Complex single-level policy mix
M	S	S	Simple multilevel instrument mix
M	M	S	Complex multilevel instrument mix
M	S	M	Simple multilevel policy mix
M	M	M	Complex multilevel policy mix

4. Energy poverty application

Policy mix depends ‘on the dynamic institutional context in which the policies are embedded’ (Magro and Wilson 2019), and energy poverty ‘exhibits great context specificity’ (Sareen et al. 2020). As identified by Sareen et al., a dilemma appears related to the ‘inexistence of universally optimal balance between context provision and removal’ since ‘if data come with too little context, accountability relations are weakened or destroyed, but providing too much context may be overwhelming and paralyze action’.

Understanding how policies for EP are designed is a good step to choose the information which should be collected in the instrument analysis step of the framework. While drafting policies to address EP, EPOV, and now EPAH, recommends the focus on four different topics: (1) measurement, (2) definition, (3) policy type, and (4) financing & funding (Table 4).

Table 4. Policy guidance on energy poverty. Source: adapted from (EPOV 2020a; EPAH 2022b; Palma and Gouveia 2022).

Measurement	Energy expenditure and income: quantify energy poverty by looking at the energy expenditure of households and income. Self-assessment: assess energy poverty by asking households directly to what extent they feel comfortable or able to afford energy. Direct measurement: measure physical variables to determine the adequacy of energy services. Proxy indicators: give an impression of the energy poverty situation through related factors, such as arrears on utility bills, number of disconnections, and housing quality.
Definition	Socio-economic and demographic groups: some socio-economic and demographic groups are particularly vulnerable to energy poverty. Housing composition: certain housing situations are known to present certain risks in terms of energy poverty such as single parents, people with disabilities or pensioners Energy carrier: use of specific energy carriers could be related to energy poverty. For example, in certain countries, the use of expensive and inefficient heating oil boilers would make households more vulnerable to being energy poor. The policies & measures section allows selecting of example policies that target specific energy carriers. Location: certain areas are known to have more households in energy poverty (e.g. rural vs urban).
Policy type	Financing support for energy efficiency improvements (and renewable energy sources uptake as Solar photovoltaics) are the most preferred option to solve energy poverty structurally. Energy audits are visits to vulnerable households to provide direct advice on how to improve their specific energy vulnerability situation. Financial assistance to reduce energy bills or ease the difficulty to pay can be given in two ways: social tariffs and energy bill payment support. Disconnection bans provide protection against disconnection for households, often in colder months during wintertime. Information and awareness are measures that indirectly facilitate households to improve their situation by providing advice, information, or education for behavioural changes or best solutions. Social support provides general income support for households to cover more general expenses.
Financing & funding	Financing specific improvements through tax incentives, grants and loans. Measures can be funded through public funding, private funding, public-private partnerships, or levies.

First, measurement can be considered in this framework as the contextualization of EP problematic. Second, the definition of an energy-poor and vulnerable household can be taken when analyzing the target groups of the instrument or general goals. Third, the policy type can be accessed when considering the instruments’ measures presented to tackle EP. Lastly, the fourth suggestion on finance & funding can be viewed by analyzing the different measures applied in the instruments and identifying the actors that are directly or indirectly related to them.

In the case of EP, instruments can be first subjected to identifying target groups, measures, and indicators. The indicators can be surveyed considering the criteria used by Siksnelyte-Butkiene et al. (2021) presented in Table 5. This selection was based on the literature on EP and the policy guidance provided by the EPOV and EPAH (EPOV 2020a; EPAH 2022a).

Table 5. Criterion and Sub criteria to evaluate Energy Poverty Indicators.

Criterion		Sub criteria
		Title
Objectivity	1	Indicators reflecting economic dimensions of the problem are involved
	2	Indicators reflecting social dimensions of the problem are involved
	3	Indicators reflecting environmental dimensions of the problem are involved
Transparency	4	Methodology for indicator selection is presented
	5	Links to data sources are clearly indicated
	6	Data sources are publicly available
Practicability and Flexibility	7	A system for indicator calculation is quite simple
	8	The indicators set can be easily adapted to another research
	9	The methodology for calculation is presented (or links provided)
Participation	10	Stakeholders or experts are involved in the process of selection of indicators
	11	Stakeholders or experts are involved in the determination of weights of the indicators
	12	Scientific methods are used for stakeholders or expert participation (selection, concordance of opinions)

Source: adapted from (Siksnelyte-Butkiene et al. 2021).

Considering that the process of measuring EP suffers from the multi-dimension and contextualization nature of the EP problem, in Table 6, three levels are proposed for assessing EP: macro, meso and micro (Siksnelyte-Butkiene et al. 2021). These three levels suggest how to apply the step of Instrument interaction analysis since it makes it possible to observe how different levels of policies will interact in the EP case.

Table 6. Levels and dimensions of Energy Poverty Assessment.

Level of assessment	Economic dimension	Social dimension	Environmental dimension
Macro (e.g. EU level)	Competitive energy prices Energy consumption Energy dependence Energy balances Policy actions	Socio-economic and demographic household characteristics Health Social policy actions	Energy efficiency Renewable Energy Greenhouse gas emissions Environmental, energy policy actions
Meso (e.g. National level)	Competitive energy prices Energy expenditure Presence or absence of political actions/ initiatives	Socio-economic and demographic characteristics Health	Energy efficiency of technologies used Renewable fraction Greenhouse gas emissions
Micro (e.g. household level)	Energy access Energy consumption (expenditure) Energy prices Income	Socio-economic and demographic characteristics Health Habits	Housing characteristics Energy efficiency of technologies used Renewable fraction

Source: adapted from (Siksnelyte-Butkiene et al. 2021).

The single instrument analysis application aims to consider the existent information on EP policies and/or should contemplate the problem. Then, the essential information collected can be regarded as a posterior analysis and support of the instruments’ interactions. Therefore, the homogeneity of the categories is an important factor in comparing the instruments better.

In summary, the framework can be applied to the EP policy mix analysis by collecting the target groups, measures, indicators, and actors during the instrument analysis, where consistency can be used as the main criterion. Then, during the analysis of instruments interactions, coherence can appear as the primary criterion for understanding how the instruments are selected to interact (figure 3). These suggestions are based on the literature previously reviewed and the policy guidance of EPOV, which highlight the most important information to consider when designing policies for the EP field.

Figure 3. Policy Mix evaluation criteria applied to Energy Poverty.

5. Discussion and conclusions

Policy mix analysis is a growing topic in the developed policies for sustainable transitions. Combining different goals, instruments, and government levels is becoming more common and necessary for the problems covered in new policies. In this way, EP appears as an example of the need for a mix of policies for its mitigation.

A new framework is proposed to disentangle the complexity of policy mix and its elements, which makes possible the comparison of policy mix by its typology and the criteria for evaluation defined. The instrument interaction analysis and the adaptive feature of the framework to the pathway are the major contributions of this work by making its application to different policy fields.

The framework for policy mix analysis was backed by state-of-the-art literature on the subject. Different methodologies for policy analysis in different fields, topics, and goals helped deconstruct the framework patterns and the policy mix concept itself. Thus, the framework proposed contemplates five major steps – definition of objectives, instrument selection, single instrument analysis, instruments interaction analysis, and evaluation – and an iterative process that forces the importance of contextualization and the pathway, ex-ante or ex-post, of the policy mix, addressed. In other words, the proposed framework cannot only be adopted for EP but is open for different topics to evaluate the policy mix with the adequate and proposed criteria. In summary, the major contributions of the proposed framework are:

• Disentangle a robust and extensive combination of instruments, condensing the relevant information on EP of instruments that encompass different goals, fields, and measures.

• Identify each instrument's targets, indicators, measures, and actors under the EP topic.

• A flexible yet systemic analysis process allowed one to visualize the interactions and find the instruments contributing more to the policy mix.

• Define the taxonomy of the policy mix.

• Categorize the instruments’ interactions based on a homogenous method which allows to compare and visualize the different interactions directly.

The complexity of the current policies which address energy, climate transition, and social matters requires an understanding of the multi-level instruments and interactions between them so they successfully meet their goals. For instance, EP eradication in the EU requires not only understanding how the European directives are able to eradicate this issue but also how they will affect the existing national policies or work together to achieve this goal. This framework is an important step for policymakers and researchers to not only understand how new policies, such as REPowerEU, will work together to sustain the continuity to eradicate EP but also evaluate how previous EU policies, for instance, the Energy Efficiency Directive, has affected the national plans on EP. The framework's capability to maintain a common analysis process facilitates the adaptation of these types of policy mix and the visualization of their benefits or prejudices to the matter being analysed.

In conclusion, the framework proposed encompasses an iterative process, and its flexible and systemic structure helps to disentangle the exhaustive works to overview the contributions of the single instruments to a defined topic and to visualize the interactions between them. Therefore, different pathways can be chosen, and the proposed framework can address various policy fields, as has been suggested for the case of EP, and to understand how the current policies for the EU energy transition will affect those in EP.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

The authors acknowledge and are thankful for the support provided to CENSE by the Portuguese Foundation for Science and Technology (FCT) through the strategic project UIDB/04085/2020.