Indicators: from Counting to Communicating

Abstract

The use of indicators for analysing quality of life conditions and differential access to resources and opportunities has gained attention in both academia as well as the planning practice and policy arena. However, teaching methods and usage of indicators is a challenge since it requires the practice of going through all the steps of developing indicators in a short period of time. These steps range from counting, construction, and operationalisation of the indicators to more policy and learning related steps that require a critical and reflective process. This gradient (from the traditional and mechanistic to the critical) is also reflected in the evolution of the indicators movement. To analyse and illustrate this development, we documented the education, fieldwork and research on indicators carried out at our faculty in the past 20 years. Our experience suggests that indicators have a great potential to bring the worlds of academics (research and education) and of planning and policy making closer together. However, we believe that in the education process it is of utmost importance to emphasise the shift in the role of indicators from one of traditional counting and description towards a more critical and communicative role. We present a framework to show the existing dichotomies and how to incorporate this shift in indicator development.

Keywords:

• indicators
• learning and teaching indicators
• indicators process

Introduction

The use of indicators for analysing quality of life conditions and differential access to resources and opportunities has gained attention in both the academic as well as the planning practice and policy arena (Wong 2000, 2006, Ghose & Huxhold 2002, Bovaird & Loffler 2003, Dopheide & Martinez 2007, Baud et al. 2009, Martínez 2009, Tesfazghi et al. 2010). The geographic variation of cases in the literature shows that the use of indicators to generate policy relevant knowledge covers both the global north and south. This increasing interest is probably influenced by global monitoring campaigns such as the Human Development Index, the Millennium Development Goals, UN-Habitat Urban Indicators and Global Urban Observatories. This also reflects the different scales at which indicators are applied from the local to the global.

This interest in indicators is evidenced in the inclusion of indicator methods in the curricula of geography, urban planning, and development studies. However, teaching methods and usage of indicators is a challenge since it requires the practice in a short period of time of the different steps of developing indicators. These steps range from counting, construction, and operationalisation of the indicators to more policy and learning related steps that require a critical and reflective process. This gradient (from the traditional and mechanistic to the critical) is also reflected in the evolution of the indicators movement that we present in the next section.

We understand indicators as qualitative or quantitative data that describe features of a certain phenomenon and communicate an assessment of the phenomenon involved. Indicators are selected specifically for their relevance to policy issues and are related to a specific time and place. With this definition we want to emphasise the policy relevance of indicators in particular if they are used for informing planning.

One of the strengths of indicators as a tool for planning is that they have the potential to communicate complex issues in a simple and understandable manner. They help in the formulation of explicit goals and in the development of a shared vision on relevant issues. They have the advantage of being objectively verifiable and they can also help in monitoring and assessing progress. They are also a powerful communication tool that can inform the public, generate awareness on various issues of concern, and encourage action and empowerment. Also at the international level, it is recognised that the design of good indicators has to reflect these matters (United Nations Centre for Human Settlements 2000). Moreover, which is particularly relevant in the context of governance and participation, if indicators are properly developed they promote transparency and accountability. However, indicators are not always capable of fulfilling the above mentioned potentials and this is reflected in the fact that they have been used for decades with intermittent degrees of acceptance, embedment and success.

The aim of this paper is twofold: to critically analyse the claimed potentials and limitations of indicators; and to provide a framework that can better inform the teaching and understanding of indicators practices as well as facilitate a shift from utilising the mere counting function of indicators towards their use as a tool for critical understanding.

In the second section of this paper we describe the origins of indicators, outline the path towards their present use and distil dimensions of indicators development. The third section presents several research and fieldwork projects utilising indicators carried out in our faculty. (To reach this aim we revisit education, fieldwork and research activities on indicators carried out at our faculty in the past 20 years.)

In the fourth section we point to some of the dilemmas, problems and limitations that emerge with the use of indicators. Finally we conclude by giving some guidelines and proposing a framework for teaching and developing indicators.

The origins and dimensions of indicators development

The use of indicators and geo-information at sub-city level could be traced back to Charles Booth's Poverty Maps in London at the end of the 19^th century (see: http://booth.lse.ac.uk/). Wong (2006) identifies the origins of quantitative indicators used for policy making in the 1940s with the appearance of the Monthly Economic Indicators in the United States. The term social indicators and its dissemination started a decade later (Wong 2006). An example of the interest in the geographical aspects of social indicators can be found in the work of Smith (1973) who carried out a study of the city of Tampa using 47 ‘intra-city indicators’ and a general index to measure general social well-being. This also relates to a concern about spatial disparities and spatial injustice in geographical studies that were accompanied by an interest in influencing public policy. Social indicators and the interest in capturing multiple dimensions was also a reaction to indicators that only captured economic dimensions (e.g. GDP/capita). Non-monetary dimensions of well-being were later taken up by the Human Development Index (United Nations Development Programme 1990, Natoli & Zuhair 2011).

In the 1980s there was a loss of interest in social indicators that could be explained by a criticism of the reductionist character of indicators and a moving away from quantitative analysis. Some of the critique towards the early social indicators movement of the 1960s was that they emphasised the measurement task, while often excluding political and institutional aspects (Innes 1990). Data reliability and problems of interpretation (among others) were also signalled (Knox 1978, cited in Wong 2006). A lack of interest in social indicators could be also explained by a loss of concern and interest in reducing disparities. On one side post-modernism emphasised or ‘celebrated’ diversity and difference (Smith 1994) and ‘governments increasingly opted for the “magic of the market” rather than social intelligence and became less interested in social engineering and reform’ (Miles 1985, cited in Wong 2006, p2).

In the 1990s a renewed interest started in social indicators that could be explained by the growing social inequality in Western societies and a focus on ideas of social exclusion that defines poverty also in terms of social participation (Knox & Pinch 2000). At the same time there has been a growing engagement of geographers with inequalities and moral and social issues, including the theme of the ethics of professional practice (e.g. Couclelis 1999). The increase of interest in indicators in general was related to a growing interest in quantification; more tangible objectives (thinking SMART); accountability; transparency; and managerial approaches (Bovaird & Loffler 2003). It is also in the 1990s that usage of indicators grew worldwide (Wong 2006). This is also reflected in the growing interest on indicators in the urban agenda at the United Nations (UN). The United Nations Human Settlements Programme (UN-HABITAT) commenced the collection of urban indicators as one of its objectives, to capture essential information on cities and to monitor the performance of cities in relation to desired policy goals (Flood 1997). In 1991, UN-Habitat initiated the Housing Indicators Programme, focusing on monitoring performance in shelter issues. This led to the establishment of the Urban Indicators Programme in 1993 with a focus on a larger range of urban issues (Flood 1997). The most recent Global Urban Indicators Database (UIP-III) produced by UN-HABITAT's Global Urban Observatory (GUO) continues to address the Habitat Agenda key issues, with a specific focus on the Millennium Development Goals (MDG), particularly, its Target 11 on the improvement in the life of slum dwellers (Martínez et al. 2008). In that sense the MDGs could be seen as the latest ‘global showcase’ of indicators attached to targets and goals. An increasing role is also given to the communication and diffusion of indicators produced by the UN on the web (e.g. www.devinfo.info/urbaninfo).

In many cases along the history of indicators, they were selected for their relevance to policy issues. In the 90s in particular, indicators usage was promoted with the claim of supporting evidence-based policies. The case of the United Kingdom (UK) is one of the examples of the government push for evidence-based practices (Wong 2006, pp36-38). In that way indicators acted as mediators between policy and research. However, there is not a monolithic view on how to approach indicators within this intersection of policy and research. Davoudi (2006, p15), explains two different views about the policy-research interface: the instrumental view and the enlightenment view. In the instrumental view the relationship between evidence and policy is assumed as ‘unproblematic, linear and direct’ and either research leads policy (evidence-based policy) or research is shaped by policy. In the instrumental view indicators can be chosen and linked to a particular policy goal (policy-related research). Once indicators are implemented and they trigger new policies they are clearly following the evidence-based policy domain.

Davoudi (2006, p16) advocates for the enlightenment view where emphasis is on evidence-informed policy. In this case the role of research would be ‘less one of problem solving than of clarifying the context and informing the wider public debate’ (p16). Also Cobb & Rixford (1998) were referring to this enlightenment function of indicators when they state that ‘it is the power of indicators to alter the common understanding of a problem – not merely to point out the problem’ (p25).

Considering these two views, the present and future role of indicators will better fit into the enlightenment model. One of the misconceptions of the instrumental view is that it considers policy making as a rational process where experts are apolitical and value-free (Davoudi 2006). In relation to problem-solving and decision making, policy choices and decisions are value judgements and cannot be determined by indicators (Wong 2006). On the contrary, in a real policy process issues such as ‘ideology, interests, institutional norms and practices and prior information’ play a key role in influencing decisions (Weiss 2001, cited in Davoudi 2006, p19). Indeed, research on the actual use of indicators (Higginson et al. 2003, Martínez 2005) show that some policy makers feel that indicators are not yet influencing action or decision making as much as they would expect. One of the explanations given is that ‘it is not always clear how policy is made and how information should be fed into the policy making process’ (Higginson et al. 2003, p6). Insufficient communication of indicator findings is also mentioned as a problem (Innes 1990, Higginson et al. 2003).

A critical view on the increasingly normative global agenda (e.g. MDG indicators) is given by Jenkins et al. (2007, p184) since it “assumes that ‘fair play’ can exist, as long as information is clear, analysis correct and dialogue possible” and ignoring e.g. power relations and the fact that “the way knowledge is selectively produced also conditions the form of analysis – whoever sets the rules and agenda dominates the action and dialogue”.

Indicators as any evidence will not trigger a decision but they will rather better inform and shed light on the problems that ultimately policy makers will tackle. Therefore the capacity and function of indicators as a learning and communication tool should be emphasised.

Figure 1 presents two dimensions identified in the evolution of the indicators movement. One represents a mechanistic vs a critical construction of indicators ranging from indicators understood predominantly as data, counting and number crunching; to indicators understood as knowledge, understanding, and communication. See also for Cobb & Rixford (1998) p31, who contrast indicators that help to develop and test the validity of cause-effect relationships rather than just pointing to the problems themselves.

Figure 1 Dimensions of indicators development.

Another dimension represents two opposing views in the process of indicators construction: indicators as an ‘objective’ exercise with ‘value-free’ information; and indicators as an inclusive process characterised by collaboration, communication, feedback, sharing, and empowerment. With these two dimensions we do not intend to provide a precise and comprehensive typology but rather reflect on what we observe as existing opposing practices. The origin of indicators in the 1960s is most probably identified in the first quadrant of the mechanistic and objective dimensions (with an instrumental policy-research view). On the fourth quadrant characterised by inclusive and critical dimensions we could locate what we call the ‘critical-inclusive’ area of ‘learning indicators’. This fourth quadrant also comes close to the integrative double-loop indicator evaluation framework as proposed by Wong (2011, p15) that allows ‘key stakeholders to express their vision in the policy formulation process as well as providing a feedback loop to frame policy problems’. Also Alexander (2011) suggests that a comprehensive evaluation practice will have analytical-technical elements as well as participative-interactive ones. A shift from the first to the fourth quadrant is what we try to advocate both in the teaching process of indicators as well as in practices. Some characteristics of this shift emerge in the following section.

Indicators studies in the last 20 years at Faculty ITC

In this section we present in chronological order a selection of education and research activities related to urban and regional indicators within the Department of Urban and Regional Planning and Geo-Information Management at the Faculty of Geo-Information Science and Earth Observation (ITC) in Enschede (the Netherlands). Faculty ITC (University of Twente) has almost 60 years of experience in capacity building and institutional development of professional and academic organisations and individuals specifically from less developed countries in the field of geo-information science and earth observation and its applications (see: www.itc.nl).

Early 1990s: Awareness of relevance of indicators

This period of indicators development is characterised by a transition from mechanistic-objective approaches – typical of indicators usage since the 1960s – to more critical-inclusive approaches. This transition is guided by policy needs including issues of performance and accountability and awareness of the relevance of indicators.

The renewed interest on indicators in the 1990s is also reflected in research and education carried out at ITC. We present a selection of student projects and MSc studies that illustrate the process through which students ‘evolve’ in their indicator development. While in the first instance indicators development is very much data-driven and mechanistic, students realise – if the education process is facilitated properly – the need for iterative steps and feedback from stakeholders in problem identification, indicator construction, data collection and indicator use.

Around the 1990s a group fieldwork for students of the Survey Integration course took place in Chiang Mai, Thailand, which involved the setting up of an indicator system at district level; which could be compared with the actual allocation of public resources among the various villages. Another example is the work of Infante (1992) where (socio-economic) indicators were the research focus. Emphasis was then given to supporting planning in different phases and in particular to identify, rank and prioritise problem areas in the Philippine provinces. One of the recommendations was directed towards the improvement of data due to ‘the absence of reliable data’ (Infante 1992, p58). In the same direction, Infante indicated that because the Local Government Code of 1991 ‘stressed the need to put into maps all data relevant to local planning’ (p58) it would be recommended to apply Geographic Information Systems (GIS) in this framework of indicators. In general, Infante (1992, p58) concluded that ‘At the provincial level, a framework using socio-economic indicators is desirable and feasible’.

Mid 1990s: Relevance of link of indicators with policy

In this period of indicators development we observe the importance given to the linkage of indicators with policy and the emergence of more critical-inclusive approaches. The relevance of spatial units and analysis to inform policy was evident in the increasing use of methods supported by GIS. In teaching we have emphasised learning aspects of indicator development rather than pure data collection practices.

The use of GIS in the construction of indicators was present in a later work of Asensio (1997) on targeting the poor within Guatemala city. One of the research objectives was to ‘analyse poverty indicators on a spatial context’ (Asensio 1997, p11). One of the main advantages of GIS in the construction of indicators was present in her work. The advantage relates to the possibility of combining different data sources such as census data and remotely sensed images (aerial photographs). Most of the attention was given to compare the resulting targeted areas from the different data sources (census, AP). Some of the conclusions were that the use of ‘homogeneity criteria to delineate census enumeration sectors can help to avoid targeting errors and to show specific characteristics of socio-spatial segregation’ (Asensio 1997, p70).

A year later another work emphasised the importance of problem identification and the role of policy makers rather than geo-information tools such as GIS or Remote Sensing. The policy relevance of indicators is then incorporated in the framework of indicators of a study carried out in the Philippines (Figure 2). The research had the aim of exploring spatial variations among the municipalities in the province of Zambales (Arjagaruka et al. 1998). The set of indicators was based on field visits and discussions with local officials. Measurability and accuracy were also the criteria for selecting the indicators. Accuracy in this case was related to ‘the accuracy in indicating a gap between the actual and the desired situation’ (i.e. the problem) (Arjagaruka et al. 1998, p5). In the identification of the indicators the role of policy makers was clearly established. A number of reiterations based on feedback and discussions with the various local stakeholders were necessary to arrive at a consolidated problem analysis and problem structure. Also in the eventual definition of the indicators, students had to go through a number of rounds. The whole exercise illustrated very well the learning aspects of indicator development rather than being a simple data collection exercise. In that sense this exercise would fit in the fourth quadrant of Figure 1 emphasising the critical and inclusive dimensions. A challenge in the construction of the indicators was related to the lack of data coherence since data varied between agencies.

Figure 2 Spatial variation of poverty across the province of Zambales. Source: Arjagaruka et al. 1998.

2000s: Inequalities, globalisation

In this period of indicators development we observe a consolidation in the use of GIS and visualisation. This is primarily guided by policy needs seeking performance comparisons, benchmarking, and trend analysis at different scales: from the global (i.e. MDG) down to the sub-urban and community scale. These two scales are reflected in the following cases of inter-urban and intra-urban comparisons.

Inter-urban comparisons

At ITC we observe a similar correlation with an extensive use of GIS and sophisticated visualisation tools of indicators and an interest in global comparisons and benchmarking.

The research carried out on global indicators at ITC (Martínez et al. 2008) is a consequence of the increasing attention to the comparison of quality of life conditions across the world. The aim of this research was to statistically analyse trends and clustering in shelter and slum conditions in different cities in developing countries and to establish how these conditions are related to other Millennium Development Goals (MDG) indicators such as under five mortality. The research carried analysis of data contained in the Global Urban Indicators Database (UIP-III).

Figures 3 and 4 show how with indicators it is possible to communicate with a simple graph the different paths that cities took in the improvement of water access and the reduction of under five mortality. This is the case of cities like Fortaleza and Kampala where an increase of access to safe water (Figure 3) is also reflected in a reduction of under five mortality (Figure 4). Both cities belong to a cluster of cities that share similar characteristics according to a group of shelter indicators (Figure 5).

Figure 3 Changes in access to safe water for cities in cluster C.

Figure 4 Changes in under five mortality for cities in cluster C.

Figure 5 Map of city clusters based on six shelter indicators (Martínez et al. 2008). Cartographic representations: Connie Blok and Jeroen van den Worm.

Within the same research, attention was given to the communication and visualisation aspects of indicators and the possibility of producing advanced visualisation tools such as the interactive flash maps that were presented for the World Urban Forum 2006 (Figure 5). By clicking on the interactive map it was possible to visualise groups of cities separately or to display a synthesis of shelter indicators of a city in particular.

In this example, indicators are mostly used to communicate and advocate for specific policy actions (e.g. improve slum conditions), however the selection of indicators is related to an ‘objective’ and value-free process. Therefore, it falls within the critical-objective quadrant presented in Figure 1. This can be explained by its relation to a global normative agenda (Jenkins et al. 2007).

Intra-urban comparisons

Another research study reflects the increasing interest in the analysis of inequalities at intra-urban level. Here the issue is not to compare cities (as global indicators do) but to analyse differences within cities (Figure 6).

Figure 6 Contrasting quality of life conditions. Source: Martinez 2009.

Martinez (2005, 2009) presents a methodology that allows the systematic monitoring of intra-urban inequalities with the use of GIS-based indicators through an indicator matrix and an approach to incorporate a geographical component into the participatory budget allocation. GIS-based indicators are constructed combining different data sources such as census and administrative data. This methodology is applied in a case study in Rosario (Argentina) and demonstrates how urban indicators and GIS combined are a valuable tool to describe and monitor inequality aspects in quality of life conditions and access to services to better target resources. In this research the inclusion of the perspective of policy makers was relevant for the selection and validation of indicators. It also looked into existing practices of indicators usage.

It is commonly argued that with the construction of policy relevant information (as indicators are) it is possible to help policy makers to reduce inequalities, increase transparency and as a consequence improve governance. However, this research also shows through some interviews with policy makers in Rosario and Liverpool that there are some signs of saturation in the use of indicators. Comments such as “people are getting tired of this”, “there are times when you just drown in the number of indicators and targets” or “indicators fatigue” were given during the interviews (Martínez 2005, p116). This shows to some extent the problem of overloading policy makers with measurements, especially if there are overlapping sets of indicators or long lists of indicators. This is mostly related to an over valorisation of its performance measurement role. Some of these learning points are the result of the self-reflective characteristic of critical-inclusive approaches.

2000s: Indicators of derived and self-expressed needs

Another strand observed in the development of indicators during the 2000s – and in particular in the second half of the decade – is the inclusion of measures of subjective or perceived conditions. These measures are more in line with critical-inclusive approaches since they tend to incorporate citizens’ perspectives. This is very much related to an increasing interest by local governments to capture quality of life conditions and citizens’ satisfaction.

Martínez (2009) incorporated the concept of derived needs (indicators obtained from census data) and expressed needs (self-expressed needs derived from administrative data). This reveals the importance of not only considering indicators that measure actual (objective) conditions but also indicators that reflect the subjective perception of the population. This aspect is particularly followed up in the research carried out by Tesfazghi (2009) and Tesfazghi et al. (2010) in the city of Addis Ababa. The research analysed, with the help of GIS, the possible outcomes of combining objective and subjective quality of life conditions. Likewise, the research work of Kumar Dashora (2009) combined GIS visualisation techniques with qualitative methods to measure the quality of life at neighbourhood level in the city of Enschede (Figure 7).

Figure 7 Quality of life at neighbourhood level in Enschede-Wesselerbrink. Source: Kumar Dashora 2009.

The main learning point of recognising and including subjective measures in the development of indicators has trickled down in subsequent MSc research topics and teaching material. This is also the result of recognising multiple sources of knowledge in the construction of indicators (e.g. expert knowledge and local knowledge).

2000s: Capturing multiple dimensions

The last element in this period of indicators development – and reflected in the teaching of indicators at ITC – is the emergence in both the global north and south of indices of multiple deprivations. In most cases these indicators practices were triggered by the need to offer policy makers a comprehensive synthesis of indicators and inform the allocation of resources to the most deprived areas.

Although indices contain a certain degree of arbitrariness in their construction, Sharpe (1999, p50) concluded, based on a review of numerous economic and social well-being indicator and indices initiatives, ‘when taken together, they do and can provide a fairly accurate picture of general trends in well-being’.

The emergence of indices of multiple deprivations in the global south is the result of a shift from income or consumption approaches in poverty studies to multidimensional approaches (see e.g. Baud et al. 2008). This is reflected in the work carried out in India by a group of ITC students which also showed the importance of giving a general picture or a synthesis of several indicators in a single index (Houndebasso Ahoga 2009, Mishra 2009).

The use of indices of multiple deprivations diffused from the UK experience towards cases in the developing world and other European countries. This is also replicated in the work carried at ITC in relation to a critical analysis of the use of indicators to rank multiple deprived areas by the Dutch government. In the Netherlands, the Minister of Housing, Communities and Integration presented a ranking of the 40 most deprived neighbourhoods with the objective of allocating extra funds during the cabinet period. They were selected following assessment of a list of 18 indicators measuring the following domains: income, employment, education, nuisance and satisfaction of neighbours over their neighbourhood (Brouwer & Willems 2007). However, some Dutch municipalities did not agree with the ranking list and claimed that other neighbourhoods were in higher need than those selected (NRC Handelsblad 2007). Some of the municipalities even started to lobby to have neighbourhoods in the list (Buddingh & Thie 2007). This is an example of how in reality policy choices and decisions cannot always be scientifically and mechanistically determined by indicators, as in this case local policy makers have a different perspective than the central government selected indicators show. A criticism on the selection of those indicators by Dopheide & Martinez (2007) pointed at the lack of transparency both in the selection as well as the communication of the indicators.

The research carried out by Trisusanti (2008) with a case study in the Dutch province of Overijsel also showed how the resulting ranking of deprived areas can vary just by altering the methods that synthesised the indicators into a final index (Figure 8). In relation to transparency of methods she also stressed the necessity to check and show the sensitivity of different techniques to policy and decision makers.

Figure 8 Multiple deprivation across Overijssel. Trisusanti (2008).

Summary

Table 1 summarises this section, considering for each timeline period and case: (1) the purpose of the indicator study; (2) the learning points; and (3) the approach in relation to the dimensions of indicators development presented in Figure 1.

The main observation from Table 1 is that through this retrospective look at indicators cases at ITC we see that from the 1990s there is an increasing recognition of the importance of critical-inclusive approaches with an emphasis on the learning aspects of indicators development and self-reflective practices (which this paper is also part of). A variation on this inclusive approach is observed in the MDG indicators case where a ‘value-free’ and ‘objective’ process characterised the choice of indicators. It is at the end of the 2000 decade that we observe cases of indicators that incorporate multidimensional framing of indicators (e.g. in deprivation and quality of life) and the incorporation of multiple sources of knowledge (e.g. subjective and perceived conditions).

The experience and reflection on the various research and educational ITC activities allows us to indicate in the following section a number of pitfalls in indicator development.

Table 1 Summary of indicators studies in the last 20 years at Faculty ITC.

Timeline	Case	Purpose	Learning points	Indicators development approach
1990s Awareness of relevance of indicators	Chiang Mai, Thailand	Developing indicator system at district level	Compare with the actual allocation of public resources	Awareness and transition point from Mechanist-Objective quadrant to Critical-Inclusive. Includes issues of performance and accountability
Philippine provinces	Developing socio-economic indicators	Absence of reliable data and recommended to apply GIS
Guatemala city, Guatemala	Analysing poverty indicators on a spatial context	Homogeneity criteria to delineate census areas
Mid 1990s Relevance of link of indicators with policy	Zambales, Philippines	Exploring spatial variation among the municipalities	Emphasis on problem identification and the role of policy makers rather than geo-information tools such as GIS or Remote Sensing. Included discussions with local officials	Critical-InclusiveEmphasised learning aspect of indicator development rather than data collection
2000s Inequalities, globalisationInter-urban comparisons	MDG/Global indicatorsInter-urban comparisons	Analysing trends and clustering in shelter and slum conditions in different cities in developing countries. Communicating how these conditions are related to other Millennium Development Goals indicators	Communicate with a simple graph the different paths that cities took in shelter indicators improvements. Communicate and advocate for specific policy actions (e.g. improve slum conditions). Interactive indicators maps	Critical-ObjectiveSelection of indicators is more related to an ‘objective’ and value-free process. Relation to a global normative agenda
2000s Inequalities, globalisationInter-urban comparisons	Indicators usageRosario, Argentina. Liverpool, UK	Monitoring of intra-urban inequalities with the use of GIS-based indicator. Incorporate a geographical component into the participatory budget allocation	“People are getting tired of this”, “there are times when you just drown in the number of indicators and targets” or “indicators fatigue” were given during the interviews. Critique on the over valorisation of indicators performance measurement role	Critical-InclusiveThe inclusion of the perspective of policy makers was relevant for the selection and validation of indicators. It also looked into existing practices of indicators usage.
2000s Indicators of derived and self-expressed needs	Addis Ababa, Ethiopia	Mapping intra-urban Subjective and objective Quality of life conditions	Importance of combining objective and subjective perspectives	Critical-InclusiveMeasure subjective and perceived conditions which are more related to inclusive and critical than ‘objective’ and mechanistic use of indicators.
Enschede, The Netherlands	Analysing variations of QoL combining GIS visualisation techniques with qualitative methods
2000s Capturing multiple dimensions	Kalyan-Dombivilli, India	Water Poverty Index and Natural Capital	Indicator Synthesis.Complement Index of Multiple Deprivation
The Netherlands	Critique of 40 neighbourhoods selection	Local policy makers have a different perspective than the central government. Importance of transparency in indicators selection	Advocate to more ‘Critical-Inclusive’ approaches.[Critique and complementation of existing Indices of Multiple Deprivation]
Province of Overijssel, The Netherlands	Ranking multiple deprived areas	How the resulting rank of deprived areas can vary just by altering the methods that synthesised the indicators

Pitfalls in indicator development

One of the main limitations we observed is that if emphasis is on the measurement, the indicator becomes more important than the issue to be measured (the mechanistic dimension in Figure 1). In those cases data availability rather than the validity often determines the selection of the indicator; which could cause an unacceptably large gap between the measurement and the issue to be measured. Academics sometimes put too much emphasis on the number crunching rather than the actual meaning of the indicators; which becomes particularly clear in the construction of indices. The strength of a set of separate indicators to policy makers is sometimes underestimated. Wong explains how sensitive indices are to different weightings on individual indicators with the following metaphor:

The weighting scheme used to combine different indicators is indeed very similar to a cooking recipe that specifies the quantity of different ingredients to make a dish. It is always intriguing how the taste of the dish can dramatically change by simply varying the relative proportion of each ingredient used.

(Wong 2006, p81).

The advantage of a set of separate indicators is that they are easier to communicate to policy makers than complex indices that are not transparent to them (Bergström 1997). A related criticism is made by Smith (1994, p142):

Many individual indicators are highly correlated with one another, and although composite indicators can overcome problems of data overlaps and redundancy, the statistical techniques required are incomprehensible to most politicians and ordinary people (and to not a few academics), which tends to obscure the meaning of the results.

Criticism also arises because composite indicators are considered too reductionist to translate a complex reality, and choices of weights are not always clear. There is also the question of whether they are meaningful to the users, such as policy makers and urban planners.

Where ‘objective’ dominates over inclusive process (Figure 1) indicator initiatives take insufficiently into account the local needs and are too much driven by the interests of the higher administrative levels (e.g. standardisation of indicators, problem perception).

On the contrary, for a critical approach towards indicators it seems necessary to consider that: an indicator focuses on and renders only intentionally selected and quantifiable areas of the reality (Innes 1990). Eventually, ‘policy choices and decisions are value judgements, which cannot be scientifically determined by statistics’ (Wong 2006, p20). No set of indicators can show the richness and diversity of reality. There are issues such as informal economy, participation, and social cohesion which are difficult to measure.

There are also some limitations in the implementation of indicators. They can be time-consuming and discouraging for policy makers if different initiatives overlap, resulting in an overload of indicators. The value of local knowledge and qualitative analysis are clearly important to complement indicators and especially for internal and local use.

Based on such pitfalls and the framework presented in Figure 1, we propose some steps for indicators development that can improve learning of appropriate usage of indicators as well as facilitate a shift from use of the mere mechanistic and counting function of indicators to that of a tool for critical understanding. For the development of indicators some generic steps are usually recognised (see e.g. Wong 2006, p106).

Ideally, throughout all the indicators development steps, involvement of the stakeholders could be highly beneficial, particularly in view of the further acceptance, use and institutionalisation of the indicators. Particularly in the first and last steps, the importance of the involvement of different stakeholders and users of indicators is stressed (Figure 9). The last step related to the communication of the indicators closes the loop of the process bringing new insights to the problem perspective and helps the concept consolidation for future development of indicators. Emphasising the iterative characteristic of this process we propose this as an open spiral learning process. This concept relates to the educational concept of spiral learning and reflective practices. Based on Bruner's theory (Reigeluth 1999, cited in Sockman & Sharma 2008) the spiral learning contemplates that ‘learners master ideas gradually, after several cycles’ (Sockman & Sharma 2008, p1078).

Figure 9 Open spiral learning process for the indicators development. Steps adapted from Wong (2006, p106).

Conclusions

We have defined indicators as qualitative or quantitative data that describe features of a certain (urban) phenomenon and communicate an assessment of the phenomenon involved. Among academicians, but also among policy support officials with a strong belief in the conviction of the 19^th century mathematical physicist Lord Kelvin’s “If you cannot measure it, you cannot improve it” the emphasis in indicator development and use has been (and often still is) on the measurement and quantification of complex phenomena. The quantification of complex phenomena is certainly an important element of indicator initiatives.

However, our experience in a number of smaller indicator development activities in research and education at our faculty shows again and again that for a successful implementation and uptake of indicators stakeholder participation, iterations and communication throughout all the steps of indicator development, particularly in the identification and presentation, is paramount. The quantification in itself is insufficient to make the indicators attractive, valid and appealing for their use among policy and decision making. Feedback and learning among the various stakeholders involved is crucial. Successful indicators initiatives move away from the mechanistic-objective approach towards a more ‘critical-inclusive’ approach. This approach will pay explicit attention to the elements of communication, information sharing, collaboration, learning and empowerment.