Introducing an integrative evaluation framework for assessing the sustainability of different types of urban agriculture

ABSTRACT

In cities, a mosaic of different types of urban agriculture can be found. However, knowledge about advantages and disadvantages of the different types is still fragmented. This paper introduces an integrative evaluation framework for assessing the environmental, social, and economic sustainability of urban agriculture by applying a multi-criteria analysis based on an Analytic Hierarchy Process and a participatory approach. Based on a German case study and on the examples of vertical farming and community-supported agriculture, the results suggest that sustainable urban agriculture is a multi-dimensional approach informed by strong sustainability that places nature in the focus. Thus, the environmental dimension received the highest weight, followed by the social and, lastly, the economic dimension. Regarding the sub-criteria, species diversity achieved the highest total weight and food quality and safety the lowest. Conceptually, this paper provides scientific fundamentals for a systematic comparative evaluation of different types of agriculture for sustainable urban development.

KEYWORDS:

• Vertical farming
• community-supported agriculture
• biodiversity
• climate regulation
• circular economy
• participation
• education
• community building
• food quality and affordability
• local value chains

1. Introduction

In times of globalisation, technologization and urbanisation, human actions are intensifying the ecological crisis such as climate change and biodiversity loss (Zeng et al. 2020). In this regard, today’s established food system is argued to be based on environmentally unsustainable agriculture being responsible, for instance, for overexploitation of water resources (Dalin et al. 2019; Mall and Herman 2019) or adding to the global anthropogenic greenhouse gas emissions (Poore and Nemecek 2018). Biodiversity is also jeopardised such as by contamination of insects in nature conservation areas with pesticides used in surrounding agricultural areas (Brühl et al. 2021).

To promote sustainable agriculture in the face of the ecological crisis, there is an increasing interest in research and policy to foster urban agriculture. In fact, several review papers show the increasing attention on the role of urban agriculture for sustainable development (Artmann and Sartison 2018; Ferreira et al. 2018; Armanda et al. 2019; Azunre et al. 2019). Urban agriculture can be considered a generic term referring to urban farming or gardening including the production, processing, and distribution of livestock and food crops within and around cities (Skar et al. 2020). This generic definition includes a mosaic of different types of urban agriculture, which can be clustered in various ways. For instance, urban agriculture-types can vary in terms of their technological shaping. Controlled urban food production is currently gaining momentum in research and practice with a special focus on vertical farming, which links agricultural with architectural technologies aiming at increasing the amount of food supply and other social-economic ecosystem services in cities (Kalantari et al. 2017; Zaręba et al. 2021). Examples of nature-based food production include farming based on synergistic and regenerative agricultural practices such as permaculture, which aims at environmentally sound food production through minimisation of external input (e.g. energy, pesticides, water) and is argued to contribute to sustainable productive landscapes which bring nature back into cities (Watkins 1993; Rhodes 2012).

Specifying urban agriculture-types is in fact meaningful when it comes to the question of its contribution to sustainable urban development. Thus, the ecological, social, and economic benefits and challenges of urban agriculture can differ depending on its configuration. For instance, in terms of biodiversity, urban domestic gardens are more beneficial compared to commercial orchards at the urban fringe where higher pesticide-contamination is threatening certain bee species (Šlachta et al. 2020). When applying social science methods, a survey among various urban agriculture-stakeholders in an Italian case study found that peri-urban farming is perceived as providing habitat services in particular, whereby socio-cultural ecosystem services are supplied by socially oriented urban agriculture-projects (Sanyé-Mengual et al. 2020). Further research systematically comparing various urban agriculture-types is rare (Artmann et al. 2021). However, taking into account that open space in cities is limited, especially given the need to foster compact cities that might risk a decrease in supply of open and green spaces (Wellmann et al. 2020; Balikçi et al. 2021), what is needed are multifunctional edible urban landscapes (Artmann and Sartison 2018; Säumel et al. 2019). Therefore, an integrative assessment of urban agriculture-types based on a multi-criteria analysis can support strategic decision-making to identify suitable urban agriculture-types reflecting various ecological, social, and economic demands on urban food supply (Artmann and Sartison 2018).

However, taking into account the fact that urban agriculture embeds a mosaic of various types and related impacts, its strategic implementation into the urban fabric can be considered as a complex task. In general, in order to reduce the complexity of wicked problems and to foster solution-oriented sustainability, a systemic and participatory approach is needed. Thus, urban sustainability is confronted with governance challenges. Their addressing requires the integration of several actors and scientific disciplines that reflect various motives, meanings, and values of urban sustainability in order to foster a resilient, liveable, and equitable ecology for cities (McPhearson et al. 2016; Frantzeskaki et al. 2021). In fact, successful implementation of urban agriculture needs institutional support through urban planning and policy, but also engaged bottom-up initiatives such as non-governmental organisations and residents bringing success to edible cities (Nemoto and Ruoppolo Biazoti 2018; Sartison and Artmann 2020).

In order to strengthen a solution-orientated sustainability approach that takes into account a multi-criteria analysis and various actors, application of an Analytic Hierarchy Process (AHP) can support complex decision-making by evaluating different alternatives using multidimensional criteria and structuring them on the basis of a hierarchical decision model (Belton and Stewart 2002; Saaty and Vargas 2012). By incorporating preferences and priorities of different stakeholder groups using multi-criteria analyses and an AHP, effective solutions for urban sustainable development can be identified, such as those revealed through research in the field of urban regeneration (Awad and Jung 2022), transport planning (Ghorbanzadeh et al. 2018), or for measures involving urban climate and storm-water adaptation (Axelsson et al. 2021), to name only a few research fields. AHP and multi-criteria analyses have also been applied in the context of agricultural research, e.g. to identify sustainable agriculture in Iran with a focus on ethical sustainability (Veisi et al. 2016) or ecological modernisation (Rezaei-Moghaddam and Karami 2008). Further studies have analysed the land use suitability for agricultural activities by using an AHP (Akıncı et al. 2013; Pramanik 2016). In fact, an AHP was also applied to detect suitable land for various urban agriculture-practices in Sri Lanka (Weerakoon 2014) or with a wider focus on peri-urban geographies (Thapa et al. 2011; Ustaogl et al. 2021). However, the urban focus has so far been underrepresented and, in addition to land suitability analyses, an AHP was used in a Chinese case to foster multifunctional urban agricultural functions (Peng et al. 2015). To the authors´ knowledge, an application of multi-criteria analyses and an AHP to systematically compare concrete forms of agriculture in general and in an urban context, in particular, has not yet been undertaken. Such an integrative assessment is also meaningful for European policy-makers to raise awareness of the relevance of (peri-)urban agriculture for sustainable development. In fact, in Europe and its member states such as Germany the agriculture system requires deep sustainability transformation that would overcome existing negative impacts from the current mode of food production and consumption. This involves improvements of the unfavourable income situation of employees dependent on agriculture, the reduction of greenhouse gas emissions to a significant extent by agriculture, and the increasing of consumers’ low willingness to pay for food purchases (European Union 2020; Zukunftskommission Landwirtschaft 2021).

In the light of these research gaps and need for actions addressing sustainable urban agriculture, the objectives of this paper are twofold: 1) to develop an integrative sustainability assessment framework to evaluate various urban agriculture-types by taking into account a multi-criteria analysis, an AHP, and judgements of relevant actors in a European context and 2) to explore this framework based on the example of urban vertical farming and community-supported agriculture as in the case of Germany. Vertical farming is chosen as a representative of a technology-based urban agriculture-type, which can be considered an innovative, production-intensive urban agriculture-form with a strong market orientation (Specht et al. 2016). Vertical farming of food in high-rise buildings promises to contribute to food security through closed systems (Besthorn 2013) and may have environmental benefits such as a reduction in both space demand and water and energy consumption compared to conventional farming methods (Kalantari et al. 2018). Futuristic forms of vertical farming, such as those developed by Despommier (2011), are still in a pilot phase. However, there is a growing number of practical examples that produce food in cities via rooftop greenhouses or greenhouse factories (Specht et al. 2016; Appolloni et al. 2021). A more in-depth assessment of multidimensional advantages and disadvantages of vertical farming can further promote the implementation of such innovative forms of sustainable agriculture (Benke and Tomkins 2017; Kalantari et al. 2018). Compared to vertical farming, community-supported agriculture can be considered a nature-based type of agriculture that aims for local production of organic food while avoiding labour- and technology-intensive forms of cultivation for environmental protection goals (Bloemmen et al. 2015). Community-supported agriculture refers to a cooperating community of farmers and consumers who share the responsibility as well as the yield risk of food production. The cooperation includes the farmer, who usually ensures sustainable field management, and livestock, and provides food to the members of the consumer community, who provide the necessary financial resources for farming (Van Elsen and Kraiß 2012). Participatory farm organisations such as solidarity agriculture focus on producing healthy food, including food for low-income groups, and they minimise negative impacts on the environment; they can also be seen as a counter design to farms focused on increasing growth and efficiency (Lang 2010; Schnell 2013). The focus of community-supported agricultureal research is primarily on the social dimensions, such as ethical implications and social justice (Thompson and Coskuner-Balli 2007; Forbes and Harmon 2008), although there is still a lack of focus on an urban context. A multi-criteria analysis, such as suggested in this paper, can contribute to a further understanding of the environmental, social, and economic relevance of community-supported agriculture for sustainable urban development.

2. Materials and methods

For an evaluation of the sustainability of different types of urban agriculture, we applied a multi-criteria analysis based on an Analytic Hierarchy Process (AHP). The AHP is a methodology established by Saaty (1987) that enables multi-criteria decision-making and allows for an integrative participatory approach (cf. Liquete et al. 2016). The multi-criteria analysis is based on a three-pillar approach of sustainability that addresses environmental, social, and economic dimensions combined with the ecosystem service concept, which can be assigned, in general, to the three dimensions (Bastian et al. 2013) and, specifically, to the assessment of urban agriculture (Artmann and Sartison 2018). The steps of the AHP we perform can be described as follows (cf. Saaty and Vargas 2012): 1) definition of problem and goal, 2) state the hierarchy and define its elements, 3) pairwise comparison of the hierarchy elements and overall priority ranking of the sub-criteria, 4) development of indicators, 5) evaluation of the alternatives and sensitivity analysis of weights. Since the focus of the paper is to introduce a conceptual framework for an integrative sustainability assessment of various urban agriculture-types (see level 1 in Figure 1) in terms of its environmental, social, and economic dimensions (see level 2) and related sub-criteria (see level 3) only steps 1 to 3 will be presented in this article. The focus of the paper is on level 3 and the selection and weighting of sub-criteria based on a literature review and two online surveys. First, we conducted a literature review for preparing lists of sub-criteria suitable for assessing the sustainability of urban agriculture in general, which have been selected then by scientific experts in a first online survey (see section 2.1). For weighting of the selected sub-criteria in a second survey on the example of vertical farming and community-supported agriculture, we involve stakeholders representing key actors for the implementation of urban agriculture (see section 2.2).

Figure 1. The five levels of the performed analytic hierarchy process (source: own illustration adopted from Liquete et al. 2016).

2.1. Selection of sustainability sub-criteria (level 3)

In a first European expert survey, the most important sub-criteria per sustainability dimension (in the following called criteria) have been identified and selected.

2.1.1. Identification of experts

The survey was addressed to scientific experts in the field of urban agriculture. Potential experts were identified through web research of international urban agriculture projects (e.g. Urban Agriculture Europe, Urban Allotments, FEW-meter, FoodE, Edible City Network, proGIreg, Urban Allotment Gardens, URBAG) and their respective project partners. Due to the geographic focus of our study on Germany and in order to assume a comparable perspective of the experts, only those scientists with a focus on urban agriculture in Europe were included. Based on the personal profiles on ResearchGate or if not available by using other scientific platforms, databases, or websites (e.g. personal institutional websites), we counted the publications of the respective scientist since 2010 (mainly articles or books) that have a focus on urban agriculture (in general or on a specific urban agriculture type). A scientist has been considered an urban agriculture-expert if s/he has at least four suitable publications. The number of four publications was based on the rationale to obtain a sufficient but still manageable number of experts, and resulted in a list of about 120 experts.

2.1.2. Preparation of sub-criteria lists

Prior to the survey, we prepared lists of 11 sub-criteria for each of the three criteria, representing the three sustainability dimensions environment, society, and economy (see, section 3.1). Taking into account that there is, to the best of our knowledge, no final official list of how sustainable urban agriculture can be assessed for decision-makers in Europe or Germany, we based the lists on recent research, which provides an integrative assessment of urban food production taking into account environmental, social, and economic dimensions and its related ecosystem services. We referred to the literature reviews by the following: Artmann and Sartison (2018) which synthesised current research on urban agriculture as a nature-based solution and its corresponding ecosystem services, Clerino and Fargue-Lelièvre (2020) who identified sustainability objectives and criteria applicable to professional intra-urban farms, and Teitel-Payne et al. (2016) who focused on relevant and measurable indicators of the health, social, economic, and ecological benefits of urban agriculture. The 33 sub-criteria in our lists were linked to ecosystem services and were orientated to the Common International Classification of Ecosystem Services (CICES) (Haines-Young and Potschin 2018).

2.1.3. Development and implementation of the first online survey

The expert survey was realised online using SoSci Survey (Leiner 2019a), and the questionnaire was made available to participants at www.soscisurvey.de in the English language. The experts were invited and reminded via email twice to participate in the survey for which the email addresses of all experts were collected by web research on personal institutional websites or the authors’ affiliations according to their publications in advance. The survey started on 27 October and ended on 3 December 2020.

The experts should first assess the importance of the three criteria with regard to evaluating the sustainability of different urban food production types (level 2). Then, they should rate the importance of each of the 11 sub-criteria per criteria (level 3) by a 5-point Likert-scale (1 – unimportant, 2 – slightly important, 3 – moderately important, 4 – important, 5 – very important) (cf. Liquete et al. 2016). The experts were also asked to name up to three further important sub-criteria per sustainability dimension if they might have missed any. At the end of the survey, socio-demographic data were requested, and the experts had an opportunity to give general comments.

2.1.4. Data processing and analysis

Based on the results of this first online survey, means and standard deviation were calculated for each sub-criterion using IBM SPSS Statistics 25 to identify the most important sub-criteria for evaluation of the sustainability of different types of urban agriculture. The three highest rated sub-criteria per criteria have been selected and in some cases slightly amended under consideration of the experts’ general comments or their further named sub-criteria.

2.2. Weighing the importance of the selected sub-criteria

In a second survey, the importance of the selected sub-criteria has been weighted by German stakeholders.

2.2.1. Selection of case study cities

The second survey focused on urban agriculture in Germany. We have selected ten German cities as our case studies by using data from the Monitor of Settlement and Open Space Development from the Leibniz Institute of Ecological Urban and Regional Development (https://www.ioer-monitor.de). Since the focus of the study is on urban food production, the cities have been selected based on the number of inhabitants, population density, and share of built-up areas as basic indicators to analyse spatial developments of open spaces and built-up areas, such as those previously completed in Germany, as our project focus (Meinel et al. 2021). The case studies had to have at least 50,000 inhabitants representing large medium-sized cities according to the classification by the German Federal Institute for Research on Building, Urban Affairs and Spatial Development (BBSR) (191 German cities, status 2018) and more than 1,500 inhabitants per km² representing cities with a high population density (52 German cities, status 2018, cf. Dijkstra et al. 2019) together representing high-density clusters or urban centres (Dijkstra and Poelman 2014). For these 52 cities, we ascertained the percentage of built-up settlement area and transport space in relation to the reference area (status 2018). In order to take into account cities with different urbanisation trends that might influence urban agriculture (e.g. the availability of space for urban agriculture, urban agriculture governance, or food supply, Artmann and Sartison 2018), we selected the five cities with the highest (Herne, Munich, Oberhausen, Bochum, Gelsenkirchen) and the five cities with the lowest values regarding this percentage (Freiburg im Breisgau, Konstanz, Aachen, Augsburg, Dresden).

2.2.2. Selection of stakeholders

The survey addressed three target groups, the urban administrations, as well as non-governmental organisations of our 10 case study cities, and practitioners or technical-scientific experts for community-supported agriculture or vertical farming in Germany.

Stakeholders of the target group city administrations were included, as they are very important for the strategic implementation of urban agriculture (Langemeyer et al. 2021). They were researched by visiting the official administrative web pages of the 10 case study cities, searching for contact persons or addresses of the planning, construction, environmental, and social resorts.

Non-governmental organisations as important actors for the realisation of urban agriculture (Kanosvamhira 2018) were identified by web research for organisations within our 10 case study cities with a focus on gardening, environmental or nature protection, nutrition, and/or environmental education.

Practitioners or technical-scientific experts for community-supported agriculture or vertical farming in Germany were incorporated in order to take into account expert opinions in weighting the sub-criteria. Stakeholders of this target group were researched in two different ways according to the urban agriculture type. In particular, for community-supported agriculture practitioners, the German community-supported agriculture network (www.solidarische-landwirtschaft.org) was used. Due to the number of German community-supported agriculture farms exceeding 300, they were limited to farms in and around the 52 German cities with more than 50,000 inhabitants, and more than 1,500 inhabitants per km² in order to achieve framework conditions similar to the case study cities. Practitioners or technical-scientific experts for vertical farming were identified through a web research using keywords such as ‘urban farming’, ‘vertical farming’, ‘aquaponics’, or ‘roof top gardening’ each in combination with ‘Germany’.

Finally, the research resulted in a list of 500 addresses in total (226 addresses for representatives of city administrations (12–36 per city), 152 for non-governmental organisations (4–37 per city), and 122 for practitioners or technical-scientific experts).

2.2.3. Development and implementation of the online survey

Similar to the first survey, the second survey was realised online using SoSci Survey (Leiner 2019b): The questionnaire was made available to participants at www.soscisurvey.de in German.

The stakeholders were also invited via email and reminded twice. The survey took place from 14 February to 15 March 2022. We created three separate questionnaires, one for each stakeholder group, to have the opportunity to ask specific socio-demographic questions based on, for example, the experience and expertise regarding urban agriculture, which included questions in reference to the represented resorts (addressed to city administrations), the work focus (addressed to non-governmental organisations, practitioners, or technical-scientific experts), and the perspective on which the expertise is based (to differentiate between practitioners and technical or scientific experts). At the end of the survey, participants had the opportunity to give general comments.

In each of the three questionnaires, stakeholders were asked to first compare the importance of the three criteria as a whole (AHP level 2) followed by the comparison of each of the three ecological, social, and economic sub-criteria (AHP level 3, see Figure 1). To simplify the AHP for the mostly non-scientific stakeholders, each pairwise comparison between the sub-criteria started with the question which of the two options is more important in the opinion of the participant, or whether both are considered equally important. If the decision by the participant was that both options are equally important, the next pairwise comparison followed directly. If a stakeholder decided that one option is more important than the other, they were asked to rate how much more important it is on the remaining 8 points of the 9 points Saaty-Scale (Saaty 2008).

2.2.4. Data processing and analysis

The weights of the three criteria and by the experts selected nine sub-criteria were calculated with R software using the ahpsurvey package (Cho 2019). In a first step, the structure of the data exported from SoSci Survey had to be adapted for processing with the ahpsurvey package. In the next step, pairwise comparison matrices were created and individual preference weights calculated using the Dominant Eigenvalues method. This was followed by measuring the consistency of the individual judgements. Only consistent judgements with a consistency ratio (CR) less than 0.1 (Saaty 1987) were then used to compute the aggregated weight for each criterion and sub-criterion by calculating normalised geometric means. Finally, the aggregated sub-criteria weights were multiplied with the corresponding aggregated criteria weight to get the total weight of each sub-criterion (Saaty 1987).

The process of data adaptation and calculation of the individual and aggregated weights, the consistency measurement (Kasper and John 2023), as well as the survey and related anonymised data (John and Artmann 2023) are made available in a data repository.

2.3. Further statistical analysis

To discuss the results of both surveys, further statistical analyses were carried out to detect group-specific differences (e.g. gender, different length or degree of experience with urban agriculture, scientific focus, target group, expertise). These included U-tests and Kruskal-Wallis tests using IBM SPSS Statistics 25. Since the focus of the paper is on the AHP and the selection and weighting of suitable sub-criteria for assessing different types of urban agriculture, the results of the statistical analyses are reflected in the discussion section 4.2.

3. Results

3.1. Selected sub-criteria

In the first online survey, scientific experts selected from 33 sub-criteria (11 per criteria) which we have identified in a literature review the most important ones (see section 2.1.2). The first online survey resulted in 35 valid datasets (response rate: 29%).

The experts identified ‘local micro-climate and hydrology regulation’ (e.g. heat release, reduction or run-off mitigation by the urban food production site/installation), ‘species diversity’ (including wild and cultivated flora and fauna) and ‘circular economy’ (i.e. principle of closed-loop systems promoting resource savings and fostering a reduction of environmental pollution) as the most important sub-criteria for the environmental dimension (Table 1). As the most important social sub-criteria, ‘knowledge sharing and education’ (e.g. strengthening practical knowledge about food production, raising public awareness about connections between food production and the ecological crisis), ‘community building’ (e.g. bringing together different population groups, strengthening social justice in the form of equality and inclusion), and ‘civic participation in food production’ (e.g. strengthening social interactions between food producers and consumers) have been identified. ‘Securing food quality and safety’ (i.e. contribution to the local economy through the supply of high quality and healthy food) and its ‘affordability for all social strata’ (e.g. opportunity for consumers to negotiate food prices with the producer) as well as ‘strengthening local value chains/networks’ (e.g. by focusing on local direct marketing) are considered by the experts to be crucial for the economic sustainability dimension of urban agriculture.

Table 1. List of sub-criteria given in the first online survey ordered by descending means regarding their importance for evaluating the sustainability of different types of urban agriculture (the original order in the survey is indicated in the column os*, SD – standard deviation).

Sub-criteria	Further explanations given in the survey	os*	N	Min	Max	Mean	SD
Local microclimate and hydrology regulation by the urban food production site/installation	e.g., heat release/reduction or run-off mitigation by the urban food production site/installation	8	35	2	5	4.11	0.90
Species diversity	including wild and cultivated flora and fauna	2	34	2	5	4.09	0.87
Circular economy or cradle to cradle life cycles	i.e., principle of closed loop systems promoting resource savings (e.g. energy, water, fertilizer) and fostering a reduction of environmental pollution (e.g. air, water, soil)	6	34	1	5	4.03	0.97
Habitat diversity	-	1	35	2	5	3.91	1.04
Genetic diversity	for instance through the use of old/seldom varieties of food species	3	34	2	5	3.76	0.89
Biological control	e.g., of pests and diseases	4	33	2	5	3.76	1.03
Greenhouse gas emmission/removal and global warming potential of the urban food production site/installation	e.g., influence on carbon sequestration by plants and soil functions or soil and crop management practices	7	35	1	5	3.69	1.13
Space efficiency	i.e., the ratio between space demand for food production and degree of food supply	5	34	1	5	3.29	1.17
Focus on plant-based food production	i.e., reduction of animal-based food production such as cattle farming	11	33	1	5	3.18	1.13
Smell regulation	i.e., smell emission or reduction by the urban food production site/installation	10	35	1	5	2.91	0.89
Noise regulation	i.e., noise emission or reduction by the urban food production site/installation	9	34	1	5	2.88	1.01
Practical knowledge gain, knowledge sharing and education for urban residents/consumers	e.g., strengthening practical knowledge about food production, raising public awarness about connections between food production and the ecological crisis	3	35	3	5	4.51	0.61
Community building	e.g., bringing together different population groups (e.g. in terms of age, ethnicity, race, language, ability, socioeconomic status, gender); strengthening social justice in the form of equality and inclusion (e.g. employing people from marginalized communities)	1	35	3	5	4.40	0.69
Participation of residents/consumers in urban food production	e.g., strengthening social relationships/interactions between food producers and consumers	2	35	2	5	4.29	0.75
Opportunity for urban residents to gain nature experiences	e.g., sensory-physical exposure to nature, enjoyable and attaching experiences in nature	4	35	3	5	4.03	0.66
Recreation potential of the urban agricultural site/installation thereby contributing to public mental and physical health	e.g., passive recreation, physical exercises	7	35	1	5	4.00	1.00
Potential of the urban food production site/installation to strengthen urban resident´s place attachment	e.g., strengthening place identity, affective attachment, social bonding	10	35	1	5	3.94	1.00
Contribution to residents’ good environmental behaviour	e.g., strengthening sustainable food consumption (for instance in terms of an increase in the consumption of regional, less meat/plant-based and seasonal food or a decrease in the consumption of processed food and in food waste)	5	34	1	5	3.94	0.92
Transparency of production process for urban residents	e.g., information about where, how and for whom the food is produced	11	34	2	5	3.91	0.75
Public access to the urban food production site/installation for urban residents	-	6	35	1	5	3.71	1.05
Aesthetic appreciation of the urban food production site/installation	-	9	35	2	5	3.60	0.88
Opportunity to gain spiritual experiences at the urban food production site/installation	-	8	34	1	5	2.85	1.05
Food quality and safety	i.e., contribution to the local economy through the supply of high quality, healthy food	1	34	2	5	4.18	0.80
Establishing and use of local value chains/networks	e.g., by focusing on local direct marketing	9	35	1	5	4.00	0.94
Promotion of affordability of food for all social strata	e.g., opportunity for consumers to negotiate food prices with the producer	2	33	1	5	3.91	1.07
Development of innovations	e.g., strengthening of innovations in business models, organizational or technical innovations	11	34	1	5	3.79	0.98
Income and job generation	-	8	35	1	5	3.69	1.08
Marketability of the urban food production type and its products	e.g., acceptance of urban agriculture type by urban residents, development of a sustainable business model, market guarantee for producers	6	35	1	5	3.51	1.15
Food production according to customers´ demands in terms of quantity and quality	quantity: e.g. precise coordination of food quantity according to the demand to avoid food overproduction; quality: e.g. supply with desired species/varieties	4	34	2	5	3.44	0.96
Business focus on a diverse food product range	-	5	33	1	5	3.30	1.07
Food supply quantity	i.e., contribution to the local economy through urban food self-sufficiency	3	35	1	5	3.17	1.04
Efficient risk management to avoid or handle crop failures	e.g., by a controlled-environment agriculture or sharing the risk of yield variations between producers and consumers	7	34	1	5	3.09	1.19
Attractiveness of urban food production site/plant for tourists	-	10	35	1	5	2.91	1.17

Under consideration of the experts’ general comments and their further named sub-criteria, the sub-criterion ‘species diversity’ has been combined with genetic diversity” and titled as ‘biodiversity’ in the second survey. Furthermore, ‘community building’ was strengthened and focused more clearly on social justice and intercultural exchange, titled in the second survey as ‘community building and social justice’ (see section 5).

3.2. Sub-criteria weights

The second online survey resulted in 141 valid data sets in total, 55 from the target group urban administrations, 45 from non-governmental organisations, and 41 from practitioners or technical-scientific experts (response rates: 28%, 24%, 30%, 34%). On average, about two-thirds of the datasets were valid and could be used for the calculation of aggregated weights (Table 2).

Table 2. Number of valid datasets per target group.

		Number of datasets
Target groups	Weighting of	in total	valid
all	criteria	141	90
	environmental sub-criteria		93
	social sub-criteria		89
	economic sub-criteria		85
Urban administrations	criteria	55	31
	environmental sub-criteria		37
	social sub-criteria		34
	economic sub-criteria		28
Non-governmental organizations	criteria	45	30
	environmental sub-criteria		30
	social sub-criteria		29
	economic sub-criteria		30
Practitioners and technical-scientific experts	criteria	41	29
	environmental sub-criteria		26
	social sub-criteria		26
	economic sub-criteria		27

The environmental dimension got the highest weight (42%) followed by the social (34%) and lastly the economic dimension (24%) (Figure 2). Within the three dimensions, differences from 6 (environmental dimension) to 15% points (economic dimension) between the lowest and the highest sub-criteria weight were detected. The highest weights within each of the three dimensions received the sub-criteria biodiversity, community building and social justice, and local value chains/networks.

Figure 2. Weights of the three criteria and the corresponding sub-criteria.

The multiplication of the criteria weights with the corresponding sub-criteria weights results in the total weights. Biodiversity achieved the highest total weight and food quality and safety the lowest (Figure 3).

Figure 3. Total weights of the nine sub-criteria.

4. Discussion

The aim of the study was to develop an integrative framework for assessing the ecological, social, and economic sustainability of urban agriculture and its interlinked ecosystem services based on a multi-criteria decision analysis and an AHP by means of a participatory approach. In general, participatory sustainability assessments are gaining increasing momentum in an urban context, for instance in dependency on specific sustainability challenges such as sustainable water management (Brombal et al. 2018; Opher et al. 2019) or urban regeneration (Omidipoor et al. 2019; Korkmaz and Balaban 2020). A specific sustainability challenge was also in the focus of this study. Thus, this paper developed an assessment framework that can guide urban stakeholders in achieving sustainability goals that address challenges related to food production and its mitigation through the example of community-supported agriculture and vertical farming. A range of methodological challenges and ways forward have been detected in our study calling for discussions on i) interrelations between sub-criteria (Ameen et al. 2015; Kaur and Garg 2019) (see section 4.1), ii) sustainability as multi-dimensional approach (Komeily and Srinivasan 2015; Sharifi 2020) (see section 4.2), and iii) selection of representative stakeholders (Brombal et al. 2018; Opher et al. 2019) (see section 4.3).

4.1. Balancing sub-criteria for an assessment of sustainable urban agriculture

In the first round of weighting the importance of sub-criteria for an assessment of the sustainability of urban agriculture, we invited scientific experts in the field of urban agriculture from Europe. Based on a pre-defined list of suitable sub-criteria elaborated through a literature review, the survey had the aim of selecting the most important characteristics of urban agriculture for sustainable development. Based on these findings, future studies can assess various types of urban agriculture with the help of sustainability indicators. In general, assessing sustainability requires (sub-)criteria and indicators which are simultaneously specific and multiple enough to reduce any intersections between them and to increase the understanding in sustainability dimensions (Ameen et al. 2015). To give experts the opportunity to identify any missing or overlapping understandings in sustainable urban agriculture, they could complement up to three missing sub-criteria per sustainability dimension as well as any open comments at the end of the questionnaire. In fact, we found that some sub-criteria have been too specific or not specific enough. In the environmental dimension, this challenge became visible with the sub-criteria related to biodiversity, which we clustered with the aim of being as clear as possible in three various sub-criteria encompassing urban habitat, species, and genetic diversity (Botzat et al. 2016). However, since the results of the expert weighting showed that genetic diversity was rated the fifth most important sub-criteria (mean value: 3.8) and therefore almost as important as the third most important sub-criteria circular economy (mean value: 4.0), we merged it with species diversity rated as the second most important sub-criteria in this dimension. Thereby, we also took into account recommendations by experts to combine species and genetic diversity following also agriculture science, suggesting close interrelations between them (Chateil et al. 2013).

In contrast, the sub-criteria ‘community building’ and ‘participation of residents/consumers in urban food production’ from the social dimension have been identified through the first survey as too much overlapped with one another. In fact, literature also suggests close interlinkages since participation in urban agriculture, such as through community gardening, can foster social bonding and cohesion (Rogge et al. 2018; Delshad 2022). To make the difference between these two sub-criteria clearer, we emphasised in the follow-up survey the role of community building in terms of social justice whereby the sub-criterion on participation was especially focused more clearly on interactions between food producers and consumers. In doing so, we followed comments by the experts pointing out the value of community building through urban agriculture in order to bring various strata of population (e.g. different generations) together, which is in fact meaningful for fostering the acceleration of edible cities as an urban intervention for sustainable transformation (Sartison and Artmann 2020). All in all, the participatory two-step approach by first selecting suitable sub-criteria by scientific experts was useful to clarify any misunderstandings regarding its definitions and meanings before the weighting was done by further academic and non-academic stakeholders in the second survey. However, that the weighting of the sub-criteria is in general not an easy task for researchers was reflected by some experts in the first survey. For instance, we received questions such as why one category is more important than another to approach sustainability, or whether or not sustainable agriculture is per se an objective that is too broad to achieve. These comments address the general importance of reflecting on sustainability as a multidimensional approach, which we discuss in the following section.

4.2. Sustainable urban agriculture as a multi-dimensional approach

Although several stakeholders added at the free comment box at the end of the first and second surveys that they are wondering about the need to rate the importance of sustainability categories for urban agriculture since all dimensions and sub-criteria are equally important and are related to each other, the survey showed a clear picture that there is in fact a ranking of importance between the three sustainability dimensions and its embedded sub-criteria. The results of the second survey suggest that the stakeholders assess the environmental dimension as being more important than the social. The economic dimension of urban agriculture was evaluated as the least important one. From a general conceptual perspective, a stream of authors argues that the three-pillar approach of sustainability must be re-thought, taking into account that a prospering economy and healthy society is dependent on a healthy environment. This understanding of sustainability is framed as strong sustainability, which stands in contrast to weak sustainability, arguing that ‘natural capital’ can be substituted by ‘human capital’ (Ayres et al. 1998; Kuhlman and Farrington 2010). In fact, our results from the second survey on urban agriculture mirror the idea of strong sustainability, putting the environmental dimension in the focus and thereby constraining the social and economic dimension (see Figure 4). A Kruskal–Wallis test based on the individual weights revealed that the weights of the three sustainability dimensions differ significantly. The connected pairwise comparisons reflect the ordering of the dimensions shown in Figure 4. Similar to urban sustainability assessment research in general (Sharifi 2020), a literature review on urban agriculture suggests that research mostly addresses the environmental dimension as well dealing in particular with challenges related with biodiversity and ecosystem services (Artmann and Sartison 2018).

Figure 4. Results from the first survey suggesting strong sustainability of urban agriculture (source: own illustration adopted from Cato 2019).

Taking into account the immense adverse impacts of the current agricultural system on the environment worldwide (see section 1), it is meaningful to foster at first hand environmental sustainability. In fact, urban agriculture holds the potential to reduce the negative environmental impacts of food production from local to global scale (Langemeyer et al. 2021). According to the results of the second survey, stakeholders weighted especially the positive contribution of urban agriculture to biodiversity, circular economy, and local microclimate and hydrology regulation as being of high importance for environmental sustainability. For instance, the planning of rooftop gardens within urban green networks and their inclusion of wild flowers for pollination can foster biodiversity locally (Orsini et al. 2014). A study from the US suggests that community-supported agriculture can also have positive impacts on biodiversity, for instance, through crop diversification as a strategy by farmers for risk-hedging, which in turn has local co-benefits for soil health or water retention (Paul 2018). By analysing the impact of urban agriculture on the urban metabolism of water as well as energy or nutrients through a life-cycle-assessment that is linked with circular economy as the second most important sub-criteria of the environmental sustainability dimension in our survey, the potential to reduce negative impacts of local agriculture compared to global food imports can also be made visible (Langemeyer et al. 2021). In general, the proposed AHP in this paper does not take into account the issue of scale. We must, as an example, look at the multi-scale implications of urban agriculture which has the potential to decrease any dependencies on global food supply chains such as by vertical farming targeting at using limited urban land for food production as efficiently as possible (Al-Kodmany 2018). In this regard, conflicts between various types of land-use play a crucial role when assessing the sustainability of urban agriculture that also results in conflicts between the sustainability dimensions. For instance, due to the frequently high level of land rents in cities, non-agricultural activities (e.g. commercial or residential uses) provide higher economic outputs compared to agriculctural land uses, which could also create fewer jobs than more labour-intensive business sectors (Azunre et al. 2019). High economic pressures force, for instance, farmers of community-supported agriculture to use the land intensively to secure sufficient revenues per land parcel (Paul 2018). However, such intensive forms of agriculture in turn might hamper biodiversity such as plant species richness (Kleijn et al. 2009; Bagella et al. 2014).

Making any trade-offs or synergies visible among the different sustainable dimensions for urban agriculture is a core benefit from reflecting on urban agriculture from a multi-dimensional perspective. However, the comprehensiveness of the criteria proposed on level 2, which is based on the triple bottom-line approach of sustainability (Elkington 1998), also needs careful reflection. For instance, an expert from the first survey proposed adding ‘health and well-being’ as a separate dimension including sub-criteria such as physical activity or horti-therapy. Taking then into account the interrelatedness between multi-scale human and ecosystem health in terms of strong sustainability, a potential fourth sustainability dimension could reflect relations between urban agriculture and planetary health. In fact, the German Advisory Council on Global Change (WBGU 2021) acknowledges, as a key issue for planetary health, the need for future research and societal debates on healthy food systems that secure not only human but also animal health, biodiversity conservation, and climate protection. A major lever to foster planetary health through food systems is food production and consumption, which is in line with the ‘Planetary Health Diet‘ proposed by the EAT – Lancet Commission (Willett et al. 2019). A shift from a diet and food production system harming the health of people and nature requires ‘(…) substantial dietary shifts, including a greater than 50% reduction in global consumption of unhealthy foods, such as red meat and sugar, and a greater than 100% increase in consumption of healthy foods, such as nuts, fruits, vegetables, and legumes’ (Willett et al. 2019, p. 448). In fact, urban agriculture can then play a crucial role in fostering a planetary health diet. From a spatial perspective, taking into account limited space as a major constraint for urban agriculture (Artmann and Sartison 2018) and that food supply for plant-based diets demands less cropland compared to livestock farming (Erb et al. 2016), cities can benefit from space-efficient plant-based food production. From a social perspective, the engagement of urban residents in urban food production, such as community-supported agriculture or urban gardening, can then foster sustainable food-behaviour (Artmann et al. 2021). In this regard, future research referring to our approach should check if vertical farming as a spatial form of cultivation and community-supported agriculture as a form of social organisation could be merged into an innovative form of community-based vertical farming. In fact, in Dresden (Germany), the innovative start-up Wolkenfarm aims to foster such a hybrid form of urban agriculture that brings together local residents from a neighbourhood to cultivate vertical farms together such as on roofs of schools or kindergartens. This could increase the amount of food produced per land parcel while engaging urban residents in food production that would foster behavioural changes towards improved planetary health diets. Interestingly, the weighting of the sub-criteria by academic experts in the first survey showed that the focus of urban agriculture on plant-based food production was assessed as less important for environmental sustainability; therefore, this sub-criteria has been skipped in the second survey. Taking into account that debates about diet, such as vegetarian (e.g. addressing ethical values such as animal suffering) vs. omnivorous (e.g. as part of a ‘traditional’ regional diet), are informing decision-making through subjective emotional tensions and cultural backgrounds (Jiménez-Aleixandre and Brocos 2021), the next section will reflect on the stakeholders involved in our study.

4.3. Stakeholders assessing sustainable urban agriculture

In general, a participatory assessment of urban sustainability is an important approach to reflection on and co-creation of various meanings, insights, priorities, and perspectives among different stakeholders such as researchers and professionals (Brombal et al. 2018; Kaur and Garg 2019). As multi-criteria decision-making always involves a certain dose of subjectivity, despite the formal process of (sub-)criteria weighting, the selection of stakeholders must be done in a representative and transparent manner (Opher et al. 2019). Taking into account the fact that the weighting of the final selected sub-criteria was done by various stakeholders in the second survey, the selection of stakeholders required a comprehensive approach. Therefore, we applied transparent selection criteria and representative case study cities integrating relevant urban agriculture stakeholders including urban administrations as well as non-governmental organisations, practitioners, and technical-scientific experts (see section 2.2.2). To detect any variables eventually influencing the weighting results (e.g. gender, different length or degree of experience with urban agriculture, expertise), we carried out statistical analyses (U-tests and Kruskal-Wallis tests) (see results in the dataset published by John and Artmann 2023).

Interestingly, no significant differences in weights between the three different stakeholder groups were detected. This could be due to the fact that the assessment was not linked to a specific urban agriculture project embedded spatially in a defined case study. Thus, other studies on AHP found that significant differences between weights among stakeholders have been found between local stakeholders and outside experts, whereby local stakeholders weight specific criteria higher than outside experts who lack specific local knowledge (Strager and Rosenberger 2006; Chow and Sadler 2010). Another kind of explanation for the lack of differences might be traced back to the consensus among the survey participants that society is facing a crucial ecological crisis to which the current food system is significantly contributing. Furthermore, the fact that the environmental dimension was particularly highly rated can be explained by the realisation that most of the participants from city administrations represented environmental departments. Representatives of other departments such as urban planning and development, healthcare, or social affairs did use the opportunity to participate in the second survey to a considerably lower extent. To get a broader view of differing perspectives, future research can include further stakeholder groups and other countries into the survey. For instance, future studies could integrate urban residents in China, where participatory processes in urban planning are still at the beginning stages (Brombal et al. 2018); subsequently, these urban residents could possibly assess social sustainability of urban agriculture as being more important than stakeholders in this survey.

Furthermore, there were almost no significant gender-specific differences in weighting the sustainability dimensions and the related sub-criteria. Regarding the three sustainability dimensions, participating women gave a significantly higher weight to the social dimension than men, whereas men weighted the economic dimension significantly higher than women. For the nine sub-criteria, only ‘circular economy’ revealed significant gender-specific differences. Men weighted this sub-criterion significantly higher than women and participants who gave no answer regarding their gender. A close link between social values and female urban agriculture participants can also be found in studies on community gardens in the Global North. For instance, in Germany and the Czech Republic more females than males are engaged in community gardens in cities (Winkler et al. 2019; Dubová et al. 2020), whereby males are more interested in technology-oriented types of urban agriculture (e.g. vertical gardens) compared to females (Winkler et al. 2019) and women are more interested in passing on their knowledge to their children than men (Dubová et al. 2020). Beyond the urban scale, further studies on sustainable agriculture suggest that women assign higher importance of social values than men regarding traditional care work such as for the family, community, future generations, and nature (Chiappe and Flora 1998; Shisler and Sbicca 2019). In fact, women engage more often in sustainable and organic farming, calling for a gender-sensitive and relational approach (Leslie et al. 2019). Such an approach is also needed for the urban context, which would take into account not only the deliberation of social values and knowledge among various participants in urban agriculture (Winkler et al. 2019) but also reflection on often hidden gender-based economic discrimination constraining women farmers' access to land, machinery, and capital (Leslie et al. 2019). To what degree these injustices influence sustainable agriculture in an urban context and related different types of urban agriculture and its geographical embedding needs further research.

5. Conclusions and ways forward

Approaching sustainable cities is a common task for researchers, urban policy and planning, and civic society. (Re-)Integrating sustainable agriculture in cities can be one piece of the puzzle in this regard. However, a structured approach is needed that could guide decision-makers in their assessments of the sustainability of various types of urban agriculture while taking into account the unique spatial, social-economic, and environmental framework conditions of the respective city (cf. Azunre et al. 2019). To address this complexity, this paper provides scientific fundamentals by developing an integrative assessment framework for a systematic comparative analysis and evaluation of different types of urban agriculture based on a participatory AHP and multi-criteria analyses. Based on a German case study and on the examples of vertical farming and community-supported agriculture, the results suggest that sustainable urban agriculture is a multi-dimensional approach informed by strong sustainability that places nature in the focus. The list of identified sub-criteria cannot only be applied to community-supported agriculture or vertical farming. Thus, strong sustainability and its targets such as to foster micro-climate regulation and protect biodiversity must be the major target of all forms of food production in cities and beyond taking into account that global agriculture puts intensifying pressures on planetary boundaries (Conijn et al. 2018; Gerten et al. 2020).

To assess sustainability strengths and weaknesses of various types of urban agriculture, this paper establishes the basis for interlinking the detected sub-criteria and weights with specific indicators, which could be developed through future research. Based on our approach, the indicator development and selection could be guided by shedding light on their suitability (e.g. the detected criteria and sub-criteria guiding sustainable urban agriculture (cf. Bizikova et al. 2019) and feasibility (e.g. whether they are easily implementable for vertical farming and community-supported agriculture (cf. Bélanger et al. 2012)). Such research should then also consider that fostering sustainable food production and consumption does not end at the border of cities but needs to take into account global food supply chains and their interrelated adverse social and ecological externalities. Future assessments of urban agriculture could then reflect these ecological externalities within the context of planetary health diet and justice.

Consent to participate

Informed consent was obtained from all individual participants included in the study.

Consent for publication

All authors mentioned in the manuscript have agreed for authorship, read and approved the manuscript, and given consent for submission and subsequent publication of the manuscript.

Acknowledgments

The authors thank Mabel Killinger and Marie Herzig for their help in stakeholder identification as well as all experts and stakeholders for their participation in the two online surveys and their helpful comments. Data processing and analysis by means of an Analytic Hierarchy Process in R would not have been possible without the help of Björn Kasper.

Disclosure statement

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Funding

This project is funded by the German Research Foundation (Deutsche Forschungsgemeinschaft—DFG) (grant number: AR 1121/1-2).

Notes on contributors

Henriette John

Henriette John is a post-doctoral researcher at the Leibniz Institute of Ecological Urban and Regional Development (IOER), Dresden (Germany. Her research interests include landscape ecology, nature conservation, biodiversity, biotope network, mapping and monitoring of species and habitats and sustainable management of habitats of open landscapes.

Martina Artmann

Martina Artmann is a professor of green infrastructure at the Department of Landscape Architecture, Weihenstephan-Triesdorf University of Applied Sciences, Freising (Germany) and leads the Research Group Urban Human-Nature Resonance at the Leibniz Institute of Ecological Urban and Regional Development (IOER), Dresden (Germany). Her research interests include urban human-nature relationships for sustainability transformations, edible cities, urban agriculture and sustainable human-food relationships.