Co-designing a research programme for impact: lessons learned from practice by Aotearoa New Zealand’s Biological Heritage National Science Challenge Ngā Koiora Tuku Iho

ABSTRACT

Doing co-design and co-production is challenging, resource intensive, and outcomes do not always translate into action. Evaluations of processes are needed to identify what enables and constrains ‘co’ efforts. This paper draws on the findings of an evaluation of a co-design process undertaken by Aotearoa New Zealand’s Biological Heritage National Science Challenge, Ngā Koiora Tuku Iho (BHNSC) in 2019. The independent evaluation, commissioned by the BHNSC, draws on process observations and 25 semi-structured interviews with BHNSC leaders and process participants. In this paper, we present key insights from the evaluation through the application of co-production quality assessment principles and a knowledge governance conceptual framework. Our analysis identifies the BHNSC’s values as a critical factor in its journey to conduct a process that would foster collaboration between mātauranga Māori and Western science knowledge systems and deliver impact-focused biodiversity and biosecurity research. We propose an additional principle for assessing the quality of co-production processes: values-inspired.

KEYWORDS:

• Co-production and co-design
• values-inspired knowledge practices
• knowledge governance
• mātauranga Māori
• biodiversity and biosecurity

Introduction

Co-design and co-production are challenging to execute and resource intensive, with outcomes not always translating into action and change (Mason and Boutilier 1996; Oliver et al. 2019; Wyborn et al. 2019; Mark and Hagen 2020; Turnhout et al. 2020). Hence, it is increasingly recognised that evaluations of ‘co’ processes are needed to understand what can enable and constrain processes and their outputs (Fazey et al. 2014; Polk 2015; Van Kerkhoff and Pilbeam 2017; O’Connor et al. 2019; Oliver et al. 2019; Wyborn et al. 2019; Mark and Hagen 2020; Norström et al. 2020; Maas et al. 2022; Plummer et al. 2022). This paper draws on an independent evaluation of a co-design process conducted during 2019 by New Zealand’s Biological Heritage National Science Challenge, Ngā Koiora Tuku Iho (BHNSC). The BHNSC process was established to co-design a set of goals and impact pathways for a nationwide strategy of research and actions in the biodiversity and biosecurity sectors with researchers, Māori partners and stakeholders.

The BHNSC is one of 11 mission-led National Science Challenges (NSCs) created by Aotearoa New Zealand’s Ministry of Business, Innovation and Employment (MBIE) to ‘tackle the biggest science-based issues and opportunities facing New Zealand’ (MBIE Undated). The NSCs have been funded for 10 years in two 5-year tranches from 2014. BHNSC funding is NZ$63.4 million over 10 years. The BHNSC’s mission is to ‘reverse the decline of New Zealand’s biological heritage, through a national partnership to deliver a step change in research innovation, globally leading technologies, and community and sector action’ (BHNSC 2023). Its objectives are to protect and manage biodiversity, improve biosecurity, and enhance resilience to global threats and pressures. Funding for tranche 2 was confirmed by MBIE (2018) in its 2018 mid-way review. Aotearoa New Zealand’s largest export earners are primary production and tourism, both of which have impacts and risks for biodiversity and biosecurity. Hence, the social, cultural, and economic prosperity of all New Zealanders is linked to the nation’s biological heritage and the safeguards of biosecurity.

In Aotearoa New Zealand (ANZ), as is the case in other colonised nations, collaborative research practices have to traverse different ontologies (i.e. what we know), epistemologies (i.e. how we know), and values of both Western and indigenous knowledge systems (Nadasdy 1999, 2021). A ‘weaving’ together of knowledge systems requires ‘effective engagement of actors, institutions and knowledge-sharing processes’ (Tengö et al. 2017, p. 17 citing Johnson et al. 2015; Jones et al. 2020). Maintaining the integrity of both knowledge systems, as both have something important to offer each other in addressing issues of concern (Duncan 2016; Macfarlane and Macfarlane 2019; Nadasdy 2021), can be a significant and difficult task to navigate in practice (Duncan 2013; Duncan 2016; Tengö et al. 2017; Robson-Williams et al. 2018; Howarth et al. 2022; Tengö and Andersson 2022). Hence, evaluations of ‘co’ processes can help improve theory and practice (Wyborn et al. 2019).

The BHNSC understood the gravity of the above knowledge issues and had been applauded by MBIE’s mid-way review for how effectively it had worked with Māori partners in tranche 1 (MBIE 2018). For example, the review states: ‘[t]he overall approach to Mātauranga Māori, the level of engagement with Māori in governance, in appropriate science activity and in delivery of meaningful outcomes to accommodate the Māori viewpoint of Biological Heritage is truly impressive’ (MBIE 2018, p. 10–11). Hikuroa (2017, p. 5) describes mātauranga Māori as spanning ‘Māori knowledge, culture, values and worldview’.

In tranche 2, the BHNSC leaders wanted to intensify efforts to deliver tangible research impact (BHNSC 2018; Duncan 2020). To do so, a co-design process was established to chart a course for future impact-focused biodiversity and biosecurity research. An independent evaluation of the process was conducted in 2019–2020 to foster reflection and learning within and beyond the BHNSC. The evaluation presents insights from process participants on what worked well and what did not work so well, and identifies several foundations for co-design success (Duncan 2020), namely:

• leadership commitment

• financial resources

• a realistic timeframe

• organisational capacity

• diverse, knowledgeable and experienced participants across researcher, Māori and stakeholder/end user groups

• clear values, rules of engagement and output expectations

• power sharing

• skilled facilitation

• a well-designed process.

Of course, not having these foundations in sufficient measure could lead to failures in co-design.

In this paper, we draw on insights from the BHNSC evaluation and seek to contribute to co-design and co-production theory and practice. To do so, we first use principles for assessing the quality of co-production processes (i.e. context-based, pluralistic, goals-oriented and interactive) (Norström et al. 2020). We then draw on the knowledge governance conceptual framework in Van Kerkhoff and Pilbeam (2017) that conceives knowledge governed at three levels: civic epistemology, knowledge systems, and interventions (see also Wyborn et al. 2019). We argue that a knowledge governance perspective deepens our understanding of what contributes to the success of co-design and co-production, and where barriers to implementation are likely to reside. Using this analytical framework, we examine the approach and actions taken by the BHNSC on its journey to conduct a co-design process that would foster collaboration between mātauranga Māori and Western science. We identify the BHNSC’s values as a critical factor in its navigation of ANZ’s knowledge governance system and propose an additional principle for assessing the quality of co-design and co-production processes: values-inspired.

Analytical framework

The ‘co’ concepts

A lack of precision in defining ‘co’ concepts such as co-production, co-design and co-creation has been a concern for social science researchers for some time, and remains the case (Brandsen and Honingh 2018; Robert et al. 2022; Vargas et al. 2022; Wyborn et al. 2019). For example, Mark and Hagen (2020, p. 5), in their ANZ snapshot of the co-design literature, explain that the term is ‘used interchangeably with a set of other terms: participatory design, experience-based design, co-production, human-centred design and others’. In the healthcare literature, Vargas et al. (2022, p. 5) conceive co-design and co-production as a subset of co-creation:

Co-creation is considered an overarching construct which is defined as the active involvement of stakeholders, from the exploration and articulation of problems or needs to the creation, implementation and evaluation of solutions or initiatives. In this vision, co-design relates to the design of an initiative that positions participants’ needs, expertise and knowledge at its centre. Co-production assists in the collaborative delivery and production of knowledge.

In this formulation, co-design is a necessary first step of co-production.

Brandsen and Honingh (2018, p. 24) note that co-creation is the more recent and ‘slippery’ term in public administration and that co-production has ‘a longer tradition’, specifically in the work of Elinor Ostrom (1996). Ostrom maintained that co-production with citizens was needed to ensure services and products that governments hope to deliver are fit-for-purpose for users. Co-design, on the other hand, has its origin in the 1970s Scandinavian ‘participatory design movement’ (Robert et al. 2022, p. 3) which has fostered a range of design-thinking principles for innovation: people-centred, visual and inclusive communication, collaboration and co-creation, and iteration, as well as conceptual tools such as the ‘Double Diamond’ which encourages iterative divergent (i.e. wide) and convergent (i.e. focused) thinking (Design Council UK 2023). These design-thinking principles and tools were used by the BHNSC in its co-design process.

To this point, we have talked about co-design and co-production as purposefully designed collaborative processes or encounters that aim to produce shared knowledge. In contrast, co-production is also a foundational concept in the field of science and technology studies (STS) where it is used to critique public policy and institutional settings (Jasanoff 2004; Wyborn et al. 2019; Miller and Wyborn 2020). Van Kerkhoff and Pilbeam (2017, p. 32) argue that the former is focused on instrumental/utilitarian goals while the latter is focused on critical/reflexive goals. These seemingly disparate modes of co-production are brought together by Van Kerkhoff and Pilbeam (2017) in a multi-layered knowledge governance conceptual framework (discussed below) that recognises instrumental/utilitarian processes sit within particular institutional settings and epistemic practices that need to be included in the analysis of co-design and co-production processes.

According to Jasanoff (2004, p. 278), co-production as critique evaluates ‘the social arrangements that prop up particular natural orders or in reverse, the epistemologies that help to sustain particular social orders’ (Jasanoff 2004, p. 278; Miller 2004; see Nastar 2023 for a critical realist approach). In other words, our knowledge of nature and society are mutually constituted – one influences and is influenced by the other. Examining how knowledge practices order nature (and vice versa) can illuminate power relations and the direction in which knowledge and policy are heading (Jasanoff 2004; see also Latour 1993; Castree and Braun 2001; Miller 2004). On this basis, co-production is not an isolated event – it is occurring all the time. Importantly, instrumental/utilitarian processes often seek to push science and policy in particular directions, for example, towards more sustainable outcomes (Van Kerkhoff and Pilbeam 2017; Tengö and Andersson 2022), which is what the BHNSC was seeking to do.

We conceive co-production as an encompassing term that captures changes over the past three decades in how knowledge is governed, produced, used and challenged (Gibbons et al. 1994; Nowotny et al. 2001). This includes the array of collaborative approaches that seek to bring people together to address issues of concern (e.g. Cash et al. 2003; Roux et al. 2004; Cash et al. 2006; Brandt et al. 2013; Polk 2015; Turner et al. 2016; Thompson et al. 2017; Srinivasan and Elley 2018; Chambers et al. 2021), as well as the challenges these approaches present for participants, knowledge brokers, governments and researchers (Mason and Boutilier 1996; Filipe et al. 2017; Oliver et al. 2019; Wyborn et al. 2019; Mark and Hagen 2020; Turnhout et al. 2020; Duncan et al. 2020a). Following Brandsen and Honingh (2018), Robert et al. (2022) and Vargas et al. (2022), we conceive co-design as a precursor to co-production. For example, co-designing what is to be researched, and how, is an important first step of knowledge co-production and what flows from it, which is where the BHNSC started its journey in tranche 2.

Principles for assessing the quality of co-production processes

There are many frameworks for evaluating co-production processes (e.g. Cash et al. 2006; Wickson et al. 2006; Lang et al. 2012; Mitchell et al. 2015; Hansson and Polk 2018; O’Connor et al. 2019; Louder et al. 2021; Plummer et al. 2022). It has been recognised, however, that caution is needed. A ‘catch-all’ approach to evaluation is not desirable as knowledge and modes of its exchange are conceived differently by different fields and require different methods (Fazey et al. 2014, p. 204). Hence, Fazey et al. (2014, p. 217) recommends the use of evaluation principles as they provide the analyst with a ‘broad guide rather than a comprehensive prescription’. Given the contributions to co-production theory and practice of its many authors, we have identified the co-production quality assessment principles in Norström et al. (2020) to be useful for reflecting on the findings of the BHNSC evaluation.

For sustainability research, Norström et al. (2020, p. 183) define co-production as: ‘[i]terative and collaborative processes involving diverse types of expertise, knowledge and actors to produce context-specific knowledge and pathways towards a sustainable future’. This definition extends beyond knowledge production to ‘develop capacity, build networks, foster social capital, and implement actions that contribute to sustainability’ (Norström et al. 2020, p. 183). These were important aims of the BHNSC.

Norström et al. (2020) identify four principles for assessing the quality of co-production processes: context-based, pluralistic, goals-oriented, and interactive (the principles framework) (Table 1).

Table 1. Principles for assessing the quality of co-production processes in Norström et al. (2020, p. 184) (left) alongside a summary of their recommendations on what to look for when assessing a co-production process presented in Norström et al. (2020, p. 186–188) (right).

Principles	What to look for …
1. Context-based Situate the process in a particular context, place or issue	Are context and issues of concern sufficiently understood? Does the process address issues and work with existing priorities, activities and relationships? Does the process enhance existing skills and relationships?
2. Pluralistic Explicitly recognise the multiple ways of knowing and doing	Does the process include a diversity of participants across multiple dimensions and recognise multiple ways of knowing and doing? Does the process help participants build trust and develop shared perspectives and understandings? Does the process involve people with a range of skills and expertise? Does the process allow participants to contribute different perspectives and understandings? How does the process address power imbalances?
3. Goals-oriented Articulate clearly defined, shared and meaningful goals that are related to the challenge at hand	How does the process go about designing goals, outcomes, impacts? Does the process recognise tangible and intangible outcomes and impacts? How well does the process build new understandings, relationships and networks? Is there a feedback loop within the process to fine tune goals? Have milestones, pathways and measures of success to achieve goals been developed?
4. Interactive Allow for ongoing learning among actors, active engagement and frequent interactions	Did the process create time for people to interact meaningfully? Did learning occur? Do participants believe they had equitable opportunities to contribute? Were perceptions changed through process? Did a shared perspective on the problem emerge?

These principles are a useful guide for assessing purposefully designed (i.e. instrumental/utilitarian) co-design and co-production processes. However, they do not focus sufficient critical attention on how knowledge is governed at different institutional levels. For this purpose, we use the following knowledge governance conceptual framework.

Knowledge governance: a multi-layered perspective

Van Kerkhoff and Pilbeam (2017; p. 30 citing Van Kerkhoff 2014) define knowledge governance as ‘the suite of formal and informal rules and conventions that shape the ways we conduct or engage in knowledge processes, such as creating new knowledge, sharing or protecting knowledge, accessing it and applying or using it’. Their knowledge governance conceptual framework has three layers: civic epistemology, knowledge systems, and interventions (Figure 1).

Figure 1. A three-layer conceptual framework of knowledge governance: civic epistemology, knowledge systems, and interventions. Reprinted from Van Kerkhoff and Pilbeam 2017, p. 32. Copyright (2017) with permission from Elsevier.

Figure 1 locates the BHNSC at the knowledge systems level (middle layer) and its co-design process as an intervention (innermost layer). From here, we will refer to the BHNSC process as the ‘intervention’.

The outermost layer is civic epistemology, which is an important concept on which to elaborate. Jasanoff (2005, p. 255) defines civic epistemology as ‘the institutionalised practices by which members of a given society test and deploy knowledge claims used as a basis for making collective choices’. Comparing civic epistemologies across the United States, Britain, and Germany, Jasanoff (2005, p. 260) identifies ‘deep-seated patterns … to which ruling institutions and actors continually return, in part because they are held in place through time-honored legal, political, and bureaucratic practices’.

These patterns, and the rules and practices that constitute them, relate not only to the production of knowledge in public decision-making, but also to how knowledge is contested, validated and mobilised. The patterns and practices of a civic epistemology are essential for demonstrating transparency and accountability in the choices that are made by governing bodies. Hence, they are central to how democratic societies are governed and operate. A civic epistemology underpins, for example, parliamentary hearings, judicial reviews, and assessment processes. To make decisions in these institutional (and often politicised) settings, particular epistemic rules and practices are used to present, test, and validate knowledge claims. For example, actors with particular forms of expertise (e.g. legal, scientific) rely on particular sources of evidence (i.e. usually scientific) and registers of objectivity (e.g. case law, numbers, predictive models) (Jasanoff 2004, 2005).

Science, and its particular methodologies and methods, dominate the civic epistemology of Western democracies and is foundational to how Western democratic societies know (and what they know). Importantly, in democratic societies, knowledge claims (science-based or otherwise) are subject to scrutiny and contestation in a range of institutional, social, cultural, and political settings (Jasanoff 2004, 2005). In this milieu, styles of public decision-making and epistemic practices that foster trust and demonstrate legitimacy, accountability and expertise are elements of a nation’s civic epistemology. This means that co-design and co-production interventions ultimately have to contend with these ‘deep-seated patterns’ that are difficult (although not impossible) to shift or modify (Jasanoff 2005, p. 260; Tengö et al. 2017; Wyborn et al. 2019).

It is for this reason an intervention might be deemed successful at one level (e.g. a collaboratively designed research agenda) but can struggle to deliver expected outcomes due to barriers at other levels of a knowledge governance system (Van Kerkhoff and Pilbeam 2017; Wyborn et al. 2019). For example, in examining the role of brokers in a co-production process in ANZ, Duncan et al. (2020a) used the Van Kerkhoff and Pilbeam (2017) knowledge governance conceptual framework to differentiate a co-production intervention from its institutional and policy settings. It was shown how factors outside the control of knowledge brokers shaped how they operated and their negotiations of the knowledge they were brokering. In this paper, we use this framework to examine the different levels of knowledge governance and how they were navigated by the BHNSC on its journey to create an impact-focused co-design process for biodiversity and biosecurity research.

Methods

The evaluation of the co-design process was commissioned by the BHNSC and undertaken independently by the first author during 2019 and 2020. The evaluation draws on BHNSC documents, intervention observations, and 25 semi-structured interviews with BHNSC leaders and intervention participants. It presents an inductive thematic analysis of participants’ responses across three cohorts (i.e. Māori partners, stakeholders, and researchers) to questions about their experiences of the intervention. It should be noted that in ANZ, Māori are recognised as partners with the Crown not stakeholders.

To differentiate those who participated in the intervention from those who contributed to the evaluation, we refer to the latter as ‘evaluation participants’ across the three cohorts of Māori partners, stakeholders, and researchers. Further details of the evaluation and its findings, including qualitative research methods, can be found in Duncan (2020).

Results

Applying the principles framework

A full description of the BHNSC intervention can be found in Duncan (2020). In the following sections, we use the principles framework (Norström et al. 2020, Figure 1) to present key aspects of the intervention and discuss where it was successful, according to evaluation participants.

Context-based: situate the process in a particular context, place or issue

The BHNSC was well linked to the biodiversity and biosecurity sectors in ANZ and internationally through its researchers and leadership team, and partnerships it had fostered and created across research, Māori partner and stakeholder institutions and organisations during the first five years of the NSC (i.e. tranche 1) (MBIE 2018).

Drawing on experience and what was learned from tranche 1, the BHNSC Strategy 2019–2024 (BHNSC 2018) articulates an overarching framework to direct research investments across a number of impacts and strategic outcomes that had been identified as essential to its mission. The Strategy was developed with Challenge Partners (see BHNSC 2023), Māori partners and stakeholders. The MBIE mid-way review notes that the BHNSC had ‘demonstrated impressive science progress in Tranche 1’ and that tranche 2 had been ‘refined to reflect the wider critical issues’ (MBIE 2018, p. 7).

Given its deep understanding of the context and issues of concern, the BHNSC convened nine scoping groups of between 8 and 10 people for the tranche 2 co-design intervention. These groups included Māori partners, researchers, stakeholders, and end users from a range of industry organisations, levels of government, and beyond. Each group was assigned a strategic outcome and tasked with working together to co-design goals (including research priorities) and pathways to impacts to culminate in an investment prospectus (discussed below) that would be used to identify research teams and needed research projects, and to encourage co-investment to deliver ‘real world’ impact (BHNSC 2018).

Pluralistic: explicitly recognise the multiple ways of knowing and doing

Evaluation participants across the three cohorts were very positive about how the intervention had brought together such a broad range of people, knowledges, talent, and experience within and across the scoping groups. For example:

I thought it was fantastic. At least our group, I thought was a really good mix from early career researchers to old grey heads … The cultural mix as well, the mix of science and end users. I thought if you were to encapsulate a cross section of the research world in just six or eight people, I thought the [BHNSC] did really well, at least in our group. (Researcher evaluation participant)

People with diverse thinking and often in competition with each other over funding were brought together to think about one thing and build a consensus around that and then work out what to do about it. (Māori evaluation participant)

It kept breaking up into groups and brainstorming stuff and then coming back together and that was quite productive. It allowed different viewpoints to come forward and then be challenged, which then sparked fresh ideas. That expansion, contraction, expansion, contraction [worked well]. (Stakeholder evaluation participant)

These insights indicate that evaluation participants were not only hearing a diversity of perspectives from people with a range of skills and expertise; they were also experiencing an intervention that recognised multiple ways of knowing and doing which fostered shared understandings and respect for others.

The amount of time dedicated to the intervention allowed participants to build understanding of each other’s ways of knowing and doing. Scoping group members were asked to dedicate approximately 5–8 days to the process, which included two, 2-day, face-to-face meetings, and possibly one other face-to-face meeting. Scoping group members were paid for their time. Payment for participation in meetings and workshops is uncommon in ANZ and opened-up the process to many people who would not otherwise have been able to participate and contribute.

Evaluation participants explained the intervention had fostered respectful exchanges and they felt it had provided a safe place for different voices, views, and ideas to be put forward. Māori evaluation participants felt that the process had addressed power imbalances by levelling out hierarchies that usually prevent frank conversations between them and government stakeholders and researchers. They maintained that the way the intervention was designed had given them a voice and they were being heard.

To give an indication of the multiple perspectives that contributed to the intervention, the BHNSC calculated that scoping group participants were:

• 45% female and 55% male

• 24% self-identified as having Māori whakapapa (i.e. having Māori ancestry)

• 16% self-identified as being early career

• 63% from a research organisation

• 16% from a government organisation

• 8% from industry

• 3% from an NGO

• 9% independent.

In all, 87 people from 34 different organisations participated in the intervention across the nine scoping groups. Intervention participants were recruited through an expression of interest process (discussed below).

Goal-oriented: articulate clearly defined, shared and meaningful goals that are related to the challenge at hand

The intervention was developed to transform the BHNSC’s impacts and strategic outcomes into collaboratively designed goals and research priorities (BHNSC 2023). From the goals and impact pathways, investment prospectuses were subsequently developed by the scoping groups. BHNSC guidelines called for the investment prospectus to articulate a compelling narrative about what was needed and what was going to be done in terms of research, collaboration, and knowledge translation to encourage co-investment and build or extend relationships and networks. The entire intervention was built around the goals that were developed by the scoping groups by the end of their first workshop.

Before finalising the goals in a second workshop, scoping group members from the nine groups spoke to around 250 people from 130 organisations to gather feedback on their group’s goals. This feedback process was recognised by evaluation participants as an invaluable step for fine tuning the goals. They said the feedback process was also useful for making new contacts, seeing the big picture, understanding where research and implementation gaps were, and who was doing what across researcher, stakeholder, Māori, and end-user groups. The process also helped scoping groups identify who might be interested in working on implementing the goals and opportunities for co-investment down the track. This feedback step is an important aspect of design-thinking (Sheppard et al. 2018), which the BHNSC used to develop the activities of the intervention.

Evaluation participants were positive about the design-thinking approach and the facilitator, who was identified as central to the positive experience of the intervention. Participants explained that design-thinking had helped teams identify what was important, avoided pre-conceived ideas, allowed assumptions to be challenged through respectful conversations, generated good ideas, encouraged integrative thinking, and subdued personal agendas and biases by keeping everyone focused on the goals and impact pathways. Participants identified the problem-focus of the design-led activities and outward-thinking towards the needs of stakeholders and Māori as reasons why it was so effective. For example:

I liked the facilitation. I thought that overall [the facilitator] did a good job of making sure that different people’s voices came out in different ways. I was impressed by some of the [design-thinking] techniques, like the empathy mapping. I can’t even really put my finger on it but just the way, particularly in that first workshop, where you went from feeling like there was just so much unstructured complexity and somehow miraculously converging on something. (Stakeholder evaluation participant)

Interactive: allow for ongoing learning among actors, active engagement, and frequent interactions

Evaluation participants were impressed by the 2-day format of the two workshops, which allowed time and flexibility for considered dialogue and meaningful conversations rather than being focused on meeting agendas and feeling rushed. Māori evaluation participants also maintained that the process broke down barriers between institutions, science disciplines, and stakeholders. Interaction also took place outside the workshops as the scoping groups worked through the various tasks required by the BHNSC, e.g. seeking feedback on goals, developing an investment prospectus and pitching goals to a mock investment panel (discussed below).

Several evaluation participants talked about how they were changed by the process, for example:

just the incorporation of mātauranga Māori and where we got to and other groups as well. It was just how we worked. It wasn’t a separate objective, it was just how we operated. That’s a big cultural shift we’re going through, I think, in New Zealand. It’s great that it’s seeping into all aspects of what we do. Personally, it challenged me to do better, learn more about that. I’ve enrolled already to do a Te Reo [Māori language] course next year. That’s driven from that stuff. I was embarrassed. I didn’t really know anything about it and I actually said on the first day, ‘All this stuff, guys, it’s just fluff, how you get to your outcome because that’s really what we’re trying to do. There’s no time to waste. We’ve just got to get there as fast and as hard as we can.’ Actually, how we get there is probably more important in some cases. It definitely changed my thinking. I was wrong. (Stakeholder evaluation participant)

The stakeholder comments above, and insights from Māori evaluation participants set out earlier, illustrate how the intervention had created a safe place for close encounters and important conversations. According to evaluation participants, the workshops allowed scoping group members to learn and gain a deeper understanding and appreciation of different perspectives, in particular, mātauranga Māori, issues of concern for Māori and how Māori encounter the world.

Overall, the principles framework highlights that the BHNSC’s intervention succeeded where it was important to do so. However, the evaluation also revealed tensions.

Intervention tensions

Overall, the evaluation concluded that Māori, stakeholder, and researcher evaluation participants experienced the intervention quite differently (Duncan 2020). Māori evaluation participants maintained they felt empowered by the process as it had given them a rare opportunity to put forward their perspectives on an equal footing with researchers, stakeholders, and other end-users. From evaluation observations, Māori partners challenged others to think about the implications of past inequities, Treaty obligations of Te Tiriti o Waitangi/the Treaty of Waitangi (te Tiriti) and past research practices from a Māori perspective. They also offered different and enriching perspectives, as well as ideas on how research could be done differently.

Stakeholder evaluation participants indicated they were encouraged by the intervention. They provided insights to scoping groups on what was working and not working within their day-to-day experiences and realms of influence and what might work better if things were done differently. They were clearly pleased to have had the opportunity to contribute.

For researchers, however, the evaluation found mixed responses and tensions. For some, the design-led intervention was a revelation and a refreshing change from past bad experiences with the NSCs, their workshops, and heated arguments about research projects and funding. For others, while they were highly positive about the intervention they were also conflicted for various important reasons. For example, it was thought that the intervention was useful for generating ideas but questions remained about how it would guide funding decisions:

I really liked the design-thinking workshop and process but we need a bit more science process on the end of it. I think if you’re trying to design something for a business, and [the facilitator] says this, when you’re going through the process, it’s great for fast fails. Actually, what we were trying to get out was some solid, robust, long-term programmes and plans. (Leadership group evaluation participant)

Hence, while the intervention was seen as productive, there was concern that more science input was needed to decide what programmes and projects should be funded.

And while there was recognition that scientists needed to be open to other ways of knowing, and acceptance of doing things differently, concerns lingered about what role science should be playing:

Scientists like me are thinking scientists like me are the best thing since sliced cheese. Those are the sort of people, especially on a Science Challenge … who should be viewing [proposals] to say yay or nay or get a perspective on and that sort of stuff. I also appreciate that we’ve got to move beyond scientists like me out there and have different perspectives and all that sort of stuff. Having the judges [at the pitch days] and that sort of thing, I could see that there would be benefits there. The other side of that coin is we’re after a step change … which is heavily science driven. (Researcher evaluation participant)

Concerns about the role of science in the NSCs existed further afield. The following experience of a researcher evaluation participant illustrates difficulties encountered when trying to explain the intervention and its design-thinking terminology to researchers outside the intervention:

When we went to talk to people [after workshop 1], particularly science people, it was very hard to get them over their cynical attitude about the [National Science Challenges] and how all they do is ‘workshops, workshops, workshops, and here’s another workshop they’re having’ sort of thing. I felt that we weren’t in a position to be able to convey the end goal to them and that there would be some concrete things, so we were kind of going, ‘We’re preparing an investment prospectus.’ ‘What’s that?’ ‘I don’t know what’s going to happen with that.’ ‘Is it like an RFP [request for proposals]?’ ‘I don’t know.’ It’s that cynical nature that scientists have, it was really hard to get them on board. (Researcher evaluation participant)

This contribution illustrates how the impact-focused co-design model of knowledge governance was challenging for researchers inside and outside the intervention. For example, the investment prospectus (discussed below) was focused on goals for research. This clashed with traditional funding conventions that usually require the articulation of tightly defined science projects.

Empathy is central to design-thinking (Design Council UK 2023). It compels a designer to walk in the shoes of who he or she is designing for. To understand who the scoping groups were designing their goals for, and to capture perspectives not in the room, empathy mapping was used as part of the feedback step discussed above to finalise goals in the second workshop. Empathy mapping involves discussions with people who will use what is being designed to understand their contexts, world views, issues and challenges. While empathy mapping was quite a revelation for some, others were challenged by it:

Personally, from my natural scientist perspective, I thought [empathy mapping] was a bit wishy-washy. I’m supportive of social science and everything but I’m still a little cynical of the fuzzy edges of it. It seemed to me that that part was right in the fuzzy edges of what I’m comfortable with for a science process. (Researcher evaluation participant)

Empathy mapping was a bridge too far for the researcher quoted above who was concerned about the scientific rigour of the information empathy mapping might generate. This illustrates how far away the BHNSC was from conventional knowledge governance practice.

The outcome of a design-thinking process is a minimum viable product (MVP), which involves developing empathy with the end-user but also creating something that is not fully formed or final when it is presented for feedback. Indeed, finality is counter to the design process (Sheppard et al. 2018; Design Council UK 2023). The BHNSC intervention had two MVPs – the goals (discussed above) and an investment prospectus.

Although guidelines were explicit that the investment prospectus was not a research plan (see Duncan 2020 Appendix 9, p. 83), this is what several teams thought they were writing. For example, ‘I wrote the prospectus in the same way that I’d write any science application really’ (Researcher evaluation participant). It was clear that researchers struggled with the impact-focused requirements of the investment prospectus. Concerns were also raised by a researcher evaluation participant that the investment prospectus was a commercial world idea that was not appropriate for science institutions which, this evaluation participant maintained, operate differently from corporations in terms of where the ideas come from (i.e. from the bottom up in science organisations and from the top down in corporations).

The final activity for the teams was to pitch key aspects of their investment prospectus to a mock investors panel to get feedback to finalise their investment prospectus (i.e. another feedback loop). While it was intended to be a learning exercise, the evaluation found that the stakes were perceived to be very high, given that the investment prospectus would be used to make research programme funding decisions. It was observed that pitch presenters were surprisingly nervous. The evaluation found that they were feeling daunted by expectations of having to sell their ideas to non-scientists who would not have the usual reference points of fellow scientists. It seems it was hard to shift the competitive mindset of contestable funding and the prospect of disappointment if feedback on their presentation was not optimal. The nervousness also showed the extent to which researchers were in uncharted waters, not only in terms of the presentation (as funding bids are usually submitted online) but also the content. The focus of the investment prospectus and presentation was on impact and how to get there, which meant the science had become a smaller part of the story. This was not conventional practice in bidding for research funds and was, again, somewhat unsettling according to evaluation participants.

While these tensions were addressed by BHNSC leaders reminding everyone the investment prospectus was a first step, in our view, the tensions reveal confusion about how and where science fits in this new impact-focused research context.

A knowledge governance perspective

Reflecting on the findings of the evaluation using the principles framework shows that the intervention succeeded where it was important to do so. It was context-based, pluralistic, goal-oriented, and interactive. However, questions remain about what contributed to success and why were there tensions? Looking beyond the intervention to other levels of the knowledge governance system, i.e. knowledge systems and civic epistemology (Figure 1) provides some insight, which we explain below.

ANZ’s changing knowledge governance system: the knowledge systems level

Initially, the NSCs were expected to engage with stakeholders and do public outreach. However, in 2015, MBIE introduced additional performance criteria that, by 2019, formally required the NSCs to ‘actively co-design (at the outset) and co-develop/create (along the way)’ (MBIE 2019a, Appendix One, p. 11; see also MBIE 2015a). With these changes, MBIE was calling for the NSCs to work far more closely with Māori partners, stakeholders, and end users to co-produce research, based on expectations that a more collaborative co-designed approach would have a better chance of delivering outcomes and research impact (MBIE 2015a, 2017, 2019a, 2019b). While these changes have presented challenges for some researchers and research institutions (Mark and Hagen 2020; Duncan et al. 2020b), they aligned well with the BHNSC’s impact-focused vision, and its decisions about how it should proceed in tranche 2 (MBIE 2018).

How knowledge is governed in ANZ has also been changing for Māori. Since colonisation and the signing of te Tiriti between Māori and the British Crown in 1840, Western science has dominated the development and implementation of the nation’s environmental policy and legislative frameworks (Harcourt et al. 2022). However, in 1991, local governments were, for the first time, required to take account of te Tiriti principles and to acknowledge the interconnected and protection-based relationship Māori have with their land and its ecologies. In practice, though, Māori have struggled to have their values, issues, and knowledge incorporated into environmental management policy, regulation, and practices (Muru-Lanning 2012). Furthermore, Māori communities have been unable to participate in science and policy-making in culturally safe ways as Western science and policy approaches have mostly failed to recognise the full inter-relatedness of te ao Māori (Mercier and Jackson 2019; Jones et al. 2020). Hence, notwithstanding te Tiriti, the ability of Māori to influence policy and legislation has been limited (Taylor et al. 2021).

However, societal and political views have been shifting about the role of Māori in managing their lands, especially given their successes in building highly profitable land-based business enterprises and their careful stewardship of land. Furthermore, recent reviews of the nation’s research science and innovation system (Rauika Māngai 2019, 2020; MBIE 2021), its policy development system (Kukutai et al. 2021), and the resource management system (MfE 2021) have now recognised the importance of mātauranga Māori to all New Zealanders and that Māori must be central to policy implementation (DOC 2020; MfE 2020, 2021). As is the case internationally, there is increasing recognition that marginalised indigenous knowledge systems are likely to hold needed responses to sustainability issues (Kawharu and Tapsell 2021; Tengö et al. 2017; Tengö and Andersson 2022; Young 2021).

Although the shift to impact-focused research policy and instituting Māori cultural principles in environmental legislation and policy has changed the rules of the game at the knowledge systems level, ANZ’s civic epistemology remains dominated by Western science. It is here that ‘deep-seated patterns’ of knowledge practice identified by Jasanoff (2005, p. 260) can impede the implementation of decisions of an intervention. For example, Māori research is not funded because its methods are not deemed to be sufficiently scientific, or the ‘holistic character’ of indigenous knowledge ‘is constantly measured against an existing western scientific preoccupation with disciplinary specialisation’ (Cole 2017, p. 128). Hence, ways are still to be found to formally embed mātauranga Māori alongside Western science within the civic epistemology of ANZ’s knowledge governance system (Macfarlane and Macfarlane 2019; Mercier and Jackson 2019; Kukutai et al. 2021; Harcourt et al. 2022). Following the journey of the BHNSC in developing its co-design intervention provides useful insights on potential ways forward for improving knowledge practices in ANZ.

Making values explicit

Reflecting on the evaluation findings using the principles framework, and questions about what, in particular, contributed to success, reveals the importance of the BHNSC’s values (Table 2). From the outset, the BHNSC had committed to ‘embrace and embed’ its values in ‘every facet’ of its operations (see BHNSC 2023, Our Values).

Table 2. BHNSC values presented on its website https://bioheritage.nz/about-us/our-values/ (BHNSC 2023, Our values).

BHNSC Values	How the BHNSC applies its values
Mahi whaipainga	We care about making a difference for Aotearoa
Ngākaunui/Uekaha/Ngakau Whakapuke	We are passionate and enthusiastic about our work
Whanaungatanga	We work as a family We have fun together
Manaakitanga	We build trust and create a place that others want to be part of We build mana in others around us to enhance the mana of the whole
Mana Motuhake/Tino Rangatiratanga	We recognise and empower sovereignty and autonomy
Whakapapa	We recognise interconnections We have an intergenerational vision
Kaitiakitanga	We enable stewardship of our biological and cultural heritage
Mātauranga	We recognise the value of blending traditional and modern knowledge
Tohungatanga/Ngā tiketiketanga o te pai	We apply excellence to everything we do
Mahi rangapū	We work respectfully in partnership

The MBIE mid-way review recognised the importance of BHNSC’s values in tranche 1:

Particularly worthy of comment is the adoption of a suite of Māori-derived values for the whole Challenge which provide a strong cultural framework for the integrated, collaborative and caring approach the Challenge embodies. This is reflective of the need for shifting cultures across the research community which the Challenge is addressing head on. It is inspiring to hear leading researchers state that the ‘values’ of the Challenge were one of the attractions of their involvement. (MBIE 2018, p. 11)

The BHNSC values are well publicised and evident in how it has operated as an organisation. For example, in recognising and empowering Māori sovereignty and autonomy, the BHNSC operates with a Director and a Kaihautū Ngātahi/Co-director Māori at the head of the organisation. They function as Co-directors and this is not tokenism; they work together, support each other, and make decisions together.

The MBIE mid-way review also noted that the BHNSC’s ‘collaborative culture is no accident with Challenge leadership … creating the environment for active collaboration to flourish and modelling collaborative behaviour in the leadership of the Challenge. This appears to be having a cascading effect’ (MBIE 2018, p. 7). BHNSC approach was seen as ‘unique, welcome and beneficial to the science being undertaken’ (MBIE 2018, p. 5) and its approach and actions in partnering with Māori as ‘exemplary’ and ‘class leading’ (MBIE 2018, p. 7).

For Māori evaluation participants, co-leadership at the top of the BHNSC demonstrated a strong commitment to te Tiriti. This action fostered much needed trust in the intervention. For example:

That [co-everything] allowed for relationship building, and for me, that’s the key underpinning thing that makes the difference, building that relationship, building that trust and sharing the learning as a part of that, so the process of co-everything, building relationships and the values [worked well]. (Māori evaluation participant).

Co-leadership roles were also taken up by Māori participants across the scoping groups.

Putting values into practice

An important value from a knowledge governance perspective is the commitment to foster collaboration between the knowledge systems of mātauranga Māori and Western science, articulated by the BHNSC as ‘blending traditional and modern knowledge’ (Table 2). Putting this value into practice was achieved in a number of ways. As discussed, co-leadership built trust with Māori partners and signalled to everyone that the BHNSC was committed to bridging ontological and epistemological divides.

The evaluation identified the importance of a range of administrative instruments that the BHNSC had developed to establish the rules of engagement for the intervention and for scoping group members to sign up to. Specifically, a letter of offer established a foundation for exchange between the BHNSC and each member, i.e. payment for time and commitment to tasks outlined in the Strategy 2019–2024 (BHNSC 2018). A non-disclosure agreement classed contributions of scoping group members (including ideas and concepts, information, data, mātauranga Māori, know-how, whether technical or not) as ‘confidential information’ that could not be disclosed or published by others without consent. Hence, everyone was bound by ‘certain duties of confidentiality and non-use in respect of the Confidential Information in both written and verbal form’ (see Duncan 2020 Appendix 6, p. 77).

Terms of reference provided guidance on what was expected, i.e. participation in workshops and the delivery of the investment prospectus (see Duncan 2020 Appendix 5, pp. 72-75). The BHNSC had also developed operating principles, a code of conduct, policies on equity, diversity, access and inclusion (see Duncan 2020 Appendix 5, p. 76) as well as a range of guidance documents on how to conduct culturally safe research with Māori (BHNSC 2023, Resources/Operational & Guidance Documents). These instruments were put in place to protect the ideas and intellectual property of all researchers, which encouraged authentic participation and knowledge exchange.

From a knowledge governance perspective, these instruments can be seen as ways to enact the BHNSC’s values to change knowledge practices. Notably, the BHNSC’s values flowed through to scoping groups’ goals. Some incorporated aligned values into their goals, while others articulated them separately as principles so the values would guide the interpretation and implementation of the goals in the future (BHNSC 2020).

Values shaped how people were recruited to the intervention:

I think the way we picked the … [scoping group] teams worked really well. Having an EOI [expression of interest] process that was focussed on values, what you brought to the team and the like, was a really good process that got us the right people. (Leadership group evaluation participant; see Duncan 2020, Appendix 6, p. 79–80)

These somewhat mundane administrative instruments paved the way for diverse, transparent, and inclusive participation in the intervention. In particular, they encouraged Māori participation. In effect, the instruments, and the values they embodied and enacted, put mātauranga Māori and Western science on the same epistemic footing. As noted earlier by a stakeholder evaluation participant about the incorporation of mātauranga Māori in designing the goals, ‘It wasn’t a separate objective, it was just how we operated’.

A shift to strategic research programmes rather than unaligned research projects

As explained, the BHNSC values, and how they were enacted, shaped how the intervention was conducted and how participants engaged with each other. The BHNSC’s values also challenged research conventions. Specifically, caring about making a difference tops the BHNSC’s list of values (Table 2). In its Strategy 2019–2024, the BHNSC maintained it needed to ‘actively lead better and faster pathways from science discovery through to delivering impacts at regional or national scales’ (BHNSC 2018, p. 4). The imperative to make a difference and move faster was in response to what the BHNSC had identified as entrenched and disabling features within ANZ’s knowledge governance system:

Research effort in New Zealand is fragmented, competitive, and often not collaborative. Since 2014 we have identified more than 1000 research projects or programmes aligned with the Challenge mission – millions of dollars of investment. Yet this diverse effort lacks overall cohesion and focus, in part because it has never been harnessed in a strategic framework to deliver measurable benefit for New Zealand (Investment Prospectus Guidelines in Duncan 2020, Appendix 9; see also BHNSC 2018, p. 4).

For the BHNSC, a new approach to funding was needed if it was to make a tangible difference (BHNSC 2018). To this end, the intervention focused on developing goals and impact pathways for overarching research programmes of work (focused on delivering impact) rather than unaligned research projects focused on publications. Rather than scientists developing research projects and applying for funding when it becomes available through a competitive process, the BHNSC committed to making strategic decisions, informed by collaboratively designed goals and impact pathways, that would guide who was to be involved and where research funds would be spent. The investment prospectus was intended to encourage co-investment, build relationships and networks, and envision what the on-ground change end points should be. Having a clear idea up-front about what research needed to be done, how it should be done, where the research might land and the benefits of doing so was seen by the BHNSC as crucial for encouraging others to invest not only money, but also time, expertise, and case study locations.

Our analysis has shown that the BHNSC’s values have been central to its success in designing a diverse, transparent, and inclusive intervention which set a course for fostering collaboration between the knowledge systems of mātauranga Māori and Western science. Yet, there have also been tensions as the BHNSC has sought to shift decision-making boundaries between science and society to fund impact-focused research programmes.

Discussion

Calls from government institutions and research funders for co-design and co-production are now routine (Mark and Hagen 2020). However, as discussed, doing co-design and co-production are resource intensive, and outcomes do not always translate into action and change. Hence, evaluations are needed to identify what can enable and constrain co-design and co-production efforts.

Reflecting on the evaluation findings across the principles framework (Norström et al. 2020), we found that the BHNSC intervention succeeded where it was important to do so. It was context-based in addressing issues of deep concern to New Zealanders. Through its expertise and existing partnerships, the BHNSC was well-placed to develop its co-design intervention. The intervention was pluralistic in that it brought together people with diverse knowledges, skills, and talents, and created a place for multiple perspectives and skills to contribute to designing new ways forward. The intervention was built around goals and focused on transforming them into ‘pathways to impact’ in an investment prospectus. The design-thinking approach meant each step involved meaningful interactions and participants were focused on goals and how to get there rather than individual agendas. A number of evaluation participants were changed by the intervention. Of course, further research is needed to understand the full impact of the intervention as we have focused here only on the first step.

While the principles framework was useful, questions remained about what, in particular, contributed to the intervention’s success and why there were tensions? From a knowledge governance perspective, what enables and constrains co-design and co-production efforts can reside outside an intervention itself. As raised by Wyborn et al. (2019, p. 326), ‘effective co-production requires more than the good will and effort of those engaged in the process. Rather, aligned structures and institutions enable (whereas misaligned ones undermine) collaborative decision making’.

We identified the BHNSC’s values as an enabling factor in the design and conduct of the intervention. We discussed how the BHNSC put its values into practice. For example, the BHNSC modelled co-leadership. It also reframed the purpose of research funding to deliver impact via strategic programmes of research rather than unaligned research projects. It also developed various administrative instruments and safeguards (e.g. terms of reference, non-disclosure agreement, intellectual property management plan, code of conduct, etc.) that validated the importance of ‘weaving’ (Tengö et al. 2017, p. 17) the knowledge systems of mātauranga Māori and Western science (Jones et al. 2020). The BHNSC’s actions and instruments embodied many of its values, in particular, recognition of the ‘value of blending traditional and modern knowledge’ (BHNSC 2023 Our Values). In effect, the BHNSC’s modus operandi instituted solid ontological and epistemic footings for mātauranga Māori alongside those that already exist for Western science. As such, these knowledge systems were conceived in the intervention as complementary ways of knowing and doing.

The evaluation also found tensions. While evaluation participants were pleasantly surprised and positive about the design-thinking activities, some researcher evaluation participants were indifferent to some of its end-user and impact-focused methods. There was also confusion about where science fitted, and the feeling its adjudicatory role was being diminished. It was also evident there were difficulties for researcher evaluation participants in changing their conventional knowledge production practices (e.g. bidding for funding) in the new impact-focused research policy paradigm where ‘effective engagement of actors, institutions and knowledge-sharing processes' is critical (Tengö et al. 2017, p. 17). Researcher evaluation participants also encountered cynicism towards the NSCs and the workshops the NSCs were conducting in their varying co-design and co-production efforts (Duncan et al. 2020b). At the end of the intervention, it was observed that ‘all the things that came out of this weren’t science, they were just big game-changing, system-changing things. There was very little black and white science projects that came out in the traditional sense’ (Leadership group evaluation participant).

While these observations indicate the process had landed where it was intended, i.e. at strategic overarching programmes of work to guide actions and research into the future, the tensions are important to consider. From an STS perspective, these tensions could be seen as the ‘boundary work’ practitioners of ruling knowledge systems do to protect their authority and credibility (Gieryn 1983, p. 781). Hence, the concerns raised could be dismissed as mere talk or a lack of understanding of the first steps of the Strategy 2019–2024 (BHNSC 2018). However, from a knowledge governance perspective, we argue that they reflect unease about how the new impact-focused research paradigm is shifting decision-making boundaries between science and society.

As discussed, much has changed at the knowledge systems level of ANZ’s knowledge governance system. For example, for some time, MBIE has been calling for research to connect more closely with impact and sustainability (MBIE 2015b, 2019b). The mission-led NSCs were created in 2014 and, since then, MBIE’s NSC performance framework has mandated the use of co-design and co-production (MBIE 2015a, 2019a). There have also been moves to embed mātauranga Māori and Māori cultural concepts in environmental legislation and policy (Rauika Māngai 2019, 2020; Kukutai et al. 2021; MBIE 2021; MfE 2021).

These are significant and needed reforms. Yet, there has been limited research and guidance on what these reforms mean for knowledge governance in ANZ. Importantly, the knowledge governance conceptual framework draws attention to the ‘deep-seated patterns’ of civic epistemologies (Jasanoff 2005, p. 260) that are likely to continue to hinder the ‘weaving’ of mātauranga Māori and Western science (Macfarlane and Macfarlane 2019; Mercier and Jackson 2019; Jones et al. 2020; Taylor et al. 2021; Harcourt et al. 2022).

As discussed in section 2, a civic epistemology embodies formal and informal rules that structure social, political, and cultural practices that shape how societies make collective choices (Jasanoff 2005; Van Kerkhoff and Pilbeam 2017). Hence, a nation’s civic epistemology embodies practices that constitute, for example, expertise (e.g. credentials), objectivity (e.g. value-free methods), and evidence (e.g. peer review) (Jasanoff 2005). Recognising how dominant (and important) these institutions and practices are in governing Western societies highlights the challenges to be confronted when co-design and co-production interventions seek to open up science decision-making to other ways of knowing and doing (Ezrahi 1990; Jasanoff 2004, 2005; Duncan 2013; Van Kerkhoff and Pilbeam 2017; Wyborn et al. 2019; Nadasdy 2021).

Hence, it could be argued that the research programmes instigated by BHNSC’s intervention will encounter the same old issues whereby mātauranga Māori is compromised by ‘Western orientated values, structures and institutions’ (Macfarlane and Macfarlane 2019, p. 55). However, we argue that the BHNSC intervention offers useful insights for moving ANZ’s knowledge governance forward. For example, BHNSC’s values were central to all aspects of the intervention. The possibilities of values instigating change across ANZ’s knowledge governance system were foreshadowed by MBIE in its mid-way review. It noted the ‘Māori-derived values’ had given the BHNSC a ‘strong cultural framework’ within which to operate (MBIE 2018, p. 11). The review maintained the BHNSC was doing ‘excellent work integrating Mātauranga Māori alongside established research and science methodologies’ and that this ‘novel work’ was ‘realising considerable insights and learnings beneficial to the science and collaborating parties’ (MBIE 2018, p. 9). Indeed, the review recommended MBIE facilitate cross-NSC interactions so other NSCs could develop the ‘same high standard of engagement’ with Māori (MBIE 2018, p. 5).

From a knowledge governance perspective, putting BHNSC’s values into practice required the creation of new conventions (e.g. leading with values), writing new rules (e.g. co-leadership and ‘co-everything’) and rewriting old ones (e.g. related to funding, engagement with Māori and protection for mātauranga Māori). In doing so, the BHNSC was shifting long-held social, political and cultural knowledge practices (Van Kerkhoff and Pilbeam 2017). Instigated at the knowledge systems level, the BHNSCs values, and how they have been put into practice, shaped the intervention and are fostering new knowledge practices for the creation, sharing, protection, use and application of knowledge into the future (Van Kerkhoff and Pilbeam 2017). We argue that these values-inspired knowledge practices have the potential to recalibrate ANZ’s civic epistemology.

Conclusions

Our analysis has demonstrated the utility of the principles framework for assessing the success of co-design and co-production interventions (Norström et al. 2020). We have shown how the BHNSC’s intervention was context-based, pluralistic, goal-oriented, and interactive. Our analysis has also demonstrated the utility of the knowledge governance conceptual framework for looking beyond an intervention to identify enabling and constraining factors. The BHNSC’s values were identified as a key enabling factor in the intervention. BHNSC’s values underpinned its design-thinking intervention, and its decision to reject the conventional competitive funding model on the basis it was not a strategic or cohesive way to address the complex and urgent biodiversity and biosecurity issues facing ANZ. The BHNSC’s values fostered a ‘co-everything’ approach and a complementary relationship between mātauranga Māori and Western science knowledge systems.

Our analysis suggests that the careful articulation and enactment of values at the knowledge systems level of ANZ’s knowledge governance system has potential to shift what might appear to be an immutable civic epistemology that is currently dominated by Western science, to accommodate the ‘knowledge, culture, values and worldview’ of mātauranga Māori (Hikuroa 2017, p. 5).

Our analysis also highlights the importance and epistemic value of organisations like the BHNSC at the knowledge systems level of ANZ’s knowledge governance system, and the role such entities can play in challenging and recasting the status quo towards more sustainable futures. From the knowledge systems level, the NSCs have power and resources and, as we have seen, the capacity to intervene to foster new knowledge practices, and funding to chart new sustainability trajectories.

Importantly, though, simply creating such organisations is not enough. How they operate is key. So too are the values that underpin everything they do. The MBIE mid-way review noted that the BHNSC’s ability to open-up spaces for important conversations, collaboration and its mission was ‘an emergent property of the quality of leadership’ and that the style and depth of collaboration was already ‘emerging as a major legacy’ of the BHNSC (MBIE 2018, p. 7).

To conclude, the BHNSC did not just seek to create a good co-design process. It was fostering systemic change by modelling a complementary relationship between mātauranga Māori and Western science knowledge systems. As such, it foreshadowed a paradigm where both knowledge systems might co-exist. The BHNSC’s values were central to its activities and all aspects of the intervention and beyond. Hence, we propose a further principle for assessing the quality of co-production processes: values-inspired. Evidence of achieving this principle could be the presence or absence of values, how the values align with the context and sustainability objectives, whether they are publicised, how they are enacted and used, leadership commitment, what instruments they are grounded in and enabled by, how they flow through an intervention, and how they challenge the existing knowledge system and civic epistemology?

Further research is needed to examine more closely how the intervention has shaped the research programmes it has instigated, and how the BHNSC’s values-inspired knowledge practices are fostering collaborative research and on-ground change.

Acknowledgements

We would like to sincerely thank the scoping group members who participated in the evaluation (Duncan 2020) for their time and insights. We would also like to acknowledge Andrea Byrom, Melanie Mark-Shadbolt, and other BHNSC decision-makers, for their commitment to learning from doing and sharing lessons learned, which has been made possible by commissioning and funding the evaluation and this work.

Disclosure statement

The authors declare that funding provided to Manaaki Whenua Landcare Research by the BHNSC to undertake the work presented in this paper and the completed evaluation may be considered a potential competing interest. It should be noted, however, that the evaluation was conducted independently, which was required by the BHNSC.

Additional information

Funding

This work was conducted with the support of (New Zealand’s Biological Heritage National Science Challenge).