The science-society relationship in Aotearoa: practicing responsible innovation in the New Zealand research and innovation system

ABSTRACT

Reports on Aotearoa New Zealand’s research, science, and innovation (RSI) system suggest the sector could improve its social responsibility and create more social value. However, researchers and innovators within the sector find this challenging. Through qualitative interviews and a national survey of RSI system participants, this study explores the current perceptions, practices enacted, and support given around responsible research and innovation practices. Findings indicate that researchers and innovators have diverse understandings of what responsible research and innovation entails, viewing anticipation and reflexivity-aligned practices as the most important aspects of responsible research and innovation. Participants feel most supported by research organisations and funding bodies to enact such aspects of responsible practice. However, they perceive a discrepancy, across all dimensions explored, between what researchers and innovators should do, and what they actually do in terms of enacting responsible practices. Thus, significant room for improvement is identified, particularly regarding practices that align with researchers’ and innovators’ obligations under Te Tiriti o Waitangi, and requirements under Vision Mātauranga.

KEYWORDS:

• Responsible innovation
• Aotearoa New Zealand
• science-society relationship
• research and innovation system

Introduction

In Aotearoa New Zealand (henceforth Aotearoa) the government is consistently one of the largest funders of research, science, and innovation programmes (MBIE 2021a). Through this funding, the government strives to encourage and support research and innovation work that is likely to provide the most social value (MBIE 2021a). Efforts to actively enhance the social value of research, science, and innovation generally require researchers to reflect on, and potentially alter, the priorities, practices, and potential impacts of their research to ensure they work in a responsible manner (Gibbons 1999; Lekka-Kowalik 2010). In light of recent calls from the Aotearoa Ministry of Business, Innovation and Employment (MBIE) for research to improve its social responsibility and social value creation, this paper draws on a mixed methods approach to explore the following two research questions. Firstly, what do participants of the research, science, and innovation (RSI) system feel constitutes responsible research and innovation in Aotearoa? And, secondly, to what extent is responsible innovation practiced, and supported, in this country? For the purposes of this paper the social value of research is understood to refer to the benefits to society that come from its conduct, impacts, and outputs. The responsibility of research and innovation is understood in terms of the degree to which the researchers and innovators producing work that may create social value engage in practices that they and others consider to be ethical, just, and equitable.

The RSI system in Aotearoa consists, in large part, of seven Crown Research Institutes (CRI), eight universities, and several independent research institutes. Established during the 1990s, the CRIs were set up as independent crown entities, with a business model that reflected the neo-liberal governance reforms of the day (Crown Research Institutes Act 1992). Reliant on industry, government, and competitive research funding, for the last 30 years they have delivered a mixture of science as ‘service’ and research science, while also being tasked with delivering research for the benefit of Aotearoa. However, a 2020 review (Te Pae Kahurangi), assessing how well this model has positioned CRIs to meet ‘New Zealand’s current and future needs’ (MBIE 2020), highlighted that fragmentation and competition within the system were limiting researchers’ and innovators’ ability to meet such needs. That 2020 review and the subsequent Te Ara Paerangi – Future Pathways Green Paper (MBIE 2021b) and Programme can be understood as efforts to reconfigure the science-society relationship in Aotearoa, in order to increase the responsibility of, and social value creation from, research undertaken by public research organisations. The key recommendations of Te Pae Kahurangi include an increase in collaboration, more customer-centric approaches, greater focus on meeting Māori and iwi aspirations, and more efficient and effective use of scarce resources (MBIE 2020).

Exploration of the social value and responsibility of scientific research has long historical roots in philosophy of technology and science (e.g. Lenk 1983; Jonas 1985; Pimple 2002; Moor 2005; Luppicini 2008). Internationally, scientists have warned of the dangers of powerful new technologies and urged a precautionary approach to technological development and use since the development of nuclear weapons and technology (Russell and Einstein 1955; Commoner 1966; Abelson 1970; Ziman 1994, 1998, 2001; Rotblat 1999; Joy 2000). The Aotearoa scientific community shared this concern with the social responsibility of science, which has been taken up within six major surveys of scientists over the last three decades (Berridge et al. 1995; Sommer and Sommer 1997; Sommer 2001, 2010; Fisher et al. 2005; Small and Fisher 2005; Small and Jollands 2006; Small 2011).

In 2010, Sommer (p. 21) concluded that in Aotearoa scientists’ sense of social responsibility while ‘already strong, strengthened over the past decade’. Regarding the development of potentially dangerous technologies, their data showed ‘a strong inclination towards societal responsibility and citizen involvement over strict expertise’ (Sommer 2010, p. 24). Small’s (2011, 2013) investigations of scientists’ understandings of social responsibility in research revealed three main themes underpinning their notions of scientific social responsibility: doing public good, public engagement, and compliance with legal, social, scientific, and personal moral norms. Small thus concluded that scientists in Aotearoa had a good awareness of the need for social responsibility in research and a strong positive attitude regarding the importance of personal behaviour to enhance social responsibility in research. The findings of these surveys, and the expectations laid out by Te Pae Kahurangi and Te Ara Paerangi, suggest that the science-society relationship and attitudes around the social responsibility of science have been evolving in Aotearoa.

Over the last two decades, the concept of Responsible Innovation (RI) emerged in Europe as one response to such evolving science-society relationships. The concept is rooted in the ‘responsibility gap’ (Owen et al. 2013) present in a long list of regulatory failures to manage the externalities of techno-scientific innovation. Efforts to address these failures and promote more responsible techno-scientific conduct have come in different guises, from participatory risk assessments to ELSA/ELSI approaches (ethical, legal, and social aspects/implications of emerging sciences) (Owen and Pansera 2019; Christensen et al. 2020). RI combines many of these approaches to ensure that ‘science and innovation are directed at, and undertaken towards, socially desirable and socially acceptable ends, with connotations of trust and integrity’ (Owen et al. 2013, p. 27). Different frameworks have been proposed to realise these aspirations including the AIRR framework which outlines four core dimensions that can be ‘heuristically helpful for decision-making on how to shape science and technology in line with societal values’ (Macnaghten 2016, p. 6; also Stilgoe et al. 2013). These are:

• Anticipation: exploring the possible futures and impacts, both positive and negative, that science and innovation could create.

• Inclusion: bringing diverse voices into science and innovation. This implies meaningful consultation on the ends and means of innovation and incorporating different forms of knowledge, including indigenous knowledge systems.

• Reflexivity: holding up a mirror to one’s activities, assumptions, and potential biases. It involves a collective effort to reflect on the values and drivers for science and innovation.

• Responsiveness: responding to stakeholders, changing public values, and new circumstances by adjusting the course of science and innovation processes. Responsiveness is also enabled on an organisational and science system level.

The RI concept and AIRR framework, or parts thereof, have been applied by some researchers in Aotearoa exploring the social responsibility of science (Dalziel et al. 2018; Lees and Lees 2018; Eastwood et al. 2019; Brier et al. 2020; Eastwood et al. 2021; Edwards et al. 2021). Others have suggested that additional dimensions also require attention, including questions concerning social justice and openness (Espig et al. 2022):

• Equity and social justice: ensuring that science and innovation do not increase socio-economic inequities and create equitable benefits for diverse societal groups (e.g. Ludwig and Macnaghten 2020).

• Openness and transparency: means ‘open and free access to and communication of data, results, purposes, risks, uncertainties, applications and implications to facilitate inclusive deliberation’ (Owen and Pansera 2019, p. 32).

While realising these dimensions in practice often depends on the willingness and skills of researchers and innovators, RI scholars have also highlighted the enabling, or limiting, role of research organisations and funding agencies (Werker 2020; Regan 2021; Espig et al. 2022). They suggest that responsible research and innovation outcomes thus also hinge on structures and processes within the wider science and innovation system. The study presented in this article follows this perspective and investigates the science-society relationship in Aotearoa based on the dimensions outlined above with the addition of questions that address researchers’ engagement with Vision Mātauranga (MoRST 2005), a policy incorporated into the CRI’s Statements of Core Purpose in 2011 which requires the enablement and equitable inclusion of Māori people, knowledge, and resources in research practice in line with these organisations’ obligations under Te Tiriti o Waitangi (Aotearoa’s founding treaty signed between representatives of the British Crown and a majority of Māori tribes in 1840).

Materials and methods

A mixed methods approach was employed to investigate what could be gained from the operationalisation and institutionalisation of RI within the context of the Aotearoa RSI system. The research was approved by AgResearch’s Human Ethics Committee (#10/20). Twenty-four in-depth, semi-structured interviews were undertaken via video calls with senior representatives from Māori agribusinesses, industry bodies, government agencies, and research organisations across Aotearoa to explore enablers and barriers to enacting RI within the RSI system. With informed consent, the interviews were audio recorded and lasted 30–60 min. Questions were tailored to individual interviewees, however all interviews covered the following topics: understandings of responsibility in research and innovation, how responsibility is currently enacted in the RSI system, challenges associated with making research and innovation responsible, the role of the government as funder and regulator of research, and what might support more responsible research and innovation conduct in Aotearoa. The interviews were transcribed and imported into NVivo12, where they were coded and thematically analysed (Clarke and Braun 2012).

The interview findings were complemented by an online, national baseline survey of those who work within the Aotearoa RSI system. The survey aimed to better understand scientists’ and researchers’ perspectives on the relationship between science and society. For the purposes of the survey, the term ‘researcher’ was defined as any person who carries out scientific or academic research in Aotearoa. The survey included nine demographic questions, two qualitative research questions, and approximately three dozen quantitative research questions (including multi-tiered questions), which covered all dimensions of the AIRR framework, as well as social equity and justice (including Vision Mātauranga), and openness and transparency. Thematically grouped sets of quantitative questions were followed by open text boxes that provided participants with an option to add comments, which assisted the interpretation of results. Some quantitative questions were adapted from a similar survey conducted in Australia (Herington et al. 2019). The survey was constructed and hosted on the Qualtrics^xm survey platform and the link to the survey was distributed via email, following a multi-tiered dissemination strategy.

All universities, CRIs, and several independent research organisations were contacted for distribution, with data collection occurring from 10th February until 26th March 2021. The survey data was downloaded into Excel, cleaned, and analysed, to calculate mean scores, standard error, and 95% confidence intervals. For all questions scored on a Likert scale (the majority), t-tests were used to calculate whether mean scores differed significantly from neutral and Cohen’s d was used to calculate the effect size of the difference. T-tests and Cohen’s d were also used to test the significance and measure the effect size of the mean difference between statements about what scientists should do and what they are currently doing, and the mean difference between support provided to scientists by research organisations and support provided by funding agencies. Further analyses were conducted using the StatsIQ software in Qualtrics^xm.

A total of 250 researchers and scientists completed the survey, however response rates varied per question. All participants except one (99.6%) provided their informed consent to participate in the survey. The individual who did not provide informed consent was immediately exited out of the survey. Just over half of the participants were university staff (55.3%), 42.3% were from CRIs, and six (2.37%) were from ‘Other research institutes’, with most CRIs and universities represented by at least one respondent.

The largest group of participants identified as researchers/scientists (52.6%), followed by teaching and research academics (22.7%) and Masters or PhD students (11.3%). Nearly half of participants identified as having a background related to the natural sciences (49.2%), and almost one-fifth were from applied sciences (including business, engineering, and technology) (17.9%). Smaller proportions of the sample identified their disciplinary backgrounds as, the social sciences (11.9%), the formal sciences (e.g. computing and maths) (7.5%), the humanities (4%) or other fields (10%). Just over half of participants identified as female (52.8%), less than half as male (44.8%), and a small number identified as non-binary/third gender (1.2%) or did not wish to say (1.2%). The age of participants followed a bell-curve, with approximately 70% of participants aged 30-59.

In terms of ethnicity, the majority of participants identified as Pākehā/NZ European (54.9%), followed by European (20.6%), Asian (7.4%), Māori (4.3%), Middle Eastern/Latin American/African (2%), and Pacific peoples (0.4%). The survey, therefore, provided a representative sample of Pākehā/NZ European and European groups (70.2% collectively according to the 2018 census (Statistics New Zealand 2019)) and of Middle Eastern/Latin American/African groups (1.5%) within the Aotearoa population. However, it significantly under-represented Māori, Asian groups, and Pacific peoples, who made up 16.5%, 15.1%, and 8.1% of the population in 2018, respectively (Statistics New Zealand 2019). This is reflective of a larger issue surrounding under-representation in the Aotearoa RSI system. Currently, Māori scientists make up less than five percent of the RSI workforce, scientists of Asian origins make up approximately five percent, and Pacific peoples make up approximately one percent (McAllister et al. 2020). Thus, while the survey provides a relatively ethnically representative sample of scientists and researchers working in the Aotearoa RSI system, this workforce is not representative of the broader population.

Results

The science-society relationship in Aotearoa

Meanings of responsibility in science and innovation

The survey commenced with two qualitative questions that prompted open-ended written feedback. The first question invited participants to describe what responsibility meant to them in the context of research and innovation in Aotearoa. The replies from respondents (n = 240) included diverse aspects related to the conduct and outcomes of research and innovation. Furthermore, responses often covered multiple facets of responsibility. The most common themes are summarised in Figure 1.

Figure 1. Main themes in respondents’ understanding of the meanings of responsibility in research and innovation in Aotearoa (percentages of 240 responses – note that some respondents listed multiple facets, thus percentages sum to greater than 100).

Participants most often reported understanding responsibility, in the context of research and innovation, to relate to various facets associated with completing high quality research (71% of respondents, n = 171). This included following principles of scientific integrity, such as accurately and honestly reporting results and maintaining ethical standards to avoid harm to research participants or subjects. Other descriptions highlighted that sticking to ethics principles also involves broader social and environmental considerations, for instance that limited resources are used effectively or that research is undertaken for the ‘right reasons’. The following response illustrates the breadth within this thematic cluster:

Being ethical. By that I mean:

Thinking through scientific results to ensure that interpretations are valid, rather than rushing to publish something high profile but exaggerated or even wrong;

Being fair to colleagues and students in acknowledging contributions to research;

Communicating science to non-scientists so that our knowledge can be used, and ensuring that even as we make that communication clear to others, we don’t cheat or say things that are wrong;

Not stealing other people’s research ideas.

A second key understanding of responsibility was a need to produce benefits from research, whether to – in decreasing order of emphasis – society and the public good, bodies of scientific and applied knowledge, or the environment (33%, n = 78). Some respondents felt that responsibility involved anticipating potential uses of research and innovation, their implications (ethical, social, cultural), and limitations (18%, n = 43). Transparency and openness were likewise considered to be significant components of responsibility to others, which included disseminating research results widely, making research accessible to the public, and not censoring results (18%, n = 43). Less frequently mentioned themes were being inclusive (10%, n = 24) and reflexive (8%, n = 20), accepting accountability (8%, n = 20), and meeting obligations associated with Te Tiriti o Waitangi (5%, n = 12).

A second open-ended question asked participants to whom they believed researchers are responsible. Most participants completed this question (n = 243), naming a range of social groups, organisations, as well as abstract entities. The most frequent responses are included in Figure 2.

Figure 2. Main groups and entities researchers believe they are responsible to (percentages of 243 responses) note as respondents were able to list multiple groups percentages sum to greater than 100.

A majority of respondents considered researchers to be responsible to the public or a similar general group (79%, n = 193). Direct references to the ‘public’ were most common, but respondents also mentioned society, community, and humanity. Some references were made to specific subsets within the general public, such as marginalised groups and taxpayers; taxpayers were mentioned because they funded their work, while responsibility to marginalised groups arose more out of a duty of care. About half of respondents considered they were responsible to governing bodies, which includes funders, government, the legal system, boards and committees, and research organisations as employers (47%, n = 113).

Other less frequently mentioned groups were the science community or part thereof, such as peers and colleagues, collaborators, or students and junior researchers (27%, n = 66). Some participants noted responsibilities towards stakeholder groups (e.g. industry, research participants, or research target groups; 21%, n = 52), while others felt that researchers were primarily or partially responsible to themselves (15%, n = 36). A smaller group stated adherence to higher standards and principles, for instance scientific principles, truth and ethical principles, or God (9%, n = 15). Other themes include responsibility to the planet, life on Earth, and/or the environment (8%, n = 19), and responsibility to Māori and Te Tiriti o Waitangi (4%, n = 10).

The current and envisioned states of the science-society relationship

The primarily quantitative sections of the survey included sets of questions that were designed to correspond to core dimensions of the responsible innovation concept (anticipation, inclusion, reflexivity, responsiveness), as well as transparency and openness, and social equity and justice. This section outlines a combined analysis of some of these dimensions in relation to respondents’ assessments of the current and envisioned states of research and innovation in Aotearoa. The dimensions have been combined in an attempt to practically anchor them to practices that happen to bridge multiple dimensions. Rated on a seven-point Likert scale, overall findings indicate that participants did not feel strongly that research and innovation practices in Aotearoa are either responsible or irresponsible, with mean scores ranging from 3.39 to 5.04 (see Figure 3).

Figure 3. Survey respondents’ assessment of the current state of responsibility in research and innovation in Aotearoa (Scale: 1.0 = strongly disagree to 7.0 = strongly agree; Assessment of Vision Mātauranga: 1 = Not at all to 7 = To a strong degree).

The practices which were scored highest, in terms of agreement that they are currently undertaken within research and innovation in Aotearoa were those aligned with the dimensions of anticipation and reflexivity. These dimensions were assessed through three questions the mean scores for which were all above 5. The practice that was scored lowest was the fair representation of all New Zealanders’ interests with a mean score significantly lower than neutral indicating that respondents disagreed that the interests of all New Zealanders are currently equally represented in science and innovation (m = 3.39, p < .001, Cohen’s d = −.044, a small effect). Respondents also disagreed that science is accessible to all members of the interested public (mean score = 3.44, p < .001 and Cohen’s d = −0.32, a small effect). These practices are associated with the dimensions of openness and transparency and social equity and justice. A closely related assessment within the dimension of inclusion was whether Vision Mātauranga (Māori knowledge, people, and resources) currently influences the direction, execution, and delivery of research in Aotearoa. The mean score of 4.14 (p < .14 ns, d = 0.10) was not significantly different from the scale mid-point, indicating respondents considered that Vision Mātauranga had a moderate influence on the RSI system in Aotearoa.

Comparing respondents’ assessments of current and envisioned practices shows that room for improvement exists across all the considered dimensions. This finding is highlighted by statistically significant differences between views on what researchers should be doing versus what they are doing (see Table 1). In the three sets of questions related to anticipatory and reflexive practices, participants’ views differed significantly on whether researchers should and do critically consider their work in terms of potential positive and negative outcomes (see Table 1 12.1–.2), underlying motivations and assumptions (13.1–.2), and potential limitations and blind spots (14.1–.2).

Table 1. Survey respondents’ assessment of what researchers should be doing, and what they are doing.

Question	Mean (95% CI) p-value^a Cohen’s d^a	Mean Diff (should-do) p-value^b Cohen’s d^b
12.1 Researchers should consider the potential outcomes of their work (both positive and negative).	6.75 (6.43–6.71) p < .001 d = 2.31	1.71 p < .001 d = 1.24
12.2 Researchers do consider the potential outcomes of their work (both positive and negative).	5.04 (4.87–5.21) p < .001 d = 0.78
13.1 Researchers should consider the motivations and assumptions underlying their work.	6.63 (6.50–6.75) p < .001 d = 2.69	1.60 p < .001 d = 1.37
13.2 Researchers do consider the motivations and assumptions underlying their work.	5.03 (4.86–5.20) p < .001 d = 0.77
14.1 Researchers should consider potential limitations and blind spots in their work.	6.69 (6.58–6.81) p < .001 d = 2.94	1.65 p < .001 d = 1.47
14.2 Researchers do consider potential limitations and blind spots in their work.	5.04 (4.88–5.20) p < .001 d = 0.80
18.4 Researchers should respond to and adjust their research according to feedback from a range of stakeholders.	5.21 (5.01–5.40) p < .001 d = 0.78	0.56 p < .001 d = 0.39
18.5 Researchers do respond to and adjust their research according to feedback from a range of stakeholders.	4.65 (4.48–4.82) p < .001 d = 0.51
20.1 Vision Mātauranga (Māori knowledge, people and resources) should influence the direction, execution and delivery of research in New Zealand*.	5.22 (5.03–5.41) p < .001 d = 0.83	1.08 p <.001 d = 0.76
20.2 Vision Mātauranga (Māori knowledge, people and resources) does influence the direction, execution and delivery of research in New Zealand*.	4.14 (3.95–4.32) p = .14ns d = 0.10
24.3 Researchers should consider and aim to address factors^† that might limit some groups' ability to participate in, and benefit from, research. (^†May include social, cultural, political, economic or other factors.)	5.75 (5.57–5.92) p < .001 d = 1.27	1.44 p <.001 d = 1.05
24.4 Researchers do consider and aim to address these limiting factors.	4.28 (4.10–4.47) p = .003 d = 0.2

^a The p-value and Cohen’s d (standardised mean difference) refer to the difference between the mean and the neutral mid-point of the scale. According to Cohen’s rules of thumb: Cohen’s d = 0.2 is a small effect, 0.5 is a medium effect, and above 0.8 is a large effect.

^b The p-value and Cohen’s d refer to the mean difference between what researchers should be doing and what researchers are perceived as doing.

Scale: 1.0 = strongly disagree to 7.0 = strongly agree; scale for question 20.1 and 20.2: 1 = Not at all to 7 = To a strong degree. NB for the Likert scales, a non-significant (ns) p-value result indicates that participants neither agreed nor disagreed with a statement.

Similar findings emerged for questions related to the dimensions of inclusion, responsiveness, social justice, and equity. Significantly different assessments were made by respondents regarding whether researchers should respond to, and adjust, their research according to feedback from a range of stakeholders, and whether they do so in current practices (see Table 1, 18.4–.5). The standardised mean difference between these assessments was a small effect size (Cohen’s d = 0.39).

Participants responded that the influence of Vision Mātauranga on the direction, execution, and delivery of research in Aotearoa should be stronger than it is at present (see Table 1, 20.1–.2); the standardised mean difference for this was a medium effect size (Cohen’s d = 0.76). It should however be noted that the mean scores for ‘researchers should respond to and adjust their research according to stakeholder feedback’ (M = 5.21) and ‘Vision Mātauranga should influence research in Aotearoa’ (M = 5.22), although affirmative, are lower than the other ‘should’ RI dimensional aspects. This indicates that respondents did not consider these aspects/dimensions of responsible research practice to be as important as anticipation and reflexivity (see Table 1, 12.1-2, 13.1-2, 14.1-2). Nevertheless, respondents also indicated that while the interests of all New Zealanders’ are not currently fairly represented (see Figure 3, mean score of 3.39), researchers should consider and aim to address the factors that might limit some groups’ ability to participate in, and benefit from, research more than they currently do (see Table 1, 24.3-4 significant mean difference between should and do = 1.44 and a large effect size between these assessments, Cohen’s d = 1.05).

The role of research organisations and funding agencies

To better understand structural and institutional factors that might influence views of current and envisioned states of the science-society relationship, survey questions also asked participants to assess the role of research organisations and funding agencies in terms of supporting or inhibiting specific practices. Across the considered dimensions, only slight differences emerged between respondents’ assessments of the role of research organisations and fundings agency. Aggregated means scores of these assessments are outlined in Figure 4, with individual scores listed in Table 2. On average, research organisations and funding agencies were seen as neither strongly supporting nor inhibiting practices associated with responsible research and innovation. However, some differences can be observed between the dimensions covered by the thematic sets of survey questions.

Figure 4. Survey respondents’ aggregated assessments of whether New Zealand research organisations and funding agencies support researchers in some aspects related to responsible research and innovation (Scale: 1.0 = strongly disagree to 7.0 = strongly agree).

Table 2. Survey respondents’ assessment of whether Aotearoa research organisations and funding agencies support researchers in some aspects related to responsible research and innovation.

Question	Mean (95% CI) p-value^a Cohen’s d^a	Mean Diff (research organisation – funding agency) p-value^b Cohen’s d^b
12.3 The way New Zealand’s research organisations operate supports researchers to consider the potential outcomes of their work (both positive and negative).	4.50 (4.31–4.69) p < .001 d = 0.33	−0.01 p = .87ns d = −0.01
12.4 The way New Zealand’s research funding agencies operate supports researchers to consider the potential outcomes of their work (both positive and negative).	4.51 (4.31–4.71) p < .001 d = 0.33
13.3 The way New Zealand’s research organisations operate supports researchers to consider the motivations and assumptions underlying their work.	4.25 (4.05–4.45) p = .013 d = 0.16	0.05 p = .55ns d = 0.04
13.4 The way New Zealand’s research funding agencies operate supports researchers to consider the motivations and assumptions underlying their work.	4.20 (3.99–4.40) p = .57ns d = 0.12
14.3 The way New Zealand’s research organisations operate supports researchers to consider potential limitations and blind spots in their work.	4.32 (4.14 - 4.50) p < .001 d = 0.22	0.11 p =  = .17ns d = 0.07
14.4 The way New Zealand’s research funding agencies operate supports researchers to consider potential limitations and blind spots in their work.	4.21 (4.01–4.41) p = .042 d = 0.13
18.2 The way New Zealand’s research organisations operate supports meaningful engagement between researchers and the public.	3.65 (3.45–3.84) p < .001 d = −0.24	0.03 p = .71ns d = 0.01
18.3 The way New Zealand’s research funding agencies operate supports meaningful engagement between researchers and the public.	3.63 (3.42–3.84) p < .001 d = −0.23
22.4 The way New Zealand’s research organisations operate encourages openness and transparency from our researchers.	3.82 (3.62 – 4.02) p = .071 ns d = −0.12	0.10 p = .49ns d = 0.06
22.5 The way New Zealand’s research funding agencies operate encourages openness and transparency from our researchers.	3.72 (3.50–3.94) p = .013 d = −0.16
24.5 The way New Zealand’s research organisations operate enables researchers to address these limiting factors*. (*see Table 1 24.3 for referenced factors)	3.91 (3.71–4.12) p = .404 ns d = −0.06	0.13 p = .22ns d = 0.09
24.6 The way New Zealand’s research funding agencies operate enables researchers to address these limiting factors.	3.78 (3.56–3.99) p = .038 d = −0.14

^a The p-value and Cohen’s d (standardised mean difference) refer to the difference between the mean and the neutral mid-point of the scale. According to Cohen’s rules of thumb: Cohen’s d = 0.2 is a small effect, 0.5 is a medium effect, and above 0.8 is a large effect.

^b the p-value and Cohen’s d refer to the mean difference between research organisations and funding agencies.

Scale: 1.0 = strongly disagree to 7.0 = strongly agree). NB For the Likert scales, a non-significant (ns) p-value result indicates that participants neither agreed nor disagreed with a statement.

Similar to respondents’ overall assessment of the current state of research and innovation in Aotearoa (see section 3.1.2), aspects of practice aligned to the dimensions of anticipation and reflexivity were scored significantly higher than those aligned to the other dimensions.¹ On average, participants slightly agreed that research organisations and funding agencies support the consideration of research outcomes (aggregated mean score of 4.50, also Table 2 12.3-.4), underlying motivations or assumptions (aggregated mean score of 4.23, also Table 2 13.3-.4), as well as limitations and blind spots (aggregated mean score of 4.26, see Table 2 14.3-.4). These scores are all significantly higher that the neutral mid-point of the scale indicating that respondents slightly agreed with the statements. However, Cohen’s d for all of these statements ranges from .12 to .33 indicating only small effect size differences from neutral, so agreement is weak. Several written comments added to this thematic section of the survey indicate that some participants thought the current funding and research system encourages an emphasis on positive outcomes, and consequently discouraged consideration of more negative aspects. For instance, one respondent noted:

Researchers are encouraged to emphasise the positive in grant applications and reports to government. A reasonable and balanced approach does not sell science, but it should.

Other comments suggest that some participants deemed oversight from their institution or external bodies (e.g. human ethics committees) to make their work more difficult, with the implication that it might be better to leave consideration of research outcomes, assumptions, and limitations to researchers.

Following the pattern in the overall assessment of the current state of research and innovation, respondents on average scored questions related to the dimensions of inclusion, responsiveness, social justice, and equity significantly lower than the anticipation and reflexivity dimensions, with aggregated mean scores below the scale neutral point of 4 indicating that respondents generally disagreed that research and funding organisations supported these activities (see Figure 4). While participants slightly agreed that researchers do consider and aim to address factors that might limit some groups’ ability to participate in, and benefit from, research (see section 3.1.2), they neither agreed nor disagreed that research organisations enable these considerations (mean score 3.91) and slightly disagreed that funding agencies do so (mean score 3.78, see Table 2, 24.5-.6). Similarly, participants slightly disagreed that the operation of funding agencies encourages openness and transparency (mean score 3.72), neither agreeing nor disagreeing to the same statements regarding research organisations (mean score 3.82, see Table 2, 22.4-.5). Again, Cohen’s d for all these statements ranged between −0.24 and −0.06 indicating only small effect size differences from neutral for these question items.

The biggest room for improvement was seen in support for meaningful engagements with the public. While participants slightly agreed that engagements between researchers and the public are meaningful in terms of being inclusive, timely and genuine (see section 3.1.2), they were somewhat more critical of research organisations and funding agencies in this regard. On average, respondents slightly disagreed that research organisations and funding agencies support engagements with the public (mean scores of 3.65 and 3.63, see Table 1, 18.2-.3). Written comments made in this thematic section identified several barriers to engagement, including lack of funding, resources, and time. For instance, one participant noted:

Very little money or opportunity is made available to researchers to consult and interact with the public or interest groups. This is usually a cost to the researcher which, with the limited amount of research funding available in NZ, is hard to do.

These written comments demonstrate that complementary qualitative insights are required to better interpreted and contextualise the presented quantitative survey findings.

Challenges for responsible practices

Given the gap between what researchers attest should be done in order to enact responsible practices within the RSI system and what they perceive is actually done, the question arises why do researchers not do what they believe they should do? The findings from the 24 qualitative interviews undertaken as part of this project shed some light on this conundrum. Indeed, while several of the interviewees spoke of researchers generally having good intentions, when asked about the challenges they perceived researchers face around making research within the Aotearoa RSI system responsible, they identified a wide range of factors that deterred or inhibited the enactment of responsible research at the individual and system level.

Individual challenges

At the individual level, interviewees spoke of responsible practices being piecemeal, a situation that was frequently attributed to a general lack of training, guidance, and ultimately requirement to engage meaningfully in responsible practices. Furthermore, alongside this lack of requirement sat what interviewees identified as attitudinal challenges amongst researchers. These challenges were perceived to manifest in some researchers as a lack of patience for addressing the questions and issues that engaging with aspects associated with RI brings, and for others as a reluctance to sacrifice autonomy over their research.

I think an important factor is that it is not easy, you do get the gnarly questions as a researcher and you have to address those and not everybody has the patience to work through that, they just want to get on and do a trial or whatever, just do their thing. (Industry representative 1)

I think all researchers would like to think they are responsible, and they’d like to do the right thing, but when it actually comes down to saying how it might impact what they can and can’t do, that’s when it’s going to start getting difficult. (CRI representative 1)

The interviewees also spoke of the epistemic cultural challenges that face some researchers within the biophysical sciences who view the questions and issues raised by thinking more broadly about aspects related to responsible research and innovation, and the practices required to engage with these aspects, as either irrelevant to, or falling outside of, their field of science.

I don’t think that there’s enough consultation and wider discussion and I still think that we’ve got the challenge of scientists being far too siloed in their own discipline and not thinking outside the box. (Industry representative 2)

the way they were thinking about their project meant they compartmentalised the engagement side away from their sciencey, in the laboratory, sort of side. (Government representative 1)

Such individual level challenges are exacerbated when it comes to engaging meaningfully with Māori. Indeed, many of the interviewees spoke of researchers lacking familiarity with, and understanding of, te ao Māori (the Māori worldview), which provides knowledge and insights to inform how research with Māori should be undertaken responsibly. Furthermore, many researchers were deemed to lack an understanding of, and familiarity with, mātauranga Māori (Māori knowledge), as a knowledge system equal in value to Western scientific knowledge.

As with other dimensions of responsible research practice, this lack of familiarity with te ao Māori and mātauranga Māori was attributed to a lack of training, guidance, and requirement to engage meaningfully with Māori stakeholders. A situation which, some interviewees asserted, results in attempts at engagement with Māori being tokenistic and often occurring towards the end of research projects.

I … get emails every day about how we [as Māori stakeholders] can be involved in a project, … [saying] ‘oh, the project is coming to an end, and we’d like you to be a part of it’, [and my response is] ‘ok, thank you but no thank you. You’ve obviously continued on your project without us and to have a true relationship we would have preferred to be [involved] at the beginning’. (Māori Agribusiness representative 1)

The need for early, meaningful, and ongoing engagement, which would underpin such true relationships, requires researchers to understand the value of responsible research practices, and to be adequately trained and supported to implement them. The dearth of such support was seen by interviewees to be a key challenge researchers face.

Systemic challenges

Indeed, at the system level, the interviewees spoke of researchers lacking the necessary time and funding to engage meaningfully in RI practices, whilst working within an RSI system that too often focuses on research outputs rather than the way in which research is undertaken.

because everybody is busy and feeling stressed and needs to chase more money and needs to write papers … One of the first things to go is acting in an altruistic fashion. (CRI representative 2)

This constraint within the RSI system was also identified by the interviewees in the perceived lack of flexibility around research plans, key performance indicators, and deadlines. These challenges are exacerbated by the fragmentation seen within the Aotearoa funding system, which makes engaging meaningfully in reflexivity and responsiveness challenging.

This isn’t just a blame of the scientist … because I think the funding system probably generates some perverse behaviours. (Government representative 2)

Interviewees also spoke of the systemic challenges that come with researchers facing multiple, and at times competing, responsibilities to colleagues, employers, funders, stakeholders, and society at large, not to mention their obligations under Te Tiriti o Waitangi. The need to honour Te Tiriti and conduct culturally safe and responsible research is an important layer to doing research responsibly within the Aotearoa context. As with other RI dimensions, applying a Te Tiriti-led approach to research not only requires specific knowledge and skills at the individual level, but it also requires system-wide support. However, at the system level, interviewees spoke of research projects suffering from a lack of time and resourcing to both engage meaningfully with Māori and build the long-term relationships required to do so. This issue was seen to be exacerbated by a severe lack of capacity within the science system both in terms of Māori researchers, and in terms of culturally competent researchers.

they identified early on that [bringing a Māori voice into the project] was something that they really needed to do but there wasn’t enough … resource or time or anything to think about it … that is part of the challenges of the system … there aren’t enough Māori in those positions. (Government representative 1)

The lack of recognition and knowledge of te ao Māori, mātauranga Māori, and Māori research approaches across the science system, the inadequate attempts at engaging meaningfully with Māori, and the dearth of Māori researchers within the science system can all be seen as systemic challenges resulting from the colonial legacy in Aotearoa. Thus, while some of the reasons for the significant discrepancy between what researchers believe should be done and what they perceive is actually being doing lie at the individual level, others are systemic. Enactment of RI therefore requires attention to both sets of challenges with particular attention paid to those that inhibit meaningful engagement with Māori and enactment of Te Tiriti obligations.

Discussion

Findings from this project suggest that, while respondents’ understandings of responsibility in the context of research and innovation differed, there exists a desire and good intentions among many researchers and innovators in Aotearoa to conduct their work responsibly and in a manner that aligns with at least some of the dimensions underlying RI approaches. This corresponds to earlier studies of the science-society relationship in Aotearoa (e.g. Sommer 2010; Small 2011, 2013). However, the present research also highlights that there is a significant discrepancy between what researchers believe they should do and what they perceive researchers actually do when it comes to enacting such responsible research practices. While some of the reasons for this discrepancy lie at the individual level, others are systemic.

Key areas identified through this research for improvement in terms of responsible research and innovation practice include the accessibility of scientific research; equitability of research and innovation outcomes; a greater focus on research processes, outcomes, and social impacts and not just outputs and economic impacts; and meaningful engagement with Māori (including cultural competencies, dedicated resources, and time). Responsibility for encouraging and enacting improvements in these areas of research practice were not just seen to sit with individual researchers and innovators, but also with research organisations and government agencies. Indeed, the qualitative data indicates that current research practices, whether responsible or not, are partly determined by what researchers can feasibly do against a background of structural challenges. That research organisations and funding agencies were seen to operate in ways which neither strongly supported nor inhibited researchers from conducting their research responsibly arguably forms part of the problem, given the noted/identified support and training needs of researchers.

While the survey results suggest that research organisations and funding agencies do provide some support around enacting practices that align with the dimensions of RI, this support is uneven. Indeed, while there was slight agreement that research organisations and funding agencies support anticipatory and reflexive practices, there was slight disagreement that research organisations and funding agencies support practices that align with the other dimensions explored (inclusion and responsiveness, social equity and justice, openness and transparency), and the difference between these perceptions of support was statistically significant. This suggests that participants both perceive anticipation and reflexivity to be the most important dimensions of RI practice, and perceive that these dimensions are given the greatest support within the RSI system.

Enabling more responsible research and innovation practices, therefore, may require addressing these perceived imbalances around, and challenges to, their enactment that exist at both the individual and at the system level. To address the latter, research organisations, funders, regulators, and government agencies could support responsible conduct through more coherent visions, overarching strategies, and flexible funding mechanisms that require and/or incentivise RI-aligned practices. Central to such requirements could be a focus on tackling the individual and system-level challenges that currently impede meaningful Māori inclusion and engagement in research and innovation. There is growing momentum towards addressing these challenges supported by the likes of Te Ara Paerangi (MBIE 2021a, 2021b) and the guide to good Vision Mātauranga practice compiled by Rauika Māngai (Rauika 2020). As the presented findings suggest, however, there is still considerable room for improvement.

Whether RI is a useful concept and approach to support such improvement in the responsible practice of research and innovation in Aotearoa is a question for further research. The expanded version of RI applied here was used as the basis for assessing the current state of research practice in Aotearoa rather than a guide for future practice. There are plenty of Kaupapa Māori-led initiatives that can direct those within the Aotearoa RSI system towards more responsible practices (Wilcox et al. 2008; Hikuroa et al. 2011; Hudson et al. 2016; Collier-Robinson et al. 2019; EPA 2020; Hudson et al. 2020). And while many RI practices align well with those supported by such initiatives and guidance documents, it remains to be investigated whether the tools, approaches, and thinking that RI promotes are useful for guiding Aotearoa towards more responsible research and innovation and thus better science-society relationships. Indeed, RI has been used internationally by some organisations and practitioners to develop tools that guide responsible research and innovation practices (e.g. Aymerich-Franch and Fosch-Villaronga 2020). However, the concept’s suitability for the Aotearoa context would need to be thoroughly assessed. Such work is emerging, but more conceptual and applied research is required to determine RI’s value proposition for the Aotearoa RSI system.

Conclusion

While previous research suggests that there is a growing recognition among researchers and innovators of the importance of conducting their work responsibly, the current study highlights that understandings of what responsibility means in this context differ considerably. Furthermore, the findings of the current study suggest that there is a bias within the individual enactment, and the systemic support, of responsible research and innovation practices towards those that align with the RI dimensions of anticipation and reflexivity. Less attention is perceived to be given to practices that align with the dimensions of inclusion and responsiveness, social equity and justice, and openness and transparency. This indicates that there is considerable room for improvement in the enactment and support of many responsible research and innovation practices at both the individual and system levels. This conclusion is supported by the gaps identified between what respondents thought researchers and innovators should do in terms of enacting responsible practices, and what they perceive is actually done. While this study provides insights into researchers’ and innovators’ perspectives on what makes for responsible practice, and the current state of responsible research and innovation in Aotearoa, an exploration of wider public perspectives would be beneficial to gain a fuller picture of these issues. Likewise, further research exploring the discrepancies between what researchers perceive they should do and what they actually do is warranted in order to better understand the barriers and enablers to adopting more responsible research and innovation practices. Furthermore, for RI scholarship to support this work in Aotearoa, additional effort is needed to adapt the approach for the Aotearoa context so that it aligns with and supports researchers and innovators commitments to Te Tiriti, their requirements under Vision Mātauranga, and the guidance emerging out of the Te Ara Paerangi review of the science system.

Acknowledgments

Firstly, we would like to thank all those who participated in our interviews and survey for sharing their time, experience, and insights with us. We would also like to thank Professor Phil Macnaghten, of Wageningen University and Research Centre for his support and guidance, and the two reviewers who provided us with helpful suggestions to strengthen our manuscript.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

Due to the nature of this research, participants of this study did not agree for their data to be shared publicly, so supporting data is not available.

Additional information

Funding

This study was funded by the Ministry of Business, Innovation and Employment (MBIE) through Strategic Science Investment Fund (SSIF) support of the Responsible Transitions programme at AgResearch (Lincoln, New Zealand).

Notes

1 The result of a 1-tailed t-test between the lowest scoring anticipation/reflexivity support statement (13.4) and the highest scoring support statement for any other dimension (24.5) results in p value of .001.