Defining success in community-university partnerships: lessons learned from Flint

ABSTRACT

The success of responsible research and innovation (RRI) work is as much about the process of partnership as it is about the products and outcomes. In this paper, we present lessons learned from the first three years of a participatory modeling (PM) research project based on RRI principles and focused on the transformation of the food system in Flint, Michigan through identification of leverage points. Participatory modeling is a type of community engaged research that seeks to build representations of a system collaboratively between researchers, decision-makers, and community members. We discuss the challenges, opportunities, and lessons learned from the Flint Leverage Points Project (FLPP) using the four ‘Ps’ framework–purpose, processes, partnerships and products. We argue a carefully designed participatory modeling process can serve to build lasting partnerships across community-university boundaries.

KEYWORDS:

• Participatory modeling
• urban food systems
• systems modeling
• participatory research

Introduction

Responsible Research and Innovation (RRI) and Community Engaged Research (CEnR) are two approaches to research that underscore the importance of collaboration and equity. RRI is an approach that anticipates and assesses potential implications and societal expectations with regard to research and innovation, with the aim to foster the design of inclusive and sustainable research and innovation. RRI includes a cast of actors (researchers, community, policy makers, organizations/institutions, businesses and government) who collaborate during the research and innovation process (Owen, Macnaghten, and Stilgoe 2012). CEnR is an approach to research that supports various points of engaging community and other stakeholders along a continuum of engagement, with Community Based Participatory Research (CBPR) being one point along that continuum. In recent years, CEnR has gained more visibility in the literature with a focus on community-academic partnerships. Both RRI and CEnR strongly encourage the input of community and other stakeholders in research and promote inclusive processes. These inclusive approaches are built off of two predecessor models: (1) the International Association of Public Participation continuum which uses five domains of participation; and (2) Arnsteins’s ladder of participation which posits eight domains of civic participation (Arnstein 1969). These models served as forerunners for engagement continuums and have been adapted for the fields of science and research.

 In this paper, we present lessons learned from the first three years of a community engaged research project based on RRI principles and focused on the transformation of the food system in Flint, Michigan through identification of leverage points. In particular, we focus on the RRI principles of co-design and co-innovation for finding solutions to socially relevant problems. This project, titled the Flint Leverage Points Project (FLPP), uses a participatory modeling approach to build models of the food system in a collaborative manner, using co-designed innovation. Participatory modeling (PM) is a methodology for constructing, parameterizing, running, and validating a model as a collaborative effort between modelers and stakeholders (Voinov and Bousquet 2010). Researchers and community partners building a model together represents an opportunity to foster a robust discussion using a systems approach in which all parties are invited to lay out and question their assumptions about the system in question (Vennix 1996). The nature of this collaboration varies depending on the context of the modeling project, but PM can be used for both community engaged and community-based participatory research projects in a manner consistent with the principles of RRI (Hovmand 2014; Van den Belt 2004; Voinov and Bousquet 2010). A key part of incorporating these principles is deliberate design of the PM project, and reflection on the part of all project participants around how the design and implementation of the project met the objectives of both researchers and community partners. As an attempt to encourage this type of reflection, as well as more consistent and standardized reporting on PM projects, researchers have developed a ‘4P framework’ (Gray et al. 2018) which reports on the purpose, process, partnerships, and products for a given PM exercise. Using that framework, we demonstrate how PM may be used to support the RRI principles of co-designed innovation, tackling relevant social problems, and ethical considerations.

The Flint Leverage Points Project

Overview of the FLPP

The Flint Leverage Points Project has been co-designed by community leaders, from design to process identification to implementation. The original idea for the project came from the Food Navigator in charge of programming around the Flint food system at the Community Foundation of Greater Flint (CFGF). The person holding this position was asked to foster connections and design holistic programming to address the food needs of Flint citizens, across government, nonprofit, faith-based, and retail sectors. In this case, the Food Navigator had observed that despite multiple interventions in the Flint food system, some of which were successful by their own objectives, the food system overall remained in a sub-optimal state of poor access to healthy food and racial inequity. From the beginning the project adopted an Academic Principal Investigator and Community Principal Investigator model. This shared model is designed to provide equity, transparency and trust in all decision making related to the study. In addition, a Community Consultative Panel (CCP), consisting of Flint stakeholders with key roles in the food system, was formed to ensure that the community has equitable input in the project. The members of the panel are compensated for their time, through funds set aside for that purpose in the research grant. They were selected by the Food Navigator to represent different sectors of the food system and of the Flint community, including growers, faith-based and non-profit emergency food providers, members of city government, and representatives of Flint’s Black and Latinx communities (see Appendix for original list of panel members). The proposed role for this panel involved them meeting with the research team quarterly.

Flint, Michigan is a majority-minority (predominately Black) city. Historically, unethical practices in research have negatively impacted communities of color and their willingness to engage in research. In research studies from the Tuskegee Syphilis Study to the harvesting of Henrietta Lacks’ genetic material without her consent, researchers’ lack of ethical practices have done damage to the ways Black Americans see research and their decisions to participate (Brandon, Isaac, and LaVeist 2005). More recently, the Flint community was failed by every level of government resulting in the Flint Water Crisis. The disregard of community voice, elimination of democracy and the blatant lies and lack of accountability on the part of the state government proved to be great ethical concerns for the residents of Flint (Pauli 2019). Thus, this experience further broadened the distrust Black residents of Flint have historically had with institutions, and more specifically their trust in academic research. Thus, ethical principles needed to engage with the Flint community include trust, transparency, equity, value of community voice and respect.

Understanding the distrust, lack of transparency and devaluation of community voice during the Flint Water Crisis, the FLPP took an extra measure to gain the community’s trust by submitting the proposed research to a community Institutional Review Board (IRB) located at Hurley Medical Center. Although university-based IRBs do effectively assess individual-level outcomes including risks, benefits, consent, and protections, the cumulative impact of the research process on studied communities, and in particular on communities of color, is not an ethical priority for these IRBs (Shore et al. 2011). Research that is not reviewed through a community lens has the potential to create harm in that inappropriate generalizations can misrepresent perspectives of the community and generate misleading conclusions (McGrath et al. 2009).

The community-based review board in Flint arose from the need expressed by the broader Flint community to further ethically safeguard residents who participate in scientific research at the local level. In 2009, author (KK) founded and developed Community-Based Organizations Partners (CBOP)’s Community Ethics Review Board (CERB) in Flint (Key 2017). The CBOP-CERB is comprised of community members and supporters that provide ethical reviews of proposed research, identify community advisory board members for researchers (if needed), and work with researchers to ensure community-level protections and mutual benefit. Therefore, CERB serves the interests of both the Flint community (by protecting them from unethical research conduct) and researchers interested in Flint (by facilitating an ethical and smooth access to the community). Unlike the traditional IRB, which focuses solely on the protection of the researcher and the subject, the CERB (1) ensures community approval; (2) provides a mechanism for community to give input on proposed research; and (3) ensures community-level benefits and protections. In addition to the academic Institutional Review Board (IRB) needed to conduct research, the FLPP leadership also went through this community ethics review process.

Partnerships

From the project inception, relationships form the core of the layers of partnerships, and are the glue that sustains this complex work. The project structure described above and roles/responsibilities evolved over time as greater clarity was required to carry out project tasks. The project is unique in two ways. The vision and commitment to form a team to model complex community issues preceded this particular project, and relationships existed with all of those invited to engage, although the relationships lived at many different tables. This foundation grounded in relationships has been key to authentic engagement of community members to inform research and responding to results, as well as to sustaining participatory decision-making, and group priority setting throughout the project.

The FLPP owes its origin to the vision of the Food Navigator at the Community Foundation for Greater Flint, a boundary spanner who deeply understood both the challenges facing the Flint food system and the role research (particularly, systems modeling) could play in addressing those challenges (Walker and Salt 2006). Boundary spanners are individuals who are essential not only for facilitating communication across research-community boundaries, but for creating the driving motivation behind the project, as they are champions for both community needs and for the value of research in addressing those needs. Halfway through the project, this individual left to take another position, which necessitated the project seeking out other boundary spanners who were firmly planted in the Flint community and versed in the process of academic research. We were fortunate to find several such individuals in author KK, a community-based health researcher; in a newly hired program manager for the Community Foundation; and in process monitor RW, who had worked on similar projects in Detroit. This period of transition between boundary spanners was a challenging time for the project, marked by miscommunication and confusion on the part of community partners around the goals and objectives of the project, and lack of cohesion of the research team. While the key boundary spanner leaving the project was unforeseen, this point to the importance of building some redundancy into the roles of key project personnel, and the list of key project roles must include Boundary Spanner.

Process

The heart of engagement revolved around the Community Consultative Panel (CCP). The ebb and flow of planning, conducting, analyzing and reviewing research findings and decision-making, all happened in monthly cycles with CCP convenings occurring quarterly initially and transitioning to once per month. CCP members could also elect to attend research team and subteam meetings to engage more deeply. Periodically they also met with the evaluation team. All engagement opportunities were made visible and accessible to the CCP through a process innovation called ‘Pathways to Participation’ (P2P) developed by the project process monitor. The master pathway coordinates all project activities, identified through monitoring conversations with team leads who determine which activities are a fit for CCP participation (creating research specific pathways). Deliberately creating visible, shareable pathways makes it possible to communicate current and upcoming opportunities to CCP members, from which they can select activities they have capacity, interest, and availability to engage in (creating an individual P2P). This approach puts community members in control of the level of their engagement, establishes reality-based expectations, and builds in accountability.

Evaluation components played a prominent role in project design. The project evaluation is participatory (Cousins and Elizabeth 1998) and utilization-focused (Patton 2012) in orientation. Participatory evaluation involves collaboration between evaluators and stakeholders in the design, implementation and use of evaluations, from establishing key evaluation questions to interpreting results and using findings. Utilization-focused evaluations are designed to facilitate the use of findings by primary intended users for a variety of uses, including modifying programs to improve their performance to making decisions about whether to continue, expand or terminate programs. The evaluation team ensured that the evaluation of the project would be participatory and utilization-focused by including three key stakeholder groups – members of the research team, community leads, and members of the CCP – in decisions about priority evaluation questions, methods to answer those questions, and the meaning and potential use of findings. The primary mechanisms to promote participation and utilization were the establishment of an Evaluation Advisory Group and discussions about every stage of the evaluation and its findings between this group, the research team and the CCP. To facilitate use, the evaluation team presented findings as soon as they became available in easily digestible formats.

The project is somewhat atypical in that it developed an innovative role for a process monitor, in addition to an evaluator. Because of the overlapping, complementary nature of these roles, the evaluator and process monitor worked closely together, with the process monitor serving as a member of the evaluation team while also maintaining independent activities. Whereas the evaluator saw his responsibility as answering stakeholder-identified evaluation questions, the process monitor assumed responsibility for designing and facilitating interventions to strengthen project integration across academic disciplines, as well as for deepening participatory interactions between the community members on the project team and the researchers.

From the research process side of the project, we were faced with the challenge of focusing in on the key ‘levers of change’ for Flint in the context of a complex food system with a rich history tied to multiple other issues, such as the economic downturn experienced by the city and the state of Michigan; the ongoing Flint Water Crisis impacts on infrastructure and public health; local, state and national political environments; etc. We perceived that building a quantitative model representing the entire food system and the factors impacting it would be both extremely difficult and unhelpful in providing clarity and focus to the key research questions. We therefore designed a ‘funnel’ approach, starting with a resilience assessment through which we collected a broad range of qualitative data from across the sectors of the food system through interviews, focus groups, participant observation, and community histories, and used that data to focus in on key problems and feedback dynamics for further quantitative modeling. As the quantitative models are constructed, we return to the broader qualitative data and turn to the community to provide context, and to iteratively check the assumptions underlying the models.

Any innovation and products produced by this study are co-designed by the entire team (academics and community). The purpose of the FLPP represents both RRI principles of co-designed innovation and tackling relevant social problems, described in greater detail below.

Overview of Flint and its food system

Situated in southeastern Michigan, the city of Flint grew in prominence as the birthplace of General Motors. Flint was the site of the first sit-down strike in 1936 when Flint’s auto workers galvanized around promoting workers’ rights and developing a strong union. Community organizing remains central to Flint residents’ identity today. However, as manufacturing jobs began to move outside the US, several major industrial plants closed in Flint. Once a manufacturing mecca, over time, Flint experienced a tremendous economic decline and industry de-investment. As a result, Flint’s population dropped from a peak of 200,000 residents in the 1970s to fewer than 100,000 residents in 2019. Today, the median household income in Flint is $28,834 and 62.4 percent of children in the city live in poverty. At the same time, racist housing policies and redlining contributed to making Flint the 14th most segregated city in the United States (US Census Bureau 2021). For the State of Michigan, the Robert Wood Johnson Foundation County Rankings ranked Genesee County, which contains Flint, 76th of 82 in health factors (health behaviors, clinical care, social and economic factors, physical environment); and 81st of 82 for overall health outcomes. More devastating is that within the city of Flint, there are zip codes with an average life expectancy of under 65 years compared to the nation’s average of 78.5 years in 2018 (US Census Bureau 2018).

Recently, Flint drew national attention for the Flint Water Crisis which began April 25, 2014. This city-wide lead poisoning resulted from a switch from Detroit City water to collection from the Flint River without the proper corrosive control measures being implemented. A cost-saving decision made by an Emergency Manager appointed by Governor Rick Snyder, the water delivered to city residents was highly contaminated with lead and bacteria including legionella (Sadler et al. 2019). Tireless work by community activists, religious leaders, and citizen scientists revealed mismanagement and deceit that resulted in lead poisoning of Flint residents, including elevated blood lead levels in the city’s children, with disadvantaged neighborhoods suffering the greatest levels of lead exposure (Hanna-Attisha et al. 2016). While approximately 10,000 lead pipes have been replaced in the city, concerns about the safety of the water remain, and residents exposed to lead will face potential lifelong health complications (Diaz 2022).

 The Flint Water Crisis shapes the food system in Flint today, as public health programs promote the consumption of lead-mitigating foods and the management of diets for health (Mahaffey 1990). Along with disinvestment and depopulation, all of the major grocery store chains closed their locations in the city by the 2010s.¹ As a result, access to healthy, fresh food became a significant challenge for residents, especially for residents who rely on public transportation. Furthermore, recent research in Flint shows that neighborhoods that are predominantly Black or have low socio-economic status also have less access to fresh fruits and vegetables (Shaver et al. 2018). Residents of these neighborhoods are more likely to access liquor or convenience stores nearby to meet food needs, rather than the grocery store (Shaver et al. 2018). Together these factors contribute to the patchwork of local, non-profit, and governmental programs aimed at increasing food access through supplemental food programs in Flint. Community organizations continue to drive efforts to improve food access, and innovative projects like the Flint FARMacy, expansions of Double-Up Food Bucks, and a fruit and vegetable prescription program for children illustrate efforts in innovation (e.g. Saxe-Custack et al. 2020). Nevertheless, this context exemplifies the need for substantial change to improve and integrate components of the food system for all residents, and is essential to the research framework and design.

Goals, objectives and expected outcome of project

The overall goal of the project as proposed was to identify ‘leverage points,’ or opportunities for intervention in the Flint food system, which would have the highest potential impact in shifting the system towards more healthy, equitable, and sustainable outcomes for Flint residents. We also wished to identify the organizations or individuals who would best be able to act on these leverage points. Proposed outcomes and outputs for the project, including products, were:

1. Identification of balancing feedback loops that keep the Flint food system in a sub-optimal state at multiple scales

2. Identification of trusted decision-makers who can disrupt these balancing feedback loops to tip the system into a trajectory of positive change

3. Identification of strategic interventions to improve health and nutritional outcomes in Flint, based on 1 and 2 above.

4. Collaboration that fosters learning within the Flint food system, and a platform for ongoing community dialogue

5. Systems-modeling training for community leaders and organizations that sustains research and learning over time

6. Publicly available dynamic simulation models which allow Flint food system actors to test potential interventions, identify leverage points, and explore a range of future scenarios

7. Publications, webinars and online tools that provide a research and engagement framework for extending the learning to other urban areas.

Progress of the Flint Leverage Points Project in the first 3 years

The overall research design began with the resilience assessment described below, consisting of qualitative and secondary data collection and analysis which then informed the semi-quantitative mental modeling and the quantitative system dynamics modeling (both approaches are described in more detail below). See Table 1 for a summary of project phases, methods and activities associated with each phase.

Table 1. Iterative FLPP research plan.

Phase	Timeframe	Method(s)
Resilience assessment	2018–2022	Timeline: informal data collection and conversations at community events and with CCP Stakeholder Mapping and Visioning: workshops and small group activities; 12 workshops N = 64 Q-Methodology: individual interview and activity; N = 25 Archival Research: US Census, demographic, government document research Synthesis and Community Feedback: from virtual and in-person events
Governance project	2018–2019	Individual Interviews: Examining efforts to organize the Flint food system (N = 26)
Household food security	2019–2022	Participant Observation: In-person and switch to online with COVID-19 Interviews: food insecure residents and food system actors providing COVID-19 food assistance Photo-Elicitation Project: residents with access to phone and computer; N = 16
Mental modeling	2020–2022	Interviews and mapping activities: with food system experts; focus on emergency, supplemental, and retail sectors; N = 51 Group model building: with food system experts; focus on emergency, supplemental, and retail sectors Additional interviews with focus on local food sector and racial equity through supplemental funding N = 25
System dynamics modeling	2021–2023	Develop model structure based on previous research Feedback and scenario development with community partners

Each phase has evaluation and process monitoring built in as critical on-going components of this project.

In the first three years of the project, we have focused on objectives 1 through 4, while also producing relevant scholarly products along the way, targeted to academic, community and policy audiences. Objectives 5 and 6 will be addressed in the final year of the project, as we build the quantitative system dynamics model. In giving an overview of project activities and outcomes so far, we describe what community engagement in a systems change project involving modeling tools can look like, drawing on participatory methods and literatures from diverse disciplines.

Resilience assessment

The resilience assessment was designed: (1) To look backwards to understand how the food system evolved to its current state with low food security; (2) to understand how the resilience of the system supports the continuation of the current (undesirable) state; and (3) to look forwards to characterize a more desirable and resilient food system. We worked with the CCP and other Flint food system actors to define key events, boundaries, and players in the food system, using timelining activities (Hodbod & Wentworth, 2022). To contribute to objective 1, the research team then identified indicators of resources (food security and funding from donor organizations) and connectivity (number of food pantries and collaboration between food system actors) within the Flint food system. Datasets of these indicators were created from interviews and secondary data and informed an adaptive cycle analysis (Hodbod and Wentworth 2022). The analysis outlined the history of the Flint food system (objective 1), but also contributed to objective 2 by identifying which resources and relationships had supported (or blocked) change to more food secure states through the last 70 years. An emerging theme was that city-scale economic and governance decisions (such as divestment by General Motors, implementation of an Emergency Manager) created a loss of autonomy which blocked innovations scaling from neighborhood food systems up to the city scale. Contributing to both objectives 1 and 2 we focused a lot of attention on the social structures within the food system. Our governance team explored previous (failed) attempts to move the food system out of this food insecure state through collective action. Understanding these previous attempts at collective action was essential to building rigorous community-engaged research that accounted for previous work, which is too often overlooked leading to redundancy, ‘research fatigue,’ and further mistrust among community members (Warren et al. 2018). To position this analysis within the present day, participatory stakeholder mapping was carried out to understand who the key stakeholders currently are within the food system and how they are connected, from the perspective of multiple key stakeholder groups identified by the CCP.

The stakeholder mapping analysis helps to identify key actors who are well-positioned now to initiate system changes. To inform objective 3, we hosted visioning workshops with these key stakeholders and a range of community members. We analyzed the transcripts inductively to identify values as priorities for the food system for different stakeholder groups, finding healthy, fresh and natural foods, and affordability to be the common priorities which then informed the subsequent modeling (Belisle-Toler, Hodbod, and Wentworth 2021). After further ranking of these values as priorities with community members and key stakeholders (a co-design approach), we identified three shared visions for the food system, reaffirming that there is not a single experience of food insecurity and thus not a singular vision for its desirable food system. However, these visions (a food system with healthy foods that residents are educated about and willing to travel for, a food system with convenient, fresh food options for those who cannot travel, and a food system that maintains community’s food traditions) can be achieved simultaneously, as explored in the modeling.

The resilience assessment has created methodological innovations for researchers and decision-makers to use, both in the overall adaptation of resilience assessment for an urban food system context, and with specific tools within it, particularly the visioning protocol. These products have been shared through a coupled approach of peer-reviewed journal articles open access to ensure they are widely available; e.g. (Belisle-Toler, Hodbod, and Wentworth 2021) and book chapters (Hodbod and Wentworth 2022) with corresponding briefing notes and outreach tools (e.g. an interactive timeline) published on the FLPP website,² and presentations at conferences and community events, corresponding to objective 7, distributing learning from the project to academic, policy, and community audiences. We will also publish a report outlining all elements of the resilience assessment.

Household food security

The aim of this component of the FLPP is to describe the experiences of food security with a primary focus on Flint residents and non-food system experts. While the resilience assessment included perspectives from residents, we also held workshops with community leaders, philanthropic and government organizations. Here we aimed to focus on resident voices and better understand how people use the food system in an effort to identify barriers. Drawing on some of the qualitative data from the Resilience Assessment, the Household Food Security component allowed for a more detailed investigation of some of the themes that emerged in our earlier work, thus supporting the iterative process of the FLPP. The broad research questions for this component of the project are listed below.

1. How do Flint residents navigate the current food system?

2. What barriers do Flint residents experience when attempting to access healthy food?

3. How do Flint residents access and interpret nutritional information?

4. What is the role of kin and neighborhood networks in accessing food?

To answer these questions, qualitative data was collected among subsets of the Flint community to represent a range of perspectives and experiences. Three primary methods were employed for this research: participant observation, individual interviews and a photo-elicitation project. Participant observation was used to generate observational data on how community members utilize food distribution services and manage food security. Based on our learning from participant observation, interview questions were developed for individual interviews. After a shift to remote data collection due to COVID-19, participant observation continued during virtual community meetings, and interview questions were adjusted to include information on the pandemic. Finally, since the photo-elicitation project had not started before the pandemic, this project was redesigned to ask participants how their experiences in the food system had changed as a result of COVID-19. This data was essential to our understanding of Flint residents’ experiences navigating the food system through the pandemic, addressing a relevant social concern in real time (objectives 1 and 3).

Fuzzy cognitive mapping

After the values and relevant stakeholders were identified in the predominantly qualitative process of the resilience assessment, we transitioned to Fuzzy Cognitive Mapping (Gray, Zanre, and Gray 2014). The purpose of this phase of the project was to begin to understand how the prioritized values (e.g. increasing access to healthy and natural foods and increasing affordability) related to the larger dynamics of the urban food system, addressing objectives 1 and 3. Therefore we used a Fuzzy Cognitive Mapping (FCM) interview protocol which was developed to engage relevant stakeholders (N = 51) to better understand how the emergency, supplemental, and retail food sectors interact with ranked values. Participants in this phase of the research were identified in the stakeholder mapping, with community partners, the CCP, and through snowball sampling. Since FCM is a semi-quantitative approach, interviews were conducted on the individual level in an attempt to gain candid and detailed data where participants defined the other factors related to the overall food system and directional and weighted causal relationships between them (Figure 1). The purpose of this step in the research was to constrain and define the system-level boundaries in a way that the research team could understand major dynamics of the system. In addition to the individual FCMs, participants also identified potential leverage points (points in the system in which to intervene to shift system outcomes), and their perceived dynamics as they related to the broader Flint food system. Once maps were created they were aggregated into a group model. Although still in progress, scenario analysis on both the sector-base aggregated model and the overall model will indicate the degree to which all proposed policy ‘levers’ are effective in achieving the preferred state defined by the resilience phase.

Figure 1. ‘Example of a Fuzzy Cognitive Map (FCM) developed from interviews with Flint food system actors. The arrows labeled with ‘+’ indicate positive (direct) casual relationships between concepts, while the arrows labeled with ‘−’ indicate negative (indirect) causal relationships.

System dynamics modeling

The system dynamics modeling work is currently ongoing in Year 4 of the project, informed by the focus groups and fuzzy cognitive modeling interviews. This modeling work will contribute to objectives 5 and 6. One of the challenges we faced was translating community perceptions about the problems in the Flint food system into model structures which could be simulated, while retaining the original meaning and rendering the model intelligible to community partners without a modeling background. We did this through the use of archetypes, or system stories, which have been defined in the system dynamics literature previously (Meadows 2008; Wolstenholme 2004). Archetypes for the Flint food system describe the maladaptive dynamics operating in the current system while pointing to potential ways to disrupt them. We developed a method of coding focus group transcripts for systems archetypes with the intention of retaining the integrity of community stories in the quantitative modeling process, and of more easily communicating system structure and behavior between modelers and community members.

Throughout the project, we have used a range of participatory methods, including modeling methods, to integrate community and academic insights in useful ways. These methods are iterative and build on one another; for example, the community values identification informing the fuzzy cognitive mapping, and the qualitative data collection informing the quantitative system dynamics modeling. This allowed us to incorporate a range of diverse community voices and perspectives into the research, and to reach conclusions informed by these perspectives.

Products

Throughout the project, we have sought to publish insights from the research in both academic and community-facing venues, as described in the ‘Progress in the First Three Years’ section. In the final upcoming year of the project, disseminating the learning in a manner that is accessible to the community and useful for changemaking is a top priority. In the past, we created briefing notes, which were useful for communicating with the CCP, but which we learned needed to be targeted more specifically to particular Flint audiences in order to be effective at community-wide communication. We have since brainstormed more creative solutions. Planned outreach and collaboration events include community art projects (artists will create works based on what we have learned about the Flint food system); a Food Summit in collaboration with the new Flint Food Policy Council; and modeling and scenario planning activities which will be open to the community. We will evaluate each of these activities for their effectiveness at fostering learning and engagement, and pivot to other activities if necessary in the final months of the project, which is set to end by May 2023.

COVID supplement and race intersection

The disproportionate impact that COVID-19 has had on communities of color nationally caused us to pause to reassess the needs of our partners, while documenting new impacts of the pandemic on food systems. In a highly segregated, low resourced city, how did Flint residents perceive or experience opportunities for participation in local food production, processing, retail and waste sectors? How might participation help them move from the oppression of food apartheid to a state of food security and sovereignty? We are currently conducting a second phase of FCM interviews with Flint residents, as well as Detroiters with similar food system experiences, and a few national experts who work at the intersection of food and racial equity to explore these research questions. This work was funded through a supplemental grant from the Foundation for Food and Agriculture Research to address food system issues arising out of the COVID-19 pandemic.

Challenges

In this section, we describe the major challenges we have faced in project implementation so far. While we have developed strategies to address these challenges (described in the following section), some remain unresolved in whole or in part, and represent tensions that most community-university partnerships will need to hold with care.

COVID

The COVID-19 pandemic arrived in Michigan approximately halfway through the Flint Leverage Points Project. Black communities in the state such as Flint were particularly devastated by the disease, because Black Michiganders are represented disproportionately among ‘essential workers’ who were unable to quarantine, and are already receiving a poorer standard of health care compared with predominantly white communities (Anderson-Carpenter and Neal 2021). Many community-based organizations in Flint that had been participating with the FLPP had to pivot quickly to serve their community’s needs differently during the pandemic. This was inspiring to witness, and further illustrates the high degree of innovation within the Flint food system. For example, a pastor who sat on the CCP and runs a food pantry worked with another CCP member representing Flint Fresh (a food delivery service) to package meals for curbside pickup, in order to provide food while complying with COVID protocols. Food banks and food pantries expanded their services, often with reduced staff, as the need in the Flint community grew. At the same time, we saw anew the potential of the FLPP to use the pandemic as a learning experience for long-term transformation of the food system, and integrated new data collection components into research design. As researchers, we held to state and university requirements to stop our in-person data collection activities and pivot to online meetings to keep our communities safe. We also had to be sensitive to the fact that many Flint residents, including those we work with closely and consider friends, were dealing with both personal loss and emergency situations, and our first concern was for their wellbeing. During this time both our research team and our community partners worked to determine the best way to maintain engagement in a virtual space. Ultimately the pandemic limited our initial level of engagement and shaped the perspectives included in the research as a whole. Prior to the pandemic, several members of our team regularly attended food distributions, community lunches, and interacted with members of the homeless community. Our long-term limitations in using virtual spaces from March 2020 through 2021 resulted in our inability to engage with partners without internet access or those who do not have reliable phone service. This is important because it likely impacts participants with significant food security challenges, and we now must acknowledge how this shaped data collection and results.

CCP availability and turnover

During the pandemic, many of our CCP members became even more busy, managing an increased workload and greater community need with diminished staff and restrictions on working in person. It is therefore understandable that some members were unable to continue engaging in the project and attending CCP meetings. However, this presented a challenge for the research team, as we no longer had access to the diverse community viewpoints to weigh in on the project. At some points during the pandemic we had only two or three CCP members (out of 12) attending the meetings. As a project team we had to grapple with whether to bring in additional CCP members to replace those who were no longer able to participate, since the time and resources needed to orient new members joining more than halfway through the project was substantial. Turnover and attrition of community partners involved in participatory modeling projects is a phenomenon that has been documented in the literature (Van den Belt 2004). Although community partners are compensated for the time they spend engaging in research activities, the project is not part of their job responsibilities as it is for the research team. With current research and funding models, this will always lead to an imbalance affecting the relative participation of university and community partners. Community-based research collaboratives represent one systemic solution to this problem, as do university-funded boundary spanners who sit in both community and university spaces.

Managing disagreement and conflict

In a research project involving dozens of personnel from the university and community side with widely different positionalities, backgrounds and perspectives, disagreements inevitably arose. In the case of the FLPP, these ranged from disputes over authorship of project documents, to general dissatisfaction with the progress or process of the research, to interpersonal conflict (i.e.., ‘getting on one another’s nerves’). It was important to us to take a stance that conflict was not a sign of failure, but an opportunity for learning and growth of both the project as a whole and the people engaged in it. Our goal was to foster healthy conflict (disagreement that is productive and leads to learning), and discourage unhealthy conflict (disagreement that is polarizing and divisive (Ripley 2021)). The project team has attempted to adopt a consistent stance of encouraging dissenting opinions and honest feedback about any aspect of the project. Part of the process monitor’s and project manager’s roles are to convey concerns from community members to research leads, should community members or junior members of the team feel uncomfortable voicing them to the research team directly.

As a response to some of the inter-team conflict, the project leads developed a community norms document outlining roles and responsibilities of research team members; the project’s approach to participatory research; authorship guidelines; logistical aspects of project management; and protocols for managing relationships and conflict on the team. University-based team members were asked to sign the document indicating their agreement with the norms in order to participate in the project. Authorship guidelines are common components of many academic departments, and using that as a framework was a beneficial launch point to collectively think through how to manage collaboration, authorship, and engagement.

Managing funds

The effective and equitable management of project budgets has been recognized as one of the key challenges for community-university partnership projects (Kellogg Commission on the Future of State Land-Grant Universities, National Association of State Universities, and Land-Grant Colleges 1999). Universities have an incentive to keep as many grant dollars under their management as possible, both for collecting overhead and for ensuring compliance with reporting requirements from the granting agency. However, for many community-engaged or community-based projects, it may be more appropriate for the community partner to hold and manage research funds, in order to spend this money in the community and to respond more nimbly to project needs. The original plan for the FLPP was to have the CFGF hold and manage the grant funds with a subcontract to MSU; however, after significant pushback from the university, the money was re-granted to MSU with a subcontract to the community partner (CFGF). This caused a long delay in project funds being released and considerable stress for project personnel and support staff managing the budget. To further complicate this process, restriction of travel during COVID, along with unforeseen expenses and opportunities, required frequent updating of the budget and re-adjustments with both MSU and CFGF. This issue is still in the process of being fully resolved, four years into the timeline of the original project. Ideally, CFGF would have held and managed the funds, for the reasons articulated above.

Time

As with many community engaged research projects, team members had to acknowledge and negotiate different expectations around time. Community partners were busy with multiple other commitments and priorities besides the research project, and were often facing immediate problems to which they were hoping the research would lend insight. Some community-based team members therefore expressed frustration with the length of time needed to collect and analyze data in order to develop an informed statement on a given topic. On the other hand, academics also have pressing commitments. Graduate students need to develop and publish thesis projects, and investigators need to submit reports to the granting agency with tangible progress. Academic team members were therefore sometimes frustrated with the length of time needed to build relationships of trust with community members necessary to develop these research products. Throughout the project, balancing the need for inclusive process (for example, checking in with all team members and community partners about a given decision) with the need to produce timely results for both community needs and academic needs was a challenge. This is a tension that will likely be present in every community-engaged research project, and which must be negotiated by surfacing expectations and communicating needs clearly and openly. For example, community-based team members began using the local team meetings to bring forward upcoming community meetings and decision processes at which interim project results could be discussed, and to flag them for attendance by research team members. This helped integrate interim project results and updates with community processes.

Changes to the project

This section describes how we responded to these challenges, through project pivots encompassing the process, partners, purpose and products. The evaluation contributed to the evolution of the FLPP by routinely sharing and discussing evaluation findings and recommendations in research team and CCP meetings as they became available. Integrating evaluation throughout the project, rather than leaving it for the end allowed us to improve over time, and is a recommended practice for community-engaged research. In year three, the evaluation and research teams collectively agreed to make the discussion of evaluation recommendations a regular feature of biweekly research team meetings. There have been several such discussions; below we concentrate on those that led to concrete changes in project procedures.

Some of the most impactful evaluation findings came from a December 2019 CCP focus group in which CCP members expressed their desire for the research team to be more transparent with them about what segments of the Flint community the research team was interacting with and what questions they were asking them. CCP members felt that such information would help them be more effective in guiding and supporting the project’s community engagement efforts. Following the discussion of these findings, the research team added questions about participants’ zip codes, wards and neighborhoods to the event evaluation questionnaires. The evaluation team subsequently mapped those data to identify the geographic distribution of participants across the city of Flint. The accumulated data, presented to the research team and the CCP in April and May of 2021, showed that residents of the northwest areas of Flint were underrepresented among FLPP participants. In response to these findings, the research team and CCP generated a list of community leaders and organizations in the northwest areas of Flint with whom the project might redouble its engagement efforts, and began collaboration on a data collection effort to inform the development of a new food coop in north Flint. Ideally, incorporating this geographic information on participant representation would have been built into the design of the project. In addition, the research team created and shared a Google Drive that contains CCP meeting minutes and presentations, research briefs, general information on participants, interview questions, and summary documents. The research team also developed a monthly newsletter to provide a brief summary of work to the CCP and alert them to what was next.

A second recommendations-driven discussion focused on rethinking how to engage the Flint community more effectively in dialogue about the study and its findings in light of the COVID-19 pandemic. This discussion contributed to the development of research briefing notes, a social media plan, and Pathways to Participation (P2P). Research briefing notes are short, accessible summaries of the activities and findings of each of the research sub-teams for specific public audiences. Over the course of the project, we have incorporated CCP suggestions on how to more effectively target and distribute these briefing notes for Flint audiences. The project research lead has kept the project visible within the Flint community during the pandemic by regularly attending virtual community events. As many CCP members have been pulled away from the project due to increased responsibilities during the pandemic, we have developed a more flexible approach to engaging with them. Project team members contact CCP members on an individual basis to get their perspectives on the direction of the project, and to fill them in on developments, if they have not recently attended meetings. Members of the research team also contact CCP members with targeted problems that require their specific expertise and feedback.

A third discussion focused on a recommendation to improve coordination with other research efforts in Flint that may be addressing similar issues to minimize research duplication and expand reach in the Flint community. With this aim in mind, the project research lead attends meetings of the Healthy Flint Research Coordinating Center, ‘a community-academic partnership with a goal of establishing equitable relationships between community and academia’ (HFRCC). Broader discussions with our colleagues at MSU Flint, UM Flint, and Kettering University, along with support from MSU University Outreach and Engagement have fostered discussions on collaboration and improvement of FLPP research activities.

A fourth discussion focused on a recommendation to develop procedures to assess participant learning due to participating in the project. Early efforts to include a question about participant learning in the event evaluation forms were unsuccessful due to low response rates and incomplete responses. In year four, the evaluation team will conduct a focus group with former and current CCP members to discuss how their understanding of the Flint food system changed due to their involvement in the project and/or their awareness of project findings.

It is fair to say that we initially underestimated the challenge of keeping everyone associated with the FLPP informed as to the many project activities. The first two years of the project were a bit of ‘trial and error’ process until we settled on a schedule of more frequent but shorter meetings with the CCP, the Flint local team, and the research team, following recommendations from the evaluation team and process monitor. This period was marked by some miscommunication and frustration on the part of project team members, who sometimes felt that they were not receiving the feedback they wanted on proposed ideas, or that they were left out of activities they didn’t know were taking place. After we implemented the new meeting structure, these feelings were reduced and team members generally felt more informed. The implications are that the central project staff (PIs, project research lead, process monitor and project manager) will need to attend a lot of meetings in order to stay informed about all aspects of the project and to communicate important updates. This is part of the additional time commitment that comes with community engaged research, as described in the literature (Khodyakov, Mikesell, and Bromley 2017).

Lessons learned

We learned lessons from the FLPP that are applicable to each of the principles of community-engaged RRI. In terms of co-designed innovations, the methodology of participatory modeling is structured for co-design. System structure and potential solutions to systemic problems are articulated by researchers and community partners together, meaning that the models addressing the research questions reflect the worldviews of both. Therefore, the results are co-discovered; the knowledge generated belongs to both the researchers and the community. This is the goal of participatory modeling (Hovmand 2014).

The role of boundary spanners in community-university research partnerships in facilitating co-design of innovations has been articulated previously. It is clear that this is still a role that is under-appreciated and under-invested in from both the community and university sides of most projects. Moreover, role redundancy is often considered in the research team, but not in the role of boundary spanner, which should change. For example, if a person with a specific skill set is necessary for the project, might someone else be trained to take over their role in the event that they are unable to continue with the project?

In a community-engaged research project, adaptability, flexibility, and learning throughout the project are key to ensuring the relevance of project results and the strength of trust and communication between partners. The integral participation of the evaluation lead and the process monitor from the beginning of the project helped us to pivot when changes needed to be made, whether externally driven (the pandemic, the departure of the Food Navigator) or internal (updates to communication plans). Similar to the role of boundary spanners, the role of evaluators is frequently under-appreciated in a research context. Moreover, evaluation activities are often planned for the end of a project, but not as formative activities throughout the project duration. Assigning these roles (evaluator and process monitor) to people other than the principal investigators allowed for more candid and objective feedback to surface from diverse project participants.

The origins of this project in a Flint-based organization helped ground it in relevance to the community. This was essential in providing traction and uptake for interim project results, and for ensuring community members would be interested in engaging the project. In almost every case, Flint residents who interacted with the FLPP expressed a desire to be updated on project results, and an interest in how those results would be implemented. It is hard to imagine this would have been the case if the research were conceptualized without input from Flint. We did face the challenge of navigating different perceptions of both the research team and the community around what participatory or engaged research looks like, which is unsurprising given that these are defined on a scale, and community members likely have interacted with researchers across the full spectrum of this scale (Shirk et al. 2012). During year 2 of the project, we clarified our position by developing a project statement on participatory research. Ideally, we would have done this earlier in year 1. In addition, our CCP taught the research team how important it was to be cognizant of the diversity of experiences and perspectives represented by the Flint community, and to reach out to as broad a spectrum of community voices as possible. Without an effort to do so, the research could be seen as illegitimate by some community constituents.

Ongoing challenges remain around the negotiation of conflict, the fluctuation of CCP participation, and the management of funds. We submit that perhaps these are not ‘solvable’ problems, but dynamics that any community engaged research team will have to negotiate through open communication and flexibility. It is easy now to look back on the first three years of the project and see where we should have anticipated a difficulty, or planned something differently. Certainly it is our hope that future projects will learn from our mis-steps and do better. However, it is impossible to anticipate every challenge (the COVID-19 pandemic being a good example), and community engaged research teams should not expect that this is a reasonable or desirable goal in project planning.

We therefore offer these reflections on a project in progress in the hope that they will benefit others doing similar work. We have attempted to paint an honest picture of our project’s successes, shortcomings, challenges, and unresolved issues from the varied perspectives of the authors, who occupy different roles on the project. Community-university research partnerships are messy, challenging, and time-consuming, which is perhaps why they continue to be less common than more ‘traditional’ research, despite their value. They require tremendous capacity for personal and professional growth and learning on the part of all people engaged in the project. Humility, empathy, deep listening, and the ability to admit mistakes and course-correct are important characteristics for researchers engaged in these partnerships, yet these are not characteristics that are encouraged or rewarded in academia (quite the opposite). It is perhaps for this very reason that these research partnerships are–or can be – ultimately highly rewarding both personally and professionally. The experiences that most challenge and stretch us in life are typically the experiences we value highly. We are also aware that this paper lacks the voices of the community members whose lives and livelihoods are most highly impacted by the Flint food system, and we plan a subsequent paper with more insights shared from community members. We know that ultimately the FLPP will be judged by whether it achieved its stated objectives and generated useful knowledge on which the Flint community can act. It is our hope that, in the interim, lessons learned along the way will help others engaged in this type of work.

Acknowledgements

The authors wish to thank the community of Flint for their contributions to this ongoing work.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by Foundation for Food and Agriculture Research.

Notes on contributors

Laura Schmitt Olabisi

Laura Schmitt Olabisi is an Associate Professor in the Department of Community Sustainability and the Environmental Science and Policy Program at Michigan State University. She is an ecologist and a participatory systems modeler, working directly with stakeholders to build models that foster adaptive learning about the dynamics of coupled human-natural systems. Laura holds a doctoral degree from the State University of New York College of Environmental Science and Forestry. She was a AAAS Leshner Leadership Institute Public Engagement Fellow in 2018–2019.

Chelsea Wentworth

Chelsea Wentworth is a Research Assistant Professor in the Department of Community Sustainability, Adjunct Professor in the Department of Anthropology, core and GJEC faculty for the Center for Gender in a Global Context at Michigan State University. Dr. Wentworth’s research examines hunger and food security, food as a human right, and sustainable food systems through a gendered lens, promoting critical reflection on the impact cultural experiences have on health access and policy. Currently, she is working on several projects, including long-term ethnographic research in the South Pacific in Port Vila, Vanuatu, where she studies infant and young child feeding practice, urban gardening, and disaster response in collaboration with the Vanuatu Cultural Centre and Ministry of Health. She also co-leads the Flint Leverage Points Project, a community-research partnership with the Community Foundation of Greater Flint. This research aims to map the Flint food system to identify leverage points to improve nutrition security and support evidence-based public policy. Additionally, Dr. Wentworth works on teams studying indigenous food systems in Michigan tribes and co-leads a team examining values surrounding farmers markets. Emphasis on feminist community-engaged research praxis and understanding food access through a systems-based approach unite Dr. Wentworth’s international and US research.

Kent Key

Kent Key is a is a Racial and Ethnic Health Disparities Researcher at the Michigan State University College of Human Medicine, Division of Public Health. Dr. Key specializes in Community Engaged Research approaches and methodologies. He has an interest in urban minority populations and use qualitative research methods to evaluate the perceptions and lived experience with a goal towards the development of community-driven solutions for health equity. He also worked to create equitable engagement for patient/provider engagement as he was a member of the National Patient Stakeholder Council for PCORI and PCORI’s National Fall Prevention Council. In his community role, Dr. Key is the Executive Deputy Director of the Community Based Organization Partners (CBOP) and Founder of the Community Engagement Studio of Flint. Dr. Key is also the Founder and Director of the Flint Public Health Youth Academy. Dr. Key is 2017 Fellow of the Robert Wood Johnson Foundation Culture of Health Leaders Program. In June of 2020, Dr. Key authored a resolution Declaring Racism a Public Health Crisis in Genesee County which was passed by both the Genesee County Board of Health and Board of Commissioners.

Renée V. Wallace

Renée V. Wallace bio: Reneé V. Wallace is a lifelong learner and entrepreneur, serving as executive director of FoodPLUS Detroit and CEO of Doers Edge, headquartered in Detroit, Michigan. Introduced to participatory modeling in 2015, she served on MSU’s Innovations in Collaborative Modeling conference planning team, helping develop a community track for people new to systems thinking and modeling. Building on that experience she now serves as community partner and co-host, supported the 2016 conference, as well as the transition to modeling field schools in 2019 and 2021. Renee’s modeling experience includes building of systems dynamic models (food insecurity in Detroit 2015, urban livestock in Detroit 2016), fuzzy cognitive models (Trusted Conversations in Flint water crisis 2016); and integrated use of FCM and SD Models (Flint Leverage Points Project: Modeling the Flint Food System 2018–2021). Reneé is working to re/define community members project roles of sponsor, community PI, advisory team, citizen researcher, visual facilitator and recorder, and process monitor and evaluator; and she is developing mastery in using qualitative techniques that complement participatory modeling methods. Reneé envisions using these disciplines to work with visionaries and doers to accelerate development and implementation of innovative solutions to diverse types of challenges facing urban communities.

Miles McNall

Miles McNall, Ph.D., is Director for Community Engaged Research in the Office of Public Engagement and Scholarship at Michigan State University. Miles has 24 years of experience with the evaluation of health and human service programs and 16 years of experience supporting faculty, staff and students in their community engaged scholarship (CES). Miles has conducted evaluations of a wide variety of interventions including HIV/AIDS prevention and care programs, comprehensive community initiatives, school-based health centers, intensive home-based treatment services for children with severe emotional disturbances, and systems change initiatives in early childhood and youth mental health systems. Miles is the President of the Michigan Association for Evaluation. Miles offers a variety of supports to faculty staff and students for CES, including assistance in the development of academic-community partnerships, educational and professional development opportunities on the theory and practice of CES, and support in developing broader impacts plans and evaluations for NSF proposals. In addition, Miles is the lead organizer for the Innovations in Collaborative Modeling conference and Participatory Modeling Field School at Michigan State University.

Jennifer Hodbod

Jennifer Hodbod is an environmental social scientist, exploring the social-ecological resilience of food systems. Jenny has a PhD in Environmental Social Science from the University of East Anglia (UK), within the Tyndall Centre for Climate Change Research, and was a Postdoctoral Research Fellow at Arizona State University. She is an Assistant Professor in the Department of Community Sustainability at Michigan State University. Her research covers multiple types of food systems – urban agriculture in the USA, Adaptive Multi-Paddock (AMP) grazing in the USA, river basin development in Ethiopia, and leverage points in Flint, MI. All these projects are linked by the use of interdisciplinary methods to investigate cross-scale impacts on adaptive capacity and transformative capacity, and how this influences social-ecological system resilience.

Steven A. Gray

Steven A. Gray is an associate professor in the Department of Community Sustainability at Michigan State University. His research focuses on socio-environmental modeling and understanding how individuals and groups make decisions about complex social-ecological systems. He is the lead editor on the book, Environmental Modeling with Stakeholders: Methods, Theories and Applications (Springer 2017). His research has been funded domestically by the National Science Foundation, the Socio-Environmental Synthesis Center (SESYNC) and by federal resource management agencies including the National Oceanic and Atmospheric Administration, the United States Department of Agriculture, and the United States Geological Survey. Internationally his research has been supported by the Leibniz-Institute, the Australian Academy of Sciences and the Belmont Forum.

Notes

1 https://www.canr.msu.edu/flintfood/uploads/files/2022-0627-Flint-Food-System-Timeline.pdf.

2 https://www.canr.msu.edu/flintfood/index.

Appendix

Table describing constituencies and identities represented by the original 12 members of the Community Consultative Panel of the Flint Leverage Points Project. Each row represents an individual member.

Table

Female (F)	Male (M)	African American (AFA)	White (W)	Latino (L)	Seniors/Senior Citizens (S)	Youth (Y)	Community Member (CM)	Returning Citizens (Prisoner Reentry)(PR)	Business Owner (BU)	Researcher (R)	Community Based Organization (CBO)	Education (EDU)	Economic Dev. (ED)	Faith Community (FC)	Food (F)
x				x			x				x	x
x			x								x				x
	x		x				x				x			x
x		x					x				x		x
	x	x					x		x						x
x															x
x			x								x				x
	x	x					x			x	x	x
x				x						x		x
x		x					x						x
x														x
x		x			x		x				x