A categorization of observed uses of operational research models for fundamental surprise events: Observations from university operations during COVID-19

Abstract

Operational research (OR) approaches have been increasingly applied to model resilience to surprise events. To model a surprise event, one must understand its characteristics, which then become parameters, decisions, and/or constraints in the model. This means that these models cannot (directly) handle fundamental surprise events, which are events that could not be defined before they happen. However, OR models may be adapted, improvised, or created during a fundamental surprise event, such as the COVID-19 pandemic, to help respond to it. We provide a categorization of observed uses for how OR models were applied by a university in response to the pandemic, thus helping to understand their role during fundamental surprise events. Our categorization includes the following adaptations: adapting data, adding constraints, model switching, pulling from the modeling toolkit, and creating a new model. Each of these adaptations is formally presented, with supporting evidence gathered through interviews with modelers and users involved in the university response to the pandemic.

Keywords:

• Resilience
• fundamental surprise

1. Introduction

Operational research (OR) models have been increasingly created to analyze the resilience of systems (Sharkey et al., 2021). Common themes in the definitions of resilience focus on the ability of a system to respond, adapt, or recover when faced with stressful events, based on the foundational work of Holling (1973). A stressful event is often defined as an event that pushes a system towards or past its normal operating boundaries or conditions. To model a stressful event, one must understand its characteristics which become inputs into the OR model. Stressful events that appear in OR models a priori for resilience must fall into the category of situational surprise as discussed by Lanir (1986) and Wears and Webb (2014), which focuses on unlikely events that have a low probability of occurring but can be imagined beforehand. Alternatively, fundamental surprise involves events that “refute basic beliefs about ‘how things work’” (Wears & Webb, 2014) where one “cannot define in advance the issues for which one must be alert” (Wears & Webb, 2014). This does not mean that OR models cannot play a role in fundamental surprise events but suggests that they must be deployed in response to the surprise event, when its characteristics become better known. The context is inherently reactive. This brings unique challenges to model development, including compressed timelines, integrating stakeholder involvement and buy-in, and data availability. There are also unique opportunities where modelers and users are aligned in the necessity to make time-sensitive, impactful decisions, and implementation may be a key goal.

The existing paradigm for uncertainty in OR models is focused on situational surprise and is insufficient in cases of fundamental surprise. To address this shortfall, we will study the following research question: How can OR models are used to improve resilience to fundamental surprise events? Given the limited number of fundamental surprise events that occur, gathering data to answer this question is challenging. This paper uses the impact of COVID-19 on university operations to identify uses of OR models in response to fundamental surprise events. Our categorization of observed uses includes both instances when an OR model was used during the response to COVID-19 and when an OR model could have been used or adapted.

A general framework for system resilience comes from Woods (2015). He presents four ways to demonstrate resilience, i.e., rebound, robustness, graceful extensibility, and adaptability, in order to unify the diversity of definitions of resilience. Rebound focuses on the ability of a system to recover from a stressful event. Robustness focuses on the ability of the system to maintain operations under stressful events. Extensibility focuses on the ability of the system to alter boundaries or operations when surprise events challenge it. Adaptability focuses on the ability of the system to adapt to future stressful events as conditions evolve. Sharkey et al. (2021) framed the contributions of the network optimization community, a subset of the broader OR community, to resilience within the four concepts of Woods (2015). These contributions tend to focus on rebound (Çelik, 2016) and robustness (see, e.g., Faturechi & Miller-Hooks, 2015; Snyder et al., 2016; Wang et al., 2016), consistent with the differences between situational and fundamental surprise events. When a system is designed or decisions are made with an OR model, the potential for the system or decisions to be resilient to events that were not considered beforehand is limited. Response to fundamental surprise events initially relates to extensibility and adaptability and often requires improvisation by the system. During a fundamental surprise event, operational strategies and models pertaining to rebound and robustness within the system can be devised, adjusted, or improvised.

Eisenberg et al. (2019) discuss that improvisation is necessary to deploy “resilience analytics” in response to fundamental surprise. They provide a framing mechanism to understand how improvisation would occur by the model, the user of the model, or the modeler (or some combination of them) to address a fundamental surprise event, increasing the extensibility and adaptability of models. Sharkey et al. (2021) highlight that optimization models that enable improvisation are an important research direction. To move towards creating models that enable improvisation, we should first seek to understand how models are deployed in fundamental surprise events.

This paper provides a categorization of OR model usage in the response to COVID-19 at Clemson University. COVID-19 was a fundamental surprise event since it altered the basic structure of how courses were delivered, how resources (such as hand sanitizer) needed to be deployed, and how distinct organizations within the university needed to coordinate efforts. We provide empirical evidence of the OR modeling improvisations and adaptations that took place during COVID-19 in creating this categorization.

At a high-level, our paper contributes to the literature providing a better understanding of how OR models were used in response to a fundamental surprise event through the case study of a university during COVID-19. More specifically, we provide an evidence-based categorization scheme of improvisations and adaptations that occurred, which builds off the work of Eisenberg et al. (2019). We provide evidence of actual improvisations and adaptations, while Eisenberg et al. (2019) discussed high-level, thought examples of how improvisations and adaptions could have occurred during fundamental surprise events. We provide a categorization scheme of uses in terms of how models were adapted, switched, or created, based on interviews with professionals involved in various decision-making processes associated with the university. We provide examples from these interviews for each of our categorizations, which helps to provide more qualitative support for the improvisation schemes laid out by Eisenberg et al. (2019). We apply our categorization to a sampling of other OR models deployed at universities in their response to COVID-19.

The remainder of this paper is organized as follows. We provide a background on OR modeling processes as well as OR model usage, and related technologies, during COVID-19 in Section 2. We provide the context of the studied university in Section 3 and discuss our research methodologies in Section 4. In Section 5, we provide the results of these methods and formally present the categorization scheme of observed uses. We analyze a sampling of OR work done at other universities in Section 6. In Section 7, we discuss insights and an agenda to better enable the use of OR models during fundamental surprise events. We conclude in Section 8.

2. Background on or modeling processes and or usage during COVID-19 response

The process to improvise OR models during fundamental surprise events is like the general OR modeling process. Morris (1967) discusses the types of tasks that are necessary to inform mathematical models. Willemain (1994, 1995) interviewed OR experts to understand how they create models for a practical problem. This process was especially important given the number of stakeholders that OR experts interact with and consider while creating models, conducting analysis, and communicating their recommendations. Similarly, Tako and Robinson (2010), Brooks and Wang (2015), and Tako (2015) investigate expert approaches to simulation modeling.

Pidd (1999) offers six principles for modeling in practice, with several of them being relevant to our observed uses of OR models during the fundamental surprise event of COVID-19. The principle of model development being a gradual process is illustrated in our iterative model development adaptations. The principle of proper use of data is illustrated when the fundamental surprise event radically changes current operating conditions. The principle of the modeling process being chaotic relates to the time-critical nature of deploying models in response to the event. The contextual considerations of Gorman (2021) emphasize the need for modelers to work carefully with stakeholders as they try to use models to support decision-making, which is a consideration of OR use in response to COVID-19. We note that existing literature on model development has not considered the context of fundamental surprise and, therefore, we provide an important contribution in this space.

In general, digital technologies played an important role in the response to COVID-19 (Khan et al., 2022a; 2022b; Liu et al., 2022). In the area of university operations and COVID-19 response, OR model usage impacted many areas including classroom layout for social distancing (Greenberg et al., 2021; Murray, 2020), transportation-related schedules (Chen et al., 2021), and issues related to class scheduling and attendance (Barnhart et al., 2022; Gore et al., 2022; Johnson & Wilson, 2022; Navabi-Shirazi et al., 2022). Simulation model usage also impacted many areas of university operations and COVID-19 response, including understanding its spread (Harper et al., 2021), capturing the impact of different testing strategies and other related policies (Brook et al., 2021; Frazier et al., 2022; Ghaffarzadegan, 2021; Saidani et al., 2021), and modeling different classroom layout and pedestrian movement policies (Islam et al., 2022). The main contribution of our paper is that it helps to provide an evidence-based categorization of how OR model usage during the surprise event of COVID-19.

3. Context of Clemson University and their or model usage during COVID-19

The response of Clemson University to COVID-19, like all colleges and universities, involved adaptations and new challenges across all its organizations (Hegde et al., 2022). Clemson University moved all of its Spring 2020 classes online on March 19, 2020 and announced that its Summer 2020 classes would be online on April 8, 2020. It was committed to providing in-person classes in Fall 2020, while adhering to federal COVID-19 guidelines for social distancing and other public health measures. Three areas in which OR approaches were considered were the scheduling of courses to classrooms (course scheduling), the assignment and scheduling of cohort of students to days of in-person classes (rotational attendance), and locating hand sanitizing stations on campus. Table 1 provides an overview of the pre-pandemic approaches to these problems as well as how changes to the modeling approaches occurred.

Table 1. An overview of model use and adaptations by the COVID-19 response of Clemson University.

	Pre-pandemic	Fall 2020	Spring 2021
Classroom scheduling	Banner, with historical classroom data, and input from department-level schedulers and students	Banner with classroom data update in accordance with COVID-19 guidelines	Banner, with classroom data update in accordance with COVID-19 guidelines
Cohort scheduling (hybrid)	Policy not in place: No model needed	Cohort based MIP model	Cohort-based MIP model
Hand sanitizer location	Resource not deployed: No model needed	Heuristic-driven demand model, (No math modeling)	Application of facility location models

In preparing for the Fall 2020 semester, social distancing policies impacted classroom capacity. Instructors had the option to request fully online or in-person courses. Students could elect to be fully online. Once the initial set of selections was analyzed, there was not enough classroom capacity to allow traditional in-person instruction. As described in Gore et al. (2022), Clemson University enlisted a research team to create a rotational attendance scheme in which each student is assigned to exactly one “in-person” day a week in each course, resulting in hybrid courses. This team created an integer program (IP) to determine the rotational attendance scheme. On July 22, 2020, the university announced that the first four weeks of the fall semester would be online, providing more time to plan. Course planning for Spring 2021 followed a similar process with the main difference being that students registered knowing both the course schedule and teaching modalities. Once registration occurred, the rotational attendance scheme was determined through the IP.

The Clemson University Facilities Department was responsible for deploying hand sanitizer across campus to mitigate the spread of COVID-19. They purchased sanitizer and built deployment stations that could be placed at entrances to different buildings. For Fall 2020, they determined locations for these stations using heuristics and building coordinator familiarity with relative door usage. In Spring 2021, OR modelers worked with the department to develop facility location models to recommend locations based on estimated demand from door access data. O’Brien et al. (2022) discuss a blended approach that integrates location modeling with qualitative interview analysis that was applied to reach these recommended solutions.

In general, many aspects of delivering academic programs have been addressed in the literature (Barnhart et al., 2022). These aspects include term planning, time tabling and room assignment, and student enrollment. As institutions faced the disruptions of the COVID-19 pandemic, especially entering the 2020–2021 academic year, these aspects were altered in unexpected ways. Analyzing the response to COVID-19 at universities offers a unique perspective on how this fundamental surprise event impacted OR model use.

4. Research methods

This study was conducted using qualitative analysis of semi-structured interviews. Participants were comprised of 14 pandemic modelers and users from Clemson University, a large public university in the southeastern United States, who helped to coordinate the university’s pandemic response. Participants include administrators and coordinators from the university Registrar’s office as well as department-level scheduling coordinators and analysts, which all helped to contribute to the university’s pandemic response decision-making. We did not interview students since they were not involved in this decision-making, although we recognize that the decisions did impact students. We selected participants that provided coverage across the three different areas of OR model usage (classroom scheduling, rotational attendance, and locating hand sanitizers on campus). In these areas, we interviewed one modeler that was knowledgeable about the modeling process for that particular area and, therefore, interviewing an additional modeler was not necessary. We further interviewed users in these areas to understand their perspectives on how they interacted with the model and the modelers. We began seeing a degree of saturation in these responses (i.e., answers in latter interviews were previously observed) from the users, which helps to indicate that further interviews were not necessary.

The semi-structured interview script (Appendix) was developed to elicit considerations for course scheduling, rotational attendance, and resource allocation. The study was approved by the institutional review board (IRB) of the authors’ institution and deemed exempt. Participants were informed about the study, and consent was obtained verbally before the start of the interview. Questions were designed to identify participants’ roles in university’s pandemic response and identify constraints and objectives that shaped decisions and policies. Further probes were based on prior knowledge elicitation and used to expand upon aspects of adaptation such as improvised workflows, resource constraints, and coordination between stakeholders. Interviews were recorded and transcribed to create the interview data. Each interview took an hour to complete.

The interview data were analyzed by HF and OR experts using the technique of thematic analysis. A rigorous thematic analysis on organizational resilience of this data was conducted by Foster et al. (2023). In terms of our categorization of observed uses, we analyzed the interview data for passages that related to decisions (or, equivalently, variables), requirements on those decisions (e.g., constraints), goals related to the decisions (e.g., objectives), and data necessary to make them (e.g., input parameters). Relevant passages were flagged and reviewed by the OR experts to understand how the passage may refer to an observed OR model use. We analyzed passages using traditional modelling considerations, based on our previous experience, and reviewing the discussions of Pidd (1999) and Gorman (2021). More precisely, relevant passages in the transcripts were reviewed by the OR members of the team and the proposed categories emerged from this analysis.

An important note is that Clemson University was not using an OR model for the scheduling of courses to classrooms. However, it would be natural to apply an optimization model that assigns courses to classrooms over time (e.g., Daskalaki et al., 2004). Therefore, we view this class scheduling process as the underlying model and believe that we can draw conclusions about how adaptations to this process would translate to using OR models during fundamental surprise events, especially when considering the rarity of such events.

5. Categorization scheme of observed model use during fundamental surprise events and supporting evidence

The focus of this section is on the categorization scheme of how OR models may be used to support the response to a fundamental surprise event. Following the framework of Eisenberg et al. (2019), we examine how adaptations or improvisations were necessary by the model, modeler, or user. Eisenberg et al. (2019) define these terms as follows: (i) the model focuses on the underlying mathematical representation of the decision-making environment; (ii) the modeler is the person (or people) responsible for creating the model; and (iii) the user is the person (or people) that will run the model and recommend decisions based on its output. Eisenberg et al. (2019) discuss the limitations of ‘resilience analytics’ to handling fundamental surprise and provide examples of how improvisation may happen across the model, modeler, or user. In this paper, we provide rigorous qualitative support for OR model usage in response to fundamental surprise events. We begin the section with an overview of our categorization of observed uses in university operations during COVID-19. We then take a deeper look at each categorization.

5.1. Overview of categorization scheme

We identified five different categories for OR model use during the response to this fundamental surprise event which involved model adaptation, switching, and creation. We classify the model use in five categories: adapt data, add constraints; switch model; pull model from toolkit; and create new model. Table 2 provides an overview of each of these categorizations, a summary of the model characteristics, and who is making the adaptation. For each categorization, we provide their definition, examples from our data, and considerations for their use.

Table 2. An overview of our categorization scheme for how OR models are used in response to fundamental surprise events.

		Model characteristics vs. pre-surprise
Category	Definition	Decision	Model	Data	Adaptation by
Adapt data	The underlying model remains the same but old data is no longer useful	Same	Same	New	User
Add constraints	The underlying model remains the same, but the decision-maker gives less freedom to the model.	Same	Modified	New	Either modeler or user
Model switch	The underlying model is substantially revised or switches to a different model altogether	Same	New or modified	New	Modeler and user
Pull model from toolkit	A new decision that needs to be made due to the surprise event could be made better by applying existing modeling approaches.	New	New	New	Modeler and user
Create new model	An entirely new model is needed for effective decision-making.	New	New	New	Modeler and user

There are similarities between the last four categories; however, the steps required to implement each observed use build off one another. To add constraints to the model, the modeler or user need to identify the additional characteristic of the decision-making environment and update the model, which can often be done quickly. For model switching, the modeler or the user need to identify the current limitations of the model and then switch the model, which will take more time than adding a new constraint. To pull a model from the toolkit, the modeler and the user need to identify the new decision that needs to be made, develop “buy-in” from the user(s) that an existing model can help, and then the modelers need to implement the model. To create a new model, the modeler and the user would take a similar approach except the modelers would need to both create and implement the model.

5.2. Adapting the data

5.2.1. Definition

The underlying model remains the same but traditional data is no longer useful and, therefore, data needs to be adapted to represent the current situation.

5.2.2. Overview and examples

Adapting the data occurs when the user believes that the fundamental surprise event does not break the model assumptions but does break traditional data sources. This can be viewed as an adaptation mainly by the user since there is no need to update the model. Many examples arose in COVID-19, especially around planning for the return to in-person classes while accounting for social distancing. Traditional data on classroom capacities was inaccurate when considering social distancing requirements for students. Data on which on-campus facilities and rooms could be used to hold classes also was no longer valid. For example, at Clemson University, basketball courts were used as classrooms to increase the size of courses that could be taught in-person under social distancing guidelines.

Data on classroom capacities was important in the class scheduling process and gathering it was discussed during our interviews:

At that point we didn’t have room capacities for our initial modeling. For every room, we extracted out the largest class that was registered in that room and then divided the capacity by a third just for our initial testing.

They said to use a six-foot rule and so from there we calculated this is how many people were in the class.

As the situation evolved, better estimates for certain classroom types were implemented:

It was determined quickly that for square classrooms, you could, with movable seating, you could sort of do square footage divided by 36 plus one for the instructor and get a good ballpark estimate.

Data on the set of rooms available to schedule classes in (the resources in a classroom scheduling model) required inputs from multiple organizations:

It was a fairly large team, it was 10ish people, and we evaluated data about the spaces and whether or not a room is suitable for us to use as a classroom, especially like the big box spaces and determining whether we could hold a class in, say, a gymnasium.

Adapting the data into the model may not always fall to a single person since the improvisation needed to understand how resources can be used may need to be made by various stakeholders with different knowledge bases.

5.2.3. Practical considerations for adaptation use

This adaptation requires that an underlying model currently exists and is deployed. It requires that users understand the underlying model assumptions and have enough information about the surprise event to believe the model is still useful if the right data is gathered.

The data-gathering process of the organization(s) impacts this adaptation because it must be appropriately accurate for the decision-making horizon. In the example of classroom scheduling, this data-gathering process included updating capacities of traditional classrooms (model parameters) and the schedulable spaces (the sets and decision variables). For Fall 2020, Clemson University had four months to gather such data. If this time horizon was smaller, then it may not have been possible for this to occur.

The intersection of the data-gathering process and the adaptability of the decisions also play a role in the potential use of this adaptation. If the data-gathering process may take longer than when the first decisions must be made, then this adaptation can still be useful if the decisions can be updated. The classroom scheduling example is one where the decisions are not easily adapted since moving one class could have significant ripple effects.

5.3. Adding constraints

5.3.1. Definition

The underlying model remains the same, but the modeler or the user provides it less freedom to help address issues around lack of information and/or uncertainty.

5.3.2. Overview and examples

This adaptation focuses on the modeler or the user identifying considerations that arose from the fundamental surprise event and adding constraints directly to the model that help capture them. An important special case is when the constraint “fixes” a variable. This adaptation may be implemented by the user if the way they interact with the model allows them to add new constraints or it may require an interaction between the modeler and the user. For example, a user may be able to directly add a constraint that fixes a variable; a constraint that focuses on a set of variables may need to be incorporated by the modeler.

For course scheduling, one consideration was to determine which classes would be fully online and would not need to request a classroom (resource). In this case, the added constraint would be to enforce that the class is online. One department indicated that large classes would have been difficult to accommodate in classrooms in a socially distant manner:

[O]ur largest classes are MSE [Materials Science and Engineering] 2100 and MSE 3190. The department here said, ‘Those are online. We’re not going to try because there’s no way we can find a room.’ Our other class sizes were much more reasonably sized with anywhere from 15 to 40–45 students at most.

There were further issues surrounding the modalities by which classes were offered, especially when considering that most campus buildings were not allowed to be regularly accessed. For example, the library required reservations which made it difficult for students that may have been going from an in-person class to an online class in only 15 min:

The biggest thing for my students was if they had classes back-to-back, … and, if they really wanted to be in person, it was going to be too difficult. So, they had to find a place on campus and just attend online. And then I had students that really, really needed to be in person all the time.

Constraints could be added that appropriately spaced-out classes within a particular curriculum (e.g., courses typically taken together during a semester) that had different modalities. However, the actual modality that was implemented by a particular instructor may have been adapted on the fly which means this situation may have arisen unexpectedly.

5.3.3. Practical considerations for adaptation use

This adaptation requires that an underlying model currently exists and is deployed. It requires that users understand the model assumptions and have enough information about the surprise event to understand the considerations that need to be updated.

A successful application of this adaptation requires differentiation between hard constraints and soft constraints. This is especially important as the user surveys the situation after the fundamental surprise event. The “conventional wisdom” was that all classes should be offered in-person as much as possible, and that some in-person teaching for each class was required. When some departments realized that a fully-online experience could provide a more uniform learning experience, and reduce demand for large classrooms, other classes were able to find classrooms. In this way, a potential hard constraint that required every class to have some in-person component was removed, and certain classes had their offering variable set to online.

5.4. Model switching

5.4.1. Definition

There was an underlying model in use, but the surprise event requires a switch to a different existing model.

5.4.2. Overview and examples

This adaptation focuses on examining the current environment after the surprise event, realizing that the existing model no longer applies, and switching to another model that better applies to the post-event environment. The decision to be made is similar, but the context surrounding the decision is not. Therefore, it involves both adaptations by the user in recognizing that the current model is no longer sufficient and by the modeler in working with the user to adapt to a better model for the current situation. Specific adaptations may include changing objectives to constraints or changing objectives completely.

Model switching may involve situations where constraints become objectives after the event, especially if they can no longer be satisfied, or objectives become constraints (Sharkey et al., 2021). During the COVID-19 response, models were switched for deploying hand sanitizer. Initial decisions were made to best deploy limited resources but eventually campus became saturated with hand sanitizer:

Of course, everything was eventually saturated but at the start we had very limited supply, so we built some stands to be able to put those hand sanitizers out in our primary use buildings.

The initial objective was focused on “covering” all people:

From my understanding, we were trying to cover everybody that was on campus.

Once everybody was covered, campus wanted to minimize the costs of deploying hand sanitizer (new objective) while ensuring the coverage of hand sanitizer (old objective, now a constraint).

Another example of model switching is the incorporation of safety considerations into decision-making surrounding campus activities:

I think that maybe the EOC [Emergency Operations Center] at times will make sure the students are safe and make sure that the people that were provided that material, our custodial staff, had all their PPE [Personal Protective Equipment] stuff and they had their mask and their personal sanitizing wipes.

Although general safety considerations are incorporated into campus maintenance, safety considerations surrounding disease spread did not arise until COVID-19.

5.4.3. Practical considerations for adaptation use

This adaptation not only requires that an underlying model is in use but there is another model that can be easily switched to after a decision-maker realizes the limitations of the currently deployed model. It requires the user to identify the limitations, including when certain constraints could no longer be satisfied and/or which would become objectives, and how other existing models would overcome them.

5.5. Pulling from the modeling toolkit

5.5.1. Definition

A new decision needs to be made due to the surprise event and can be made by applying variants of pre-existing optimization models.

5.5.2. Overview and examples

This adaptation focuses on using an existing model to address a new decision brought on by the event. The modeler uses an existing framework, i.e., “pulling from the modeling toolkit,” to create a model in partnership with the users. The classic model may be supplemented with minor side constraints, but its core is from an existing framework. We illustrate this principle with the example of locating the limited resource of hand sanitizer stations across campus at Clemson University. Prior to COVID-19 response, there were no hand sanitizer stations, outside of bathrooms and, therefore, this was a new decision. As will be discussed, the goal of the university was to ensure access to sanitizer stations, which an OR modeler would recognize as being related to maximum coverage location problems.

At Clemson University, a new decision that needed to be made during COVID-19 response was the location of hand sanitizer stations in academic buildings. To prepare campus for student return, the Facilities Department was directed to locate sanitizer stations throughout campus:

Most of the academic areas there really weren’t many [with sanitizer] and we quickly realized that was going to be a problem. … then we had to start making hard decisions on where those stations should be located and where the most impactful locations are.

The initial allocation occurred quickly and without a model:

[The stations] were initially located primarily via expert judgement … from [those]… who are responsible for … each building on campus.

To improve allocation, modelers partnered with the Facilities Department to develop an optimization model to recommend locations. This adaptation involved: (1) a new decision (hand sanitizer locations); (2) a new model to make recommendations on where to locate the stations; and (3) new data. Because the decision context fit primarily within the location problem framework, the modelers could pull directly from the toolkit during model development:

The main goal was getting students faculty and staff … access to hand sanitizer stations. … What we did on our [OR modeling] team was figure out ways to help allocate sanitizer stations effectively to be able to achieve that

During model development, it was important to understand the context—including the goal (objective), decisions to be made, restrictions on the decision, the ability to implement the recommended decisions, and data available. Each facet informed what type of modeling framework would be appropriate. Partnership with users was critical:

Facilities informed every aspect of the model. … We had several conversations with them about the structure of the model. We… didn’t talk about constraints or objectives, but rather we asked questions and had conversations to get at … their priorities

Through the iterations, the models were all “based on a conventional and existing framework”:

At one point we were considering a p-median based model. Through different model iterations and subsequent conversations with the Facilities Department we ended up landing on a maximum coverage style modeling approach.

The Facilities Department guided what version of the model was considered final:

At their behest, we decided to stop iterating, and so it was not only the structure of the model, but rather the decision to complete the modeling effort that came from discussions with the Facilities Department.

In this example, the final model was a variant of the classic maximum coverage problem (O’Brien et al., 2022), in which a set of locations (doors), either have a hand sanitizer station installed or not (binary decision variable) to maximize the number of people who use those doors with hand sanitizer stations (objective), subject to a limited hand sanitizer budget.

5.5.3. Practical considerations for adaptation

A toolkit-style adaptation is useful when the surprise event causes a new decision to need to be made, there is enough time in the build-up to the decision (or the decisions are adaptable) that a model can be implemented, the decision clearly fits within an existing optimization modeling framework that can be quickly adapted to create a specific model to aid users, and the modelers and users can collaborate.

Throughout the process, integrated development with users is critical. It may be worthwhile to consider an iterative approach to model development that incorporates qualitative interviewing of stakeholders (O’Brien et al., 2022). The process of developing model iterations was conducted in a feedback loop with the Facilities Department review and input at many stages. Model development itself can be expedited by using an existing modeling framework.

Modelers should also seek to understand the full range of users and stakeholders and coordinate to the degree possible. There may be multiple stakeholders seeking to address the new decision. With hand sanitizer stations, “Some of the individual departments or colleges had previously gone out on their own and put in sanitizer stations.” Because the decision is new, there may not be a mechanism for implementation of the model recommendations, and modelers should take care to discuss this process with users.

The context may continue to change after the surprise event. For example, the university began to record door accesses and these were able to be used as a proxy for demand in the location model. In addition, response recommendations may change. As the pandemic progressed, small hand sanitizer pumps were also placed in each classroom. These additional resources affected the candidate locations for the pumps; the decisions shifted from being classroom-centric to locations near exterior doors.

5.6. Creating an entirely new model

5.6.1. Definition

A new decision needs to be made due to the surprise event and an entirely new model needs to be developed in order to effectively address it.

5.6.2. Overview and examples

This adaptation involves examining the environment after the surprise event, realizing that a new decision needs to be made that has never been made before, the decision not clearly fitting into an existing modeling framework, and creating a new model to address these decisions in collaboration between modelers and users.

In Summer 2020, Clemson University administrators were faced with an uncertain environment; students had already registered for courses for Fall 2020 based on the expectation that conditions would be “normal.” In response to concerns about exposure to COVID-19, both students and faculty were given the option to be in class or be online, but political pressures meant that not all courses could be moved online. All faculties teaching in-person needed to engage in hybrid teaching. Rough estimates of classroom capacities indicated that not all currently registered in-person students would be able to attend class in a socially distant manner as originally scheduled. At this point, the administration determined that “something else had to change … students would not be able to attend every class meeting in person, but instead, students would only be allowed to attend a subset of class meetings in person.”

In early Summer 2020, the Facilities Department had not yet determined the actual classrooms for Fall 2020 courses, but modelers started exploring how to model the situation:

So I knew [that the facilities people] knew that we weren’t going to be able to put everyone in the same rooms, because the capacity was going to be wonky. But we didn’t know what the capacity was going to be, because it took a good while to actually map out the rooms and figure out how many rooms [were available].

The modeling team worked simultaneously on three modeling approaches: a graph theoretic approach, an integer programming approach and a heuristic approach:

I kept trying to do a graph partitioning approach. I was just constantly trying to create partitions of students and find natural partitions and it was a disaster.

After some iterations between the modeling team and the administration—the users (Gore et al., 2022), a new integer program was developed and utilized in Fall 2020 and Spring 2021 to assign students to days in which they could attend courses in-person:

We were creating this top-down assignment scheme, and then we had a town hall. Somebody asked the question “how will I know what day my students will attend my class” and we already had an answer. We’re going to give you that list of information.

The simplified hierarchy of university course scheduling problems presented by Barnhart et al. (2022) does not include any sort of assignment of students to attendance days, illustrating the degree to which this presented an entirely new decision in response to COVID-19.

5.6.3. Practical considerations for adaptation

The situations in which an entirely new model may be needed echo those in which a toolkit-style adaptation may be appropriate with the distinction being that the decision does not fit well enough within the context of a known OR model.

The importance of integrated development with users applies here. In this example, the modelers attempted to develop models based on several different frameworks but did not explicitly utilize an existing model. In contrast with the sanitizer station project, the rotational attendance project did not lend itself to iterative improvement since it would not be easy to adapt the assignment of students to in-person class days; moreover, many stakeholders (faculty, students) were unavailable to provide feedback during model development. This is an artifact of the type of decision that needed to be made, not the fact that a new model was needed.

6. Categorization of or-informed adaptations at other universities

In this section, we apply our categorization of observed uses of OR models in fundamental surprise events to published papers at other universities during COVID-19. A sample of six optimization models is presented in Table 3. We selected these papers based on the fact that our categorization of observed uses was focused on areas where optimization is frequently applied and, therefore, were similar in nature to the approaches deployed at Clemson University. They represent adaptation categories 3–5. We did not find papers that focus on adapting data or adding constraints, possibly reflecting a publication bias in terms of the requirements necessary to publish OR models in practice or the fact that OR models were not in frequent enough use before COVID-19 to allow for categories 1 and 2.

Table 3. Categorization of a sampling of OR papers for university response to COVID-19 within our framework.

					Type of Adaptation
Reference	University	Problem	Model/approach	Implemented?	Adapt data	Add constraints	Model switch	Model toolkit	New model
Murray, 2020	University of California, Santa Barbara	Classroom physical distancing (capacity and layout)	Location modeling	Yes				x
Chen et al., 2021	Michigan	Bus operations to reduce transmission (locations, routes, schedules)	MIP; simulation	Yes			x
Greenberg et al., 2021	Cornell	Classroom seating layout	IP and computer vision	Yes				x
Johnson & Wilson, 2022	Oklahoma State University	Assigning hybrid classes to classrooms while considering rotations of students	IP	Yes				x
Navabi-Shirazi et al., 2022	Georgia Tech	Decide class modality; assign classrooms for in-person classes	MIP	Advisory					x
Barnhart et al., 2022	MIT	Term planning (which terms to offer courses in) and timetabling (scheduling the courses)	IP	Advisory and implemented					x

The papers address classroom layout decisions, classroom assignments and modalities, term planning and timetabling, and bus operations. One paper is classified in the “model switch” adaptation category (Chen et al., 2021). The bus-related decisions that the paper addresses, i.e., locations of hubs, stops, and routes, were also made pre-pandemic. However, COVID-19 changed the context surrounding these decisions; transit needed to be revisited with transmission risk in mind. In contrast, the classroom seating layout models (Greenberg et al., 2021; Murray, 2020) address new pandemic-induced challenges. Because they use facility location modeling tools, we classify them as “model toolkit” style adaptations. Similarly, the work of Johnson and Wilson (2022) utilize the classic assignment problem to analyze tradeoffs that arose during the pandemic to assign classes to classrooms. Class modality integrated with classroom assignment was new and addressed by Navabi-Shirazi et al. (2022) and Barnhart et al. (2022).

Many of the papers in Table 3 were used in an advisory manner to guide decisions leading up to the Fall 2020 and Spring 2021 semesters. This was especially true for one-time decisions for each semester (e.g., classroom layouts, course scheduling). The models were used to support an adaptation by the university but are not necessarily designed to be adaptable themselves. In some ways, it appears that fundamental surprise events are particularly well-suited to incorporate models into decision-making environments since the modeler-user incentives are aligned to understand the impacts of decisions within a new environment. The response to COVID-19 also allowed iterations between the modelers and users in many instances. However, the unique characteristics of COVID-19 may have driven the ability of such modeling efforts as well: there was a sufficiently long lead-time to the fall semester of 2020 from when the event occurred and there was a significant desire by many universities to move away from the initial response that occurred in the spring of 2020. The time between the onset of the event and when new decisions were made impacts the ability to build new models or incorporate existing models into decision-making.

7. Discussion

The categorization described in this paper can be used as a roadmap for leveraging modeling capability to support adaptation in response to fundamental surprise. We view adaptation to these events from a cognitive systems perspective, centered around the user, the modeler, the model itself, and the external dynamics observed in the world. Operationalizing the categorization requires designing for interaction between each of these components and requires the modeler and the user to interface with each other in the context of external dynamics being monitored. The modeler then matches the state of the world to the appropriate category of the modeling categorization to determine the level of adaptation required (Table 2). The user provides input and feedback to the modeler based on their decision needs and to refine the real-world representation for the modeling, including the level of model adaptation. These interactions are particularly pressing in the context of fundamental surprise—where the external dynamics are fundamentally altered from status quo.

Figure 1 represents these notional relationships, including the feedback loop between the modeler and user, and the adaptive loops between each pair of entities. The modeler and user interact with the external dynamics of the world and then subsequently interact with the model. This addresses one of the issues with solely relying on resilience analytics raised by Eisenberg et al. (2019) where data from the world (external dynamics) is directly fed into models without understanding context. Figure 1 is a specific application of the framework of Eisenberg et al. (2019) to how OR models can be used during the response to surprise events, supported by the evidence we gathered in creating our categorization.

Figure 1. Interactions between the user, modeler, OR model, and the external dynamics in the real world. This is a specific application of the general framework of Eisenberg et al. (2019) presented in their Figure 1, p. 1873 to OR model usage.

Sharkey et al. (2021) discussed a research agenda for the OR community that would allow it to continue contributing to resilience, which includes: models and/or methods that enable improvisation and methods for model switching. We found instances where improvisation and potential model switching occurred during the response to COVID-19 at Clemson University. Users improvised model inputs such as parameters, such as classroom capacities, and resources, such as altering the set of spaces that were considered classrooms. Model switching could have occurred as hand sanitizer was deployed to cover all of campus at minimal cost rather than cover as much of campus as possible under cost constraints (note that the focus of the deployment had changed; however, OR modelers did not provide decision-makers with a location model focused on minimizing cost). These observations may help to identify modeling guidelines to better address graceful extensibility and adaptability since we have empirical evidence of how improvisation occurred across the modeler and user.

An important takeaway from our analysis is that OR can play an important role in the response to fundamental surprise events by appropriately understanding the improvisations and/or adaptations that are being made in response. This may require modelers to (quickly) collaborate with users to understand the type of support the users, or other stakeholders, are seeking in responding to the event and defining the scope of the model appropriately (Barnhart et al., 2022; O’Brien et al., 2022). It also implies that OR should understand its own limitations in resilience, recognizing that resilience models for examining future events only handle situational surprise events—a critical point made by Eisenberg et al. (2019). Bridging from recommended decisions to implementation in practice should also embed user expertise and awareness not captured by the model.

OR can use the knowledge gained during fundamental surprise events to help identify considerations that should be built into models to expand the set of models available to its end users. Taking COVID-19 as a case study, safety considerations became critical in the operations of systems and existing literature on supply chain disruptions did not consider this. There were not many models that incorporate safety considerations that were available, so this could be an area of future research to help support other pandemic-based surprise events.

Our results are limited by the fact that we focused only on the use of models at Clemson University during its response to COVID-19. While we do observe similar OR model uses at other universities during the pandemic, the empirical evidence upon which our categorization is based is only from Clemson University. Therefore, the observed uses may not be exhaustive since there could be other styles of modeling adaptations. Further, there is overlap between the authors of this paper and the modelers at Clemson University. We do not feel that this significantly impacts the categorization or the empirical evidence supporting it. Given the infrequency of fundamental surprise events, we would argue that our approaches to gather empirical evidence help to deepen our understanding of how OR models can be used in the response to these events.

8. Conclusions

This paper focused on the research question: How can OR models be used to improve resilience to fundamental surprise events? In order to answer this question, we provided a categorization of observed uses of how OR models were deployed to university operations responding to the fundamental surprise event of the COVID-19 pandemic. Although we cannot claim that our categorization is exhaustive, it does provide important contributions in that it provides examples, based on interviews with modelers and users, on the real-world deployment of OR model usage during a fundamental surprise event. The contributions of this paper are that it provides rigorous, qualitatively-grounded support for how OR models were adapted, created, and improvised during the COVID-19 pandemic at a university, thus building on the initial framing done by Eisenberg et al. (2019). The proposed categorization helps to better understand how models can still impact events that cannot be anticipated or modeled in advance. It helps to provide a research agenda on how to best enable adaptations and improvisations by modelers and users to address fundamental surprise events.

There is several future research directions that could be pursued based on this work. First, OR modelers could use issues arising during COVID-19, such as safety considerations, to build out the modeling toolkit. This would shorten the time required to address similar issues should they arise in future events since the toolkit-style adaptation would replace the create a new model adaptation. Second, research could be done on how to design interfaces between the user and the model to enable their ability to quickly implement the adapting data and adding constraints adaptations. Third, research could be done to understand how our categorization could be applied across an event-continuum characterized by levels of surprise.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This research was partially supported by Clemson University through a seed grant.