Here’s something we prepared earlier: Development, use and reuse of a configurable, inter-disciplinary approach for tackling overcrowding in NHS hospitals

Abstract

Overcrowding affects hospital emergency departments (ED) worldwide. Most OR studies addressing overcrowding develop bespoke models to explore potential improvements but ignore the organisational context in which they would be implemented, and few influence practice. There is interest in whether reusable models, for ED crowding and in healthcare generally, could have more impact. We developed a configurable approach for tackling ED overcrowding. A reusable queuing model for exploring drivers of ED performance was augmented by a qualitative approach for exploring the implementation context and a generic framework for assessing the likely compatibility of interventions with a given organisation. At the hospital where the approach was developed it directly informed strategy. We describe reuse of the approach at three hospitals. One project was completed and well-received by hospital management, two were terminated partway when data problems surfaced. The primary contribution of this work is its novelty in considering, alongside quantitative modelling, evidence-based interventions to overcrowding and qualitative assessment of a hospital’s aptitude and capability to adopt different interventions. A secondary contribution is to further the debate on model reuse, particularly by introducing more complex, modelling-centred approaches that acknowledge how models must relate to tangible interventions with reasonable prospects of being adopted locally.

Keywords:

• Practice of OR
• health services
• queuing

1. Introduction

Emergency department (ED) overcrowding (physical congestion and/or delays) affects hospitals worldwide and can adversely impact the quality and safety of care, staff morale and cost (Boyle & Higginson, 2018). Time-based targets for ED stays have been introduced in many healthcare systems in an attempt to reduce overcrowding, for example until recently hospitals in the UK were judged against a “4-hour target” that at least 95% of patients attending an ED must be admitted or discharged in under 4 hours (Campbell et al., 2017). However, the evidence for time-based targets improving quality of care is relatively weak (Jones et al., 2021).

Interventions to reduce ED crowding are often categorised as tackling input factors (e.g., diversion to primary care), throughput factors (e.g., patient streaming) or output factors (e.g., increased inpatient capacity) (Asplin et al., 2003). The evidence for such interventions varies, and studies reporting their success often lack sufficient information on the context of implementation for others to understand whether it would work for their system (Morley et al., 2018). Additionally, guidance on reducing ED crowding often promotes a single intervention or a vast range of interventions, without support for organisations to prioritise those most relevant to their hospital (NHS England, 2015; NHS Improvement, 2017). Configurable solutions appropriate and useful for different sites are required (Hurwitz et al., 2014).

The majority of papers on ED crowding in the OR literature report case studies of modelling a single hospital’s ED and its specific operating conditions (Salmon et al., 2018), with the alternative operating scenarios explored limited to those already being considered by stakeholders locally. Modellers tend not to consider the appropriateness of interventions to the local organisational context or draw on the existing evidence base for their effectiveness in the wider health services research literature. This may help to explain why very few recommendations from ED modelling studies have been implemented (Salmon et al., 2018; Mohiuddin et al., 2017).

In addition to this preponderance of bespoke models, there has been interest in developing ED models to study overcrowding and possible solutions in multiple hospitals using fewer modelling resources (Centeno et al., 2003; Eiset et al., 2016; Fletcher et al., 2007; Rashwan et al., 2015; Salmon et al., 2018). Such generic or configurable models are appealing in a system like the NHS where EDs have broadly similar objectives, demands, resources and operating structures. Fletcher et al. (2007) describe a generic ED simulation model developed to inform national policy on tackling ED delays in the NHS and its subsequent use to help individual hospitals improve ED performance. They found that the biggest challenges of using a national model for local use were issues such as data quality, organisational dysfunction, hospitals’ motivations for engaging in the modelling and the number of recent and ongoing improvement initiatives at the ED rather than the generic nature of the model.

Fletcher and Worthington (2009) compare generic and bespoke models in the literature and their success in improving ED performance. They found important differences in the project initiation, design and build, data requirements, validation and implementation of generic models for central use compared with configurable models and bespoke models. Mahdavi et al (Mahdavi et al., 2013) found that few healthcare modelling and simulation studies met their defining criteria for a generic model, whilst others note scarce model reuse (Günal & Pidd, 2010; Sobolev et al., 2011). The argument has been made that reusable models could enable quicker, more standardised and widespread uptake of OR approaches in practice (Jun et al., 1999; Taylor et al., 2013).

Our research makes two distinct contributions in this area. Firstly, we report the development of a novel, configurable approach for characterising the underlying causes of ED overcrowding for a given hospital and identifying evidence-informed interventions most likely to alleviate the problem given the hospital’s aptitude and capability to adopt different interventions. The approach is interdisciplinary, centred on a generic framework for identifying evidence-based interventions likely to be compatible with a given organisation based on findings from a reusable queuing model and a qualitative assessment of the implementation context (Crowe et al., 2019). Secondly, we contribute to the on-going debate about the potential for model reuse by introducing, through case study applications of the approach in the hospital where it was developed (primary case study) and three further hospitals (secondary case studies), a more complex view of reuse that acknowledges how models must relate to tangible interventions with reasonable prospects of being adopted.

In what follows, we first describe the development of our configurable approach, comprising a generic compatibility framework, reusable queuing model and qualitative research at an initial hospital. We then present the primary and secondary case study applications of the approach. Finally, we discuss the insights generated through the research, with reference to the extant literature on the challenges and benefits of translating models to different settings.

2. Conceptual approach

Our approach was developed when several of the authors worked in a multi-disciplinary embedded research team (Vindrola-Padros et al., 2017) at a large UK hospital that had tried numerous initiatives to tackle ED overcrowding with limited success. We were tasked with determining which interventions for reducing ED overcrowding were likely to be most effective and feasible for the hospital.

Our interdisciplinary approach comprised:

A generic compatibility framework for analysing the feasibility and likely impact of evidence-informed interventions;

A reusable mathematical model of ED performance;

Qualitative research exploring organisational practices and attitudes.

2.1. Generic compatibility framework

Firstly, a list of evidence-informed interventions for tackling ED crowding reported in the literature was established. Review articles relating to interventions published between January 2000 and July 2017 on PUBMED and google scholar were screened for relevance by title, then abstract, then full text. In total 32 relevant articles were identified. For each intervention identified, we then extracted the following information from the review articles:

•  • The aspect of the system targeted (e.g., input, throughput or output factors, or specific areas of the ED such as triage);

•  • The patient subgroups targeted (e.g., low acuity patients);

•  • Any details of the context(s) in which the intervention had been applied;

•  • Any potential barriers to or enablers of effective implementation.

This formed the basis of a generic “compatibility framework” (see Supplemental online material A) designed for identifying evidence-informed interventions that might be compatible with a given organisation based on their capacity to adopt different types of intervention and the nature of the problems driving their ED overcrowding. To apply the compatibility framework at a given hospital would require an end user to:

•  • Apply a reusable mathematical model (described below) to assess the likely impact of each intervention on the ED crowding problems they face;

•  • Interview hospital staff using a reusable topic guide (described below) to identify: relevant potential barriers to, or enablers of, adoption of each intervention; what, if any, attempts had already made to implement each intervention (or their plans to do so).

2.2. Reusable mathematical model

A mathematical queuing model was developed to explore the intrinsic limits and drivers of ED performance under current operating conditions and estimate the likely impact on ED performance of changes to external factors such as arrival rates and downstream capacity. Stolletz’ approximation method was adapted to the case of queueing networks (Stolletz, 2008), enabling us to incorporate the desired level of detail in the model without recourse to pseudo-random observations as in a simulation. Full details of the model are given in Supplemental online material B, and an assessment using simulation of the robustness of our quantitative findings to key modelling assumptions is given in Supplemental online material C. The model was designed to be generic and flexible for reuse in different settings, e.g., accommodating different ED areas, and was implemented in R as this is the language used most by NHS analysts. The R code is publicly available for download via the Zenodo repository (Grieco et al., 2023) and enables users to set up key model features and conduct scenario analyses.

As detailed in Supplemental online material B, the ED was modelled as a queueing network with each node representing a potential patient location: triage, in-house general practitioner (GP), Minors (for less severe conditions), Majors (for more severe conditions) and the resuscitation area (Resus) (see Figure 1), each characterised by a capacity (e.g., the number of beds or cubicles available). Whilst the nodes and patient flows in Figure 1 were informed by discussions with ED clinicians at the hospital where the model was developed, the model can be configured to include alternative ED areas or pathways relevant to a different setting.

Figure 1. The model structure, which treats each area of the ED as a “node” in a queueing network. GP - in-house general practitioner; Resus - resuscitation area; ED - emergency department.

Patients are assumed to undergo one or more of a set of activities while at a given area and the subsets of activities will differ from patient to patient. Activities correspond either to services (e.g., assessment by an ED or specialist clinician), or waiting for something external to the ED (e.g., waiting for a specialist clinician or a bed to become available in the hospital). Whether the activities at an area occur in sequence or in parallel can vary between patients. All patients following the same sequence of ED areas and same set of activities within each area form a patient type. Before entering a given ED area, a patient waits until a place becomes available and their waiting time depends on the mix of patients already present in that area. This gives a measure of the time a patient spends “blocked” during their stay in the ED.

Table 1 summarises the parameters of the model and how values for each were estimated for our primary case study (see Supplemental online material D for further information). The majority were determined through analysis of routinely collected data for patients attending the ED from 01/11/2016 to 31/03/2017, with a smaller set elicited through discussions with ED staff.

Table 1. A description of the parameters required for the mathematical model and a summary of model parametrisation in the primary case study (see Supplemental online material D for further details).

Model parameter	Description	Parametrisation in the primary case study
Patterns of arrival	The average number of patients arriving at the ED each hour of the day is required for: a baseline (“typical day”) scenario; a “low” arrival scenario; a “high” arrival scenario. Arrivals at each hour of the day need to be stratified by: the area of the ED the patient visits (e.g., Majors, Minors, Resus); mode of arrival (“walk-in” or ambulance); whether or not the patient is later admitted to the main hospital.	The stratified average number of patients arriving at the ED each hour of the day was determined from the routine data for: the days with a total arrival rate in the middle 50%; the 25% of days with the lowest total arrival rate; the 25% of days with the highest total arrival rate. Simplified analytical versions of these three patterns of arrival were derived (using piecewise linear functions with the same total daily arrivals as those obtained from the data) to use as inputs to the model for the baseline, low and high scenarios (see Supplemental online material D).
System capacities	Capacities are required for each of the nodes in the model, i.e., areas of the ED, which we assume to be the maximum number of patients that can be in that area at the same time. This is the number of cubicles or beds, for example. Capacities can vary by hour of the day.	The following system capacities were assigned in consultation with ED staff: Triage = 3; GP area = 1; Minors = 8; Majors = 16; Resus = 5. These were assumed to be constant throughout the day (but accounting for 8am-10pm GP service).
Activity durations	Average activity durations are required for: clinical assessments in each of the nodes of the model (e.g., triage assessment, GP assessment, assessment in Minors etc.); assessments by a specialist clinician (in any area of the ED); admission processes (e.g., paperwork and porter transfer); diagnostic tests (MRI, CT, ultrasound, x-ray). Activity durations are assumed to be constant throughout the day.	The following average activity durations were assigned in consultation with ED staff: triage assessment = 5 min; GP assessment = 10 min; assessments by an ED clinician in Minors/Majors/Resus = 15/30/45 min respectively; assessments by a specialist clinician = 30 min; admission processes = 15 min. From the routinely collected data, the average time to conduct a diagnostic test was determined to be 10.8 min.
Time-varying average waiting times	Average waiting times for patients arriving at each hour of the day are required for each of the exogenous factors considered in the model: waiting for a specialist assessment; waiting for diagnostic tests; waiting for a hospital bed to be available; waiting for additional processes before transfer to an available hospital bed (e.g., waiting for porters).	The waiting times for patients arriving at different times during the day were estimated from the data for specialist assessments, bed availability and pre-transfer processes (the latter required a number of simplifying assumptions), see Supplemental online material D. Based on discussions with ED staff, the average waiting time for diagnostic tests at any time in the day was set at 20-minutes.
Proportion of patients assessed by ED clinicians and/or specialists	The model requires the proportions of patients assessed only by an ED clinician, only by a specialist clinician, and by both, each stratified by model node (i.e., ED area in which the clinical assessment took place) and whether the patient is admitted to hospital.	Supplemental online material D shows the proportions used based on analysis of routinely collected data and some simplifying assumptions agreed with ED staff.
Sequences of activities	The model contains a parameter for the proportion of activities that are carried out in parallel rather than sequentially, e.g., whether patients undergo diagnostic tests while they are waiting for a bed to be ready rather than waiting for a diagnostic test to be complete before requesting (and waiting for) a bed.	The routine data did not provide enough information to estimate this, so through consultation with an ED clinician we assumed that: assessment by an ED clinician or GP is never carried out in parallel to any other activity, and that all other activities happen fully in parallel in 75% of cases and sequentially in the remaining 25% of cases.

The following ED performance measures are generated by the model:

•  • The percentage of patients discharged or admitted to hospital within 4 hours (cf 95% target);

•  • average time to initial assessment (i.e., the wait to be triaged);

•  • average time to treatment;

•  • average length of stay in the ED;

•  • expected number of patients in the system at any given time.

The baseline model was calibrated to replicate observed system performance at the primary case study hospital under low, central and high observed arrival rates, and the mechanisms of the model were deemed to have clinical face validity by an ED consultant at the hospital.

The model allows “what-if” scenario analyses to assess how changes in the following exogenous factors affect the ED performance: arrival rates; waiting time for a specialist clinician; waiting time for diagnostic tests; waiting time for an available hospital bed; waiting time for admission to hospital. In addition to halving and doubling exogenous waiting times, we also ran scenarios with average waiting times for a bed or specialist clinician set at the hospital’s target times (60 mins and 30 mins respectively).

Scenario analyses associated with specific interventions included: decreasing ambulance arrival rates (e.g., through diversion policies); decreasing GP/Minors arrival rates (e.g., through diversion to alternative services such as walk-in centres); increasing ED capacity (overall, and separately in Minors, Resus and Majors).

2.3. Qualitative research

Consented, recorded and transcribed semi-structured interviews were conducted with a purposive sample of staff from the primary case study’s ED (n = 15, Table 2) and related departments (n = 12, Table 3). A topic guide developed and piloted in the first three interviews was used thereafter. Interview transcripts were imported to NVivo and analysed using thematic analysis (Silverman, 2016) in conjunction with data obtained from meeting observations and staff shadowing (see below). Analysis focused on identifying the potential factors that contribute to overcrowding, the main links of the ED with other hospital departments, and how staff members (in the ED and other areas) respond to overcrowding.

Table 2. ED staff interviewed.

Professional group	Number of interviews	Details
Nursing	7	Sister, band 6s, band 5s, matron. Working in: Minors, EDU, AAA, Resus, AC, Triage
Doctors	5	GP, consultants, registrars. Working in: Minors, EDU, AAA, Resus, AC, Triage
Managers	3	Divisional manager, ED manager, bed manager
Total	15

EDU - emergency day unit; AAA - acute assessment area; AC - acute care.

Table 3. Staff in non-ED departments and services interviewed.

Areas	Number of interviews	Details
Imaging	2
Pharmacy	1
Acute Medical Unit	2	Ward sister, clinical lead
Operations centre	1
Discharge team	1
Specialities	4	Representatives from: general surgery, medicine, paediatrics and gynaecology
Ambulance services	1
Total	12

Structured observation guides were developed and used in observing ED handovers and nursing huddles, senior leadership meetings, ED team meetings, and hospital-wide meetings (n = 20 h over 19 observations, see Table 4). Prior permission and verbal consent were obtained from meeting chairs and participants respectively. Five members of ED staff were shadowed with signed consent (2 nurses, 2 consultants and one healthcare assistant). Shadowing took place at different times of the day and different days of the week to capture a range of ED operating conditions (n = 10 h in total, spread across three mornings and two afternoons). Data were imported into NVivo and analysed thematically (Silverman, 2016) with a focus on understanding current operating conditions, patient journeys and key interconnections between the ED and other hospital areas.

Table 4. Meeting observations.

Type of meeting	Recurrence	Number of observations
Handovers / nursing huddles	Weekly	10
Senior leadership meetings	Fortnightly	7
ED meetings	Monthly	1
Special events / Trust-wide meetings	As relevant	1
Total		19

Recognising that this detailed, time-consuming and highly skilled qualitative research would not be easy to replicate in other settings, we used the findings to design an interview topic guide for use by operational researchers or hospital analysts, clinicians or mangers to assess aspects of the organisational context that might hinder or aid specific ED crowding interventions (see Supplemental online material E).

3. Case studies

Our primary case study describes the application of our approach in the hospital where it was developed, which was an NHS teaching hospital in a major city with an ED struggling to meet performance targets and where several of the authors were working in a multi-disciplinary embedded research team. Our secondary case studies describe the reuse of our approach in three additional NHS teaching hospitals failing to meet ED performance targets, one in a major city, another in a medium-sized city and the other in a rural location.

3.1. Primary case study

3.1.1. Findings from mathematical model

Table 5 shows the estimated system performance of our primary case study hospital in our baseline scenario and each of our “what-if” scenarios in terms of the 4-hour target, average ED length of stay and average time to treatment, for different patient groups. Figure 2 shows the impact on the expected number of patients in the ED over the course of the day under different scenarios. Table 6 and Figure 3 show performance against the 4-hour target and the expected number of patients in the ED over the course of the day in those scenarios related to particular interventions (see Supplemental online material F for further detail and additional results).

Figure 2. The expected number of patients in the ED over the course of the day for our baseline scenario (black) and what-if scenarios.

Figure 3. The expected number of patients in the ED over the course of the day for our baseline scenario (black) and for the scenarios involving fewer GP/Minors patients (squares) and greater capacity in Minors, Majors and Resus (triangles).

Table 5. Modelling estimates of the relative influence of different external factors on the ED performance measures generated by the model for the primary case study. Current averages (italics) refer to the daily average of the time-varying parameters used in the baseline scenario.

Scenario		% Patients leaving ED within 4 hrs					Average length of stay in the ED (minutes)					Average time to treatment (minutes)
		GP	Minors	Majors	Resus	Overall	GP patients	Minors patients	Majors patients	Resus patients	All patients	GP patients	Minors patients	Majors patients	Resus patients	All patients
Baseline scenario		100%	86%	75%	66%	82%	24	130	185	235	145	12	81	46	27	61
ED arrival rate (Current: 12.5 patients per hour)	Low	100%	95%	82%	68%	90%	22	84	151	220	109	10	35	11	12	24
	High	100%	77%	62%	60%	73%	25	175	249	258	192	14	127	110	51	109
Average waiting time for hospital bed (Current: 160 mins)	60 mins	100%	98%	91%	82%	95%	24	71	101	140	81	12	31	4	4	19
	Halved	100%	96%	89%	80%	93%	24	78	110	154	88	12	37	5	6	23
	Doubled	100%	72%	45%	45%	64%	24	205	396	373	263	12	149	222	109	161
Average waiting time for specialist assessment (Current: 74 mins)	30 mins	100%	92%	80%	68%	88%	24	100	159	221	120	12	56	28	23	42
	Halved	100%	91%	80%	68%	87%	24	104	162	223	123	12	59	30	23	44
	Doubled	100%	80%	69%	63%	77%	24	160	216	247	172	12	108	68	32	84
Average waiting time for diagnostic tests (Current: 20 mins)	Halved	100%	91%	77%	67%	86%	22	108	177	229	130	11	62	40	25	49
	Doubled	100%	81%	73%	64%	79%	25	155	193	240	162	13	103	52	30	75
Average waiting time for admission (Current: 5 mins)	Halved	100%	87%	76%	66%	83%	24	127	181	231	142	12	79	43	26	59
	Doubled	100%	86%	74%	65%	82%	24	132	189	238	148	12	83	48	29	64

Table 6. Model estimates for the primary case study of ED performance against the 4-hour target, overall and for each of the ED areas (GP, Minors, Majors, Resus), for the baseline scenario and our scenarios related to particular interventions.

Scenario		% Patients leaving ED within 4 hrs
		GP	Minors	Majors	Resus	Overall
Baseline scenario		100%	86%	75%	66%	82%
100% overlap of activities^a		100%	90%	80%	70%	86%
50% fewer ambulance arrivals		100%	89%	82%	70%	87%
15% fewer GP/Minors patients		100%	94%	75%	66%	86%
25% fewer GP/Minors patients		100%	96%	75%	66%	86%
Increase the capacity of Minors, Majors and Resus	By 1 bed / cubicle each^b	100%	93%	78%	69%	87%
	By 25%^c	100%	96%	82%	69%	90%
	By 50%^d	100%	97%	82%	69%	91%

^aIn the baseline scenario the overlap of activities is 75%, i.e.75% of activities (excluding GP and ED clinician assessments) happen in parallel rather than sequentially.

^bCapacities: Minors from 8 to 9, Majors from 16 to 17, Resus from 5 to 6. ^cCapacities: Minors from 8 to 10, Majors from 16 to 20, Resus from 5 to 6.

^dCapacities: Minors from 8 to 12, Majors from 16 to 24, Resus from 5 to 7.

The key findings for decision-making were:

•  • The external factor primarily driving ED overcrowding is bed availability and reducing waiting times for beds could markedly improve the performance of Majors, Resus and Minors;

•  • Lower arrival rates would primarily benefit Minors patients with little benefit for Resus patients;

•  • Delays to specialist assessment have a moderate influence on ED performance, mostly within Majors and Minors (rather than Resus or GP);

•  • Reducing waiting time for diagnostic tests would have limited impact on performance other than in Minors;

•  • Some interventions may significantly reduce time-to-treatment without having a major impact on the 4-hour target;

•  • Some interventions (e.g., reducing arrival rates of low acuity patients) reduce physical congestion without having a major impact on the 4-hour target;

•  • Increasing ED capacity (across Minors, Majors and Resus) reduces congestion and could improve ED performance against the 4-hour target to some extent but with rapidly diminishing returns.

These findings from our analytical model were corroborated in our series of simulation experiments (see Supplemental online material C) and echo those in other EDs, with a review of the emergency medicine literature concluding that ED overcrowding is often driven by delays in admitting patients from the ED and by the number and type of people attending ED (Morley et al., 2018).

3.1.2. Findings from qualitative research

In addition to informing the structure of the mathematical model, the qualitative analysis identified concerns among some staff that insufficient time was given to embedding each new system or ED improvement initiative before the next change occurred, with the unintended consequence of variability in practice, poor staff morale and change fatigue:

… there’s a lot of still learning from a lot of the members of staff or lack of understanding of what that [ED operating] model is and how it works and there is a lot of change that isn’t always sustained and embedded before the next change occurs. (Manager)

Participants reported negative experiences of the physical congestion associated with ED overcrowding, including the stress generated by having patients lined up in the corridors or overflowing in the waiting areas and instances where they felt that this was not safe for staff or patients. ED staff discussed internal ED problems including the low quality of triage and challenges in referring ED patients to inpatient specialty teams or other hospitals. Non-ED staff raised concerns including the internal organisation of ED, lack of clear escalation policy and poor communication with other parts of the hospital. Staff perceptions relating to problems with policies in place at the time for escalation processes under conditions of crowding included:

•  • ED escalation policies were not linked to hospital-wide escalation policies;

•  • Imperfect awareness of and adherence to ED escalation policies among staff;

•  • Lack of inter-departmental collaboration and communication;

•  • Lack of ownership of the 4-hour performance target by non-ED staff.

3.1.3. Application of compatibility framework

Figure 4 shows the interventions identified from the literature according to the aspects of the system they target, along with summary findings from our assessment of their likely impact and feasibility for the study hospital (see Supplemental online material G for details of our assessments for individual interventions).

Figure 4. This figure is amended from Crowe et al. (Crowe et al., 2019). In the central diagram, different interventions (shown in lower case) target different aspects of the system (shown in BOLD CAPITALS) in an attempt to reduce ED overcrowding. The text surrounding the diagram summarises our findings about the potential impact and feasibility of ED crowding interventions in the primary case study.

The findings of using the compatibility framework for the primary case study hospital were presented to ED staff and hospital managers including the Divisional Clinical Director of Emergency Services, who subsequently presented them to the hospital’s Board. There was significant interest in the findings, particularly that delays for hospital beds were driving poor performance, and the work informed the hospital’s subsequent decision to focus on exit blocks (output factors), prioritising action on speeding up admission processes and freeing up beds earlier in the discharge process. Our findings of the likely feasibility of interventions suggested that many would depend on successful strategies for strengthening team dynamics and inter-departmental collaboration, and that interventions outwith the immediate control of the hospital would require them to strengthen their skills in cross-system collaboration.

3.2. Secondary case studies (reusing the configurable approach)

We attempted to implement our approach for screening ED crowding interventions in three additional case studies. Table 7 describes the initiation and intent of these secondary case studies.

Table 7. Descriptions of how projects were initiated in each of the case studies and the manner in which the approach for tackling ED crowding was adopted (or planned to be adopted) in each case.

Hospital context	Project initiation	Project approach
Primary case study NHS teaching hospital in major city ED failing to meet performance target	Research team already embedded within the hospital so strong existing relationship with clinicians and senior management ED crowding selected by senior managers as a key priority area for research to focus on Research proposal for the project agreed at a senior level within the hospital	Research designed and conducted by the hospital’s embedded research team (comprising academics from partnering university, including operational researchers and a medical anthropologist) The approach developed comprises: a reusable mathematical model; qualitative analysis of organisational context relevant to ED crowding; a generic compatibility framework
Secondary case study 1 NHS teaching hospital in major city ED failing to meet performance target	No prior history of collaboration between the research team and hospital Research team approached whilst delivering a short course on flow by a participant who was an ED consultant looking for support managing problems with ED flow The ED consultant gained support from other ED staff and agreement from senior hospital managers (e.g., Director of planning) to share data and provide analyst time Research proposal shared with the ED consultant and the hospital’s performance team only (no direct contact with senior management or other ED staff)	Translation of approach used by the original research team at primary case study hospital, with plan for: An analyst in the hospital’s performance team to identify and process relevant data to populate the mathematical model The analyst to run the reusable mathematical model with support from the original research team The original research team to use a modified qualitative approach(reusable interview topic guide) to identify potential barriers and facilitators of ED crowding interventions The original research team to apply the compatibility framework in collaboration with hospital staff to assess the potential impact and feasibility of different interventions
Secondary case study 2 Large NHS teaching hospital in medium-sized city ED failing to meet performance target	No prior history of direct collaboration between the research team and hospital Existing connections between research team and an operational research team connected with the hospital, who proposed the approach to an ED clinician they had worked with previously Research proposal shared with the ED consultant, who later sought support from other ED staff and senior hospital management	Translation of approach used by the original research team at primary case study hospital, with plan for: The operational research team connected with the hospital to utilise existing data they had to populate the mathematical model, with support from the original research team and input from an ED clinician The operational research team connected with the hospital to run the reusable queuing model with support from the original research team The original research team to use a modified qualitative approach(reusable interview topic guide) to identify potential barriers and facilitators of ED crowding interventions, in collaboration with the operational research team The original research team to apply the compatibility framework in collaboration with hospital staff to assess the potential impact and feasibility of different interventions
Secondary case study 3 NHS teaching hospital in rural location ED failing to meet performance target	No prior history of direct collaboration between the research team and hospital Existing connections between research team and an operational research team connected with the hospital, who proposed the approach to an analyst in the hospital strategy team who they had worked with previously Research proposal shared with the analyst in the hospital strategy team, who gained support from an ED service manager and permission to share data	Translation of approach used by the original research team at primary case study hospital, with plan for: An analyst in the hospital’s strategy team and another in the information services department to identify and process relevant data to populate the mathematical model, with support from the original research team The two analysts to run the reusable queuing model with support from the original research team and operational research team The operational research team to use a modified qualitative approach (reusable interview topic guide) to identify potential barriers and facilitators of ED crowding interventions, with support from the original research team The operational research team to apply the compatibility framework in collaboration with hospital staff to assess the potential impact and feasibility of different interventions (with support from the original research team)

The secondary case study hospitals had similar ED structures and patient flows, with small differences accounted for in the parametrisation of the model. For secondary case study 2, there was a case for introducing additional structure to the model to incorporate a pronounced difference in waiting times for beds on an observation unit compared to an inpatient ward (a difference not experienced at the primary case study hospital). We chose to note this limitation to stakeholders rather than make structural amendments to the model. The model was re-calibrated to reproduce observed performance at the hospital in secondary case study 2, and an ED clinician there attested to its clinical face validity.

The generic compatibility framework developed and applied in the primary case study was applied in the same manner in secondary case study 2 after incorporating findings from literature published in the intervening months (nine additional relevant articles were identified using the same search criteria). The interview topic guide developed using insights from the qualitative research in the primary case study was also used in secondary case study 2. Case studies 1 and 3 were terminated before the qualitative and compatibility framework stages of the approach.

Table 8 summarises our engagement in each case study, along with the project outcome and factors that potentially influenced that outcome, drawing on the work of Fletcher et al. in categorising key barriers and stages of the consultancy process completed in each case (Fletcher et al., 2007).

Table 8. A summary of the engagement involved in each case study, the outcome and the key factors that potentially influenced that outcome.

Case study	Engagement	Outcome and stages of the consultancy process^a completed	Potential influencing factors on project outcome^b
Primary case study	Regular meetings and email correspondence with an ED consultant throughout the project (e.g., contributing to model structure regarding patient pathways, expert view on model parameters not available from data, sense-checking emerging findings). Close collaboration with an ED nurse in gaining access to meeting observations/shadowing and identifying participants for the qualitative research. Input in the later stages of the project from an ED manager about the hospital’s historic and planned use of interventions for tackling ED crowding. Straightforward access to data through existing honorary contracts and strong support from IT/performance team. Regular update meetings with senior hospital managers on the advisory group of the embedded research team, e.g., identifying how the research might feed into the hospital’s strategy. Presentation of final results to senior hospital managers and the hospital’s board.	Outcome: The research was completed and the findings well received by senior management in the hospital. The work informed the hospital’s implementation of improvement strategies, but there was no evaluation to see whether the predicted improvements were made. Stages of the consultancy process (Fletcher et al., 2007) completed: Initial demo of the generic model to key stakeholders in the hospital (A&E consultants, A&E managers, nurses, performance directors, analysts); Work with local analysts to get accurate data; Replicate the performance of the hospital from the key parameters with the model and agree with hospital; Run “what if” scenarios on potential improvement strategies; Implement improvement strategies.	Key enablers: Research team was already embedded within the hospital so had strong existing relationship with clinicians and senior management ED crowding was selected by senior managers as a key priority area for research to focus on Research proposal for the project agreed at a senior level within the hospital Strong staff engagement throughout Key barriers (based on those observed by Fletcher et al. (2007)): Changes in A&E departments: Before, during and after the research was conducting, the hospital tried numerous means to improve ED performance, often at the same time, which would have made evaluating the impact of strategies informed by the research challenging.
Secondary case study 1	Several meetings and email exchanges with the hospital’s deputy lead for performance regarding data requirements and availability, how to run the model, and sharing illustrative results from model scenario analysis. Initial meetings with an ED consultant about the scope of the modelling and illustrative results from model scenario analysis. No direct contact with any other ED staff or senior hospital managers (their support for the work was established indirectly via the ED consultant and deputy lead for performance). Plans to conduct a workshop with key hospital stakeholders to discuss modelling scenarios and establish wider buy-in did not go ahead because the project was ended before reaching that stage.	Outcome: The hospital ended the project prior to finalising the parametrisation of the re-useable model because they felt that they lacked the resources to complete the work (e.g., not enough time available from analysts to check the quality of the data or ED staff to engage and interpret the findings). Stages of the consultancy process (Fletcher et al., 2007) completed: Initial demo of the generic model to key stakeholders in the hospital (A&E consultants, A&E managers, nurses, performance directors, analysts); Work with local analysts to get accurate data.	Key enablers: Enthusiasm from ED clinician, who initiated the work. Good engagement initially from the hospital performance department. Key barriers (based on those observed by Fletcher et al. (2007)): Motivation (main barrier): Those engaged in the project saw significant potential for its usefulness, but senior managers were far less engaged and had other priorities so felt that analysts’ time needed to be focused elsewhere. Data quality (partial barrier): Much of the data required was available, but there were gaps and quality issues that required some analyst time to address. At the time, the hospital was addressing fundamental data issues and introducing a new IT system, so it was felt that this project would be more feasible after that. The data barrier would likely have been surmounted relatively easily had there been motivation to prioritise it.
Secondary case study 2	Several meetings and email exchanges with the operational research team regarding data requirements and availability, how to run the model, sharing illustrative results, and interpreting the model findings. Several meetings and email exchanges with an ED consultant about the scope of the modelling, modelling assumptions, clarifications about how data were collected and interpreting the model findings. Straightforward access to data through the operational research team. Close collaboration with the ED consultant in identifying participants for the qualitative research, and in establishing the hospital’s historic and planned use of interventions for tackling ED crowding. Indirect dissemination of the findings to senior managers at the hospital via and ED consultant.	Outcome: The research was completed and well received by senior management in the hospital, who felt that the findings were in line with their strategic view. The research team attempted to engage further with the hospital about how the research could best inform their improvement plans, but this was disrupted by the outbreak of the COVID-19 pandemic. Stages of the consultancy process (Fletcher et al., 2007) completed: Initial demo of the generic model to key stakeholders in the hospital (A&E consultants, A&E managers, nurses, performance directors, analysts); Work with local analysts to get accurate data; Replicate the performance of the hospital from the key parameters with the model and agree with hospital; Run “what if” scenarios on potential improvement strategies.	Key enablers: Operational research team had strong existing relationships with clinicians at the hospital and access to relevant data. Enthusiasm and significant input from an ED clinician, who initiated dissemination of findings to senior hospital management. Key barriers (based on those observed by Fletcher et al. (2007)): Changes in A&E departments: The hospital was already making or planning a number of changes aimed at improving ED performance, so the research findings may have had little impact other than to back-up existing plans. Motivation: Given the commitment of the hospital to existing plans, it is unclear whether the findings would have had any influence had they been counter to, rather than aligned with them. Additionally, as the research was finishing the COVID-19 pandemic began and managing that became the major priority for the hospital.
Secondary case study 3	Several meetings and email exchanges with a senior analyst in the strategy team and an analyst in the information services department regarding data requirements and availability, how to run the model, and sharing illustrative results from model scenario analysis. No direct contact with any ED staff or senior hospital managers (their support for the work was established indirectly via the analyst in the strategy team).	Outcome: The project was ended by the hospital prior to finalising the parametrisation of the re-useable model because senior managers felt that addressing the data deficiencies would take up analyst time that needed to be spent on other priorities, and also concerns that the model was overly simplistic. Stages of the consultancy process (Fletcher et al., 2007) completed: Initial demo of the generic model to key stakeholders in the hospital (A&E consultants, A&E managers, nurses, performance directors, analysts); Work with local analysts to get accurate data.	Key enablers: Strong link between the operational team and senior analyst in the hospital’s strategy team. Good engagement and enthusiasm initially from the two analysts in the hospital. Key barriers (based on those observed by Fletcher et al. (2007)): Motivation: Senior managers were not closely engaged in the project and had other priorities so felt that analysts’ time needed to be focused elsewhere. Data quality: Some of the data required was available, but there were gaps and quality issues that required some analyst time to address. These issues would likely have been surmounted relatively easily had there been motivation to prioritise them.

^aBased on a 6-stage classification of the consultancy process by Fletcher et al. (2007), which they use to assess how many of the hospitals where they tried to apply their generic ED model passed through each stage: 1. Initial demo of the generic model to key stakeholders in the hospital (A&E consultants, A&E managers, nurses, performance directors, analysts); 2. Work with local analysts to get accurate data; 3. Replicate the performance of the hospital from the key parameters with the model and agree with hospital; 4. Run “what if” scenarios on potential improvement strategies; 5. Implement improvement strategies; 6. Test if the predicted improvements were made.

^bIncluding assessment of the following four potential barriers observed by Fletcher et al. (2007) in their work applying their generic ED model at a number of NHS hospitals: Poor data quality; Organizational dysfunction (e.g., ineffective hospital management); Insufficient motivation (particularly at the point where staff time needed to be committed to the project); Changes in the ED (trying numerous means to improve ED performance so difficult to validate the model).

In two of the secondary case studies, initial enthusiasm from collaborators proved insufficient as senior managers were quick to divert analysts to tasks they felt were higher priority when it became apparent that data issues would need to be addressed. It is likely that these data issues could have been overcome with modest analytical input had the motivation been there, but in both cases the projects lacked the engagement and buy-in at a senior level that was present in the primary case study. For secondary case study 2, where for much of the project we also lacked engagement with senior hospital management, we were able to progress further because our operational research collaborators already had data to parameterise the model and we had strong clinical engagement. In the later stages of the project, senior managers were more engaged and received the findings positively, but the timing coincided with the outbreak of the COVID-19 pandemic, which disrupted further engagement about how the research could inform their improvement plans.

4. Discussion

This paper reports the development of a configurable, reusable approach for addressing ED overcrowding that combines quantitative and qualitative methods, and our findings in attempting to apply it in four UK NHS hospitals. The primary contribution of this work to the literature is its novelty in accounting for not only the underlying drivers of overcrowding in a hospital but also that hospital’s capacity to adopt different types of solutions. A secondary contribution is how our findings add to current debates on the relative merits of bespoke, generic and reusable models.

Evidence from implementation science and related fields suggest that a failure to account for organisational context (Kaplan et al., 2010) may explain the well documented failure of quantitative modelling and simulation studies of ED overcrowding to influence practice (Salmon et al., 2018; Mohiuddin et al., 2017). Of particular relevance is the absorptive capacity of the whole organisation and/or a specific department (“the set of routines and processes characterized by knowledge acquisition, assimilation, transformation, and application” (Ashoor et al., 2021)). For example, a quantitative OR model might predict a drop in ED waiting times if patients were triaged by consultants rather than nurses, but such a change would be very unlikely in practice if the consultants of that hospital were resistant to taking on what they perceived as monotonous work (Ming et al., 2016).

Our work addresses this by innovatively augmenting quantitative modelling with qualitative research methods designed to explore the potential barriers to and enablers of implementing proposed interventions and to assess an organisation’s absorptive capacity (Leslie et al., 2014). In our primary case study, the qualitative component comprised in-depth ethnographic research combining interviews with shadowing and non-participant observation of meetings, and we consider that this feature of the work was crucial to its positive reception among senior decision makers.

While the “light-touch” qualitative approach we distilled from this to make our approach readily reusable was successfully deployed by non-experts in a second case study, we should highlight the limitation that relying on interviews captures staff perceptions but not the actual practices and behaviours captured through expert observation. This and other limitations are acknowledged in the wider literature on rapid qualitative techniques in healthcare (Vindrola-Padros & Vindrola-Padros, 2018), including analysis of the trade-offs between the breadth and depth of data captured, appropriate sampling approaches, the documentation of complex processes, and the ability to capture changes over time (Vindrola-Padros & Johnson, 2020, 2021).

The other key feature of our approach is a characterisation of each commonly proposed intervention for ED crowding (see Supplementary online material A). This enables proposed solutions to be filtered, first by their potential impact on the underlying problem causing crowding at their hospital and secondly by the feasibility of successfully deploying each intervention given the hospital’s capacity to overcome the implementation challenges. This comprehensive approach goes beyond existing OR studies, which typically focus on a small set of possible changes and do not draw on the wider evidence base about those changes or alternative interventions.

While our approach directly informed plans for tackling ED crowding at our primary case study, we did not evaluate the impact on ED performance of changes made following the study, nor any changes to collaborations or team dynamics. In this sense, the work ended prior to full completion of the consultancy process as described by Fletcher et al. (2007).

The approach to addressing ED crowding was designed in response to calls for more generic modelling in OR (Mahdavi et al., 2013; Taylor et al., 2013), and a desire to create something that could be reused in other hospitals in contrast to the vast majority of ED studies that embed within a model the specific features of a particular hospital, its problems and potential solutions. The queuing model met the criteria for generic models adopted by Mahdavi et al. (2013) and the criteria for “generic multiple applicability models” set out by Fletcher and Worthington (2009).

Crucially, however, our research broadens the conceptualisation of model reuse in the OR literature by recognising that, if the intention of model reuse is to inform change in multiple organisations, each with its own strengths and weaknesses when it comes to adopting change, then a configurable quantitative model is only a partial intervention as it does not account for whether specific changes have a reasonable prospect of being adopted in practice. By augmenting our reusable queuing model with the generic compatibility framework, we introduce the notion of more complex modelling-centred interventions for reuse. Expanding the conceptualisation of reuse in this way opens the possibility of learning from the broader literature on complex interventions in healthcare, which amongst other things stresses the importance of “involving stakeholders, reviewing published research evidence (…) understanding context, [and] paying attention to future implementation in the real world”(O’Cathain et al., 2019).

Although developing an approach to support future reusability took extra time and resource than a standard ED modelling project would have, we hoped that this upfront cost would provide time savings for other hospitals as well as some consistency of output (Robinson et al., 2004). However, we experienced mixed success in our attempts to apply the model elsewhere. As Robinson et al. (2004) note, model reuse is not necessarily straightforward even when the original and reuse modelling contexts are similar, which they tend to be (at least structurally) within NHS emergency departments. In line with the findings of Fletcher et al, the barriers we faced were not related to the generic nature of the model (e.g., difficulty establishing required levels of abstraction, the importance of model (re-)validation or “force fitting” inappropriate models (Robinson et al., 2004)), but rather to local issues such as data quality, the hospital’s motivation to engage with the modelling, and the number of recent and ongoing changes in the ED (Fletcher et al., 2007). Whilst such factors are exogenous in a model-only intervention, they are a central part of the broader conceptualisation of complex interventions that we propose as useful to the debate on reuse.

Many papers in healthcare OR talk about the importance of stakeholder engagement in promoting the use of OR solutions and insights in practice, although there is a lack of consensus about the processes required for effective stakeholder engagement (Zabell et al., 2021). Facilitated workshops are one means of engaging stakeholders considered by many to be effective, particularly if tailored to different stages of the project life-cycle (Proudlove et al., 2017). For example, PartiSim (Tako & Kotiadis, 2015; Kotiadis et al., 2014) is an approach that uses a series of carefully designed formal workshops to engage stakeholders in a simulation project. Our approach is closest to the PartiSim post-model coding stages (Kotiadis & Tako, 2018), i.e., experimentation and implementation, albeit outside a formal structured workshop setting. Another example is the “Simtegr8” approach (Tako et al., 2019), which uses facilitated simulation workshops to embed the perspective of health service users as well as providers. In focusing on model reuse rather than bespoke modelling through facilitated workshops, our work offers complementary insights on engagement. For example, Jahangirian et al. (2015) report that engagement is challenging when stakeholders have high workloads, as in our case study hospitals, and model reuse helped here by enabling the quick production of quantitative results. Additionally, our “light-touch” qualitative approach provided the flexibility needed to engage stakeholders in smaller groups or 1-1 conversations in times that worked for them, whilst filtering commonly proposed interventions by their potential impact and feasibility at a hospital given its particular context and capabilities potentially mitigated a “communication gap between modelling and stakeholder groups” (Jahangirian et al., 2015) and a “not invented here” mentality. With respect to the debate on model reuse, Bowers et al. (2012) found that bespoke models can be effective in stakeholder engagement but resource intensive and slow, whereas the generic route is quicker but can struggle to gain acceptance. They consider model reuse a “reasonable compromise” because it is relatively efficient yet retains some stakeholder engagement and has scope for local adaptations. Monks et al. suggest that the extra cost and time associated with model development might be beneficial for changing management attitudes and improving learning and understanding for clients (Monks et al., 2010, 2016). However, although Fletcher & Worthington find that sufficient client involvement is the most consistently cited enabling factor of model implementation in papers on emergency patient flows, they found no clear difference in the likelihood of implementation between generic and specific models of emergency patient flows and note that local politics can be an influencing factor in both (Fletcher & Worthington, 2009). Implementation levels are not thought to be considerably different for generic and non-generic modelling studies in healthcare applications beyond emergency care either, with a systematic literature review of generic models in healthcare showing similarly low levels of implementation to earlier reviews of healthcare models generally (Mahdavi et al., 2013).

Given the evidence that healthcare organisations are unlikely to adopt or be informed by a piece of operational research, generic or otherwise, one could argue that operational researchers should adopt a “take it, or leave it” approach with an existing model-centred intervention, with an onus on front-loading engagement and considerations of implementation, rather than investing significant time building a model specially for an organisation to ignore. In our study, for example, the lack of sustained interest by senior managers in using operational research to tackle ED overcrowding in two of the secondary case studies was identified with relatively little expended effort on our part, and the circumstances did not suggest that they would have been any more motivated if we had developed a bespoke model, which would have involved far more sunken cost. In the assessment of pharmaceuticals, there is the useful concept of the number of patients that need to be treated to avoid an undesirable outcome in one of them, which is three or above for many successful drugs (e.g., paracetamol). Should operational researchers draw on this concept and be more ready to accept that their interventions are simply not likely to work in every organisation?

Our study poses some important questions for further research. Firstly, how can qualitative methods for assessing the absorptive capacity of organisations and the feasibility of implementation be harnessed to increase the uptake of modelling solutions? Such research could include examining the extent to which benefits can be realised through streamlined qualitative analysis conducted by operational researchers rather than in-depth studies conducted by specialists. In our work, an in-depth qualitative analysis in the primary case study informed a more targeted approach to analysing staff perspectives in the secondary case studies, which generated useful insights and enabled us to filter interventions on the basis of implementation feasibility. Would a deeper analysis have added sufficient value to warrant the additional time and resource commitment? Secondly, to what extent does the manner in which a project using a generic model is initiated impact on its likely success? The initiating conditions were very different in the primary case study (where the research team was already embedded, had strong relationships and the work was picked out by senior managers as a key priority area) to the secondary case studies where the approach was reused - one a speculative approach from a clinician, the others connections to the hospitals via other operational research teams. This strengthens the finding of Crowe and Utley (2022) that trust and a prior history of collaboration is important in project initiation as well as the direct involvement of senior hospital management, but further research is needed to systematically examine this.

Acknowledgements

We would like to thank Chris Laing for helping us to understand the impact of the work, and Sean Manzi and Daniel Chalk for connecting us with a key contact at one of the hospitals.

Disclosure statement

No potential competing interest was reported by the authors.

Additional information

Funding

SC, LG, CV-P, NJF and MU were in part supported by the University College London Hospitals (UCLH) NHS Foundation Trust through matched funding to the National Institute for Health Research Collaboration for Leadership in Applied Health Research and Care (NIHR CLAHRC) North Thames at Bart’s Health NHS Trust. NJF is an NIHR Senior Investigator. TM was supported by the NIHR CLAHRC South West Peninsula and NIHR CLAHRC Wessex. BK and MP were supported by NIHR CLAHRC Wessex. The views expressed in this article are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care.