Using a stakeholder co-design approach to develop interventions for quality improvement based on patient complaints

ABSTRACT

Background

While research into complaints made about hospitals is increasing, this has yet to be translated into interventions to improve quality and safety. Incorporating the views of stakeholders into learning from complaints can be an effective means of bridging the gap between research and implementation in patient safety research.

Aims

The aim of the study is to assess whether a co-design approach involving stakeholders is feasible and effective for identifying interventions to address issues from patient complaints.

Methods

A series of online workshops and surveys were utilized to collate the views of stakeholders including patients, healthcare providers, health system researchers, and healthcare managers on how to improve quality and safety in care. Findings of previous analyses of patient complaints were used to identify the focus of this study and guide participants. The strength and feasibility of the interventions outlined by participants was subsequently assessed.

Results

Thirty-two interventions were outlined across the two workshops. These were ranked by participants using the APEASE model. Participants considered the co-design approach an appropriate and effective way to identify interventions for quality and safety improvement.

Conclusions

Stakeholder co-design demonstrated excellent potential for suggesting interventions to improve patient safety based on the findings of complaints analyses.

KEYWORDS:

• Patient complaints
• hospital care
• patient safety
• quality of care
• stakeholder survey
• intervention design

1. Background

Patient and family complaints about healthcare are an important source of data on the quality and safety of care [1]. Although complaints address problems in individual care episodes, when analysed collectively, they can reveal previously unnoticed trends within the broader healthcare system [2]. Reports and research that first emerged from the UK highlighted the importance of patient insights through complaints for providing a ‘warning signal’ of quality and safety issues [2,3]. With limited resources for quality improvement common in healthcare systems internationally [4], complaints as an existing source of quality and safety may prove to be a cost-effective means of improvement.

There is a growing body of research utilizing complaints. Analyses of samples of complaints about hospital care in a number of countries have shown that common issues in complaints relate to institutional processes such as bureaucracy, waiting times, or accessing care [5–8]. Patient harm was reported in approximately half of a sample of 641 complaints about hospital care in the Republic of Ireland (RoI) [5], with levels of harm similar to those found in an analysis of complaints in the UK [6]. In the RoI and the UK, hotspots for harm (areas where harm is reported by patient complaints more frequently than statistically expected) were found for care on the ward, examination and diagnosis, and operations and procedures [5,6]. In the Irish sample, blind spots for harm (areas in care that are difficult for staff members to observe) were identified at admissions and discharge, and for errors of omission (i.e. when something is not done that should have been).

One of the major challenges to progressing patient safety has been a focus on collecting data on quality and safety, rather than using this information to inform changes to the delivery of healthcare [9]. It is also very difficult when attempting to utilize data to make and sustain improvements across a system as complex as healthcare [10]. It is recommended, in order to maximize the likelihood of success, that a range of stakeholders should be involved in the development of interventions [11–13]. An increasingly commonly used approach to involving stakeholders is co-design. Co-design in healthcare involves the equal partnership of those who work within the system, those who use the system, and those who design interventions for the system, with a shared goal of achieving better outcomes or more efficient processes [14]. It has been found that involving stakeholders from different backgrounds can provide a balanced insight into how interventions may work in practice [15], and can ensure that any suggestions for implementation of interventions will be feasible and acceptable to the staff working in those contexts. A co-design approach uses a shared decision making approach that draws on the input from all those involved in the co-design activity with active collaboration. The idea of co-design within healthcare is for both patients and healthcare professionals to shape the system by creating value through new and ongoing interactions [16]. Rather than using a traditional researcher-participant relationship, co-design is a collaborative means for academics and other stakeholders to knowledge [17].

The aim of the study reported is to assess whether a co-design approach is feasible and effective for identifying interventions to address issues from patient complaints. This paper describes: (1) how complaints analysis data were used to identify areas for potential interventions; (2) the co-design approach that was used to identify potential interventions to address these areas of concern; and (3) a feasibility assessment of this approach to learning from complaints data to identify potential interventions.

2. Methods

2.1. Setting

This study was conducted in November 2021 in the context of the Republic of Ireland hospital system. Two workshops were held virtually on the videoconferencing software Zoom©, one week apart, each lasting 90 min.

2.2. Ethics

This study was approved by the [University] Research Ethics Committee (Reference: 2021.07.002). Participants were provided with an information sheet describing the requirements of the study, and asked to provide explicit consent prior to the workshops. The data protection measures put in place for the study were found to be compliant with GDPR regulations, with all participants providing informed explicit consent for the processing of their data. No identifying information on the participants is shared throughout this paper.

2.3. Stage 1: Identifying of exemplar complaints

The findings of an analysis of 641 complaints (n unique issues = 1308) about hospital care in the RoI [5] were used to inform the identification of two specific complaint issues to study. These complaint issues were considered to exemplify the types of issues that emerged commonly within the complaints analysis, and were also considered serious examples of patient safety issues in hospital care. Figure 1 indicates the process for selecting these complaints as the focus of the study, which began with the hot spot for harm identified by the original analysis as occurring during care on the ward.

Figure 1. Process for selecting complaints for consideration in this study.

2.4. Stage 2: Co-design workshops

2.4.1. Workshop participants

Participants were recruited using purposive sampling. Random sampling was not used as expert opinion was required. It was decided that a broad, diverse group of stakeholders would be important for this study, and should include healthcare professionals, patients, health services researchers, and managers within the health service. Recruitment emails were sent to individuals in Ireland and the UK with which the researchers had previously collaborated on the wider patient complaints study and others. A total of 20 people were initially contacted, with 12 agreeing to participate (60% response rate). Participants were offered One4All gift vouchers to the value of €100 for each workshop that they attended.

2.4.2. Workshop procedure

Workshop 1 (quality) focused on identifying interventions that would have prevented an elderly patient being left to fast for four days on the ward while waiting for surgery. Workshop 2 (safety) was focused on identifying interventions that would have prevented an elderly patient falling out of bed while receiving care on the ward. The format of both workshops was the same.

1. Pre-workshop reading. Prior to each workshop, the participants were given approximately 20 min of reading on the topic of the co-design workshop.

2. Introduction (20 min). The workshop opened with a brief presentation by the workshop facilitators describing the intervention design task.

3. Small group intervention development exercise (50 min). The participants were divided into two small groups, each with a workshop facilitator. The goal was to generate ideas for interventions to prevent the issue identified in the complaint. Each group was asked to identify interventions to resolve the issues in question.

4. Presentation of interventions (20 min). Workshop participants reconvened to share their ideas and comment on the identified interventions.

2.4.3. Workshop analysis

Both workshops were audio recorded. The recordings and facilitator notes were used to form a list of interventions. For each intervention the level of the socio-technical systems model at which the intervention was designed to act was recorded. Also, each intervention was classified using the hierarchy of intervention effectiveness framework [18] through discussion by two of the researchers. The framework delineates interventions according to six levels of effectiveness from 1 (most effective) to 6 (least effective):

1. Forcing functions: designing processes so that errors are virtually impossible or difficult to make.

2. Automation and computerization: automating and computerizing processes and tasks.

3. Simplification and standardization: developing and implementing protocols and standardization.

4. Reminders, checklists, and double checks: developing tools that can reduce the risk of error by using checklists and/or having one person independently check another’s work

5. Rules and policies: establishing and enforcing rules and policies related to safety.

6. Education and training: educating and training to promote and ensure patient safety and quality of care.

Frameworks describing the hierarchy of intervention effectiveness have been elaborated by many organizations and researchers [19], and for the purpose of this study the Institute for Safe Medication Practices framework will be employed. This is referenced by a number of patient safety organizations as an approach to guide the identification of suitable safety interventions (e.g. Irish Health Information and Quality Authority [20]).

2.5. Stage 3: Surveys

2.5.1. Survey participants

The same people that participated in the workshops in stage one also completed the surveys (n = 12).

2.5.2. Survey procedure

Five days after each of the workshops, an online survey was distributed to the participants. The survey in turn utilized the APEASE framework to gain insight on participants’ opinions on each of the proposed interventions. The APEASE framework as defined by Michie et al. [21] is a set of criteria that can be used to assess the feasibility of implementing a new intervention in a particular context. This framework has been used in various healthcare settings to assess whether interventions can be implemented in their intended context [22,23] (REFS). For each of the interventions that were generated from the workshop, the participants were asked to rate their agreement with each of the following five APEASE dimensions [21] on a 0 (strongly disagree) to 100 (strongly agree) slider scale:

• Affordability: can be delivered within an acceptable budget.

• Practicability: can be delivered with minimal disruption to patient care.

• Effectiveness: is likely to improve care.

• Acceptability: will be considered appropriate by ward staff.

• Side effects/safety: will not have any unwanted side-effects or unintended consequences.

• Equity: can be delivered in any ward in the Republic of Ireland.

A slider scale was chosen over a Likert scale to reduce likelihood of a ceiling effect on the scores and for ease of use for participants answering on mobile devices [24].

At the end of the second survey, participants were asked to provide feedback on the usefulness of the co-design to generate ideas and identify feasible solutions, and on whether the process should be adopted by hospitals. The participants were also asked to provide open-ended feedback on what was most useful, and what could be improved.

2.5.3. Survey analysis

For each intervention, the mean of the responses to each of the five APEASE dimensions was calculated. An overall mean APEASE score for each intervention was calculated by deriving the mean of the dimension scores. Feedback from participants on the feasibility assessment was reported using descriptive statistics. A narrative synthesis was carried out of the qualitative feedback on the feasibility assessment.

3. Results

3.1. Quality workshop

There were 12 participants in the quality workshop. The workshop participants were stakeholders with a variety of backgrounds, including nursing (n = 2), health service and patient safety managers (n = 4), physicians (n = 1), patients (n = 3) and health system researchers (n = 2). A total of 32 interventions to prevent patients from fasting for long periods of time were identified by the participants in the quality workshop. Table 1 provides an overview of how these interventions were classified using the socio-technical model and the hierarchy of intervention effectiveness. In the quality workshop, the majority of suggested interventions were at the organization and management level of the socio-technical model (n = 11, 34%), and at the ‘Education and training’ level of the hierarchy of intervention effectiveness. More detail on the classifications of the individual interventions from this workshop is available in Supplementary material 1.

Table 1. Classifications of interventions identified at the workshops.

Levels of the socio-technical systems model	Quality workshop (n = 32)	Safety workshop (n = 32)
• Patient/carer	5 (16%)	4 (13%)
• Individual Staff	5 (16%)	6 (18%)
• Team	6 (18%)	3 (10%)
• Organization and management	11 (34%)	9 (28%)
• Equipment/technology	5 (16%)	10 (31%)
Hierarchy of intervention effectiveness
1. Forcing functions	0 (0%)	10 (31%)
2. Automation and computerization	0 (0%)	0 (0%)
3. Simplification and standardization	7 (23%)	11 (34%)
4. Reminders, checklists, & double checks	6 (18%)	2 (6%)
5. Rules and policies	8 (25%)	2 (6%)
6. Education and training	11 (34%)	7 (23%)

3.2. Quality survey

All of the participants (n = 12) who attended the quality workshop completed the survey. Table 2 presents the top quartile of interventions as identified by the highest mean APEASE score. The highest ranked interventions largely related to simplification and standardization, with a focus across different levels of the socio-technical systems model.

Table 2. Top quartile of quality interventions for patients who are left fasting on the ward.

Intervention	Mean APEASE score (SD)*	Sociotechnical model level	Intervention type
• Ensure that staff know, and follow the most recent guidelines on fasting	91.27 (11.04)	Individual	Rules and policies
• Healthcare providers must keep clear and explicit notes for each patient.	90.92 (11.39)	Individual	Simplification and standardization
• Conduct standardized handovers in which any delays in treatment for specific patients are discussed and any issues addressed.	90.17 (12.67)	Team	Simplification and standardization
• Improve communication between theatre and ward team on delays and which patients will be seen on a particular day.	89.30 (12.10)	Team	Reminders, checklists and double checks
• Ensure that patients are informed about delays and changes in their care by staff.	88.28 (14.90)	Individual	Simplification and standardization
• Provide information to patients and their advocates on admission, e.g. what to expect, the name of their consultant, time ward round is conducted	88.25 (14.87)	Management	Education and training
• Conduct regular review of current guidelines for staff to ensure they are up-to-date and accurate.	88.05 (13.06)	Management	Simplification and standardization
• Conduct regular audit of patient leaflets to ensure they are up-to-date and accurate.	87.08 (13.95)	Management	Simplification and standardization

*Maximum score = 100.

3.3. Safety workshop

There were 10 participants in the safety workshop. These included patient experience managers (n = 4), physicians (n = 1), patients (n = 3) and health system researchers (n = 2). No nursing staff were available for this workshop, although one of the managers was an experienced nurse working in an administrative role. The safety workshop identified 32 intervention to prevent patients falling out of bed on the ward. The breakdown of these can be found in Table 1, with more detail in Supplementary material 2. The majority of interventions developed in this workshop fell at the ‘Equipment and technology’ level of the socio-technical model (n = 10, 31%), and from the hierarchy of interventions framework the most frequent intervention type was ‘Simplification and standardisation’ (n = 11, 34%).

3.4. Safety survey

Following the safety workshop, all participants who attended completed the survey, in addition to a nurse who was unable to make the safety workshop (n = 11). Table 3 presents the top quartile of interventions as rated by participants in the safety survey. It can be seen that the top-rated interventions from this workshop and survey were distributed both in terms of their type and level on the socio-technical model.

Table 3. Top quartile of quality interventions for patients who fall on the ward.

Intervention	Mean APEASE score (SD)*	Sociotechnical model level	Intervention type
• Ensure patient has everything they need near them.	88.70 (15.55)	Individual	Simplification and standardization
• During the ward round, specifically discuss the fall risk of a patient and strategies to mitigate this as necessary.	88.59 (13.89)	Team	Simplification and standardization
• Ensure there is an appropriate falls prevention and management policy.	87.4 3(15.40)	Management	Rules and policies
• Conduct a risk analysis of areas where falls are likely and address any issues identified.	87.30 (14.52)	Management	Simplification and standardization
• Family members should make healthcare staff aware if a patient is frail.	86.96 (16.99)	Patient/family	Education and training
• Ensure patients and family members are aware of any medications that may increase the likelihood of a fall.	86.93 (15.43)	Individual	Education and training
• Use visual cues to indicate a patient is at risk of a fall.	86.30 (15.02)	Equipment	Reminders, checklists and double checks
• Lower beds to lowest setting for at-risk patients.	85.91 (17.68)	Individual	Simplification and standardization

*Maximum score = 100.

3.5. Feasibility assessment

As can been seen from Table 4, the participants in the co-design study were broadly positive about the process used to identify the interventions. The participants particularly liked the interactions between the groups, and the broad approach to identifying interventions. To illustrate ‘multidisciplinary and non-medical participants provide a more rounded perspective’ (nurse 2), and ‘blue sky brainstorming was useful to allow freer thinking rather than always just feeling we can’t afford it/don’t have enough staff/resources’ (doctor 1). However, there were some suggestions for improving the process. These suggestions included obtaining input from people with expertise in the area, ‘people with specific expertise on the topic being discussed would be valuable to provide insight and current evidence’ (manager 1), to bring expertise on the issues, and provide information on ‘what solutions are already in place or have been tried; why they work/don’t work’ (manager 2).

Table 4. Participant perceptions of method used.

Statement	Mean response (SD)*
• I think the process we used to identify the interventions was effective in generating ideas.	82.33 (13.42)
• I think the process we used to identify the interventions was effective in identifying feasible solutions to issues in hospitals.	78.11 (18.61)
• I think the process we used to identify the interventions should be adopted by hospitals.	78.56 (15.71)

*Maximum score = 100.

4. Discussion

This study identified and rated multiple interventions to improve specific safety and quality issues which emerged from real patient complaints. The study highlighted the benefits of using a variety of stakeholders to develop interventions and help to bridge the gap between research and implementation in a hospital setting.

Patients who are left fasting on the ward inappropriately is a serious issue, and the consultation on this complaint led to the development of interventions that have the potential to improve overall care quality. The highest rated intervention for this issue was to ensure that staff were aware of the guidelines around patients who are fasting. While staff are already well trained in monitoring fasting patients [25], the guidelines for fasting are often unclear and can quickly become outdated [26]. Ensuring that staff are up to date with the guidelines could reduce the likelihood that a patient is fasting for too long. Moving away from individual staff members, the third highest-rated intervention focused on the staff team as a whole, and on improving handover, a point with multiple demands on the team members, and monitoring of, for example, fasting can be forgotten [27]. Highlighting this issue at handover, and implementing checklists to ensure patients are not overlooked could reduce the incidence of issues such as that described in this complaint [28]. The next step will be to expand upon and apply these interventions in a clinical setting.

Regarding falls on the ward, the top-rated interventions were broadly distributed across the socio-technical model, and in terms of intervention type. The top intervention was a ‘simplification and standardisation’ type intervention directed at individual staff, ensuring that the patient has everything they need within reach. This would be a simple way to reduce the need for patients to mobilize without staff supervision. Falls often occur when patients anticipate a delay in staff coming to their assistance and undertake mobilization independently [29]. Moving away from the emphasis on individual staff standards, other interventions focused on the management level of the socio-technical model. Ensuring there are appropriate policies and frequent risk analyses of the patient care area could reduce the incidence of falls, while also taking pressure off the ward staff. Ward staff may not see risks that build up over time [30], therefore involving management could help to identify safety concerns. Finally, many of the top-rated falls interventions in this study were linked to technology and equipment and forcing functions (e.g. the introduction of simple visual cues to identify at-risk patients). Technology such as this is in place in many hospitals already, for example in the form of wristbands [30], however ensuring that these remain meaningful for staff is important, as there can be an overload of visual cues on the ward setting, and over-reliance may cause too much cognitive load [31]. Similarly to the quality issue, the next step to follow on from these promising findings will be to implement the interventions in a ward setting.

This co-design led to the identification of interventions to improve safety and quality in hospital care that span both the socio-technical model and the hierarchy of intervention types framework [21,32]. The socio-technical model was a useful way to structure both the discussion during the workshops and the findings of this study. Combining this with the hierarchy of intervention types framework [32] has given greater insight into how we can best learn from complaints. Healthcare is an incredibly complex system, and addressing issues at multiple levels, along with considering the types of interventions that we implement can provide the best chance of reducing patient safety incidents and quality issues [33]. There is often not one individual person or team that is solely responsible for issues [33], and this needs to be taken into account when designing interventions. Care is needed to ensure interventions improve safety and quality without adding to the burden on staff, who are already under immense pressure in a resource-limited system [34]. Implementing interventions at the management and technology levels, as well as interventions that are focused on ‘forcing functions’, could possibly improve patient outcomes. Researchers and management who intend on conducting stakeholder consultations should consider employing these or similar frameworks (e.g. The Yorkshire Contributory Factors Framework [33]) to take a systems level view on improvement.

The APEASE framework [21] was also a useful tool to prioritize the interventions. Effectiveness is a key aspect of quality healthcare [35], and it is important that interventions intended to improve healthcare also reflect this. There is often a disconnect between management and staff working at the sharp end of healthcare [30], and by asking participants to rate the interventions using APEASE, this disconnect is considered. It is important that we now move from development to implementation and apply the interventions in practice. Individual hospitals could use this method with their own complaints, and move from resolving complaints one-on-one to learning from them at a unit or hospital level [36].

At the end of the study, participants were asked for feedback on their experience of the process, which was well received by all. The participants indicated that they found the methods effective for developing ideas and identifying solutions. One point that emerged was the benefit of having representatives from different backgrounds in healthcare and from patients. Previous research has indicated that involving a broad range of stakeholders in research is beneficial, and can in fact help to translate research into practice [37,38]. Involving patients in care improvement is also a key focus of contemporary health research [39], and this study ensured that their perspective was represented.

4.1. Strengths and limitations

The present study had several strengths and limitations. Firstly, the inclusion of stakeholders from multiple professional backgrounds was a strength. This ensured that different insights were gained into the issues that were under consideration [38]. Next, the fact that the issues under consideration were supported by the analysis of complaints data was a strength of the study. Using empirical findings from the previous complaints study allowed this study to focus on issues that are identified as particular areas of hot-spots of harm for patients.

Despite these strengths, there were some limitations also. Firstly, while there were a broad variety of stakeholders involved, no participants were in a position to immediately action the findings on the ground (e.g. governing board members, government officials). Future consultations could involve decisionmakers who could make the required recommendations around budgeting and resourcing that may be needed to improve these systemic issues, and actually action the suggested interventions. The study was also limited in its participant numbers, in that the numbers differed between the two workshops due to participant work commitments, and in the different numbers that were representing each stakeholder group. The workshops were held during a peak time in the COVID-19 pandemic in Ireland, and it was therefore particularly difficult to recruit healthcare professionals and managers. However, there remained a good distribution of skills and backgrounds on the stakeholder panel, and care was taken to ensure we had this mix of representation. As there were different numbers in each professional group in the workshops, it is possible that may have introduced a level of bias in the statistical findings. However, as only descriptive statistics were conducted, this bias was considered to be minimal. Were future studies to utilize inferential statistics, the authors would recommend including balanced groups of stakeholders.

Finally, the study was limited in that it took place in the context of secondary care in only one country. Future studies could apply the methodology in this paper to other settings to test whether similar interventions would be developed.

5. Conclusion

There is a large body of patient insight and complaints data available to the healthcare system, which continues to grow, however it is not currently utilized to its full potential. Stakeholder consultations have demonstrated excellent potential across healthcare for distilling insights about complaints into actionable interventions to improve patient safety and quality of care. Utilizing the expertise of stakeholders for the development of patient safety interventions based on complaints analysis should continue to be explored nationally and internationally. It is however important to focus on the types of interventions that are suggested, and where they fit in to the socio-technical model, with the aim of improving safety and quality in healthcare without additional burden being placed on individual staff. This study could have implications for the management of healthcare units, hospitals, and systems, as it clearly demonstrates the value of engaging with a broad group of stakeholders to make improvements to patient safety. Managers should utilize the expertise in their units to learn from patient complaints, and make direct changes to their practice as a result. This study demonstrated the feasibility of utilizing stakeholder insights to move from empirical findings to developing interventions for use in real life healthcare settings.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

Data will be made available by authors upon reasonable request.

Additional information

Funding

This study was funded by the Health Research Board and the HSE Quality Assurance and Verification Division [grant number APA-2017-006].

Notes on contributors

Emily O’Dowd

Dr Emily O'Dowd is a Senior Lecturer in Surgical Education at the Royal College of Surgeons in Ireland. She is an Adjunct Lecturer at the Irish Centre for Applied Patient Safety and Simulation at the University of Galway.

Sinéad Lydon

Sinéad Lydon is a behavioural psychologist working in the School of Medicine at the University of Galway. Sinéad is a Senior Lecturer in Primary Care, teaching on the School's undergraduate and postgraduate programmes, and a member of the Irish Centre for Applied Patient Safety and Simulation's research team.

Chris Rudland

Mr Chris Rudland is the Assistant National Director for the Quality Assurance and Verification Division, Health Service Executive.

Alex Gillespie

Dr Alex Gillespiev is a Professor of Psychological and Behavioural Science at LSE, an Editor of Journal for the Theory of Social Behaviour and Visiting Professor II at the Oslo New University.

Elaine Ahern

Ms Elaine Ahern is a Senior Data Quality Manager at the National Complaints Governance and Learning Team at the Health Service Executive.

Marie E. Ward

Dr Marie E. Ward is the Health Systems Learning and Research Facilitator at St James' Hospital Dublin. She is Adjunct Assistant Professor & Lecturer Managing Risk and System Change at Trinity College Dublin.

Maria Kane

Maria Kane is the Quality Manager in QSID at St James's Hospital, Dublin.

Tom Reader

Dr Tom Reader is an Associate Professor of Organisational Psychology, and directs the MSc in Organisational and Social Psychology at London School of Economics.

Paul O’Connor

Dr Paul O'Connor is a Senior Lecturer in Primary Care and Human Factors Psychologist. He is the Research Director of the Irish Centre for Applied Patient Safety and Simulation (ICAPSS), and Co-director of the Diploma and Masters in Patient Safety and Simulation at the University of Galway.