Quality Management of Prehospital Airway Programs: An NAEMSP Position Statement and Resource Document

Abstract

Prehospital airway management encompasses a multitude of complex decision-making processes, techniques, and interventions. Quality management (encompassing quality assurance and quality improvement activities) in EMS is dynamic, evidence-based, and most of all, patient-centric. Long a mainstay of the EMS clinician skillset, airway management deserves specific focus and attention and dedicated quality management processes to ensure the delivery of high-quality clinical care.

It is the position of NAEMSP that:

• All EMS agencies should dedicate sufficient resources to patient-centric, comprehensive prehospital airway quality management program. These quality management programs should consist of prospective, concurrent, and retrospective activities. Quality management programs should be developed and operated with the close involvement of the medical director.

• Quality improvement and quality assurance efforts should operate in an educational, non-disciplinary, non-punitive, evidence-based medicine culture focused on patient safety. The highest quality of care is only achieved when the quality management program rewards those who identify and seek to prevent errors before they occur.

• Information evaluated in prehospital airway quality management programs should include both subjective and objective data elements with uniform reporting and operational definitions.

• EMS systems should regularly measure and report process, outcome, and balancing airway management measures.

• Quality management activities require large-scale bidirectional information sharing between EMS agencies and receiving facilities. Hospital outcome information should be shared with agencies and the involved EMS clinicians.

• Findings from quality management programs should be used to guide and develop initial education and continued training.

• Quality improvement programs must continually undergo evaluation and assessment to identify strengths and shortcomings with a focus on continuous improvement.

Key words:

• prehospital
• EMS
• quality management
• quality improvement
• quality assurance
• intubation
• airway
• ventilation
• capnography

Resource Document

All EMS agencies should dedicate sufficient resources to a patient-centric, comprehensive prehospital airway quality management program. These quality management programs should consist of prospective, concurrent, and retrospective activities. Quality management programs should be developed and operated with the close involvement of the medical director.

The complexity of prehospital airway management necessitates careful and continuous monitoring to ensure the best possible outcomes for patients experiencing serious illness or injury (1). Robust quality management programs are key to making sure that high-quality care is consistently provided by an EMS agency. Specifically, quality managment includes both quality assurance, measuring compliance against a predefined standard; and quality improvement, the process for assessing systems and processes to identify and act on measurable improvement opportunities (2). While retrospective quality assurance activities may be important to detect deviations from established standards created by the agency, organization, or accrediting bodies, a quality improvement program should be adopted to ensure that a comprehensive, continuous, systems-based approach is used to prevent problems from occurring before they happen (3). Retrospective, concurrent, and prospective reviews of EMS airway cases are all important aspects of oversight and should be incorporated into the quality management program. However, the design of this work, and the point of these reviews, should be focused on identifying challenges in individual cases that may point to larger issues. A comprehensive, patient safety-centered approach to individual cases should lead to systemic improvements, recommendations for training, equipment improvements, and other interventions

Quality management and oversight of airway management in the EMS context requires specific attention from agency leadership and medical direction. The role of a medical director in quality management is to ensure that the overarching program is anchored in medical evidence, supports best practices for airway management, and is aligned with the goal of improving overall patient care. A medical director can focus on process-oriented practice to improve care; however, it is essential that operational and training leadership collaborate to develop shared goals and approaches.

Despite the benefits of robust airway quality management, these programs are not commonplace in most EMS systems (4). Meanwhile, prehospital performance related to airway management is variable in terms of successful placement rates (5–8), errors such as unrecognized misplacement (9,10), and complications like hypoxia (11). The available literature regarding outcomes after out-of-hospital intubation is mixed (12–17).

It is difficult to know the details of quality assurance activities that existed within the systems involved in the studies cited. Quality programs have been shown to improve both airway placement success, and minimize peri-intubation risk. In one study, a structured quality program decreased peri-intubation hypoxia from 44.2% to 3.5% after the institution of a checklist-driven intubation process (18). One agency’s airway educational program, combined with a new quality program, showed an increase in endotracheal tube usage with increased success (19). An air medical provider instituted an agency-wide airway educational program associated with continuous quality improvement and showed significant and continual increases in successful intubation by its crews (20).

An evidence-based, patient-centric approach to airway management in the quality context should integrate the existing data and focus on elements that are important to patient outcomes. These include the avoidance of hypoxia and hypotension, improving first-pass success, and the use of appropriate airway equipment (21). For instance, solutions such as airway checklists can improve adherence to key measures, such as peri-intubation hypoxia (18). This is an example of an evidence-based, patient-centric solution to support EMS clinicians in their efforts to improve overall airway management. Aligning the goals of an airway management program with these types of evidence-based practices will support the best care for patients.

Effective quality improvement processes are designed in a cyclical fashion rather than as single events. This ensures a continuous, consistent movement toward predetermined goals. For example, the Institute for Healthcare Improvement’s Model for Improvement uses the Plan-Do-Study-Act cycle methodology (22) (Figure 1). Determining goals, measures, and methods for improvement requires adequate input for all stakeholders. For airway management, this should include not only EMS agency leadership but also front-line EMS clinicians, other medical first responders, and destination facilities. The expectation for performance should be clear and set early in the improvement process. Related metrics can be grouped into “performance bundles” and reported in aggregate form, further clarifying that individual results affect overall system performance (23).

Figure 1. IHI Model For Improvement.

Reference (50).

Quality improvement and quality assurance efforts should operate in an educational, non-disciplinary, non-punitive, evidence-based medicine culture focused on patient safety. The highest quality of care is only achieved when the quality management program rewards those who identify and seek to prevent errors before they occur.

Traditionally, quality management programs and individual case reviews have been perceived as negative or punitive quests to find “bad apples” with a focus on individual EMS clinicians (24). This type of culture is associated with hesitancy in participation and lack of engagement. Results are achieved as a result of not only individuals, but also the systems in which they operate. It is imperative that the aim of system enhancement, rather than individual blame, be woven throughout the entire airway quality management program, with a keen focus on education rather than discipline (25). Adhering to well-established just culture principles and a culture of safety ensures a balance between accountability for individuals and organizations, leading to an environment of empowerment, knowledge, openness, inclusiveness, and improvement (26,27).

The quality program should also encourage, recognize, and reward those who identify potential problems within the organization. A robust airway quality management program will include leaders within the organization with aptitude for critical thinking who are open-minded, fair, and have professional ethics. Engagement from both EMS clinicians and system leadership, both medical and operational, is essential to instilling cultural change and ensuring lasting improvement (28). Every airway management program must recognize the existing dyad within EMS Ssystems between operational and medical oversight. Successful quality programs are built around a collective understanding of the unique challenges that exist across both elements. A commitment to mutual respect, open dialogue, negotiation, and compromise will ensure a consistent culture for the quality program and the organization (29).

Information evaluated in prehospital airway quality management programs should include both subjective and objective data elements with uniform reporting and operational definitions.

Efforts have been made to create an Utstein-style uniformity to describe operational definitions and key data points for airway management in EMS (30,31). These include case demographics, pre-, peri-, and post-intubation data, and outcomes metrics (see Table 1).

Table 1. Example data elements.

Case Demographics Comorbidities Age Sex Indication (e.g. trauma, medical, cardiac arrest) Level of experience of EMS clinician Drugs available Transportation means available Response Intervals

Pre-intubation Data Initial cardiac rhythm Heart rate Blood pressure Respiratory rate Glasgow Coma Scale score End-tidal CO2 Oxygen saturation Oxygen delivery device/rate Anticipated difficulty of intubation Mallampati score

Peri-intubation DataUpdated data from pre-intubation list Lowest measures of continuous oxygen saturation, blood pressure, and EtCO2 Description of airway difficulty

Post-intubation & Outcome Data Complications Drugs used Subjective success (as documented by clinician) Objective success (as determined by quality assurance program) First-pass success of airway placement Number of airway attempts Airway device used Patient survival

Subjective data, documented in the EMS patient care record, is a primary source of information for quality review. This can drive questions about the approach to patient care and is the source for much of the data described in Table 1. There are obvious challenges with relying on reported data given the inherent subjectivity involved in documentation. It is crucial to limit potential bias by establishing clear definitions for each metric, e.g. what constitutes an airway attempt. For example, one paramedic may consider an intubation attempt as when initially scissoring the mouth open, while another may believe it is when a laryngoscope reaches the vallecula. This may weaken the validity of these values when comparing between individual clinicians, or across different EMS services.

The ability to capture data has advanced over the past decade. The increased availability of objective cardiorespiratory monitoring data, and the ability of quality staff and medical directors to review these data, has changed the depth of oversight possible. Elements of continuous monitoring, such as waveform capnography, pulse oximetry, other vital signs, and audio recordings (if available) can provide a sophisticated understanding of airway management, give real data to share with clinicians to demonstrate the course of care for a particular patient, and provide teaching tools to enhance EMS clinician understanding of the effects of their care (31). If using a video laryngoscope with recording capability, it too should be included, as well as video footage from body cameras if worn by EMS clinicians. Quality management programs should integrate these data in all elements of training and oversight as well.

Unfortunately, while the availability of the novel data sources have improved over the past several years, they are not universal. Many smaller and rural EMS agencies do not have this technology due to resource limitations, however, effective quality management is still possible without this information.

Reportable measures include the above-mentioned data elements and ultimate success and complication rates associated with prehospital airway placement. Most notably, these metrics must be uniformly defined, including the number of airway attempts, first-past success of airway placement, overall intubation success, failed rescue attempts, and peri-intubation hypoxemia, bradycardia, and hypotension. An attempt at endotracheal intubation has been previously described as any time the laryngoscope blade enters the oral cavity with the intent of endotracheal intubation, while a supraglottic airway attempt is marked as soon as the tube passes a patient’s teeth (32,33). Recent literature regarding overall advanced airway placement success rates has shown results ranging from 53% to over 90%, with first past success rates yielding comparable variability (34–36). While the cause of this variability has not yet been clearly defined, these strict definitions must be uniformly appreciated to minimize future bias. Each time the above criteria are met, an additional airway attempt must be documented. Final airway success requires confirmation of correct airway placement by waveform capnography, and should be confirmed through retrospective review of objective data such as the cardiac monitor file waveforms.

EMS systems should regularly measure and report process, outcome, and balancing airway management measures.

Measurement is a key aspect to understanding when changes implemented as part of an airway quality management program have led to the desired improvement. There are three main types of quality measures: outcome, process, and balancing. Outcome measures, defined as an endpoint-based quantification believed to evaluate the success of a particular action or program, are often the gold standard by which we measure patient-centered impact of care (37). However, in EMS, it is often challenging to tie the care to these measures. Instead, process measures, defined as adherence to evidence-based best practices, serve as key drivers for improvement because these are more easily captured and measured (37). Balancing measures are often overlooked but are used to identify and measure unintended consequences when attempting to improve key process and outcome measures. Example quality measures for airway-related quality management include, but are not limited to, the items listed in Table 2. During quality improvement efforts, leaders must not only identify process and outcome measures to drive improvement, but also use balancing measures to ensure that any changes made do not negatively affect other aspects of care. For instance, if an EMS agency is aiming to improve first pass intubation success, quality leaders must also ensure that there is no unintended increase in first attempt peri-intubation hypoxia.

Table 2. Example airway quality metrics.

Metric	Numerator	Denominator
Number of airway attempts	Number of times the laryngoscope blade enters the oral cavity with the intention of endotracheal intubation; or, as soon as a supraglottic airway passes the patient’s teeth.
Intubation first-pass success rate	Number of successful initial intubation attempts	Total number of patients on whom airway placement was attempted
Overall intubation success	Number of successful intubation attempts	The total number of intubation attempts
Patient intubation success	Number of patients successfully intubated	Total number of patients on whom intubation was attempted
Frequency of rescue airway deployment	Number of airway attempts resulting in rescue airway placement	Total number of intubation attempts
Peri-intubation hypoxemia	Number of patients undergoing intubation who desaturate to an oxygen saturation below 90%	Total number of intubation attempts
Peri-intubation hypotension	Number of patients undergoing intubation whose systolic blood pressure drops below 90 mmHg	Total number of intubation attempts
Rate of complications	Number of airway management attempts that encountered a complication	Total number of airway management attempts
Unrecognized failed airway rate	Number of advanced airway attempts that have no objective success verified on quality assurance review	Number of advanced airway attempts documented as successful by the performing clinician
Unmanaged airway rate	Number of patients without objective success verified for any basic or advanced airway maneuver on quality assurance review	Number of patients undergoing advanced airway management

Quality improvement methodology relies on data reporting of these quality measures over time. Viewing data over time is essential to separate random common cause variation from nonrandom special cause variation, distinctions which are often lost with use of single aggregate summary measures such as annual means or proportions. A simple and important way to visualize data from quality measures is through run charts. A run chart is a graphical display of data plotted in order by time (38). Statistical process control charts, also known as Shewhart charts add upper and lower control limits to data presented over time, help distinguish normal variation from special cause variation and decrease variability in a system (22). Process changes should be noted on charts to visualize when the change occurred and evaluate its effect on the process.

Evaluation of airway management performance must include assessment for equity of patient care in order to reduce disparities in patient outcomes based on demographic and socioeconomic characterisitics (39–41). All data and evaluation reports should be filtered to look for differences in patient care depending on race, gender, and age. Socioeconomic status differences in care can also be evaluated based on scene location, primary payer, or homeless status.

Quality management activities require large-scale bidirectional information sharing between EMS agencies and receiving facilities. Hospital outcome information should be shared with agencies and the involved EMS clinicians.

Bidirectional data sharing between EMS agencies and receiving facilities should be a routine occurrence with every airway case, both on an individual scale for EMS clinician learning, and on a larger scale for research purposes and health care advancement (42). Bidirectional data sharing allows receiving hospitals to review prehospital care, prehospital clinicians to follow up on patient outcomes and treatments, and prehospital clinicians to review patient history and documentation en route to a call. Access to patient interactions including procedures, treatments rendered, and patient outcomes is critical for prehospital clinicians to learn from previous cases to improve for future ones.At a minimum, bidirectional data sharing should include, the “eOutcomes” fields of the National EMS Information System (NEMSIS) standard, to ensure uniformity across systems.

Despite these benefits, bidirectional data sharing is not yet common practice. The EMS Agenda 2050 recognized the need for further integration of prehospital care into the larger health care arena, appreciating that “EMS has often struggled to find a ‘seat at the table’” (43). Hospitals have traditionally avoided data sharing with EMS for fear of expense and HIPAA privacy concerns. However, these concerns are laregely unfounded as evidenced by a recent whitepaper has drawn attention to the legally feasible and important goal of data sharing between EMS and the larger health care arena, and vice versa (4,44).

In the recent era of COVID-19 infections, bidirectional data sharing has become even more important for EMS clinician safety, to inform responders of potential infectious exposures, particularly in the case where aerosolizing airway procedures were performed. Similarly, availability of patient data in the prehospital setting not only enables EMS clinicians to review subsequent care rendered after EMS handoff to the ED team, but also proactively allows clinicians to review patients’ medical histories, medications, allergies, recent procedures, and documentation while responding to a patient, saving vital time on scene in critical cases.

Findings from quality management programs should be used to guide and develop initial education and continued training.

A well-organized EMS airway quality program can and should guide initial and on-going educational programming (3). Educational interventions, both didactic and skills-based, are often key components of quality improvement efforts. EMS education and training programs are grounded in ensuring initial competency in the understanding and application of core physiologic principles and psychomotor proficiency in both frequently used and high-risk skills. Ventilation and airway management are core features of both basic and advanced life support management for prehospital patients (1). High-functioning quality improvement programs should be aligned with educational curricula in the following ways:

1. Ensure and track initial credentialing of prehospital clinicians based on completion of accredited educational programs and maintenance of license and certification

2. Base prospective quality improvement tracking and reporting of outcomes (including success rate and complications) for pre-specified low-frequency, high-risk skills (e.g. intubation, needle cricothyrotomy)

3. Ensure on-going didactic and simulation opportunities for low-frequency, high-risk skills

4. Use quality improvement program outcomes and findings to inform and augment ongoing educational curricula in the form of didactic and skills review programs, introduction of new devices with updated training, and individualized performance improvement plans

Each level of prehospital clinician needs to master the procedures and considerations applicable to their national and local scope of practice (1). EMS airway quality improvement programs should ensure and document that each clinician has completed the minimal requirement of training and proctoring for such skills. Quality improvement program managers should be familiar with the core requirements essential for accredited training programs. Many programs may find such accreditation sufficient. However, some programs may also find benefit from a proctoring and review period upon a clinician’s matriculation to an EMS service, given that differences may exist in a clinician’s performance in the training versus direct practice environment (such as equipment, patient acuity, or environmental factors).

Prospective monitoring of critical skills, and their outcomes, is essential for any quality improvement program to identify areas for improvement. Regular review of trends in volume of skills performed, and procedure rate variation between individual clinicians, can help to inform both systemic and personalized educational programming, including the frequency of simulation programs to maintain critical skills (45). For example, the expansion of a paramedic workforce to meet growing demands in lower-acuity call volume could result in lower numbers of endotracheal intubations per paramedic performed each year (46). This should inform the need for increased simulation and skills-based practice, which have been demonstrated to improve clinician confidence and success in skill application, within the annual training and educational curriculum (47).

Prospective outcomes monitoring can also help to separate “common cause” versus “special cause” quality assurance issues (38). For example, if there is a sudden increase in endotracheal tube dislodgments following the introduction of a new airway device, this could trigger a change in device or a more intensive educational effort around appropriate use and securing techniques for the device. A robust quality improvement process can also allow the EMS leadership of a program to evaluate a product or technique in airway management, such as video laryngoscopy or routine bougie use, before introducing it to the larger educational curriculum for all clinicians (48,49).

Case review of sentinel events also offers an opportunity to reexamine existing educational material or reinforce best practices for the larger group of clinicians. For example, reviewing a case of anaphylaxis that required a needle cricothyrotomy with a larger group can help to review key material for a rare procedure along with important treatment algorithms to address the condition. Additionally, a missed diagnosis or a relatively rare condition, can serve as fodder for formal or informal educational review to refresh the clinicians’ knowledge and recall of a condition.

Quality improvement programs must continually undergo evaluation and assessment to identify strengths and shortcomings with a focus on continuous improvement.

As quality management programs use improvement science to set goals and change airway processes, these same principles must also be applied to the quality management systems themselves. Quality managers should regularly evaluate for program performance gaps, develop goals, and design measures specific to the airway quality management program for continued growth and success.

Often automated programs are used to help evaluate calls for compliance and identify those calls that require a manual review. Part of the quality improvement review process should be to regularly audit the automated programs to make sure the protocols are up to date, the data are pulling from the correct areas of the patient care report, and the correct calls are flagging for review. Computer technology is only as smart as its programming, and quality managers should work closely with program vendors to constantly improve these processes.

Another key aspect related to making continuous improvements is to consider is the time to complete retrospective review of airway cases. Reviews should be completed and feedback should be provided as close to the event as possible, to help with recall of the incident for the clinician, and to quickly implement any change ideas gleaned from the review process. Using review systems that capture data about when a review is opened and closed will help evaluate the time it takes to start a review and the time it takes to complete the review process. Evaluating these times and working for improvement to make the time to complete a review as short as possible is a good example of how to evaluate and improve quality improvement work. In large systems with multiple case reviewers, it is important to audit the reviewers periodically for compliance with the review process, appropriateness of feedback, and time management. Making sure that feedback is constructive, timely, and accurate is important to the overall culture of the organization.

Smaller systems with low call volume may be able to accomplish 100% manual chart review, but larger systems may need to employ automated programs to evaluate calls for compliance and identify those calls that require a manual review. Part of the quality improvement review process should be to occasionally audit the automated programs to make sure the protocols are up to date, the data are pulling from the correct areas of the patient care report, and the correct calls are flagging for review. Computer technology is only as smart as its programming, and quality managers should work closely with program vendors to constantly improve these processes.

In large systems with multiple case reviewers, it is important to audit the reviewers periodically for compliance to the review process, appropriateness of feedback, and time management. Making sure feedback is constructive, timely, and accurate is important to the overall culture of the organization.

Reviewing cases does little good if the information is not shared. Part of improving the system is embracing transparency and sharing results, good or bad. Feedback on all reviews performed should be shared in private with the individual clinician. System results and feedback should be shared with all clinicians and stakeholders. Care should be taken to visualize the data in a way that is easily understood by all. Coaching in private and praising in public will provide a safe space for clinicians to learn and grow and foster trust and teamwork between the field clinicians and the quality improvement department.

Just as you will evaluate and set goals for airway processes, quality improvement systems must also be reviewed for gaps, goals set, and improvement science processes implemented for continued growth and success.