Abstract
Background: Performance measures are a key component of implementation, dissemination, and evaluation of evidence-based guidelines (EBGs). We developed performance measures for Emergency Medical Services (EMS) stakeholders to enable the implementation of guidelines for fatigue risk management in the EMS setting. Methods: Panelists associated with the Fatigue in EMS Project, which was supported by the National Highway Traffic Safety Administration (NHTSA), used an iterative process to develop a draft set of performance measures linked to 5 recommendations for fatigue risk management in EMS. We used a cross-sectional survey design and the Content Validity Index (CVI) to quantify agreement among panelists on the wording and content of draft measures. An anonymous web-based tool was used to solicit the panelists' perceptions of clarity and relevance of draft measures. Panelists rated the clarity and relevance separately for each draft measure on a 4-point scale. CVI scores ≥0.78 for clarity and relevance were specified a priori to signify agreement and completion of measurement development. Results: Panelists judged 5 performance measures for fatigue risk management as clear and relevant. These measures address use of fatigue and/or sleepiness survey instruments, optimal duration of shifts, access to caffeine as a fatigue countermeasure, use of napping during shift work, and the delivery of education and training on fatigue risk management for EMS personnel. Panelists complemented performance measures with suggestions for implementation by EMS agencies. Conclusions: Performance measures for fatigue risk management in the EMS setting will facilitate the implementation and evaluation of the EBG for Fatigue in EMS.
Background
This paper complements the article “Evidence-Based Guidelines for Fatigue Risk Management in Emergency Medical Services” (Citation1). In the guidelines paper, we outlined 5 recommendations for the mitigation of fatigue in Emergency Medical Services (EMS) operations (summarized below). The recommendations were constructed following a rigorous, evidence-based process based on the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) methodology (Citation1–3). This process included evaluating the best available science linked to multiple strategies for fatigue mitigation in high-risk operations. The origins of this project are linked to an advisory issued by the National EMS Advisory Council (NEMSAC) and to increased concern for the safety of EMS clinicians, their patients, and the public (Citation4).
The National Prehospital Evidence-Based Guideline Model recommends that development of evidence-based guidelines (EBGs) be closely followed by implementation and evaluation of those guidelines (Citation5). Without a plan to turn guidelines into actions and ensure that they are being followed, EBGs will not reach their full positive impact. Implementation and evaluation of EBGs have represented a substantial challenge for EBG developers and are a focus of recent investigation and efforts to increase the use of guidelines in prehospital care (Citation6,7). Common barriers to the adoption of EBGs include a lack of tools disseminated directly to the targeted audience that can assist in guideline implementation (Citation8). Evaluating whether guidelines are being executed as recommended and whether they are having their intended effect on outcomes is similarly challenging. Brown et al. assessed the implementation of an EBG for prehospital pain management in a statewide EMS system and found incomplete documentation of pain scores as a barrier to assessing the true impact of implementing the EBG (Citation9). This example highlights the importance of performance measures in any planned evaluation process.
The Federal Interagency Committee on EMS, the National EMS Advisory Council, and the National Prehospital Evidence-Based Guidelines Strategy developed by the National Association of EMS Physicians, identified additional emphasis on implementation and evaluation as a priority in the advancement of prehospital EBGs (Citation10–12). Development of performance measures that complement EBGs has become a critical tool to aid in these important steps. We seek to facilitate the adoption of the Fatigue Risk Management Guidelines in EMS (Citation1) by developing performance measures tailored to each evidence-based recommendation. We also offer practical suggestions from the project's expert panel that may facilitate implementation of the guidelines across EMS systems.
Methods
Two investigators (CMG and PDP) developed a draft set of performance measures linked to 5 previously developed recommendations for fatigue risk management (Citation1). Each of these recommendations was guided by research questions framed in the Population, Intervention, Comparison, Outcome (PICO) framework. The draft measures were presented to the expert panel for editing and refinement in preparation for rating relevance and clarity. The panel completed several rounds of edits, facilitated via conference calls and e-mail, before they were asked to rate the relevance and clarity of performance measure statements. We used a cross-sectional survey study design to quantify agreement among panel members on relevance and clarity.
Protocol
The University of Pittsburgh Institutional Review Board approved our study protocol. After performance measures were drafted and revised with input from the expert panel, we used an anonymous, web-based survey tool to solicit the panel members' perceptions of relevance and clarity separately for each draft performance measure. The panel was presented the draft performance measure statement for each recommendation separately, as shown in . Immediately below each draft performance measure statement, we presented the panel with two questions that solicited their perceptions of the content for each of the draft performance measures. The question of relevance was defined as addressing whether the statements were connected/germane to: (a) “the findings of the systematic review for [the guideline recommendation];” (b) “the balance between benefits and harms for [the guideline recommendation];” and (c) “the values and preferences of the EMS community of shift worker clinicians and administrators.” Additionally, relevance addressed whether there were (d) “any concerns for resource use (costs)” and whether the statements (e) “are suitable in current form for purposes of guiding the EMS community with regards to fatigue risk management?” The panel rated their perception of relevance for each draft performance measures statement separately on a scale anchored from 1–4, with (1) the statements are not relevant; (2) the statements need major revisions to be relevant; (3) the statements need minor revisions to be relevant; and (4) the statements are relevant. The question of clarity was presented separately for each draft performance measures statement as: “Are the statements for [the performance measure] clear, intelligible, appropriately worded, sharp, and easy to understand by a diverse audience?” The panel recorded their perception of clarity on a scale anchored from 1–4, with (1) the statements are not clear; (2) the statements need major revisions to be clear; (3) the statements needs minor revisions to be clear; and (4) the statements are clear.
Table 1. Performance measures
Analysis
We calculated separate scores for relevance and clarity of each performance measure using procedures corresponding to the calculation for the Content Validity Index (CVI) (Citation13). The CVI score was quantified as the total number of 3s and 4s (on a scale of 1–4) divided by the total number of raters. The CVI scores for relevance and clarity were not combined, averaged, or otherwise collated. The score for relevance was quantified separately from the score for clarity. We used the standard CVI score benchmark of ≥0.78 (on a 0 to 1 scale) separately for relevance and clarity as our indicator of consensus on the content of draft performance measures (Citation13).
Results
The panel reached agreement on all five measures of performance (). The panel exceeded the 0.78 cut point with only a single round of voting for each of the five draft performance measures. The panel agreed upon the following performance measures as indices to evaluate the impact of adopting and implementing the five recommendations for fatigue mitigation in the EMS setting. Fulfillment of each of these evidence-based recommendations should be reassessed on an annual basis to better incorporate the recommendations into practice and ensure their ongoing use.
Recommendation 1: We recommend using fatigue/sleepiness survey instruments to measure and monitor fatigue in EMS personnel.
Performance Measure 1: Demonstrated use of reliable/valid fatigue and/or sleepiness survey instruments to measure and monitor fatigue in EMS personnel on at least a quarterly basis.
Goal: Assess fatigue/sleepiness of EMS personnel with reliable/valid survey instrument(s) at least quarterly.
Details on Use of Performance Measure: There is insufficient evidence to recommend a specific fatigue/sleepiness survey instrument. Using any of the 14 survey instruments identified in the systematic review as reliable and/or valid is a priori acceptable. Random sampling, as well as targeted assessments, is recommended. Specifically, assessment of fatigue/sleepiness is recommended with reliable/valid survey instruments for any shift schedule (pattern/structure) suspected of elevating the risk of fatigue, such as extended duration shifts (e.g., ≥12 hours).
Recommendation 2: We recommend that EMS personnel work shifts shorter than 24 hours in duration.
Performance Measure 2: Percent of all shifts that are <24 hours in duration.
Goal: 100% of all shifts are <24 hours in duration.
Details on Use of Performance Measure: Shifts performed contiguously should be counted as a single shift period with a total duration (e.g., two 12-hour shifts performed contiguously by a single provider should be counted as a 24-hour shift). Shifts include: 1) scheduled and unscheduled work periods; and 2) all work periods performed by EMS personnel, regardless of patient care responsibilities.
Recommendation 3: We recommend that EMS personnel have access to caffeine as a fatigue countermeasure.
Performance Measure 3: Percent of all shifts where EMS personnel have access to caffeine.
Goal: 100% of shifts with access to caffeine.
Details on Use of Performance Measure: Example of access to caffeine includes availability of caffeinated beverages or gum for free or for purchase while on duty within reasonable access to on-duty EMS personnel.
Recommendation 4: We recommend that EMS personnel have the opportunity to nap while on duty to mitigate fatigue.
Performance Measure 4: Percentage of all shifts where EMS personnel are provided with access to and permission to take a nap while on duty.
Goal: EMS personnel are provided with access to and permission to take a nap while on duty in 100% of extended shifts (i.e., ≥12 hours) and shifts taking place overnight.
Details on Use of Performance Measure: We define a nap as a short period of sleep (duration is not specified). To ensure reasonable access to take a nap while on duty, there should be a scheduled time to take a nap or an unrestricted opportunity to take a nap throughout the shift, and an appropriate place to take an uninterrupted nap. Agencies may wish to consider the napping strategy regardless of shift duration and include shifts <12 hours as part of the performance measure if personnel work contiguous shifts and/or consecutive shifts with limited recovery between shifts (including combinations of shifts involving different agencies). Providing EMS personnel the opportunity to nap on duty is best demonstrated with a written policy.
Recommendation 5: We recommend that EMS personnel receive education and training to mitigate fatigue and fatigue-related risks.
Performance Measure 5: Percent of EMS personnel who have: 1) received education and training to mitigate fatigue and fatigue-related risks during new employee orientation/training; and 2) received education and training to mitigate fatigue and fatigue-related risks within the previous 2 years.
Goal: 1) 100% of EMS personnel have received fatigue education and training as part of new employee orientation/training; and 2) 100% of EMS personnel have received fatigue education and training within the previous 2 years.
Details on Use of Performance Measure: Functional memory, knowledge, and skills can decay rapidly after initial education and training (Citation14). Education and training every 2 years is recommended to address decay in memory, knowledge, and skills in dealing with fatigue recognition and mitigation in the workplace.
Discussion
Guidance for Implementation of Performance Measures
The panel produced five measures to evaluate the impact of adopting and implementing evidence-based strategies to mitigate fatigue in the EMS setting. Implementation of these measures may be met with unique challenges, which may be predicted and addressed as discussed in the following sections.
Guidance on implementation of performance measure 1: Demonstrated use of reliable/valid fatigue and/or sleepiness survey instruments to measure and monitor fatigue in EMS personnel on at least a quarterly basis. Administrators of EMS operations should assess and monitor the fatigue/sleepiness status of personnel. A list of 14 survey instruments that may be useful for fatigue and sleepiness assessment and monitoring is published separately (See Online Supplemental Material in a separate publication) (Citation15). The expert panel that developed this recommendation considered whether any specific survey instrument should be recommended for use. However, the systematic review leading to this recommendation was not designed to identify which survey instrument is best and different instruments may be better for different operational EMS settings. Therefore, EMS administrators should consider use of any of the instruments identified as reliable and/or valid. Assessing fatigue/sleepiness for a random sample of scheduled shifts (rather than all shifts) may reduce respondent burden, improve participation by EMS personnel, and improve the representativeness of fatigue assessment findings. Targeted assessments of extended duration shifts (e.g., ≥12 hours), shifts occurring overnight, or shifts with limited inter-shift recovery are recommended. Shifts with high unit hour utilization, short inter-shift recovery, remote basing, and system status management that preclude rest periods should also be targeted.
Guidance on implementation of performance measure 2: Percent of all shifts that are <24 hours in duration. Administrators of EMS operations should provide adequate opportunity to rest, recover, and sleep between shifts. Ideally, shift duration would be <24 hours. Shift schedules should be designed with recovery periods greater than 10 hours between shifts to give personnel their best chance of obtaining the National Sleep Foundation's recommended 7–9 hours of sleep per 24 hours (Citation16–18). When assessing shift duration, unscheduled contiguous work periods should be considered as a single shift, and its total duration measured. The implementation procedure should also track both scheduled and unscheduled work periods to determine total hours worked, whether intentional or unintentional. The total hours worked should form the basis for this performance measure.
Not all EMS systems will be able to implement shifts of <24 hours duration and in some cases, shorter shifts may not be an optimal strategy due to inherent difficulties with staffing and the need for continuity of operations. EMS systems that utilize shifts ≥24 hours in duration should implement other fatigue mitigation tools, including the other recommendations outlined in the Fatigue Risk Management Guidelines in EMS (Citation1). Additionally, a “time out” policy, whereby EMS personnel identified as fatigued by self-report, colleagues, or management are granted a reasonable rest period, should be considered for all shifts and especially for shifts of extended duration. Calling a “time out” should be without punitive action or undue pressure to continue in this circumstance, and use of the policy should be monitored.
Guidance on implementation for performance measure 3: Percent of all shifts where EMS personnel have access to caffeine. Administrators of EMS operations should provide adequate access to caffeine. EMS agencies do not need to provide caffeine-containing products for free to meet this measure, but should at least ensure EMS personnel can obtain caffeine (in the form of coffee, tea, caffeinated sodas, or gum) while on duty if they choose to use it. Attention should be paid to obtaining or using caffeine during periods where alertness may be affected most (e.g., during typical sleep periods). Caffeine availability and use should be integrated with education on fatigue risk management that addresses the planned use of caffeine to achieve maximal on-duty benefit, while mitigating potential negative consequences on physiology and off-duty sleep. Because of potential adverse physiological effects of caffeine use (and/or withdrawal) in persons vulnerable to such effects and personal beliefs regarding caffeine use, EMS agencies should not require the use of caffeine, nor is mandatory caffeine consumption necessary to meet the performance measure. The systematic review of literature on use of caffeine did not identify an optimal dose or timing of caffeine; consequently, we do not offer a recommendation on these aspects of caffeine use. However, EMS personnel should be educated on both the potential benefits and risks associated with excessive caffeine use as part of a comprehensive fatigue education and training program.
Guidance on implementation for performance measure 4: Percent of all shifts where EMS personnel are provided with access to and permission to take a nap while on duty. Administrators of EMS operations should provide the opportunity to nap during work hours. The optimal duration of on-duty naps cannot be determined from the available literature due to the heterogeneity of methods and outcome measures in existing studies, as well as differences in human physiology. Even a nap as short as 10 minutes can reduce sleepiness and fatigue, and improve cognitive performance (Citation19). However, rest breaks should only be considered nap opportunities if there is a reasonable expectation that personnel will be able to sleep. Rest periods that do not provide a reasonable expectation of sleep have been described in prior research as placebo or control conditions and do not provide the outcome benefits of sleep-containing naps (Citation20, 21).
The possibility of sleep inertia should also be considered when implementing the opportunity for naps (Citation22, 23). EMS personnel may experience a period of reduced alertness or impaired cognition when suddenly awakening from a nap, such as when immediately responding in an ambulance or other apparatus. Education of EMS personnel regarding this potential occurrence should be considered as part of a comprehensive fatigue risk management system. Caffeine may be used to mitigate the psychomotor effects of sleep inertia (Citation24, 25). Alternately, having EMS crew members who can both serve as driver nap at separate times may help mitigate the effects of sleep inertia after one takes a nap.
The available literature does not describe the optimal location to take a nap. However, many studies describe an area where an individual can lay horizontally and is reasonably shielded from light, noise, and distractions to achieve sleep (Citation20, 26, 27). Accommodations are expected to differ among EMS agencies. Administrators will need to consider existing accommodations, as well as the potential perception of administrators, colleagues, and the lay public who could encounter personnel who are attempting to nap.
Guidance on implementation for performance measure 5: Percent of EMS personnel who have: 1) received education and training to mitigate fatigue and fatigue-related risks during new employee orientation/training; and 2) received education and training to mitigate fatigue and fatigue-related risks within the previous 2 years. Administrators of EMS operations should incorporate fatigue and sleep health education and training as part of new employee orientation and continuing education. Multiple educational programs that include sleep health, fatigue, and general wellness education are summarized in a separate publication (Citation28). Currently, there is no gold standard program and we anticipate the scale, scope, depth, and breadth of programs will differ based on an agency's preferences and needs. EMS personnel should receive instruction inclusive of but not limited to the use of fatigue/sleepiness survey instruments, shift scheduling, nap opportunities, and caffeine use, as appropriate to the guidelines' recommendations that the EMS agency has adopted. Considering that functional memory, knowledge, and skill can decay rapidly after initial education and training, we recommend education and training occur every 2 years (Citation14), similar to training renewal in multiple medical content areas (e.g., Advanced Cardiac Life Support education and training). Fatigue or sleep health modules could be combined with existing orientation and continuing educational programs to mitigate cost of implementation.
Implementation Checklist
Based on the guideline recommendations and performance measures, we developed an implementation checklist that EMS stakeholders can use to incorporate the recommendations into their EMS system (). This practical guide can help assess whether the EMS system is already compliant with the individual recommendations and guide additional steps to meet the core elements of the guideline recommendations (Citation1).
Table 2. Checklist for the implementation of fatigue in EMS evidence-based guidelinesFootnote*
Limitations
The expert panel was comprised of a variety of EMS and sleep science stakeholder representatives as recommended by the Institute of Medicine (Citation29). Panel members included EMS administrators and risk managers, EMS clinicians, emergency medicine personnel, sleep and fatigue scientists, and researchers. A different panel may have created different performance measures based on their review of the evidence and development or interpretation of the guidelines.
We measured agreement among panelist using the CVI calculation, which is an established metric for quantifying consensus on item content and wording (Citation13). We believe use of the CVI is superior to simple verbal agreement or an alternative subjective approach to evaluating group consensus. The CVI provides a quantitative summary of agreement. There may be different methods for determining agreement among panel members using quantitative or subjective measurement approaches. Use of a different technique could have led to different measures of performance.
Our approach to guideline development was informed by the GRADE framework, which has emerged as a standard for development of EBGs that inform clinical practice and occupational health (Citation3, 30). We also used the Model Process for EBG development and sought to address the key steps of implementation and evaluation (Citation5). Other processes for guideline development might yield different results.
Conclusions
Implementation and evaluation are critical components of the Model Process for Prehospital EBGs. We propose performance measures and offer a practical guide for implementing and evaluating the fatigue risk management guidelines in EMS.
References
- Patterson PD, Higgins JS, Van Dongen HPA, Buysse DJ, Thackery RW, Kupas DF, Becker DS, Dean BE, Lindbeck GH, Guyette FX, et al. Evidence-based guidelines for fatigue risk management in Emergency Medical Services. Prehosp Emerg Care. 2018;22(S1):89–101.
- Patterson PD, Higgins JS, Weiss PM, Lang ES, Martin-Gill C. Systematic review methodology for the Fatigue in Emergency Medical Services project. Prehosp Emerg Care. 2018;22(S1):9–16.
- Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, Schunemann HJ, GRADE Working Group. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336(7650):924–6. https://doi.org/10.1136/bmj.39489.470347.AD.
- NEMSAC. Fatigue in Emergency Medical Services. National EMS Advisory Council Quarterly Meeting. 2013; http://ems.gov/pdf/nemsac/2013/NEMSAC-AdvisoryFatigueJan2013.pdf.
- Lang ES, Spaite DW, Oliver ZJ, Gotschall CS, Swor RA, Dawson DE, Hunt RC. A national model for developing, implementing, and evaluating evidence-based guidelines for prehospital care. Acad Emerg Med. 2012;19(2):201–9. https://doi.org/10.1111/j.1553-2712.2011.01281.x.
- Browne LR, Shah MI, Studnek JR, Ostermayer DG, Reynolds S, Guse CE, Brousseau DC, Lerner EB. Multicenter Evaluation of Prehospital Opioid Pain Management in Injured Children. Prehosp Emerg Care. 2016;20(6):759–67. https://doi.org/10.1080/10903127.2016.1194931.
- Shah MI, Carey JM, Rapp SE, Masciale M, Alcanter WB, Mondragon JA, Camp EA, Prater SJ, Doughty CB. Impact of high-fidelity pediatric simulation on paramedic seizure management. Prehosp Emerg Care. 2016;20(4):499–507. https://doi.org/10.3109/10903127.2016.1139217.
- Flodgren G, Hall AM, Goulding L, Eccles MP, Grimshaw JM, Leng GC, Shepperd S. Tools developed and disseminated by guideline producers to promote the uptake of their guidelines. Cochrane Database Syst Rev. 2016(8):CD010669.
- Brown KM, Hirshon JM, Alcorta R, Weik TS, Lawner B, Ho S, Wright JL. The implementation and evaluation of an evidence-based statewide prehospital pain management protocol developed using the national prehospital evidence-based guideline model process for emergency medical services. Prehosp Emerg Care. 2014;18(Suppl 1):45–51. https://doi.org/10.3109/10903127.2013.831510.
- NHTSA. FICEMS Releases Collaborative Action Plan to Further Support EMS Nationwide. 2014; https://www.ems.gov/ficems/plan.htm. Accessed June 14, 2017.
- NEMSAC. Successful Integration of Improvement Science in EMS. 2016; https://www.ems.gov/pdf/Final-Draft-Integration-of-Improvement-Science.pdf. Accessed June 14, 2017, 2017.
- Martin-Gill C, Gaither JB, Bigham BL, Myers JB, Kupas DF, Spaite DW. National Prehospital Evidence-Based Guidelines Strategy: A summary for EMS stakeholders. Prehosp Emerg Care. 2016;20(2):175–83. https://doi.org/10.3109/10903127.2015.1102995.
- Lynn MR. Determination and quantification of content validity. Nurs Res. 1986;35(6):382–85. https://doi.org/10.1097/00006199-198611000-00017.
- Yang CW, Yen ZS, McGowan JE, Chen HC, Chiang WC, Mancini ME, Soar J, Lai MS, Ma MH. A systematic review of retention of adult advanced life support knowledge and skills in healthcare providers. Resuscitation. 2012;83(9):1055–60. https://doi.org/10.1016/j.resuscitation.2012.02.027.
- Patterson PD, Weaver MD, Fabio A, Teasley EM, Renn ML, Curtis BR, Matthews ME, Kroemer AJ, Xun X, Bizhanova Z, et al. Reliability and validity of survey instruments to measure work-related fatigue in the Emergency Medical Services setting: a systematic review. Prehosp Emerg Care. 2018;22(S1):17–27.
- Eldevik MF, FLo E, Moen BE, Pallesen S, Bjorvatn B. Insomnia, excessive sleepiness, excessive fatigue, anxiety, depression and shift work disorder in nurses having less than 11 hours in-between shifts. PLos One. 2013;8(8):e70882. https://doi.org/10.1371/journal.pone.0070882.
- Kurumatani N, Koda S, Nakagiri S, Hisashige A, Sakai K, Saito Y, Aoyama H, Dejima M, Moriyama T. The effects of frequently rotating shiftwork on sleep and the family life of hospital nurses. Ergonomics. 1994;37(6):995–1007. https://doi.org/10.1080/00140139408963713.
- Van Dongen HP, Maislin G, Mullington JM, Dinges DF. The cumulative cost of additional wakefulness: dose-response effects on neurobehavioral functions and sleep physiology from chronic sleep restriction and total sleep deprivation. Sleep. 2003;26(2):117–26. https://doi.org/10.1093/sleep/26.2.117.
- Brooks A, Lack L. A brief afternoon nap following nocturnal sleep restriction: which nap duration is most recuperative? Sleep. 2006;29(6):831–40. https://doi.org/10.1093/sleep/29.6.831.
- Amin MM, Graber M, Ahmad K, Manta D, Hossain S, Belisova Z, Cheney W, Gold MS, Gold AR. The effects of a mid-day nap on the neurocognitive performance of first-year medical residents: a controlled interventional pilot study. Acad Med. 2012;87(10):1428–33. https://doi.org/10.1097/ACM.0b013e3182676b37.
- Purnell MT, Feyer AM, Herbison GP. The impact of a nap opportunity during the night shift on the performance and alertness of 12-h shift workers. J Sleep Res. 2002;11(3):219–27. https://doi.org/10.1046/j.1365-2869.2002.00309.x.
- Tassi P, Muzet A. Sleep inertia. Sleep Med Rev. 2000;4(4):341–53. https://doi.org/10.1053/smrv.2000.0098.
- Hilditch CJ, Centofanti SA, Dorrian J, Banks S. A 30-Minute, but Not a 10-Minute Nighttime Nap is Associated with Sleep Inertia. Sleep. 2016;39(3):675–85. https://doi.org/10.5665/sleep.5550.
- Van Dongen HP, Price NJ, Mullington JM, Szuba MP, Kapoor SC, Dinges DF. Caffeine eliminates psychomotor vigilance deficits from sleep inertia. Sleep. 2001;24(7):813–19. https://doi.org/10.1093/sleep/24.7.813.
- Newman RA, Kamimori GH, Wesensten NJ, Picchioni D, Balkin TJ. Caffeine gum minimizes sleep inertia. Percept Mot Skills. 2013;116(1):280–93. https://doi.org/10.2466/29.22.25.PMS.116.1.280-293.
- Bonnefond A, Muzet A, Winter-Dill AS, Bailloeuil C, Bitouze F, Bonneau A. Innovative working schedule: Introducing one short nap during the night shift. Ergonomics. 2001;44(10):937–45. https://doi.org/10.1080/00140130110061138.
- Chang YS, Wu YH, Lu MR, Hsu CY, Liu CK, Hsu C. Did a brief nap break have positive benefits on information processing among nurses working on the first 8-h night shift? Appl Ergon. 2015;48:104–08. https://doi.org/10.1016/j.apergo.2014.11.005.
- Barger LK, Runyon MS, Renn ML, Moore CG, Weiss PM, Condle JP, Flickinger KL, Divecha AA, Coppler PJ, Sequeira DJ, et al. Effect of fatigue training on safety, fatigue, and sleep in Emergency Medical Services personnel and other shift workers: A systematic review and meta-analysis. Prehosp Emerg Care. 2018;22(S1):58–67.
- Institute of Medicine. Clinical Practice Guidelines We Can Trust. Washington, D.C.: The National Academies of Sciences; 2011.
- Morgan RL, Thayer KA, Bero L, Bruce N, Falck-Yitter Y, Ghersi D, Guyatt G, Hooijmans C, Langendam M, Mandrioli D, et al. GRADE: Assessing the quality of evidence in environmental and occupational health. Environ Int. 2016;92–93:611–6.