Systematic Review and Meta-analysis of the Effects of Caffeine in Fatigued Shift Workers: Implications for Emergency Medical Services Personnel

Abstract

Background: Emergency Medical Services (EMS) workers may experience fatigue as a consequence of shift work. We reviewed the literature to determine the impact of caffeine as a countermeasure to fatigue in EMS personnel and related shift workers. Methods: We employed the GRADE methodology to perform a systematic literature review and search multiple databases for research that examined the impact of caffeine on outcomes of interest, such as patient and EMS personnel safety. For selected outcomes, we performed a meta-analysis of pooled data and reported the pooled effect in the form of a Standardized Mean Difference (SMD) with corresponding 95% confidence intervals. Results: There are no studies that investigate caffeine use and its effects on EMS workers or on patient safety. Four of 8 studies in shift workers showed that caffeine improved psychomotor vigilance, which is important for performance. Caffeine decreased the number of lapses on a standardized test of performance [SMD = 0.75 (95% CI: 0.30 to 1.19), p = 0.001], and lessened the slowing of reaction time at the end of shifts [SMD = 0.52 (95% CI: 0.19 to 0.85); p = 0.002]. Finally, 2 studies reported that caffeine reduced sleep quality and sleep duration. Conclusions: Although the quality of evidence was judged to be low to moderate, when taken together, these studies demonstrate that caffeine can improve psychomotor performance and vigilance. However, caffeine negatively affects sleep quality and sleep duration. More systematic, randomized studies need to be conducted in EMS workers in order to address the critical outcomes of health and safety of EMS personnel and patients. The risk/benefit ratio of chronic caffeine use in shift workers is currently unknown.

Key words::

• caffeine
• EMS workers
• shift work
• psychomotor vigilance
• safety

Background

Fatigue during shiftwork may threaten the safety of Emergency Medical Services (EMS) personnel and their patients (1). Previous research suggests caffeine is an effective fatigue countermeasure for shift workers (2). Despite a large number of observational and experimental studies, the impact of caffeine on safety and other relevant outcomes of EMS professionals remain unclear.

Caffeine is the most commonly consumed psychoactive substance in the world (3). In the United States, over 85% of adults regularly consume caffeine, with an average intake of 180 mg/day (4, 5). Although the neural mechanism of caffeine is well-understood, the conditions under which caffeine improves cognitive and physical performance remain equivocal (6). Low to moderate doses of caffeine improve performance on cognitive tasks (7–10), yet higher doses may result in decrements in performance and increased negative mood or affects, such as anxiety and jitteriness (8, 11). It is unclear if caffeine has primary effects on performance or whether it merely reverses a deficit in performance introduced by caffeine withdrawal (12–14). Individuals who consume caffeine daily often become desensitized to the effects and experience dampened performance. Yet, individuals who consume caffeine on a daily basis may report greater mood enhancing effects compared to low- and non-caffeine consumers (15). Definitive conclusions of the impact of caffeine on performance are hampered by inconsistency in study design and methodology, including different caffeine doses, routes of caffeine administration, amounts of time between caffeine and testing, participant age groups, and states of usual caffeine use and/or withdrawal. The average intake of caffeine in U.S. adults is equivalent to about 2 cups of coffee, yet the use of caffeine may be greater in populations that experience regular periods of sleep disruption or fatigue, such as shift workers (16), active duty military personnel (17), and EMS workers (18). It is important to understand whether caffeine is an effective countermeasure to sleep disruption in this population and if there are any potentially harmful effects of caffeine as well.

We assessed caffeine as a countermeasure to fatigue for EMS shift workers. This systematic review was guided by a single research question developed by members of an expert panel: “In EMS personnel, does the worker's use of fatigue countermeasures mitigate fatigue, mitigate fatigue related risks, and/or improve sleep?” (PROSPERO 2016: CRD42016040101) (19). This was achieved by examining a series of more specific outcomes, including personnel performance, acute states of fatigue, alertness, sleepiness, and indicators of sleep (e.g., sleep quality).

Methods

We used a systematic review of multiple databases of peer-reviewed and non-peer-reviewed literature. Our search methodology, study protocol, and procedures are described in a separate paper in the Supplemental Material (20). The unique features of our protocol for this systematic review are described in the following sections.

Study Design

We assessed the retrieved literature for studies that described use of experimental study designs (i.e., randomized controlled trials and crossover designs).

Types of Participants

Our targeted population of interest was defined a priori by a panel of experts as: EMS personnel or similar worker groups, defined as shift workers whose job activity requires multiple episodes of intense concentration and attention to detail per shift, with serious adverse consequences potentially resulting from lapses in concentration (19). We retained studies that involved EMS personnel or similar shift worker groups 18 years of age and older (20). Studies that involved healthy volunteers, students, and other non-traditional shift worker populations (e.g., military personnel) were considered on a case-by-case basis and included if the protocol tested involved work-related scenarios analogous to EMS shift work, such as long periods of waking that resulted in fatigue, repeated stress during the night, and participants who were accustomed to performing shift work. Disagreements were addressed through discussion among co-investigators (JLT, DH, and PDP).

Types of Interventions

We searched the literature for interventions using caffeine as a countermeasure to fatigue, sleepiness, or for the improvement of alertness during shift work or simulated shift work. We retained studies that included multiple comparisons with caffeine as a component part of one or more study arms (e.g., caffeine versus placebo, caffeine plus sleep versus caffeine only versus placebo, and so on). Studies that did not report on the effects of caffeine as a fatigue countermeasure were excluded.

Types of Outcome Measures

The primary (critical) outcomes of interest were patient safety and EMS personnel safety as defined by the individual study (19). Secondary (important) outcomes of interest were personnel performance, acute states of fatigue, alertness, and sleepiness; indicators of sleep (e.g., sleep quality); employee retention/turnover; long-term health indicators (e.g., cardiovascular disease); and cost to the system.

Search Methods for Studies

A research librarian (PMW) searched 5 bibliographic databases and one website. The details of our methods and search strategy are published separately (20), where we identify all sources searched, the search terms incorporated, and the description of search vocabulary. See Online Supplemental Material for search strategy specific to this systematic review.

Data Collection and Selection of Studies

Screening

Co-investigators (CMG and JPC) independently screened titles and abstracts to identify potentially relevant publications. Co-investigators (PDP and DJS) adjudicated disagreements against inclusion/exclusion criteria: a) the study describes the population of interest; b) the study describes shift duration as the primary comparison of interest; c) the title and/or abstract describes one or more outcomes of interest. The Kappa statistic was used to determine agreement between screeners (21).

Full-Text Review

Two investigators (PDP and JPC) worked independently to abstract key information from full-text articles and then verified the abstractions of the other investigator. Key information included study design, participant characteristics, intervention characteristics, comparisons, outcome measures, and key findings (Online Supplement Material). Disagreements were handled with discussion. Book chapters, conference abstracts, newsletters and similar publications, dissertations, and theses were excluded. Two co-investigators (PDP and JPC) searched bibliographies to identify additional relevant research and reviewed the full-text article to determine inclusion or exclusion.

Risk of Bias Assessment

Our team's 3 senior co-investigators (JLT, DH, and PDP) determined risk of bias using the Cochrane Collaboration's Risk of Bias tool for experimental studies (22). The Cochrane tool evaluates the risk of bias across 6 domains: selection bias (i.e., sequence generation and allocation concealment); performance bias (i.e., blinding of participants and personnel); detection bias (i.e., blinding of outcome assessment); attrition bias (i.e., incomplete outcome data); reporting bias (i.e., selective reporting); and other bias (i.e., other sources of bias not addressed in other domains) (22). Disagreements in assessing bias were handled by discussion.

Statistical Analysis

Co-investigators (JLT, DH, and PDP) used a categorical system adopted by Bolster and Rourke (23) to describe the impact of a caffeine intervention on critical and important outcomes as favorable, unfavorable, mixed/inconclusive, or no impact. Favorable was assigned when co-investigators JLT, DH, and PDP determined findings favored the intervention (caffeine). Unfavorable was assigned when findings did not favor the intervention. Mixed/inconclusive was assigned when findings show both positive and negative impacts on select outcomes with multiple components (e.g., a composite or index measure) or when the results reported on an outcome were insufficient to draw a conclusion or interpret impact. No impact was assigned when co-investigators JLT, DH, and PDP determined the intervention showed no statistical and/or clinically meaningful impact on outcomes.

We pooled data for purposes of a meta-analysis when 2 or more studies used an experimental study design and reported results for the PVT outcome. We used the RevMan software to calculate the pooled standardized mean difference (SMD) and 95% confidence interval (CIs) for each outcome. The SMD is the estimated intervention effect relative to variability (22). The I² statistic was calculated as a component of the meta-analysis and presented as a standard measure of heterogeneity (22). The I² is the percentage of total variation across studies relative to heterogeneity (not chance). The I² ranges from 0% to 100% with higher values implying sizeable heterogeneity (22).

Co-investigators (JLT, DH, PDP, and ESL) used the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework to summarize and rate the quality of retained research (evidence) (20, 24, 25). Evidence profile tables present information regarding the quality of evidence connected to outcomes. Quality of evidence is rated from very low, low, moderate, to high.

Results

The search strategy yielded n = 1,401 unique records of which n = 23 duplicates were removed manually (Figure 1). Co-investigators (CMG and JPC) independently screened n = 1,378 titles and abstracts. The inter-rater agreement for inclusion/exclusion was fair (Kappa = 0.55). Eighteen records were judged potentially eligible based on title and abstract. Twenty-one additional studies were identified during bibliography searches as potentially relevant and reviewed in full-text format. There are no studies that investigate caffeine use and its effects on EMS workers or on patient safety. Eight experimental studies were determined relevant to the study's research question and key^{Table 1} findings (Table 1, Online Supplement Material). These studies involved other shift worker groups and/or tested protocols germane to EMS shift work. Thirty studies were excluded with reasons and reported in the Population, Intervention, Comparison, Outcome (PICO) format (See Online Supplement Material) (26–28). See Online Supplemental Material for completed risk of bias forms for the retained research.

Figure 1. PRISMA Flow Diagram PICO#3 PROSPERO 2016:CRD42016040101.

Impact of Caffeine as a Countermeasure to Fatigue in EMS Shift Workers: Performance

The impact of caffeine on performance was analyzed using meta-analytic techniques on 2 measures of performance linked to the psychomotor vigilance test (PVT): reaction time and number of lapses. For both measures, caffeine improved performance in the form of faster reaction time and reduced number of lapses compared to the placebo/control condition.

Caffeine vs. Placebo: Effect on Psychomotor Vigilance

Reaction Time

Four experimental studies measured reaction time defined as the average amount of time it took an individual to make a response after the cue was presented in the task (29–32). Caffeine lessened the slowing in reaction time at the end of shift [SMD = 0.52 (95% CI: 0.19 to 0.85); p = 0.002; Figures 2a–2b]. There was no evidence of heterogeneity [ChiSq = 1.63; df = 3 (p = 0.65); I² = 0%].

Table 1. Synthesis of findings of the impact of caffeine as a countermeasure to mitigate fatigue, fatigue related risk, and or improve sleep for selected outcomes

EXPERIMENTAL STUDIES
			Critical Outcome	Important Outcomes
Author, Year	Distiller RefID^* PMID^*	Study Design	Personnel Safety	Patient Safety^†	Personnel Performance^*	Acute Fatigue^‡	Sleep and Sleep Quality^§	Long-Term Health^║
Kamimori	RefID-587	Double-blind, placebo	—	—	Favorable	—	Unfavorable	—
et al., 2015 (31)	PMID-25527035	controlled – caffeine (800 mg) vs. placebo
Kamimori	RefID N/A	Double-blind, randomized,	—	—	Favorable	No	—	—
et al., 2005 (33)	PMID-16313140	counterbalanced design: effects of 0, 50, 100, or 200 mg of caffeine on psychomotor vigilance				impact
Schweitzer	RefID-1104	Cross-over design with	—	—	Favorable	Favorable	Unfavorable	—
et al., 2006 (32) (field study only)	PMID-16453980	participants randomized to study order. Two conditions: caffeine (all nights) + evening nap vs. placebo (all nights) and no nap
Ronen et al.,	RefID-1053	Within-subjects crossover	Favorable	—	Favorable	Favorable	—	—
2014 (34)	PMID-24913484	design comparing control, energy drink, and energy drink + rest
McLellan,	RefID N/A	Double-blind, placebo	—	—	Favorable	—	—	—
Kamimori, Voss, et al., 2005 (29)	PMID-16018347	controlled design comparing placebo vs. caffeine
McLellan,	RefID N/A	Double-blind, placebo	—	—	Favorable	—	—	—
Kamimori, Bell, et al., 2005 (30)	PMID-15672985	controlled comparison of placebo vs. caffeine
McLellan,	RefID-818	Double-blind, placebo	—	—	Favorable	—	—	—
2004 (36)	PMID-15328782	controlled comparison of placebo vs. caffeine
Doan et al.,	RefID-329	Double-blind, placebo	Favorable	—	Favorable	Favorable	—	—
2006 (35)	PMID-17042248	controlled comparison of placebo vs. caffeine

Findings are classified as favorable, unfavorable, mixed/inconclusive, or no impact.

^* Includes external objective and subjective ratings of the study subject's performance including perceived satisfaction with the subject's performance.

^† Includes quality of care.

^‡ Includes acute states of fatigue, sleepiness, and alertness.

^§ includes sleep latency, total sleep time, recovery, and related measures.

^║ General wellness or well-being measures included.

Figure 2. Forest plots for Standardized Mean Difference between placebo and caffeine reaction time and lapses on the PVT test. a) Reaction time at end of shift. b) Reaction time change/delta from start-to-end of shift. c) Lapses at end of shift. d) Lapses change/delta from start-to-end of shift. Figures 2a–2d Notes: report the standardized mean difference (SMD) for reaction time (a and b) and lapses (c and d) on the psychomotor vigilance test in participants receiving caffeine or placebo. The mean ± SD reaction time (ms) was calculated using data presented in each of the manuscripts and entered into RevMan to generate forest plots. All of the studies defined lapses as responses that were made after a specified amount of time had passed (500 ms – 5 seconds). The mean ± SD number of lapses were available for the 2 McLellan manuscripts, but only the mean number of lapses was available for the Kamimori et al. manuscript (29, 30, 33). In order to estimate the SD for this measure, we used a conservative approach of doubling the mean, which was a significantly greater proportion of the mean than either of the other studies. In order to generate data, we relied on means and standard deviations/errors reported in each manuscript. We estimated the means and standard errors presented in graphs when means and standard deviations/errors were not reported. In all cases, we selected all time points after caffeine administration and averaged them across placebo and caffeine conditions. Data for PVT lapses were collected as follows: For the Kamimori et al. 2005 manuscript, data were found in Figure 1 (33). Caffeine administration was indicated with dashed lines. All points after the first caffeine administration (305) were averaged up until time point 1100 hours for placebo and 200 mg of caffeine. The final time point was considered too long after caffeine administration, given a half-life of approximately 4.5 hours. For the McLellan, Kamimori, Voss, et al. 2005 paper, means and standard deviations were reported in the manuscript (29). For the McLellan, Kamimori, Bell, et al., 2005 paper, data were extracted from Figure 4b from the referenced manuscript (30). Caffeine was administered on Day 2 at 2130. We averaged the number of lapses for all time points on Day 3 for the placebo and caffeine conditions. We obtained PVT reaction time data as follows: For the Kamimori et al. 2015 manuscript, data were taken from Figure 1 and converted to reaction times (31). We averaged data from all time points after the initial caffeine dose at 2100 hours, excluding the time points 1815 and 2100 on Days 3 and 4, as those were too long after the previous caffeine administration to be considered relevant. For the McLellan, Kamimori, Voss, et al. 2005 manuscript, data were taken from Figure 4 from the referenced manuscript (29). We averaged all time points on Day 3 after the initial caffeine administration at 0140 hours for both placebo and caffeine. For the McLellan, Kamimori, Bell, et al. 2005 manuscript, data were extracted from Figure 4a from the referenced manuscript (30). Caffeine was administered on Day 2 at 2130 hours. We averaged the reaction time for all time points on Day 3 for the placebo and caffeine conditions. For the Schweitzer et al. 2006 manuscript, data were taken from Figure 5 in the referenced manuscript and converted to reaction time (32). We used the second time point, as the first time point on each night was prior to caffeine administration.

Number of Lapses

Three experimental studies measured the number of PVT lapses (29, 30, 33). Lapses were defined as the number of reaction time responses exceeding arbitrary cutoffs. The cutoffs were defined as minor lapses (1000 – 2999 ms), moderate lapses (3000 – 4999 ms), or major lapses (> 5000 ms) (33). Reported lapse data varied among studies. The McLellan et al. studies reported total lapses (29) or minor lapses only (30). We used the moderate lapses from the Kamimori et al. study in order to have values in between the other 2 studies (33). Caffeine decreased the number of lapses [SMD = 0.75 (95% CI: 0.30 to 1.19), p = 0.001; Figures 2c–2d]. There was no evidence of heterogeneity [ChiSq = 0.29; df = 2 (p = 0.86); I² = 0%].

Impact of Caffeine as a Countermeasure to Fatigue in EMS Shift Workers: Safety, Sleep, and Acute Fatigue

Of the 6 critical and important outcomes, we were able to categorize the impact of caffeine on personnel safety, acute fatigue, and sleep/sleep quality as favorable, unfavorable, mixed/inconclusive, or no impact (23).

Personnel Safety

Two experimental studies assessed personnel safety in two ways (34, 35). The Doan et al. study used an adaptive tracking task to simulate a night mission in pilots (35). Deviation from a vertical line within the horizontal plane was used as the measure of accuracy. In the Ronen et al. study, a driving simulator was used to assess lane position and deviation in wheel position and speed (34). We interpreted these studies as being related to personnel safety because EMS workers have to be able to transport patients and themselves; thus impaired performance on these simulators would indicate reductions in EMS personnel safety. Caffeine improved simulator performance relative to the placebo condition. Interpretation of impact was favorable.

Sleep/Sleep Quality

Two experimental studies reported sleep or sleep quality data (31, 32). The Schweitzer et al. study asked participants to record sleep duration in a sleep diary and had participants complete the Karolinska Sleepiness Scale (32). The Kamimori et al. study used Actigraphs to objectively record the duration of sleep and wakeful bouts (31). Both studies showed that caffeine was associated with reduced sleep duration and reduced perceived sleep quality relative to placebo. Interpretation of impact was unfavorable.

Acute Fatigue

Acute fatigue was assessed^{Figure 2} in four experimental studies (32–35). The Kamimori et al. study examined fatigue using the Stanford Sleepiness Scale which asked participants to rate statements about fatigue and alertness on a scale from 1 – 7 with 1 = “feeling active, vital, wide-awake” and 7 = “sleep onset soon, losing struggle to remain awake” (33). The Schweizter et al. study asked participants to rate fatigue using the Karolinska Sleepiness Scale, which also uses a scale from 1 – 7 with 1 = very alert and 7 = very sleepy (32). The Ronen et al. study examined acute fatigue using the Swedish Occupational Fatigue Inventory-20, which asked participants to rate various subjective states, such as lack of energy or sleepiness, on a scale from 1–9 (34). The Doan et al. study reported that participants reported fatigue every 2 hours and also reported the “fatigue” values from the Profile of Mood States (35). The retained studies assessed fatigue/sleepiness using different instruments. Three of four (32, 34, 35) reported that caffeine reduced fatigue (at 2–8 hours after caffeine or placebo administration) relative to placebo and the fourth study (33) reported no impact of caffeine on fatigue.

Quality of Evidence

We determined that most of the studies presented a low risk of bias, with randomized, double blind procedures specified in all but one study, nearly complete outcome data, and no evidence of selective reporting. We found good consistency among reported studies. We downgraded for indirectness of evidence and imprecision for several of the outcomes. This was due to differences in study populations, outcome measures, and interventions as well as small sample sizes in all of the studies. When these considerations were taken together, we viewed the quality of the evidence as moderate-to-low (personnel performance), low-to-very low (acute fatigue, sleep/sleep quality, and personnel safety), and non-existent (patient safety and long-term health of providers) (See Online Supplemental Material).

Discussion

Summary of Main Results

Evidence from experimental studies suggests that caffeine can act as a countermeasure for fatigue under conditions of sleep deprivation like those experienced by EMS workers. Caffeine improves psychomotor vigilance, which is important for performance. In addition, three of eight studies demonstrated that caffeine reduced acute fatigue/sleepiness during the experiment. Two of the studies showed that caffeine may be unfavorable to sleep duration and sleep quality. A side effect of caffeine consumption is reduced sleep duration, which should be considered in the context of shift workers. This review of the published literature highlights the fact that direct studies are needed in EMS workers to better understand the broader range of potential impacts of dietary caffeine. None of the studies reviewed examined the impact of caffeine on patient safety or long-term health of EMS workers. It is important to address these areas in order to have a comprehensive understanding of the impact of caffeine on the health and safety of both EMS workers and their patients.

Inclusion/Exclusion of Prior Research

The decision to include or exclude a research study was based on relevance to EMS workers. We included papers that used shift workers, military personnel, and healthy volunteers as long as the protocol followed was judged to be relevant to EMS operations. For example, studies in healthy volunteers were included if those individuals were put through a sleep deprivation protocol that simulated shift work. We included studies based on the dependent measures used. For example, studies were included if they examined psychomotor vigilance, fatigue, or a relevant measure of performance (e.g., adherence to a flight path), but we excluded manuscripts where the dependent measures were less relevant for EMS workers (e.g., marksmanship or transcranial stimulation). We included studies that had sleep deprivation in their protocol, mimicking shift work or night shifts.

Quality of Evidence

Incomplete descriptions of randomization procedures and allocation concealment were common in most of the studies we reviewed and limited our ability to accurately assess risk of bias and study quality. In addition, each of the studies reviewed used different doses, route of administration, and patterns of caffeine administration. This is important because the half-life of caffeine is relatively short (3–6 hours) (37) and, thus, the timing and dose of administration is important for delivery of sustained effects. Our pool of included studies was too small to be restrictive regarding the specific details of caffeine administration. Overall, we determined the research retained in this systematic review presented a low risk of bias. Our assessments of evidence quality were guided by the GRADE methodology and varied by outcome from low to moderate. This was, in part, due to the fact that the majority of the studies were not conducted in EMS workers. The GRADE process allows us to include articles that we deem relevant to our population of interest (EMS shift workers), but then downgrade the quality for indirectness.

Agreement and Disagreement with Other Systematic Reviews

This systematic review is similar in scope to an earlier review by Ker and colleagues (2). They examined the impact of caffeine on prevention on injuries and error in shift workers. Their interpretation of the evidence is similar to ours, with evidence that caffeine can improve personnel performance (measured as cognitive performance), but difficulty determining whether these improvements translate directly into improvements in safety. The current review advances our understanding of caffeine and its impact by focusing on psychomotor vigilance, which is critical for EMS worker performance, and by restricting the analysis to study protocols and study participants exposed to shift work conditions. Both the current review and the paper by Ker et al. agree with the larger body of evidence involving healthy adults, which shows that acute caffeine administration effectively combats fatigue and improves alertness.

Limitations

Our collection of relevant literature (data) was limited to select databases. Other databases may index literature and research relevant to our PICO question. Agreement between screeners on the decision to include or exclude a record during the initial screening of titles and abstracts was fair (Kappa = 0.55). This Kappa value is slightly less than other systematic reviews, yet is comparable (38–40). We acknowledge that fair agreement during the initial screening phase may have led to some research related to our PICO being overlooked. We addressed this limitation by adding the additional step of reviewing the bibliographies of retained literature and searching for potentially relevant literature involving use of caffeine in experimental studies. This added step, which is not common to all systematic reviews, yielded 20 additional studies that underwent full-text review. We also evaluated the decision making of our screeners by randomly selecting n = 50 records from the initial pool of screened literature and tasking a third co-investigator (PDP) to render an include/exclude decision. The percentage agreement between the 2 screeners and the third reviewer was 100% based solely on the title/abstract. We believe our added search of bibliographies and the 100% agreement between three reviewers germane to the initial screening is evidence that our search was comprehensive and that few studies germane to our PICO, if any, were excluded from our analysis.

One study retained for meta-analysis failed to report standard deviation values for PVT lapses (33). We imputed the standard deviation for purposes of computing the SMD for PVT lapses. We evaluated the impact of using estimations of standard deviation of PVT lapses by varying the imputed values using the proportion of SD to the mean from the other included studies, using the mean as the SD, and using 150% and 200% of the mean as the SD. All variations resulted in favorable impacts of caffeine on PVT lapses. We chose to use the most conservative approach in our analysis (200% of the mean as the SD). The impact was non-significant.

Although this meta-analysis was not meant to be inclusive of the totality of caffeine research, there are some concepts that are lacking from the literature that are particularly relevant to our population of interest. None of the studies reviewed here address the impacts of long-term caffeine use on these performance measures or on the broader concept of EMS personnel and patient health and safety. There are several important interactions to consider. First, chronic caffeine use can result in tolerance to the effects of caffeine (41,42). This could make acute caffeine use a less effective countermeasure and, therefore, lead to escalating doses of caffeine to maintain the same effect. Second, caffeine use can result in sleep disruption (43). This could also impact fatigue among EMS workers and reduce patient safety. Third, chronic caffeine use can disrupt circadian rhythms, resulting in a phase delay of circadian melatonin secretion when used within several hours of bed time (44), especially when used during overnight shifts. This contributes to “night shift syndrome”, which increases the risk of chronic disease and other health problems among night shift workers (45). Fourth, Caffeine Use Disorder and Caffeine Withdrawal Disorder are emerging in the literature as legitimate diagnoses in a subset of the population with high, chronic consumption of caffeine (46). It is possible that the fatigue associated with night shifts and shift work may promote excess intake of caffeine, which could increase the risk of these disorders. As the classification of these disorders develops, it will be important to determine if shift workers and EMS workers are an at risk population. Conversely, there are some studies reporting potential benefits of chronic caffeine use on health, including a lower incidence or risk of death from Type II Diabetes, cardiovascular disease, respiratory illness, and injury (47–49). However, these data were overwhelmingly collected from individuals who are not shift workers. The risk/benefit ratio associated with chronic caffeine use in shift workers in unknown (45). Clearly, more work is needed to evaluate the relative risks and benefits in the EMS worker population.

Conclusions

Caffeine is a commonly used stimulant that has known effects on fatigue, alertness, energy, and cognitive performance. Its use among shift workers who are expected to be awake and alert when most others are sleeping is even more widespread. Given that EMS workers are often dealing with acute threats and are asked to perform life-saving procedures in patients, it is important to understand the empirical data on the effectiveness and efficacy of caffeine as a countermeasure to fatigue in this population and under the conditions in which these individuals work. The studies reviewed here demonstrate that caffeine improves personnel performance and reduces acute fatigue. However, we unfortunately found that caffeine also reduces sleep duration and quality. We judged the quality of evidence as low to moderate based on bias affecting selective reporting, random sequence generation, and allocation concealment. Other factors affecting quality of evidence included a perceived serious impact of indirectness and imprecision for some of the reviewed articles. More studies need to be conducted to determine if these improvements in performance translate into better personnel and patient safety. In addition, no studies on shift workers have investigated the long-term impact of caffeine use on personnel health. This systematic review and meta-analysis is important because it not only highlights what we know, but also what we do not yet understand.