542
Views
0
CrossRef citations to date
0
Altmetric
Research Article

A cross-sectional assessment of the effects of select training modalities on vaccine cold chain management

, , , &

ABSTRACT

Background

Vaccines offer arguably the most cost-effective public health intervention. Vaccine supply chain management which is a critical building block faces many Human resources challenges mainly due to the special attributes of vaccines.

Objective

This study attempted to measure the effect of training on vaccine cold chain handler knowledge and practices.

Methods

A cross-sectional research design, using predominantly quantitative data collection techniques, was used. Facilities that have offered vaccination services for more than a year and report through the HMIS system were eligible for selection. Observation checklists and structured questionnaires were used. SPSS was used to analyse data.

Results

Vaccine cold chain management among the study group had an average score of 65.33% range (31–85%). The average knowledge score among the study respondents was 62.42% with a range (45–95%). The knowledge of respondents generally increases with an additional increase in the number of training modalities.

Conclusions

The status of VCCM is at about 65.33% below the target of 80% set by the EVM. The trainings have an effect on both knowledge of handlers and their practice especially when deployed in a multi-pronged design and thus these trainings need to be aligned to achieve synergy.

ABBREVIATIONS

CCE, Cold Chain Equipment; DHIS2, District Health Information Systems 2; DHO, District Health Officer; DPT, Diphtheria, Pertussis, Tetanus; DVS, District Vaccine Stores; EPI, Expanded Program for Immunisation; EVM, Effective Vaccine Management; FEFO, First Expiry First Out; GAVI, Global Alliance for Vaccines and Immunisation; HMIS, Health Information Management Systems; IRC, International Rescue Committee; KII, Key Informant Interview; LIAT, logistics indicator assessment tool; PATH, Program for Appropriate Technology in Health; PHC, Primary Health Care; QPPU, Quantification and Planning and Procurement Unit; SOPs, Standard Operating Procedures; SPSS, Statistical Package for Social Sciences; UNEPI, Uganda National Expanded Program for Immunisation; UNICEF, United Nations Children's Fund; VPD, Vaccine Preventable Diseases; VVM, Vaccine Vial Monitors; WHO, World Health Organisation

Background

Vaccines have a pivotal role in public health being described as ‘the most cost-effective public health intervention’ with a return on investment of 16 USD accrued from a cost saving and reduced illness-induced loss of productivity for every 1 USD spent on vaccination (Ozawa et al., Citation2016; Riedmann, Citation2010; Sillanp, Citation2015)

To obtain the full benefits of vaccination, it is imperative that the vaccines are maintained in appropriate and adequate cold chain conditions throughout the supply chain (Lloyd & Cheyne, Citation2017). Any break in the cold chain may result in an unrecoverable loss in potency lowing effective coverage leading to outbreaks of some vaccine-preventable diseases regardless of reported good coverage (The Maternal and Child Survival Program (MCSP), Citation2016; Orenstein et al., Citation1985). At the facility level, all the vaccines are stored within a recommended temperature range of 2–8°C in most cases (Kartoglu & Milstien, Citation2014). Critical to note is that these vaccines have two major categories with a bunch of them being sensitive to heat events and another bunch being sensitive to freeze events (Hanson et al., Citation2017). These special sensitivities determine the section of the cold chain storage equipment where these categories can be stored. Exposure to heat events can be tracked on the Vaccine Vial Monitor (VVM) which is a chemical indicator that records cumulative heat events and reports them as a colour change (Eriksson et al., Citation2017). Freeze events which have a quicker blow on freeze-sensitive vaccines do not have any form of direct tracker as such need to be monitored using temperature monitoring equipment (Hanson et al., Citation2017). Any suspicion of a freeze event should be investigated using the shake test (Hanson et al., Citation2017)

An adequate number of knowledgeable, skilled and motivated human resources is critical in maintaining an effective and efficient vaccine cold chain (Steele, Citation2015). Unfortunately, this is not the case on the ground as seen in the citations below: In Ethiopia, about 45.4% of the vaccine handlers were classified as not having satisfactory knowledge and skills, for example, inability to read and interpret the readings on the temperature monitoring equipment (Lutukai et al., Citation2019). Suboptimal data use, analysis, and interpretation are into actionable decisions (Woldemichael et al., Citation2018) among vaccine handlers. This lack of adequate knowledge has been determined as a predictor of practice. In Ghana, whereas the knowledge of most respondents was generally considered satisfactory, it was noted that the application of this knowledge was limited. For example, out of the 100% who had heard of VVM, 85% could correctly read VVM but only 19% could correctly state the implication if the colour of the inner square and outer circle of a Vaccine Vial Monitor (VVM) matches (Osei et al., Citation2019). The use of unqualified community vaccinators, who lack formal training or formal contracts with the facility, was observed in Uganda (Karlsson, Citation2012). These observed gaps impact vaccine management in many ways most importantly causing a break in the cold chain and exposing the vaccines to quality compromise which negatively affects the potency and safety of vaccines at the vaccination site (Zaffran et al., Citation2013).

As an affirmative action to address these gaps, a plethora of human capital development interventions have been deployed to build the capacity of vaccine handles at all levels. These include the creation of international professional networks (Brown et al., Citation2017), the creation of regional centres of excellence like the East African region has one hosted in Kigali Rwanda with a focus on the area of vaccines immunisation and health supply chain (Brown et al., Citation2017), multiple certifications at certificate, diploma, degree and post-graduate are widely available all over the world, and supporting toolkits and resource repositories (Brown et al., Citation2017). The knowledge, skills, morale and performance of staff can be enhanced through interactive in-service training usually organised as workshops (Masresha et al., Citation2020), pre-service and in-service training and supervision (Adebimpe & Adeoye, Citation2021; Masresha et al., Citation2020)

Despite this conducive training environment, it was noted that 60% of supply chain roles including those in vaccine cold chain management are performed by human resources without supply chain certifications (Kasonde & Steele, Citation2017). A study in Kenya showed that little or no time was allocated to EPI topics in preservice training and lecturers in nursing schools also needed some refresher training (Zaffran et al., Citation2013). These factors jointly call for a focus on on-the-job training with many comprehensive relevant training known yet with limited access and a general lack of strategy to deliver training.

The commonest modalities cited include but are not limited to Offsite training, Onsite training, support supervision, provision of reference materials in the form of (manuals, guidelines, SOPs, and Job Aids) and self-paced learning on virtual platforms (Kasonde & Steele, Citation2017)

The outcome of these interventions in knowledge and practice improvement however has not been extensively studied in Uganda, particularly in the Lango Sub region. This study set out to determine the effect of the human resources’ capacity-building interventions on the knowledge and practices of the vaccine cold chain handlers.

Methods

Study setting and intervention

The study was conducted in the Lango sub-region located on the northern side of Uganda with a population of close to 2.5 million people. The region has 10 administrative districts including the recently chartered Lira city with a mix of public and private facilities across all levels of care offering immunisation services. The city, which is the most urban business hub in the region, is about 344 km away from the country's capital Kampala. The region has a tropical climate characterised by very hot dry seasons which could prove challenging for cold chain logistics. The facility mix, weather aspect, and poverty levels jointly formed the ground for the choice of the sub-region for the study.

Study design

A cross-sectional study design was employed to assess the effects of HR interventions on knowledge and practice. Quantitative techniques were deployed to measure interventions, knowledge and practices. With a cross-sectional study, rich data on exposure and outcome were abstracted fast and all at a single point in time.

Study population

The sub-region has 375 facilities comprising 1 regional referral hospital (RRH), 7 general hospitals (GH), 15 health centre four (HCIV), 187 health centre, three (HCIII) and 165 health centre two (HCII) (The Ministry of Health Uganda, Citation2018)

Selection of participants

Inclusion criteria

Health facilities in the Lango sub-region that had offered immunisation services for at least one year before May 2023 were included in the study. Additionally, facilities that granted the investigators permission to take part in the study were included.

Exclusion criteria

Health facilities in the Lango sub-region that did not have functional cold chain equipment by May 2023 were excluded from the study.

Sample size calculation

Using the USAID/DELIVER PROJECT guidance in the logistics indicator assessment tool (LIAT) a sample size of 15% of the population is representative of the assessment of pharmaceutical logistics indicators. (USAID | DELIVER PROJECT TO Citation1, Citation2009), 37515%=56.25facilities

The estimated sample size of 57 was stratified by district and then the level of care to obtain a representative sample composition. Within the strata, simple random sampling was used to obtain the facilities to recruit as below 1 RRH, 3 GH, 6 HCIV, 25 HCIII, and 22 HCII.

Data collection and analysis

A structured questionnaire, developed with ideas from multiple data collection tools of similar studies (Mohammed et al., Citation2021; Woldemichael et al., Citation2018), was used to collect data on participant biodata, individual, and professional characteristics and knowledge. The tool was tailored to collect sequential specific relevant cold chain knowledge areas required of a cold chain handler. Quantitative data on cold chain practices at the facility were obtained using observation checklists which were used to measure on-spot cold chain practices at the facility. The data collection was conducted by the PI to ensure consistency, reliability and validity.

The raw coded data from the tools was entered in Microsoft Excel and exported to Statistical Package for Social Scientists (SPSS) version 23 for analysis.

Descriptive statistical techniques were used to analyse data and results were presented in frequencies, percentages, average ranges, and standard deviations and later categorised and presented as dichotomous categories based on satisfactory or non-satisfactory grading. The outputs were then summarised using tables and where applicable visualisation was enhanced using appropriate figures. Relevant tests, particularly the chi-square tests, were run to determine the associations between the dependent and independent variables. A critical value of p < 0.05 was considered as the cut-off for statistical significance to assert an association.

Results

Health facility characteristics

A total of 57 facilities stratified to represent all levels of care and ownership were targeted out of 375 health facilities to participate in the study. All 57 facilities were enrolled on the study yielding a response rate of 100%. The ownership spread was as follows. Public 47(82.46%) Private Not for Profit (PNFP) 9(15.79%) and Private for profit 1(1.75%).

The level of care distribution was as follows: 1(1.75%) Regional Referral Hospital (RRH) in the region participated in the study. 3(5.26%) General Hospitals (GH), 10(17.54%) Health centre fours (HCIV), 23(40.35%) health centre threes (HCIII) and 20(35.09%) health centre twos (HCII).

Respondents’ demographics

describes the respondent demographics. There was a gender balance among the respondents. Professional certificates 30(52.6%) followed by professional diplomas 21(36.8%) were the most predominant level of education. The majority of the respondents were EPI focal persons 52(91%).

Table 1. Respondent characteristics.

The majority of the respondents were nurses and midwives with 46(80.7%) having more than 1 year of experience in vaccine cold chain management.

The status of vaccine cold chain management practices and performance in participating facilities

shows that among the many areas of vaccine cold chain management practice assessed, the areas of best performance included: VVM application and vaccine storage unit access control at 100% each, and vaccine storage unit positioning also had a good compliance level at 91%. Temperature excursion incident reporting and wastage recording were the worst performed with 0(0%) and 1(1.75%) correct practice, respectively.

Table 2. Facility compliance to set standards of key vaccine cold chain management practices.

Vaccine cold chain management among the study group had an average score of 65.33% with a maximum score of 85% and a minimum of 31%. On a binary scale with 60% or more practice scores as a cut for a satisfactory level of performance, the majority of facilities 46(80.7%) were considered to have a good level of performance while 11(19.3%) were considered as having an unsatisfactory level of practice.

Training deployed to improve the knowledge and skills of vaccine cold chain handlers

shows the exposure level to the training investigated in this study. Reference materials were the most populous modality at 98.2%, while the use of the internet was the least utilised modality to gain knowledge and skills at 42%.

Table 3. Trainings deployed.

shows the multiplicity of the different trainings. Key to note is that at least all respondents had exposure to more than one modality of intervention. Most respondents 31(54.4%) had been exposed to four types of interventions.

Table 4. Number of different types of training participating facilities were exposed to.

Knowledge of vaccine cold chain handlers that participated in the study

Using a 22-theme area questionnaire, the average knowledge score among the study respondents was 62.42% with a minimum score of 45 and a maximum score of 95%.

On a binary scale with a 60% or more knowledge score as a cut for a satisfactory level of knowledge, the respondents were almost equally split with 28(49.12%) not satisfactory and 29(50.88%) with satisfactory knowledge level.

The effects of the training deployed on the knowledge of respondents

Five training models were investigated for their effects on the knowledge of handlers. It was found that of these, Peer-to-peer learning which was termed as learning from a colleague at work and the use of the internet had the greatest positive impact on knowledge gain as demonstrated by a mean score difference exposed compared to the non-expose groups. The difference was however not statistically significant. Technical support supervision was found to have a negative impact on knowledge as demonstrated by the reduction of knowledge score mean in the exposed group from 68.3% in the non-exposed to 62.09 in the exposed group.

shows that the mean knowledge score generally increased with an increasing number of interventions a participant was exposed to previously. With an average mean increase of 3% for every additional intervention a participant is exposed to.

Figure 1. Comparison of means against the number of different types of training participants were exposed to.

Figure 1. Comparison of means against the number of different types of training participants were exposed to.

The effects of the interventions deployed on vaccine cold chain management practices

shows that the mean vaccine cold chain management practice scores change for a number of background and direct interventions. Particular mention can be made for facility ownership which had a p-value of 0.007 and respondent qualification with a p-value of 0.000. Whereas training in itself did not have a significant statistical impact on practice, the duration since the training had a p-value of 0.005 implying it affected practice. The use of the internet was the final intervention which had a significant effect on practice with a p-value of 0.045.

Table 5. Comparison of vaccine cold chain management score means against the various interventions.

Unlike in Knowledge, the number of interventions did not seem to directly impact the mean practice scores.

The effect of the knowledge of respondents on vaccine cold chain management

highlights that the study found a strong association between participant knowledge and practice with a p-value of 0.000 as in the table above.

Table 6. Correlation between knowledge scores and vaccine cold chain management practice scores.

Discussion

The status of vaccine cold chain management practices

This current study showed that 95% of the facilities had temperature monitoring equipment. This was higher than in a study in Cameroon, where 76% of the facilities in the study population had functional temperature equipment (Yakum et al., Citation2015), However, of the 95% with temperature monitoring equipment in this current study, 5(9%) had non-functional equipment and were unaware of it. Further analysis showed that of these, 4/5 had temperature readings for the morning of the visit yet had non-functional equipment in the vaccine storage unit this casts doubt on the reported readings in the temperature charts across the study population. In this study, 77% of the facilities monitored had recorded temperature twice a day this was higher than a study in Ethiopia that had only a 51.2% complete temperature monitoring rate (Feyisa, Citation2021).

This current study showed that 95% of the respondents knew that a shake test was necessary to investigate a suspected freeze event this was not in agreement with a study conducted in Uganda and Senegal which revealed a challenge in freeze event detection (Luzze et al., Citation2017).

In this study, despite the demonstrated knowledge of shake tests, only 39% knew how to safeguard freeze-sensitive vaccines from freeze events by placing them in the right section of the vaccine storage unit this agreed with the same study above conducted in Uganda and Senegal.

This study had an outstanding 100% non-compliance with the recommendation to track wastage this reiterated the same findings in a previous study in Uganda and Senegal which demonstrated a low level of tracking of wastage (Luzze et al., Citation2017)

Training deployed to improve vaccine cold chain management

In this study, 5 training modalities were interrogated and it was shown that 57(100%) of the participants were exposed to at least a mix of two different trainings. The study further goes on to show that the average knowledge score increased by about 3% with every additional exposure to an additional training modality. These findings were in agreement with findings from another study that evaluated vaccine management human capital factor in observed practices, according to the researchers, it was concluded that a multi-pronged training deployment yielded better results when aiming at improving practices (Kasonde & Steele, Citation2017).

Knowledge of vaccine handlers

This current study found that 50.9% of the vaccine handlers had satisfactory knowledge of vaccine cold chain management principles. This finding is similar to a study in Ethiopia which had a result of 54.6% of the vaccine handlers classified as having satisfactory knowledge (Woldemichael et al., Citation2018). However, this differs from a study in Yemen which found 80% as the proportion of handlers with sufficient knowledge (Sule, Citation2022).

In this current study, the investigators demonstrated that 81% of the respondents could correctly define a VVM and this could be slightly lower than a study in Ghana which showed that 100% of the respondents had heard of VVM (Osei et al., Citation2019). In this current study, however, the investigators went a little deeper than merely hearing about VVM to define it. Furthermore, in this study, 70% could correctly identify from a picture of vaccines at different VVM levels which vaccine should be used first based on the VVM readings. This test tested both the ability to read a VVM and decide based on the reading. Although lower than the 85% who could correctly read a VVM in the Ghana study, it was higher than the only 19% who stated the implication of the inner colour matching the outer colour in the same study (Osei et al., Citation2019).

Association between training and practice

In this current study, experience was found a non-contributory factor for both knowledge and practice. This was contrary to a study in India that concluded that longevity in service referred to as experience had an effect on practice (Osei et al., Citation2019), Also cadre referred to as qualification in this current study was non-impactful on practice contrary to the assertions of the study in India (Osei et al., Citation2019).

Association between knowledge and practice

In this current study, the average knowledge of all the participants was 62.42%, while the average practice score for all the participants was 65.33%. This was different from the paradox observed in the Nigerian study where practice scores lagged behind Knowledge at 77.2 and 83.9%, respectively (Adebimpe & Adeoye, Citation2021).

Study limitations

The study being a cross-sectional one cannot conclude with certainty that exposure to the training and the knowledge and practice outcomes are associated, as depicted in the study population. To arrive at this conclusion, a controlled study would yield more reliable results.

The biggest proportion of the study participants were exposed to at least 2 trainings the outcomes were therefore confounded when each training was being analysed in isolation.

The study focused on the lead personnel for vaccine cold chain management at the facilities under the assumption that their knowledge and practices are the closest proxies to the facility’s level of practice. This however may not entirely be the case. As such a follow-up study involving more stakeholders needs to be taken to cover the gaps.

Conclusions

The study found that the level of practice at 65.33% is below the 80% WHO targets set in the EVM although all the six types of training investigated had been utilised across the board among the facility vaccine cold chain handlers. At least each cold chain handler had been exposed to a minimum of two combined different types of training.

The knowledge of the vaccine cold chain handlers is above average at 62.42% but below the WHO target of 80% set out in the EVM. The study found that the training has an incremental effect on knowledge of handlers and their practices at the facility level. It also found that the deployment of multiple trainings yielded slight but increasingly better results as demonstrated in the results section above. The study further found that the training has an advantageous effect on the practices of handlers and their practices at the facility level. It also found that the deployment of multiple trainings yielded slight but increasingly better results as demonstrated in the results section above.

The study also found that as would be expected, knowledge impacted practice therefore highlighting that efforts made to increase knowledge also impacted practice in the long run.

Declarations

Ethics approval and consent to participate

Ethical approval was obtained from the Lira University Research Ethics Committee under approval number LUREC-2023-29. Permission to conduct study activities was obtained from the District Health Office of the selected districts. Written informed consent was obtained from all study participants. Appropriate coding for all potential identifiers of facilities and participants was done with unique codes only known to the research team. The data collected for the study was and will remain accessible by the research team and has been reported in aggregated form with minimal trace-back potential.

Authors’ contributions

AD designed the study, collected data and drafted the manuscript. MFM and OJ participated in the designing of the study, data analysis, and critically reviewed the manuscript. IH and TR reviewed and offered guidance on the manuscript right from the draft stage and made critical reviews to arrive at this final version. All authors have approved the final version for publication.

Availability of data and materials

The datasets used and/or analysed during the current study are available upon reasonable request.

Acknowledgements

The authors of this paper acknowledge the funding of the Masters of Health Supply Chain Management by the German Federal Ministry for Economic Cooperation and Development (BMZ) through the KfW Development Bank and the East African Community Regional Center of Excellence for Vaccines, Immunisation, and Health Supply Chain Management. In addition, this research would not have been possible without the assistance of the very supportive staff of the College of Medicine and Health Sciences, University of Rwanda. Special acknowledgement to the classmates for the shared experiences that contributed to the conception, conduction and completion of this research.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Correction Statement

This article has been corrected with minor changes. These changes do not impact the academic content of the article.

Additional information

Funding

The research from which this manuscript was developed did not receive funding either for data collection or manuscript publication. However, it is prepared from a master’s dissertation that was conducted in fulfillment of the requirement for the award of a master’s degree in Health Supply Chain Management, in EAC RCE – VIHSCM. This master’s degree was funded by the German Federal Ministry for Economic Cooperation and Development (BMZ) through the KfW Development Bank.

Notes on contributors

Aguma Daniel

Dr. Aguma Daniel is a pharmaceutical supply chain specialist who has an MSc in Health supply chain management from the EAC Regional Centre of Excellence for Vaccines, Immunization and Health Supply Chain Management, College of Medicine and Health Sciences, University of Rwanda, Kigali, Rwanda. Aguma is passionate about his patients and has focused his energies on improving supply chain efficiency such that his patients get the right medicines in the right quality, quantity and cost. He is dedicated and committed to make his contribution to the health sector in Uganda, the EAC, Africa and the world at large. Currently working for the ministry of health in the northern region of Lango Aguma heads the supply chain technical working group of the Lango region and has made strides in synergizing the individual efforts of the vast supply chain stake holders in the region seen in the raising trend of most supply chain indicators in the region.

Joseph Oloro

Joseph Oloro is a Lecturer of Pharmacology at the Department of Pharmacology & Therapeutics, Faculty of Medicine, Mbarara University of Science and Technology, Uganda. His major area of interest is Toxicology. He holds a Diploma in Clinical Medicine and Community Health, BSc. In Pharmacology, MSc in Pharmacology and Currently pursuing a Ph.D. in Toxicology. He previously worked as a Clinical officer at Pope John Hospital Aber, Northern Uganda, and Kampala International University Teaching Hospital, Uganda. He has taught at the university level from 2006 to date at different levels. At Mbarara University, he is the Chairman of the Faculty of Medicine Curriculum Review Committee and a Member of the Faculty of Medicine Postgraduate Committee. He had training in Basic Laboratory Animal Science at the University of Utrecht in the Netherlands with a Species-Specific specialization in Rodents in 2019. He is also a member of several grants at the local level.

Innocent Hahirwa

Dr Innocent Hahirwa is a Senior Lecturer and a Senior Consultant Pharmacist with a Ph.D in Biomedical and Pharmaceutical Sciences/Clinical Toxicology from University of Liege (Belgium). Technically, he has an extensive experience in teaching, research, administration and clinics. In addition to the formal academic training, Dr Hahirwa has been trained in different areas of Pharmacy profession including, Pharmacy profession regulation, Drug registration, Pharmacovigilance, Pharmaceutical products (including hazardous products) handling and Clinical Trials. Dr HAHIRWA is teaching different courses related to Toxicology, Pharmacology, Pharmacovigilance and Clinical trials at both Undergraduate and Postgraduate levels, and has supervised a number of research works for UG and PG students. He is also in charge of clinical pharmacy care and training in Kigali University Teaching Hospital. His research areas of interest includes mainly Toxicology, Pharmacology, Clinical Biology as well Pharmacy practice and regulation. Having occupied different managerial positions including being the Chairperson and a member of the National Pharmacy Council Board, Head of Pharmacy Department and Deputy Dean of the School of Medicine and Pharmacy at the University of Rwanda for several years, Dr HAHIRWA has strong leadership and management skills.

Theogene Rizinde

Theogene Rizinde has more than 15 years of doing research and teaching in higher learning institutions. Theogene is a Lecturer at University of Rwanda (UR). He has been a head of Department of Applied Statistics at University of Rwanda in the School of Economics, College of Business and Economics for 4 years since 2018. He is author of more than 6 journal articles, 3 book chapters and he is an international consultant. Theogene holds master’s degree of Mathematical modelling and Scientific computing, and He is finalizing his PhD in Data Science applied in Biostatistics.

Marie Francoise Mukanyangezi

Dr. Marie Francoise Mukanyangezi is a PhD holder in Medical Science from Gothenburg University, Sweden. Her PhD research focused on translational study of the immunity responses of the uterine cervix in case of inflammation and infection. Following her graduation, Dr Marie Francoise is determined to contribute to the Rwandan Government in his efforts to the fight against cervical cancer. Currently teaching Research Methodology at both UG and PG Pharmacy programs, at University of Rwanda. Dr. Marie Francoise aims not only promoting medical research in the field of women cancers but also promoting health professional’s education.

References

  • Adebimpe, W. O., & Adeoye, O. A. (2021). Knowledge and practice of vaccination logistics management among primary health care workers in Nigeria. Human Vaccines & Immunotherapeutics, 17(5), 1490–1495. https://doi.org/10.1080/21645515.2020.1827609
  • Brown, A. N., Prosser, W., & Zwinkels, D. (2017). Who is preparing the next generation of immunization supply chain professionals? Vaccine, 35(17), 2229–2232. https://doi.org/10.1016/j.vaccine.2016.12.076
  • Eriksson, P., Gessner, B. D., Jaillard, P., Morgan, C., & Le Gargasson, J. B. (2017). Vaccine vial monitor availability and use in low- and middle-income countries: A systematic review. Vaccine, 35(17), 2155–2161. https://doi.org/10.1016/j.vaccine.2016.11.102
  • Feyisa, D. (2021). Cold chain maintenance and vaccine stock management practices at public health centers providing child immunization services in Jimma zone, Oromia regional state, Ethiopia : Multi-centered, mixed method approach. Pediatric Health, Medicine and Therapeutics, 12, 359–372.
  • Hanson, C. M., George, A. M., Sawadogo, A., & Schreiber, B. (2017). Is freezing in the vaccine cold chain an ongoing issue? A literature review. Vaccine, 35(17), 2127–2133. https://doi.org/10.1016/j.vaccine.2016.09.070
  • Karlsson, S. (2012). Routine child vaccination in Uganda: Designing a vaccination service delivery model.
  • Kartoglu, U., & Milstien, J. (2014). Tools and approaches to ensure quality of vaccines throughout the cold chain. Expert Review of Vaccines, 13(7), 843–854. https://doi.org/10.1586/14760584.2014.923761
  • Kasonde, M., & Steele, P. (2017). The people factor: An analysis of the human resources landscape for immunization supply chain management. Vaccine, 35(17), 2134–2140. https://doi.org/10.1016/j.vaccine.2017.01.084
  • Lloyd, J., & Cheyne, J. (2017). The origins of the vaccine cold chain and a glimpse of the future. Vaccine, 35(17), 2115–2120. https://doi.org/10.1016/j.vaccine.2016.11.097
  • Lutukai, M., Bunde, E. A., Hatch, B., Mohamed, Z., Yavari, S., Some, E., Chweya, A., Kania, C., Ross, J. C., Keddem, C., & Chandani, Y. (2019). Using data to keep vaccines cold in Kenya: Remote temperature monitoring with data review teams for vaccine management. Global Health: Science and Practice, 7(4), 585–597. https://doi.org/10.9745/GHSP-D-19-00157
  • Luzze, H., Badiane, O., Mamadou Ndiaye, E. H., Ndiaye, A. S., Atuhaire, B., Atuhebwe, P., Guinot, P., Sosne, E. F., & Gueye, A. (2017). Understanding the policy environment for immunization supply chains: Lessons learned from landscape analyses in Uganda and Senegal. Vaccine, 35(17), 2141–2147. https://doi.org/10.1016/j.vaccine.2016.10.089
  • Masresha, B., Dochez, C., Bumgarner, A., Pienkowski, N., & Mihigo, R. (2020). The world health organization African regional training course for mid-level immunization managers: Lessons and future directions. The Pan African Medical Journal, 37(194), 1–6.
  • Ministry of Health Uganda. (2018, November). National Health Facility Master List 2018. Minist Heal Uganda, 1–164. http://library.health.go.ug/sites/default/files/resources/National Health Facility Master List 2018_0.pdf
  • Mohammed, S. A., Workneh, B. D., & Kahissay, M. H. (2021). Knowledge, attitude and practice of vaccinators and vaccine handlers on vaccine cold chain management in public health facilities, Ethiopia: Cross-sectional study. PLoS One, 16(2 February), 1–12.
  • Orenstein, W. A., Bernier, R. H., Dondero, T. J., Hinman, A. R., Marks, J. S., Bart, K. J., & Sirotkin, B. (1985). Field evaluation of vaccine efficacy. Bulletin of the World Health Organization, 63(6), 1055–1068.
  • Osei, E., Ibrahim, M., & Kofi Amenuvegbe, G. (2019). Effective vaccine management: The case of a rural district in Ghana. Advances in Preventive Medicine, 2019, 1–8. https://doi.org/10.1155/2019/5287287
  • Ozawa, S., Clark, S., Portnoy, A., Grewal, S., Brenzel, L., & Walker, D. G. (2016). Return on investment from childhood immunization in low- and middle-income countries, 2011-20. Health Affairs, 35(2), 199–207. https://doi.org/10.1377/hlthaff.2015.1086
  • Riedmann, E. M. (2010). Report: State of the world’s vaccines and immunization. Human Vaccines, 6(2), 157–163.
  • Sillanp, I. (2015). Supply chain performance measurement in the manufacturing industry A single case study research to develop a supply chain performance measurement framework.
  • Steele, P. (2015, October). Assessment of the human resources landscape for immunization supply chain management.
  • Sule, V. (2022). Vaccine management knowledge and practice of health workers in Yemen. Texila International Journal of Public Health, 10(4), 111–125. https://doi.org/10.21522/TIJPH.2013.10.04.Art011.
  • The Maternal and Child survival program (MCSP). (2016, June). Strengthening the routine immunization system through a reaching every child–quality improvement approach in Uganda a how-to guide. www.mcsprogram.org
  • USAID | DELIVER PROJECT TO 1. (2009, January). Logistics System Assessment Tool (LSAT). Arlington, Va USAID | Deliv Proj Task Order 1\r.
  • Woldemichael, B., Bekele, D., & Esmael, A. (2018). Cold chain status and knowledge of vaccine providers at primary health care of units bale zone, southeast Ethiopia: Cross-sectional study. Immunome Research, 14(1), https://doi.org/10.4172/1745-7580.1000152
  • Yakum, M. N., Ateudjieu, J., Pélagie, F. R., Walter, E. A., & Watcho, P. (2015). Factors associated with the exposure of vaccines to adverse temperature conditions: The case of North West region, Cameroon Health Services Research. BMC Research Notes, 8(1), 1–7. https://doi.org/10.1186/1756-0500-8-1
  • Zaffran, M., Vandelaer, J., Kristensen, D., Melgaard, B., Yadav, P., Antwi-Agyei, K. O., & Lasher, H. (2013). The imperative for stronger vaccine supply and logistics systems. Vaccine, 31(Suppl 2), B73–B80. https://doi.org/10.1016/j.vaccine.2012.11.036