Adaptation and validation of the Turkish version of the vaccine hesitancy 5 point Likert Scale

ABSTRACT

The decrease in vaccine acceptance has been recognized as an emerging public health problem and there is therefore a need for reliable and validated tools that identify vaccine hesitancy. The objective of this study was to adapt and validate the Turkish version of the Vaccine Hesitancy 5-point Likert Scale which was originally developed by the WHO Strategic Advisory Group of Experts on Immunization. The study was carried out in a Family Health Center (FHC) in Istanbul over the period June 1-November 30, 2020. The participants were parents who had applied to the FHC for well-child visits and had a child ≤18 months of age. After the process of translation and back-translation, the Turkish version was pilot-tested, and its test-retest reliability was evaluated among 40 parents at a two-week interval. The validation was carried out with 306 parents through exploratory factor analysis. There was no statistical difference between the test-retest scores (p = .17). The intraclass correlation coefficient was 0.98 (p < .001). Cronbach’s alpha coefficient was 0.81. Factor analysis yielded two subscales that were named “confidence’ and ‘risk perception” and explained 63% of total variance. Our results suggest that the Turkish version of the Vaccine Hesitancy 5-point Likert Scale is a reliable and valid instrument.

KEYWORDS:

• Vaccine hesitancy
• reliability
• validity
• scale
• vaccine
• immunization

Introduction

According to the World Health Organization (WHO), 2–3 million deaths and 75.000 disabilities are prevented annually by global vaccination.¹ While the benefits of vaccination are well documented, vaccine hesitancy has emerged as a major public health problem worldwide over the last two decades.² Hence in 2019, WHO declared vaccine hesitation as the delay in acceptance or refusal of vaccination despite the availability of vaccination services,³ and identified it as one of the top ten global health threats.⁴ Vaccine hesitancy has been recognized as an emerging public health problem in Turkey as well. The number of parents who had refused vaccination for their children in 2011 was only 183. However, the number of refusals increased to 5400 in 2015, and to nearly 23,000 in 2018.⁵ Although there are no studies at the national level, some local studies indicate that vaccine hesitancy rates range between 16%-42%. It is yet hard to compare these figures due to the lack of a validated tool that identifies vaccine hesitancy in Turkey.^6–9 The WHO Strategic Advisory Group of Experts (SAGE) on Immunization recognized vaccine hesitancy as a global challenge and established a Working Group.¹⁰ SAGE has developed several different types of tools to measure parental attitudes and vaccine hesitation. One of these is the Vaccine Hesitancy 5-point Likert Scale which is a diagnostic measure used to identify and compare hesitancy across different global settings.² Questions on other scales are closed- or open-ended in nature. The questions on the Scales were adapted from the Parent Attitudes About Childhood Vaccines (PACV) survey previously developed by Opel et al.¹¹ The Vaccine Hesitancy 5-point Likert Scale includes 10-items, with responses ranging from “strongly disagree” to “strongly agree.”

The psychometric properties of the Vaccine Hesitancy 5-point Likert Scale were evaluated and the instrument was shown to be reliable and valid in several low to high-income settings in Canada,¹² the United Kingdom,¹³ Guatemala,¹⁴ China,¹⁵ and Ethiopia.¹⁶ The Vaccine Hesitancy 5-point Likert Scale enables us to compare parental hesitancy levels across regions and evaluate changes over time. It can also be beneficial in identifying the sociodemographic characteristics of hesitant groups for the purpose of addressing their needs.¹³

To our knowledge, the Vaccine Hesitancy 5-point Likert Scale had not been adapted to Turkish. Hence the objective of this study was to adapt and validate the Turkish version of the Vaccine Hesitancy 5-point Likert Scale originally developed by the SAGE Working Group.

Materials and methods

This is a validation study evaluating the psychometric properties of the Turkish version of the Vaccine Hesitancy 5-point Likert Scale.

Study protocol

The Vaccine Hesitancy 5-point Likert Scale was translated into Turkish by two of the authors. The translators were fluent in both languages, familiar with the cultures under study, had some knowledge of the test construction, and of the construct being measured.¹⁷ Discrepancies between the English version and the first Turkish draft were evaluated and resolved by an expert panel that included the authors and five other health professionals. Back-translation was carried out by a bilingual translator. The authors compared the back-translated and the original English versions, resolved the inconsistencies, and developed the final version of the scale. The final version was applied and pilot-tested among ten parents with a child of <18 months of age. Test-retest reliability was evaluated among 40 parents at a two-week interval. Validation was assessed among 306 parents through exploratory factor analysis.

Setting and participants

The study was carried out in a Family Health Center (FHC) in Istanbul over the period June 1-November 30, 2020. Tuzla Training FHC was selected as the study setting since it serves as the research and training site of Marmara University School of Medicine for both undergraduate and postgraduate students.

The participants were parents who had applied to the FHC for well-child visits and had a child ≤18 months of age. In Turkey, the Expanded Program on Immunization includes 13 antigens for diphtheria, pertussis, tetanus, measles, rubella, mumps, tuberculosis, polio, hepatitis A, hepatitis B, chickenpox, Haemophilus influenzae type B and pneumococcal diseases; vaccination services are provided free of charge.

Study participants were selected by convenient sampling from the parents of ≤18-month-olds since the first doses of all vaccines were expected to be completed within this time interval in line with the Turkish Immunization Program. Only native Turkish speakers were included in the study.

To perform factor analysis, the sample size is recommended to be at least 5–10 times the number of the items used in the scale.¹⁸ Also, a rough guideline for grading adequate sample size suggests that 300 participants is a “good” number for validation studies.^19,20 Since the number of items in the scale was 10, we aimed to recruit a minimum of 300 participants. Participants were selected through convenient sampling as well. The study protocol is summarized in Figure 1.

Figure 1. Study protocol.

Data collection

Data were collected through a face-to-face questionnaire and the Turkish version of the Vaccine Hesitancy 5-point Likert Scale. The questionnaire included sociodemographic characteristics, the history of delay and refusal of previous childhood vaccines and the father’s involvement in childcare. Items 1–4 and 6–8 in the scale indicate positive, while items 5, 9, and 10 indicate negative attitudes toward vaccination. Higher scores reveal low vaccine hesitation, and the maximum score is 50.

Only the participants who had fully responded to all the scale items were included in the study.

Analysis

Descriptive statistics are presented with numbers, percentages, mean ± SD and median (25^th-75^th percentiles). Normality of the data was evaluated by means of histograms and the Skewness and Kurtosis test. Continuous variables for two and more groups were compared with the Mann-Whitney U and Kruskal-Wallis tests, respectively. The Chi-square and Fisher’s exact tests were used in comparing categorical variables.

Test-retest reliability was evaluated among 40 parents at a two-week interval with the Wilcoxon test, Spearman’s correlation and the Intraclass correlation coefficient. Cronbach’s alpha was used to evaluate internal consistency. Exploratory factor analysis was performed after the assessment of Bartlett’s Test and the Kaiser-Meyer-Olkin (KMO) Test. Varimax rotation was used. The subscales were determined and compared with variables and parents’ characteristics. A p value less than 0.05 was used as the level of statistical significance. The SPSS version 20.0 was used for statistical analysis. (IBM SPSS Statistics 20.0 – August 2011 – SPSS Inc., Chicago Ill).

Ethical approval

The study was approved by the Ethics Committee of Marmara University School of Medicine (Approval No:09.2020.247). The informed written consent of the participants was received.

Results

Reliability of the scale

Forty parents (65% mothers; 35% fathers) at a mean age of 31.2 ± 4.8 years (min: 24, max: 49) were included in the reliability analysis. Cronbach’s alpha coefficient of the scale was 0.73. There was no statistical difference between the test-retest scores of the scale applied two weeks apart [median (25^{th −7}5^th percentiles): 45.0 (40.25–47.0) vs. 45.0 (41.0–47.0); p = .17]. Spearman’s rho coefficient was 0.94 (p < .001). The intraclass correlation coefficient was 0.98 (p < .001).

Validity of the scale

Among the 316 parents invited, 306 (response rate: 96.8%) participated in the validation study. The participants had a mean age of 31.6 ± 4.88 years (min: 20, max: 49). Sociodemographic characteristics of the participants are presented in Table 1.

Table 1. Sociodemographic characteristics of the parents

		n	%
Relationship to child	Mother	186	60.8
	Father	120	39.2
Mother’s educational status	No education	3	1.0
	Primary school	33	10.8
	Middle school	48	15.7
	High school	60	19.6
	University and above	162	52.9
Mother’s employment status	Working	90	29.4
	Not employed	216	70.6
Father’s educational status	No education	0	.0
	Primary school	18	5.9
	Middle school	42	13.7
	High school	84	27.5
	University and above	162	52.9
Father’s employment status	Working	285	93.1
	Not employed	21	6.9
Father’s participation in childcare	Participates	276	90.2
	Not participates	30	9.8
Family monthly income level^a,b	<₺2200	39	12.7
	₺2200-4999	117	38.2
	₺5000-7499	87	28.4
	₺7500-9999	45	14.7
	>₺10000	18	5.9
Living in their own home	Yes	99	32.4
	No	207	67.6
Own car presence	Yes	204	66.7
	No	102	33.3
Residence (longest lived)	City	270	88.2
	Town	12	3.9
	Village	24	7.8
Family status	Nuclear	270	88.2
	Traditional	36	11.8

^aSince the minimum wage was ₺2200 while preparing the survey, the lower limit was determined as ₺2200.

^b₺: Turkish Lira

Cronbach’s alpha coefficient for the scale was 0.81. The Kaiser-Meyer-Olkin coefficient of the scale was 0.894 and Bartlett’s Test was statistically significant (p < .001). In the exploratory factor analysis, two subscales explaining 63% of total variance were revealed; these were named “confidence’ and ‘risk perception” (Table 2). The correlation coefficient between the scores of the sub-scales was 0.34 (p < .001).

Table 2. Item loadings in factor analysis

Vaccine hesitancy scale items	EFA loadings
	Confidence	Risk Perception
Getting vaccines is a good way to protect my child/children from disease	.911
All childhood vaccines offered by the government program in my community are beneficial	.884
Childhood vaccines are important for my child’s health	.881
Childhood vaccines are effective	.829
Generally, I do what my doctor or health care provider recommends about vaccines for my child/children	.804
Having my child vaccinated is important for the health of others in my community	.672
The information I receive about vaccines from the vaccine program is reliable and trustworthy	.647
New vaccines carry more risks than older vaccines		.780
My child/children does or do not need vaccines for diseases that are not common anymore		.762
I am concerned about serious adverse effects of vaccines		.576
	Eigenvalue	Explained variance	Total variance (%)
Confidence	5.004	50.044	50.044
Risk Perception	1.331	13.312	63.356

EFA: Exploratory factor analysis. Extraction Method: Principal Component Analysis. Rotation Method: Varimax with Kaiser Normalization

The median score of the scale was 43.5 (25^{th −7}5^th percentiles: 40.0–47.0; min-max 22–50). The total scale score was lower among mothers than the fathers (p = .021). Participants with higher income had higher total scale scores than those who had lower income (p = .009) (Table 3). As the ages of the mothers and the fathers increased, the total scores of the scale decreased (r = −0.24; p < .001 and r = −0.12; p < .05, respectively).

Table 3. Comparison of the scale total score with some variables

		n	Median	25th-75th Percentile	p
Relationship to child	Mother	186	43.0	39.0–46.0	0.021^a
	Father	120	44.0	40.0–47.0
Mother’s educational status	Primary school and below	36	43.5	40.5–47.0	0.275^b
	Middle school	48	44.0	40.0–46.0
	High school	60	44.5	41.0–47.0
	University and above	162	43.5	40.0–47.0
Mother’s employment status	Working	90	44.5	40.0–47.0	0.106^a
	Not employed	216	43.0	39.5–47.0
Father’s educational status	Primary school and below	18	44.5	43.0–45.0	0.717^b
	Middle school	42	46.0	40.0–47.0
	High school	84	43.0	40.0–47.0
	University and above	162	43.0	39.0–47.0
Father’s participation in childcare	Participates	276	43.5	40.0–47.0	0.141^a
	Not participates	30	43.0	38.0–46.0
Family monthly income level^c	<₺2200	39	43.0	38.0–47.0	0.009^b
	₺2200-4999	117	43.0	40.0–46.0
	₺5000-7499	87	43.0	40.0–47.0
	₺7500-10000	45	45.0	40.0–47.0
	>₺10000	18	47.0	42.0–50.0
Living in their own home	Yes	99	45.0	39.0–47.0	0.611^a
	No	207	43.0	40.0–47.0
Refusing their child’s vaccinations	Yes	6	26.0	23.0–27.0	<0.001^a
	No	300	44.0	40.0–47.0
Delay in acceptance some of their child’s vaccines or not allow them to be administer	Yes	33	40.0	37.0–46.0	0.002^a
	No	273	44.0	40.0–47.0
Presence of parents in the environment who refuse their child’s vaccinations	Yes	72	41.0	40.0–46.0	0.071^b
	No	181	45.0	40.0–47.0
	Do not know	53	43.0	38.0–47.0

^aMann-Whitney U test; statistical significance p < 0.05

^bKruskal-Wallis test; statistical significance p < 0.05

^c₺: Turkish Lira

The “confidence” subscale score was lower among university graduate mothers compared to s mothers who had a primary school education or less; this subscale score was also lower among those who had previously exhibited childhood vaccine refusal and hesitancy for their children as compared to those who had not (Table 4). As the mother’s age increased, the “confidence” subscale score decreased (r = −0.17; p = .003).

Table 4. Comparison of the sub-scale scores with some variables

		Confidence				Risk perception
		n	Median	25th-75th percentile	p	Median	25th-75th percentile	p
Relationship to child	Mother	186	34.0	29.0–35.0	0.190^a	11.0	9.0–12.0	0.003^a
	Father	120	34.0	29.0–35.0		12.0	10.0–13.0
Mother’s educational status	Primary school and below	36	35.0	34.0–35.0	0.019^b	10.0	6.0–12.0	0.026^b
	Middle school	48	33.5	28.5–34.5		11.0	8.5–12.0
	High school	60	34.0	28.0–35.0		12.0	9.5–13.0
	University and above	162	33.5	29.0–35.0		11.0	10.0–13.0
Mother’s employment status	Working	90	34.0	29.0–35.0	0.981^a	11.5	10.0–13.0	0.007^a
	Not employed	216	34.0	28.0–35.0		11.0	9.0–12.5
Father’s educational status	Primary school and below	18	34.5	33.0–35.0	0.111^b	9.5	9.0–11.0	0.129^b
	Middle school	42	34.5	29.0–35.0		12.0	11.0–12.0
	High school	84	34.0	31.5–35.0		10.5	9.0–13.0
	University and above	162	34.0	28.0–35.0		11.0	9.0–13.0
Father’s participation in childcare	Participates	276	34.0	29.0–35.0	0.331^a	11.0	9.0–13.0	0.006^a
	Not participates	30	33.5	31.0–34.0		10.0	6.0–12.0
Family monthly income level^c	<₺2200	39	34.0	32.0–35.0	0.529^b	10.0	8.0–12.0	0.001^b
	₺2200-4999	117	34.0	28.0–35.0		11.0	9.0–12.0
	₺5000-7499	87	34.0	28.0–35.0		12.0	9.0–13.0
	₺7500-10000	45	34.0	31.0–35.0		11.0	10.0–12.0
	>₺10000	18	34.0	31.0–35.0		13.0	11.0–15.0
Living in their own home	Yes	99	35.0	29.0–35.0	0.067^a	11.0	8.0–12.0	0.024^a
	No	207	34.0	29.0–35.0		11.0	10.0–13.0
Refusing their child’s vaccinations	Yes	6	21.0	16.0–22.0	<0.001^a	5.0	5.0–7.0	<0.001^a
	No	300	34.0	29.0–35.0		11.0	9.0–13.0
Delay in acceptance some of their child’s vaccines or not allow them to be administer	Yes	33	32.0	27.0–35.0	0.012^a	10.0	6.0–12.0	0.034^a
	No	273	34.0	29.0–35.0		11.0	9.0–13.0
Presence of parents in the environment who refuse their child’s vaccinations	Yes	72	32.0	28.0–35.0	0.013^b	12.0	8.0–13.0	0.089^b
	No	181	34.0	31.0–35.0		11.0	10.0–12.0
	Do not know	53	34.0	28.0–35.0		10.0	8.0–12.0

^aMann-Whitney U test; statistical significance p < 0.05

^bKruskal Wallis test; statistical significance p < 0.05

^c₺: Turkish Lira

The risk perception subscale score was lower among the mothers than the fathers, also among unemployed mothers compared to working mothers, and in families where fathers did not participate in childcare compared to those families in which fathers did participate. The risk perception subscale score was higher in mothers who had graduated from high school compared to the mothers who had a middle school, primary school education or less. Also, the risk perception subscale score was higher in parents with a monthly income higher than TRY 10,000 compared to parents who had a lower income (Table 4). As the ages of the mothers and fathers increased, the risk perception subscale scores decreased (r = −0.21; p < .001 and r = −0.12; p < .05, respectively).

Discussion

In this study that researched the Turkish reliability and validity of the Vaccine Hesitancy 5-point Likert Scale, which had been originally developed by the WHO Strategic Advisory Group of Experts on Immunization, a good test-retest reliability with a high correlation coefficient (rho: 0.97) and a two-factor structure as in the original version were found in the factor analyses.

Reliability

The reliability of an instrument is represented by using the Cronbach alpha coefficient. A high correlation coefficient indicates that the scale is reliable. Cronbach’s alpha coefficient values above 0.70 are considered acceptable; values above 0.80 are preferable.²¹ Therefore, we can conclude that the internal consistency of the Turkish version of the Vaccine Hesitancy 5-point Likert Scale was good, revealing a Cronbach’s alpha of 0.81 when the scale was applied to 306 parents. A study conducted by Ren et al. in China determined a Cronbach’s alpha coefficient of 0.73.¹⁵ Another study, this time carried out in Canada, revealed two subscales emerging from the factor analysis with Cronbach’s alpha coefficients of 0.64 and 0.92.¹² The test-retest correlation coefficient (rho: 0.97) was also high, indicating a good level of repeatability.^22,23

Validity

Original factor structures in scales might not be fully sufficient to explain the theory tested in a new population and cultural setting. Exploratory factor analysis is therefore used instead of confirmatory factor analysis in order to enable the discovery of factor structures in a new cultural setting.^24–26 In our study, the exploratory factor analysis revealed a structure of two factors, which were named ‘confidence” and “risk perception,” explaining 63% of total variance. In scales with more than one subscale, explaining more than 50% of variance is accepted as adequate criteria for factor analysis.²⁷

The two subscales were consistent with the theoretical framework of “healthism/risk culture” and “level of confidence in health authorities and mainstream medicine” proposed by Peretti-Watel et al.²⁸Also, a similar two-factor structure was revealed in the studies conducted by Shapiro et al., Domek et al. and Luyten et al. While the item “My child/children does/do not need vaccines for diseases that are not common anymore” was not clearly loaded onto any factor in the mentioned studies, it did load onto the “risk perception” subscale in our study. It is worthy of note to find that the results of our study are generally consistent with studies conducted in cultures with diverse economic and cultural characteristics.

In the studies of Shapiro and Luyten, 7 items with positive expressions were scored between 1 = strongly agree to 5 = strongly disagree, therefore, unlike our study, high scores indicated vaccination hesitation. Although the scores differed, the common finding in these studies and the present study was that parents were not very hesitant in general but were mostly more concerned about the risks associated with vaccines rather than displaying a lack of confidence in vaccination programs and health authorities.^12,13

Studies have reported significant differences between the sociodemographic characteristics of parents and vaccine hesitancy. In the present study, fathers were less concerned than mothers about the risks of vaccinations, but there was no difference in terms of the confidence scores. Similarly, Shapiro et al. also showed that fathers were less concerned than mothers, but further reported that fathers had a greater lack of confidence compared to mothers.¹² Also, in the study conducted by Luyten et al., fathers were less confident than mothers but there were no differences as to risk perception.¹³ These variations might be related to the influence of different cultures on fathers’ perceptions, attitudes, and roles in child-raising. Mothers with a high school level of education were less concerned about the risks of vaccines than those less educated, while no relationship was observed between the educational level of both mothers and fathers and concerns about confidence. Shapiro et al. reported that as the educational level of the parents increased, concerns about confidence decreased.¹² Ren et al. reported that concerns about the risks of vaccination decreased as education levels increased but confidence concerns did not change.¹⁵ In some other studies that investigated vaccine hesitancy but which did not use the Vaccine Hesitancy 5-point Likert Scale, higher levels of education were found to be associated with high vaccine hesitancy.^29,30 These differences may be due to the fact that parents with a higher level of education have easy access to many sources of information about vaccines, including true as well as false information, compared to lesser educated individuals. We can therefore conclude that the Turkish Vaccine Hesitancy 5-point Likert Scale is a selective scale due to the difference in scale scores in the study subgroups.

Another interesting finding of this study was the relationship between the father’s involvement in childcare and risk perceptions regarding vaccines. The fathers who participated in childcare demonstrated a lower level of risk perception. Further studies are needed to confirm and explain why fathers have less hesitation compared to mothers, and also to understand why fathers participating in childcare have higher vaccine acceptance. It was observed that this issue was not investigated in other studies using the Vaccine Hesitancy 5-point Likert Scale. One of the main findings of our study is that vaccine hesitation in fathers was less than in mothers. These two findings lead to the conclusion that fathers especially should be investigated in terms of vaccination hesitation using the Vaccine Hesitancy 5-point Likert Scale, and those who have no vaccine hesitation should be encouraged to participate in childcare.

During the study period, a short and a long version of a Turkish vaccine hesitancy scale had been developed by Kılınçarslan et al. The long version has 21 and the short version has 12 Likert-type questions similar to the items on our scale. However, to the best of our knowledge, this scale has not been evaluated in a large population up until now.³¹

One of the secondary findings of our study was related to the rate of vaccine refusal among the study population; 1.96% of the parents had refused all vaccines for their children. This rate is consistent with the findings of the Turkish Demographic and Health Survey (TDHS), which has a national representation. TDHS findings indicate that 2% of 12–23 month-old children had never been vaccinated.³² Although vaccine hesitancy is still low in Turkey compared to other countries, we should be aware that refusal, delay and skipping doses might increase and threaten herd immunity in the foreseeable future.^33,34

Strengths and limitations:

One of the strengths of this study was that the study population consisted of parents with a child at the age of ≤18 months, which was a factor that minimized recall bias. While the sample size was adequate enough to carry out factor analysis,¹⁹ we should be aware that our study was limited to only one FHC. Turkey is a large country encompassing a diverse population in terms of sociodemographic characteristics. Therefore, further studies with larger samples that also encompass different subpopulations in Turkey would be beneficial. We should also note that Likert-type scales are unidimensional and, in our case, had only five options which might have limited the tool from exploring the real attitudes of the participants. Also, most of the participants might have been reluctant to choose the most extreme options of “strongly agree and/or disagree” as has been seen to happen with Likert- type scales. Moreover, attitude scales are limited in that we cannot know if the attitude measured can be fully translated into behavior.

In conclusion, the WHO recognized vaccine hesitancy as a global challenge and called for the tools that were developed to define this problem to be adapted to different global settings. Following this call, we translated the Vaccine Hesitancy 5-point Likert Scale into Turkish and evaluated reliability and validity. Our results suggest that the Turkish version of the Vaccine Hesitancy 5-point Likert Scale is a reliable and valid instrument and can be used with Turkish parents. But we should emphasize that the Vaccine Hesitancy 5-point Likert-type scale was developed to explore general vaccine hesitancy and does not provide information on specific antigen refusals. Further studies with different populations would be help to challenge with vaccine hesitancy.

Disclosure of potential conflicts of interest

No potential conflicts of interest were disclosed.

Acknowledgments

The authors would like to thank to Emine Mizyal for her contributions to the English editing.

Additional information

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or non-profit sectors.