COVID-19 vaccine hesitancy among Mexican outpatients with rheumatic diseases

ABSTRACT

Vaccine hesitancy (VH) has emerged as a recognized threaten to contain the COVID-19 pandemic. Historically, low vaccine acceptance rates had been described among patients with rheumatic diseases (RMDs). The study objective was to determine COVID-19 VH among Mexican outpatients with RMDs and validate the COVID-19 VH questionnaire. This cross-sectional study was developed in three steps. Step 1 consisted of translation/cultural adaptation of the Oxford-COVID-19-VH questionnaire. Step 2 consisted of pilot testing and questionnaire feasibility, content, construct and criterion validity, reliability (internal consistency and temporal stability) and questionnaire sensitivity to change. Step 3 consisted of VH phenomenon quantification in patients from two metropolitan tertiary-care-level centers. Step 1 followed ISPOR-task-force recommendations. Patients who participated in step 2 (n = 50 for pilot testing/feasibility and n = 208 for questionnaire validation [91 in test–retest and 70 in questionnaire-sensitivity to change]) and step 3 (n = 600) were representative outpatients with RMDs. The seven-item COVID-19 VH questionnaire was found feasible, valid (experts’ agreement ≥80%; a 1-factor structure accounted for 60.73% of the total variance; rho = 0.156, p = .025 between COVID-19 VH questionnaire and score from the Spanish version of the Vaccine Hesitancy Scale; and lower questionnaire scores in patients who reported 5 years-previous influenza vaccination), reliable (Cronbach’s ɑ = 0.889, intra-class correlation coefficient = 0.933 and 95% confidence interval = 0.898–0.956) and sensitive to change (effect size = 1.17 and 0.86, respectively, in patients who decreased [n = 34] and increased [n = 31] questionnaire-score after intervention). VH phenomenon was 35.5%. VH phenomenon was present in a substantial number of Mexican patients with RMDs. The COVID-19 VH questionnaire showed good psychometric properties to assess COVID-19 VH in our population.

KEYWORDS:

• Covid-19 vaccine
• vaccine hesitancy
• rheumatic diseases patients

Introduction

The coronavirus disease 2019 (COVID-19) pandemic has imposed an enormous disease burden worldwide, and there are currently limited treatment options.^1,2 The risk of developing severe complications or death is more significant among older patients and those with comorbidities than their counterparts.³ There are conflicting data concerning the actual incidence and severity of SARS-CoV-2 infection in patients with rheumatic diseases (RMDs). Registry data and meta-analyses have shown that the incidence is greater among patients with RMDs and that there might be an association with a prednisone dose of >10 mg/day.^4–6 Meanwhile, recent viewpoints suggest overall a similar risk and similar course to those of the general population, with some rheumatic diagnoses, disease activity statuses, and disease-specific treatments associated with worse outcomes.²

Developing a safe, effective, and affordable vaccine has been recognized as the most promising strategy to contain the COVID-19 pandemic. However, vaccine hesitancy (VH) confers a different and unique hurdle to researchers, scientists, governments, and community leaders.⁷ VH is the term used to define refusal or reluctance in accepting vaccination despite the availability of vaccination services.⁸ Before the pandemic, the World Health Organization (WHO) identified VH as a top global health threat in 2019.⁹ More recently, this phenomenon has been defined as an imminent threat in the battle against COVID-19 because achieving herd immunity depends on the population’s willingness to accept vaccination.^10,11 A 32-country survey in 26,758 individuals, conducted between October 21 and December 16, 2020, confirmed significant variations among countries of potential acceptance of COVID-19 vaccines.¹²

Moreover, results from additional surveys overall seem to suggest that willingness to vaccinate against COVID-19 has declined globally between the early months of the pandemic and December 2020.^13–19 However, rates tend to fluctuate and might be associated with COVID-19 risk perception,²⁰ and vaccine safety-related issues.²¹ In addition, vaccination campaigns are becoming part of the political debates. At the same time, there is considerable scientific uncertainty. Available vaccines are limited, particularly in developing countries, and politicians – rather than experts – are the public face of crisis management, all of whom might impact VH rates.¹⁵

Currently, there are limited studies that assess COVID-19 VH among patients with RMDs.^22–27 Previous studies highlight that vaccine uptake in general, is deficient among specific rheumatic diagnoses such as Rheumatoid Arthritis (RA), although with significant variations among countries.^28,29 People with RMDs may have specific concerns on how their underlying disease and their immunomodulatory therapies affect the benefit and safety of receiving COVID-19 vaccination, in addition to fear about potential RMD´s flares after vaccination, that might be related or not to holding antirheumatic medications.^6,30 A recent international survey assessed patient perception and outcomes related to COVID-19 vaccines among 2860 vaccinated adult patients with RMDs (mean age 55.3 years, 86.7% female, 86.3% white, and 42.3% with RA).³⁰ The study showed that adverse events affected up to 48% of the participants (primarily fatigue, headache, and widespread muscle/joint pains), while flares of the underlying rheumatic disease that required medication changes, occurred in less than 5% of the population. Finally, recommendations from healthcare professionals are strong drivers of vaccine acceptance among individuals and patients with RMDs,^22,23,31 and international organizations currently agreed that rheumatic patients might benefit from COVID-19 vaccination.^32–34 Moreover, among the vaccinated adult patients with RMDs previously described, 82% had discussed COVID-19 vaccination with their healthcare provider, and 96% reported that vaccination was recommended.³⁰ Nonetheless, when it comes from VH, what matters most varies across time and communities, highlighting the relevance of addressing the topic from a cultural perspective, which has other ethical implications.

The study’s primary objective was to determine COVID-19 VH among Mexican outpatients from two national referral centers for RMDs. As a secondary objective, we aimed to adapt and validate the Oxford COVID-19 Vaccine Hesitancy Scale¹⁹ in Mexican outpatients with RMDs.

Patients and methods

Setting and study population

This cross-sectional study was performed between March 1st and August 13th, 2021, and two tertiary-care level and academic centers for RMDs located in Mexico City, contributed with participants to assess VH phenomenon (ClinicalTrials.gov Identifier: NCT04775563): The Instituto Nacional de Ciencias Médicas y Nutrición Salvador-Zubirán (INCMyN-SZ) and the Hospital General de México Dr. Eduardo Liceaga (HGM). Before the pandemic, around 5,000 and 13,000 patients were attending each outpatient clinic and suffered from a variety of RMDs. The most frequent diagnoses were Systemic Lupus Erythematosus (SLE) and RA (Table 1).

Table 1. No (%) of patients with at least one visit to the outpatient clinic with the ten most frequent diagnoses specified (January 2019 to December 2019)

Outpatient clinic INCMyN-SZ n = 4476		Outpatient clinic HGM n = 12935
Specific rheumatic diagnosis^a	no (%)	Specific rheumatic diagnosis^a	no (%)
Systemic Lupus Erythematosus	1652 (36.9)	Rheumatoid Arthritis	5673 (43.9)
Rheumatoid Arthritis	1578 (35.3)	Systemic Lupus Erythematosus	3109 (24)
Systemic Sclerosis	239 (5.3)	Osteoarthritis	851 (6.6)
Systemic Vasculitis	220 (4.9)	Fibromyalgia	773 (6)
Primary Sjögren’s Syndrome	190 (4.2)	Gout	759 (5.9)
Spondyloarthritis	174 (3.9)	Systemic Sclerosis	691 (5.3)
Inflammatory myopathies	150 (3.4)	Inflammatory myopathies	373 (2.9)
PAPS	150 (3.4)	Ankylosing Spondylitis	283 (2.2)
MCTD	94 (2.1)	Human Adjuvant Disease	259 (2)
Adult Still disease	29 (0.6)	PAPS	164 (1.3)

PAPS = Primary Anti-Phospholipid Syndrome; MCTD = Mixed Connective Tissue Disease.

^aDiagnosed based on the attending rheumatologist criteria. INCMyN-SZ = Instituto Nacional de Ciencias Médicas y Nutrición Salvador Zubirán. HGM = Hospital General de México Dr. Eduardo Liceaga.

During the study period, consecutive outpatients with a definite rheumatologic diagnosis according to the attendant rheumatologist criteria (among those included in Table 1) were invited to participate. Exclusion criteria included patients on palliative care.

Study design

This cross-sectional study was performed in three phases.

Phase 1 consisted of the translation and cross-cultural adaptation to Mexican patients with RMDs of the Oxford COVID-19 Vaccine Hesitancy Scale, a 1-factor predominant, validated seven-items instrument that assesses VH in adult individuals.¹⁹ Briefly, the original scale was tested and validated in 5114 adult individuals from the UK; gender, age, ethnicity, income, and region quotas were considered. Initially, a pool of 15 items was developed, and feedback from public involvement groups, including ethnic minority health groups representatives, was obtained. Exploratory and confirmatory factor analyses were used to derive a final seven-item scale, with higher scores indicating a higher level of VH. Principles of good practice for the translation and cultural adaptation process for patient-reported-outcomes measures from the ISPOR task force were followed,³⁵ and the primary author of the original scale agreed to collaborate.

Phase 2 consisted of validating the translated and adapted to patients with RMDs version, performed after pilot testing in 50 outpatients with RMDs to examine questionnaire feasibility (Table 2). Judgment experts determined COVID-19 VH questionnaire content validity. The expert committee was integrated by six rheumatologists and four specialists in infectious diseases. Experts were directed to evaluate items and their scale responses according to the following criteria: Relevance, good wording, and appropriate language and meaning for the target population; they were also directed to evaluate the questionnaire face validity. Construct validity was evaluated using factorial analysis. Criterion validity was examined according to the ten-item Likert scale from the Spanish version for Guatemala of the Vaccine Hesitancy Scale (convergent validity)³⁶ and according to self-referred previous 5-years influenza vaccination (divergent validity). COVID-19 VH questionnaire reliability was assessed with internal consistency and temporal stability (test-retest), which was tested after the questionnaire was applied to (at least) 50 patients, twice in the same day, with (mean) 5 hours between both applications; this short lag-time was chosen due to daily intensive information related to COVID-19 vaccines safety in the social media, while considering recall bias. Finally, questionnaire sensitivity to change was examined in (at least) 20 patients per group, those who increased and those who decreased VH score after an informative/educative intervention. The questionnaire was applied twice, at baseline and two weeks apart, after patients received COVID-19 vaccine infographics and were allowed to communicate with dedicated healthcare professionals to resolve doubts. The information was locally adapted from the Center for Disease Control and Prevention COVID-19 vaccine.³⁷ The two-week lag time was considered convenient to ensure patient review of the educative material and comfortable communication with experts.

Table 2. COVID-19 VH questionnaire feasibility

Feasibility categories	n (%) of patients
Adequate time to fill the questionnaire	50 (100)
Incomplete questionnaires^a	2 (4)
Perceived instructions clarity	49 (98)
Perceived items clarity	49 (98)
Perceived items scale response clarity	49 (98)
Format acceptance	49 (98)

^aTwo patients referred skipped items. Mean ± SD minutes to fill the questionnaire was: 1.6 ± 0.6.

Phase 3 was designed to assess the VH phenomenon in the target population and consisted of the application of the validated version of the COVID-19 VH questionnaire to consecutive outpatients with RMDs, which is a seven-items instrument, based on a five-point Likert scale from one to five, with higher scores translating into a higher level of VH (Appendix 1). Standardized formats were used to assess in all the patients, the current level of disease activity (without disease activity and low, moderate, and high disease activity level), disease activity control (Yes/No), and (direction of) treatment recommendations at the end of the consultation (no changes, treatment intensification, treatment reduction, treatment changes because of adverse events and treatment re-initiation after withdrawal).³⁸

Relevant sociodemographic variables (sex, years of age and formal education, occupation, living with a partner, and universal access to health care), disease-related variables (specific rheumatic diagnosis, years of disease duration, comorbid conditions based on the Rheumatic Diseases Comorbidity Index,³⁹ one year-previous hospitalization and number), and treatment-related variables (immunosuppressive treatment and number of immunosuppressive drugs/patient, and corticosteroid use) were obtained in standardized formats after a careful chart review and patient interview, during phases 2 and 3.

Questionnaires application

Consecutive patients with a definite rheumatic diagnosis were invited to participate. Patients were identified at the outpatient clinic while waiting for a scheduled consultation. Upon patient´s agreement to participate, interviews were performed in a designated area for research purposes, and questionnaires and scales were administered to patients, who were instructed to fill them. Instruments were applied in a specific and standard order, with the COVID-19 VH questionnaire first applied. If necessary, patients could be assisted to fill questionnaires by trained personnel.

Strategies for quality control

Before the study initiation, its design and objectives were shared with the physicians involved in outpatients’ care, and agreement on terminology and concepts was achieved. Also, patients with RMDs participated in items adaptation to RMDs and pilot testing to avoid bias in question design and pretest bias.⁴⁰ In all the cases, questionnaires and scales were applied by trained personnel not involved in patient care, on the same day that patients visited their primary rheumatologists, and in a particular location within the outpatient clinic that was suitable for clinical research, to facilitate confidentiality and to reduce bias associated to questionnaire administration.⁴⁰ Finally, questionnaires and scales were reviewed to identify missing information.

Description of sample and sample size calculation

Three different convenience samples of consecutive outpatients were included, and quotes were considered to represent the distribution of the other rheumatic diagnoses for each outpatient clinic. In addition, nine outpatients participated in phase 1.

The first sample (S-1) included 50 outpatients from center 1. The number of patients had followed the recommendations for pilot testing.⁴¹

The second sample (S-2) included (at least) 200 outpatients from center one and was used for the validation process. The sample size was based on published methodological recommendations.⁴¹ The sample size for questionnaire sensitivity to change was estimated in (at least) 20 randomized patients per group (with post-intervention increase and decrease COVID-19 VH questionnaire score), to detect a moderate effect size (Cohen’s d = 0.5), 90% of power, and alpha = 0.05, one-tailed.^42,43 Similarly, (at least) 50 additional patients were randomly selected for test-retest.

The patient’s randomization process was conducted through a computer-generated randomization scheme using randomly permuted blocks (software available at website www.randomization.com). A priori, an independent collaborator generated the random allocation sequence. The list was divided into two groups of patients, those allocated to sensitivity to change maneuver and those allocated to test-retest; it also included a consecutive number and the assigned group.

The third sample (S-3) included (at least) 600 outpatients derived from both centers. Together, outpatient clinics registered 18,000 patients with RMDs in 2019. We estimated the sample size to detect VH considering a prevalence of 30%.²⁰ We obtained 577 patients with a 95% confidence level and 4% precision.

All sample size calculations were performed using G power 3.1.9.2 version.

Statistical analysis

The COVID-19 VH questionnaire score was calculated as the mean of individual items scores, and the “Do not know” option response was excluded from scoring.¹⁹ In addition, we agreed that to provide a score, at least six out of the seven items should be scored, which led that (mean) score ranged from 0.86 to 7.00. VH was defined if the score assigned was ≥ 1.86. This cutoff was based on the 75-percentile data logarithmic transformation.⁴¹

Phases 1–3: Descriptive statistics were performed to describe the variables of the patients included in the three samples, with frequencies and percentages for categorical variables or the mean/median and standard deviation (SD)/Q25-Q75 for continuous variables with normal/non-normal distribution.

Phase 2: Content and face validity by experts were examined with agreement percentages. Construct validity was evaluated using exploratory factorial analysis (principal components). Criterion validity (convergent) was analyzed using Spearman rank correlation coefficient (rho) to determine the strength of the relationship between the COVID-19 VH questionnaire score and the ten-item-Likert-scale score from the Spanish version for Guatemala of the Vaccine Hesitancy Scale;³⁶ the strength of the correlation was interpreted according to published recommendations.⁴¹ Divergent validity was examined using the Mann Whitney U test to compare the COVID-19 VH questionnaire score between patients who self-reported five years of Influenza vaccination and those who denied it. Cronbach’s α was used to assess the internal consistency of the questionnaire. For temporal stability/test-retest, intra-class correlation coefficients (ICC) and their 95% confidence intervals (CI) were calculated using a single measurement, absolute-agreement, 2-way mixed-effects model. Cronbach’s α, ICC, and 95% CI interpretations followed published recommendations.^42,44,45 COVID-19 VH questionnaire sensitivity to change was defined with the distribution-based indicator effect size.^42,43 Finally, floor and ceiling effects of the questionnaire were determined as the percentage of patients who achieved the lowest and highest score of the scale, respectively.

Missing data were below 1%, and no imputation was performed.

All statistical analyses were performed using Statistical Package for the Social Sciences version 21.0 (SPSS Chicago IL). A value of p < .05 was considered statistically significant.

Ethics

The Internal Review Board from each Institution approved the study. Written informed consent was obtained from all the patients who participated.

Results

Phase 1. Translation to Spanish and adaptation to patients with RMDs of the Oxford COVID-19 vaccine hesitancy scale

The process included ten steps, which are summarized in Table 3. The resultant COVID-19 VH questionnaire was subjected to psychometric validation.

Table 3. Summary of the translation and adaptation to RMDs of the Oxford COVID-19 vaccine hesitancy scale

STEPS	MODIFICATIONS AND INCORPORATIONS
Preparation	Two local collaborators were recruited; both were bilingual (Spanish, English) and specialists in infectious diseases and rheumatology. In addition, the Oxford COVID-19 Vaccine Hesitancy Scale developer (Daniel Freeman) agreed to collaborate.
Forward translation	Two independent forward translations were received (English-Spanish), which were identical. Translations were provided by two different certified translators.
Reconciliation of the forward translation into one single forward translation and adaptation to RMDs	The translation was adapted to RMDs by two rheumatologists and one PhD in Health Sciences (expert in RMDs). Each one proposed one RMDs-adapted version. RMDs-adapted versions were evaluated by an experts-committee, integrated by nine patients with RMDs, five rheumatologists, four infectious diseases specialists, one general physician and one psychologist. Experts evaluated items and instructions from each version and a final version was integrated after ≥ 80% of the participants agreed.
Back translation	One backward translation into the source language (English) was performed by a certified translator (version 1).
Back translation review	The developer of the original instrument approved the back translation, although suggested modifications to item one and to item six (version 2).
Harmonization	The project responsible (VPR) did not identify potential problematic items.
Cognitive debriefing	The version 2 was evaluated and approved by nine outpatients with RMDs: three patients had Rheumatoid Arthritis, two patients had Systemic Lupus Erythematosus, two patients had IgG4-related disease, one patient had Primary Antiphospholipid Syndrome and one patient had Overlap Disease.
Review of cognitive debriefing results and finalization	The version 2 was approved by the project responsible. Terms were harmonized along the questionnaire.
Proofreading	The version 2 was proofread by the project responsible and the local collaborators.
Final report	Elaborated by the responsible of the project.

Phase 2. Validation of the translated and adapted to patients with RMDs version: the COVID-19 VH questionnaire

Samples’ description (Table 4)

Table 4. Description of the samples´ characteristics

	S-1 n = 50	S-2 n = 208	S-3 n = 600
Sociodemographic characteristics
Female sex*	45 (90)	185 (88.9)	519 (86.5)
Years of age	47 (36.8–56)	48 (36–58.8)	46 (35–56)
Years of formal education	12 (9–17)	12 (9–16)	12 (9–15)
Formal and non-formal job*	24 (48)	83 (39.9)	252 (42)
Married or Living with a partner*	22 (44)	101 (48.6)	314 (52.3)
Universal access to health care*	11 (22)	56 (26.9)	148 (25.5)
Disease related-variables
Patients with RA diagnosis*	15 (30)	71 (34.1)	202 (33.7)
Patients with SLE diagnosis*	21 (42)	90 (43.3)	245 (40.8)
Years of disease duration	10.5 (6–17)	12 (6–20.8)	9 (4–16)
Rheumatic Diseases Comorbidity Index score	1 (0–1)	1 (0–1)	0 (0–1)
One year-previous hospitalizations*	5 (10)	33 (15.9)	48 (13)
Patients with adequate disease activity control*	37 (74)	155 (74.5)	453 (75.6)^b
Treatment-related variables
Immunosuppressive drugs*	44 (88)	165 (79.3)	529 (88.2)
No of immunosuppressives/patient^a	2 (1–2)	2 (1–2)	1 (1–2)
Corticosteroids use*	23 (46)	103 (49.8)	259 (43.2)
Treatment direction (after consultation)*^b
No changes	29 (58)	116 (55.8)	294 (49.1)
Treatment intensification	8 (16)	47 (22.6)	153 (25.5)
Treatment reduction	12 (24)	35 (16.8)	112 (18.7)
Treatment changes because adverse events	1 (2)	9 (4.3)	17 (2.8)

Data presented as median (Q25-Q75) as otherwise indicated.

*Number (%) of patients.

^aAmong those with the characteristic.

^bSeven missing data.

S-1 and S-2 included patients which data were used for the validation process. Twenty-three patients scored <six items, and their data were excluded from analysis, although their characteristics were similar to those of the patients analyzed. The final number of patients included in each sample was 50 in S-1 and 208 in S-2.

Patients from both samples were representative of outpatients with RMDs from center 1. Briefly, they were primarily middle-aged females, with (median) 12 years of formal education; patients had substantial disease duration, and the underlying RMD was under control; the majority of them were on immunosuppressive drugs, and almost half of them on corticosteroids.

Figure 1 summarizes the most frequent diagnoses.

Figure 1. Rheumatic diagnoses distribution in S-1, S-2 and S-3.

Pilot testing for COVID-19 VH questionnaire feasibility

The questionnaire was found feasible by 50 patients with RMDs, as summarized in Table 2.

Judgment experts’ questionnaire content validity

Experts generally agreed on the item and scale response evaluation (≥90% agreement for item relevance and appropriate language and meaning for the target population and ≥80% agreement for good wording), but item 6, which reached 70% agreement for the “good wording” section; after modifications, 90% of the experts approved item 6. In addition, the questionnaire showed 100% face validity.

COVID-19 VH questionnaire construct validity

Table 5 summarizes results from factor analysis. The KMO measure was 0.886, and a significant result (X² = 744.956, p ≤ 0.001) for the Bartlett sphericity test confirmed the adequacy of the sample. A 1-factor structure was extracted, which accounted for 60.73% of the total variance. The post-validation structure of the questionnaire did not differ from that of the original scale.

Table 5. Item loadings for the 1-factor of the C19-VH questionnaire

	1
Would you take a COVID-19 vaccine (approved for use in the Mexico) if offered?	0.791
Currently there are different COVID-19 vaccines. You consider that:	0.853
I would describe my attitude toward receiving a COVID-19 vaccine as:	0.800
If a COVID-19 vaccine was available for you, you would:	0.781
If my family or friends were thinking of getting a COVID-19 vaccination, I would:	0.740
Regarding COVID-19 vaccination, I would describe myself as:	0.812
I consider that taking a COVID-19 vaccination is:	0.663

Matrix to 1-component extracted with the method of principal component analysis.

COVID-19 VH questionnaire criterion validity

Convergent validity

There was a low but significant correlation between the questionnaire score and the ten-item-Likert-scale score from the Spanish version for Guatemala of the Vaccine Hesitancy Scale,³⁶ rho = 0.156, p = .025.

Divergent validity

Patients who reported five years of previous influenza vaccination had lower questionnaire scores than those who denied previous vaccination: 1.6 (1.1–1.9) vs. 1.9 (1.4–2.3), p ≤ .001.

COVID-19 VH questionnaire reliability: internal consistency and temporal stability

Results of internal consistency (Cronbach’s α) and temporal stability/test-retest (ICC and 95% CI) for individual items scores and global questionnaire score are presented in Table 6, which additionally includes median (Q25-Q75) scores and floor and ceiling effects. Test-retest was performed in 91 patients randomly selected from S-2.

Table 6. Psychometric characteristics of the C19-VH questionnaire

	Median (Q25-Q75)	Cronbach Alpha^a	ICC (95% CI)^b	Floor/ceiling effects (%)
Item 1. Would you take a COVID-19 vaccine (approved for use in Mexico) if offered?	1 (1–2)	0.891	0.765 (0.645–0.845)	67.3/3.4
Item 2. Currently there are different COVID-19 vaccines. You consider that:	2 (1–2)	0.874	0.807 (0.709–0.873)	45.2/2.4
Item 3. I would describe my attitude toward receiving a COVID-19 vaccine as:	2 (1–3)	0.863	0.646 (0.466–0.766)	24/1
Item 4. If a COVID-19 vaccine was available for you, you would:	1 (1–1)	0.873	0.721 (0.578–0.816)	79.8/2.4
Item 5. If my family or friends were thinking of getting a COVID-19 vaccination, I would:	2 (1–2)	0.875	0.781 (0.668–0.856)	40.9/1
Item 6. Regarding COVID-19 vaccination, I would describe myself as:	2 (2–2)	0.880	0.804 (0.703–0.871)	21.6/1
Item 7. I consider that taking a COVID-19 vaccination is:	1 (1–2)	0.871	0.506 (0.257–0.672)	55.8/1
COVID-19 VH questionnaire	1.6 (1.3–2)	0.889	0.933 (0.898–0.956)	12/0.5

^aIf item deleted (but for COVID-19 VH questionnaire data).

^bIn 91 randomized patients from S-2. ICC: Intraclass correlation coefficient. CI: Confidence Interval.

COVID-19 VH questionnaire sensitivity to change

There were 70 patients from S-2 (excluding 91 patients randomized for test-retest) who were randomly selected for sensitivity to change analysis. They all received infographics as previously described, and 29 patients (41.4%) communicated with experts. Following the educative intervention, 34 patients decreased the COVID-19 VH questionnaire score (change of 76%) and 31 patients increased the score (change of 50%), and effect size values were 1.17 and 0.86, respectively. Five patients maintained the same score.

Phase 3. Covid-19 VH phenomenon

S-3 description

Table 4 summarizes patients’ characteristics. Briefly, the 600 patients included were primarily middle-aged (46 years [35–56]), females (86.5%), with (median) 12 years of formal education; almost half were married or living with a partner (52.3%), while 42% referred having a job (formal or non-formal). The most frequent diagnoses were SLE (40.8%) and RA (33.7%), and additional diagnoses are presented in Figure 1. Patients had (median) 9 years of disease duration and Rheumatic Disease Comorbidity Index score of 0. The majority of them had adequate control of the disease activity status (75.6%) according to the physician evaluation and were on immunosuppressive drugs (88.2%), while 43.2% received corticosteroids.

VH phenomenon

There were 19 questionnaires with less than six items scored, and they were discarded from the analysis. Median (Q25-Q75) COVID-19 VH questionnaire score in the 581 questionnaires left was 1.57 (1.29–2). Based on the cutoff proposed (≥1.86, which corresponds to 75-percentile data logarithmic transformation), 206 patients (35.5%) had VH.

Discussion

The present study revealed that VH was present in 35.5% of Mexican outpatients with RMDs, from two academic centers in Mexico City. To our knowledge, this is the first study that uses a validated questionnaire/scale to assess COVID-19 VH among patients with RMDs. In them, VH-related knowledge has been conceived based on online independent survey applications,^22–25 phone survey applications,²⁶ and face-to-face interview/survey application,²⁷ all of whom lack a solid development and validation process. Meanwhile, the use of self-report or poorly designed instruments can result in misclassification bias. VH is a complex theoretical construct, which measurement should ideally involve its operationalization in a defined variable and the development and application of an instrument to its adequate quantification. Key indicators of the quality of a measuring instrument are its reliability and validity; in addition, the responsiveness of the measure to change is of interest in the VH phenomenon, where its modification in the appropriated direction as a result of healthcare professional’s intervention is a primary target.⁴⁶

The COVID-19 VH questionnaire development and validation processes were rigorous methodological.^19,35 Relevant healthcare professionals (considering the target population), primarily rheumatologists, infectious disease specialists and psychologists, and patients with representative RMDs contribute to reframing the construct under validation. Clinicians may be the best observers for certain aspects of RMDs, but only patients can report on more subjective elements such as the expressed intent to accept a COVID-19 vaccine.⁴⁷ The wording for single items was optimized through a process of face validity testing with a multidisciplinary group of healthcare providers. Meanwhile, instruction’s wording and clarity were submitted to patient’s evaluations, as was the final version of the questionnaire, before its formal validation process; the changes subsequently made were crucial for enhancing acceptance by the intended population. Importantly, different samples of consecutive outpatients with RMDs, representative of real-world outpatients attending two tertiary-care-level centers, were used for analysis. Accordingly, our results could be generalized to populations of Spanish-speaking patients with similar characteristics.

The COVID-19 VH questionnaire showed adequate internal consistency with a good Cronbach’s α coefficient for the total scale.⁴⁴ The same researcher assessed the test-retest reliability in 91 patients, and we observed an ICC and 95% CI indicating good reliability.⁴⁵ The construct validity was demonstrated by KMO sampling and Bartlett's test of sphericity, both confirming the adequacy of the sample size for conducting factor analysis,⁴⁸ and a single factor structure was extracted, accounting for 60.73% of the variance. Face and content validity were examined by a multidisciplinary group of experts involved in RMDs and COVID-19 management (during the pandemic, the Institution was declared a dedicated COVID-19 hospital). Divergent validity was documented by significant differences in VH COVID-19 scores between those who reported five years-previous Influenza vaccination and their counterparts. Also, the COVID-19 VH global score showed neither floor nor ceiling effect, defined when more than 15% of the patients achieved the lowest or highest score, respectively; floor and ceiling effects can reduce the possibility of detecting change over time. The questionnaire showed sensitivity to change, was feasible based on patients’ evaluation, and was suitable for low-literacy patients. A more significant percentage of change was detected in the patients who improved the score compared to those who deteriorated the score. Surprisingly, a substantial number of patients increased the score and indicated that the educative maneuver was unsuccessful in them, probably related to personal solid anti-vaccine beliefs; also, in these patients, the educative intervention might have been insufficient to compensate anti-vaccine feedback from social media, friends and/or family. Finally, the COVID-19 VH questionnaire had a single score which ensures simplicity in routine clinical practice, while a short time was required to fill the questionnaire (mean 1.6 minutes). Also, we propose that at least six out of seven items be scored to guarantee that the COVID-19 VH construct is comprehensively assessed.

Our rate of COVID-19 VH found (35.5%) was within the range of previous descriptions in different populations of patients with RMDs,^{22–27,49–51} all of whom were survey-based. Four studies were performed in Italy,^24,26,49,50 and patient’s willingness to potentially receive a SARS-CoV-2 vaccine ranged from 54.9%²⁴ to 82%.⁴⁹ Gauer et al.²⁷ did face-to-face structured interviews with 280 Indian patients with RMDs and 102 controls, using a questionnaire previously tested in 20 patients; they found a vaccine acceptance of 54% among the patients. Yurttas et al.²⁵ conducted a web-based questionnaire survey in three groups of participants from Istanbul (Turkey), among whom 732 had RMDs; the online survey was designed and executed following published recommendations⁵² and was tested in 22 hospital workers; authors reported that 29.2% of the patients with RMDs were willing to get vaccinated, 19.0% were unwilling, and 51.8% were undecided. Boekel et al.²² reported data from an online survey answered by 1361 patients with RMDs, 366 patients with multiple sclerosis, and 682 controls from the Netherlands; the proportion of patients and controls who would be willing to get vaccinated against SARS-CoV-2 was similar, 61% in the patient group and 65% in the control group. Felten et al.²³ applied 57 web-based questions regarding COVID-19 vaccination to 1266 patients from 56 countries with a self-reported diagnosis of systemic autoimmune or inflammatory rheumatic disease, and healthcare professionals; they found that 54.2% of the patients were willing to get vaccinated against SARS-CoV-2, 32.2% reported uncertainty and 13.6% refused vaccination. A cluster analysis of the previous results, showed that there were 53 Mexican patients with RMDs, further characterized in cluster 1 (“Most willing to get vaccinated,” n = 5), cluster 2 (“More hesitant,” n = 25) and cluster 3 (“Mostly opposed to getting vaccinated,” n = 23).⁵³ Finally, Ko et al.⁵¹ applied an online de-identified survey to 3451 adult Australian patients from rheumatology clinics, to describe the SARS-CoV-2 VH rate and factors influencing it; the survey response rate was 18.6%, and 34.4% of the patients were vaccine-hesitant (unwilling or undecided); survey content validity and feasibility were explored with the participation of four specialist clinicians and three patients research partners. As shown, a wide range of COVID-19 VH rates had been described across different populations of patients with RMDs, which might be associated with factors related to population characteristics, the timing of the VH phenomenon measurement, and how the COVID-19 VH construct was defined and measured. Importantly, none of the studies considered Mexican patients, but the cluster analysis international study included 53 Mexican patients with RMDs, although specific diagnoses distribution was not detailed.⁵³ Recent local and vaccine-related scientific literature highlights that 79.8% of Mexican patients with RMDs were vaccinated for influenza at least once during 2019; the study addressed knowledge and attitudes about influenza vaccination in 223 patients with RMDs, primarily integrated with RA patients (64.8%).⁵⁴

Some limitations of the study need to be addressed. At first, the COVID-19 VH questionnaire validation process was conducted at a single academic center. At the same time, the VH phenomenon rate was established in two tertiary-care-level centers located in a metropolitan area, where attending RMDs patients might have particular characteristics which might affect the VH phenomenon rate. Second, most of the patients included had SLE and RA diagnoses, while other diagnoses were mildly represented in our patients. Third, the study was performed in Spanish-speaking Mexican patients, which might affect the generalizability of the results to all Spanish-speaking patients with RMDs; nonetheless, the seven-item questionnaire was found feasible in low literacy patients, and we consider it can be generalized to Spanish speaking patients from the Latin-American region. Forth, we did determine a limited number of psychometric characteristics of the questionnaire, those considered necessary for a first approach. Also, alternative and sophisticated strategies for judging the quality of a measuring instrument, such as Rasch models and item-response theory or latent-trait models, were not used. Fifth, patients included might have responded socially acceptable or in line with the impression they want to create, which might be particularly true in the context where the COVID-19 VH questionnaire was applied. Sixth, it is relevant to note that reported intentions may not always translate into vaccine uptake,⁵⁵ and the VH phenomenon rate obtained should be considered cautiously.

Conclusions

We presented the rationale and the methods for the translation and cultural adaptation and validation process of a questionnaire allowing for assessing the COVID-19 VH phenomenon in Spanish-speaking patients with RMDs. The questionnaire was valid and reliable and can be easily implemented in the outpatient setting and for research purposes, particularly in low literacy patients. Documenting VH rates is the first step and should be followed by collecting and analyzing data on the reasons for patients (and physicians) VH, which is critical for informing the design of effective vaccine messaging in the community of patients with RMDs.⁵³ We are currently performing a mixed-methods design study to address the topic comprehensively.

Published evidence regarding the RMDs-related COVID-19 VH phenomenon has been carried out mainly in developed countries. There is a need to address the research gap and consider data from Latin-American region patients, where highly represented low- and middle-income countries. As recently published,⁵⁶ a higher willingness to take a COVID-19 vaccine was found in low-middle income countries compared with the United States and Russia, in the general population. At the same time, health workers were the most trusted sources of guidance about COVID-19 vaccines. Finally, vaccination campaigns should focus on enhancing vaccine acceptance among patients with RMDs and translating the levels of stated acceptance into actual uptake.

Acknowledgments

We acknowledge the participation of Daniel Freeman, Gabriel Medrano, José Francisco Moctezuma, Conrado García, Julio Casasola, Janitzia Vázquez-Mellado, Gabriela Huerta, Roberto Cruz, Alfonso Gastélum-Strozzi, Lexli D. Pacheco-Santiago and Vivian A. Estrada-González.

Supplemental data

Supplemental data for this article can be accessed on the publisher’s website at https://doi.org/10.1080/21645515.2021.2003649.

Disclosure statement

No potential conflict of interest was reported by theauthor(s).

Additional information

Funding

The author(s) reported there is no funding associated with the work featured in this article.