Implementation of the human papillomavirus school-entry requirement in Puerto Rico: barriers and facilitators using the consolidated framework for implementation research

ABSTRACT

In 2018, Puerto Rico (PR) enacted a Human papillomavirus (HPV) vaccine school-entry requirement for students ages 11 to 12. Using the Consolidated Framework for Implementation Research (CFIR), we aimed to identify potential barriers and facilitators of this implementation. We conducted 38 qualitative interviews with stakeholders in PR from different organizations (Department of Health, Schools, Healthcare Providers, and Community organizations). We evaluated construct rating variability between the organizations to determine barriers and facilitators. The strongest facilitator determined was stakeholder’s awareness of the parent’s and student’s needs to meet the HPV school-entry requirement. Other facilitators included initiatives for school-entry policies and the relative advantage of this requirement over different strategies. The strongest barriers included was the cost for private providers to administer the HPV vaccine, the negative influence of social media about the vaccine, which affected parents’ acceptance, and the lack of school nurses as available staff resources for the school entry requirement. Findings from this study can be used to improve implementation (adaptations/modifications) and inform other states and countries in earlier stages of consideration of the adoption of similar immunization policies. Most barriers can be modifiable with the implementation of educational programs/training across schools, considering that they are the first line of response to parents of this school entry requirement.

KEYWORDS:

• CFIR
• HPV vaccine
• school entry requirement
• Puerto Rico
• implementation research
• Hispanic
• stakeholders

Introduction

The Advisory Committee on Immunization Practices (ACIP) recommended the HPV vaccine as part of the routine vaccination for girls (June 2006) and boys (October 2011) ¹ of age 11–12 years. This initiative was in response to the high incidence of HPV infection and HPV-related cancers affecting about 35,300 of the United States (US) population each year.² Based on this recommendation, at least 42 states or jurisdictions have introduced legislation regarding the HPV vaccine, ranging from efforts including providing funding for the HPV vaccine, to educational programs directed to awareness regarding HPV school-entry requirements.^3,4

School-entry requirements for vaccines have been an effective strategy to increase child immunization rates and decrease the incidence of vaccine-preventable diseases.^5,6 For the HPV vaccine, five states or territories/jurisdictions in the US (Rhode Island, Virginia, Puerto Rico [PR], the District of Columbia [DC], and more recently Hawaii) have enacted this policy.³ Opt-out provisions and students’ characteristics vary. For Virginia⁷ (enacted in 2008) and Washington DC³ (2009), this policy requires that all girls entering sixth grade should be vaccinated against HPV. However, parents could opt-out through medical, religious, or personal belief exemptions.⁸ In 2015, Rhode Island implemented this policy, gradually, in which the school requirement indicates that seventh graders were required to have the first dose, eighth graders were required to have the second dose, and ninth graders were required to have the third dose.⁹

PR has the highest incidence rates of cervical cancer of all US states and territories (PR:11.7 vs. US:7.4 per 100,000),¹⁰ and incidence rates are higher for the Hispanic population overall compared to non-Hispanic whites in the US.¹¹ Specifically for cervical cancer, incidence rates showed an increase of 2.4% per year from 2001 and 2017 in PR¹² while for anal cancer, incidence increased 4.9% from 2001 and 2016.¹³ Due to its public health relevance, in 2018 the Secretary of Health of PR implemented the HPV vaccine school-entry as a requirement for all students ages 11–12 years old.¹⁴ Since the implementation enactment, two different amendments to this requirement have occurred: (1) extension of school-entry requirement’s age range to 14 years for the 2019–2020 academic school year,¹⁵ and (2) extension of the age range to 16 years for 2020–2021.¹⁶ The last jurisdiction to enact an HPV school entry requirement in the US was Hawaii (July 2020) for 6^th grader males and females.¹⁷

The effectiveness of school-entry mandates to reach optimal HPV vaccination uptake might be affected by several factors that can impede its successful implementation. There is plenty of qualitative research conducted in the US that addresses HPV vaccination uptake. These studies have identified that low parental acceptance,^18,19 invasion of parental autonomy (governmental interference),¹⁹ limited providers recommendation,^19–21 vaccine safety concerns,¹⁹ stigma associated with vaccines against STIs,¹⁹ financial burden,^3,18 the flexibility of opt-outs,²² and lack of knowledge^22,23 are related to low HPV vaccine uptake. Although most stakeholders interviewed prior in the literature were part of the clinical setting (including healthcare providers), few studies have included stakeholders from other organizations such as school and government agencies (Department of Health).^22,24,25

Understanding the process of the implementation of mandatory vaccination for preventable diseases is critical to determine the facilitators and barriers across different settings or contexts that could be involved in its success. The Consolidated Framework for Implementation Research (CFIR), a meta-theoretical framework, has been used across various studies as a practical application to examine the barriers and facilitators across the implementation process.^26–28 This framework has been used previously to describe similar interventions related to HPV vaccination in adolescents.^29–31 CFIR allowed us to obtain a full scan of the HPV school-entry requirement, including barriers and facilitators and the magnitude, strength, and direction of the implementation. The purpose of our study was to evaluate the implementation process of the HPV school-entry requirement in PR using the CFIR framework. To our knowledge, our Human Papillomavirus Policy Implementation Vaccination (HPV-PIVac) study is the first study in PR using the CFIR to explore the barriers and facilitators of this HPV vaccine policy from the perspective of stakeholders, concurrently with the implementation of this critical immunization requirement.

Methods

Study design and participants

This analysis is part of the HPV-PIVac study, which aims to understand the geographic variation in HPV vaccine requirements and outcomes across US states and territories, and to examine the implementation and impact of the new school-entry HPV vaccine policy in PR. As part of the aims, we conducted qualitative semi-structured interviews with 46 key informants (KI), who were stakeholders from: the PR Department of Health (DOH), school systems (either public or private), healthcare organizations island-wide, as well as community organizations, which collaborates with the school-entry requirement. Additionally, participants against this policy were part of the total sample interviewed (n = 8). However, the analysis of this group will be reported separately because their interview responses mainly focused on themes outside of our selected framework, CFIR. Participants were selected by recommendation from the study’s Community Advisory Board members and from other KIs. Participants had to meet the following inclusion criteria: older than 21 years old, work in an area related to education or health, collaborated with the HPV school-entry requirement in their healthcare setting or organization and agreed to audio-record the interview.

Data collection

Interviews were conducted from July 2018 to January 2020 in PR. Only one interview [date: 07/13/2018] was performed prior to the implementation of this school-entry requirement in August 2018. Participants were interviewed either face-to-face in the PR Comprehensive Cancer Center facilities, in their workplace, or by phone. Interviews were conducted in Spanish, were audio recorded, lasted from 60 to 90 minutes, and were led by one of four team members: VC, RS, DM, and a research assistant.

Conceptual framework

The CFIR consists of 39 constructs organized into five major domains that focus on different stages of the implementation.^27,32 For this study, the CFIR was used to develop the interview guides, data coding and analysis. The interview guides were similar across the four KIs organizations, although we adapted the questions to each group’s expertise area (i.e., health, education). The interview guide had relevant constructs selected under each of the domains. Examples of the five domains and constructs included: (1) Characteristics of individuals (knowledge, attitudes, and beliefs about HPV vaccination), (2) Outer setting (perceived needs for immunization services in the community) (3) Inner setting (access to knowledge and information about the implementation, staff resources), (4) Intervention characteristics (actual practices, advantages and disadvantages of this implementation), and (5) Process (leadership and commitment among implementers of HPV immunization) among others.

Interview analysis

Data coding

We conducted a series of steps for this qualitative analysis using CFIR constructs. First, we adapted and developed a new codebook (from the CFIR official website)³² with the framework constructs and additional constructs that emerged through the analysis. The research team discussed and modified the codebook as they coded the first interviews. All the interviews were digitally recorded and transcribed verbatim in Spanish, using qualitative data Atlas.ti (version 8.4.23.0) software package.³³

We coded the quotes based on the codebook constructs and definitions. Additional constructs relevant to the implementation were created and included as subthemes of the CFIR original constructs. These additional codes were evaluated independently of the CFIR constructs but within the same theme (e.g., new code created as a sub-construct: unawareness (independent construct) below parent’s needs and resources (CFIR construct).

Then, we selected the constructs most cited for this analysis (n = 47). The data and analysis trustworthiness was verified independently and discussed by three investigators (VC, RS, and DM). The quotes selected for inclusion in the manuscript were translated to English after the analysis was completed.

Construct ratings

After coding the quotes (n = 1,398), we used a rating system adapted from the CFIR website to categorize the selected constructs.³² From this analysis, we excluded the constructs of recommendations and job description because they were primarily informative. The rating system was based on two concepts: valence and strength. Valence refers to the factors in the quote that have a positive influence, negative influence, or no influence (neutral) on the school-entry implementation. While strength refers to how strong the influence of the content of the quote is on the implementation, giving values from ‘0ʹ (no influence), ‘1ʹ (low influence), and ‘2ʹ (high influence). After rating each quote, we determined the most frequent rating value of the constructs stratified by type of organization, which we denominated as ‘rating’s score mode’ (Table 2).

Analysis and interpretation

After the assignment of ratings per construct, we summarized the results in a rating matrix, considering the pattern of ratings grouped by the organizations interviewed, which we denominated ‘variability of the ratings’ (Table 2). This matrix led the research group to visually identify patterns in the construct ratings between the four organizations interviewed to determine multilevel barriers and facilitators to the school-entry requirement. We compared the differences in rating scores by row for the variability analysis, considering each organization’s construct rating score. The following classification was used: ‘none’ when the scores of the construct ratings were the same for each organization; ‘low’ when the value was mixed but only in strength; and ‘high’ when both strength and valence were different. Those constructs that were categorized to have low or no variability that presented positive ratings were classified as ‘facilitators’ or ‘barriers’ (if presented negative ratings). Additionally, we considered the sum of the ratings by construct (row) to determine the strongest facilitators and barriers to be shown in the descriptive results. On the other hand, those constructs that had high variability between the organizations were categorized as ‘mixed’ (both facilitator and barrier).

Results

Characteristics of the participants

Table 1 presents the role of each of the KI interviewed. Most were from the school system (n = 15) followed by the Department of Health and healthcare providers (n = 9 for each group) and community organizations (n = 3). Within the school system, school nurses and directors were most interviewed (n = 5 for each group).

Table 1. Distribution of KI participants, regarding to stakeholder group and role within the organization (n = 38)

Stakeholder group	Role	n (%)
Department of Health
Program director	Senior administrative supervisor with knowledge or experience in immunization	3 (7.9)
Medical consultant	Professional with experience in advising about immunization	1 (2.6)
Promotion manager	Promotion of vaccination facts, data, and requirements in campaigns and social platforms	1 (2.6)
Program coordinator	In charge of vaccination data quality	2 (5.3)
Program epidemiologist	Assist in the data quality and develop statistics	1 (2.6)
Health educator	In charge of promoting HPV vaccine and other health topics in schools	1 (2.6)
Schools
School directors	Supervisors in charge of private or public schools	5 (13.2)
School nurses	Regional nurses in charge of a series of schools to monitoring vaccine data and vaccine school requirements, among other roles	6 (15.8)
Administrative assistant	Assistant of the school directors in charge of keep record and monitoring the HPV school-entry policy	4 (10.5)
Nurse supervisor	Administrative supervisors in the Central Offices of the Department of Education	1 (2.6)
Social worker	Assistant of the school director who assists in establish communication with parents regarding the completion of the vaccine requirements	1 (2.6)
Healthcare providers
Clinical nurses	Immunization nurses of the Federally Qualified Health Centers (FQHCs)	2 (5.3)
Pediatrician	A physician from private insurance who have the license to vaccinate	1 (2.6)
Gynecologist	Physicians and investigators specialized in HPV	2 (5.3)
Dentist	Academic and investigator in oropharyngeal cancer and HPV related cancers	1 (2.6)
Health Educators	Health providers from the FQHCS	2 (5.3)
Vaccination Program director	Physician in charge of the vaccination clinic of a private Hospital	1 (2.6)
Community organizations
Advocacy organizations	Representatives from two organizations in PR in favor of the school-entry policy committed to advocating for the immunization of the entire population through education, to prevent future diseases.	2 (5.3)
Religious	Priest from the Catholic denomination who collaborates with the HPV school-entry policy in PR.	1 (2.6)
Total		38 (100)

Table 2. Ratings assigned to CFIR construct by organization

Construct	n quotes	Rating’s score mode by organization^a				Total rating score^b	Variability^c	Category^d
		DOH	Schools	Health-care providers	CBOs
(I) Intervention characteristics
Implementation source (internal)	2	2	-	-	-	2	Not	Facilitator
Evidence strength & quality	56	1	1	1	1	4	Not	Facilitator
Adverse effects*	7	−1	−2	0	-	−3	Low	Barrier
Relative advantage	36	1	2	2	2	7	Low	Facilitator
Adaptability	15	1	-	0	2	3	Low	Facilitator
Trialability	3	−1	−1	-	-	−2	Not	Barrier
Complexity	40	−2	−2	−2	-	−6	Not	Barrier
Vaccine complexity completion*	22	0	−1	−1	−1	−3	Low	Barrier
Design quality and packaging	19	−2	−2	−2	0	−6	Low	Barrier
Cost	19	−2	−2	−1	−2	−7	Low	Barrier
(II) Outer setting
Parent needs & resources	52	2	2	2	2	8	Not	Facilitator
Unawareness*	21	1	2	1	1	5	Low	Facilitator
Parental/patient autonomy*	10	0	1	0	0	1	Low	Facilitator
HPV hesitancy*	43	1	1	0	0	2	Low	Facilitator
Vaccine distrust*	24	2	1	1	0	4	Low	Facilitator
Cosmopolitanism	43	1	0	−2	2	1	High	Mixed
External policy & incentives	40	2	2	1	2	7	Low	Facilitator
Negative influence of social media	23	−2	−2	−2	−1	−7	Low	Barrier
Excessive government interference	6	−1	-	1	−1	−1	High	Mixed
(III) Inner setting
Network & communications	37	1	2	1	-	4	Low	Facilitator
Culture	4	−2	-	-	1	−1	High	Mixed
Implementation climate	14	2	2	1	-	5	Low	Facilitator
Tension for change	14	−1	2	2	-	3	High	Mixed
Relative priority	37	1	2	1	2	6	Low	Facilitator
Organizational incentives & rewards	4	2	-	−2	-	0	High	Mixed
Goals and feedback	15	1	-	−1	2	2	High	Mixed
Learning climate	1	-	2	-	-	2	Not	Facilitator
Readiness for implementation	72	2	1	0	2	5	Low	Facilitator
DOH not enough	22	−1	−1	−1	−1	−4	Not	Barrier
Leadership engagement	27	2	1	2	1	6	Low	Facilitator
Available resources	54	2	−1	1	−2	0	High	Mixed
Available resources staff*	61	−2	−1	−2	−2	−7	Low	Barrier
Access to knowledge and information	75	−1	−1	−1	−1	−4	Not	Barrier
(IV) Characteristics of individuals
Knowledge & beliefs about the innovation	62	2	1	2	1	6	Low	Facilitator
Views & knowledge about exemptions	93	−1	0	−2	−1	−4	Low	Barrier
(V) Process
Planning	30	2	−2	2	2	4	High	Mixed
Engaging	4	2	−1	-	2	3	High	Mixed
Formally appointed internal implementation leaders	4	-	2	1	1	4	Low	Facilitator
Champions	10	1	1	2	2	6	Low	Facilitator
External change agents	33	1	1	1	1	4	Not	Facilitator
External change agents-capacity*	14	2	1	1	2	6	Low	Facilitator
External change agents-materials*	13	1	2	1	2	6	Low	Facilitator
Key stakeholders	37	1	1	1	1	4	Not	Facilitator
Implementation participants	41	1	2	2	1	6	Low	Facilitator
Communication to community*	63	2	1	1	1	5	Low	Facilitator
Implementation differences between schools	30	−1	−2	−1	−1	−5	Low	Barrier
Reflecting and evaluating	46	0	0	0	−1	−1	Low	Barrier

^aScore mode: after assigning a rating for each quote, we determined the most frequent rating for each construct stratified by organizations.

^bTotal rating score: the sum of rating’s score modes by constructs.

^cVariability: the evaluation of the pattern of ratings by each construct (high, low, not).

^dCategory: Classification of facilitator, barrier, or mixed based on the pattern of the construct’s rating scores.

*Additional new codes created as sub-constructs of the CFIR original framework. These new codes were coded and evaluated independently (e.g., mutually exclusive quotes assigned to each of the quotes indicated in this table).

Distribution of CFIR construct ratings by organization

We identified a total of 32 constructs aligned with CFIR constructs (out of 39 original CFIR constructs) and created an additional 15 constructs for a total of 47 constructs evaluated for valence and strength (Table 2).

Ratings and variability

Nine (n = 9) constructs have high variability between the organizations interviewed, which implies that organizations have different views and influences about the HPV school-entry requirement. These constructs were: communication between organizations, government interference, goals and feedback, culture, tension for change, organizational incentives and rewards, goals and feedback, available resources, planning, and engaging. Most constructs (38 out of 47) have low (n = 28) or no (n = 10) variability between the construct ratings across the organizations, which implies homogeneity of opinions regarding HPV school-entry requirement. From these, a total of 25 constructs positively influenced the implementation and were classified as facilitators. Most of the facilitators identified were on Process (n = 8) and Outer Setting (n = 6) domains. On the other hand, thirteen (n = 13) constructs negatively influenced implementation in all the organizations and were classified as barriers. Of these barriers, six (n = 6) were on the domains of Intervention characteristics, and three (n = 3) were on the Inner Setting domain.

Details of the findings are described below, categorized by facilitators and barriers to the HPV school-entry policy.

Facilitators

The strongest facilitators reflected across all the organizations interviewed (DOH, healthcare providers, schools, and organizations in favor of the implementation) were identified on the domains of Outer Setting and Intervention characteristics. Implementers were aware of the parents’ needs and resources to meet with the school-entry policy. Additionally, initiatives for school-entry policies were mentioned as a facilitator for the external policies and incentives construct. On the other hand, the fact that this implementation is required gives a relative advantage over different strategies. VOCES (Coalition of Vaccination of Puerto Rico) was the external change agent most mentioned who positively influenced providing education to stakeholders. School and clinical nurses documented messages for engaging parents and adolescents (implementation participants).

Outer setting

The facilitator with the highest score was the parent’s needs and resources construct, part of the Outer Setting domain. Implementers demonstrated their awareness and knowledge of parent’s needs and resources to accept and understand the school-entry policy. Among those needs and resources that impede a successful vaccination are HPV vaccine hesitancy, parents’ unawareness, vaccine distrust, and the violation of parental/patient autonomy.

Parent’s Needs Resources (total rating score = 8). This was the construct with a higher score under the Outer Setting. The implementers demonstrated their awareness and knowledge of the factors that impede that parent vaccinated their youths.

“They [parents] know that must vaccinate them [adolescents] specifically with the HPV vaccine, but they don’t know specifically what the vaccine is [for]. They have heard that it [HPV vaccine] ‘is bad, is bad’ … that is why they are hesitant because what they hear of the vaccine is negative” -School director

External Policy and Incentives (total rating score = 7). Initiatives from organizations to spread the HPV school entry requirement were documented, such as campaigns, meetings, development, and support of policies and programs.

“Well, nowadays the Department [of Education] has a ‘family engagement’ program, a program directed to parents … which is an area of opportunity to give orientations to leading parents so they can help us promote vaccines” –School nurses’ supervisor

Intervention characteristics

Relative advantage (total rating score = 7). The most common benefit of this implementation is that it is required, increasing the probability that every adolescent of 11–12 years be vaccinated and, hence, increase HPV vaccination rates compared to other strategies.

“I think it [the school-entry requirement] would be very successful because it is the only way we get parents to take their children to vaccinate” – Health provider

Process

External change agents (total rating score = 6). 62% of quotes mentioned VOCES as an external organization that helped provide education to the community, gave training to school staff, and provided materials with information to assist the Department of Health. Another organization highly mentioned was the pharmaceutical company Merck, which aids in providing educational material in schools and clinics.

“An external entity helps to break the existing barrier, and I can attest to that, that this has been the role that VOCES has done so this [HPV school-entry requirement] can be achieved” -Community organization

Implementation participants (total rating score = 6). Under this construct, school and clinical nurses expressed how they engaged parents to vaccinate their youths. Strategies included clear enforcement messages to educate the community about the benefits of vaccination and the use of personal anecdotes to convince them.

“I always tell them [parents]: My children have all the vaccines, and all three have the HPV vaccine. My children are all protected, and I don’t have the stress that they will be infected [by HPV] because I protected them. And many parents that I already have friendships with see when I tell them: ‘look at my children, they have it’. There, they say, ‘she is not going to expose her children to something that she does not believe in’ because she is a health worker. There they [parents] change their minds” –School nurse

Champions (total rating score = 6). Another facilitator was the engagement and dedication from implementers with the school-entry requirement in PR. Most of the interviewees mentioned the Secretary of Health and school nurses as champions committed to this implementation.

“Since he [the Secretary of Health] entered, in 2017, he committed that was going to maintain that [HPV school-entry mandate] … that vaccination was important, that he was going to maintain that, and … that was the commitment.”–Department of Health vaccination program

Inner setting

Relative priority (total rating score = 6). This construct documented the importance and the focus of having this requirement in the adolescents’ population to prevent future malignancies.

“Super necessary for me [the school-entry requirement], there you have the vast majority of the population, of the target population that you want, where you can control this epidemic at an epidemiological level. So, for me, it’s perfect.” -Health provider

Characteristics of individuals

Knowledge and Beliefs about the implementation (total rating score = 6). This construct showed the implementers’ beliefs, and mostly it was optimistic about the vaccine and the requirement.

“I believe in the vaccine, in the benefits of the vaccine and more to protect young people … it is a vaccine that certainly one is not going to see the results now, but in the long term, I hope so, that we can see the reduction in the rates of cervical cancer and other types of cancers that are associated” -Department of Health educator

Other constructs

Other constructs that had a total rating score of 5 points were: perception of lack of awareness in parents, implementation climate, readiness for implementation, and community communication. Regarding the Outer Setting domain, key informants documented the misinformation from parents about the HPV school-entry requirement or the HPV vaccine (lack of awareness). Under the Inner Setting domain, the construct of implementation climate includes the receptivity of the key informants regarding the HPV school entry requirement, and the construct of readiness for implementation included the organizational commitment to implement the requirement. Additionally, under the Process domain, the construct of communication to the community documented information delivered to the community related to the HPV vaccine or its implementation in schools.

Barriers

The strongest barriers to the school entry requirement in PR most mentioned by organizations were in the domains of Implementation Characteristics, Inner and Outer Setting. The cost for private providers to administer the vaccine, the negative influence of social media about HPV vaccine affected the acceptance of parents, and the lack of school nurses as available staff resources for the school entry requirement were the most substantial barriers mentioned that affected the implementation.

Implementation characteristics

Barriers mentioned in the Implementation Characteristics domain reflect issues with the cost of the vaccines to private providers, the program’s complexity, and the content of the material presented to the community.

Cost (total rating score = −7). The main barrier was the cost of the vaccine for private providers. Participants mentioned barriers related to the vaccine coverage by the health insurance and documented that it is below the vaccine’s cost.

“When a [private] physician is going to administer vaccines, he has to buy the vaccine and store it in the office in a cold chain … I have to pay for the storage, for the administration, and the nurse. … the time of the person who enters the vaccine [data] to the PRIR [immunization registry] … There are administrative expenses for storage and of administering the vaccine that the health insurance plans, if they pay, pay a pittance. In Puerto Rico, I understand that they pay between four and six dollars. In the United States, they pay 25. USD The other case is that there are times I buy a vaccine that costs me 100, USD and the insurance reimburses me 90 USD … ” –Pediatrician

Complexity (total rating score = −7). One of the difficulties expressed by key informants was guiding and disseminating the school-entry requirement to the community (health providers, school, parents). Participants also mentioned the process of accepting religious exemptions by schools (in instances, religious exemptions are only accepted if ‘included in what the law says’). Many implementers expressed that the rules to determine if the exemption is valid are not clear and have difficulties dealing with it.

Another theme mentioned by the participants was the delayed implementation process due to Hurricane Maria’s arrival on September 20, 2017: The DOH, health providers, and the vaccination clinics suffered disruption and vaccine losses island-wide.³⁴

“We had some challenging years because we had many stumbling blocks. The hurricane [Maria] has left us in bad shape. Not only us here, the Department of Health, the entire island: the doctors, the vaccination centers, VFC, all of those suffered a lot of damage.” - Department of Health

Design quality and packaging (total rating score = −6). The promotion materials used to inform about the vaccine initially affected parents and the community’s acceptance because they focused on it being a vaccine that prevents a sexually transmitted infection.

“How they have marketed it [HPV vaccine] in the beginning saying and stimulating that it is a vaccine that prevents a sexually transmitted infection is one of the things that, perhaps most affected the acceptance of the vaccine worldwide” –Department of Health

Inner setting

Barriers mentioned in the Inner Setting reflected the need for training to implementers and the lack of materials and human resources to attend the school entry mandate’s demands.

Available staff resources (total rating score = −7). Key informants discussed the need for more school nurses’ staff to successfully implement this requirement. In many schools, the administrative assistant or the principal must assume this duty, representing an additional responsibility and work overload. Regional nurses of public schools also take this role. Participants discuss the excess of work this task means, as one nurse can have up to 20 schools assigned, and the time to dedicate to each school is limited.

“The difficulty we have is in the limited amount of [school] nurse resources that we have. Now we are blessed this year because we have [school nurses] … But if we don’t have this resource, it limits us because the roster of my nurses, we only have 33” –School nurses’ supervisor

Outer setting

Negative influence of social media (total rating score = −7). Key informants expressed how the misleading information about the HPV vaccine posted on social media affected the acceptance of HPV by parents.

“Regarding HPV vaccine, I think they [parents] have received more negative information through the social media than information from professionals who have adequately guided them for this vaccine, which is why many parents are hesitant to this vaccine” –School nurse

Other constructs

Related to the Process domain, the construct of implementation differences between public and private schools had a total score of −5 points. This construct documented the organizational differences in engaging the same school entry policy between public and private schools.

Discussion and conclusion

During the first two years of implementing the HPV school entry policy in PR, this study described the factors that facilitated or impeded the successful implementation of this policy in this population and the magnitude of the influence of those factors on the implementation. Our study shows high variability across organizations interviewed concerning nine constructs related to Process, Outer Setting and Inner Setting. On the other hand, our study’s main findings revealed 25 CFIR constructs that showed low or no variability between the organizations interviewed. These constructs facilitated or represented a challenge to the school-entry requirement implementation. One of the most influential facilitators was constructs of the Outer Setting domain, as the participants acknowledge the parents’ needs in understanding the policy. Other facilitators include their strengths in using personal experience and testimony to promote the vaccine and the policy (implementation participants) and the influence of champions (community coalitions) that provide health providers training and capacity. Participants’ beliefs about the HPV vaccine and relative advantage of having this requirement were part of the highlights that facilitated the implementation. On the other hand, the cost of the HPV vaccine and the physicians’ low reimbursement for services related to the vaccine administration, storage, and operational costs were among the primary barriers mentioned by key informants. Moreover, some of the strongest barriers documented include the complexity in informing the school-entry requirement to the community, the lack of available school nurses, and the negative influence of social media about the HPV vaccines’ promotion.

Despite the consensus in most of the constructs across organizations, nine constructs reflected high variability in opinions about the implementation between the organizations interviewed. These results demonstrate two things: first, healthcare provider organization participants had different views when compared with all other organizations interviewed. For example, the construct of cosmopolitanism, which refers to the degree of networking between organizations, had positive ratings for the Department of Health and community-based organizations. On the other hand, it obtained the lowest rating score (−2) for the healthcare provider organization. Participants from healthcare organizations also had different rating valence regarding the construct of government interference of mandatory vaccination against parents’ will, compared with the organizations from the Department of Health and CBOs, who had negative ratings. Moreover, in the constructs of organizational incentives & rewards, and goals & feedback, the healthcare providers had negative rating scores compared with the other organizations’ positive scores.

On the other hand, participants from school organization had negative ratings compared with all other organizations interviewed in the constructs of planning (the degree to which tasks for implementing the school-entry requirement are developed) and engaging (involving appropriate individuals in the implementation and use of the school-entry requirement through combined strategies). We speculate that these findings of differences in opinions from healthcare providers and school organizations might be due to the lack of information delivered, particularly in the context of the policy’s initial implementation (August 2018). This launch was 11 months after Hurricane Marias’ arrival, which triggered other public health problems, leaving aside the priority of the HPV vaccine, which could affect the dissemination of information.^34–36 These interruptions caused doubts in the school personnel particularly and might affect the Department of Health’s communication with providers. Although trainings with physicians, journalists, and school staff were conducted prior to implementation, divergences are still observed; hence, more efforts are still needed, as healthcare providers (clinical nurses and physicians)³⁷ and school nurses³⁸ are the first lines of contact with parents regarding HPV vaccination.

The construct classified as the strongest facilitator was parents’ needs and resources, which describes the extent to which vaccine recipients’ needs are accurately known and prioritized and the barriers to reaching them. Participants demonstrated their knowledge and awareness about the factors that promote vaccine hesitancy in parents. Some of the factors documented by key informants interviewed and mentioned in previous articles by Hispanic and African American parent-level focus groups included: unawareness³⁹ and misinformation about the benefits⁴⁰ of the HPV vaccine, stigma attributed to an STI-vaccine related,^30,39 vaccine distrust due to long-term effects,^39–41 effectiveness,⁴¹ among others. Additionally, this facilitator is consistent with what has been found in a previous study using CFIR;²⁹ however, the reasons identified were not the same. As many of these factors have been recognized in focus groups of Hispanic parents, stakeholders must understand the gaps that impede parents’ vaccinating their sons and daughters. Especially in those in the first line of contact (school and clinical nurses, pediatricians) documented that they know their population, know their barriers to up-to-date vaccination. These qualities make them a strong source of influence to parents.^40,42

Three barriers obtained the same highest total rating score. These were available staff resources, negative influence of social media, and cost. Limited staff resources was a barrier mentioned by key informants to discuss mainly the need of school nurses’ staff to implement this requirement. Previous studies documented the lack of staff resources allocated to the school-entry vaccination initiatives, leading to work overload by nurses.^38,42 Other studies using CFIR identified time as a barrier to staff resources that affected productivity.³⁰ In Puerto Rico, the economic crisis, which was documented to start in 2014, could affect hiring government personnel in the Department of Health and Education Department prior, which can aid, provide support, and monitor the correct implementation of this school entry policy August 2018. The second barrier of social media’s negative influence refers to posts from social media platforms attacking the HPV or the school-entry requirement. Participants were concerned about the information available through the internet (unreliable sources), which are often the most common source accessible and influential to this population of young parents. These findings were consistent with other studies using CFIR expressing frustration about messages of opposition from religious groups linking it to sexuality for young girls,³¹ the sources of information and influences such as the internet.⁴⁰ Another study found that the vaccine coverage is lower than the expected in US states where negative opinions about HPV vaccine are popular through mainstream media sources, taking into account sociodemographic factors.⁴³ On the other hand, a systematic review found evidence that engagement with HPV related social media content was associated with improved awareness and knowledge of HPV vaccination.⁴⁴

Additionally, the barrier of cost was mentioned mainly by private physicians (from the healthcare organizations) and focused on the cost associated with the coverage of the vaccine by health insurance and all the costs associated with the maintenance and administration of the vaccine. Another factor mentioned was the high cost of the vaccine for those who were uninsured, although in Puerto Rico health insurance is required to cover this vaccine and almost all are eligible for the Vaccine for Children program. The cost of the vaccine as a barrier to patients was also mentioned in previous studies using CFIR.^31,40 Other studies also mentioned the barrier of the cost associated with the infrastructure, storage, and administration.^22,30

This study is the first to evaluate the implementation of the HPV vaccine school-entry requirement in Puerto Rico with interviews with stakeholders from different organizations using CFIR. We found the CFIR framework a useful and practical tool to construct the interview guide and analyze the data. Because it allows combining qualitative (data coding) and quantitative (construct rating score) to better measure the implementation and make conclusions, we considered this a notable strength. Moreover, all the domains were evaluated in this analysis. We interviewed four organization types that include implementers from multiple sectors involved directly in the HPV vaccine school entry requirement; this helped us see different and broad perspectives of the implementation. This analysis provides new information about challenges that emerged in the implementation and factors that support the process.

In terms of limitations, although we interviewed four organization types related to the implementation, we acknowledge that there were organizations that were not included in the study sample, such as policymakers from the government of Puerto Rico (both in the senate and the House of Representatives), which were part of the initial implementation process. Additionally, the number of key informants is unbalanced, as the school organizations have a higher number of participants recruited (n = 15) compared with the lowest number of community organizations (n = 3), which might influence the number of quotes explored and analyzed.

In conclusion, our study has shown both positive and negative factors that affected the implementation of the HPV school entry requirement in PR. Because school-entry requirements are concentrated primarily in school settings, it is crucial to maintain trained personnel with access to vaccine-related information, as they are the first line of contact with parents.⁴⁵ This study highlights the importance of providing training and tools for school and health stakeholders to boost parents’ capacity to engage parents who are unaware or hesitant to vaccinate. On the other hand, health insurance should be regulated to compensate private physicians at a level that will cover their actual costs. Additionally, negative influences of social media should be offset with the development of effective strategies to inform about the vaccine and school entry requirements, via social media information from reliable and official sources. Most barriers presented in this study can be modifiable with the implementation of educational programs/training across organizations, considering the first line of response to parents of this school entry policy. These findings generate recommendations for our stakeholders to influence policy implementation improvements and can be used to guide other implementers that want to replicate this implementation in other jurisdictions or countries.

Abbreviations

HPV	=	Human Papillomavirus
PR	=	Puerto Rico
CFIR	=	Consolidated Framework for Implementation Research
ACIP	=	Advisory Committee on Immunization Practices
US	=	United States
DC	=	District of Columbia
HPV-PIVac	=	Human Papillomavirus Policy Implementation Vaccination study
KI	=	Key informants
DOH	=	Department of Health
CBOs	=	Community based organizations
VOCES	=	Puerto Rico Coalition of Vaccination

Disclosure of potential conflicts of interest

VCL has received compensation from Merck and Co., Inc. for consultancy in June 2020. APO reported has received personal fees from serving as a consultant for Merck (October 2019) outside the submitted work. The other authors have declared that they have no competing interest.

Ethical approval

Ethical approval for the Implementation of School Entry Policies for Human Papillomavirus Vaccination (HPV-PIVac) was secured from the University of Puerto Rico Medical Sciences Campus- Institutional Review Board (Protocol #A8060218).

Acknowledgments

We want to thank to Dr. María E. Fernández, Manuel E. Rivera-Encarnación, Vilnery Rivera-Figueroa; Glizette O. Arroyo-Morales, Luis O. León Martinez, Michelle Ramirez-López, Kevin J. Colón-Rivera, and Andrea N. Suárez-Delgado to their support and collaboration.

Data availability statement

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request. The URL with the databases and supplementary material is https://drive.google.com/drive/folders/14-puy7vXHjVdDiByXLUuRzpm2T7SpN1f?usp=sharing.

Additional information

Funding

This work was supported by National Cancer Institute (NCI), grant R01CA232743 (“Implementation of School-Entry Policies for Human Papillomavirus Vaccination”) within University of Puerto Rico Comprehensive Cancer Center. The National Cancer Institute had no role in the study design; collection, analysis, and interpretation of the data; writing of the manuscript; or decision to publish. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NCI.This publication was supported by the Comprehensive Cancer Center of the UPR (a public corporation of the Government of Puerto Rico created in virtue of Law 230 of August 26, 2004 as amended). The content is entirely the responsibility of the authors and does not necessarily represent the official views of the Comprehensive Cancer Center UPR.