ABSTRACT

Vaccination is an effective way to prevent infectious diseases. However, parental resistance to childhood immunizations has recently been a growing trend due to various factors. This systematic review and meta-analysis study focused on randomized controlled trials to investigate interventions for preventing parental vaccine refusal/hesitancy. This study was conducted according to the preferred reporting items for systematic review and Meta-analysis Protocols. PubMed, Science Direct, Web of Science, MEDLINE, Cochrane Library, Ovid, CINAHL, and ProQuest databases were screened with no year restriction. Two researchers each used the Cochrane risk-of-bias tool to determine the quality of the trials. Data were synthesized using a meta-analysis package. The study population consisted of 3,302 articles. The sample consisted of 22 randomized controlled trials, which implemented the interventions of text message reminders, digital training in vaccination, training in physician communication, training in parental decisionmaking, web-based activities, perinatal training in vaccination, immunization reminder and follow-up bracelets, and mother-daughter training. The results showed that slightly more parents agreed to get their children vaccinated after the interventions. The trials were moderately heterogeneous due to differences in sample size, country, and year. This paper investigated the effect of interventions on parental vaccine refusal/hesitancy and provided level-A evidence to suggest such interventions can be used to change parents’ vaccination comprehension and decisions.

Keywords:

Vaccine hesitancy, vaccine refusal, child, systematic review, meta-analysis

Introduction

Vaccination has proven to be a cost-effective method for reducing mortality and morbidity rates caused by infectious diseases. However, recently, global childhood vaccination rates have declined. Sociocultural, environmental, economic, and individual factors play a role in parental vaccine refusal/hesitancy (1-3).

Recent outbreaks of vaccine-preventable diseases, such as measles and pertussis, demonstrate the dangers of low vaccination rates (4,5). For instance, Europe observed an eightfold increase in measles cases in 2018 (41,000 cases) compared to 2016 (5,000 cases; (6), and 87% of cases are attributed to those who refuse to be vaccinated (3,7). Diphtheria-tetanus-pertussis immunization rates have dropped to 92% and 91% in Europe and the United States, respectively. Therefore, the World Health Organization (WHO) identified vaccine refusal/hesitancy as one of the 10 threats to global health in 2019 (8,9). Moreover, the anti-vaccination trend has extended beyond childhood immunizations to campaigns against COVID-19 vaccines, a virus that has claimed millions of lives worldwide (10).

Many researchers have devoted their efforts to understanding and preventing parental refusal of childhood vaccinations (11-18). Specifically, some researchers have investigated interventions for the prevention of parental vaccine refusal/hesitancy. For example, some researchers sent vaccine-hesitant parents short texts a few days or more before a vaccination program to persuade them to get their children vaccinated (19-26). Other researchers implemented training-focused interventions, such as digital training in vaccination (27), training in physician communication (28), web-based interventions (29,30), and perinatal training in vaccination (31).

However, no comprehensive meta-analysis research exists assessing the overall effectiveness of interventions tailored to the prevention of parental vaccine refusal/hesitancy and identification of standards to inform further research on this topic. We aimed to fill this gap in the literature by determining the best intervention(s) against vaccine refusal/hesitancy so that healthcare professionals can use them to increase immunization rates.

Aims

This systematic review and meta-analysis focused on randomized controlled trials to determine the effectiveness of interventions for increasing parental vaccine acceptance rates.

The research questions were as follows:

-What kind of interventions are implemented to prevent parental vaccine refusal/hesitancy?

-Which interventions can help persuade parents to get their children vaccinated?

-How effective are interventions for increasing vaccine acceptance rates?

Methods

Search Strategy and Outcomes

The study followed the Preferred Reporting Items for Systematic Review and Meta-analysis Protocols (PRISMA-P). Figure 1 shows the flow diagram. This study satisfied all items recommended by the PRISMA-P checklist (32) and was recorded in the International Prospective Register of Systematic Reviews (PROSPERO; code: CRD42020157785).

Two researchers each searched the Science Direct, Web of Science, Springer Link, Ovid, CINAHL, PubMed, Cochrane Library, and ProQuest databases (August 2020) with no year restriction based on the four elements of PICOS (patient/population, intervention, comparison, outcome and study design). They screened the databases using the keywords of “vaccine,” “refusal,” “hesitancy,” “parent,” “intervention,” and “clinical trials” alone or in combination. They also used keywords/MeSH terms within articles from search alerts in PubMed. They compared the articles they accessed independently. They discussed the articles and reached a consensus on which articles to include in the study.

The literature review based on the search strategy yielded 3,302 articles, which were transferred to the EndNote 9 package. First, duplicates were removed (n=228), after which irrelevant articles by title or abstract were removed (n=2,899). The remaining 175 articles were reviewed based on the inclusion and exclusion criteria (Table 1), and 143 were removed. The full texts of the remaining 32 articles were reviewed and 10 were removed. One of the remaining 22 articles was regarded as two different articles because it involved two different interventions. Therefore, some sections and tables include 23 articles.

Risk of Bias and Quality Appraisal

The researchers each used the Cochrane risk-of-bias assessment tool (RoB 2) to evaluate the risk of bias of the articles based on the following bias domains (33) (Figure 2). GRADE was used to grading quality of evidence and the strength of recommendation of studies.

Data Extraction and Analysis

The researchers each extracted the data based on (a) author, year, country, (b) participant characteristics, (c) intervention, (d) intervention details, (e) control group, (f) outcome measures, (g) results, and (h) certainty of the evidence (GRADE).

The data were analyzed using the Comprehensive Meta-analysis (version 2.0). The main outcome was pre- and postintervention vaccine acceptance rates. The data were prepared for meta-analysis using Microsoft Excel. The effect size (ES) was calculated using Borenstein and Hedges (34), which were rated as small (d=0.20), medium (d=0.50), and large effects (d=0.80) between the intervention and control groups (35). Cochran’s Q test and I² statistics were employed to determine heterogeneity (36). A significance level of less than <.05 indicated heterogeneity for Cochran’s Q test. I² of 25%, 50%, and 75% were classified as low, moderate, and high heterogeneity, respectively (37). The random-effects model was used to show the average difference and to calculate the weighted average difference (38). Publication bias was determined using Egger’s test, Begg and Mazumdar’s test for rank correlation, and Duval and Tweedie’s Trim-and-Fill test (39,40). Analog ANOVA and meta-regression analysis were performed to identify the cause of heterogeneity (41-44).

Results

Research Characteristics

The sample consisted of 22 randomized controlled trials, with a sample size of 57 to 15,786 (n=55,138). The trials were conducted between 2011 and 2020 in the United States (n=14), the United Kingdom (n=2), China (n=1), Australia (n=1), the Netherlands (n=1), Japan (n=1), Pakistan (n=1), and Guatemala (n=1).

Most trials implemented the intervention of sending parents short reminder messages (SMS alerts) before scheduled immunization visits for their children (19-26,45-49). The other trials provided digital training in vaccination [DTV; n=1; (27)], training in physician communication [TPC; n=1; (28)], training in parental decision-making [TPDM; n=1; (50)], perinatal training in vaccination [PTV; n=1; (31)], web-based interventions [WBI; n=2; (29,30], office-based educational brochure [OBEB; n=1; (51)], immunization reminder and follow-up bracelets [IRFB; n=1; (52)], and mother-daughter training [MDT; n=1; (53)].

The details of the interventions differed among trials. The SMS trials (19-26,45-49) involved sending parents short reminder messages just before scheduled immunization visits, or 4 days, 1 week, 4 weeks, 1 month, 2 months, or 4 months before the visits. The frequency of the SMS alerts depended on the dose of vaccination. The DTV trial (27) used only reinforcement messages depending on the parents’ views or administered training in vaccination within the scope of a program. The TPC trial (28) administered physicians 45-minute training and examined its effect on parental vaccine hesitancy. The TPDM trial (50) provided training to parents eight times for 2 months. One WBI trial (30) provided online training (via email) only once, while the other WBI trial (29) administered intervention six times with intervals of 2, 4, and 6 weeks. The PTV trial (31) presented two sessions (10 minutes per session) of training to parents in the perinatal period. The OBEB trial (51) used brochures laid out in the waiting room for visitors to take. The 1-year IRFB trial provided vaccinated children with bracelets that had a symbol for each vaccination (52). The MDT trial held 11 dinner meetings for mother-daughter training (53). The control groups in all trials received standard procedures (Table 2).

Risk of Bias

In the domain of randomization process, 16 trials were at low risk of bias (19,23,25-31,45,47-51,53), five trials raised some concerns (21,22,24,46,52), and the remaining one was at high risk (20). All trials were at low risk of bias for deviations from intended interventions and missing outcome data. Five trials raised some concerns in the domain of bias in the measurement of the outcome (20,24,47,51,53), while the others were at low risk. Eight trials were at low risk in the domain of bias in the selection of the reported result, while the others raised some concerns. Six trials were at low risk in the domain of general bias, 16 raised some concerns, and one was at high risk (Figure 2).

Study Outcomes

This paper focused on 22 trials to investigate the effect of interventions on vaccine acceptance rates. Meta-analysis results showed that interventions for vaccine acceptance were effective [p<0.001, Hedge’s g=0.10, 95% CI (0.06, 0.14)]. The Cochran’s Q test results showed a moderate level of heterogeneity (Q=61.26, p<.001, I²=64.09%; Figure 3). The moderating variables of intervention type (Q=11.114, df=9, p=0.268) and vaccine type (Q=4.467, df=4, p=0.347) had no significant effect on the effect size between subgroups. The moderating variable of country significantly affected the heterogeneity (Q=32.891, df=7, p<0.001). The meta-regression analysis showed that sample size significantly affected the ES (Q=61.26, df=22, p<0.001), suggesting a negative correlation between sample size and ES. The variable of year also significantly affected the heterogeneity (Q=61.26, df=22, p<0.001).

Publication bias was determined using a funnel plot, Begg and Mazumdar’s test, and Egger’s test. The funnel plot presents the standard error for the Y-axis and the ES for the X-axis (Figure 4). The Begg and Mazumdar’s tau with continuity (tau=0.34, Z=2.32 p=0.02) and Egger’s regression test results indicated publication bias. Therefore, Duval and Tweedie’s Trim-and-Fill test was used to calculate publication bias again. The results indicated that nine studies be added to avoid publication bias, which would result in a general ES of 0.04 [95% CI (0.04, 0.11); Figure 4].

Discussion

This systematic review and meta-analysis focused on 22 randomized controlled trials to examine the effectiveness of interventions for promoting vaccine acceptance rates. The results indicated that interventions helped reduce vaccine refusal/hesitancy and increase vaccine acceptance rates, albeit insignificantly (Hedge’s g=0.10). We believe the difference in sample size, intervention, and outcome across the trials affected the ES of the interventions. Twelve trials showed interventions resulted in increased vaccine acceptance rates (19,20,23,25-27,30,31,48-51), while the remaining 11 found no significant difference in vaccine acceptance rates between the intervention and control groups (21-24,28,29,45-47,52,53).

The results showed that the trials had moderate heterogeneity, which was affected by sample size, intervention, and outcome (54,55). The ANOVA and meta-regression analysis results showed that the difference in sample size, year, and country caused the heterogeneity. The trials with large samples had insignificant ESs, adversely affecting both overall ES and heterogeneity. We suggest researchers employ the right statistical methods and consider missing data and errors when determining the right number of participants for accurate results in terms of ES and effectiveness. Differences in the country also affected the heterogeneity of the trials. Hofstede et al. (56) attributed cultural differences to vertical-collectivism and horizontal-individualism. They argued that horizontally individualistic cultures are the source of heterogeneity because they have loose social ties, resulting in high heterogeneity.

Most trials implemented the intervention of SMS alerts. Their results indicate that texting reminder messages to parents before their scheduled visits for their children’s vaccination reduces their vaccine refusal/hesitancy (19,23,26,48,49). However, texting messages sent too early before the scheduled visits or texting multiple messages did not affect vaccine acceptance rates. Face-to-face training, decision-making processes (30,31,50,51), and tech-based interventions (27) were most effective, increasing the overall ES. Given recent advances in technology and low health literacy rates, we believe it is of paramount importance to provide tech-based training in vaccination to the public. However, the training should be short and occasional given that trials of long (28) and frequent training (29,52,53) found no difference in vaccine acceptance rates between the intervention and control groups. If an intervention should be administered more than once, it should be implemented in multiple sessions over an extended period rather than short intervals (50).

The trials had a low or uncertain risk of bias, which was affected by differences in the number of samples and research content. We can state that the trials addressed the risk of methodological thinking and bias and met standards for randomizing and blinding. Most trials generated CONSORT flow charts, which should be used for accurate randomization, low risk of bias, and high research quality (57).

Although the trials reported different results, researchers and practitioners should take interventions as a whole and set standards regarding the frequency, duration, and application of interventions tailored to reduce vaccine refusal/hesitancy and increase vaccine acceptance rates. Such interventions should be short and occasional with tech-based interventions administered face to face right before visits for vaccinations. Researchers should be more careful when determining the right sample size. Future studies should consider these suggestions when it comes to designing interventions tailored to reduce parental vaccine refusal/hesitancy rates.

Strengths and Limitations

This research has several strengths. First, it is a comprehensive systematic review and meta-analysis on vaccine hesitancy in children. Second, a protocol based on Cochrane principles was followed. Another strength was that databases were scrupulously scanned and the Cochrane tools manual was followed to minimize bias. Further, the review was conducted by two authors independent of screenings and bias controls.

Concerning limitations, gray literature, such as conference papers, research, and committee reports, was not included. We wanted to keep the evidence to the highest level. Gray literature can be added to future studies, considering the risks of lowering the level of evidence.

Conclusion

This systematic review and meta-analysis focused on 22 randomized controlled trials to determine the effect of interventions on the reduction of parental vaccine refusal/hesitancy rates. The results show that interventions can be used to persuade parents to get their children vaccinated. Our results will be of considerable value to physicians, nurses, academics, primary healthcare staff, and the public. Overall, the results indicate that interventions should be short tech-based interventions administered face to face. We believe that the results can help primary healthcare staff develop new interventions to reduce parental vaccine refusal/hesitancy rates and provide high-quality care. The results introduce Level-A evidence that can pave the way for further research on the topic. We recommend that more researchers undertake more detailed and specific national and international trials concerning the effectiveness of interventions for promoting parental vaccine acceptance rates.

Ethics

Conflict of Interest: No conflict of interest was declared by the authors.

Financial Disclosure: The authors declared that this study received no financial support.

References

Larson HJ, Jarrett C, Schulz WS, Chaudhuri M, Zhou Y, Dube E, et al. Measuring vaccine hesitancy: The development of a survey tool. Vaccine 2015;33:4165-75.

Siddiqui M, Salmon DA, Omer SB. Epidemiology of vaccine hesitancy in the United States. Hum Vaccin Immunother 2013;9:2643-8.

Düzgün MV, Dalgıç Aİ. Can vaccine rejection, an increasing danger to public health, be prevented? The Journal of Current Pediatrics 2019;17:424-34.

Coombes R. Europe steps up action against vaccine hesitancy as measles outbreaks continue. BMJ 2017;359:j4803.

Gowda C, Dempsey AF. The rise (and fall?) of parental vaccine hesitancy. Hum Vaccin Immunother 2013;9:1755-62.

Holt E. 41 000 measles cases in Europe since the beginning of 2018. Lancet 2018;392:724.

Bozkurt HB. An overview of vaccine rejection and review of literature. Kafkas Journal of Medical Sciences 2018;8:71-6.

Thangaraju P, Venkatesan S. WHO Ten threats to global health in 2019: antimicrobial resistance. Cukurova Medical Journal 2019;44:1150-1.

World Health Organization. Ten threats to global health in 2019. Available from: https://www.who.int/news-room/spotlight/ten-threats-to-global-health-in-2019

Dror AA, Eisenbach N, Taiber S, Morozov NG, Mizrachi M, Zigron A, et al. Vaccine hesitancy: the next challenge in the fight against COVID-19. Eur J Epidemiol 2020;35:775-9.

Dube E, Vivion M, MacDonald NE. Vaccine hesitancy, vaccine refusal and the anti-vaccine movement: influence, impact and implications. Expert Rev Vaccines 2015;14:99-117.

Yaqub O, Castle-Clarke S, Sevdalis N, Chataway J. Attitudes to vaccination: a critical review. Soc Sci Med 2014;112:1-11.

Brown KF, Kroll JS, Hudson MJ, Ramsay M, Green J, Long SJ, et al. Factors underlying parental decisions about combination childhood vaccinations including MMR: a systematic review. Vaccine 2010;28:4235-48.

Favin M, Steinglass R, Fields R, Banerjee K, Sawhney M. Why children are not vaccinated: a review of the grey literature. Int Health 2012;4:229-38.

Rainey JJ, Watkins M, Ryman TK, Sandhu P, Bo A, Banerjee K. Reasons related to non-vaccination and under-vaccination of children in low and middle income countries: findings from a systematic review of the published literature, 1999-2009. Vaccine 2011;29:8215-21.

MacDonald NE, Hesitancy SWGoV. Vaccine hesitancy: Definition, scope and determinants. Vaccine 2015;33:4161-4.

Salmon DA, Dudley MZ, Glanz JM, Omer SB. Vaccine hesitancy: Causes, consequences, and a call to action. Vaccine 2015;33 Suppl 4:D66-71.

Luyten J, Bruyneel L, van Hoek AJ. Assessing vaccine hesitancy in the UK population using a generalized vaccine hesitancy survey instrument. Vaccine 2019;37:2494-501.

Domek GJ, Contreras-Roldan IL, Bull S, O’Leary ST, Ventura GAB, Bronsert V, et al. Text message reminders to improve infant immunization in Guatemala: A randomized clinical trial. Vaccine 2019;37:6192-200.

Henrikson NB, Zhu W, Baba L, Nguyen M, Berthoud H, Gundersen G, et al. Outreach and Reminders to Improve Human Papillomavirus Vaccination in an Integrated Primary Care System. Clin Pediatr (Phila) 2018;57:1523-31.

Niederhauser V, Johnson M, Tavakoli AS. Vaccines4Kids: Assessing the impact of text message reminders on immunization rates in infants. Vaccine 2015;33:2984-9.

Porter RM, Amin AB, Bednarczyk RA, Omer SB. Cancer-salient messaging for Human Papillomavirus vaccine uptake: A randomized controlled trial. Vaccine 2018;36:2494-500.

Rand CM, Vincelli P, Goldstein NP, Blumkin A, Szilagyi PG. Effects of Phone and Text Message Reminders on Completion of the Human Papillomavirus Vaccine Series. J Adolesc Health 2017;60:113-9.

Szilagyi PG, Schaffer S, Rand CM, Goldstein NPN, Younge M, Mendoza M, et al. Text Message Reminders for Child Influenza Vaccination in the Setting of School-Located Influenza Vaccination: A Randomized Clinical Trial. Clin Pediatr (Phila) 2019;58:428-36.

Tull F, Borg K, Knott C, Beasley M, Halliday J, Faulkner N, et al. Short Message Service Reminders to Parents for Increasing Adolescent Human Papillomavirus Vaccination Rates in a Secondary School Vaccine Program: A Randomized Control Trial. J Adolesc Health 2019;65:116-23.

Yeung KHT, Tarrant M, Chan KCC, Tam WH, Nelson EAS. Increasing influenza vaccine uptake in children: A randomised controlled trial. Vaccine 2018;36:5524-35.

Dixon BE, Zimet GD, Xiao S, Tu W, Lindsay B, Church A, et al. An Educational Intervention to Improve HPV Vaccination: A Cluster Randomized Trial. Pediatrics 2019;143:e20181457.

Henrikson NB, Opel DJ, Grothaus L, Nelson J, Scrol A, Dunn J, et al. Physician Communication Training and Parental Vaccine Hesitancy: A Randomized Trial. Pediatrics 2015;136:70-9.

Pot M, Paulussen TG, Ruiter RA, Eekhout I, de Melker HE, Spoelstra MEA, et al. Effectiveness of a Web-Based Tailored Intervention With Virtual Assistants Promoting the Acceptability of HPV Vaccination Among Mothers of Invited Girls: Randomized Controlled Trial. J Med Internet Res 2017;19:e312.

Shourie S, Jackson C, Cheater FM, Bekker HL, Edlin R, Tubeuf S, et al. A cluster randomised controlled trial of a web based decision aid to support parents’ decisions about their child’s Measles Mumps and Rubella (MMR) vaccination. Vaccine 2013;31:6003-10.

Saitoh A, Nagata S, Saitoh A, Tsukahara Y, Vaida F, Sonobe T, et al. Perinatal immunization education improves immunization rates and knowledge: a randomized controlled trial. Prev Med 2013;56:398-405.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffman TC, Mulrow CD, et al. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ 2021;372:n71.

Higgins J, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al. Cochrane handbook for systematic reviews of interventions (version 6). 2020: The Cochrane Collaboration; 2019.

Borenstein M, Hedges LV, Higgins JP, Rothstein HR. Introduction to meta-analysis. 2nd ed. John Wiley & Sons; 2011.

Cohen J. Statistical power analysis for the behavioral sciences-second edition. 12 Lawrence Erlbaum Associates Inc. Vol 13. Hillsdale, New Jersey: 1988.

Higgins JP, Thompson SG. Quantifying heterogeneity in a meta-analysis. Statistics in Medicine 2002;21:1539-58.

Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ 2003;327:557-60.

Borenstein M, Hedges LV, Higgins JP, Rothstein HR. A basic introduction to fixed-effect and random-effects models for meta-analysis. Research Synthesis Methods 2010;1:97-111.

Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ 1997;315:629-34.

Duval S, Tweedie R. Trim and fill: a simple funnel-plot–based method of testing and adjusting for publication bias in meta-analysis. Biometrics 2000;56:455-63.

Hedges LV, Olkin I. Statistical methods for meta-analysis. Academic press; 2014.

Hedges LV. Fitting categorical models to effect sizes from a series of experiments. Journal of Educational Statistics 1982;7:119-37.

Bigby M. Understanding and evaluating systematic reviews and meta-analyses. Indian Journal of Dermatology 2014;59:134.

Cornell JE, Liao JM, Stack CB, Mulrow CD. Annals Understanding Clinical Research: Evaluating the Meaning of a Summary Estimate in a Meta-analysis. Annals of Internal Medicine 2017;167:275-7.

Ahlers-Schmidt CR, Chesser AK, Nguyen T, Brannon J, Hart TA, Williams K, et al. Feasibility of a randomized controlled trial to evaluate Text Reminders for Immunization Compliance in Kids (TRICKs). Vaccine 2012;30:5305-9.

Hofstetter AM, DuRivage N, Vargas CY, Camargo S, Vawdrey DK, Fisher A, et al. Text message reminders for timely routine MMR vaccination: A randomized controlled trial. Vaccine 2015;33:5741-6.

Hofstetter AM, Vargas CY, Camargo S, Holleran BAS, Vawdrey DK, Kharbanda EO, et al. Impacting delayed pediatric influenza vaccination: a randomized controlled trial of text message reminders. Am J Prev Med 2015;48:392-401.

Rand CM, Brill H, Albertin C, Humiston SG, Schaffer S, Shone LP, et al. Effectiveness of centralized text message reminders on human papillomavirus immunization coverage for publicly insured adolescents. J Adolesc Health 2015;56:S17-20.

Stockwell MS, Kharbanda EO, Martinez RA, Vargas CY, Vawdrey DK, Camargo S. Effect of a text messaging intervention on influenza vaccination in an urban, low-income pediatric and adolescent population: a randomized controlled trial. JAMA 2012;307:1702-8.

Jackson C, Cheater FM, Harrison W, Peacock R, Bekker H, West R, et al. Randomised cluster trial to support informed parental decision-making for the MMR vaccine. BMC Public Health 2011;11:475.

Scott VP, Opel DJ, Reifler J, Rikin S, Pethe K, Barrett A, et al. Office-Based Educational Handout for Influenza Vaccination: A Randomized Controlled Trial. Pediatrics 2019;144:e20182580.

Siddiqi DA, Ali RF, Munir M, Shah MT, Khan AJ, Chandir S. Effect of vaccine reminder and tracker bracelets on routine childhood immunization coverage and timeliness in urban Pakistan (2017-18): a randomized controlled trial. BMC Public Health 2020;20:1086.

Winer RL, Gonzales AA, Noonan CJ, Buchwald DS. A Cluster-Randomized Trial to Evaluate a Mother-Daughter Dyadic Educational Intervention for Increasing HPV Vaccination Coverage in American Indian Girls. J Community Health 2016;41:274-81.

Gartlehner G, West SL, Mansfield AJ, Poole C, Tant E, Lux LJ, et al. Clinical heterogeneity in systematic reviews and health technology assessments: Synthesis of guidance documents and the literature. Int J Technol Assess Health Care 2012;28:36-43.

Zhou Y, Dendukuri N. Statistics for quantifying heterogeneity in univariate and bivariate meta-analyses of binary data: The case of meta-analyses of diagnostic accuracy. Stat Med 2014;33:2701-17.

Hofstede G, Hofstede GJ, Minkov M. Cultures and organizations: Software of the mind. Vol 2. Mcgraw-hill; New York: 2005.

Grant S, Mayo-Wilson E, Montgomery P, Macdonald G, Michie S, Hopewell S, et al. CONSORT-SPI 2018 Explanation and Elaboration: Guidance for reporting social and psychological intervention trials. Trials 2018;19:406.

The Effect of Interventions on the Prevention of Parental Vaccine Refusal and Hesitancy: A Systematic Review and Meta-analysis of Randomized Controlled Trials