A Mixed-Methods Evaluation of The ¡Vamos Por Más! Parenting Program Implementation in Chile

Abstract

Introduction:

Substance use is a significant global concern. Strengthening parenting in families with adolescents has been shown to reduce substance use initiation. The ¡Vamos por Más! (¡VxM!) program is a positive-parenting program developed in Chile to improve family relations and reduce adolescent substance use that combines in-person school workshops, multimedia messaging and personalized support.

Objective

This manuscript reports a mixed-methods evaluation of the pilot implementation of the ¡VxM! program utilizing the Consolidated Framework for Implementation Research and Proctor’s taxonomy for process outcomes.

Methods

An explanatory sequential design was used. Quantitative methods evaluated program use, acceptability, appropriateness, and fidelity, and were followed by qualitative focus groups (FGs) to assess the implementation process and understand these outcomes. Thirteen FGs stratified by school and role, including school leaders (n = 7), program facilitators (n = 19), participants (n = 36), and researchers (n = 4), were conducted.

Results

The program was implemented in three schools, reaching 253 families with in-person workshops (40.5% of potential participants), 257 parents who viewed on average 72.1% of sent multimedia messages, and 2 families who used the personalized support (0.3%). Overall, the program was viewed as acceptable and appropriate by participants and implementers due to the high quality of program materials, targeted content, and activities. Implementation differed by schools. Key implementation factors were the outer context, inner school setting, and implementation processes.

Conclusion

This comprehensive evaluation, including both intervention implementers and participants, identified implementation facilitators, barriers, and outcomes. Future ¡VxM! implementations should alter program components of schools with lower engagement to improve program implementation and outcomes.

Introduction

Substance use is a global concern, due to both individual and population level burdens of the disease.  Substance use patterns and subsequent burden differ greatly by region and country (1, 2). Despite these regional variations, adolescence is the peak period of substance use initiation globally (3) and has been identified as a top priority by numerous multinational organizations (4-6). Addressing adolescent substance use requires targeted and effective prevention strategies (7, 8). 

Chilean adolescents have a high prevalence of substance use and quantities of consumption. According to the last National Student Survey, of students consuming alcohol in the last month, 61.7% had drank more than 5 drinks at one time on at least one occasion (9). Additionally, the average age of cigarette smoking initiation was 13.8 years and by 12^th grade 8.8% of students reported daily smoking (9) .

Interventions which target adolescents and their families have proven successful at preventing substance use. This age is an appropriate time for intervention as childhood characteristics predict health outcomes later in life, including substance use (10). Additionally, once substance use is initiated, patterns of use are often difficult to modify (11). Parenting programs targeting families with adolescents have been successful in decreasing risky behaviors, including substance use (10, 12-14). Implementation of parenting programs in schools have shown particularly high success rates (12).

Even though parenting programs promise improved public health outcomes, translatability of these programs outside of their initial implementation has had mixed outcomes in Latin America and Chile (15-18). A recent literature review reported that many countries have implemented adaptations of the Strengthening Families Program 10-14 (Familias Fuertes), without transferring the observed outcomes of the original studies (18). Thus, it recommended either adapting interventions validated elsewhere or building the evidence for locally developed interventions (18).

The ¡Vamos por Más! (¡VxM!) program was created as a local, school-based family-centered alternative to prevent substance use, focusing on the needs and preferences of Chilean families and available community resources (19). This manuscript reports the ¡VxM! pilot implementation and lessons learned for the implementation of parenting programs in Chile and Latin America.

Description of the ¡Vamos por Más! Program 

Details of the ¡VxM! program development are presented in (19). The ¡VxM! program is a school-based parenting intervention designed to be implemented over two years during 5^th and 6^th grade. The program was developed with the input of school staff, parents, students, and experts in intervention design and parenting adolescents. The program has five components:
1) School partnership aligning intervention objectives and focus with the practices and available resources of the schools; 2) In-person workshops, one per semester, for students and their guardians. Each workshop is independent, guided by a facilitator using standardized videos and materials. Workshops addressed Autonomy and self-esteem, and Parent-child communication in 5^th grade and Positive discipline and Monitoring and support in 6^th grade. Workshops begin with separate activities for parents and youth, followed by a family activity to reflect and practice learned skills; 3) Virtual engagement with biweekly media messages reinforcing program content; 4) Family support to personalize introduction of new skills; and 5) External supervision to support program implementation. 

This study reports the implementation of the ¡VxM! program as the fourth phase of its iterative intervention design process. As proposed by the Medical Research Council’s (MRC) framework for developing complex interventions (20), this evaluation will inform future versions of the program.

Evaluation Frameworks

This study was guided by Consolidated Framework for Implementation Research (CFIR) and Proctor´s taxonomy for implementation outcomes. The CFIR synthesizes several existing theories and frameworks, providing an overarching structure for intervention implementation and evaluation (21).  It has five major domains: (1) Intervention characteristics, (2) Outer setting, (3) Inner setting, (4) Characteristics of implementers, and (5) Implementation processes, with subsequent subconstructs. To date, few parenting programs have utilized the CFIR in evaluations (22)

In addition to utilizing an evaluation framework, it is essential to identify key implementation metrics. Proctor et al. (2010) developed a taxonomy of eight distinct outcomes -acceptability, adoption, appropriateness, feasibility, fidelity, implementation cost, penetration, and sustainability- which were used this evaluation.

Methods

Study Design

The ¡VxM! program was piloted using a cluster randomized trial. This manuscript focuses on the schools that implemented the program as part of the intervention group using an explanatory sequential mixed-method study design (23). In this design, quantitative implementation measurements were followed by qualitative focus group exploration to illuminate initial findings. Data analysis and integration were done through merging, in which quantitative and qualitative analyses were done separately, and then combined at the interpretation phase for comparison (24). This design provides convergences of findings and allows for insights into implementation factors not possible with a single method. All procedures were approved by the XXXX Review Board. 

Setting

The ¡VxM! program was implemented at schools in Santiago, Chile. Chilean primary education is offered through both private and public institutions. Within private institutions many use government subsidies. All participating schools were subsidized private schools with varying sizes and socioeconomic vulnerability (Table 1). 

¡Vamos por Más! Implementation 

The ¡VxM! program was implemented during the 2019 school year in three schools randomized to the intervention group. The program was offered to all families with a student in 5^th or 6^th grade. School leadership appointed a school coordinator to lead the implementation. Teachers served as workshops facilitators and were required to have a professional degree (either as a teacher, psychologist, nurse, or social worker) and experience leading groups. All school staff attended three meetings. The first introduced the program, and the other two were 90- minute trainings prior to the in-person workshops, which were divided by school grade (e.g., teachers facilitating the 5^th grade workshops only received training in those activities). At the end of the trainings, facilitators completed an evaluation. 

In-person workshops were implemented during scheduled 2-hour group parent-teacher conferences, on weekday evenings. Some schools provided snacks for the participants, although traditionally uncommon. Media messages were sent through WhatsApp® to all participants who provided their phone number at enrollment. Messages were sent at best reception times (25). At the workshops and in messages, participants were invited to reach out to the school coordinator or the program´s email for additional support. The research team provided technical and content support as needed. 

Implementation Evaluation

All parents or guardians (hereafter referred to as “parents”) were invited to participate in the program evaluation. Parents read and signed an informed consent form for themselves and their child. Youth read and signed an assent form. Families were considered enrolled within the evaluation if consent was provided and a baseline survey was completed. Regardless of enrollment status, all families could participate in program activities and workshop attendance was collected via sign in sheets. Participants and facilitators completed anonymous evaluations after workshops. Message reception rates were tracked by research staff and classified as not received, received but not viewed, or received and viewed.

Focus groups (FGs) organized by participant type were conducted with parents, teachers and school leadership after implementing the program. All FGs were divided by school, and parents were further divided by grade. An additional FGs was completed with the research team. Semi-structured, audio recorded FGs were conducted by researchers with experience in qualitative methods. FGs lasted 1.5–2 hours and were conducted in Spanish. Open-ended questions explored specific areas of the CFIR and varied slightly between FGs.  

Participants

Table 1 summarizes participant’s demographic information. Schools had 625 students in 5^th and 6^th grade, of which 308 were female (49.4%) and 317 were in 5^th grade (50.7%). 

Program Evaluation Participants 

Overall, 38.4% of eligible students and 22.1% of eligible parents completed a baseline survey (n=240 and 138, respectively). Over half of the students were female and exactly half were in 5^th grade. Comparing students who were eligible and those completing the baseline survey, there were no statistically significant differences in gender or grade (p=.066 and .850, respectively). 

Of enrolled parents, the majority identified as female and were born in Chile. Their average age was 39.9 years (SD 7.1). About half of the total parents were married and the majority worked outside their home. There were statistically significant differences between schools in rates of parents born in Chile and employment (p<.001 and .003, respectively).

Twenty-one facilitators completed a demographic survey (58.3%). School staff from School #2 (S#2) were less likely to complete their training evaluation forms (p=.032). Approximately half worked as teachers and most identified as female. 

Focus Group Participants

In total, thirteen FGs were conducted at the end of program implementation. Participants included school leadership (n=3 FGs, 7 participants), teachers (n=3 FGs, 19 participants), parents (n=6 FGs, 36 participants), and researchers (n=1 FG, 4 participants). 

Outcomes

Proctor’s intervention implementation outcomes (2010) were evaluated through program participation, surveys and FGs. Penetration was measured using participant engagement. Acceptability was operationalized as overall satisfaction with the program; Appropriateness as perceived utility and interest in the program; and Fidelity as the degree to which facilitators implemented the curriculum without adaptations. These data were collected in the quantitative evaluation phase. Feasibility, implementation cost, and sustainability were evaluated in FGs and operationalized as the degree to which the program could be implemented, the cost of reproducing the program in the future, and what would be needed to sustain the program. 

Data Analysis

Quantitative Data Analysis

Demographics, perceived satisfaction, utility and fidelity were summarized using descriptive statistics. Message view rates were calculated based on the total number of messages sent for each school. Data from continuous variables were compared between schools using one-way analysis of variance using Bonferroni correction to assess between-group differences. Categorical data were compared between schools using Chi² or Fisher’s exact tests. No covariate adjustment was conducted. Resulting p-values <.05 were considered statistically significant.

Qualitative Data Analysis 

FGs were transcribed verbatim and analyzed with NVivo (QSR International, v.12) using an inductive-deductive process following Content Analysis procedures (26). Guided by the CFIR domains, one analyst coded the data identifying emerging categories and sub-categories. The coding was revised by a second researcher, and discrepancies were discussed to achieve full agreement. A third researcher conducted an external review validating the coding. Data were further organized according to the participant´s role and school. Representative quotes by CFIR domain were translated and are presented as a word document entitled Additional File 1.   

Mixed-methods Integration

Quantitative and qualitative data were integrated through connecting and building (24). The connecting strategy was used to guide sampling, as FG participants participated in the intervention. The building strategy was used to guide data collection, as FG questions were based on the quantitative findings. Findings are reported using a continuous narrative approach (24), where mixed methods are reported separately in the same manuscript.  

Results

The ¡VxM! program was only fully implemented in School #3 (S#3). Schools #1 (S#1) and S#2 only offered half of the workshops because of nation-wide curfews and school closures due to unprecedented protests during October 2019-February 2020 (27). Although pending activities were postponed for the following year (2020), the COVID-19 pandemic changed school priorities, further prohibiting program implementation. Despite experiencing similar national and global constraints, the three schools achieved different implementation outcomes. 

Program Penetration

Participation results are presented in Table 2. Workshops were attended by 40.5% of total eligible families (n=253) and by more than half of consented families (53.7%, n=73). Overall, S#2 had the lowest participation rates (p<.001). The virtual component was utilized by most parents providing contact information. The average rate of messages viewed was 72.1% across schools, and most parents who received messages viewed greater than 80% of sent messages (68.5%, n=176). No significant differences were found when comparing the message viewing rates between schools (p = .152). The family support component had minimal use across all schools (0.3%, n=2). 

Program Acceptability, Appropriateness, and Fidelity 

School staff unanimously reported high satisfaction with the facilitator trainings (Table 1). Most parents, students, and teachers reported being satisfied with the workshops and perceived them as useful and interesting (Table 2).

Although all parents reported high satisfaction with the program, parents from S#3 had significantly fewer positive reactions (p<.001). Satisfaction with the first semester workshops were evaluated similarly between schools (p=.877). However, the workshops implemented during the second semester produced significantly lower satisfaction within S#3, the only school that implemented them (93.4% vs 68.1%, p=.027). Students across schools differed in their perceived utility ratings (p=.035), where students at S#3 had significantly higher perceived utility than students at S#1 (p=.010). Perceived utility ratings were similar between the first and second semester workshops within S#3 (p=.934).

Regarding workshop implementation fidelity, 92.0% of school staff reported teaching all workshop content and 76% reported making adaptions (Table 2). These rates did not differ between schools.

Program Implementation Perceptions

Perceptions are organized by CFIR domain and data source (informant and school). Overall, all participants were very satisfied with the intervention. However, the implementation contexts and processes differed between sites.

Perceptions about ¡Vamos por Más!

Table 3 summarizes perceptions about the program. Overall, all stakeholders, across schools, valued the program contents and the excellence in how the intervention was assembled, bundled, and presented through workshops and media messages. Having the program developed by an external team provided an increased sense of legitimacy. The use of high-quality audiovisual materials was advantageous because parents identified with the parenting scenarios which they had not experienced with other programs and the use of standardized materials could facilitate future dissemination. Participants emphasized the importance and usefulness of integrating students, as youth reported positive experiences and enjoyed learning with their peers and then sharing with their parents. In addition, the program was perceived as highly adaptable across all groups, because it could be embedded into local school culture and integrated within changing national and global contexts. Most teachers and school leadership reported that the workshop implementation did not disrupt the school schedules and mentioned that the facilitator role was compatible with the role of a teacher, but the intervention content was complex for many teachers. Leadership teams noted that they would simplify materials in future implementations to reduce costs, such as using black-and-white printing, and emailing or printing the diploma instead of the magnetic diploma given to families. The family support component was perceived as unnecessary as some parents reported there was no need, preferred other sources of support, email was perceived as impersonal, and some did not know about it. 

Perceptions about the context, implementers and processes

Perceptions on how the program was implemented across schools, organized by CFIR domains, are presented in Table 4. 

Outer setting. All participants reported that the program addressed an existing need to strengthen parenting. At the school with the highest vulnerability and lowest attendance (S#2), concerns of the school building’s safety limited attendance. Protests produced school closings, limiting access to the program across schools. This also changed family needs and dynamics with some families reporting a greater need for the program and while others less.

Inner setting. The program was perceived as embedding well within schools. School culture was considered the most important factor for adoption, and was difficult to modify. School staff viewed implementing the program as a professional development opportunity but did not perceive it as a core responsibility. Demands on teachers’ time and their concerns of not having skills to work with parents or address questions limited feasibility. Participants from the two schools with higher attendance reported a culture of supporting parents and feeling support from school leadership.

Characteristics of individuals. School staff valued implementing the program as part of their commitment to the families they serve, particularly because they knew the families’ contexts. Facilitators were viewed as having the knowledge and skills to correctly implement the program. Parents at the school with lowest engagement (S#2) reported receiving different levels of information about the program including who could participate, the different components of the program, and the date of when meetings were held. 

Processes. The facilitator trainings were perceived as high quality. Program adoption was limited by disinformation about the program and technology dependent challenges (e.g., audio issues when playing the videos during the workshops, etc.). At the schools with higher workshop attendance, school staff and leadership reported modifying their outreach plans to increase participation. Participants were concerned that evaluation data may not reflect program impact as questionnaires were not answered thoughtfully, the evaluation process changed (e.g. virtual FGs), and that it was too soon to evaluate behavioral changes. 

Recommendations

Several suggestions to improve the program and its implementation arose from the FGs (Table 5). To strengthen relationships with school staff, it was recommended to dedicate a semester to foster relationships before formal implementation. Implementers at the school with the lowest engagement and highest vulnerability (S#2) suggested including updates during school staff meetings and having the school director attend facilitator trainings to improve integration into the school. Teachers across all schools proposed reconsidering the ideal workshop facilitator, particularly someone with more time, energy, and facilitation skills. Logistical recommendations emphasized improving information dissemination, technical support during workshop delivery, and reconsidering the ideal time for in-person workshops. 

Discussion

This study reports the pilot implementation of the ¡VxM! program to strengthen family relations and prevent adolescent substance use. Overall, the program had greater penetration, acceptability, and appropriateness than other parenting programs in Chile (15, 16) and Latin America (18). Programs with high quality intervention design that engage multiple stakeholders maximize the program fit with local needs, beliefs, and context, producing high acceptability and use (28, 29). Although parents had different backgrounds across schools, they all emphasized the high relatability of program content, as compared to other Chilean parenting programs which have been considered “reductionist” and lacking cultural contextualization (15). Guided by the CFIR, stakeholders positively evaluated the intervention and its implementation, identifying recommendations for future delivery.

Pilot implementation is an essential part of the design of complex behavioral interventions (20). Each implementation round is an opportunity to learn what works and what doesn´t, allowing for further intervention refinement. For example, the family support component produced minimal interest and use, despite its initial support at the intervention design phase (19). Additionally, workshops were delivered during parent-teacher conferences to minimize participation barriers but participants recommended re-thinking this strategy. These findings emphasize the use of pilot implementation as a key phase of behavioral intervention design. If implementing internationally developed interventions, a formal cultural adaptation process, including program piloting, is needed to ensure proper implementation and outcomes (18, 30).

Effective implementation evaluation should be guided by established frameworks (21). This manuscript is one of few publications using the CFIR and Proctor’s taxonomy that evaluates the implementation of a parenting program (22, 31). This theory-driven approach allowed for a detailed examination of key implementation factors, increasing our understanding of parenting intervention delivery. Practitioners and researchers should consider the fit of the proposed intervention with potential users, implementation contexts, implementer characteristics, and processes involved. All these elements are essential to outcomes (21).

Even though the ¡VxM! intervention was highly valued, its adoption was limited in the school with higher student vulnerability (S#2). The ¡VxM! program was not initially created to target this population because it was believed that the program alone would not prevent adolescent substance use due to the high impact of external factors beyond the scope of the program (19). In this pilot we observed that implementation was also limited in this context. Although delivery of school-based parenting programs is feasible (32) and parents from S#2 reported concerns for their children’s substance use, it is essential to consider the intervention implementation contexts (e.g., socio-political context, site safety, available resources, competing demands, local culture, participant situations, etc.). This was further emphasized when parents from S#3 reported lower intervention acceptability after attending 2nd semester workshops implemented under constraints at the national level. This finding also highlights the need for local intervention development and adaptations as needed.

As expected, the program was a better fit for schools with lower vulnerability (S#1, S#3), who obtained higher in-person workshop participation. We believe that the School Partnership component of ¡VxM! established a positive school-wide perception about the program. This ownership enabled implementers to make calculated changes to participant outreach strategies and delivery, creating positive familiar learning environments and experiences. On the contrary, when a strong partnership was not achieved and local culture was not integrated (e.g., S#2), teacher motivation was lower and implementation harder, resulting in decreased penetration. As seen in other programs, impacting local culture is closely tied to intervention adoption, acceptability, feasibility, and use (14, 33, 34), and therefore strong partnerships with implementing sites are essential for successful delivery.

Facilitators reported high fidelity and made few changes. High rates of fidelity have been associated with increased attendance and positive parenting in other program evaluations (33, 35). In both program outreach and workshop delivery changes likely reflected skilled facilitators responding to unmet needs and using personal knowledge to enhance implementation, as certain adaptations have indicated success (36–38). Though most facilitators viewed the program as part of their commitment to serve families, some did not perceive parent training as the school’s responsibility or having the skills to deliver the program. If the implementation timeline had not been truncated, facilitator confidence may have increased overtime (15). Because implementers are essential for successful program implementation, future implementations of parenting programs in Latin America need to pay attention to facilitator´s knowledge, skills, and disposition towards the intervention.

The high participation rate of the virtual component across schools was key for content delivery over time. This reflects that school context has less impact on virtual delivery compared to in-person delivery. As recommended by participants, online delivery should be explored further to better address both individual and contextual participation barriers, particularly after the Coronavirus Disease 19 Pandemic (39).

Findings reported in this manuscript are limited by the pilot nature of the evaluation, as it was only implemented in three schools, engaging a small number of implementers, not capturing all contexts of potential implementation locations. Despite this, program refinement will continue with future implementations. The shortened timeline of the intervention also leaves gaps in predicting the sustainability of the program. As participation in school-based parenting programs often changes over time (28, 33, 40), continuity matters for long-term success. This manuscript also has some noteworthy strengths. First, we conducted the implementation evaluation guided by the CFIR and Proctor´s taxonomy for implementation outcomes. Thus, findings from this study address a gap of implementation research in positive parenting and school-based programs. Second, we implemented a robust mixed methods evaluation including several informants and different types of data, noting convergence in quantitative and qualitative reports. We also conducted site-specific implementation evaluations. This thorough evaluation process enabled us to obtain deep learning about successes and challenges for the implementation of parenting programs in schools.

Conclusion

This evaluation identified numerous factors affecting the quality of the ¡VxM! program pilot implementation, highlighting the importance of the outer and inner contexts where innovations are implemented. These findings are essential for the implementation of parenting programs in new settings and contribute to the larger implementation evaluation literature.

Abbreviations

Declarations

Ethics approval and consent to participate: All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Informed consent was obtained from all individual participants included in the study. All procedures approved by the XXX Ethics Committee, Reference Number XXX.

Consent for publication: Not applicable. 

Availability of data and materials: The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Competing interests: The authors declare that they have no competing interests.

Funding: This research was supported by Fondo Nacional de Desarrollo Científico y Tecnológico (FONDECYT 11170834); The University of Rochester International Medicine Research Fellowship and the University of Rochester David Hamilton Smith Pediatric Research Fellowship

Authors' contributions: JKVM collected and analyzed qualitative and quantitative data and was a major contributor in writing the manuscript. GD collected and analyzed qualitative and quantitative data. IM contributed to program implementation, collection of qualitative data and content expertise. CB contributed to program implementation, collection of qualitative data and content expertise.  DGH contributed to program implementation and content expertise, analyzed qualitative and quantitative data, and was a major contributor to writing the manuscript. All authors read and approved the final manuscript.

References

1. Peacock A, Leung J, Larney S, Colledge S, Hickman M, Rehm J, et al. Global statistics on alcohol, tobacco and illicit drug use: 2017 status report. Addiction. 2018;113(10):1905–26.
2. Degenhardt L, Charlson F, Ferrari A, Santomauro D, Erskine H, Mantilla-Herrara A, et al. The global burden of disease attributable to alcohol and drug use in 195 countries and territories, 1990–2016: a systematic analysis for the Global Burden of Disease Study 2016. The Lancet Psychiatry. 2018;5(12):987–1012.
3. Degenhardt L, Chiu W-T, Sampson N, Kessler RC, Anthony JC, Angermeyer M, et al. Toward a global view of alcohol, tobacco, cannabis, and cocaine use: findings from the WHO World Mental Health Surveys. PLoS Med. 2008;5(7):e141.
4. Organization WH. Global strategy to reduce the harmful use of alcohol. Geneva; 2010.
5. Organization WH. WHO Framework Convention on Tobacco Control. 2005.
6. Nations U. Outcome Document of the 2016 United Nations General Assembly Special Session on the World Drug Problem. 2016.
7. Organization WH. Global Status report on alcohol and health 2018. Geneva; 2018 2018.
8. Organization WH. WHO global report on trends in prevelance of tobacco use in 2000–2025 Geneva2019 [updated 2019. Third Edition:[Available from: https://apps.who.int/iris/bitstream/handle/10665/330221/9789240000032-eng.pdf?ua=1.
9. Servicio Nacional para la Prevención y Rehabilitación del Consumo de Drogas y Alcohol (SENDA). Décimo Segundo Estudio Nacional de Drogas en Población Escolar de Chile, 2017 8º Básico a 4º Medio. Santiago, Chile: Gobierno de Chile; 2018 2018.
10. Moffitt TE, Arseneault L, Belsky D, Dickson N, Hancox RJ, Harrington H, et al. A gradient of childhood self-control predicts health, wealth, and public safety. Proceedings of the Natlonal Academy of Sciences of the United States of America. 2011;108(7):2693-8.
11. Moss HB, Chen CM, Yi H-y. Early adolescent patterns of alcohol, cigarettes, and marijuana polysubstance use and young adult substance use outcomes in a nationally representative sample. Drug Alcohol Depend. 2014;136:51–62.
12. Allen ML, Garcia-Huidobro D, Porta C, Curran D, Patel R, Miller J, et al. Effective Parenting Interventions to Reduce Youth Substance Use: A Systematic Review. Pediatrics. 2016;138(2).
13. Lachman JM, Cluver L, Ward CL, Hutchings J, Mlotshwa S, Wessels I, et al. Randomized controlled trial of a parenting program to reduce the risk of child maltreatment in South Africa. Child Abuse Neglect. 2017;72:338–51.
14. Sanders MR. Development, evaluation, and multinational dissemination of the Triple P-Positive Parenting Program. Ann Rev Clin Psychol. 2012;8:345–79.
15. Correa M, Muñoz V, Astrain C, Estrada YA, Sánchez M, Tapia MI A Qualitative Assessment of Implementing an Evidence-Based Intervention in Chile, from the Perspective of the Facilitators Implementing Familias Unidas. [Poster [online]]. Society for Prevention Research 26th Annual Meeting. [online] Society for Prevention Research; 2018.
16. Sánchez M, Astrain C, Pardo D, Correa ML, Estrada YA, Prado G, Velazquez MR. Pilots Studies of Familias Unidas in Chile [Conference Poster]. Society for Prevention Research 26th Annual Meeting. [online]: Society for Prevention Research 26th Annual Meeting. [online]; 2018.
17. Corea VML, Zubarew GT, Valenzuela MMT, Salas PF. Evaluación del programa "Familias fuertes: amor y límites" en familias con adolescentes de 10 a 14 años. Revista médica de Chile. 2012;140:726–31.
18. Mejía A, Bertello L, Gil J, Griffith J, López AI, Moreno M, et al. Evaluation of family skills training programs to prevent alcohol and drug use: a critical review of the field in Latin America. International Journal of Mental Health Addiction. 2020;18(2):482–99.
19. Garcia-Huidobro D, Ivan M, Burmeister C, Macchiavello C, Lendzion L, Bernard A. Development of a School-Based Parenting Intervention to Prevent Substance Use Among Chilean Adolescents [Poster]. Society for Prevention Research 27th Annual Meeting: Society for Prevention Research; 2019.
20. Campbell M, Fitzpatrick R, Haines A, Kinmonth AL, Sandercock P, Spiegelhalter D, et al. Framework for design and evaluation of complex interventions to improve health. BMJ. 2000;321(7262):694–6.
21. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implementation Science. 2009;4:50.
22. Kirk MA, Kelley C, Yankey N, Birken SA, Abadie B, Damschroder L. A systematic review of the use of the Consolidated Framework for Implementation Research. Implementation Science. 2016;11(1):72.
23. Creswell JW, Creswell JD. Research design: Qualitative, quantitative, and mixed methods approaches. Sage publications; 2017.
24. Fetters MD, Curry LA, Creswell JW. Achieving integration in mixed methods designs-principles and practices. Health Serv Res. 2013;48(6 Pt 2):2134–56.
25. Abroms LC, Whittaker R, Free C, Mendel Van Alstyne J, Schindler-Ruwisch JM. Developing and Pretesting a Text Messaging Program for Health Behavior Change: Recommended Steps. JMIR Mhealth Uhealth. 2015;3(4):e107.
26. Krippendorff K. Content analysis: An introduction to its methodology. Sage publications; 2018.
27. Londoño E. What You Need to Know About the Unrest in Chile. The New York Times. 2019 October 21, 2019.
28. Christian DL, Todd C, Rance J, Stratton G, Mackintosh KA, Rapport F, et al. Involving the headteacher in the development of school-based health interventions: A mixed-methods outcome and process evaluation using the RE-AIM framework. PLoS One. 2020;15(4):e0230745.
29. Smith JA, Baker-Henningham H, Brentani A, Mugweni R, Walker SP. Implementation of Reach Up early childhood parenting program: acceptability, appropriateness, and feasibility in Brazil and Zimbabwe. Annals of the New York Academy of Sciences. 2018;1419(1):120 – 40.
30. Parra-Cardona JR, Bybee D, Sullivan CM, Rodriguez MM, Dates B, Tams L, et al. Examining the impact of differential cultural adaptation with Latina/o immigrants exposed to adapted parent training interventions. J Consult Clin Psychol. 2017;85(1):58–71.
31. Baumann AA, Powell BJ, Kohl PL, Tabak RG, Penalba V, Proctor EE, et al. Cultural Adaptation and Implementation of Evidence-Based Parent-Training: A Systematic Review and Critique of Guiding Evidence. Child Youth Services Review. 2015;53:113–20.
32. Bettencourt AF, Gross D, Breitenstein S. Evaluating Implementation Fidelity of a School-Based Parenting Program for Low-Income Families. Journal of School of Nursing. 2019;35(5):325–36.
33. Álvarez M, Rodrigo MJ, Byrne S. What Implementation Components Predict Positive Outcomes in a Parenting Program? Research on Social Work Practice. 2016;28(2):173–87.
34. Mytton J, Ingram J, Manns S, Thomas J. Facilitators and barriers to engagement in parenting programs: a qualitative systematic review. Health Education Behavior. 2014;41(2):127–37.
35. Breitenstein SM, Fogg L, Garvey C, Hill C, Resnick B, Gross D. Measuring implementation fidelity in a community-based parenting intervention. Nurs Res. 2010;59(3):158–65.
36. Stern SB, Alaggia R, Watson K, Morton TR. Implementing an evidence-based parenting program with adherence in the real world of community practice. Research on Social Work Practice. 2008;18(6):543–54.
37. Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015;350:h1258.
38. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M, et al. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008;337:a1655.
39. Wong CA, Ming D, Maslow G, Gifford EJ. Mitigating the impacts of the COVID-19 pandemic response on at-risk children. Pediatrics. 2020;146(1).
40. Wright CM, Chomitz VR, Duquesnay PJ, Amin SA, Economos CD, Sacheck JM. The FLEX study school-based physical activity programs - measurement and evaluation of implementation. BMC Public Health. 2019;19(1):73.

Tables

Table 1. Demographics of participants in the ¡Vamos Por Mas! Program

Note: Same superscripts denote statistically significant differences between the observed demographics between schools (p<.05). 

Table 2. Participation Rates and Evaluation of the ¡Vamos Por Mas! Program

Note: Same letter superscripts denote statistically significant differences between the observed demographics between schools (p<.05). 

^z Families did not need to enroll in order to participate in the workshops. 

^y Total number of parents, students, and school staff who completed the workshop evaluations

Table 3 – Intervention Participant perceptions about the ¡Vamos Por Mas! Program implementation based on the CFIR Framework

Table 4. Implementation. Participant perceptions about the implementation of the ¡Vamos Por Mas! Program implementation based on the CFIR Framework 

Table 5 – Recommendations Participant recommendations for changes to the ¡Vamos Por Mas! and its implementation based on the CFIR Framework