Study Design
The ¡VxM! program was piloted using a cluster randomized trial. This manuscript focuses on the schools that implemented the program as part of the intervention group using an explanatory sequential mixed-method study design (23). In this design, quantitative implementation measurements were followed by qualitative focus group exploration to illuminate initial findings. Data analysis and integration were done through merging, in which quantitative and qualitative analyses were done separately, and then combined at the interpretation phase for comparison (24). This design provides convergences of findings and allows for insights into implementation factors not possible with a single method. All procedures were approved by the XXXX Review Board.
Setting
The ¡VxM! program was implemented at schools in Santiago, Chile. Chilean primary education is offered through both private and public institutions. Within private institutions many use government subsidies. All participating schools were subsidized private schools with varying sizes and socioeconomic vulnerability (Table 1).
¡Vamos por Más! Implementation
The ¡VxM! program was implemented during the 2019 school year in three schools randomized to the intervention group. The program was offered to all families with a student in 5th or 6th grade. School leadership appointed a school coordinator to lead the implementation. Teachers served as workshops facilitators and were required to have a professional degree (either as a teacher, psychologist, nurse, or social worker) and experience leading groups. All school staff attended three meetings. The first introduced the program, and the other two were 90- minute trainings prior to the in-person workshops, which were divided by school grade (e.g., teachers facilitating the 5th grade workshops only received training in those activities). At the end of the trainings, facilitators completed an evaluation.
In-person workshops were implemented during scheduled 2-hour group parent-teacher conferences, on weekday evenings. Some schools provided snacks for the participants, although traditionally uncommon. Media messages were sent through WhatsApp® to all participants who provided their phone number at enrollment. Messages were sent at best reception times (25). At the workshops and in messages, participants were invited to reach out to the school coordinator or the program´s email for additional support. The research team provided technical and content support as needed.
Implementation Evaluation
All parents or guardians (hereafter referred to as “parents”) were invited to participate in the program evaluation. Parents read and signed an informed consent form for themselves and their child. Youth read and signed an assent form. Families were considered enrolled within the evaluation if consent was provided and a baseline survey was completed. Regardless of enrollment status, all families could participate in program activities and workshop attendance was collected via sign in sheets. Participants and facilitators completed anonymous evaluations after workshops. Message reception rates were tracked by research staff and classified as not received, received but not viewed, or received and viewed.
Focus groups (FGs) organized by participant type were conducted with parents, teachers and school leadership after implementing the program. All FGs were divided by school, and parents were further divided by grade. An additional FGs was completed with the research team. Semi-structured, audio recorded FGs were conducted by researchers with experience in qualitative methods. FGs lasted 1.5–2 hours and were conducted in Spanish. Open-ended questions explored specific areas of the CFIR and varied slightly between FGs.
Participants
Table 1 summarizes participant’s demographic information. Schools had 625 students in 5th and 6th grade, of which 308 were female (49.4%) and 317 were in 5th grade (50.7%).
Program Evaluation Participants
Overall, 38.4% of eligible students and 22.1% of eligible parents completed a baseline survey (n=240 and 138, respectively). Over half of the students were female and exactly half were in 5th grade. Comparing students who were eligible and those completing the baseline survey, there were no statistically significant differences in gender or grade (p=.066 and .850, respectively).
Of enrolled parents, the majority identified as female and were born in Chile. Their average age was 39.9 years (SD 7.1). About half of the total parents were married and the majority worked outside their home. There were statistically significant differences between schools in rates of parents born in Chile and employment (p<.001 and .003, respectively).
Twenty-one facilitators completed a demographic survey (58.3%). School staff from School #2 (S#2) were less likely to complete their training evaluation forms (p=.032). Approximately half worked as teachers and most identified as female.
Focus Group Participants
In total, thirteen FGs were conducted at the end of program implementation. Participants included school leadership (n=3 FGs, 7 participants), teachers (n=3 FGs, 19 participants), parents (n=6 FGs, 36 participants), and researchers (n=1 FG, 4 participants).
Outcomes
Proctor’s intervention implementation outcomes (2010) were evaluated through program participation, surveys and FGs. Penetration was measured using participant engagement. Acceptability was operationalized as overall satisfaction with the program; Appropriateness as perceived utility and interest in the program; and Fidelity as the degree to which facilitators implemented the curriculum without adaptations. These data were collected in the quantitative evaluation phase. Feasibility, implementation cost, and sustainability were evaluated in FGs and operationalized as the degree to which the program could be implemented, the cost of reproducing the program in the future, and what would be needed to sustain the program.
Data Analysis
Quantitative Data Analysis
Demographics, perceived satisfaction, utility and fidelity were summarized using descriptive statistics. Message view rates were calculated based on the total number of messages sent for each school. Data from continuous variables were compared between schools using one-way analysis of variance using Bonferroni correction to assess between-group differences. Categorical data were compared between schools using Chi2 or Fisher’s exact tests. No covariate adjustment was conducted. Resulting p-values <.05 were considered statistically significant.
Qualitative Data Analysis
FGs were transcribed verbatim and analyzed with NVivo (QSR International, v.12) using an inductive-deductive process following Content Analysis procedures (26). Guided by the CFIR domains, one analyst coded the data identifying emerging categories and sub-categories. The coding was revised by a second researcher, and discrepancies were discussed to achieve full agreement. A third researcher conducted an external review validating the coding. Data were further organized according to the participant´s role and school. Representative quotes by CFIR domain were translated and are presented as a word document entitled Additional File 1.
Mixed-methods Integration
Quantitative and qualitative data were integrated through connecting and building (24). The connecting strategy was used to guide sampling, as FG participants participated in the intervention. The building strategy was used to guide data collection, as FG questions were based on the quantitative findings. Findings are reported using a continuous narrative approach (24), where mixed methods are reported separately in the same manuscript.