A longitudinal trend survey (Fraenkel et al., 2019) was used because it permits data collection at different points (to study changes in the variables of interest). Different samples from a population, whose members may change, are surveyed at different times in trend studies.
The Center for School Behavioral Health (CSBH) at Mental Health America of Greater Houston (MHAGH) provided more than 150 training sessions, using EBP interventions in selected Texas school districts from 2019 to 2020. A need for interventions arose from students’ increased trauma from natural (i.e., Hurricane Harvey) and manmade (e.g., Santa Fe Texas school shootings) disasters. MHAGH’s in person workshops were delivered in schools before the novel Coronavirus 2019 lockdown; some virtual workshops were delivered after March 2020. Three MHAGH training specialists led the workshops. Training methods included experiential activities such as role play simulation, problem solving scenarios, and small group discussion about youth mental health issues.
The EBP is a year-long, train-the-trainer program. Intervention topics were based on the EBP curricula for students’ behavioral and mental health needs and included Children’s Mental Health, Trauma Informed Classrooms, Youth Suicide Prevention, Trauma 102, and Self-Care and Mindfulness. MHAGH’s train-the-trainer workshops lasted nine hours; training for school personnel was about 1.5 hours/topic. After EBP training, trainees teach and equip school staffs to support students and families with youth behavioral and mental health needs. Briefly, EBP intervention topics included:
-
Children’s Mental Health (CMH): defines mental health, symptoms of mental health disorders, and presents strategies for empathetic conversations.
-
Trauma Informed Classrooms (TIC): examines the effects of trauma on brain development, creating trauma informed spaces, and addresses grief in childhood.
-
Youth Suicide Prevention (YSP): presents myths and facts of suicide and intervention strategies for youth with suicidal ideation.
-
Advanced Trauma (T102): strategies to create trauma informed classrooms and campuses, including impacts of cultural awareness, equity, and racial/historical trauma in the classroom.
-
Self-Care and Mindfulness (SCM): addresses employee burnout, establishing achievable SMART goals, and developing self-care plans.
Participants included school personnel (i.e., teachers, principals, counselors), parents, and community members (i.e., child-serving agencies in school districts) (Table 1). All participants were affiliated with public or private schools.
Table 1
Participants at Intervention Trainings (EBP Training), 2019-2020.
EBP Interventions (Topics)
|
2019
|
2020
|
Total
|
Children’s Mental Health (CMH)
|
3,896
|
703
|
4,599
|
Trauma-Informed Classrooms (TIC)
|
3,055
|
1,188
|
4,243
|
Youth Suicide Prevention (YSP)
|
2,625
|
799
|
3,424
|
Trauma 102 (T102)
|
-
|
727
|
727
|
Self-Care and Mindfulness (SCM)
|
4
|
885
|
889
|
Note. Participants represented 28 Texas public school districts, four charter school systems, seven special needs schools, four private schools, 28 behavioral health providers, 26 community education and advocacy groups, 11 public entities, 20 child-serving organizations, two funders and four institutions of higher learning.
|
An online consent script informed each participant that anonymous responses would be used in group format for technical reports and publications; consent required acknowledgement (I Agree) before entry to the training surveys. Pre-intervention data were collected during registration. Following the EBP training, participants answered post-intervention questions like the pre questions. We developed retrospective only (post-then-pre) post instruments to reduce response-shift bias. Colosi and Dunifon (2006) mentioned threats to validity with the retrospective method as,
-
Recall: inability to accurately recall past attitudes and behaviors.
-
Social desirability: need to report improvement to fit expectations or inflate perceived improvement for most important items.
-
Effort justification: report improvement (subconsciously) to justify time and energy invested in program attendance.
-
Cognitive dissonance: report improvement even if unrealized, to meet self-expectations of change.
Not all participants completed all intervention assessments; therefore, only paired data are reported herein.
Perceptions and knowledge of EBP interventions were collected with instruments based on Kirkpatrick’s second level of evaluation (i.e., learning); all instruments used Likert-type, 5-point scales (i.e., 1 = strongly disagree, 5 = strongly agree; 1 = low, 5 = high; 1 = very uncomfortable, 5 = very comfortable; 1 = poor, 5 = excellent; 1 = never, 5 = always). Concerning perceptions, the CMH intervention included five pre/post items rated on a 5-point scale (strongly disagree to strongly agree). Cronbach’s (1951) alphas were .92 (pre) and .94 (post); both were highly reliable. Five retrospective-only items were rated with 5-point scales (poor to excellent; very uncomfortable to very comfortable). Cronbach’s (1951) alphas were .80 (before) and .79 (after), also deemed reliable. The TIC intervention used retrospective only items with 5-point scales (low to high). Cronbach’s (1951) alphas were .89 (before) and .92 (after); deemed highly reliable. The YSP intervention included eight items (retrospective only) with a 5-point scale (poor to excellent). Cronbach’s (1951) alphas were .93 (before and after); deemed highly reliable. T102 intervention included eight items for knowledge and comfort levels (retrospective only) on a 5-point scale (poor to excellent). Cronbach’s (1951) alphas were .95 (before) and .97 (after): deemed highly reliable. The SCM intervention had three pre/post and ten retrospective items for knowledge and comfort on 5-point scales (never to always; poor to excellent). Cronbach’s (1951) alphas were .74 (pre) and .97 (post) and .84 (before) and .95 (after) in retrospective only; all were highly reliable.
Knowledge was tested with questions from the EBP curricula. Twelve pre/post questions in T102 and 11 pre/post SCM knowledge questions were posed as single (4-item multiple-choice) or multiple response (check all correct) items. Testing effect threats were minimized by randomizing question and response order (Campbell & Stanley 1963). Participants’ responses were dichotomously coded (0 = incorrect, 1 = correct). Kuder-Richardson 20 (KR20) reliability coefficients was used because the KR20 appropriately determines internal consistency of measurements with dichotomous data (Fraenkel et al., 2019). KR20 for T102 was .71 (pre) and .64 (post); SCM yielded reliability .47 (pre) and .58 (post). Reliability was attributed to heterogeneity of items and item discrimination (Frisbie, 1988), and limited number of questions. Results should not be generalized beyond the sample when KR20 scores are less than .60 (Ursachi et al., 2015). Knowledge results represent only those in MHAGH’s EBP trainings.
Participants demographic characteristics were recorded during registration. All valid paired data were assigned unique numbers and personal information was removed before analyses. Descriptive and inferential statistics described the data. Alpha was .05 a-priori to determine if significant differences existed in pre/post perceptions or knowledge. Non-parametric tests (e.g., Friedman, McNemar) were used because normality of data from non-random samples could not assumed.
Multiple purposive (non-probability) samples were drawn from EBP intervention trainees. Due to space limitations, only their primary demographics are in Table 2. Each intervention included more than 400 participants. We calculated power using G*Power 3.1.9.7 (Faul et al., 2007); sensitivity analysis indicated a sample of 400 was large enough to detect within-subject interactions (i.e., post hoc matched pairs; α = .05) as small as d = .2 with 98% power. Participants were characterized as female (~62%), white (~31%), teachers (~63%), who were 36-45 years old (~20%) (Table 2). They were 41 years (SD = 10.90) old.
Table 2
Participants’ Characteristics from EBP Intervention Trainings (2019-2020).
|
CMH
(N = 1,066)
|
TIC
(N = 2,133)
|
YSP
(N = 1,673)
|
T102
(N = 428)
|
SCM
(N = 401)
|
Variables*
|
n
|
%
|
n
|
%
|
n
|
%
|
n
|
%
|
n
|
%
|
Sex a
|
|
|
|
|
|
|
|
|
|
|
Female
|
618
|
58.0
|
754
|
35.3
|
747
|
44.7
|
379
|
88.6
|
340
|
84.8
|
Male
|
179
|
16.8
|
202
|
9.5
|
183
|
10.9
|
35
|
8.2
|
42
|
10.5
|
Race b
|
|
|
|
|
|
|
|
|
|
|
White
|
430
|
40.3
|
527
|
24.7
|
545
|
32.6
|
213
|
49.8
|
22
|
5.5
|
Hispanic
|
163
|
15.3
|
175
|
8.2
|
201
|
12.0
|
92
|
21.5
|
86
|
21.4
|
Black
|
131
|
12.3
|
139
|
6.5
|
107
|
6.4
|
61
|
14.3
|
96
|
23.9
|
Age
|
|
|
|
|
|
|
|
|
|
|
26-35
|
254
|
23.8
|
279
|
13.1
|
283
|
16.9
|
101
|
23.6
|
99
|
24.7
|
36-45
|
244
|
22.9
|
262
|
12.3
|
274
|
16.4
|
154
|
36.0
|
116
|
28.9
|
46-55
|
162
|
15.2
|
208
|
9.8
|
212
|
12.7
|
92
|
21.5
|
76
|
19.0
|
Profession
|
|
|
|
|
|
|
|
|
|
|
Teacher
|
792
|
74.3
|
1360
|
63.8
|
1,092
|
65.3
|
215
|
50.2
|
239
|
59.6
|
Paraprofessional
|
63
|
5.9
|
178
|
8.3
|
113
|
6.8
|
36
|
8.4
|
49
|
12.2
|
Counselor
|
7
|
.7
|
115
|
5.4
|
109
|
6.5
|
109
|
25.5
|
47
|
11.7
|
Other Staff c
|
57
|
5.3
|
106
|
5.0
|
116
|
6.9
|
44
|
10.3
|
37
|
9.2
|
Note. Frequencies may not equal 100% because of missing data. *Self-reported gender, race or ethnicity, age, and profession. a Responses included Decline to answer or Other. b Responses included Asian, Two or more races, American Indian or Alaska Native and Native Hawaiian or Other Pacific Islander. c Other Staff included coaches, librarians, school nurses, behavioral health specialists, etc.
|
The professional development activities and data reported herein are not considered research involving human subjects as defined by DHHS and/or FDA regulations. This study was exempt by an Institutional Review Board at [STATE UNIVERSITY].