We tested an upscaled version of the Coping With Depression/Coping With Strain Course (CWD/CWS), called MindPower (17, 22). The traditional CWD/CWS courses have mainly targeted high risk groups for depression and been delivered within a health service context. The MindPower version differs from previous versions by being upscaled to a universal program and being delivered class wise to high school students during ordinary school hours, independent of their mental health status. In spite of classical recommendations to go for universal preventive interventions when a health problem is widespread (23), to our knowledge, nobody has tried to test the delivery of a course in the CWD/CWS-family universally since Clarke and coworkers in 1993(16).
To test the efficacy of MindPower, we used a two-groups delayed intervention design combined with linear mixed models statistical analyses. To indicate whether the absolute symptom levels in the two intervention groups before, during and after the intervention differed significantly from the relevant general population, we compared the results with results from a large-scale population survey covering all students in the same age group and catchment area (UngData) (33).
We measured symptom levels of anxiety and depression with SCL-8 in order to be able to compare our results with the relevant general population, and with RADS-2:SF, since this instrument is well tested and shown to have good psychometric properties, is adapted particularly to this population, and is assumed to be sensitive to change (41).
The monitoring test showed that only half of the students in both intervention groups reported to have completed the tenth and last MindPower session (34). This deviates from the instructions in the study design and the course leader manual. All the ten sessions are regarded as necessary, because it takes time to learn all the content in MindPower. Less than complete implementation may attenuate effects of the intervention seriously.
The findings indicate no reduction or preventive effect on the anxiety and depression levels of the students. The main findings show, however not conclusively because of the limitations of this study, a small but significant symptom increase during the eight weeks course period, and throughout the study period. This slight increase in self-reported symptom levels of anxiety and depression in the MindPower sample was almost entirely attributable to those who at start of the study had the lowest symptom scores, which again were markedly lower than the scores of their peers in the comparable UngData population sample. Boys had significantly lower levels of depression and anxiety than girls throughout the MindPower study period. This is consistent with repeated reports that girls tend to have an elevated level of common mental distress in the general population as compared to boys (33).
Why did we not find positive effects of MindPower on symptoms levels of anxiety and depression?
The most obvious explanation is that a reduction of symptom levels among these participants would be hard to achieve because most of the students before the start of the study had only moderate or low levels of symptoms and even lower than a comparable group from the general population of peers (UngData).
Another possible explanation is selective dropout. The high dropout across time in this study leads to loss of statistical power and possibly also patterns of selection which again hampered our conclusions. The dropout was associated with age and gender; the older male students being overrepresented.
However, those who dropped out, did not differ significantly from complete responders on their last measurement of either SCL-8 or RADS-2:SF (all p > 0.05). Drop out was not systematically associated with the outcome variables of interest. If the intervention had had a significantly symptom dampening effect in the two groups, such an effect should have been revealed in spite of the high level of dropout.
In fact, although modest, we saw a slight increase of symptom levels across time. Since the students included in this study seem to be a selected group with lower than peer population levels of symptoms, this may be due to statistical regression. In that case, the increased symptom levels were not due to MindPower, but to statistical effects.
However, it is not unusual that when measures of levels of anxiety and depression are repeated in non-clinical populations, the level of self-reported symptoms increases. This has been attributed to destigmatizing and to increased awareness.
This study included seven measurement points, all with the same symptom measures, across more than one year in a group of participants with little previous experience in reporting on their mental health. This may have facilitated a small awareness effect. The fact that the students during the project period were trained in how to deal with issues of life, may also have facilitated such an effect (42). From a mental health literacy point of view, increased awareness of one’s state of mental health may be regarded as a positive result.
Although several of these explanations may have been operating, we cannot exclude the possibility that the results turned out as they did because MindPower, as it was implemented, did not meet the students’ expectations or for other reasons was ineffective.
In that case, it should be noted that at the end of the project, the levels of anxiety and depression were still low and not higher than the level found in the comparable population study among students who had not received any intervention. Consequently, although a slight symptom increase was observed, it can hardly be argued that the intervention was harmful.
What can we learn from this study?
This study was launched and conducted in a very positive contextual atmosphere. The Norwegian government had pledged that all schools must provide life-skills education. A new national curriculum plan on how to include public health and life skills in the schools was in the process of being launched. Researchers, students, teachers, local, regional, and national politicians, and administrators had for a long time argued for equalization of mental and physical health in the school. Openness about mental health issues had increased significantly and continuously since The Ten Years National Task Force On Mental Health (1999–2008).
In the case of this study specifically, the top administrator of education in the county welcomed the study and was hands on, enthusiastic, and effective in supporting communication and collaboration with the school administration and the schools. The Deputy Minister of Health of Norway visited the project, which was also mentioned in the text of the National Budget (2020). Several motivation meetings with the teachers before and during the project period indicated high motivation among school leaders, teachers, and students to participate.
Comprehensive preparations were laid out before implementation. All the involved teachers received intensive training, arranged by the organization ‘Fagakademiet’. This included theoretical lectures, study design, practical guidelines and instructions, role-playing, and homework, ending in certification to teach MindPower. In an internal evaluation from Fagakademiet, the teachers reported very high satisfaction regarding the training. Prior to each data collection, the teachers received a short list of key points to ease remembering of what to do. In addition, all 170 teachers received the principal researcher’s phone number and e-mail address in case they had questions about the study.
With such a positive context and such a thorough implementation and enthusiastic follow up, what could have been done to improve intervention implementation and data collections? A qualitative study has addressed this question (43). In addition, we have collected information through a number of informal sources. This has provided us with several learning points for others who intend to launch large-scaled universal mental health initiatives among high school students.
The first problem identified was that, at some schools, the teachers did not have sufficient support from their school administration. In particular, some lacked assistance from their co-teachers to adapt the MindPower program into the curriculum.
A solution was, as some schools did, to establish a forum at the school that could include mental health personnel. Their intention was to share positive and negative experiences and to seek advice and support. Such a team might give the teachers the support they needed to feel competent in teaching life-skills after their training program.
Another problem is lack of clarity with regards to teachers’ job descriptions. Teachers questioned whether they alone should have the responsibility of reducing symptoms of anxiety and depression among adolescents. Furthermore, they questioned whether schoolteachers have the mandate, the skills and the resources needed in order to effectively promote students’ mental health. Other teachers maintained that teaching mental health is not a school-teacher’s job. Teaching and promoting students’ mental health is rather the job of the health services. Others were reluctant to address questions about difficulties in life in classroom settings, as they themselves were struggling with such difficulties.
Solutions to this kind of challenges could be that the program leadership and principals together give clear directions on these concrete issues. This must be sorted out well before the teachers go through comprehensive training to become competent life-skill teachers. However, relevant authorities had not provided clear mandates and descriptions with regards to how life-skills should be implemented in schools, and what to expect from the teachers.
Based on experiences from the present study, we have come to the conclusion that teachers have not been sufficiently well mandated and resourced for the task of taking the lead in implementing and conducting school-based life skills training programs. By teaching life-skills, the teachers alone cannot be expected to reduce anxiety and depression levels among adolescents. Instead, life skills can be taught in terms of “how life is”, implying that challenges in life are normal and that no one is alone in experiencing such feelings. By learning about yourself, being aware of one’s feelings and being able to manage some tools, one might be better off in approaching such challenges in life.
A third problem is that teachers were instructed to participate in the preparatory course, whether they were motivated or not. Especially some of the teachers who teach subjects such as electronics and carpentry were not comfortable carrying out teaching activities implying handling of issues related to emotions. In some schools, the school administrations did not allow teachers to choose not to teach MindPower.
To address this, a solution is to only admit motivated teachers to teach life-skills programs. Unmotivated teachers are seldom good teachers. Consequently, as far as possible, teachers who see this as a natural part of their day-to-day practice as a teacher should perform life-skills education. It is necessary to respect that it cannot be mandatory to teach life-skills. On the other hand, some teachers maintained that all teachers are practicing life-skills education with their students anyhow when they are communicating with students. An example is mobilizing self-efficacy by teaching the students how to overcome their fear of making mistakes in math.
A fourth problem may be that the MindPower program has not been sufficiently well designed and tailored for classroom-based educational purposes. Teachers experienced considerable difficulties running the MindPower program in the classroom.
A solution might be to involve teachers and students in a process of program revision. Ideas and opinions of teachers and students are crucial in order to succeed. In this way the program will be better tailored for school-settings. However, in the present project, students and teachers were asked to give feedback on the course book before the implementation of the program. Furthermore, MindPower lasts for 90 minutes rather than two and a half hours as in traditional Coping With Depression/Coping with Strain-courses (17, 21). This change was made by the designer of MindPower to ease the implementation of MindPower in schools.
The fifth problem is general implementation issues. Adequate implementation requires strengthening the program on several dimensions. During the program implementation in schools it turned out that the teachers had deviated from the implementation plan on several points, for instance by not completing all the ten MindPower sessions, and by shortening the 90 minutes sessions. For many teachers, the intervention proved to be too difficult to handle. It turned out to be too time consuming and for practical reasons not easy to fit into the curriculum. The implementation varied considerably across schools. The ten schools had different needs and different curricula, and therefore had to develop their own local implementation plans to make the implementation possible. A demotivating factor for the students was when their self-selected favorite course was replaced with MindPower.
Solutions for this issue, is to test for fidelity. The first author of this study did not receive the necessary approval from the first author’s research institute to administer a questionnaire on monitoring fidelity. Despite of an approval from The Regional Committee for Medical and Health Research Ethics (REC).
There are at least four major problems that should have been addressed in a fidelity test, as described above, and confounding variables affect the findings of the study. To which extend the course book and the course leader manual have been used according to intent, affect the results. It is essential to test whether the 170 course leaders followed the course leader manual, the course book and the study design. Otherwise we cannot know whether we measure the depression and anxiety effects of the MindPower course, or something else.
Fidelity may contribute to control for some of the impact on the results. Especially when the program is changed, e.g. shortened according to the length of hours, and being not completed at all. It takes time for the students to learn and do the practical assignments in life-skills training, and it is essential to make the time and effort to learn life-skills (45).
Another problem was difficulties with data collections. The students had difficulties in retrieving the questionnaire due to technical issues. There were difficulties in administering the questionnaires with the correct e-mail addresses, and the participants had difficulties in both retrieving and opening the questionnaires at the seven time points. An additional problem in this study was that after we had received the student e-mail addresses directly from the teachers, during the preparations for the data collections, these e-mail addresses were replaced with less updated addresses retrieved from The Common Contact Register of Norway. Consequently, a significant portion of the eligible students did not receive the initial invitation to participate.
This issue has several solutions. It is necessary that the technical partner, which is administering the data collections, accepts the plan for what to do if there are technical failures. Such a plan includes making additional links with the questionnaires, which can be sent to the participants and teachers if technical problems occur. Pretesting data collection procedures is important, including making sure you have the best possible records of e-mail addresses and telephone numbers of study participants.
Strengths and limitations
This study, like all studies has its strengths and weaknesses. Strengths include the adapted randomized control trial design (RCT), with inclusion of a delayed intervention group, which functioned as a control group until the intervention was implemented. For ethical reasons, it is good to let all study participants benefit from the intervention, only with a period of delay of four months before the intervention is implemented in the second group. Compared with a pure RCT design, the two-group design with a delayed intervention in one of the groups may be less strong. Still the latter design has several strengths. If a pattern of stability before the intervention in the second group is observed, followed by a decline in depression and anxiety during and possible after the interventions in the second group and a similar pattern during and after the intervention is observed in the first group, we have strong indications of program effects.
In the context of the present study, also, data from the UngData study made it possible to compare outcome variables with corresponding prevalence and means estimates of the general population. Further, utilization of mixed models statistical analysis, with its handling of missing with maximum likelihood estimation contributes to making results less biased. The present study is based on relatively large numbers of participants, over 1 600 in MindPower and over 18 000 in UngData. The outcome measures used in our study are high quality instruments commonly utilized in studies on adolescents. Furthermore, when two such measures are administered repeatedly, changes over time can be examined.
Results presented above show that mean scores on outcome measures show the same patterns of change in both groups. This strengthens our trust in the findings in this study.
Limitations include insufficient fidelity assessment. Beyond systematic reports from students, there are only verbal reports and sporadic emails from teachers revealing problems of implementation and data collections that occurred during the study. Also, during the electronic data collection serious problems were encountered. Due to problems with the list of e-mail addresses, questionnaires often did not reach students. This may have contributed to attrition and selection bias, even at the first measurement occasion. This might explain why we observed significant differences between the two intervention groups already at baseline. Teachers as well as students reported problems with retrieving the questionnaire as stressful and time consuming, which caused a bad teaching situation during implementation of the MindPower program (43). Furthermore, even though analyses indicate that drop-out did not produce the observed increases in anxiety and depression levels, a limited number of participants filled in and submitted the questionnaire on all 7 data collection occasions. In this study, the mean score from the follow-up (T7) is not sufficient, because of the low participation at this last data collection. Furthermore, the lack of further data collections (after M7) makes it impossible to examine more long-term preventive effects of MindPower on depression and anxiety. The findings might not have generalisability if the school-samples e.g. background variables, teacher training, organization and school structures, differs in great extend from the current samples.
Although several studies have found positive effects on youth mental health after exposure to life-skills programs, reviews have concluded that there are serious risks of biases in a number of previous studies. This cast doubt on conclusions from these studies (46, 47). Therefore, there is limited evidence that educational setting-based interventions focused solely on the prevention of depression or anxiety are effective (47). However, in this paper we have provided a number of advises for future research on universal mental health promotion and prevention projects in order to improve implementation as well as data collection quality.