Design, Settings and Participants
This study is a RCT in which patients were randomised to the guided online intervention or to a wait-list control group. The control group had direct access to a website with psycho-education about the reasons for suicidal thoughts, risk factors and where to seek help (e.g. 113 online, Samaritans, mental health organisations) and could start with the intervention after the post-test measurement at 6 weeks. As part of the feasibility, semi-structured telephone interviews were held at 6 weeks with the intention to obtain in-depth information about how the intervention was implemented and the contextual factors related with the uptake of the intervention in daily life. Participants were recruited between January 2017 and October 2018.
Recruitment took place among the Turkish-speaking population (i.e. people of Turkish, Kurdish and Turkish Cypriot background) (23,44) in the Netherlands and in the UK. Participants were recruited from the general population through newspaper advertisements, TV programmes, banners on relevant websites (e.g. of community and health organisations), social media and public events. These events were organised in collaboration with the community organisations representing the population of interest.
Eligible participants were 18 years and older, and had Turkish background (being born in Turkey or having at least one parent being born in Turkey) had a suicidal ideation score of 1 or higher on the Beck Scale for Suicide Ideation (BSS), had access to a PC and internet, and consented to provide their name, telephone number, e-mail address and the e-mail of their General Practitioner (GP). These data were needed for our safety procedure. Exclusion criteria were: 1) Being younger than 18 at the time of the study; 2) not having a Turkish background; 3) not being registered with a GP; 4) not residing in the Netherlands and/or in the UK where the recruitment took place; 5) being computer illiterate, and 6) not having access to internet. Already receiving help, regardless of the source, was not an exclusion criterion.
A study website (http://kiymacanina.org/) was created where potential participants could find more information about the study and register in Turkish, English and Dutch. Those who registered received an e-mail with further information, an informed consent form and a link to the baseline questionnaire. Those who returned the informed consent, provided their contact details and those of their GPs`, filled out the baseline questionnaire, and did not fulfil our exclusion criteria, were included. The randomisation scheme was derived using random allocation software by an independent researcher. Randomisation was stratified for the UK and the Netherlands and took place in a 1:1 ratio. The outcome was communicated to the participant by e-mail with either a log-in code for the intervention or a link to a website with general information on suicide for the wait-list control group. The study protocol is described in more detail elsewhere (39) and was registered in the Netherlands Trial Register NTR5028 (see https://www.trialregister.nl/trial/4926).
This study was approved by the Medical Ethics Committee of the VU University Medical Centre in the Netherlands (registration number 2014. 187) and by the Queen Mary University of London Research Ethics Committee in the UK (registration number QMERC2014/46). Written informed consent of participants was obtained after the study and all procedures had been fully explained in writing. Participants could ask questions by e-mail or telephone if wanted.
As this study involved vulnerable people who are at risk of suicide, a safety protocol was used (39). In summary: every participant (in both conditions) was asked to fill out the Beck Suicidal Ideation Scale (BSS) once in two weeks throughout their participation. If a participant exceeded the cut-off score of 29 on the BSS, we were obligated to phone the participant to perform a risk assessment. If deemed necessary, the next step of action was contacting their GP. The phone calls were going to be made by a psychologist in the research team under the supervision of a licensed clinical psychologist in the Netherlands and a licenced consultant psychiatrist in the UK who were both experienced in suicide prevention. As a last resort, participants’ GPs were also going to be contacted if they could not be reached.
The original version of the intervention is developed by van Spijker and colleagues (45). RCTs investigating the unguided version of this intervention showed its effectiveness in reducing suicidal ideation compared to the treatment as usual in general Dutch (d=0.2) (30) and Belgian populations (d= 0.34) (32), but it showed no effect when implemented in a general Australian population (31).
The intervention is based on the CBT framework (46). Within this framework, some mindfulness exercises were also included. The main principle is that, worry, rumination and repetitive suicidal ideation each produce obsessive attention to particular thoughts, sometimes resulting in a desire to end consciousness as a way to end the tantalizing repetition of suicidal thoughts (45). Thus, the aim of this intervention is to enhance controlled thinking (i.e. focusing on postponing worrisome thoughts to specific time-slots “worry times’’ of the day and not thinking of these thoughts for the rest of the day).
The intervention consists of six modules: 1) the repetitive character of suicidal thoughts, 2) regulating intense emotions, 3) identifying automatic thoughts; 4) thinking patterns, 5) thought challenging, and 6) relapse prevention (38,45). Each module contains a theory section, a weekly assignment and several exercises. Weekly assignment is an essential element of the intervention encouraging participants to practice the techniques they learn from the intervention in daily life. For example, the first module explains that suicidal thoughts can develop out of self-protection, as keeping on living may seem worse than dying. Similarities between worry and suicidal thinking are also outlined. A weekly assignment involves tallying suicide-related thoughts to obtain an idea of how often these thoughts occur, while the exercises are aimed at learning to manage these repetitions better by introducing worry postponement. Participants are advised to do one module per week.
Guidance was available from coaches (2 in the UK and 2 in the Netherlands), who were supervised by a team of three experts including a psychologist, a licenced clinical psychologist (NL) and a consultant psychiatrist (UK). The coaches were bi-lingual (English-Turkish or Dutch-Turkish) and were either students at a Masters level or practitioners seeing patients. They received training about how the online intervention works, safety protocol, referral system in both countries and their roles and responsibilities whilst providing guidance to the participants. Regular supervision meetings through skype and/or e-mail communications were arranged.
Coaches provided online feedback to participants after their completion of each module. The aim of this personalised feedback was to help participants to understand the exercises and homework assignments as explained in the lessons. Moreover, it was used to motivate the participants to continue with the intervention.
We adapted this intervention linguistically and culturally for the Turkish migrant populations in line with the evidence indicating that interventions work better when adapted to local settings (41). The decisions made during the cultural adaptation process are outlined in appendix B. In summary, we used the ecological validity model of Bernal and colleagues (40). This framework retains the core principles of CBT in order to preserve treatment validity but permits flexibility (26,47). The model delineates eight dimensions when culturally adapting an intervention: the use of appropriate language, persons (cultural similarities/differences between the client and clinician which shape the therapeutic relationship), metaphors (symbols and concepts), content (cultural knowledge), concepts (treatment concepts that are culturally congruent), goals (that support adaptive cultural values), methods (cultural enhancement of treatment methods) and context (consideration of acculturation, social context) (40) (see Appendix B).
First, the intervention was forward translated and back-translated. The consistency of the adaptations in three languages (English, Dutch and Turkish) was checked by bi-lingual speakers. The cultural and linguistic adaptation was based on the results of 6 focus groups and 7 individual interviews with 38 Turkish-speaking lay people and 4 professionals living in the Netherlands and in the UK during the year 2014/2015 (23). The key suggestions for the cultural adaptation of the intervention were: enriching the content of the intervention with relevant case studies, including quotations from participants, who have found the programme useful, and reducing textual information and including more visuals such as pictures of nature and people who look happy, calm and/or relaxed (23). In light of these, we made modifications in the concepts (see Appendix B). For instance we included some well-known idioms and metaphors describing psychological distress and suicide in Turkish language. Furthermore, modifications in the context included cultural case examples (see Appendix B). Additionally, some theoretical modifications have also been made. In line with the well-documented evidence supporting the value of MBT (19-21) for instance, a well-known MBT based exercise called “safe place” was incorporated into the crisis plan in the intervention. This exercise uses guided imaginary and encourages people to create an imaginary safe place that they could visit whenever they feel the need to be grounded (19) (see appendix B).
Deviations from the study protocol
The sample size of this RCT was calculated based on the expected effect of the primary outcome measure: the reduction in frequency and intensity of suicidal thoughts as measured with the BSS (d=0.40). The rationale for this decision was based on the trial of van Spijker who found an effect size of d=0.2 for an unguided version of the treatment (30). We expected a higher effect size since we provided personal coaching which generally leads to higher effect sizes (28). Based on power of 0.80 and an alpha of 0.05, 100 participants were needed in each condition. Given the expected drop-out of 30%, the total sample size was determined as being 286 (39).
Although intended to be a definitive trial, at best it became a feasibility study with process evaluation to understand the components and how they operate. We conducted in depth interviews with the participants who completed the intervention and consented to be interviewed on feasibility issues. Furthermore, we have excluded the following questionnaires from the study: Suicidal Ideation Attribution Scale, the Hospital Anxiety and Depression Scale and an item measuring the satisfaction with the treatment. This was done to reduce the burden on participants and to increase their motivation to fill out our questionnaires. Moreover, we did not include three months follow-up assessments (T4) in the analyses and have done completers analyses. The reason was the underpowered study design which did not allow us to perform complex statistical procedures such as multiple imputation. We have used T4 to monitor the safety of the participants only. Finally, in the original protocol we described that we would add two additional modules on self-harm to the original intervention. In the end we decided to keep the focus on suicidal thoughts only (39).
As part of the feasibility, the system usability scale was used and interviews were conducted. The feasibility was measured at post-test (6 weeks after baseline: T3). Usability of the online intervention was measured with the System Usability Scale (SUS). The SUS is composed of 10 statements that are scored on a 5-point scale of extent of agreement (score 0 to 100). The reliability is good (α=0.91) (48). In our study the scale also showed good internal consistency (α=0.81). Interventions with scores of 70 and above are accepted as highly usable (48) and scores between 50 and 70 indicate acceptable usability of an intervention. Interventions with scores 50 and below are subject to concerns about their usability by the target population and should be investigated further (48).
In order to answer the research question about the indications of effectiveness, online questionnaires were used. The primary outcome measure in this study is the reduction in the frequency and intensity of suicidal thoughts. This was measured with the BSS (49) at baseline (T0), 2 and 4 weeks into the intervention (T1 and T2), and at (T3). The BSS is a 21-item measure assessing the severity of the suicidal ideation (49-51). Each item is scored from 0 to 2. The total score is obtained by adding the first 19 items and ranges from 0 to 38. High score represents high suicidal ideation. The BSS has good psychometric properties in English (49-51) and in Turkish (52,53). In our study the scale had excellent internal consistency (α=0.92).
Secondary outcome measures are measured at T0 and at T3 and included the following: depression, hopelessness, worrying, quality of life, self-harm behaviour and suicide attempt, acculturation.
Depression was measured with the Beck Depression Inventory (BDI) comprised of 21 items (54). Each item is scored from 0 to 3. The severity ranges from minimal depressed (score lower than 13) to severely depressed (scores between 29 and 63) (54). It is a reliable and valid (54) measure for assessing depression. The BDI has been validated in Turkish and Dutch population (55). In our study the scale had good internal consistency (α=0.82).
Hopelessness was measured with the Beck Hopelessness Scale (BHS), and contains 20 true and false statements (56). Each statement is scored from 0 to 1 and the total score ranges from 0 to 20. A high score indicates high degree of hopelessness. The instrument has good psychometric properties (57,58). In our study the scale showed excellent internal consistency (α=0.92).
Worrying The Penn State Worry Questionnaire (PSWQ-PW) is a 15 item inventory assessing the weekly status of pathological worry (59). Each item is scored on a 7 point rating scale, ranging from never 0 (never) to 6 (almost always). The total score ranges from 0 to 90 with a high score indicating more worrying. PSWQ-PW shows good reliability and convergent validity (59). The Turkish version demonstrated good reliability (60). Additionally, in our study the internal reliability was good (α=0.78).
Quality of life The Euro Quol (EQ-5D) is an instrument measuring health quality of life and has 5 items: mobility, self-care, usual activities, pain/discomfort, and anxiety/depression (61). Each item is required to be rated as 1 (no problem), 2 (some problem) or 3 (extreme problem). The current health state is also rated on a scale ranging from 0 (worst imaginable state) to 100 (best imaginable state). Both Dutch and Turkish versions have been validated (62,63). In our study the scale showed sufficient internal consistency (α=0.60).
Suicide Attempt and Self-Harm (SASH) Four questions measuring the previous suicide attempt and the presence of self-harm were taken from the original Self-Harm Questionnaire (64). The original scale showed good psychometric properties (64,65). In our study the scale items showed sufficient internal consistency (α=0.60).
Acculturation was measured with an adapted version of the Lowlands Acculturation Scale (LAS) representing the difficulties that migrants might face (66). On a 6 point rating scale, item scores range from 1 (not applicable) to 6 (very applicable). The instrument is validated among Turkish migrants living in the Netherlands (66). It has been adapted to measure the 2 dimensions of acculturation: participation (4 items) and maintenance (11 items) (67,68). The subscale participation measures tendency to participate in social life of the host country such as interacting with other minority and majority groups (score range 4 to 23). The subscale maintenance measures the tendency to maintain one`s culture of origin such as preferring to interact with people from same ethnic background (score range 11 to 60). Higher scores indicate a greater degree of participation and maintenance. The new sub scales were reliable and internally consistent in Turkish migrant populations (68,69). In our study both sub scales showed good internal consistency (α=0.82 for both sub scales)
We defined feasibility as engagement with the intervention (33), its usability in daily life, as well as the potential for delivering a full trial in future. The following components were assessed during the interviews in order to identify the facilitators and barriers influencing the engagement with the intervention. Cultural relevance (i.e. familiarity and relevance of the therapeutic content to one`s cultural background), cultural appropriateness (i.e. appropriateness of the therapeutic content in terms of the cultural context) and acceptance (i.e. feedback on the experience of using the intervention in real life) (26).
Telephone interviews were conducted with the participants who completed the intervention and consented to be interviewed (N=12) and started with an open question: ‘What was your overall experience during your participation?’ A topic guide was used for the remainder of the interview which was created on the basis of the relevant literature and discussions with the rest of the researchers taking part in the study (see Appendix A for the topic guide). The interviews were approximately 30 minutes long and were recorded for verbatim transcription.
We used thematic analysis as a means of identifying, analysing and reporting explanatory models, and for understanding which elements of the intervention facilitated or hindered participants` progress during their participation and how the intervention can be optimised to increase its acceptability, relevance and user-friendliness (70). Pseudonyms were assigned to each interviewee. First, the code system (and categories and themes developed on the basis of the coding process) was developed gradually and collaboratively. The code system was developed on theoretical grounds and included following categories: Definition of facilitators and barriers, specific examples for cultural relevance, appropriateness and acceptability of the intervention and specific recommendations for further improvement. Each of these categories had a number of sub-categories and codes. The definition of therapeutic gains as a result of using the intervention (e.g. therapeutic alliance) emerged during the analysis. This category was considered as important and decided to be analysed separately. The coding system was developed by the first author and was checked independently by a second person (Y.A.) who was not involved in the research. Once an agreement was reached, they were further developed, refined and applied to the transcripts. The first author was the main coder and Y.A. involved as a second coder, who systematically counter-checked the coding, to assure the robustness and the internal validity. The data was coded manually. Disagreements over the coding were discussed between the main and the second coder and where necessary experts (S.R.) were consulted. Detailed descriptive accounts were produced for each major theme alongside the related extracts from participants’ transcripts. Analysis continued until no new themes emerged from the transcripts (see table 4 for the themes).
The RCT was carried out in accordance with the CONSORT guidelines (see consort checklist). First, t-tests and chi-square tests, as implemented in SPSS, were used to compare the baseline characteristics of those who were allocated in the intervention group with those who were allocated to the waitlist control group. Second, we tested for the likely effects of the intervention compared to the waitlist control group. We used Bayesian Repeated Measured ANOVA as implemented in the JASP (version 0.9.2), which is a free and open-source graphical programme for statistical analysis. The rationale for choosing Bayesian approach over the classical inferential approach is based on the following advantages: Bayesian approach considers all possible models (null model, between and within group differences) and assigns more weight to those models that predict data relatively well (71,72). Classical inferential model selects the best model, estimates its parameters and might produce overconfident conclusions on data by neglecting model uncertainty (71,72). Since the present study design is underpowered, the Bayesian approach accounts for the uncertainty of the all possible models and allows us to make more reliable conclusions based on the existing data (71). Additionally, we did a sensitivity analysis on those participants who reported severe suicidal ideation at baseline to see if the indications for an effect of the intervention were stronger than in the whole group including those with very mild suicidal ideation.