Setting
Sri Lanka is a lower-middle income country in South Asia with a multi-ethic and multi-religious population of 21.3 million, of whom 38.7% is under the age of 24 and 80% resides in rural areas [17, 18]. The Indian Ocean tsunami in 2004 and three-decades of civil war which ended in 2009, resulted in over 100,000 lives lost and left 300,000 civilians internally displaced [9, 19]. This study was carried out in three divisions in the Eastern Province, with a total population of 25,591 children aged 5-19 years [20]. Despite Sri Lanka’s overall economic growth, poverty rates in the Eastern and Northern Province, where most of the armed conflict was concentrated, are far above the national average [19]. There is no available data on the prevalence of mental health problems among children and adolescents in the Eastern Province. A 2011 study conducted in the Northern and Eastern Province showed that 92% of the children experienced life-threatening events such as bombings, attacks on homes and loss of family members during the conflict [21]. Furthermore, a qualitative study in the Eastern Province reported that adolescents perceived disrupted family relationships, separation and migration of parents, violence at home and sexual abuse as the main factors affecting their mental and physical well-being [22].
Design
This study assessed the accuracy of the CCDT in proactively detecting children and families in need of mental healthcare (i.e., CCDT positives). In addition, concurrent validity of the CCDT positives was assessed against the Strengths and Difficulties Questionnaire (SDQ) [23], a widely used alternative instrument to detect mental health problems among children and adolescents. As opposed to universal screening, proactive case detection with the CCDT will not lead to negative cases. A small proportion of CCDT negative cases (i.e., those detected by community members using the CCDT as probably not in need of mental healthcare) were therefore only included to avoid confirmation bias.
Instruments
Community Case Detection Tool. The CCDT is a tool for ‘community gatekeepers’, trusted and respected community members, who do not have any professional mental health background. The tool is developed to support proactive community-level detection of children and adolescents aged 6-18 years and families in need of mental healthcare to encourage help-seeking. It uses an adapted version of the ‘prototype-matching approach’, originally developed to simplify and standardize diagnosis. It presents related symptoms in a paragraph-length vignette to help recognise mental healthcare needs in daily life [24]. The vignettes are supported by illustrations, two questions and a simple decision algorithm to determine the level of match and decide on the follow-up action. If there is a match with a vignette, the gatekeeper is advised to support the child and family to seek help from available services. Based on qualitative research in Sri Lanka, we developed and evaluated three Tamil versions of the tool: internalising problem vignette, externalising problem vignette and family-related problem vignette (see Supporting File 1). For the purpose of this study, an identification card was developed for community gatekeepers to administer the version that was used for the identification and their knowledge about the need of mental healthcare prior to the introduction of the CCDT. A positive match with one of the three vignettes was scored as ‘CCDT positive’.
Ten Question Screen for Childhood Disability. An abbreviated four-item version of the Ten Questions Screen for Childhood Disability (TQS) was used to assess hearing, speaking, or severe cognitive disabilities prior to participation in the study [25]. Children who scored positive on one of the four items were excluded from the sample as the research methods were deemed inappropriate.
Mini-International Neuropsychiatric Interview for Children and Adolescents (MINI-KID). In this study, we used the Indian Tamil MINI-KID 6.0 to evaluate the mental health of children and adolescents the gatekeepers had detected. The MINI-KID is a short structured clinical interview to assess the presence of current DSM-IV (Diagnostic and Statistical Manual of Mental Disorders) and ICD-10 (International Classification of Diseases) disorders in children and adolescents aged 6-17 years [26]. Each diagnostic module starts with a screener followed by more detailed symptom, severity and functionality questions. The MINI-KID has shown to generate reliable and valid diagnosis [26] and has been used with children in Sri Lanka before [27]. Relevant modules were selected by a child psychologist (MJ), supervising psychiatrist from Sri Lanka (JJ) and Indian child psychiatrist and master trainer (JVK). The selected modules were depression, suicidality, dysthymia, panic disorder, separation anxiety disorder, obsessive compulsive disorder, post-traumatic stress disorder, alcohol and substance dependence, attention deficit hyperactivity disorder, conduct disorder, oppositional defiant disorder, generalized anxiety disorder and adjustment disorder. The modules on suicide, alcohol and substance dependence were only deemed appropriate for children aged 13 years and older. Standard relevant scoring and instructions were used for functional impairment caused by the symptoms and the time frame (i.e., current, past 6 or 12 months).
Family functioning. We used an adapted version of the Safe Environment for Every Kid - Parent Questionnaire-R (SEEK) to assess family problems and child protection needs [28]. Relevant items of the SEEK were selected based on the construct captured in the family vignette. The questionnaire was further adapted and translated through a systematic process in which the items were first translated into Tamil. The research team provided feedback to ensure separate items and translations were culturally appropriate, followed by a blind back-translation. The final questionnaire consisted of 14 items that addressed harsh punishment, child neglect, parental stress, intimate partner violence and substance abuse.
Indication for treatment. At the end of the interview, the senior counsellor administering the MINI-KID and SEEK answered a concluding dichotomous question regarding the need for any psychological treatment from a mental health counsellor or psychiatrist or child protection service. This indication for treatment was scored (i.e., yes/no) based on the counsellors’ judgement following the information provided in the structured clinical interview, the family assessment.
The Strengths and Difficulties Questionnaire. This study used the Sri Lankan Tamil parent version of the SDQ [29]. This widely used 25-item behavioural screening questionnaire for 3-16 year old children, covers emotional symptoms, conduct problems, hyperactivity/inattention, peer relationships problems and prosocial behaviour. A three point Likert scale allows the respondent to indicate how each item applies to the participating child [23]. All items, except those related to prosocial behaviour, generate a total difficulty score classified as SDQ ‘normal’ (i.e., a score between 0-13) or SDQ ‘borderline’ and ‘abnormal’ (i.e., a score between 14-40).
Training and supervision
This study was carried out through an existing partnership between War Child Holland (WCH), an international non-governmental organisation (NGO) and the Eastern Self-reliant Community Awakening Organisation (ESCO), a local NGO. Community gatekeepers with regular interaction with children and families participated in a two-day training by the research coordinator (RC; PP). The training covered a basic introduction to child and adolescent mental health, the use of the CCDT and ethical considerations related to proactive case-detection such as confidentiality, stigma and child safeguarding.
A master trainer and child psychiatrist (JVK) with extensive experience in conducting the MINI-KID trained a supervising psychiatrist (JJ) and back-up psychiatrist for three days. The supervising psychiatrist subsequently trained five senior community counsellors (four female, and one male) for five days to administer the MINI-KID and the SEEK. Supervision meetings were held with the counsellors for quality control and to support with referrals. Ten research assistants (RA) were trained for six days in research basics, ethics, informed consent and assent procedures, the SDQ, and data management. All research team members were trained in an adverse events reporting mechanism and the supervising psychiatrist followed up on children and families in need of immediate assistance.
Participants and procedures
Community gatekeepers in this study were all female, older than 18 years and included youth club leaders (n=11), women society group members (n=22) and community health volunteers (n=12). Youth club leaders organise recreational and awareness-raising activities in their village, women society group members mobilise women to improve their social and economic conditions, and health volunteers assist midwives and medical health officers to organise monthly health clinics and conduct home visits. They used CCDT for six months during their daily routine activities and detected a total of 238 children aged 6-18 years. After obtaining informed consent and assent, a study ID was created, the TQS was administered and an appointment with the counsellor was arranged by the RA. Within two weeks of identification, the counsellor met with the family at their home or at any other convenient location to conduct the clinical interview and the family assessment. Children aged 13-18 years were interviewed individually, younger children in the presence of their caregiver. The RA followed up within two days after the counsellors’ visit to administer the SDQ with the same caregiver. Direct contact between the counsellors and gatekeepers was limited for potential confirmation bias.
Ethics
Ethical approval was obtained through the Ethics Review Committee of the Faculty of Health-Care Sciences at the Eastern University in Batticaloa. Divisional and district level approval was obtained before the start of this study. Prior to official informed consent and assent procedures, the gatekeepers asked the caregiver whether they were interested in participating in a research study. The identification card was only completed for those families that were willing to participate. All children and families were informed about available and free of charge support, regardless of their participation in the study. Help-seeking was only encouraged, never imposed.
Analyses
The CCDT results and outcomes of the MINI-KID, SEEK and SDQ were analysed using the Statistical Package for Social Sciences (SPSS version 19.0). The interrater reliability (IRR) of the MINI-KID among the five counsellors was assessed using Krippendorff’s alpha for dichotomous variables [30]. The IRR was calculated for a selection of the screener, diagnostic, indication for treatment items, and total of these items.
The accuracy of the CCDT was assessed through the Positive Predictive Value (PPV), which is calculated as the percent of children and families detected using the CCDT (i.e., probable positives) who are in need of mental healthcare based on the clinical interview and two separate criteria. The primary criterion was the indication for treatment. The secondary criterion was a diagnosis of a psychiatric disorder.
The primary outcome was the PPV for all CCDT positives, regardless of the vignette used, assessed against the indication for treatment. The secondary outcome was the PPV for the subsample of CCDT internalising or externalising positives against diagnostic criteria. CCDT positives detected using the family vignette or cases detected with multiple vignettes were excluded from this subsample because diagnosis of a mental disorder is not applicable to these cases. Exploratory analyses were done to assess the differences in PPV for each individual vignette (i.e., internalising, externalising and family vignette), for each gatekeeper group separately (i.e., youth club leaders, women society group members and community health volunteers), for different age groups and gender against the indication for treatment. CCDT internalising positive cases were also compared against selected MINI-KID modules representing anxiety, depressive and somatic symptoms and the CCDT externalising positive cases with modules related to impulsive, disruptive conduct, and substance use symptoms. The Negative Predictive Value (NPV), the proportion of the CCDT negative cases that were not in need of mental healthcare, was calculated against both criteria.
Since our main sample only included CCDT positives, we could not establish the concurrent validity with a correlation coefficient. It was therefore assessed as the proportion of agreement between the CCDT positives and the SDQ positives, i.e., borderline and abnormal scores. As with other studies in Sri Lanka, we used the internationally applicable original three-band cut-off scores for the SDQ [29]. Additionally, the PPVs of the CCDT were compared to the PPV of the SDQ against the indication for treatment criterion.