Identification and selection of studies
Search strategy
A review protocol was developed based on the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA)-statement (www.prisma-statement.org) (27). The protocol for this meta-analysis was registered at PROSPERO (CRD42018091080).
We conducted a comprehensive systematic literature search in the bibliographic databases PubMed, Embase.com and PsycINFO up to 20th December 2018, in collaboration with a medical librarian. Detailed search strategies for these databases are given in the appendix A. The following terms were used (including synonyms and closely related words) as index terms or free-text words: “Social stigma”, “Common mental disorders”, “Mood disorders”, “Anxiety disorders”, “Depression”, “Stress disorders”, “Migrants”, “Minority groups”. The search was performed without date or language restriction. Search strategies for other databases were built accordingly. We also checked the references of the included studies to identify additional relevant studies (see figure 1 for the Prisma Flowchart).
Inclusion criteria
The searches were limited to the following criteria: 1) Peer-reviewed papers, 2) Racial minorities (i.e. defined based on the classification of the country of the included studies) 2) Racial majorities (i.e. defined based on the classification of the country of the included studies) 3) Adults aged 18 and above and 4) Participants with/without common mental disorders (i.e. common mental disorders are identified as depression and anxiety spectrum disorders), 2) Empirical studies with cross-sectional designs measuring mental illness stigma about common mental disorders among racial minorities in comparison to majorities 3) Studies were not limited to the European populations only and studies carried out in other continents were included.
Exclusion criteria
The exclusion criteria: 1) Publications focusing on stigma about help-seeking, HIV, physical disorders or sexual minorities (if they are not from a racial minority group), 2) Publications focusing on stigma about severe mental health disorders (e.g. schizophrenia), 3) Empirical studies without a comparison group (studies which are not comparing different racial groups were excluded), 4) Qualitative studies and 5) Publications focusing on adolescent and children sample.
Quality assessment
We assessed the quality of the included studies using the Effective Public Health Practice Project Quality Assessment Tool (EPHPP). This tool assesses possible sources of bias in observational studies and RCTs. Since we have included cross-sectional studies, the following domains of the tool were used in this study: (1) selection bias; (2) study design; (3) confounders; (4) data collection method and; (5) analyses. The studies received an overall assessment in one of the following: 1) high risk of bias (studies which scored high risk of bias in 3 or more of the assessment domains); 2) moderate risk of bias (studies which scored high risk of bias in 2 of the assessment domains) and; 3) low risk of bias (studies which scored high risk of bias in 1 of the assessment domains). Assessment was carried out by two independent assessors and disagreements were solved through discussions.
Data extraction
A customised data extraction form was generated and included the following characteristics: Method of recruitment into the study (community, clinical samples or other recruitment type), target group (adults in general, older adults, student population or other target group), types of stigma perspectives, types of stigma mechanisms and types of outcome measures used to measure stigmas (self-report instruments, vignettes). Vignettes are case descriptions of an individual, presenting symptoms of a CMD (22) (see Appendix B for an example).
Covidence, online software for screening and data extraction for systematic reviews and meta-analyses, was used for the review and extracting data. First, the results of the online database searches were imported to covidence. Two reviewers had personal accounts and selected papers independently in a random order. A third reviewer carried out the reference list search of the selected papers. The titles of all studies were screened, and the abstracts of the studies were checked regarding the inclusion criteria. When no definitive decision could be made based on the abstract, the original papers were used. Discrepancies between the reviewers' selections were resolved through discussions. If not resolved, the opinion of a fourth researcher was sought. The corresponding author filled in the extraction form.
Conceptual frameworks
We conceptualised mental illness stigma based on the MISF (11). The types of perspectives, measured by the studies, were categorised into three groups: 1) the perspective of the stigmatizer (i.e. Public attitudes and beliefs that other people devalue or discriminate against individuals with mental illness. The specific components are: stereotype, prejudice and discrimination); 2) The perspective of the stigmatized (i.e. Personal beliefs, attitudes and perceived anger for having a mental illness. The specific components are: experienced stigma, anticipated stigma and internalized stigma) and; 3) The perceived stigma (i.e. shared experiences of stereotypes, prejudices and discrimination between people who stigmatize and who are stigmatized).
The concept of race-ethnicity was defined based on the minority or majority classifications of the country of the included studies. Because there were many ethnic groups (e.g. Chinese, Indian) within the included studies, we decided to use broad racial categories (e.g. Asian, Black) in order to make the studies comparable (28). Six racial groups were identified in consultation with the categories previously defined by Ünlü İnce and colleagues (29). These categories were: Black (African background), Asian, Hispanic (Latin American and Spanish background), Native American (referring to the indigenous people of North America), and White (Caucasian and white European) and other (people from racial-ethnic minority group who could not be identified in one of these categories).
Outcome
For each comparison between ethnic groups in stigma, the effect size indicating the difference between groups was calculated (Hedge`s g). Effect size of 0.8 was accepted as large, effect size of 0.5 was accepted as moderate and effect size of 0.2 was accepted as small (30). Effect sizes were calculated by subtracting the means of stigma between racial minorities and majorities and dividing the result by the pooled standard deviation. If means and standard deviations were not reported, we used the procedures of the Comprehensive Meta-Analysis software (see below) to calculate the effect size using dichotomous outcomes; and if these were not available either, we used other statistics (such as t-value or p-value) to calculate the effect size.
In order to calculate effect sizes we used all the self-report measures and vignettes examining mental illness stigma (see table 1 for the outcome measures) [such as Perceived Devaluation and Discrimination Scale (PDD) (31), Internalized Stigma of Mental Illness Scale (ISMI) (32), Community Attitudes to Mental Illness scale (CAMI) (33)]. The decision on which outcome measure is capturing which specific stigma perspective and mechanism was based on the Fox and colleagues` classification (11).
Analyses
To calculate pooled mean effect sizes, we used the computer programme Comprehensive Meta-Analysis (version 3.3070; CMA). We expected considerable heterogeneity among the studies for various reasons. First, our definition of racial minorities and majorities were too broad and did not capture specific ethnic minorities and majority groups within the samples. Second, we pooled the studies employing different outcome measures (self-report and vignette). Third, the included studies are investigating different stigma perspectives and mechanisms associated with them. Fourth, the included studies often investigated mental illness stigma for various common mental illnesses (both anxiety and depression spectrums) and lastly, we included studies from both High Income and Low and Middle Income countries which have variations in how mental illness stigma is defined, measured and experienced. In the light of these, we employed a random effects pooling model in all analyses.
As a test of homogeneity of effect sizes, we calculated the I2 statistic, which is an indicator of heterogeneity in percentages. A value of 0% indicates no observed heterogeneity, and larger values indicate increasing heterogeneity, with 25% as low, 50% as moderate and 75% as high heterogeneity (34). We calculated 95% confidence intervals around I2 (35) using the non-central chi-squared based approach within the heterogi module for Stata (36).
We tested publication bias by inspecting the funnel plot on primary outcome measures and by Duval and Tweedie`s trim and fill procedure (37) yields an estimated effect size after publication bias has been taken into account (as implemented in CMA). We also conducted Egger`s test of the intercept to quantify the bias captured by the funnel plot and to test whether it was significant.
We also examined whether specific characteristics of the studies were related to the effect sizes. We conducted subgroup analyses according to the mixed effects model, in which studies within subgroups are pooled with the random effects model, while the tests for significant differences between subgroups are conducted with the fixed effects model. The priori decided sub-groups were: ethnicity, type of stigma outcome, type of stigma perspective and the quality of the studies. Further, we used multi-variate meta-regression analyses as implemented in CMA.