Validation of the Arabic Version of the Refugee Health Screener-13 Among Perinatal Syrian Refugees in Lebanon

Background: Refugee mothers are at a significantly increased risk of suffering from perinatal depression. However, available tools to screen for it often use a western paradigm of mental health, may not be culturally appropriate, and may not account for the background stress experienced by refugees. The objective of this study is to test and validate RHS-13, which was developed by consultation with refugees from different backgrounds, among a group of Syrian refugee mothers living in Beka’s valley in Lebanon. Methodology: The tool was tested on a sample of 103 women. Internal consistency of the items for each subscale was assessed using Cronbach’s alpha. An interitem correlation was performed to examine the most correlated items on the scale. Pearson correlation coefficient was calculated between each subscale and its diagnostic proxy. To find the best fit cut-point between sensitivity and specificity, ROC curves were used. Results: RHS-13 is a reliable and valid tool to detect depression and anxiety among perinatal Syrian mothers (Internal consistency 0.803) (correlation with PHQ: 0.63), (correlation with GAD-7: 0.73). The best fit cut-points were 12, 15, and 25 indicating the severity of symptoms from mild to severe, respectively. Conclusion: RHS-13 is a valid tool among perinatal refugees in Lebanon and can be used by primary care physicians, mental health providers, social workers working with refugees from the Middle East to screen mothers for depression and anxiety.


Background
Perinatal depression is the most common complication of childbirth with a global prevalence of 13% (O'hara & Swain, 1996). This prevalence is almost quadrupled (40%-50%) among refugee mothers who suffer from major lifetime stresses including displacement, poverty, and marginalization (Dennis, et al., 2016;Mohammad et al., 2018). Some studies attribute part of the discrepancy between the global prevalence and among refugee groups to the inaccuracy of the measurement tools (Dadi et al., 2020).
Many mental health tools used in humanitarian emergencies are adapted from western countries and are not validated in that population. A study that reviewed 183 studies concerning the mental health of refugees found that over 50% did not validate the instruments used (Hollifield et al., 2002). To accurately measure the magnitude of mental illness, it is essential to choose an instrument that is rigorously designed to capture the cultural aspects of the specific population studied (Ali et al., 2016;Wells et al., 2015).
The Refugee Health Screener (RHS-13) was developed by Pathway to Wellness through consultation with different refugees and was translated and validated in seven languages (Hollifield et al., 2013). It was designed to screen newly resettled refugees and asylum seekers in the United States for emotional distress and common mental health problems (depression, anxiety, and PTSD). It was adapted by other western countries such as Germany and Sweden for screening their refugees. It was found valid and appropriate for use among refugees who speak Arabic, Nepali, Karen, Burmese, Russian, Spanish, and Somali (Hollifield et al., 2016). The majority of uses for this instrument have been for refugees resettled in western countries. However, Fellmeth attempted to validate the instrument among refugees at the Thai Myanmar border and found acceptable psychometric properties among pregnant women (Fellmeth et al., 2018).
Although the population and the context may differ widely between Lebanon and Thailand, this study shows promise that this tool could be adapted to other low-resource settings. The use of this instrument, mostly in western countries, is not surprising. In fact, much of the available research on refugees' mental health is derived through the lens of refugees resettled in western countries (Fazel et al., 2005;Morina et al., 2018). However, those refugees are systematically different from refugees still left behind for three main reasons. First, the resettlement process considers health conditions, the number of members in the family, ages, and the presence of relatives in the country of resettlement.
Second, the psychological nature of those who apply for asylum and decide to begin a life in another country may be systematically different from those who do not (An Overview of U.S. Refugee Law and Policy Humanitarian Protection Refugee Status, 2020). Third, the challenges of the daily lives of refugees in low resource settings, including extreme poverty, crowdedness, and risk of violence, are no longer experienced by refugees resettled in western countries. All these differences are vital factors that could impair the already established reliability.
In this research, we chose to test the Refugee Health Screener with its shorter version (RHS-13) for the following reasons: 1) It was developed with consultation from different groups of refugees from different backgrounds 2) It is short and simple to administer by lay health workers 3) It is validated among Arabic-speaking refugees in multiple western countries (USA, Germany) 4) It captures the anxiety and PTSD in addition to depression (Bjärtå et al., 2018;Hollifield et al., 2013). The last point overcomes one of the main challenges in assessing mental health among refugees, which is the non-specific symptomatology. Previous studies have found that refugees' mental health symptoms are often not characteristic of one psychiatric disorder, as defined in its western definition, but multiple common mental illnesses (Hollifield et al., 2016). The RHS-13 instrument was not explicitly developed for perinatal depression. However, we argue that it can be helpful in this situation because the only distinction between perinatal depression and major depression in the DSM-V is the timing of occurrence (during pregnancy or after giving birth) (American Psychiatric Association, 2013). In this research, we will test the reliability and validity of this tool among a group of perinatal women in Lebanon.

Data Collection:
The data was collected in collaboration with Multi-Aid Programs (MAPS) Organization in Lebanon in April-May 2021 in Beka'a area, Lebanon. A sample of 103 mothers were phone interviewed by three social workers at MAPS. The phone numbers of participants were collected from MAPS health care clinics. MAPS provided a list of all women who gave birth in the past year (April 1, 2020 -April 1, 2021). Then names were randomly selected using a simple random sample. Social workers communicated with participants through the phone and obtained verbal consent, conducted the survey, and entered the information using Kobo toolbox at the same time of the interview (KoBoToolbox, n.d.)

Statistical Analysis:
Demographics and basic characteristics of the samples were reported based on the severity level, means, and percentages. To compare the different groups for the categorical variables, Chi-square and Fisher's exact tests were used, and for scale values, one-way ANOVA was used (Kim, 2017;Park, 2009). Interrater reliability between each social worker was tested for RHS-13 using Kappa statistics (McHugh, 2012). Cronbach's alpha was used to assess internal consistency of the items for each subscale (depression, anxiety, PTSD). An alpha score of 0.7 was considered acceptable for all. An interitem correlation was performed to examine the most correlated items on the scale. The correlation was calculated between each item and other items in the scale. Pearson correlation coefficient was calculated between each subscale and its diagnostic proxy (Benesty et al., 2009). Diagnostic proxies used were the Patient Health Questionnaire-9 (PHQ-9), the Generalized Anxiety Disorder 7 (GAD-7) and Primary Care PTSD-5 (PC-PTSD-5). To find the best fit cut-point between sensitivity and specificity, ROC curves were used. After establishing cut-points for each severity level, the cut-points which provided higher sensitivity and specificity were selected. IBM SPSS was used for data analysis (IBM Corp, 2016).

Results
Overall, 75% of the sample scored positive with a cut-point of 11. The highest proportion of participants were between 20-25 years (38%  Note. N = 103. Participants were on average 27 years old (SD = 10.1), and participant age did not differ by condition.

Reliability testing
The interrater reliability for the different social workers (data collectors) is acceptable at 0.72 with a significance level of <0.001. Kappa statistics are demonstrated in Table 1. The calculated Cronbach's Alpha level for the (RHS-13) in this sample is 0.80, while it is 0.78 for PHQ-9, 0.76, and 0.49 for PC-PTSD-5. The summary of Alpha statistics for RHS-13 and its diagnostic proxies are shown in Table   The highest correlation between the two items is between item 2 (feeling down, sad, or blue most of the time) and item 4 (feeling helpless). Table 5 shows the inter-item correlation between each item and other items.  The correlation of each item with all other items combined is >0.4 for items (1,2,3,4,5,7 and 8). The lowest correlation is of element nine (crying easily). The value of Cronbach's Alpha did not increase when any of the items were deleted. Table 6 shows the correlation between each item and all other items combined. The highest percentage of mothers answered 'not at all' to the following elements: element one (muscle, bone, joint pains) (33%), element five (suddenly scared for no reason) (64%), element six (faintness, dizziness, or weakness) (64%), element seven (nervousness or shakiness inside) (37%), element eight (feeling restless) (64%), element ten (had the experience of reliving the trauma) (36%), element eleven (having physical reactions when reminded of the trauma) (56%), element thirteen (jumpier, more easily startled) (73%).
The highest percentage of mothers answered 'quite a bit' to the following elements: element two (feeling down, sad, or blue most of the time) (30%), element three (too much thinking), 49%, element four (feeling helpless) (31%), element nine (crying easily) (37%). Table  Half of the participants answered that they could handle a few things but could not cope with others. At the same time, very few reported that they are not able to cope with most things (9%) or not able to cope with anything (14%). Table 8 shows the distribution of answers for the resilience question.

RHS-13 correlates well with the depression scale (PHQ-9) and the anxiety scale (GAD-7)
with a score of 0.63 and 0.73, respectively. However, it has a weak correlation with PTSD (PC_PTSD5) (0.43). Table 9 shows the correlation between RHS-13 and its diagnostic proxies.   Table 10 shows the area under the curve for RHS-13 in comparison to the severity levels of the diagnostic proxies. Figure 1 depicts the ROC curves for RHS-13 in contrast to the severity levels of the diagnostic proxies.
We chose three cut-points, 12, 15, and 25 indicating the severity of symptoms from mild to severe, respectively. The previously established cut-points by Bjärtå et al. (2018) of 11 and 18 did not seem to be the best cut-points. The first cut-point (11), which was established for mild cases, showed a better balance between sensitivity and 1-specificity (0.83 and 0.17) when compared to mild depression than when compared to anxiety (0.79 and 0.47). Similarly, the second cut-point, which was established for moderate cases (18), showed a good balance for depression (0.70 and 0.16), while for anxiety, it was less favorable (0.56 and 0.26). Using the newly established cut-points, the highest prevalence of symptoms in this sample was for moderate level distress which was indicated by a score of 15-24, followed by normal, which was indicated by a score of <12. Table 11 shows the prevalence of each level of distress.   Table 12 shows a summary of our key results compared to other studies. The internal consistency (Cronbach's alpha) for the Refugee Health Scale (RHS-13) in our sample was (0.803), which was lower than that reported by Bjärtå (0.92) and Hollifield (0.96) but higher than Fellmeth, (0.63 in Burmese; 0.56 in Sgaw Karen) (Bjärtå et al., 2018;Hollifield et al., 2013;Fellmeth et al., 2018). Similarly, the correlation between RHS-13 and the diagnostic proxies in our sample (Depression=0.63, anxiety=0.73, PTSD=0.49) was lower than what was reported by Bjärtå (Depression=0.82,anxiety=0.82, PTSD=0.56) despite using the same diagnostic proxies (Bjärtå et al., 2018). Our established cut-point of 12 for mild level distress differed from the previously established cut-point of 11, by Hollifield and Bjärtå, 13 by Kaltenbach and 14 by Fellmeth (Bjärtå et al., 2018;Hollifield et al., 2013;Kaltenbach et al., 2018;Fellmeth et al., 2018).
The variation in the scale's psychometric properties between our study and previous studies could be attributed to three main reasons. The first is the language; although some of these studies included Arabic-speaking individuals, variation in dialect between different countries (e.g., Syria and Iraq) might have affected the internal consistency. The second is increased homogeneity in our sample, and our sample is focused on a specific group of women who share similar living circumstances and are of a similar age group, while other studies were inclusive of persons from a wider spectrum of ages and backgrounds. The third point is the administration method, and some studies used a self-administrative survey while we used it as a phone interview-based survey. Due to mobility constraints of the pandemic and the low literacy level expected among Syrian refugees, an interview-based survey was more feasible for this study. Despite the differences, we concluded that the RHS-13 showed acceptable reliability when administered as a phone interview by lay health care workers in the community of Syrian refugee mothers.
Approximately 70% of the participants in our study were positive for mild distress. This high prevalence was also found among refugees awaiting asylum in Germany (77%) (Bjärtå et al., 2018). However, studies of refugees already resettled in western countries found a much lower prevalence (38%) in the United States and (42%) in Germany (Hollifield et al., 2016;Fellmeth et al., 2018). Interestingly, Hollifield et al. (2016) found the highest prevalence among Arabic-speaking Iraqi refugees (58%). When comparing with women in the perinatal period, we found that our result was much higher than that of Fellmeth, 2018 (44%), which studied pregnant refugees in the Thai Myanmar region. These disparities could be attributed to the differences between populations and the stress levels experienced in various settings. In Bjärtå's study, the higher prevalence of distress reflected the urgency and high stress among refugees who fled their countries but have not yet been granted asylum (Bjärtå et al., 2018). Similarly, in our study, the high prevalence reflects the economic crisis and the stressful situation the Syrian mothers are experiencing during the pandemic.
Overall, our findings correspond with previous studies showing a high prevalence of perinatal depression among Syrian refugees in Lebanon and Jordan (74% and 50% respectively) as well as in Canada (58%) (Stevenson et al., 2019;Mohammad et al., 2018;Bowen et al., 2017).
Although some of these studies differ in the use of instruments and cut-points, it generally reflects a much higher level of perinatal depression than what is recorded as the prevalence in lower-middle-income countries (20%-25%) and what was estimated for immigrant mothers in Canada (8-10%) (Dadi et al., 2020;Ganann et al., 2016). The differences between refugees in western countries and those living in lower/middle -income countries are considerable and systematic as asylum and resettlements options are only given to a small percentage of refugees and according to a very specific set of conditions. This highlights the importance of our research in testing and validating useful tools to be used by humanitarian and development agencies. The greatest strength of our study is that, to our knowledge, it is the first to validate the RHS-13 among Syrian refugees in a in lower/middle -income country. Using this instrument which addresses three common mental health disorders (depression, anxiety, and PTSD), is more appropriate given the complexity of the situation faced by refugee mothers. Having this instrument validated and ready to use has major implications for aid organizations and mental health providers in Lebanon, Jordan, Turkey, and Egypt. Furthermore, the use could be extended for other Arabic-speaking refugees, and conflict-affected populations such as Iraqis and Yemenis.
Limitations of this study included the use of phone interviews rather than face-to-face interviews. However, due to the pandemic and social distancing precautions, this approach was the most feasible. Another limitation is the use of verbal administration of RHS-13 rather than the self-completed version. We thought this method was the most appropriate because of the expected low literacy level and to ensure data quality. This method was endorsed by Hallifield and used by Bjärtå (Bjärtå et al., 2018;Hollifield et al., 2013) Additionally, the screening instrument (RHS-13) was not developed to screen for postpartum depression, and for this, comparing the prevalence of postpartum depression to our study should be considered.
Moreover, the diagnostic proxies were not the gold standard. However, given the difficulty of finding mental health providers to conduct the diagnostic interviews and as previous studies suggested, of the same validation proxies by Bjärtå, the rendered cutoffs can be valuable as a general screening tool in community settings (Bjärtå et al., 2018). Finally, although RHS-13 appears to be a valid instrument to assess depression and anxiety among Syrian mothers in Lebanon, its generalizability to other groups such as men and elderly women is not known.
Further validation studies are required to use the instrument among all Syrian refugees.

Ethics approval and consent to participate
The study was approved by the Institutional Review Board at the University of Nebraska Medical Centre, Omaha, Nebraska, USA (IRB#:0787-20-EP) Informed consent was taken from all participants.

Consent for publication
Not Applicable

Availability of data and materials
The data that support the findings of this study are available on request from the corresponding author NA. Data are not publicly available due to them containing information that could compromise research participant privacy.