Construct Validity of the Psychosis Screening Questionnaire in Ugandan Adults

Background: Psychotic disorders are common and contribute significantly to morbidity and mortality of people with psychiatric diseases. Therefore, early screening and detection may facilitate early intervention and reduce adverse outcomes. Screening tools that lay persons can administer are particularly beneficial in low resource settings. However, there is limited research evaluating the validity of psychosis screening instruments in Uganda. We aimed to assess the construct validity and psychometric properties of the Psychosis Screening Questionnaire (PSQ) in Uganda in a population with no history of a psychotic disorder. Methods: The sample consisted of 2101 Ugandan adults participating as controls in a larger multi-country case-control study on psychiatric genetics. We used confirmatory factor analysis (CFA) and item response theory (IRT) to evaluate the factor structure and item properties of the PSQ. Results: The overall prevalence screening positive for psychotic symptoms was 13.9%. “Strange experiences” were the most endorsed symptoms (6.6%). A unidimensional factor was the best fitting model based on the fit indices including the root mean square error of approximation (RMSEA of 0.00), comparative fit index (CFI of 1.000), and Tucker-Lewis Index (TLI of 1.000). The most discriminating items along the latent construct of psychosis were items assessing thought disturbance followed by items assessing paranoia, with a parameter (discrimination) value of 2.53 and 2.40, respectively. Conclusion: The PSQ works well in Uganda as an initial screening tool for moderate to high-level of psychotic symptoms.


Introduction
Psychotic disorders, including schizophrenia spectrum and bipolar affective disorders, are chronic severe mental illnesses that contribute signi cantly to high morbidity (years lived with disability) and mortality, mainly due to suicide risk [1]. Moreover, studies have demonstrated that people with psychotic disorders have an increased risk of cardiovascular disease [2], metabolic syndrome [3] and diabetes mellitus [4], which all predict premature mortality and can negatively impact quality of life [5].
The prevalence of psychotic disorders has been shown to vary widely worldwide with estimates between 0.8-31.4% [6]. However, many people have psychotic symptoms without a psychotic disorder and these too are associated with signi cant distress and impairment in personal, family, social, educational, occupational, and other important areas of life [7]. Research points to a higher prevalence of psychotic symptoms among individuals from low-resource settings [7]. For example, a study done in Dar-es-Salaam, Tanzania, showed the prevalence of psychotic symptoms to be 3.9% [8], and in another in a rural Kenyan setting, the prevalence of psychotic symptoms was found to be 8.1% [9] .
Early screening for psychosis may facilitate early detection and prompt treatment for high-risk populations. In low-resource settings like Uganda, with a small number of mental health practitioners, screening tools that laypersons can administer are needed [10,11]. Several instruments have been used to assess and screen for psychotic disorders, including self-report questionnaires, such as the Psychosis Screening Questionnaire (PSQ). Assessing the psychometric properties of PSQ in the Ugandan setting would help screen high-risk populations for psychosis, which could then be followed up with a diagnostic assessment.
The PSQ is preferred by clinicians and researchers because of its brevity and reliability and has been used with ethnic and cultural minorities in high-income countries. The PSQ was tested for equivalence in the United Kingdom in a study across ve different ethnic groups in reporting psychotic symptoms [12]. It has also been adopted and used in Kenya [9], Tanzania [8] and Ethiopia [13].
There have been no studies in Uganda to date, to our knowledge, that examine the psychometric properties of the PSQ outside of our group. Our group has published a cross-cultural examination of the PSQ across Uganda, Ethiopia, Kenya, and South Africa [1]. However, this prior study was focused on a broad comparison of the scale's performance across the four countries to test its equivalence across settings, without focusing on the speci cs of the performance of PSQ from each country. The goal of this study was to examine the performance of PSQ in Uganda in depth, including ne grained analyses at the item level to understand its cultural relevance for the setting. Therefore, this study aimed to investigate the construct validity of the PSQ by exploring factor structure and item properties through item response theory analyses in Uganda with adults from a general medical setting.

Method
We utilized data from the Neuropsychiatric Genetics of African Populations-Psychosis (NeuroGAP-Psychosis) study in Uganda, Ethiopia, Kenya, and South Africa. NeuroGAP-Psychosis is a case-control study aimed at expanding the understanding of genetic and environmental risk factors for psychotic disorders across different African populations, for the purposes of the current study, we analyzed data from Uganda only.

Participants And Study Procedure
Study participants for the current study consisted of participants without psychosis (i.e., controls) in Uganda who were recruited between February 2018 and March 2020. Participants were individuals seeking outpatient general medical care, caretakers of individuals seeking care, and staff or students working at general medical facilities. Only controls were included in this study because patients with clinical diagnoses of psychosis (i.e., cases) were not administered the PSQ. Participants were recruited from the following medical facilities: Butabika National Mental Health Referral Hospital, Naguru, Arua, Mbarara, and Gulu Regional Referral Hospitals. Inclusion criteria for controls were age 18 years or older and able to provide consent. Individuals were excluded who had acute levels of alcohol or substance use, including being under inpatient hospitalization or acute medical care for one of these conditions. Ethical approval was obtained from all participating sites, including the Makerere University School of Medicine

Psychosis Screening Questionnaire
The presence of psychosis was assessed using the Psychosis Screening Questionnaire (PSQ), a selfreported brief screening instrument designed to detect psychotic symptoms. The PSQ has ve primary (root) questions that assess the presence of psychotic symptoms: mania, thought-interference, paranoia, strange experiences, and hallucinations. Endorsement of any of the primary questions are followed by one to two secondary questions to further screen for psychotic experiences. The original PSQ assesses symptoms in the past year, but for our purposes we focused on lifetime symptoms "ever" in one's life. We Standard sociodemographic variables were collected, including age, sex at birth, level of education, marital status, and current living situation. Participant characteristics were described using means and standard deviation for continuous variables and counts and percentages for categorical variables. Prevalence estimates of psychotic symptoms were also calculated. All study participants from Uganda were included in all analyses, but three individuals from the total sample had missing data on the PSQ and were excluded from the below listed analyses.

Con rmatory Factor Analysis
We examined the construct validity and factor structure of the PSQ by conducting con rmatory factor analysis (CFA) in Mplus 8 v.1.7. We tested a unidimensional factor structure based on past research and theory for the measure [14] including a study that reported one latent factor on a multi-ethnic British sample comparing PSQ's equivalence across groups [12]. A split sample exploratory factor analysis was not possible due to a oor effect in our sample of controls with a low prevalence of psychotic disorders. Our model t was calculated with a weighted least square mean and variance adjusted (WLSMV) estimator for categorical data, and measurement error was not assumed to be correlated among items.

Item Response Theory
The study further explored the factorial validity of the PSQ for Ugandan adults with item response theory (IRT) to better understand the relationship between the latent trait of psychosis and items on the PSQ. IRT accounts for how each item measures the latent construct and individual variation across the construct's severity levels. IRT uses two main parameters, item discrimination and item di culty, to describe the relationship between the participant, the latent construct (psychosis), and each PSQ item. The discrimination (or ) parameter describes the ability of each item to distinguish between degrees of psychotic symptom severity. The item di culty (or ) indicates the location along the psychosis latent construct at which individuals have a ≥ 50% likelihood of endorsing a particular item.
We examined the assumptions required for an IRT model: unidimensionality, local independence, and monotonicity. The unidimensionality assumption was assessed by examining a one-factor CFA model, while the monotonicity was investigated via Mokken scaling analysis. After checking the assumptions, a 2-Parameter Logistic model was tted using a unidimensional latent structure. Item information curves (IICs), item characteristic curves (ICCs), and the total information curves were generated using the R statistical program, version 3.6.2, packages Mokken and ltm.

Demographic characteristics
The nal analytic population consisted of 2,104 adults (Table 1, includes three individuals with missing data), with more participants who identi ed as female (56%) than male (44%). A large proportion of the participants were between the ages of 30 and 44 (40%), and a majority reported being married or cohabiting with a partner (53%). In addition, 42% reported attending at least some secondary school, and 32% reported nishing primary school.  The overall prevalence of individuals screening positive for at least one psychotic symptom in their lifetime in our sample was 13.9%. Strange experiences was the most endorsed item (6.6%), followed by paranoia (5.1%), hallucinations (4.9%), and thought interference (4.1%). Mania was the least endorsed symptom (1.4%).

Con rmatory Factor Analysis
We carried out a con rmatory factor analysis to examine the construct validity and factorial structure of the PSQ to assess model t for a unidimensional model. The model t indices and standardized factor loadings of each item are presented in Table 2. Indicators of goodness of t, including RMSEA, CFI, and TLI indicate excellent t (RMSEA of 0.00, TLI of 1.000 and CFI of 1.000) for a unidimensional model of the PSQ. The perfect model t for categorical data was likely due to skewed binary indicators (i.e., oor effect) where we did not have su cient power to reject the null model (Hu & Bentler, 1999;Methuen, 2014). Furthermore, all items loaded strongly (ranging from 0.72 to 0.79) on the unidimensional model of PSQ and provided additional evidence for a good model t. The item with the weakest loading on the factor, although still strong, was mania (0.72).

Item Response Theory
The IRT discrimination ( ) and di culty ( ) parameters for each of the ve PSQ items are presented in Table 3, and the IICs, ICCs, and test information function curves are found in Fig. 2. The most discriminating items along the latent construct were thought disturbance followed by paranoia, with a parameter (discrimination) values of 2.53 and 2.40, respectively. This is shown visually in the item information curves (Fig. 2a), where thought disturbance has the highest peak, followed by paranoia. Therefore, these two items (thought disturbance and paranoia) might best discriminate between participants with higher levels of psychotic experiences severity and participants with lower levels of psychotic experiences severity. The least discriminating item was strange experiences ( ), indicating that this item does not discriminate as effectively between participants at high and low levels of psychotic experiences relative to the other items. Overall, the items tend to discriminate between participants showing higher levels of psychotic experiences severity, given that the item information function peaks at the upper level of the latent construct (Fig. 2c). The test information function indicates that the PSQ is useful for screening moderate to high levels of psychotic symptoms rather than average or below-average levels. The ICCs visually represent the probability of endorsing items by the underlying construct severity (Fig. 2b). The ICC plot indicates that the probability of endorsing mania is highest when a person's psychotic experience severity is high. In contrast, the other four items are similar in their probability of being endorsed at slightly moderate psychotic experiences levels, with strange experiences being the least di cult.

Discussion
This study was one of the rst to examine the psychometric properties of the PSQ in a Ugandan sample. The lifetime prevalence of psychotic symptoms in our sample was 13.9%, with strange experiences as the most endorsed item with a prevalence of (6.6%) and mania as the least endorsed symptom (1.4%). All items loaded strongly (ranging from 0.72 to 0.79) on a single psychosis latent factor measured by the PSQ and provided evidence for a good model t. The IRT analysis indicated that the PSQ may provide more information for higher-than-average psychosis levels and that the PSQ will more likely identify individuals with a high level of psychosis compared to those with low levels.
Given that our study was one of the rst of its kind in Uganda, the prevalence of lifetime psychotic symptoms of 13.9% found in our study cannot be compared with previous research in Uganda. The only other study in Uganda that used the PSQ outside of our reports from the NeuroGAP-Psychosis study noted a prevalence of 63% in rst psychosis episode patients at Butabika National Psychiatric Referral Hospital [17]. However, this study does provide an adequate comparison since it was based on a retrospective chart review of people presenting for the rst time speci cally for mental health disorders while our study consisted of people with no history of psychotic disorders. In a study done in a Chinese b α adult population found a lifetime prevalence estimate of 5.5% for positive screens on the PSQ for the rst phase of their study [18]. This is lower than the one found in our setting, and this could potentially be due to cultural differences and the different meanings attached to the different psychotic symptoms between the two settings.
On the con rmatory analysis, the PSQ performed well as a unidimensional construct. These ndings are aligned with prior theory and research on the PSQ in high-income countries [7,12]. IRT analyses showed that the most discriminating items along the latent construct were thought disturbance followed by paranoia, and these items gave the most precise information regarding psychosis. This nding is comparable to a study on psychotic symptoms in the United Kingdom examining diverse ethnic groups, which showed that paranoid symptoms were the most endorsed for participants of Caribbean ancestry [12].
Strange experiences, on the other hand, was the least precise symptom for identifying positive screens compared to other items. Given that the strange experiences item was the least discriminating item for psychosis and was the most endorsed, this item may not work well to screen for psychosis in Uganda. One explanation may be the description of a strange experience varies with different cultures in Uganda, with many experiences that would be considered strange being normalized due to the popular belief in supernatural experiences such as communication with the gods [19,20]. Therefore, the prevalence proportion of 13.9% may have been overin ated by the strange experiences item.
Overall, the IRT analysis indicates that the PSQ provides valuable information about psychosis as a construct at higher levels of the latent trait, thus more accurately detecting moderate to severe levels of psychosis. This may mean that in Uganda, the measure may be better suited as a screen for individuals suspected of having mental health di culties (e.g., as identi ed by community leaders or healthcare workers). The measure may be less adept as a screen in generalist settings (e.g., as a screen administered to all primary care patients).

Limitations
This study should be understood within its limitations. One major limitation is the lack of evaluation of measurement invariance analysis by key demographic (e.g., sex at birth) and clinical characteristics due to the low prevalence of psychotic experiences. In this study, we only recruited participants in general hospital settings, and therefore the ndings may not be generalizable to other populations. However, study strengths include administering and evaluating the PSQ in an understudied population using a large sample size. Lastly, the study did not assess the criterion validity using a clinical diagnostic gold standard. Future research examining the PSQ against a clinical gold standard measure will shed further light on the measure's utility in Uganda.

Page 11/14
This is one of the rst study to assess the psychometric properties of the Psychosis Screening Questionnaire in a Ugandan population to the best of our knowledge. This study estimated a lifetime prevalence of psychotic symptoms at 13.9% in a population with no history of a psychosis spectrum disorder, with strange experiences as the most endorsed symptom. Our ndings show good construct validity and a one-dimensional structure for the PSQ in Uganda. The measure may be adept at screening individuals with higher levels of psychosis and be more helpful when administered to individuals displaying mental health symptoms for further diagnostic assessment. There is need for further studies to examine the criterion validity of the PSQ in Uganda.