Screening for symptoms of anxiety and depression in patients treated with renal replacement therapy: utility of the Edmonton Symptom Assessment System-Revised

The Edmonton Symptom Assessment System-revised (ESASr) is widely used in clinical oncology to screen for physical and emotional symptoms. The performance of the anxiety and depression items (ESASr-A and ESASr-D, respectively) as screening tools have not been evaluated in patients treated with renal replacement therapy. Kidney transplant recipients and patients on dialysis were recruited in Toronto. Patients were classified as having moderate/severe depression and anxiety symptoms using the established cut-off score of ≥ 10 on the Patient Health Questionnaire-9 (PHQ-9) and the General Anxiety Disorder-7 (GAD-7) questionnaires. This study included 931 participants; 62% male, mean age (SD) 55(16), and 52% White. All participants completed ESASr, however only 748 participants completed PHQ-9 and 769 participants completed GAD-7. Correlation between ESASr item scores and legacy scores were moderately strong (ESASr-D/PHQ-9: 0.61; ESASr-A/GAD-7: 0.64). We found good discrimination for moderate/severe depression and anxiety [area under the receiver operating characteristics curve (95% CI) ESASr-D 0.82(0.78–0.86); ESASr-A 0.87 (0.82, 0.92)]. The cut-off ≥ 2 for ESASr-D [Sensitivity = 0.76; Specificity = 0.77; Likelihood Ratio (LR) + = 3.29; LR − = 0.31] and ≥ 4 for ESASr-A (Sensitivity = 0.75; Specificity = 0.87; LR + = 5.76; LR − = 0.29) had the best combination of measurement characteristics. The identified ESASr-D and ESASr-A cut-off scores may be used to rule out patients without emotional distress with few false negatives. However, the low sensitivity identified in our analysis suggests that neither ESASr-D or ESASr-A are acceptable as standalone screening tools.

Valid, reliable patient reported outcome measures (PROMs) that have good discrimination may be used to screen for emotional symptoms [14][15][16][17]. Recent studies considered the Edmonton Symptom Assessment System (ESAS) to routinely measure symptoms experienced by patients on dialysis [18,19]. Routine symptom assessment with ESAS improved awareness of symptom burden among providers and patients, particularly for psychosocial symptoms but it had minimal impact on symptom management. One potential barrier to efficiently use ESAS may be the lack of standardized triggers for when to respond to results [18].
The ESAS has been developed for symptom assessment in patients with cancer and has been translated and validated in several languages [20,21]. It assesses nine symptoms using a single rating scale item for each symptom. The revised tool (ESASr) specifies timeframe and definitions of symptoms [20,22]. ESASr poses minimal burden to patients [21] and is used to monitor symptom burden in patients with cancer in Ontario [23][24][25] and in other jurisdictions [26][27][28]. The reliability and validity of ESASr have been assessed in patients on RRT [29][30][31].
In this study, we assessed the discrimination and screening performance of ESASr-depression (ESASr-D) and ESASr-anxiety (ESASr-A) items in relation to the Patient Health Questionnaire-9 (PHQ-9) (depression) and the Generalized Anxiety Disorder-7 (GAD-7) (anxiety) in patients treated with RRT. We also assessed the precision of potential cut off scores to identify patients with moderate/severe symptoms of depression and anxiety who may benefit from further multidimensional assessment.

Study design
This is a secondary analysis of data collected in a multicenter, cross-sectional study drawn from the COmprehensive Psychosocial REsearch Data System (Co-PreDS-CKD), a research database that we established (REB#17-5916). We used data from patients on maintenance dialysis and kidney transplant recipients (KTRs) in Toronto, Ontario who completed all ESASr items, PHQ-9 and GAD-7 questionnaires at the same time between November 2015 and Dec 2019. All patients included in the studies provided written informed consent. Patients included in the study initiated dialysis treatment at least 90 days or had their kidney transplant a minimum of 30 days prior to enrollment. Exclusion criteria were: insufficient English proficiency, diagnosis of dementia and/or severe acute medical conditions. Patients were approached during regular clinic visits or dialysis treatment. Participants completed questionnaires on iPads using an electronic data capture system (DAta Driven Outcomes System-DADOS, Techna Institute, University Health Network, Toronto). The study questionnaires also included questions about sociodemographic characteristics including age, sex, marital status, education level, income and ethnicity. Clinical data was abstracted from medical records and entered into the database after quality checks and audits. Clinical variables included time since dialysis start or receipt of transplant, blood hemoglobin, serum albumin, and estimated glomerular filtration rate (eGFR). Glomerular filtration rate (eGFR, in mL/min/1.73 m 2 ) was estimated using the Chronic Kidney Disease Epidemiology Collaboration equation [32]. Comorbid conditions were obtained from medical records to calculate the Charlson Comorbidity Index (CCI) [33].
The GAD-7 and PHQ-9 are publicly available screening tools which ask about the extent to which respondents are bothered by symptoms of anxiety and depression, respectively. These questionnaires contain 7 items and 9 items, respectively, with responses rated on a scale of 0-3 (0 "not at all bothered" and 3 "bothered nearly every day"). The total score is a summation of the item scores and ranges from 0 to 21 (GAD-7) and 0 to 27 (PHQ-9) [34,35]. The British Columbia Renal Agency has recommended the use of GAD-7 and PHQ-9 as screening tools in patients with chronic kidney disease [36]. Patients were classified as "anxious" and "depressed" based on a cut-off score of ≥ 10 on the GAD-7 and PHQ-9, respectively. These cut off scores identify patients with moderate/severe anxiety or depression [34,35].

Statistical analysis
Baseline characteristics are presented as frequency, mean, standard deviation, median and interquartile range. Floor and ceiling effects were reported as the proportion of patients with minimum and maximum possible scores, respectively. Criterion validity is assessed through correlations between ESASr-D or ESASr-A and the corresponding legacy questionnaire (PHQ-9 or GAD-7) using Spearman's rank sum correlation. Score differences between groups were assessed using Mann-Whitney U test.
Receiver Operating Characteristics (ROC) analyses were conducted to assess the discrimination of ESASr items against moderate/severe depression and anxiety defined by the PHQ-9 or GAD-7 cut-off score ≥ 10. Test discrimination is characterized by the area under the receiver operating characteristic curve (AUROC), with an AUROC of 0.75 or greater representing good discrimination [37].
We identified cut-off scores to identify "cases" with moderate/severe symptoms for further assessment using Youden's J [38]. The performance of ESASr-D and ESASr-A items was further assessed by calculating sensitivity, specificity, positive and negative predictive value (PPV/NPV), positive and negative likelihood ratio (LR +/LR −). The 95% confidence interval for optimal cut-off scores identified by the Youden index were reported using bootstrapping techniques with the "cutpt" program in STATA [39].
To further assess pre-test probability and likelihood ratios of the suggested cut-off scores were integrated using Bayesian principles to create a conditional probability plot [40].
All statistical analyses were conducted using STATA 14.0 (StataCorp, College Station, TX).

Sample characteristics
Data from 456 (49%) KTRs and 475 (51%) patients on maintenance dialysis were used in this analysis ( Figure S1). The mean (SD) age of the entire cohort was 56 (16) years and 60% of participants were male. The baseline characteristics of the participants are shown in Table 1.

Discrimination and potential cut-off score
In the ROC analysis for the entire cohort of patients on dialysis and KTRs who completed PHQ-9 (n = 748) good discrimination was noted for ESASr-D regarding moderate/severe depression [Area under ROC = 0.82, 95% CI: (0.78-0.86); Fig. 1]. Similarly, good discrimination was seen for ESASr-D when the transplant and dialysis cohorts were analyzed separately ( Figure S2A and S2B). Based on the Youden's J, the best cut-off score for ESASr-D in the entire cohort was ≥ 2 (95% CI: 0.35-2.65; Sensitivity = 0.76; Specificity = 0.77; LR + = 3.29; LR − = 0.31; Table 3). The same cut-off score for ESASr-D was identified in both sub-cohorts (dialysis and KTR), with similar NPV and PPV. Results of the threshold analysis for the entire cohort and the sub-cohorts are shown in Tables 3, S1 and S2, respectively. For the suggested cut-off of ESASr-D ≥ 2, the NPV (0.92) was high, however, the PPV (0.47) was moderate to low.
The prevalence of moderate/severe depression is reported to range from 20-40% in patients with ESKD [41,42]. Considering a theoretical cohort of 100 patients with an underlying prevalence of moderate/severe depression of 30%, compared to a cut-off score of ≥ 5, a cut-off of ≥ 2 identifies less patients correctly, but misses twice as many patients with moderate/severe depression ( Figure  S3A and S3B).

Post-test probability
Positive and negative post-test probability plots for depression using ESASr-D are shown in Figure S4. At a prevalence of 30%, a positive result on ESASr-D with a cut-off score of ≥ 2 would result in a post-test probability of depression of 59%. A negative test with a cut-off score of ≥ 2 would result in a post-test probability of 12%. The results are overall similar in the prevalence range of 20-40%.

Discrimination and potential cut-off score
ESASr-A exhibited good discrimination for moderate/ severe anxiety for the entire cohort of patients on dialysis and KTRs who completed GAD-7 [n = 769; AUROC: 0.86, 95% CI (0.82, 0.90); Fig. 2]. The optimal cut-off for ESASr-A in the entire cohort based on Youden's J was ≥ 4 (95% CI: 2.27-4.73; Sensitivity = 0.75; Specificity = 0.87; LR + = 5.76; LR − = 0.29; Table 4). This cut-off had a high NPV (0.96), but the PPV (0.46) was moderate to low. Similarly, good discrimination was seen for ESASr-A when assessed in the KTRs and dialysis groups separately ( Figure  S5A and S5B) with the same cut-off score of ≥ 4 identified in both sub-cohorts (Tables S3 and S4). The prevalence of anxiety is reported to range from around 10-50% in patients with ESKD [7,10]. Taking a theoretical cohort of 100 patients with an underlying prevalence of moderate/severe anxiety of 30%, compared to a cutoff score of ≥ 5, a cut-off score of ≥ 4 identifies more patients correctly, at the expense of somewhat more false negatives ( Figure S6A and S6B).

Post-test probability
Positive and negative post-test probability plots of anxiety using ESASr-A are shown in Figure S7. At a prevalence of 30%, a positive result on ESASr-A with a cut-off score of ≥ 4 would result in a post-test probability of anxiety of 71%. A negative test would result in a post-test probability of 11%. The results were similar across the 10-50% range of anxiety prevalence.

Discussion
In this study we assessed the discrimination of the ESASr emotional symptom items relative to widely used legacy tools (PHQ-9 and GAD-7) in patients treated with RRT. Both ESASr-D and ESASr-A have good discrimination. Potentially clinically useful cut-off score of ≥ 2 and ≥ 4 were identified as potential cut-off scores for ESASr-D and ESASr-A, respectively. These were similar for patients on dialysis and KTRs, supporting the use of these cut-off scores for patients with ESKD treated with different renal replacement modalities. Based on the Youden's J, we identified a potential cutoff score of ≥ 2 for ESASr-D. Notably, this score was also recommended to identify clinically relevant depression in oncology care and in patients on dialysis respectively [43][44][45]. The sensitivity and specificity of this cut-off was similar to what was previously found for other screening tools [44,46]. A higher cut off for ESASr-D would have higher specificity, thus reducing the potential burden on the healthcare system associated with unnecessary further assessment of false positives. However, we would miss many more patients with moderate/severe depressive symptoms due to higher proportion of false negatives. Considering the potential negative implications of depression (e.g. impaired HRQOL, non-adherence) we do not suggest using higher cut-off scores.
For ESASr-A, we identified a potential cut-off of ≥ 4 with better specificity compared to other ultra-short screening tools [46]. Collister et al. have recommended an ESASr-A cut-off of ≥ 2 to identify patients with anxiety [44]. In our analysis the ESASr-A cut-off of ≥ 2 yielded less false negatives but substantially more false positives compared to ≥ 4. This may increase screening burden, since many more  patients would need additional assessment to "clear" patients erroneously identified as having anxiety.
The results of our analyses were similar for both KTR and patients on dialysis. Moreover, the cut-off scores identified performed similarly across the prevalence range reported for depression and anxiety in all patients with ESKD, supporting their use for both for patients on dialysis and KTRs and this may facilitate continuity of care.
The low post-test probability we observed for a negative ESASr-D and ESASr-A score (below potential cut off) in the reported prevalence range for moderate/severe depressive symptoms suggests that a negative ESASr score may be enough to rule out moderate/severe depressive and anxiety symptoms. However, a positive test on ESASr-D and ESASr-A would not increase the post-test probability sufficiently for confident case finding. Thus, positive screening results will need subsequent confirmatory assessment.
The single item ESASr-D and ESASr-A offer may be considered as ultra-brief screening tools for symptoms of depression and anxiety. However, the relatively low sensitivity and specificity of the potential cut off scores identified in our analyses suggests that neither ESASr-D or ESASr-A is acceptable as a standalone screening tool. Even at the lowest cut-off of 1, we found low sensitivity, similar to what is reported in patients on dialysis and oncology settings [45,46]. The overlap in symptoms of emotional distress with other symptoms experienced in patients on RRT may interfere with accurate screening of symptoms of depression and anxiety. Moreover, given the culturally and ethnically diverse patients in our sample, the wording related to symptoms of depression and anxiety in ESASr-D and ESASr-A may have resulted in under-reporting. As a result of the high number of false negatives, many patients who would benefit from further assessment would miss an opportunity discuss options to manage their symptoms. In addition, the lack of specificity at low cut-off scores would result in many false positive patients, thus overwhelming the healthcare system with patient who do not need any further assessment for emotional symptoms.

Strengths and limitations
The strength of our study includes a relatively large sample size, and inclusion of both patients on dialysis and KTRs, making our results more generalizable. Our sample was also ethnically and culturally diverse. The detailed analysis of ESASr-D and ESASr-A cut-off scores also increases confidence in our findings. Use of probability plots allows for the comparison of the identified cut-off scores while varying the prevalence rates of depression and anxiety.
One limitation includes the enrollment of a convenience sample of stable patients with ESKD in a Canadian cohort which may limit the generalizability of the results to less stable patients with ESKD. In addition, a significant proportion of the sample (56%) scored 0 on the ESASr-D scale. This skewed distribution was also seen in other studies using ESAS and does not necessarily present clinically meaningful impact as all scores at the floor level would indicate the absence of symptoms [45,47,48]. Further work is needed to assess additional properties of ESASr not addressed in this study, such as test-retest reliability and responsiveness. Moreover, the use of self-reported questionnaires instead of clinical diagnostic interviews as "legacy" comparison is an important limitation of the study. This may have introduced a common-method bias that may have led to inflated estimates of the diagnostic utility.

Conclusion
In this study among patients on RRT we found strong evidence of criterion validity and good discrimination for ESASr-D and ESASr-A when compared to legacy PROMs. We propose cut-off scores of ≥ 2 and ≥ 4 for ESASr-D and ESASr-A, respectively for pre-screening anxiety or depression symptoms in patients on RRT. These cut-off scores screen out many patients that do not need any further assessment for depression and anxiety. However, due to the low sensitivity, ESASr-D and ESASr-A would not be appropriate as standalone screening tools.
Informed consent Informed consent was obtained for all individuals in the study.