The Validity and Reliability of the Patient Health Questionnaire-9 in Screening for Poststroke Depression

Abstract

Background

Depression is the most common psychological problem experienced by survivors of a stroke. The pool frequency is 31% of stroke survivors at any time up to five years after their stroke ¹. However, poststroke depression (PSD) often remains unrecognized and undertreated ². PSD is associated with decreased participation in rehabilitation programs, resulting in less functional improvement ³. After patients are discharged, they tend to become physically inactive and socially isolated ⁴. This may lead to more depression, cognitive impairment, increased caregiver distress ⁵, and increased mortality during the 2–5 years period following a stroke ⁶. A study of the impact of PSD on healthcare use by veterans with acute stroke reported that stroke patients with PSD had significantly more hospitalizations, outpatient visits, and longer lengths of stay than stroke patients without PSD ⁷.

It is difficult to make a diagnosis of depression after stroke because the symptoms of depression can be confused with certain symptoms that are typical of stroke patients ⁸. For example, swallowing impairment after stroke can cause weight loss, yet a stroke patient might be suffering from anorexia nervosa resulting from depression. Clinicians would tend to pay more attention to the swallowing impairment than ask about other constitutional symptoms. In addition, the weakness customarily experienced after stroke presents as a reduction in physical movement and feelings of fatigue after movement. However, such weakness is also indicative of the fatigue specified in the DSM 5 criteria as a prerequisite for the diagnosis of major depressive disorder (MDD). These symptoms might confuse clinicians and lead them to underdiagnose, as well as undertreat, PSD.

Screening for mood disorders after stroke is recommended by many stroke and stroke-rehabilitation guidelines ^{9, 10}. Given that the availability of psychiatrists is limited in Thailand, there is a need for a screening tool to assist primary care physicians and other specialists with the detection of PSD. Currently, several depression screening tools have been translated into Thai and are used for the general population ¹¹. The tool selected for this study was the Patient Health Questionnaire-9 (PHQ-9). Extensively studied in the general population and poststroke patients, it has been reported to be one of the good PSD screening tools with the highest sensitivity ¹². The PHQ-9 has been translated into Thai and validated in primary care patients. The cutoff score of the Thai PHQ-9 for major depression in primary care patients is 9, which differs from the corresponding figure, 10, for the original version of the PHQ-9 ^{13, 14}. As to PSD, Williams et al. reported a cutoff score for the original version of 10, with a sensitivity of 91% and a specificity of 89%. However, the PHQ-9 has not been validated for PSD ¹⁵ among Thais. Because Thailand and western countries have different health care systems, cultures, attitudes, mindsets, and family support systems, this study investigated the validity and reliability of the PHQ-9 in screening for PSD among Thais.

Methods

Subjects and procedures

Ethical approval was obtained from the Human research protection unit, Faculty of Medicine Siriraj hospital which is a medical ethical committee of Faculty of Medicine Siriraj Hospital (COA no.623/2017). The patients gave written consent to participate and were recruited between November 2017 to December 2018 from the Department of Rehabilitation Medicine, Faculty of Medicine Siriraj Hospital which served as a tertiary hospital in Thailand. The patient inclusion criteria were age > 45 years; having first ever stroke as per WHO criteria, with a stroke duration 2 weeks - 2 years; stable medical and neurological conditions; and the ability to communicate in Thai. Excluded were patients with cognitive impairment of < 24, as measured by the Thai Mental State Examination,¹⁵ or a previous diagnosis of dementia, a psychiatric disorder, or another neurological disease.

Demographic characteristics were retrieved from interviews with the enrolled patients, and information related to their stroke (such as any comorbid illnesses, and the types of stroke diagnosed from imaging studies) were obtained from medical records. The Barthel Index (BI) and Modified Rankin Scales (MRS) were also obtained to determine the level of physical function and disability of the participants. The PHQ-9 ¹³ was administered by one of the researchers. On the same day, a psychiatrist interviewed each patient in a private area and gave a diagnosis according to the DSM-5 criteria. The researcher and a psychiatrist were blind to each other’s assessment.

Measures

Thai Mental State Examination (TMSE)¹⁵

TMSE is the first neuropsychiatric test for the standard mental status examination for Thai people. The total score of TMSE was 30 points. The cut-off point for the diagnosis of normal healthy Thai elderly was 24 points.

Modified Rankin Scale (MRS)

The Modified Rankin Scale (MRS) is a clinician-reported measure of global disability and has been widely applied for evaluating recovery from stroke ^{16, 17}. It is an ordinal scale with 7 categories ranging from zero (no symptoms) to six (death). The MRS assesses ability to ambulate and complete activities of daily living. MRS greater than 3 was defined as disability ¹⁸.

Thai Patient Health Questionnaire-9 (PHQ-9)¹³

The PHQ-9 consists of 9 questions based on the 9 DSM-IV criteria for major depressive

episode. It refers to symptoms experienced by the patients during the two weeks prior to answering the questionnaires. Scores for each item in the PHQ-9 range from 0 (not at

all), to 1 (several days), 2 (more than half of the days) and 3 (nearly every day), while summed scores range from 0 to 27. The PHQ-9 can be used as a screening tool for the diagnosis of major depression. It can be used as a screening tool with recommended cutoff score for diagnosis of depression. The cutoff score of 9 or greater of Thai PHQ-9 revealed a sensitivity of 0.84 and specificity of 0.77. It can also be used to establish a diagnosis following a categorical algorithm. A major depressive disorder is diagnosed if 5 or more of the 9 symptoms have been present at least more than half the days of the past 2 weeks and 1 of these symptoms has been either depressed mood or anhedonia.

DSM-5 criteria for major depressive episode

The DSM-5 criteria for major depressive episode was used as a reference standard ¹⁹. The psychiatric interview was performed to an individual patient. The major depressive episode could occur in major depressive disorder (MDD), depressive disorder due to another medical condition with MDD, and MDD episode in bipolar disorder

Data Analysis

PASW Statistics for Windows, version 18.0 (SPSS Inc., Chicago, Ill., USA) and MedCalc Statistical Software, online version (MedCalc Software bvba, Ostend, Belgium) were used for the statistical analyses. The demographic data, MRS, BI, and PHQ-9 scores were analyzed by descriptive statistics. The quantitative data (age and total BI) were analyzed by an independent-sample t-test, while the stroke durations and PHQ-9 scores were analyzed with the Mann–Whitney U test. The qualitative data (gender, education levels, risk factors, pathology of stroke, side of weakness, and MRS scale) were analyzed by Chi-square.

The depression scores of the normal and depression groups were analyzed by the independent-sample t-test. All analyses were significant at a p-value of < 0.05. Internal consistency was analyzed by Cronbach’s alpha. As a bivariate response, the psychiatric diagnosis was used as the gold standard to calculate the sensitivities and specificities of all possible PHQ-9 cutoff scores. The positive and negative predictive values as well as the positive and negative likelihood ratios were calculated for each PHQ-9 cutoff score. The ROC analyses combined instrument sensitivity and specificity into one measure (referred to as the area under the curve, or AUC) for all possible cutoff scores.

Results

In all, 190 stroke patients were recruited. Seventy-five of these were excluded: 21 had recurrent stroke, 17 had cognitive impairment, 17 had aphasia, 10 were < 45 years, and 10 had a stroke duration > 2 years (Fig 1). Of the remaining 115 stroke patients enrolled, there were 63 males (54.8%) and 53 females (45.2%), with a mean age of 64 + 10 years (45,88). The majority had graduated primary school, followed by lower-secondary school and upper-secondary school. The comorbid illnesses found were, in descending order of frequency, hypertension, dyslipidemia, diabetes mellitus, and heart disease. The median duration of stroke was 59 days. Most patients (81.7%) suffered from ischemic stroke, and left-side weakness was dominant (61%). Most patients (65.2%) were recruited from inpatient rehabilitation.

All patients were administered the index test (the PHQ-9) and the reference standard (the DSM-5 criteria for MDD) on the same day. The mean PHQ-9 score was 5.2 + 4.8. According to the DSM-5 criteria, 23 patients (20%) had PSD, while 92 (80%) were normal. In the PSD group, 8 (6.9%) were diagnosed with MDD, 2 (1.7%) had depressive disorder not otherwise specified, and 1 (0.9%) had other specified depressive disorder and 12 patients (10.5%) had adjustment disorder with depressed mood (Table 1).

Table 1. The incidence of psychological status among the stroke patients

The demographic characteristics of the normal and depression groups revealed no statistically significant (age, gender, education level, risk factors, median duration after stroke, pathology of stroke, side of weakness, and settings of the patients). However, the MRS and the median PHQ-9 scores of the groups differed. MRS scores of 0–3 were defined as no disability, while an MRS score > 3 was defined as disability; more stroke patients were disabled in the depression group (78%) than in the normal group (55.4%) (table 2; data not shown).

Reliability and item analysis

As shown in Table 3, the highest mean score of the nine PHQ-9 items was found for item 3 (“trouble falling or staying asleep, or sleeping too much”). Item 9 (“thoughts that you would be better off dead or of hurting yourself”) had the lowest score. As to the internal consistency of the PHQ-9, Cronbach’s alpha was 0.78, which is considered acceptable. All items, if deleted, would consistently decrease the total scale alpha. The least item-total correlation was item 5 poor appetite or overeating.

Table 3 Mean score, standard deviation and reliability of the PHQ-9 score

Validity analysis

The performance of the PHQ-9 against the diagnosis of major depressive disorder by the DSM-5 criteria for major depressive disorder as a criterion standard was examined. According to the DSM-5 criteria, 23 patients (20%) met the diagnosis of poststroke depression. The median PHQ-9 score for the depression group was 10 (IQR25,75: 7,15) whereas the median score of the normal group was 4 (IQR25,75: 0.5,5.75). These median PHQ-9 score showed statistically different between 2 groups.

Table 4 The performance of different PHQ-9 cut-off scores in detecting depression

The validity of the index test PHQ-9, using categorical algorithm, had a sensitivity of 34.8%, specificity of 97.8%, positive predictive value (PPV) of 80%, and negative predictive value (NPV) of 85.7%, and positive likelihood ratio of 16.0 (table 3). When using the summed item based diagnosis, a sensitivity, specificity, PPV, NPV, and likelihood ratio of different PHQ-9 thresholds in diagnosing PSD showed in table 4. At the cutoff score of 6 showed the highest Youden’s index. This cutoff score had a sensitivity of 87.0 % (95% CI=66.4, 97.2), specificity of 75.0% (95% CI=64.9, 83.4), positive predictive value of 46.5% (95% CI=37.1, 56.2), negative predictive value of 95.8% (95% CI=88.8, 98.5), positive likelihood ratio of 3.5 (95% CI=2.4, 5.1), and negative likelihood ratio of 0.2 (95%CI=0.1, 0.5) (table 3). The ROC curve illustrates that the PHQ-9 performed well in identifying patients with PSD (figure 2). The area under the curve (AUC) in our study was 0.87 (95%CI=0.78,0.96) which demonstrated good discrimination.

Discussion

This study was the first in Thailand to determine the validity of a depression screening questionnaire with stroke patients. The questionnaire investigated was the PHQ-9, one of the good screening tools for PSD ²¹. The reference standard was a psychiatric interview according to the DSM-5 criteria for major depressive disorder. In this study, the validity of the PHQ-9 in screening poststroke depression was good in term of its discriminatory power (AUC=0.87) relative to the gold standard, DSM-5 criteria. In addition, its internal consistency was acceptable (Cronbach’s alpha=0.78).

Based on the DSM-5 criteria, PSD was found in 23 patients (20%) which was less than the previous studies. A meta-analysis conducted by Hackett and Pickles ² found that 31% of stroke patients developed depression in any setting at any time up to 5 years following their stroke. Robinson ²² undertook a pool analysis and reported mean incidence for major and minor depression of 19.3% and 18.5% respectively, among hospitalized patients in acute or rehabilitation hospitals. By comparison, the low incidence in the present study probably stemmed from having the inclusion criterion that only stroke patients be aged > 45 years. Previous research found that younger stroke survivors were more likely to become depressed than older survivors ^{23, 24}. Nevertheless, the incidence established by the current study was in line with that of research by Fuentes et al., which recruited stroke patients of the same age group and found a low depression incidence of 9.9% ²⁵.

Moving on to the demographic characteristics of stroke patients with and without PSD, our study revealed no significantly different of demographic-related variables between groups.

For the disability-related variable, the Modified Rankin Scale (MRS) is a common clinimetrical instrument for measuring disability after stroke. The patients with MRS score 4-5 who were classified as having disability appeared more frequently in the depression group. PSD has been found to be associated with more severe neurological deficit and physical disability in the acute and chronic phases ²⁶. Drawing on the systematic review and meta-analysis of longitudinal studies, Blöchl et al ²⁷ found that depressed stroke patients were generally more disabled.

The internal consistency of the PHQ-9 administered among stroke patients in this study was 0.78 which is considered acceptable. However, the level of internal consistency differed from the original version of the PHQ-9. The original studies, performed in primary care and obstetrics and gynecology, showed internal consistency of 0.89 and 0.86 respectively ¹². In addition, Turner et al, who employed PHQ-9 to screen for PSD, found an internal consistency of 0.82 ²⁸. In the case of the Thai version of the PHQ-9, a validity study on the Thai population reported an internal consistency of 0.79 ¹³. Later Lee and Dajpratham, who employed the Thai version on elderly Thais, reported an internal consistency of 0.76 ²⁹. In the present study, the internal consistency was 0.78, which is highly congruent with those two previous studies using the Thai version of the PHQ-9.

The PHQ-9 can be used as a screening tool since the AUC showed a good level of discriminatory power (AUC, 0.87). The result of this study was in line with other studies that reported a good discriminatory power of the PHQ-9, with AUC > 0.8 ^{14, 28, 30-32}. As to its validity, PHQ-9 score can be used in 2 ways to diagnose depression. The first way is an algorithm-based diagnosis with cutoff score of 10 to diagnose major depression. In 2015, Manea et al ³³. conducted a diagnosis meta-analysis of the PHQ-9 algorithm based scoring method as a screening for depression. They found that the sensitivity was as low as 53% (95%CI, 42-65) whereas the specificity was as high as 94% (95%CI, 91-96). Our study applied the algorithm based diagnosis of PSD in the setting of tertiary hospital. The diagnostic accuracy revealed low sensitivity and high specificity (table 4) as of the result from the study of Manea et al ³³. The low sensitivity is not a good property of a screening tool. Therefore, all the previous validation studies of PHQ-9 used summed item based score to compare with the structured interviews as their reference standard. Interestingly, Pettersson et al ³⁴ performed a systematic review to explore the diagnostic accuracy of the structured interviews as index tests. The structured interviews which had sufficient accuracy for depression diagnosis were only the Structured Clinical Interview for DSM-IV (SCID) and the Mini International Neuropsychiatric Interview (MINI). Therefore, some structured interviews currently in use are not accurate enough to be the reference standard for diagnosis of depression. The psychiatric interview as the reference standard is generally accepted as the gold standard. The summed score of PHQ-9 in this study was validated with the DSM-5 criteria interviewed by the psychiatrist and revealed the optimum cutoff score of 6. This cutoff score yielded the highest Youden’s index and showed a sensitivity of 87.0 % (95% CI=66.4, 97.2), specificity of 75.0% (95% CI=64.9, 83.4) for the diagnosis of depression (table 4).

There were some limitations of this study. Firstly, the participants’ recruitment age of ≥ 45 years meant that the findings cannot be generalized to younger stroke patients. Secondly, only participants who could communicate were recruited. We realized that stroke patients who cannot communicate would probably be very depressed; however, the mood assessment scale for noncommunicative patients is different.

Conclusion

The Thai version of the PHQ-9 had good validity and acceptable reliability for screening PSD. The summed score based depression diagnosis should be employed for screening and the cutoff score of 6 were positive for poststroke depression.

Abbreviation

The Patient Health Questionnaire 9: PHQ-9; positive predictive value: PPV; negative predictive value: NPV; area under the curve: AUC; receiver operating characteristic curve: ROC; the Structured Clinical Interview for DSM-IV (SCID) and the Mini International Neuropsychiatric Interview (MINI).

Declaration

Ethical approval was obtained from the Human Research Protection Unit, Faculty of Medicine Siriraj hospital which is a medical ethical committee of Faculty of Medicine Siriraj Hospital (COA no.623/2017). All the participants gave written consents to participate. Consent for publication was "Not applicable".

Availability of data and materials

All data generated or analyzed during this study are included in this published article and its supplementary information files.

Competing Interests

The authors declare that they have no competing interests.

Funding

No funding was obtained for this study.

Authors’ contribution

PD conceived the study, design protocol, analyzed the data, and prepared the manuscript. PP participated in the study design, designed protocol, assisted in data collection, and gave comment on the manuscript. WA participated in the study design, assisted in data collection, and gave comment on the manuscript. KW participated in the study design, assisted in data collection, and gave comment on the manuscript. JB designed protocol and gave comment on the manuscript. KP designed protocol and gave comment on the manuscript. All authors have read and approve the final version of the manuscript.

Acknowledgement

The authors would like to thank Dr. Chulaluk Komoltri, Mr. Sutthipol Udompunturak for their assistance in statistical analysis of the data, the counseling psychologists, and the nursing staff of the Department of Rehabilitation Medicine, Faculty of Medicine Siriraj Hospital for their assistance in collecting data.

References

1. Rogers S. Poststroke depression screening: an executive summary. J Neurosci Nurs. 2017;49(2):66-8.

2. Hackett M, Pickles K. Part I: frequency of depression after stroke: an updated systematic review and meta-analysis of observational studies. Int J Stroke. 2014;9:1017-25.

3. Rigler S. Management of poststroke depression in older people. Clin Geriatr Med. 1999;15(4):765-83.

4. Gillen R, Tennen H, McKee T. Depressive symptoms and history of depression predict rehabilitation efficiency in stroke patients. . Arch Phys Med Rehabil. 2001;82:1645-9.

5. Parikh R, Lipsey J, Robinson R, Price T. Two-year longitudinal study of post-stroke mood disorders: dynamic changes in correlates of depression at one and two years. Stroke. 1987;18(3):579-84.

6. Cameron J, Cheung A, Streiner D, Coyte P, Stewart D. Stroke survivor depressive symptoms are associated with family caregiver depression during the first 2 years poststroke. Stroke. 2011;42:302-6.

7. Jia H, Damush T, Qin H, Ried L, Wang X, Young L, et al. The impact of poststroke depression on healthcare use by veterans with acute stroke. Stroke. 2006;37(11):2796-801.

8. Bartoli F, Lillia N, Lax A, Crocamo C, Mantero V, Carrà G, et al. Depression after stroke and risk of mortality: a systematic review and meta-analysis. Stroke Res Treat. 2013;2013:862978.

9. Lipsey J, Spencer W, Rabins P, Robinson R. Phenomenological comparison of functional and poststroke depression. Am J Psychiatry 1986;143:527-9.

10. National institute for health and care excellence. Stroke rehabilitation: long term rehabilitation after stroke [Clinical guideline 162  Methods, evidence and recommendations ]. 2013 [updated 29 May 2013 cited 2016 Nov 21]. Available from: https://www.nice.org.uk/guidance/cg162/resources/cg162-stroke-rehabilitation-full-guideline3.

11. Charernboon T. Measurement and screening tools for depressive disorders in Thailand: consideration in selection. Thammasat Medical Journal 2011;11(4):667-76.

12. Kroenke K, Spitzer R, Williams J. The PHQ-9: validity of a brief depression severity measure J Gen Intern Med. 2001;16(9):606–13.

13. Lotrakul M, Sumrithe S, Saipanish R. Reliability and validity of the Thai version of the PHQ-9. BMC Psychiatry 2008;8:46-52.

14. Williams L, Brizendine E, Plue L, Bakas T, Tu W, Hendrie H, et al. Performance of the PHQ-9 as a screening tool for depression after stroke Stroke. 2005;36(3):635-8.

15. Train the brain forum committee. Thai Mental State Examination. Siriraj Hosp Gaz 1993;45:359-74.

16. Rankin L. Cerebral vascular accidents in patients over the age of 60. II. Prognosis Scott Med J. 1957;2:200 –15.

17. Swieten Jv, Koudstaal P, Visser M, Schouten H, Gijn Jv. Interobserver agreement for the assessment of handicap in stroke patients. Stroke. 1988;19:604 –7.

18. Sulter G, Steen C, Keyser JD. Use of the Barthel index and modified Rankin scale in acute stroke trials. Stroke. 1999;30(8):1538-41.

19. American Psychiatric Association. Diagnostic and Statistical manual of mental disorders –DSM-5. 5th ed. ed. Washington DC: American Psychiatric Press, Inc; 2013.

20. Youden W. Index for rating diagnostic tests. Cancer. 1950;3:32-5.

21. Meader N, Moe-Byrne T, Llewellyn A, Mitchell A. Screening for poststroke major depression: a meta-analysis of diagnostic validity studies. J Neurol Neurosurg Psychiatry 2014;85:198-206.

22. Robinson R. Poststroke depression: prevalence, diagnosis, treatment and disease progression. Biol Psychiatry. 2003;54:376-87.

23. Barker-Collo S. Depression and anxiety 3 months poststroke: prevalence and correlates. Arch Clin Neuropsychol. 2007;22:519–31.

24. Chatterjee K, Fall S, Barer D. Mood after stroke: a case control study of biochemical, neuro-imaging and socio-economic risk factors for major depression in stroke survivors. BMC Neurol 2010;10:125–34.

25. Fuentes B, Ortiz X, Sanjose B, Frank A, Díez-Tejedor E. Post-stroke depression: can we predict its development from the acute stroke phase? . Acta Neurol Scand. 2009;120(3):150-6.

26. Kutlubaev M, Hackett M. Part II: predictors of depression after stroke and impact of depression on stroke outcome: an updated systematic review of observational studies. Int J Stroke. 2014 9(8):1026-36.

27. Blöchl M, Meissner S, Nestler S. Does depression after stroke negatively influence physical disability? A systematic review and meta-analysis of longitudinal studies. J Affect Disord. 2019;15(247):45-56.

28. Turner A, Hambridge J, White J, Carter G, Clover K, Nelson L, et al. Depression screening in stroke: a comparison of alternative measures with the structured diagnostic interview for the diagnostic and statistical manual of mental disorders, fourth edition (major depressive episode) as criterion standard. Stroke. 2012;43(4):1000-5.

29. Lee S, Dajpratham P. Criterion validity of the Thai version of the PHQ-9 and the PHQ-2 for screening major depression in Thai elderly. J Thai Rehbail 2017;27(1):30-7.

30. deMan-vanGinkel J, Hafsteinsdóttir T, Lindeman E, Burger H, Grobbee D, Schuurmans M. An efficient way to detect poststroke depression by subsequent administration of a 9-item and a 2-item Patient Health Questionnaire. Stroke. 2012;43(3):854-6.

31. Prisnie J, Fiest K, Coutts S, Patten S, Atta C, Blaikie L, et al. Validating screening tools for depression in stroke and transient ischemic attack patients. Int J Psychiatry Med. 2016 51(3):262-77.

32. Wang Z, Zhu M, Su Z, Guan B, Wang A, Wang Y, et al. Post-stroke depression: different characteristics based on follow-up stage and gender-a cohort perspective study from Mainland China. Neurol Res. 2017 39(11):996-1005.

33. Manea L, Gilbody S, McMillan D. A diagnostic meta-analysis of the Patient Health Questionnaire-9 (PHQ-9) algorithm scoring method as a screening for depression. Gen Hosp Psychiatry. 2015;37(1):67-75.

34. Petterson A, Bostrom K, Gustavsson P, Ekselius L. Which instruments to support diagnosis of depression have sufficient accuracy: a systematic review. Nord J Psychiatry. 2015;69(7):497-508.

Table

Table 1. The incidence of psychological status among the stroke patients

Table 2 The characteristics of the stroke patients

*mean + SD, ** median (IQR 25,75), ***significant at p value < 0.05

Table 3 Mean score, standard deviation and reliability of the PHQ-9 score

Table 4 The performance of different PHQ-9 cut-off scores in detecting depression