Database search and risk of bias assessment. A total of 2,276 articles were identified in the initial search through the PubMed database, and thirty-six articles were potentially eligible after the first screening of the titles and abstracts. Next, the full text was reviewed, and seventeen articles were excluded. Eventually, nineteen observational studies were selected for this study (Fig. 1). The summary of the risk of bias in the included studies is shown in Fig. 2. Most studies were judged to be a “low” risk of bias.
Included studies. The characteristics of the included studies are shown in Table 1. Among these nineteen studies, prospective and retrospective studies were five 6,13,18,27,28 and fourteen 4,7,14,19,29−38, respectively. The category of pneumonia was 8, 5, and 6 for HCAP 7,19,28, 30–33,37, NHCAP 13,18,34,36,38, and NHAP 4,6,14,27,29,35, respectively. Twelve studies for PSI 4,7,14,19,27,28,31,33,36−39, ten for A-DROP 4,7,18,19,30,32,34,36,38,39, six for I-ROAD 7,18,19,36,38,39, and fourteen for CURB-65 4,6,7,14,19,27–29,31,33,35–37,39 were included in this study, respectively.
Table 1
Characteristics of included studies
First author, year | Country | Design | Category | Severity scores assessed | Patients number | Mean age(years) | Outcome | Mortality rate (%) |
El-Solh A, 2010 | USA | Retrospective | NHAP | CURB-65 | 457 | 77.4 | 30-day mortality | 23.4 |
Shindo Y, 2010 | Japan | Retrospective | HCAP | A-DROP | 141 | 81.3 | In-hospital mortality | 21.3 |
Fang WF, 2011 | Taiwan | Retrospective | HCAP | PSI, CURB-65 | 444 | 72.1 | 30-day mortality | 20.9 |
Man SY, 2011 | Hong Kong | Prospective | NHAP | PSI, CURB-65 | 767 | 83.4 | 30-day mortality | 12.4 |
Watanabe M, 2011 | Japan | Retrospective | HCAP | A-DROP | 117 | - | 30-day mortality | 5.1 |
Carrabba M, 2012 | Italy | Prospective | HCAP | PSI, CURB-65 | 307 | 72.8 | 30-day mortality | 24.1 |
Jeong BH, 2013 | Korea | Retrospective | HCAP | PSI, CURB-65 | 419 | 67.0 | 30-day mortality | 15.8 |
Lee JC, 2013 | Korea | Retrospective | NHAP | PSI, CURB-65 | 208 | 80.0 | 30-day mortality | 22.1 |
Oshitani Y, 2013 | Japan | Retrospective | NHCAP | A-DROP | 477 | 84.0 | 30-day mortality | 14.0 |
Matsunuma R, 2014 | Japan | Retrospective | HCAP | PSI, CURB-65, A-DROP, I-ROAD | 74 | 80.0 | 30-day mortality | 19.0 |
Ugajin M, 2014 | Japan | Retrospective | NHAP | PSI, CURB-65, A-DROP | 138 | 85.0 | 28-day mortality | 18.1 |
Ma HM, 2015 | Hong Kong | Prospective | NHAP | CURB-65 | 464 | 85.2 | 30-day mortality | 15.5 |
Pereira R, 2016 | Portugal | Retrospective | NHAP | CURB-65 | 103 | - | In-hospital mortality | 46.6 |
Koizumi T, 2017 | Japan | Retrospective | NHCAP | PSI, CURB-65, A-DROP, I-ROAD | 144 | 81.5 | 30-day mortality | 9.7 |
Murillo-Zamora E, 2018 | Mexico | Retrospective | HCAP | PSI, CURB-65 | 109 | 68.2 | 30-day mortality | 59.6 |
Noguchi S, 2018 | Japan | Retrospective | NHCAP | PSI, A-DROP, I-ROAD | 289 | 85.2 | In-hospital mortality | 6.9 |
Asai N, 2019 | Japan | Retrospective | HCAP | PSI, CURB-65, A-DROP, I-ROAD | 229 | 78.1 | 30-day mortality | 7.0 |
Ito A, 2020 | Japan | Prospective | NHCAP | PSI, CURB-65, A-DROP, I-ROAD | 828 | 78.0 | 30-day mortality | 12.8 |
Imamura Y, 2022 | Japan | Prospective | NHCAP | A-DROP, I-ROAD | 563 | - | 30-day mortality | 11.9 |
HCAP, healthcare-associated pneumonia; NHAP, nursing home-acquired pneumonia; NHCAP, nursing and healthcare-associated pneumonia |
PSI. Twelve studies 4,7,14,19,27,28,31,33,36−39 were included in the meta-analysis for the PSI score. Using a cut-off value of ≥ IV (moderate; n = 12), the pooled sensitivity, specificity, PLR, and NLR for mortality were calculated as 0.97 (0.94–0.98), 0.15 (0.10–0.21), 1.14 (1.08–1.20), and 0.22 (0.12–0.38), respectively. The DOR and the AUCs were 5.09 (2.95–8.78) and 0.72 (0.67–0.75), respectively (Table 2). Using a cut-off value of V (severe) (n = 11), the pooled sensitivity, specificity, PLR, and NLR for mortality were 0.69 (0.60–0.77), 0.66 (0.60–0.72), 2.03 (1.82–2.27) and 0.47 (0.38–0.58), respectively. The DOR and the AUCs were 4.32 (3.35–5.59) and 0.72 (0.68–0.76), respectively. The forest plots and estimated sensitivities and specificities from each study are shown in Figs. 2a and b.
Table 2
Pooled characteristics of severity scores for predicting mortality
| Sensitivity | Specificity | PLR | NLR | DOR | AUC |
PSI score | | | | | | | | | | | | |
≥Ⅳ (n = 12) | 0.97 | (0.94–0.98) | 0.15 | (0.10–0.21) | 1.14 | (1.08–1.20) | 0.22 | (0.12–0.38) | 5.09 | (2.95–8.78) | 0.72 | (0.67–0.75) |
Ⅴ (n = 11) | 0.69 | (0.60–0.77) | 0.66 | (0.60–0.72) | 2.03 | (1.82–2.27) | 0.47 | (0.38–0.58) | 4.32 | (3.35–5.59) | 0.72 | (0.68–0.76) |
A-DROP | | | | | | | | | | | | |
≥Ⅲ (n = 9) | 0.7 | (0.61–0.78) | 0.53 | (0.43–0.62) | 1.49 | (1.30–1.72) | 0.56 | (0.47–0.67) | 2.67 | (2.03–3.51) | 0.67 | (0.63–0.71) |
I-ROAD | | | | | | | | | | | | |
≥moderate (n = 5) | 0.92 | (0.69–0.98) | 0.44 | (0.30–0.59) | 1.66 | (1.39–1.98) | 0.18 | (0.05–0.61) | 9.32 | (2.86–30.3) | 0.66 | (0.64–0.72) |
≥severe (n = 6) | 0.68 | (0.53–0.80) | 0.6 | (0.46–0.73) | 1.7 | (1.37–2.12) | 0.54 | (0.40–0.71) | 3.18 | (2.15–4.70) | 0.69 | (0.65–0.73) |
CURB-65 | | | | | | | | | | | | |
≥Ⅱ (n = 13) | 0.91 | (0.84–0.95) | 0.28 | (0.20–0.37) | 1.26 | (1.17–1.36) | 0.33 | (0.23–0.46) | 3.86 | (2.74–5.44) | 0.65 | (0.61–0.69) |
≥Ⅲ (n = 14) | 0.63 | (0.52–0.73) | 0.63 | (0.53–0.71) | 1.7 | (1.52–1.90) | 0.58 | (0.49–0.70) | 2.91 | (2.34–3.62) | 0.67 | (0.63–0.71) |
PLR, positive likelihood ratio; NLR, negative likelihood ratio; DOR, diagnostic odds ratio; AUC, are under the curve |
A-DROP. Ten studies 4,7,18,19,30,32,34,36,38,39 were included in the meta-analysis for the A-DROP score. Using a cut-off value of ≥ III (severe; n = 9), the pooled sensitivity, specificity, PLR, and NLR for mortality were 0.70 (0.61–0.78), 0.53 (0.43–0.62), 1.49 (1.30–1.72), and 0.56 (0.47–0.67), respectively. The DOR and the AUCs were 2.67 (2.03–3.51) and 0.67 (0.63–0.71), respectively (Table 2). The forest plots using these cut-offs and estimated sensitivities and specificities from each study, except one 7, are shown in Figs. 2c and d. The excluded study 7 had an unknown death number for each severity grade was unknown.
I-ROAD. Six studies 7,18,19,36,38,39 were included in the meta-analysis for the I-ROAD score. Using a cut-off value of ≥ moderate (n = 5), the pooled sensitivity, specificity, PLR, and NLR for mortality were 0.92 (0.69–0.98), 0.44 (0.30–0.59), 1.66 (1.39–1.98), and 0.18 (0.05–0.61), respectively. The DOR and the AUCs were 9.32 (2.86–30.3) and 0.66 (0.64–0.72), respectively (Table 2). Using a cut-off value of severe (n = 6), the pooled sensitivity, specificity, PLR, and NLR for mortality were 0.68 (0.53–0.80), 0.60 (0.46–0.73), 1.70 (1.37–2.12), and 0.54 (0.40–0.71), respectively. The DOR and the AUCs were 3.18 (2.15–4.70) and 0.69 (0.65–0.73), respectively. The forest plots using these cut-offs and estimated sensitivities and specificities from each study are shown in Figs. 2e and f.
CURB-65. Fourteen studies 4,6,7,14,19, 27–29,31,33, 35–37,39 were included in the meta-analysis for the CURB-65. Using a cut-off value of ≥ II (moderate; n = 13), the pooled sensitivity, specificity, PLR, and NLR for mortality were 0.91 (0.84–0.95), 0.28 (0.20–0.37), 1.26 (1.17–1.36), and 0.33 (0.23–0.46), respectively. The DOR and the AUCs were 3.86 (2.74–5.44) and 0.65 (0.61–0.69), respectively (Table 2). Using a cut-off value of III (severe; n = 14), the pooled sensitivity, specificity, PLR, and NLR for mortality were 0.63 (0.52–0.73), 0.63 (0.53–0.70), 1.70 (1.52–1.90), and 0.58 (0.49–0.70), respectively. The DOR and the AUCs were 2.91 (2.34–3.62) and 0.67 (0.63–0.71), respectively. The forest plots using these cut-offs and estimated sensitivities and specificities from each study are shown in Figs. 2g and h.
Comparisons of overall AUC among PSI, A-DROP, I-ROAD, and CURB-65. The mean differences in overall AUC were calculated using the overall AUC (95% CI) of each study (Fig. 3). The mean differences were 0.70 (0.68–0.72), 0.71 (0.63–0.78), 0.68 (0.63–0.73), and 0.67 (0.63–0.71) for PSI, A-DROP, I-ROAD, CURB-65 scores, respectively (Fig. 4). No significant differences were observed (p = 0.63).