DOI: https://doi.org/10.21203/rs.2.10225/v1
Objective To compared the Sepsis 1.0 criterial with the Sepsis 3.0 criteria predict the efficacy of all-caused mortality of in-hospital in critically ill patients with severe infection.
Design This is a retrospective and cohort study based on the database of severe infection.
Setting A 48-bed general intensive care unit in affiliated hospital of University. Patients Critically ill patients with suspected infection based on the electronic health records from 1 January to 31 December, 2015.
Interventions None.
Measurements The variables of exposures included: quick sequential organ failure assessment (qSOFA), systemic inflammatory response syndrome (SIRS) score and sequential organ failure assessment (SOFA). Main outcomes and measures: for predictive validity, we found that the discrimination for hospital mortality was more common with sepsis than with uncomplicated infections. Results are reported as the area under the receiver operating characteristic curve (AUROC).
Main Results In the primary cohort, 873 patients had suspected infection cohort (n=634), of whom 188 (29.7%) died; and with the non-infection cohort (n=239), 26 patients died (10.9%). Among intensive care unit (ICU) cases in the infection cohort, the predictive validity for hospital mortality was higher for Sepsis 3.0 (SOFA) criteria (AUROC=0.702; 95%CI, 0.665 −0.737; p≤0.01 for both) than for Sepsis 1.0 (SIRS) criteria (AUROC=0.533; 95% confidence interval [95%CI], 0.493−0.572).
Conclusions In our study, we found the Sepsis 3.0 criteria is able to accurately predict the prognosis in critically ill patients with severe infection, and its predictive efficacy is superior to Sepsis 1.0 criteria.
Study design
This is a retrospective cohort study, and it was approved by the Biomedical Ethics Committee. This study was registered at the Chinese Clinical Trial Register (CCTR number: ChiCTR-ORC-16010138, registered 12 December 2016). URL: http://www.chictr.org.cn/listbycreater.aspx. Written informed consent was not obtained from the patients or their relatives due to the retrospective study design of using the electronic health records and no additional interventions were given to the subjects.
Setting
We conducted the study in a 48-bed general intensive care unit (GICU) of Sichuan University West China Hospital, (Chengdu, Sichuan Province, China). The periods of recruitment in the study included one whole year from 1 January to 31 December 2015. We identified those with suspected infection for the purpose of criteria comparison among 1243 electronic health records. The dealline for follow-up time was defined as the end of in-hospital for every patient according to the electronic health records.
Participants
The included criteria of participants:
1. The participants with suspected infection need to simutanuously meet the following three requirements ( 17 ). Firstly, the initial episode of suspected infection was defined as a combination of body fluid cultures and antibiotics. Secondly, it is necessary that the combination of antibiotics and culture sampling occurred within a specific time limits. If the antibiotic was administered first, the culture sampling must have been obtained within 24 hours. If the culture sampling occurred first, antibiotic administration must have been instigated within 72hours. Finally, the “onset” of infection included the time when the first of the 2 events took place.
2. Age ≥ 18 years.
3. Length of stay in GICU ≥24 hours.
Outcome Variables
In the study, we followed up all patients after hospital discharge using their medical records and all-cause in-hospital mortality was the primary outcome. The secondary outcome was the risks of an ICU stay of 3 or more days. The exposures of risk factors for in-hospital death included scores of SIRS, qSOFA and SOFA.
Effect Modifiers and Potential Confounders
The major effect modifiers were age and acute physiology and chronic health evaluation (APACHE) II in the study due to the hospital mortality gradually increased with the increasing age and APACHE II; sex was the potential confounders due to the difference between male and female was evident in hospital mortality.
Data Sources and Bias Control
Each component of SIRS, qSOFA and SOFA derived from indicators of every medical record. For the time window from 48 hours before to 24 hours after the onset of infection, we calculated the maximum score of SIRS, qSOFA, and SOFA ( 17 ). In sepsis occurring before, near the moment of, or after infection, organ dysfunction is recognized by clinicians. Before to up to 24 hours after the onset of infection, we calculated a change of 2 points or more in the SOFA score from up to 48 hours.
The researchers in this study collected general information from medical records of patients admitted to the ICU: medical identification, demographic characteristics, vital signs, and results of laboratory tests. We calculated the qSOFA, SIRS and SOFA scores for each patient using those data. Acute Physiology and APACHE II collected to assess the illness severity of members of the enrolled participant.
These study designers did not participate in the data collection, and those who participated in data collection of the study were blind to the study design.
Comparability of Assessment Methods
The comparable cohorts including Sepsis 1.0 and Sepsis 3.0 were generated from the database of critically ill patients with infection. We compared with the baseline characteristics and in-hospital death of both cohorts.
Statistical Analysis
Descriptive variables with a normal distribution were expressed as means ± standard deviations and were analysed using an independent sample t test. We expressed variables with a skewed distribution as medians and quartiles and analysed them using the Mann-Whitney U test. We employed the χ2 test for comparison of frequencies. To assess the baseline risk of outcomes, we analysed the demographic variables that significantly differed among patients with opposite outcomes by means of univariate and multivariate logistic regression analysis; we determined the independent predictors. We constructed and compared the area under the curve of receiver operating characteristic (AUROC) was determined to assess predictive values.
We performed all the statistical analyses using MedCalc® (version 15.8) statistical software ( 18 ), and Empower Stats software. All the statistical tests were two-tailed, and P<0.05 was considered significant. We considered the area under the receiver operating characteristic curve (AUROC) to be poor at 0.6 − 0.7, adequate at 0.7 − 0.8, good at 0.8 − 0.9, and excellent at 0.9 or higher ( 19 ).
In the present study from the data of severe infection in GICU, we found the predictive validity of prognostic assessment is higher for the Sepsis 3.0 criteria than the Sepsis 1.0 criteria. In contrast, the predictive efficacy of SOFA and SIRS scores in our study was lower than that of the data from the developed counties.
As noted earlier, sepsis was defined as a−“life-threatening organ dysfunction due to a dysregulated host response to infection” by the Third International Consensus Definitions Task Force( 8 ). With its new definitions, the task force constructed potential clinical criteria and validated using the data from the developed countries( 15 ).
Using the qSOFA assessment system as a screening tool, we may miss approximately one−third of patients with infections at high risk of death; that, suggests it may be unfit for critically ill patients with severe infection in ICUs. A similar result was reported in 2016 ( 15 ). In our study, if we use the Sepsis 1.0 criteria, we found that we may miss more than one-sixth of patients with infection at high risk of death. Interestingly, using the Sepsis 3.0 criteria, the rate of exclusion was zero; this suggests that the clinical criteria in the new definitions for sepsis and septic shock may be more appropriate for critically ill patients with infection.
In the present investigation, hospital mortality was reduced to zero in patients with infection and SOFA scores under 5. When SOFA scores exceeded 5 points, it increased sharply (Fig. 1C). Regarding hospital mortality and SIRS scores, the number of patients who died was higher with 0 point than with 1 point, but the difference was not statistically significant (p>0.05); hospital mortality was likewise not statistically significant in variance analysis of subgroups with scores of 0-4 points (p>0.05) (Fig. 1B). With respect to hospital mortality and qSOFA scores, variance analysis of subgroups with 3 points in a pairwise comparison with those with 2, 1, and 0 points showed statistical significance; the cut-off value of 2 points was optimal (Fig. 1A).
Hitherto, comparative studies on the use of qSOFA as a screening tool in non-ICU settings have demonstrated its efficacy in sepsis diagnosis( 9-11 , 13 ). However, the position was not clear with ICUs? In the present investigation, we found the AUC of qSOFA to be much lower than in the original study for predicting hospital mortality in patients with infection in ICUs (0.585 vs. 0.66) ( 15 ). One reason for the discrepancy may be that the illness severity in the cohort of the present study was more severe than that in the matching cohort of the original study.
Strengths
Our study has several strengths. Firstly, the data of our study are current and present over a period of one whole year (2015) due to the time that we designed the study was just at which the Sepsis 3.0 had been published for more than one year. Secondly, the data we collected during the time window of infection or suspected infection consist of physiological or laboratory measurements that were retrospectively collected for routine monitoring data and are therefore unlikely to be biased.
Limitations
This study has several limitations. First, we conducted a single-centre clinical investigation in a province of southwest China. Second, we did not include comorbidities owing to the states of construction of the database, which was still in an initial stage. Third, the high morbidity and mortality in this study may to some extent limit its generalization under the Sepsis 3.0 definition..
Conclusions
In our study, we found the Sepsis 3.0 criteria is able to accurately predict the prognosis in critically ill patients with severe infection, and its predictive efficacy is superior to Sepsis 1,0 criteria.
Ethics approval and consent to participate
This study was approved by the Ethics Committee of Sichuan University West China Hospital (No. 315, 2016). Due to the retrospective study design involving electronic health records and no additional interventions, written informed consent was not obtained from the patients or their relatives.
Consent to publish
All authors have read and approved the manuscript version, and agree to submit for consideration for publication in the journal.
Competing interests
(1) All authors have no relationships with companies that might have an interest in the submitted work over the previous 5 years; (2) their spouses, partners, or children have no financial relationships that may be relevant to the submitted work; and (3) none of the authors have nonfinancial interests that may be relevant to the submitted work.
There are no ethical/legal conflicts involved in the article.
Availability of data and materials
We stated that all the data and materials were true and available in the study. The data in the study were deposited to the Chinese Clinical Trial Register Center (URL: http://www.chictr.org.cn/ listbycreater. aspx).
Funded This study was supported by the Zunyi Medical College 2017 Academic New Seedling Cultivation and Innovative Exploration Fund (Qian Ke He Talents Platform 【2017】5733-019) and Science and Technology Support Plan of Guizhou Province in 2019 (Qian Ke He Support 【2019】2834).
Authors' Contributions
Wei Zhang had full access to all of the data in the study and accepts responsibility for the data management and accuracy of the data analysis. Study concept and design: Wei Zhang and Yan Kang. Acquisition, analysis, and interpretation of data: Wei Zhang, Juan Gu, and Yan Zheng. Drafting of the manuscript: Wei Zhang, Yan Zheng, and Juan Gu. Critical revision of the manuscript for important intellectual content: Juan Gu and Yan Zheng. Administrative, technical, or material support: Wei Zhang and Yan Zheng. Study supervision: Juan Gu and Yan Kang. All authors agreed to the final version before submission. Wei Zhang is the study guarantor.
Acknowledgements
The authors thank Xiaoli He, Xiaolei Yang, Yangting Li, and Jie Yang for helping to construct the Sepsis-3 database at Sichuan University West China Hospital. The authors also thank all of the patients who participated in the clinical trial.
1. Angus DC, Linde-Zwirble WT, Lidicker J, et al. Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care. Critical Care Medicine. 2001;29(7):1303-10.
2. Liu V, Escobar GJ, Greene JD, et al. Hospital deaths in patients with sepsis from 2 independent cohorts. JAMA. 2014;312(1):90-2.
3. Bone RC, Balk RA, Cerra FB, et al. American College of Chest Physicians/Society of Critical Care Medicine Consensus Conference: definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. Critical Care Medicine. 1992;20(6):864-74.
4. Levy MM, Fink MP, Marshall JC, et al. 2001 SCCM/ESICM/ACCP/ATS/SIS International Sepsis Definitions Conference. Critical Care Medicine. 2003;31(4):1250-6.
5. Zhao H, Heard SO, Mullen MT, et al. An evaluation of the diagnostic accuracy of the 1991 American College of Chest Physicians/Society of Critical Care Medicine and the 2001 Society of Critical Care Medicine/European Society of Intensive Care Medicine/American College of Chest Physicians/American Thoracic Society/Surgical Infection Society sepsis definition. Critical Care Medicine. 2012;40(6):1700-6.
6. Stow PJ, Hart GK, Higlett T, et al. Development and implementation of a high-quality clinical database: the Australian and New Zealand Intensive Care Society Adult Patient Database. Journal of Critical Care. 2006;21(2):133-41.
7. Kaukonen KM, Bailey M, Pilcher D, et al. Systemic inflammatory response syndrome criteria in defining severe sepsis. The New England Journal of Medicine. 2015;372(17):1629-38.
8. Singer M, Deutschman CS, Seymour CW, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016;315(8):801-10.
9. Brabrand M, Havshoj U, Graham CA. Validation of the qSOFA score for identification of septic patients: A retrospective study. European Journal of Internal Medicine. 2016.
10. Chen YX, Wang JY, Guo SB. Use of CRB-65 and quick Sepsis-related Organ Failure Assessment to predict site of care and mortality in pneumonia patients in the emergency department: a retrospective study. Critical Care. 2016;20(1):167.
11. Churpek MM, Snyder A, Han X, et al. qSOFA, SIRS, and Early Warning Scores for Detecting Clinical Deterioration in Infected Patients Outside the ICU. Am J Respir Crit Care Med. 2016.
12. Wang JY, Chen YX, Guo SB, et al. Predictive performance of quick Sepsis-related Organ Failure Assessment for mortality and ICU admission in patients with infection at the ED. The American Journal of Emergency Medicine. 2016;34(9):1788-93.
13. Kolditz M, Scherag A, Rohde G, et al. Comparison of the qSOFA and CRB-65 for risk prediction in patients with community-acquired pneumonia. Intensive Care Medicine. 2016.
14. Vincent JL, Martin GS, Levy MM. qSOFA does not replace SIRS in the definition of sepsis. Critical Care. 2016;20(1):210.
15. Seymour CW, Liu VX, Iwashyna TJ, et al. Assessment of Clinical Criteria for Sepsis: For the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016;315(8):762-74.
16. Shankar-Hari M, Harrison DA, Rowan KM. Differences in Impact of Definitional Elements on Mortality Precludes International Comparisons of Sepsis Epidemiology-A Cohort Study Illustrating the Need for Standardized Reporting. Critical Care Medicine. 2016.
17. Shankar-Hari M, Phillips GS, Levy ML, et al. Developing a New Definition and Assessing New Clinical Criteria for Septic Shock: For the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016;315(8):775-87.
18. Schoonjans F, Zalata A, Depuydt CE, et al. MedCalc: a new computer program for medical statistics. Computer Methods and Programs in Biomedicine. 1995;48(3):257-62.
19. A. J, Hanley, J. B, et al. The Meaning and Use of the Area under a Receiver Operating Characteristic (ROC) Curve1. Radiology 143: 29-36:29-36.
Table 1. Baseline characteristics of the study cohort
Characteristic | Entire cohort N=634 | Sepsis 1.0 N=540 | Sepsis 3.0 N=631 | P value | Survivors N=446 | Non-survivors N=188 | P value |
Age, years | 58.4 ± 17.6 | 57.7 ± 17.7 | 58.4 ± 17.6 | 0.744 | 56.4 ± 17.1 | 62.3 ± 18.0 | <0.001 |
male Sex, (N-%) | 418 (65.9) | 360 (66.7) | 417 (66.1) | 0.675 | 287 (64.3%) | 131 (69.7%) | 0.196 |
APACHEⅡ | 24.6 ± 7.2 | 24.9 ± 7.4 | 24.7 ± 7.2 | 0.660 | 22.9 ± 6.6 | 28.8 ± 7.1 | <0.001 |
SOFA | 9.7 ± 3.9 | 9.8 ± 3.9 | 9.7 ± 3.8 | 0.452 | 8.9 ± 3.6 | 11.7 ± 3.7 | <0.001 |
SIRS | 2.6 ± 1.0 | 2.9 ± 0.8 | 2.6 ± 1.0 | <0.001 | 2.5 ± 1.0 | 2.6 ± 1.0 | 0.173 |
qSOFA | 1.8 ± 0.8 | 1.9 ± 0.7 | 1.8 ± 0.8 | 0.068 | 1.7 ± 0.8 | 2.0 ± 0.7 | <0.001 |
Hospital length of stay, median (IQR), d | 22 (12.2-35) | 21.0 (12.0-35.0) | 21.0 (12.0-33.0) | 0.824 | 25 (15-39) | 16 (8.8-25) | <0.001 |
ICU length of stay, median (IQR), d | 13 (7-24) | 11.0 (6.0-21.0) | 11.0 (6.0-21.0) | 0.774 | 14 (7.2-26) | 11.5 (6-20) | 0.002 |
28-days mechanical ventilation, median (IQR), d | 8 (4-16) | 6.0 (3.0-14.8) | 6.0 (3.0-14.0) | 0.649 | 8 (3-15) | 10.5 (6-18) | <0.001 |
Duration of CRRT, median (IQR), d | 10 (5-20) | 8.0 (4.0-16.5) | 9.0 (4.2-17.0) | 0.872 | 9.5 (5.2-23.5) | 10 (5-15) | 0.381 |
Mechanical ventilation, (N-%) | 573 (90.4%) | 489 (90.6) | 573 (90.8) | 0.767 | 389 (87.2) | 184 (97.9) | <0.001 |
**CRRT (N-%) | 109 (17.2) | 96 (17.8) | 109 (17.3) | 0.652 | 56 (12.6) | 53 (28.2) | <0.001 |
Hospital mortality, (N-%) | 188 (29.7) | 168 (31.1) | 188 (29.8) | 0.331 | NA | NA | NA |
Vasopressin, (N-%) | 212 (33.4) | 189 (35) | 212 (33.6) | 0.316 | 119 (26.7) | 93 (49.5) | <0.001 |
Abbreviation: APACHE Acute Physiology and Chronic Health Evaluation; SOFA Sepsis-related Organ Failure Assessment; SIRS Systemic Inflammatory Response Syndrome; qSOFA quick Sepsis-related Organ Failure Assessment; CRRT Continuous renal replacement therapy. Normal distributed data are expressed as mean ± standard deviation
The Comparison between Sepsis 1.0 and Sepsis 3.0 in Database of Severe Infection in Critical Illness
Table 2. Outcomes of univariate analysis for hospital mortality and ICU length of stay 3 days or more
Hospital mortality | ICU length of stay ≥3 days | |||
Variables | Odds ratio(95%CI) | P value | Odds ratio(95%CI) | P value |
Age, years | 1.02(1.01, 1.03) | <0.01 | 1.00 (0.99, 1.01) | 0.91 |
Sex | ||||
Male | 1.0 | 1.0 | ||
Female | 0.79(0.54, 1.13) | 0.20 | 1.07 (0.77, 1.48) | 0.71 |
Hospital length of stay | 0.98(0.96, 0.99) | <0.01 | 1.08 (1.06, 1.10) | <0.01 |
ICU length of stay | 0.99 (0.97, 1.00) | 0.09 | − | − |
28-days mechanical ventilation | 1.03 (1.01, 1.05) | 0.01 | 1.31 (1.26, 1.37) | <0.01 |
Mechanical ventilation, (N-%) | 6.74(2.37, 19.16) | <0.01 | 1.62 (0.94, 2.78) | 0.08 |
Duration of CRRT | 0.99(0.96, 1.02) | 0.45 | 1.19 (1.07, 1.31) | <0.01 |
**CRRT (N-%) | 2.73 (1.79, 4.18) | <0.01 | 1.77 (1.16, 2.71) | <0.01 |
SOFA | 1.22 (1.16, 1.28) | <0.01 | 0.99 (0.95, 1.03) | 0.51 |
APACHE Ⅱ | 1.14 (1.10, 1.17) | <0.01 | 1.01 (0.99, 1.03) | 0.48 |
Vasopressors | 2.69 (1.89, 3.84) | <0.01 | 0.78 (0.56, 1.09) | 0.14 |
SIRS | 1.13 (0.96, 1.33) | 0.15 | 1.15 (0.98, 1.34) | 0.09 |
qSOFA | 1.58 (1.25, 1.99) | <0.01 | 1.26 (1.02, 1.56) | 0.03 |
Abbreviation: APACHE Acute Physiology and Chronic Health Evaluation; SOFA Sequential Organ Failure Assessment; SIRS Systemic Inflammatory Response Syndrome; qSOFA quick Sequential Organ Failure Assessment; CRRT Continuous renal replacement treatment. Normal distributed data are expressed as mean ± standard deviation
Table 3. Multivariate regression analysis of the risk of hospital mortality and ICU length of stay 3 days or more
Independent variables | Non-adjusted OR, 95%CI, p | Adjusted I (age and sex) OR,95%CI, p | Adjusted II (APACHE II ) OR,95%CI, p |
Hospital mortality | |||
qSOFA | 1.58 (1.25, 1.99), <0.01 | 1.70 (1.34, 2.15), <0.01 | 1.17 (0.90, 1.53), 0.23 |
SIRS | 1.13 (0.96, 1.33), 0.15 | 1.24 (1.04, 1.49), 0.02 | 1.00 (0.84, 1.20), 0.96 |
SOFA | 1.22 (1.16, 1.28), <0.01 | 1.23 (1.17, 1.29), <0.01 | 1.12 (1.05, 1.19), <0.01 |
ICU length of stay 3 days or more | |||
qSOFA | 1.26 (1.02, 1.56), 0.03 | 1.27 (1.03, 1.57), 0.03 | 1.27 (1.02, 1.58), 0.04 |
SIRS | 1.15 (0.98, 1.34), 0.09 | 1.16 (0.99, 1.36), 0.07 | 1.14 (0.98, 1.33), 0.10 |
SOFA | 0.99 (0.95, 1.03), 0.51 | 0.99 (0.95, 1.03), 0.53 | 0.97 (0.92, 1.02), 0.18 |
Abbreviation: APACHE Acute Physiology and Chronic Health Evaluation; SOFA Sequential Organ Failure Assessment; SIRS Systemic Inflammatory Response Syndrome; qSOFA quick Sequential Organ Failure Assessment