Diagnosing Sepsis in the Intensive Care Unit: A Retrospective Cohort Study

Background: Sepsis is defined as life-threatening organ dysfunction caused by a dysregulated host response to infection. In the intensive care unit (ICU), organ dysfunction is common. The challenge lies in determining when organ dysfunction can be attributed to infection. We aimed to retrospectively determine what proportion of patients commenced on antibiotics for presumed sepsis in a mixed ICU had blood-culture positive sepsis, blood-culture negative sepsis, or an aseptic mimic. Methods: One hundred antibiotic naïve ICU patients who were clinically deemed to have an infection were enrolled. Retrospective interpretation of clinical history, biochemical, and microbiological data was performed by three clinicians from the fields of intensive care and infectious disease who aimed to differentiate infective from non-infective insults. Results: There was good interrater reliability amongst clinician assessors using this approach (Krippendorf’s alpha 0.868) for the retrospective diagnosis of infection. In the examined cohort, 35 percent of patients met the criteria of blood culture positivity and an additional 41 percent of patients were assessed as having probable blood culture negative sepsis. Twenty-four percent of patients were retrospectively determined to not have had sepsis. Conclusions: Misdiagnosis of infection as a cause for organ dysfunction in the ICU is common. The false attribution of organ dysfunction to infection in the ICU has significant clinical and research implications, and highlights the need for accurate point-of-care sepsis diagnostic tools.


Background
Sepsis is defined as life-threatening organ dysfunction caused by a dysregulated host response to infection [1]. Organ dysfunction is common in the intensive care unit (ICU) 3 affecting up to 72% of patients at some point during their admission [2]. The challenge in the ICU patient is identifying when infection is the cause of the organ dysfunction.
The timely diagnosis of sepsis and treatment of infection by means of source control and antibiotics is key for reducing sepsis-related morbidity and mortality [3][4][5]. On the other hand, antibiotics do not benefit patients without an infection, and place them at risk of unnecessary antibiotic-associated harms [6].
Microbiological cultures are the 'gold standard' of confirming infection and a positive culture facilitates the administration of appropriate antibiotics. However, many bacterial and fungal culture techniques take time (>24 hours) and culture results may be confounded by contamination, colonization, and "culture negative" infection [6].
Up to fifty percent of patients admitted to the ICU with a syndrome consistent with likely sepsis have negative cultures [3,7,8]. Failure to identify an organism may be due to the patient receiving prior antibiotics, thus obscuring conventional cultures; poor collection technique or failure to obtain cultures; severe localised infection without bacteraemia; or the presence of unusual or slow-growing organisms [9,10]. Alternatively, the patient may have a non-infectious cause for their clinical syndrome.
We aimed to determine what proportion of patients commenced on antibiotics for presumed sepsis in an ICU had blood culture positive sepsis, blood culture negative sepsis, or an aseptic mimic based on retrospective expert interpretation of clinical history, biochemical, and microbiological data. We additionally highlight clinical situations where the retrospective diagnosis of sepsis remains vexed.

Design, setting and population
The study was conducted as part of a larger trial aiming to investigate transcriptome changes in patients with sepsis at a 30-bed, public mixed ICU in Queensland, Australia 4 between October 2015 and July 2018. Patients or their surrogate decision makers were approached for inclusion who met the following criteria: 18 years of age or older, evidence of a systemic inflammatory response syndrome (SIRS), and commencement on antibiotics within the last 24 hours to treat clinically determined sepsis. Patients were excluded if they were: less than 18 years of age, had an underlying haematological malignancy, or were pregnant. Individuals were also excluded if they were administered antibiotics in the 7-day period prior to study inclusion. This criterion was added to remove the confounding variable of partially treated sepsis. The institutional Human Research Ethics Committee approved the study (HREC/15/QRBW/28).

Data Collection
Data was collected from a combination of hospital medical records and laboratory information systems by a medically-trained data abstractor (RL). Data collected from medical records included patient demographics, length of stay (ICU/hospital), mortality (ICU/hospital), need for ventilation/vasopressors/renal replacement therapy during ICU admission, antimicrobial treatment, and source of sepsis. Data collected from the laboratory information system, AUSLAB, included total leucocyte counts, neutrophil counts, trends in serum lactate and results of all microbiological cultures. Derived variables included SIRS criteria [11], Acute Physiology, Age, Chronic Health Evaluation II (APACHE II) score [12], Sequential Organ Failure Assessment (SOFA) and quick SOFA (qSOFA) scores [13]. Study data were collected and managed using REDCap electronic data capture tools hosted at The University of Queensland. If there was disagreement between assessors based on their independent assessmentwherein disagreement was defined as less than 100% agreement between the 3 assessors as to whether sepsis was present or not -the case was discussed at a round-table meeting involving all 3 of the above specialist physicians with the aim of achieving consensus.

Definitions and Expert Assessment
During the face-to-face roundtable meeting, the specialist assessors compared individual notes, revisited clinical notes and laboratory data, and a final decision as to whether the patient had sepsis or not was determined by majority.

Statistical analysis
Variables were presented as number (%) and mean (95% confidence interval [CI] or standard error of the mean [SEM]) or median (bootstrapped 95% CI) as appropriate for 6 categorical and continuous variables respectively. Visual inspection (box plots, histograms and Q-Q plots) of continuous variables and the Shapiro-Wilk test were used to assess data distribution. Differences between the three groups (blood culture positive sepsis, blood culture negative sepsis and aseptic inflammation) for continuous outcome variables were compared using a one-way analysis of variance (ANOVA) or a Kruskal-Wallis H test for parametric and non-parametric outcome variables respectively. Family-wise error post-hoc tests, Tukey's and Dunn's tests, were used for multiple comparisons following ANOVA and Kruskal-Wallis H tests respectively. Proportional differences in ICU mortality, hospital mortality, gender and qSOFA criteria were assessed using a Pearson's χ 2 test or Fisher's Exact test as appropriate. A two-sided P value <0.05 was considered statistically significant. Inter-rater reliability for patient group assignment was assessed using Krippendorf's alpha. An alpha value ≥0.8 was considered as acceptable inter-rater agreement.
Analysis was completed using GraphPad Prism 8 software (Graphpad Software Inc., USA) and R (R Core Team, Vienna, Austria).

Results
One hundred ICU patients were recruited, with an overall ICU mortality of 5% and inhospital mortality of 8% (see Table 1). The mean APACHE II score was 22.5 (SD, 16.6) and mean SOFA score was 8.6 (SD, 8.4).
Of all patients commenced on antibiotics for treatment of presumed sepsis, 35 patients were assessed as having blood culture positive sepsis and 41 with blood culture negative sepsis. Twenty-four patients were retrospectively assessed to not have infection, despite being commenced on antibiotics for presumed sepsis by the treating clinician. All but one patient had blood cultures collected and 57% of patients had a blood culture collected 7 prior to the initiation of antibiotics.
A significant difference existed between mean APACHE II (F[2,97]=3.928, P=0.023) and SOFA (F[2,97]=4.744, P=0.011) scores between groups. Patients given a diagnosis of blood culture positive sepsis had significantly higher APACHE II (difference 6.0, 95% CI 0.9-11.2, P=0.018) and SOFA (difference 2.7, 95% CI 0.6-4.9, P=0.008) scores than patients assessed to have aseptic inflammation (see Table 2). There was no significant difference between APACHE II and SOFA scores for patients with blood culture negative sepsis and aseptic inflammation, and between patients with blood culture positive and blood culture negative sepsis. Renal replacement therapy use differed between groups (P = 0.03). The lactate distribution significantly differed between groups (χ 2 (2)=9.79, P=0.007), however this was only significantly different following post-hoc testing between patients with BC positive sepsis and aseptic inflammation (P=0.003) (see Table 2).
Similarly, the hospital length of stay differed between groups (χ 2 (2)=7.22, P=0.027), that was only significantly different on post-hoc testing between patients with blood culture positive sepsis and those with aseptic inflammation (P=0.011). There were no significant differences in age, gender, leucocyte count, ICU length of stay and ICU-or hospitalmortality between the groups.
The majority of patients with a diagnosis of blood culture positive sepsis had a blood culture collected prior to the administration of antibiotics (80%), compared with 39% in the blood culture negative sepsis group and 54.2% in the aseptic inflammation group.
There was good interrater agreement between assessors (Krippendorf's alpha 0.868, 95% CI: 0.727-0.863). Thirteen patients (13%) required roundtable discussion. Of the 13 patients discussed, 10 were assessed as having likely culture negative sepsis and 3 were assessed as having likely aseptic inflammation. Further details of each of the case 8 presentations that went to roundtable discussion are seen in Table 3.
One hundred ICU patients were recruited, with an overall ICU mortality of 5% and inhospital mortality of 8% (see Table 1). The mean APACHE II score was 22.5 (SD, 16.6) and mean SOFA score was 8.6 (SD, 8.4).
Of all patients commenced on antibiotics for treatment of presumed sepsis, 35 patients were assessed as having blood culture positive sepsis and 41 with blood culture negative sepsis. Twenty-four patients were retrospectively assessed to not have infection, despite being commenced on antibiotics for presumed sepsis by the treating clinician. All but one patient had blood cultures collected and 57% of patients had a blood culture collected prior to the initiation of antibiotics.
A significant difference existed between mean APACHE II (F[2,97]=3.928, P=0.023) and SOFA (F[2,97]=4.744, P=0.011) scores between groups. Patients given a diagnosis of blood culture positive sepsis had significantly higher APACHE II (difference 6.0, 95% CI 0.9-11.2, P=0.018) and SOFA (difference 2.7, 95% CI 0.6-4.9, P=0.008) scores than patients assessed to have aseptic inflammation (see Table 2). There was no significant difference between APACHE II and SOFA scores for patients with blood culture negative sepsis and aseptic inflammation, and between patients with blood culture positive and blood culture negative sepsis. Renal replacement therapy use differed between groups (P = 0.03). The lactate distribution significantly differed between groups (χ 2 (2)=9.79, P=0.007), however this was only significantly different following post-hoc testing between patients with BC positive sepsis and aseptic inflammation (P=0.003) (see Table 2).
Similarly, the hospital length of stay differed between groups (χ 2 (2)=7.22, P=0.027), that was only significantly different on post-hoc testing between patients with blood culture positive sepsis and those with aseptic inflammation (P=0.011). There were no significant 9 differences in age, gender, leucocyte count, ICU length of stay and ICU-or hospitalmortality between the groups.
The majority of patients with a diagnosis of blood culture positive sepsis had a blood culture collected prior to the administration of antibiotics (80%), compared with 39% in the blood culture negative sepsis group and 54.2% in the aseptic inflammation group.
There was good interrater agreement between assessors (Krippendorf's alpha 0.868, 95% CI: 0.727-0.863). Thirteen patients (13%) required roundtable discussion. Of the 13 patients discussed, 10 were assessed as having likely culture negative sepsis and 3 were assessed as having likely aseptic inflammation. Further details of each of the case presentations that went to roundtable discussion are seen in Table 3.

Discussion
In the present study, 24% of patients who were commenced on antibiotics by the treating intensive care team for presumed sepsis were retrospectively assessed as having aseptic inflammation. This has significant clinical and research implications ( Figure 1).
Clinically, approximately one in four patients in the current cohort were subject to potential antibiotic-associated harms without any benefit. Antibiotic-associated harms include adverse drug events, risk of secondary opportunistic infections, Clostridioides difficile infection, disruption to the microbiome, and antimicrobial resistance [6,14,15].
Antibiotic-associated harms are common in ICU patients, partly due to the underlying critical illness that increases susceptibility to organ injury [6]. On the other hand, the harms of missing a diagnosis of sepsis and delaying appropriate antimicrobial therapy and opportunities for source control also need to be considered. In the absence of an accurate, rapid and accessible point-of-care test to aid in sepsis diagnosis, the inadvertent administration of antibiotics to patients with aseptic inflammation is inevitable.
There are also research implications. Many large and widely-cited prospective sepsis trials have inclusion criteria that rely heavily on whether the treating clinician commences antibiotics for the treatment of presumed sepsis [1,[16][17][18][19]. Whilst this may be a current necessity for conducting feasible clinical trials, the inadvertent inclusion of patients who have aseptic inflammation is likely to obscure the results of studies investigating the effectiveness of various sepsis interventions.
Only 35 percent of all patients with evidence of systemic inflammation who were commenced on antibiotics by the treating clinician met criteria of blood culture positivity.
This low rate of bacteraemia occurred despite all included patients being antibiotic naïve for the 7 days prior to study inclusion. This bacteraemia rate is consistent with that found in many large, multi-centre sepsis trials including the ADRENAL trial (35%) [17], ARISE trial (38%) [19], and PROCESS trial (29.5%) [20]. Notably, in the current cohort, the majority (24/40, 60%) of patients in the blood culture negative sepsis group had blood cultures taken subsequent to the commencement of antibiotics. This compares to the blood culture positive group, where the majority (28/35, 80%) of blood cultures were collected prior to antibiotic administration. It has been previously demonstrated in a prospective cohort trial that obtaining blood cultures after the initiation of antibiotics is associated with a significant loss of pathogen detection [10].
To address the problem of blood culture negative sepsis, the present study utilised multidisciplinary expert-based retrospective interpretation of clinical history and treatment response, laboratory data, and other available microbiology results. Using this method, inter-rater reliability between experts was good, with clinically substantial disagreement only occurring for 13 percent of patients. The cases wherein disagreement occurred highlight clinical situations wherein, even with the luxury of retrospective interpretation of all available data, the diagnosis of sepsis remains challenging.
The in-hospital and ICU mortality observed in the present study was surprisingly low, less 11 than predicted by the APACHE II and SOFA scores for the cohort and less than that of larger international sepsis trials with comparable illness severity scores to the present cohort [17,19,21]. This may be explained in part by the exclusion criteria, wherein patients who were not antibiotic naïve for the previous 7 days, or those with haematological malignancies were not approached for inclusion.
The above study has several limitations. Firstly, the study was conducted at a singlecentre, thus potentially limiting the generalisability of the data. Secondly, there is the potential for interpreter bias as a result of expert assessment of clinical and laboratory data. Attempts were made to minimise this through using multiple assessors from both intensive care and infectious disease specialities and inter-rater reliability using this approach was good. Finally, only patients commenced on antibiotics were included in this study. Thus, this study was not designed to investigate the possibility that some patients with sepsis were not appropriately treated.
The retrospective approach to sepsis determination used in the present study may not be appropriate for larger, multi-centre studies as it requires significant clinician engagement.
However, it does provide a "gold standard" approach for smaller, expensive biomarker discovery studies which, in turn, can generate novel diagnostic strategies to assist in accurate sepsis identification.

Conclusions
The diagnosis of sepsis in the ICU is challenging. We recommend future research efforts focus on improved diagnostic tools that rapidly and accurately identify infection in critically ill patients.