Development and Validation of a New Risk Scoring System for Cancer Patients with Suspected Infection

Purpose: This study aimed to develop a new prognostic model for predicting 30-day mortality in cancer patients with suspected infection. Methods: This study is a retrospective cohort study and was conducted from August 2019 to December 2019 at a single center. Adult active cancer patients with suspected infection were enrolled among visitors to the emergency room (ER). Logistic regression analysis was used to identify potential predictors for a new model. Results: A total of 899 patients were included; 450 in the development cohort and 449 in the validation cohort. Six independent variables predicted 30-day mortality: Eastern Cooperative Oncology Group (ECOG) performance status (PS), peripheral oxygen saturation (SpO 2 ), creatinine, bilirubin, C-reactive protein (CRP), and lactate. The C-statistic of the new scoring system was 0.799 in the development cohort and 0.793 in the validation cohort. The C-statistics in the development cohort was signicantly higher than those of SOFA [0.723 (95% CI: 0.663–0.783)], qSOFA [0.596 (95% CI: 0.537–0.655)], and SIRS [0.547 (95% CI: 0.483–0.612)]. Conclusions: The discriminative capability of the new cancer-specic risk scoring system was good in cancer patients with suspected infection. The new scoring system was superior to SOFA, qSOFA, and SIRS in predicting mortality.


Introduction
Patients with cancer are susceptible to infection, which can lead to poor outcomes. In a recent cohort study of 1 million sepsis hospitalizations in the United States, one in ve cases was associated with malignancy, and in-hospital mortality was higher in cancer-related sepsis hospitalizations (27.9% vs. 19.5% in non-cancer-related sepsis). 1 The vulnerability of these patients to infection is driven by many factors, including their immunocompromised state caused by anti-cancer treatments, frequent use of broad-spectrum antibiotics, and indwelling catheters. 2,3 Malnutrition caused by disruption of mucosal integrity and insu cient oral intake can also aggravate the immunosuppressive condition of these patients. 4 Therefore, it is critical to recognize the severity of their condition and promptly provide appropriate treatment.
The clinical presentation of cancer patients with infection can differ from the typical signs and symptoms of infection alone. Immunomodulation can often prevent the onset of fever, even in severe cases of infection. Additionally, in ammatory markers can be elevated in both infectious and noninfectious patients with cancer. 5 Organ dysfunction indicators such as elevated levels of creatinine or bilirubin, confused mentality, or respiratory distress are also often chronically held in patients with cancer regardless of infection. Such altered and inconstant clinical features can lead to an inaccurate understanding of the severity of the patient's condition and poor outcomes.
Existing severity scoring systems such as Systemic In ammatory Response Syndrome (SIRS), Sequential Organ Failure Assessment (SOFA), and quick SOFA (qSOFA) have been used to predict the outcomes of critically ill patients. 6,7 However, while studies have reported that SOFA is superior to qSOFA and SIRS, there have been few reports of the accuracy of these scoring systems to assess patients with cancer. 8,9 There is no cancer-speci c prognostic model that considers the characteristics of patients with cancer; therefore, to accurately risk-stratify these patients with suspected infection, a specialized, optimal prognostic model is needed. This study aimed to develop a new scoring system for predicting mortality in cancer patients with suspected infection.

Study design and patients
This study was retrospectively conducted in the emergency room (ER) of a tertiary referral center in Seoul, South Korea. Patients who attended the ER between August 1, 2019 and December 31, 2019 and met the following criteria were included in the study: 1) aged ≥ 18 years, 2) had active solid cancer, and 3) were suspected of having an infection and needed antibiotic treatment systemically by the physician. Active cancer was de ned as any of the following; 1) cancer that has been newly diagnosed within 6 months of study initiation; 2) receiving anti-cancer treatment; and 3) cancer that has progressed within the past 6 months. 8 All patients with suspected infection were performed with laboratory tests, blood and body uid cultures, and imaging tests for detecting infection foci, and then administered antibiotics for therapeutic purposes. Patients diagnosed with hematologic malignancies, had already used antibiotics before the ER arrival, lost at follow-up, did not have adequate workup at the ER, or refused even minimal life-sustaining treatment were excluded. Patients with no suspected infection or low probability of infection were also excluded.
According to the time of ER visit, patients were divided into two groups: a development cohort from August 1st, 2019 to September 30th, 2019 and a validation cohort from October 1st, 2019 to December 31st, 2019. For multiple visits during the study period, only information on the rst visit was collected.
Any patient-identifying information was excluded from the study.

Data collection and evaluation
Data were collected retrospectively from the hospital's electronic medical records. Clinical variables included demographics, comorbidities, type of cancer, cancer stage, Eastern Cooperative Oncology Group (ECOG) performance status (PS) 10 , and initial vital signs, including mental status. Comorbidities, including hypertension, diabetes mellitus, chronic renal disease, chronic liver disease, chronic lung disease, and cardiovascular disease were analyzed based on the medical records at ER presentation. Mental status was assessed according to the Alert/responsive to Voice/responsive to Pain/Unresponsive (AVPU) scale. The AVPU values could be substituted to Glasgow Coma Scale (GCS) scores of 15, 13, 8, and 6, respectively. 11 A score of < 15 on the GCS was de ned as an indication of altered mental status.
Three sets of blood cultures were obtained before the administration of antibiotics during the ER stay; if patients had a catheter, one set of the three was drawn from the catheter. Laboratory data included complete blood count, chemistry, electrolytes, coagulation battery, in ammatory markers such as Creactive protein (CRP), and serum lactate. The rst obtained value was used in cases with multiple test results. For lactate, the values higher than 15.0 mmol/L were reported as '>15.0 mmol/L'. The SIRS, qSOFA, and SOFA scores were calculated based on the ER's physiological and laboratory data.

Statistical analysis
The primary outcome was 30-day mortality. Data were reported as the mean ± standard deviation or median and inter-quartile range for continuous variables. They were compared between groups using the Student's t-test. Categorical variables were presented as the number and percentage and were compared using a chi-squared test.
To develop a new scoring system for 30-day mortality, univariate and multivariate logistic regression analyses were performed with an entering procedure in the development cohort. Results were summarized as odds ratios (OR) and respective 95% con dence intervals (CI). Variables with a P-value of < 0.1 in the univariate analysis and clinical relevance were considered for the multivariate analysis.
A simple risk score was devised using the penalized maximum likelihood estimates of the predictors in the multivariable model. The constant of the scoring system was de ned as one-third of the regression coe cient of ECOG PS 3-4. The score was the weighted sum of those predictors. The weights were de ned as the rounded integer value of the regression coe cients' quotient value divided by the constant. The risk score's discrimination capability was assessed using the C-statistic. The calibration capability of the risk score was assessed by the Hosmer-Lemeshow test and calibration plot. Internal validation was performed by bootstrapping with 1000 iterations and calculated optimism-corrected C-statistic. The risk score was then categorized into low, intermediate, and high-risk groups based on the likelihood of 30-day mortality. A P-value of < 0.05 was considered statistically signi cant. 12,13 The predictive performances of the SOFA, qSOFA, SIRS scores, and the new scoring system were analyzed by using the area under the receiver operating characteristic (AUROC) values and compared with Delong's test. An AUROC of 1.0 denotes perfect, whereas a value close to 0.50 indicates no apparent accuracy. All statistical analyses were performed using SAS software version 9.4 (SAS Institute Inc., Cary, NC), R version 3.6.1 (R Foundation for Statistical Computing, Vienna, Austria, http://www.R-project.org), and IBM SPSS Statistics for Windows, version 21.0 (IBM Corp., Armonk, NY, USA).

Baseline characteristics of the total population
Among the 1067 patients with cancer screened, 899 patients were included in the analysis: 450 in the development cohort and 449 in the validation cohort. The remaining 178 were excluded for the following reasons: 111 patients had hematologic malignancies, 37 patients did not have adequate workup, 22 used antibiotics before ER arrival, and 8 were lost at follow-up (Supplementary S.1).

Discussion
We have developed a new cancer-speci c scoring system for solid cancer patients with suspected infection. This new system had a strong discriminative power to predict 30-day mortality, showing a Cstatistic of 0.799 in the development cohort and 0.793 in the validation cohort. In comparison with existing scoring systems, the new scoring system was superior to SOFA (C-statistic, 0.723), qSOFA (Cstatistic, 0.596), and SIRS (C-statistic, 0.547). To our knowledge, our new prognostic model is the rst cancer-speci c scoring system to predict mortality for cancer patients with suspected infection. This new scoring system was developed and validated in two groups of similar size, recruited at different times in the same hospital. There was no signi cant difference in baseline characteristics except anticancer treatment between the two development and validation cohorts. In our study, the overall 30-day mortality rate was 22.5%. There was a difference in 30-day mortality between development and validation cohorts (19.3% vs. 25.6%, P = 0.024). However, the C-statistic (0.799 in the development cohort, 0.793 in the validation cohort) of our new scoring system showed a good discriminative capability to predict 30day mortality in both groups. The Hosmer-Lemeshow goodness to t test suggested that predicted mortality re ects true mortality, and thus our scoring system is well-calibrated. Several prognostic scoring systems have been used to predict prognosis in patients with infection. [14][15][16] SOFA is one of the most frequently validated systems and is an excellent predictor of mortality. 17 However, most of the SOFA studies were on non-cancer patients, and studies on those with cancer were limited with inconsistent results 8, 9,18 . In a recent study, SOFA had good discriminative power in patients with cancer and was superior to qSOFA. 8 In two previous studies, both SOFA and qSOFA showed weak discriminative ability with a C-statistic of < 0.7 in predicting mortality for cancer patients with infection. 9,18 As shown in this study, our new scoring system can be a good alternative to predict mortality in cancer patients with suspected infection.
The new cancer-speci c scoring system consisted of six components: ECOG PS, SpO 2 , creatinine, total bilirubin, CRP, and lactate. These six components re ect the underlying condition of patients with cancer and acute responses to infection. The ECOG PS of these patients has been considered as an essential prognostic factor. 19,20 ECOG PS had the highest score distribution among the new scoring system variables in our study. Performance status is affected by many factors such as the patients' age, cancer stage, and side effects of anti-cancer treatment. Patients who have a poor PS and limited functional capacity tend to have more di culty tolerating rigorous cancer treatments. These patients have less favorable outcomes than those with a better PS, regardless of distant metastasis or treatments given. 21 In this study, ECOG PS was a signi cant prognostic factor in cancer patients with suspected infection, whereas advanced stage or anti-cancer treatment were not. Lactate represents tissue hypoperfusion, and lactate > 2 mmol/L was introduced as diagnostic criteria of septic shock. 22 Furthermore, lactate has been shown to have prognostic power in cancer patients with sepsis. 8 Increased CRP is an indicator of in ammation in patients with sepsis. A recent study reported that CRP carries signi cant independent prognostic information. 23 , which was consistent with the results of our study; elevated CRP was associated with increased 30-day mortality in cancer patients with suspected infection. Creatinine and total bilirubin are indicators of hepatic and renal function, which are also components of SOFA. 24 SOFA included PaO 2 /FiO 2 as a respiratory indicator. We used SpO 2 in place of PaO 2 /FiO 2 given the ease to continuously and noninvasively obtain at the ER without drawing arterial blood. A previous study reported that SpO 2 was consistently associated with mortality in patients with septic shock. 25 This study has several limitations. First, it was conducted in a single hospital, which was a tertiary referral cancer center. There may have been a high proportion of severe disease. In our study, the 30-day mortality rate was 22.5%; however, in the US population report, cancer-related sepsis hospitalizations had high inhospital mortality of 27.9%. 1 Second, this study used a retrospective design. However, there was very few missing data. In our hospital, there is a separate ER for patients with cancer 26 , in which data collection and the treatment process are standardized. Lastly, we included only solid tumors and excluded hematologic malignancies in this study. Generalization of applying the new scoring system to patients with hematologic malignancies is therefore di cult. Many studies have shown that prognosis of patients with solid tumors and hematologic malignancies is quite different. 27,28 We consider that the prognostic model would have to be different to assess these two groups appropriately.
In conclusion, the new scoring system had a robust discriminative capability to predict prognosis in cancer patients with suspected infection. This new scoring system can be a good alternative for patients with cancer compared with existing scoring systems.

Conclusion
A new risk scoring system in active cancer patients with suspected infection consisted of six components: ECOG PS, SpO 2 , creatinine, total bilirubin, CRP, and lactate. The new scoring system was superior to the existing scoring systems of SIRS, qSOFA, and SOFA in predicting 30-day mortality. We believe that our system can help physicians at the ER to predict prognosis for cancer patients more accurately and inform treatment decisions. Figure 1 The curve; CI, con dence intervals; qSOFA, quick sequential organ failure assessment; SIRS, systemic in ammatory response syndrome; SOFA, sequential organ failure assessment

Supplementary Files
This is a list of supplementary les associated with this preprint. Click to download. Supplementary.docx