Predicting length of stay in intensive care units for Cardiovascular Surgery patients using APACHE II, APACHE IV, SAPS II and SOFA Scores

doi:10.21203/rs.3.rs-3807484/v1

Background and objective:

Length of Stay (LOS) in the ICU may serve as a marker for the effectiveness of care. In intensive care medicine, prediction models and scoring systems are frequently used for prognosis, quality assessments, and comparing different intensive care units (ICUs) and resource allocation. To measure morbidity and distinguish patients in ICUs, a collection of models has been developed. The purpose of this research is to evaluate and compare the prognostic performance of the SAPS II, SOFA, APACHE II, and APACHE IV models for predicting length of stay in the Cardiovascular Surgery Intensive Care Unit (CSICU) on a large sample of Cardiovascular surgery patients.

Method

In a retrospective cohort study, data on 2587 consecutive CSICU patients were collected in the Imam Reza hospital between December 2013 and April 2022. These data gathered in an CSICU registry. We used these four models to predict ICU LOS via linear regression and the original and recalibrated, SOFA, APACHE IV, APACHE II, and SAPS II for all cardiovascular patients. We assessed the predictive performance of the models (R squared (R2), Adjusted R2, Intraclass Correlation Coefficient (ICC) for agreement, F-test, Akaike information criterion (AIC), Bayesian Information Criterion (BIC), Root Mean Squared Prediction Error (RMSPE)), discrimination using Area Under the Curve (AUC), and calibration by calibration graph.

Results

Of 3114, only 2587 patients were included. They were 56.8 (13) years old on mean (SD), 40.2% were female, the mean (SD) score for SOFA, SAPS II, APACHE IV, and APACHE II were 8,5 (2.1), 50.4 (14.3), 82.7 (20.2), and 22.8 (6), respectively. they had 11.1% overall mortality rate, and 29.7% were mechanically ventilated. Aggregate mean observed ICU stay was 3.6 (5) days and the means estimated by SOFA, SAPS II, APACHE IV, and APACHE II were 4, 4, 5.67, and 3.03 days, accordingly. For the APACHE II and APACHE IV, the R-squared was 0.005, while it was zero for the SOFA and SAPS II. The RMSE for SOFA, SAPS II, APACHE IV, and APACHE II were 5.181, 5.182, 5.878, and 5.170 respectively. APACHE II and ICU LOS had a very significant correlation (r = 0.89). Mortality, mechanical ventilation, gender, GCS, Serum creatinine, blood sugar, white blood cell, respiratory rate, age, and APACHE II were the variables that had an effect on length of stay (p < = 0.05).

Conclusion

These four models were studied to predict CSICU LOS in Iran as a developing country for the first time. The APACHE IV and APACHE II models better results than the other two models. Furthermore, APACHE IV's calibration for estimating length of stay, discriminations, and fit for data than others was just moderate. Although, APACHE II had better prediction the target value (accuracy) than the other three models. None of these four models completely satisfies our demands for CSICU LOS prediction models or our particular needs for models for resource allocation planning or benchmarking purposes. External validation, customizing the models, and using machine learning techniques could be helpful to predict CSICU LOS via these models.

Health sciences/Cardiology/Interventional cardiology

Health sciences/Health care/Prognosis

Regression

Intensive care units

Prediction models

Length of stay

Scoring system

CSICU

A hospital's intensive care unit (ICU) is a dedicated department that diagnose, manage and follow up seriously ill patients who may have respiratory problems, multiple organ failure, or different serious issues that need intensive monitoring (1, 2). Hospitals are under pressure to reduce costs and optimize efficiency because of ICUs expensive and complex care (3, 4). Several quality indicator are presented in ICU (5, 6). Length of stay (LOS) as one of them, is extremely associated with the ICU costs and ICU care efficiency (7). Many scoring systems were developed to estimate severity of illness in ICU that can provide a common understanding patient status and risk(8). It is noteworthy that predictive behavior of scoring systems is affected by various admission/discharge policies and case mixes.

Clinicians predict ICU LOS for three primary reasons: 1) enabling case-mix correction when comparing average LOS between ICUs (benchmarking); 2) determining particular patients or collections of patients with unintentionally long ICU LOS for lead improvements in quality; and 3) planning the number of beds and staff members required to fulfill demand for ICU care (9, 10).

The Cardiovascular Surgery Intensive Care Unit (CSICU) is an inpatient department that provide treatment of adult patients before and after the critical period of Cardiovascular and thoracic surgery. Cardiovascular surgery, covering aortic, Cardiovascular valve, and coronary artery bypass surgeries, is one of the most often carried out surgical procedures in Iran.

A system for classifying severity is called the Acute Physiology and Chronic Health Evaluation (APACHE) that prognose ICU main outcomes. APACHE II uses 12 physiologic measurement and scoring range from 0 to 71(11) and the most effective forecasting indicator in intensive care units for many years. APACHE IV was released as a new adopted version to improve the accuracy of earlier models in 2006 and used for evaluating mortality from clinical data (18 variables) and predicting the length of stay of ICU in the first day after admission (12). The sepsis-related organ failure assessment (SOFA) is a ranking system according 6 different score that was developed in 1994 and explain the severity of organ breakdown or malfunction (13). Another classification system is the Simplified Acute Physiologic Score (SAPS II) including 17 variables that estimate mortality probability in the ICU and developed in 1993 (14). All these scoring measurements used to estimate ICU outcomes.

This study aimed to estimate and compare the SOFA, SAPS II, APACHE II, and APAPCHE IV scores' prognostic performance for CSICU patients in Iran. Data provided in the Cardiovascular Surgery Intensive Care Unit registry were used in this research.

Study Population and setting

In a retrospective cohort study, data were collected between December 2013 and April 2022 in the Imam Reza hospital - at the time of the study, The greatest academic medical center in northeastern and eastern Iran - aiming to include all admitted patient in cardiovascular surgery ICU. In Imam Reza hospital, a 21-bed separate unit known as the Cardiovascular Intensive Care Unit (CSICU) treats critically ill children, teens and young adults with heart problems. It is staffed by a group of trained specialists. At least one Cardiovascular intensive care physician is present in the CSICU 24 hours a day, all week. In 2020, first phase of ICU registry named “Cardiovascular Surgery Intensive Care Unit (CSICU)”, under the supervision of specialists, the approval of Mashhad University of Medical Sciences and ministry of Health was designed and developed. Along with daily registration by an expert and final confirm by the ICU specialist, previous data from hospital information system (HIS) and paper files, after validation, imported into current registry from the last nine years.

Exclusion and Inclusion Criteria

Each mature patients (18 years of age and older) with severe cardiovascular disease like acute coronary syndrome, acute heart failure, and fatal arrhythmia who likely need surgery – transferred from other department- were considered. The subsequent patients were not included in the analysis: discharged patients before four hours, died upon arrival, or under 18 years old. Also, the study did not include individuals without a medical examination on the floor or in another section. Therefore, these patient types are not intended to benefit from our model. Additionally, the patient's record was omitted from the data set since it included missing values, if it was not possible to calculate the prediction score for the patient.

Study Variables

All patient’s information including demographics, operation data (preoperative, intraoperative, postoperative), laboratory data, drugs, imaging, admission and discharge were stored in CSICU registry. Key clinical characteristics to calculate scores were extracted from this registry. Age, mechanical ventilation, chronic health condition, admission information, Glasgow Coma Scale (GCS), and vital signs on arrival were extracted. Laboratory factors include urea, creatinine, arterial blood gas analysis, electrolytes, blood sugar, albumin, bilirubin and so on were added for each particular patient. Also, Blood pressure (systole/diastole), heart rate, respiration rate, temperature, and oxygen saturation all instances of vital indicators. After extracting all the required data, the SAPS II, SOFA, APACHE IV, and APACHE II scores were estimated. The variables to estimate these scores, were calculated during the first twenty-four hours of the patient's admission. Two observed outcomes of ICU –ICU length of stay and ICU mortality – as factors to compare and analysis results were extracted from registry. The day of admission with the day of ICU discharge were subtracted to determine the length of stay in the ICU.

Statistical Analyses

The predictive performance measures for the models’ analysis were overall performance, discrimination, and calibration. The distance between the observed outcome and the predicted outcome is the basis to measuring overall performance of the model (15). Since our desired outcome in this study -ICU length of stay- is continues, this distance is subtraction of actual outcome from predicted outcome (\(Y-\widehat{Y}\)). Smaller gaps exist between expected and actual outcomes in better models. The most common performance metric for continuous outcomes is coefficient of determination, which is more commonly known as explained variation (R-squared/R²), quantifies the degree of any linear correlation between observed outcome and predicted outcome(16). The R-squared formula is:\({R}^{2}=1-\frac{S{S}_{res}}{S{S}_{tot}}\)

\([ S{S}_{res}=\sum _{i}{({actual outcome}_{i}-{predicted outcome}_{i})}^{2}, S{S}_{tot}=\sum _{i}{({actual outcome}_{i}-mean of observed data)}^{2}\) ]

The Adjusted R² (\({\stackrel{-}{R}}^{2}\)) as a modified version of R², considers different independent variables against the model and has less bias. The adjusted R² is defined as: \(1-\frac{S{S}_{res}/(n-p)}{S{S}_{tot}/(n-1)}\), (n: sample size, p: number of variables). If the model exactly matches the observed values, result is \({R}^{2}=1\). R² is always less or the same as adjusted R². A value that is less than or equal to 0 indicates a model that has no predictive value. The actual adjusted R² value is in the middle of these ranges. The Root Mean Square Error (RMSE) is another main performance indicators for regression models that assess average difference between predicted outcome and actual outcome. It indicated model accuracy. The RMSE formula is: \(\sqrt{\frac{{\sum }_{i=1}^{N}{({predicted outcome}_{i}-{actual outcome}_{i})}^{2}}{N}}\). A lower Root Mean Square Error (RMSE) indicates a better model. RMSE of zero value shows a perfect model. The sensitivity, specificity, and accuracy were all determined using the Youden Index technique. Model performance metrics that take model complexity into consideration can be obtained by the Bayesian information criterion (BIC) and the Akaike information criterion (AIC). The BIC and AIC used to compare models and determine which one is the best fit for the data. AIC is defined as \(2k-2\text{l}\text{n}\left(\widehat{L}\right)\) and BIC formula is \(k\text{ln}\left(n\right)-2\text{l}\text{n}\left(\widehat{L}\right)\), which L̂ is the model's likelihood function's maximal value. An F test often used for comparing models to determine the model that best fit the population. F is calculated by \(\left(\right({R}^{2}/k)/(\frac{1-{R}^{2}}{n-k-1}\left)\right)\). Intraclass correlation (ICC) is a measure of reliability used to assess the degree of agreement between two or more models when they rate the same subject or object. It is a statistical measure of the degree of agreement among raters. The mean square between raters (MSB), the mean square within raters (MSW), and the number of raters (k) form the formula for ICC, which is ICC = (MSB - MSW) / (MSB + (k-1) MSW). The degree of agreement between the two outcomes was measured using a calibration graph that had the proportion of observed occurrences on the y-axis and the expected results on the x-axis. Perfect predictions are on the 45-degree line. For our continuous outcome, LOS, a typical scatter plot acts as the calibration plot (16). Using SAPS II, SOFA, APACHE IV, and APACHE II scores (independent variables), To forecast the ICU LOS (dependent variables), linear regression analysis was employed. All variables were classified in two categories, continuous and categorical, and descriptive statistics apply to them. With continuous variables, the t-test provided the mean, standard deviation, and p-value. Using Fisher's exact test or the Chi-square test, the frequency (%) and p-value for categorical data were determined. R studio and SPSS were used for all statistical studies.

Of 3114 included patients to study, after employing the exclusion criteria, 527 patients were excluded and 2587 patients remained for further analysis. Based on variable types (continuous, categorical), Tables 1 and 2 describe the key features of the included patients. Of the included patients, 1546(59.8%) were men and 1041(40.2%) were women. Patients' mean (SD) age was 56.8 (13) years, with a range of 18 to 93 years (P-value < 0.05). The median age of the patients was 58. 11.1% of the patients passed away in the CSICU, of which 75% had LOS \(\le\) 7 and 25% had LOS \(>\) 7 (P-value < 0.05). Aggregate mean observed ICU stay was 3.6 (5) days and the means estimated by SAPS II, SOFA, APACHE IV, and APACHE II were 4., 3.03, 5.67, and 4 days, accordingly (See Fig. 1, 2, and 3). Considerable differences have been observed for the LOS of patient with regards to age, respiratory rate, white blood cell, serum creatinine, blood sugar, gender, GCS, mortality, and mechanical ventilation. However, For the patient LOS, no significant differences in terms of other parameters were observed. More detail of clinical, hematologic and biochemical presented in the tables mentioned above and the information is categorized based on CSICU LOS days less or equal to 7 and more than 7 for further analysis.

Table 1

Baseline continuous characteristics of patients admitted to the CSICU
Characteristics	LOS \(\le 7\)(N=2386)		LOS \(>7\)(N=201)		Total (N = 2587)		P-value
Characteristics	Mean\(\pm\)SD	Min-Max	Mean\(\pm\)SD	Min-Max	Mean\(\pm\)SD	Min-Max	P-value
Age (year)	56.4 ± 13.1	18–90	59.4 ± 12.6	22–93	56.8 ± 13	18–93	0.000
Clinical Parameters
Temperature	36.4 ± 0.5	35.2–37.3	36.4 ± 0.5	35.5–37.3	36.4 ± 0.5	35.2–37.3	0.945
MAP	95.2 ± 22.2	46–146	96.5 ± 23.1	46–146	95.3 ± 22.3	46–146	0.317
Heart Rate	89 ± 29.3	40–140	90.9 ± 28.9	40–140	89.2 ± 29.3	40–140	0.281
Respiratory Rate	14.9 ± 3.6	5–35	15.8 ± 5.8	5–35	15 ± 3.9	5–35	0.000
Urine Output	1226.7 ± 448	450–2000	1229.6 ± 440.5	460–2000	1227 ± 447	450–2000	0.916
Gasometry
PH	7.4 ± 0	7.34–7.8	7.4 ± 0	7.34–7.8	7.4 ± 0	7.34–7.8	0.361
FIO2	64.5 ± 29.2	21–99	63.6 ± 30.4	21–99	64.4 ± 29.4	21–99	0.617
PO2	87.2 ± 7.1	75–99	87.2 ± 7	75–99	87.2 ± 7.1	75–99	0.895
PCO2	39.5 ± 2.8	35–44	39.5 ± 2.8	35–44	39.5 ± 2.8	35–44	0.924
Hematology
Platelet	260.1 ± 146.5	0-603	254.1 ± 156.4	2-561	259.4 ± 147.6	0-603	0.506
Hematocrit	39.4 ± 13.5	11.3–68.2	40.6 ± 15.3	11.3–68	39.5 ± 13.7	11.3–68.2	0.145
White Blood Cell	19.8 ± 12.7	2.9–47	21.5 ± 13	2.9–47.1	20 ± 12.8	2.9–47.1	0.035
Biochemistry
Sodium	137.1 ± 15.1	106–168	135.7 ± 16.2	106–169	136.9 ± 15.2	106–169	0.144
Potassium	4.7 ± 1.3	2.3–7.9	4.7 ± 1.3	2.3–7.7	4.7 ± 1.3	2.3–7.9	0.933
Urea	14.6 ± 5.1	6–23	14.6 ± 5.2	6–23	14.6 ± 5.1	6–23	0.900
Blood sugar	247.5 ± 101.3	71–419	233 ± 98.9	72–420	245.8 ± 101.1	71–420	0.019
Serum creatinine	0.9 ± 0.4	0.4–3.6	1.1 ± 0.6	0.5–3.6	1 ± 0.4	0.4–3.6	0.000
Albumin	3.5 ± 1.1	1.6–5.9	3.5 ± 1.1	1.6–5.8	3.5 ± 1.1	1.6–5.9	0.904
Bilirubin	3.2 ± 2	0.2–7.1	3.2 ± 2	0.2–13.5	3.2 ± 2	0.2–13.5	0.592
Risk scores
APACHE IV	82.5 ± 20	33–139	84.9 ± 22.7	11–137	82.7 ± 20.2	11–139	0.105
APACHE II	22.6 ± 6	8–41	24.5 ± 6.4	9–41	17.8 ± 6	8–41	0.000
SAPS II	50.4 ± 14.4	6–94	50 ± 15.9	13–95	50.4 ± 14.5	6–95	0.712
SOFA	8.5 ± 2.1	3–16	8.6 ± 2.4	3–16	8.5 ± 2.1	3–16	0.787
Length of Stay	2.6 ± 1.4	1–7	15.4 ± 12.3	8–99	3.6 ± 5	1–99

Abbreviations: PaO2, Partial pressure of arterial oxygen; FiO2, Fraction of inspired oxygen; PCO2, Partial pressure of carbon dioxide; HCO3, Bicarbonate; MAP, Mean arterial pressure; GCS, Glasgow coma scale; APACHE, Acute Physiologic and Chronic Health Evaluation (version II and IV); SAPS, Simplified Acute Physiology Score (version II); LOS, Length of Stay.

Table 2

Baseline categorical characteristics of patients admitted to the CSICU
Characteristics	LOS \(\le 7\)(N=2386)	LOS \(>7\)(N=201)	Total (N = 2587)	P-value
Characteristics	Count (Percent)	Count (Percent)	Count (Percent)	P-value
Gender
Male	1438(93%)	108(7%)	1546(59.8%)	0.042
Female	948(91.1%)	93(8.9%)	1041(40.2%)	0.042
GCS
3	348(90.6%)	36(9.4%)	384(14.9%)	0.036
4–6	357(93%)	27(7%)	384(14.9%)
7–8	349(93.8%)	23(6.2%)	372(14.4%)
9–10	656(92%)	57(8%)	713(27.6%)
11–13	353(92.2%)	30(7.8%)	383(14.8%)
14–15	321(92%)	28(8%)	349(13.5%)
Mortality
Yes	224(78%)	63(22%)	287(11.1%)	0.000
No	2162(94%)	138(6%)	2300(88.9%)	0.000
Mechanical Ventilation
Yes	695(90.5%)	73(9.5%)	768(29.7%)	0.021
No	1691(93%)	128(7%)	1819(70.3%)	0.021
Readmission
Yes	115(89.8%)	13(4.9%)	128(4.9%)	0.190
No	2271(92.4%)	188(7.6%)	2459(95.1%)	0.190
Emergency Surgery
Yes	140(90.9%)	14(9.1%)	154(6%)	0.306
No	2246(92.3%)	187(7.7%)	2433(94%)	0.306
Smoker
Yes	92(93.9%)	6(6.1%)	98(3.8%)	0.349
No	2292(92.2%)	195(7.8%)	2487(96.2%)	0.349
Addiction
Yes	132(90.4%)	14(9.6%)	146(5.6%)	0.241
No	2252(92.3%)	187(7.7%)	2439(94.4%)	0.241
Cardiovascular
Yes	412(91.4%)	39(8.6%)	451(17.4%)	0.250
No	1972(82.7%)	162(7.6%)	2139(82.4%)	0.250
Myocardial Infarction
Yes	42(1.8%)	3(1.5%)	45(1.7%)	0.532
No	2342 (92.2%)	198(7.8%)	2540 (98.3%)	0.532
COPD
Yes	20(95.2%)	1(4.7%)	21(0.8%)	0.505
No	2364(92.2%)	200(7.8%)	2564(99.2%)	0.505
HLP
Yes	170(91.9%)	15(8.1%)	185(7.2%)	0.473
No	2214(92.3%)	186(7.8%)	2400(92.8%)	0.473
Diabetes
Yes	216(90%)	24(10%)	240(9.3%)	0.113
No	2168(92.5%)	177(7.5%)	2345(90.7%)	0.113
CVA
Yes	20(0.8%)	3(1.5%)	23(0.9%)	0.263
No	2364(99.3%)	198(7.7%)	2562(99.1%)	0.263
HTN
Yes	339(91.6%)	31(8.4%)	370(14.3%)	0.263
No	2045(92.3%)	170(7.7%)	2215(85.6%)	0.263
Pre ICULOS (Days)
1	1199(92.4%)	98(7.6%)	1297(50.1%)	0.369
2	1187(92%)	103(8%)	1290(49.9%)	0.369

Abbreviations: GCS, Glasgow Coma Scale; COPD, Chronic obstructive pulmonary disease; CVA, Cerebrovascular Accident; HLP, hypertriglyceridemia.

The linear predictor formula for length of stay based on scoring systems were as follows:

4.757–191× APACHE IV
2.667 + 0.058 × APACHE II
4.101 − 0.008 × SAPS II; and
3.731 − 0.003 × SOFA

With the APACHE IV and APACHE II, the R-squared was 0.005, while it was zero for the SAPS II and SOFA. F-values were 12.197, 1.263, 13.783, and 0.004 for the APACHE II, SAPS II, APACHE IV, and SOFA, in that order. Significant differences were observed for APACHE IV and APACHE II. The RMSE measure for APACHE IV (better) is lower among other models. Using AIC and BIC to compare models, shows that APACHE IV and SOFA indicates lower penalty terms hence a better model. The AUC was 0.50 (95%) for SAPS II, and SOFA, 0.54 (0.95%) for APACHE IV, and 0.75 (0.95%) for APACHE II (See Fig. 5). The ICC for the SAPS II, SOFA, APACHE IV, and APACHE II was − 0.019, -0.002, -0.042, and 0.088, respectively. The Pearson’s correlation coefficient for the SAPS II, APACHE IV, APACHE II, and SOFA were, in order, -0.37, -0.07, 0.89, and − 0.37 (See Fig. 4). The details are shown in Tables 3 and 4.

Table 3

Pearson’s correlation coefficient between CSICU length of stay and other variables.
	Pearson Correlation	P-value		Pearson Correlation	P-value
Age	0.056	0.002	Sex	0.043	0.014
Temperature	-0.015	0.229	Mortality	-0.206	0.000
MAP	0.024	0.107	Mechanical Ventilation	-0.038	0.027
Heart Rate	0.029	0.068	Readmission	0.011	0.287
Respiratory Rate	0.037	0.031	Emergency surgery	-0.002	0.460
Potassium	0.006	0.375	Smoker	-0.042	0.016
Platelet	-0.005	0.403	Addiction	-0.021	0.144
FIO2	-0.040	0.022	Previous AF	0.075	0.000
PO2	0.010	0.298	Cardiovascular	-0.006	0.388
PCO2	-0.006	0.382	Myocardial Infarction	-0.013	0.254
PH	0.004	0.410	HTN	-0.014	0.234
Sodium	-0.045	0.011	COPD	0.007	0.366
Urine Output	0.012	0.277	HLP	-0.005	0.403
Serum creatinine	0.086	0.000	Peptic ulcer	-0.004	0.425
Urea	0.001	0.479	CVA	0.009	0.331
Blood sugar	-0.019	0.161	GCS	0.018	0.183
Albumin	0.001	0.480	Diabetes	-0.005	0.409
Bilirubin	0.000	0.493	APACHE IV	-0.073	0.347
Hematocrit	0.030	0.065	APACHE II	0.891	0.000
White Blood Cell	0.033	0.044	SAPS II	-0.372	0.000
			SOFA	-0.372	0.000

Abbreviations: PaO2, Partial pressure of arterial oxygen; FiO2, Fraction of inspired oxygen; PCO2, Partial pressure of carbon dioxide; HCO3, Bicarbonate; MAP, Mean arterial pressure; GCS, Glasgow coma scale; APACHE, Acute Physiologic and Chronic Health Evaluation (version II and IV); SAPS, Simplified Acute Physiology Score (version II); LOS, Length of Stay; GCS, Glasgow Coma Scale; COPD, Chronic obstructive pulmonary disease; CVA, Cerebrovascular Accident; HLP, hypertriglyceridemia.

Table 4

Performance measures of the APACHE II, APACHE IV, SAPS II and SOFA models to predict cardiovascular ICU length of stay and Pearson’s correlation coefficient of the APACHE II, APACHE IV, SAPS II and SOFA models with ICU length of stay.
Models	APACHE IV	APACHE II	SAPS II	SOFA
Overall accuracy
R2	0.005	0.005	0.000	0.000
Adjusted R2	0.005	0.004	0.000	0.000
F-statistic	13.783	12.197	1.263	0.004
P-value	0.000	0.000	0.261	0.947
RMSE	5.878	5.170	5.181	5.182
AIC	15823.04	15936.94	15972.35	15933.97
BIC	16017.34	16142.11	16476.49	16021.91
ICC	-0.042	0.088	-0.019	-0.002
Pearson’s correlation
	-0.073	0.891	-0.372	-0.372

Abbreviations: R², R-squared;RMSE, Root Mean Square Error; AIC, Akaike information criterion; BIC, Bayesian Information Criterion; AUC, Area Under the receiver operating characteristic Curve; CI, Confidence Interval; APACHE, Acute Physiologic and Chronic Health Evaluation (version II and IV); SAPS, Simplified Acute Physiology Score (version II); SOFA, sepsis-related organ failure assessment;

Graphical evaluation shows that in comparison with the other three models, the APACHE IV model has higher calibration over the whole range of predicted probabilities. (See Fig. 6, Table 5).

Table 5

Discrimination and calibration measures of the APACHE II, APACHE IV, SAPS II and SOFA models to predict cardiovascular ICU length of stay
Models	APACHE IV	APACHE II	SAPS II	SOFA
Discrimination
AUC (CI = 0.95%)	0.54	0.75	0.50	0.50
Calibration
Residual standard error	0.005	0.793	0.002	0.138
Shapiro-Wilk normality test	0.94489	0.726	0.375	0.096
P-value	< 2.2e-16	< 2.2e-16	< 2.2e-16	< 2.2e-16
Intercept	5.780	2.869	4.030	4.004
Slope	-0.027	0.044	-0.009	-0.001

Main finding

Analysis of Cardiovascular surgery patients’ outcomes are becoming increasingly important as statistically trustworthy data on risk and prognosis for particular patients, treatments, and procedures is required by organizations, patients, and service providers. (17). On the other hand, in any ICU setting, acute overcrowding and a shortage of resources, particularly in staff and medical supplies, tend to be problems. The issue is particularly noticeable in countries that are developing. The most frequently used risk-stratified scoring systems were assessed in this study for their predictive abilities (SAPS II, SOFA, APACHE II, and APACHE IV) in the ICU to predict length of stay.

Comparing measures between models revealed that APACHE IV has better fit for data comparing with SOFA, SAPS II, and APACHE II. APACHE II indicated that 79% of the variation in the output can be explained by the input variable and has best goodness of fit between models. This model is truly linear and the regression predictions perfectly fit the data. Also, the APACHE II has better prediction the target value (accuracy) than the other three models and more reliability between groups. Moreover, the APACHE II has the strongest correlation between selected models. The APACHE II prediction model outperformed the other three models by a significant amount, according to quantitative discrimination metrics. The dotted line of the angle of 45 degrees, which represents suitable calibration for APACHE IV and APACHE II, indicates that the observed and predicted probabilities are similar. Two other models had poor calibration (SOFA and SAPS II). These results are due to the fact that these two models (APACHE II and APACHE IV) are basically designed for surgery, they provided better results.

Patients' ICU stays were influenced by a number of variables, such as administrative, structural, and patient-related aspects. Longer ICU stays have been observed to be associated with patient factors like underlying disorders, responsiveness to pharmacological treatment, desire for prolonged hospitalization, and development of complications. Variations in the type of ICU physician or specialists, the nurse-to-patient ratio, whether the ICU is open or closed, and the availability of intermediate care or ward beds are examples of structural aspects that could impact an ICU stay. Different ICU rules, the application of standard care procedures including sedation and analgesia, and weaning protocols are examples of managerial factors. Furthermore, the model's efficacy degrades over time due to a number of factors, including advancements in medical science, longer life expectancies, and newly discovered diseases.

Consequently, before using the score models for the ICU benchmark, external validation should be carried out. Customizing the models may help to improve the APACHE II's moderate capacity to predict ICU LOS and the other three models' poor performance. Recalculating the original variables' coefficients or including new variables, including hospital discharge guidelines and patient issues during ICU stays, allowed for customization. Additionally, the severity scoring system's effectiveness can be enhanced by the application of machine learning techniques. According to earlier research, machine learning algorithms perform well in predicting both the length of an ICU stay and its duration.

The predictive power of the SOFA, SAPS II, APACHE II, and APACHE IV models for the last several decades of ICU length of stay (LOS) has been evaluated in recent publications.

A study published in 2023 by Zangmo showed that sepsis patients' ICU length of stay was not well predicted by the APACHE IV, SOFA, and SAPS II models. (AUC = 0.627, 0.459, and 0.450) (18). In 2022, A study by Widyastuti revealed In predicting PLOS, the APACHE IV score impaired discrimination and calibration and prone to overestimating forecasting (AUC = 0.68) (19). When predicting the length of stay for ICU patients, the APACHE IV score performed better than the SAPS III, SOFA, and MPM II scores since it was considerably greater among non-survivors (Pearson’s correlation coefficient = 0.56) in Elmahdy et al study in 2022 (20). In 2018, El-Naggar noticed that none of the three models' scores - SOFA, SAPS II, and APACHE IV - could accurately predict how long a patient will stay in the intensive care unit (R² = 0.49, 0.48, 0.46) (21). The accuracy of the APACHE IV scores in predicting the length of stay is low (R² = 0.001, Pearson’s correlation coefficient = 0.26), particularly if sepsis is severe and trauma cases in a study by Chattopadhyay in 2015 (22).

generally, in the review of previous studies, no study has provided significant results in predicting the length of stay with our mentioned models, and in a few studies APACHE IV scores have had moderate results.

Limitations and Strengths

The vast majority of patients who were enrolled in the retrospective study during a nearly nine-year period is its main strength. Also, for the first time in the world, four models were studied for prediction CSICU LOS. We investigated four well-known models' internal performance in terms of calibration, discrimination, and overall accuracy. After that, we statistically tested and compared each of the four models' performances.

The present investigation is subject to certain limitations. Firstly, because it is a single-center study, there may be limited generalizability of the findings. However, we have been able to incorporate a wide range of acute diseases equivalent to most CSICU in Iran because we are the largest referral hospital in the eastern half of the country. Secondly, we possibly missed some patient characteristics which influence ICU LOS after CABG surgery and could indicate potential covariates for predicting ICU LOS. Thirdly, our study excluded young patients (< 18 years old). As a result, extending the findings to other CSICUs was not possible. Finally, the retrospective nature of this investigation posed limitations.

The APACHE IV and APACHE II models better results than the other two models. Additionally, compared to other methods for predicting duration of stay in our sample, APACHE IV showed moderate levels of calibration, discriminations, and fit for data. Although, APACHE II had better prediction the target value (accuracy) than the other three models. None of these four models meet our particular demands for models to plan resource allocation or benchmarking purposes, nor do they meet our requirements for prediction models for ICU LOS. External validation, customizing the models, and using machine learning techniques could be helpful to predict ICU LOS via these models.

Acknowledgements

This study is the result of a research project approved by Mashhad University of Medical Sciences. The authors would like to appreciate the cooperation of all the patients in conducting this study.

Authors’ contribution

R.M prepared the manuscript, gathered the dataset, developed, validated and analyzed the dataset. S.A as clinical consultant conducted medical issues. S.E. and K.B. assisted in designing the study and helped in the idea of the study. All authors have read and approved the content of the manuscript.

Funding

The present study has no funding.

Availability of data and materials

The data that support the findings of this study will be available by corresponding author upon request.

Ethical approval and consent to participate

The Institutional Review Board of Mashhad University of Medical Sciences approved this study. The study’s retrospective design ensured the meticulous examination of past cases while safeguarding the confidentiality and anonymity of the patients' data. We declare that all techniques were developed in accordance with the regulations and rules. Written informed consent was obtained from all participants.

Consent for publication

This paper didn’t include any individual person’s data. Rights and permissions Not applicable.

Competing interests

The authors declare no competing interests.

Chowdhury D, Duggal AK. Intensive care unit models: do you want them to be open or closed? A critical review. Neurology India. 2017;65(1):39.
Smith SE, Bronwyn Harris. "What is an ICU" wiseGEEK. Sparks, Nevada: Conjecture Corporation.2013-03-24 [
Halpern NA, Pastores SM. Critical care medicine beds, use, occupancy and costs in the United States: a methodological review. Critical care medicine. 2015;43(11):2452.
Halpern NA, Pastores SM, Greenstein RJ. Critical care medicine in the United States 1985–2000: an analysis of bed numbers, use, and costs. Critical care medicine. 2004;32(6):1254-9.
Berenholtz SM, Dorman T, Ngo K, Pronovost PJ. Qualitative review of intensive care unit quality indicators. Journal of critical care. 2002;17(1):1-12.
Flaatten H. The present use of quality indicators in the intensive care unit. Acta anaesthesiologica scandinavica. 2012;56(9):1078-83.
Kahn JM, Rubenfeld GD, Rohrbach J, Fuchs BD. Cost savings attributable to reductions in intensive care unit length of stay for mechanically ventilated patients. Medical care. 2008:1226-33.
Gao H, McDonnell A, Harrison DA, Moore T, Adam S, Daly K, et al. Systematic review and evaluation of physiological track and trigger warning systems for identifying at-risk patients on the ward. Intensive care medicine. 2007;33(4):667-79.
Marik PE, Hedman L. What’s in a day? Determining intensive care unit length of stay. Critical care medicine. 2000;28(6):2090-3.
Rapoport J, Teres D, Zhao Y, Lemeshow S. Length of stay data as a guide to hospital economic performance for ICU patients. Medical care. 2003:386-97.
Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: a severity of disease classification system. Critical care medicine. 1985;13(10):818-29.
Zimmerman JE, Kramer AA, McNair DS, Malila FM. Acute Physiology and Chronic Health Evaluation (APACHE) IV: hospital mortality assessment for today's critically ill patients. Crit Care Med. 2006;34(5):1297-310.
Vincent J-L, Moreno R, Takala J, Willatts S, De Mendonça A, Bruining H, et al. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. Springer-Verlag; 1996.
Le Gall J-R, Lemeshow S, Saulnier F. A new simplified acute physiology score (SAPS II) based on a European/North American multicenter study. Jama. 1993;270(24):2957-63.
Vittinghoff E, Glidden DV, Shiboski SC, McCulloch CE. Regression methods in biostatistics: linear, logistic, survival, and repeated measures models. 2006.
Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, et al. Assessing the performance of prediction models: a framework for some traditional and novel measures. Epidemiology (Cambridge, Mass). 2010;21(1):128.
Iezzoni LI. The risks of risk adjustment. Jama. 1997;278(19):1600-7.
Zangmo K, Khwannimit B. Validating the APACHE IV score in predicting length of stay in the intensive care unit among patients with sepsis. Scientific Reports. 2023;13(1):5899.
Widyastuti Y, Zaki WA, Widodo U, Jufan AY, Pratomo BY. Predictive accuracy of the APACHE IV scores on mortality and prolonged stay in the intensive care unit of Dr Sardjito Hospital. Med J Malaysia. 2022;77:53.
Mohamed Elmahdy N, Elsenousy T, Abdallah Abdelatief D, Mohamed Maarouf D. Prognostic Scoring Systems as a Tool to Predict the Clinical Outcomes for Patient with Critical Condition. Egyptian Journal of Health Care. 2022;13(4):1385-402.
El-Naggar TAE-H, Raafat RH, Mohamed SA. Validity of three scoring systems in assessing the severity and outcome in Al-Abbassia Chest Hospital Respiratory Intensive Care Unit patients. Egyptian Journal of Bronchology. 2018;12:208-17.
Chattopadhyay A, Chatterjee S. Predicting ICU length of stay using APACHE-IV in persons with severe sepsis–a pilot study. Journal of Epidemiological Research. 2015;2(1):1-8.

No competing interests reported.

Predicting length of stay in intensive care units for Cardiovascular Surgery patients using APACHE II, APACHE IV, SAPS II and SOFA Scores

Status:

Version 1

Abstract

Background and objective:

Method

Results

Conclusion

Figures

Introduction

Methods

Results

Discussion

Conclusion

Declarations

References

Additional Declarations

Status:

Version 1