A distinctive feature of our F + L + model is its high discriminative power with an AUROC that exceeded 0.97 in both the cross-validation and hold-out settings. Previous prediction models for determining the clinical deterioration of COVID-19 patients have reported accuracies between 0.77 and 0.91 [2–5]. Additionally, these models require specific diagnostic data, including laboratory data, peripheral oxygen saturation, or radiographic findings, to maintain their accuracies. Moreover, to what extent these models’ performance abilities are maintained during the partial absence of data has not been studied. For our F + L + model to be effectively implemented in practice, we confirmed that our reduced models maintained an adequate discriminative power even in the partial absences of data. The advantages of our reduced models include not only their generalizability to unseen data, but also their applicability within scenarios wherein there is limited clinical data. As timely triage is important for COVID-19 patients, our reduced models can be utilized during the early triage stages at a patient’s arrival while waiting for the F + L + model to more accurately stratify their disease severity risk. Given the acute exacerbation of pneumonia in COVID-19 patients, our model can also be used to re-evaluate hospitalized patients in the short-term, so that those whose clinical manifestations are likely to exacerbate can be early identified [19].
A noteworthy feature of our model is its ability to discriminate between patient-specific contributing factors for disease exacerbation and their individual contributions using SHAP values. Current COVID-19 treatment guidelines provide recommendations based on the average-risk patient under limited available insights into their disease stage [10]. These recommendations provide a one-size-fits-all approach to all patients, which is problematic for those with more complex or atypical disease presentations. Our model obviates the need for arbitrary patient risk groupings and is, therefore, useful in maximizing their survival odds based on individual risk stratification. Furthermore, our models can be integrated into electronic medical record systems, which utilize coding algorithms, as a notification system that helps in the early identification of disease exacerbation risk factors.
The validity of our model is supported by the high consistency between the results of its interpretation using SHAP, and previously reported prognosticators of COVID-19 severity [20–25]. We noted that old age, followed by lymphopenia and thrombocytopenia, exhibit the highest Shapley values for disease exacerbation. We presumed older age interacts with relevant features in older adults, including poor functional performances and increased frailty, which are associated with adverse outcomes and increased mortality among patients with respiratory syndromes [26]. Our findings also add to the literature that lymphopenia plays an important role in COVID-19 exacerbation [22–25]. Lymphopenia is exerted by the lowering of lymphocytes due to injured alveolar epithelial cells and is commonly observed in COVID-19 patients [27]. Consistent with previous studies, thrombocytopenia was also found to be associated with adverse COVID-19 outcomes [23, 28]. It has been suggested that a reduction or morphological alternation of the pulmonary capillary bed exerts pathological platelet defragmentation because the lung is a platelet release site with mature megakaryocytes [29]. Our prediction model supports the notion that early identification of COVID-19 exacerbation, before a hematological crisis occurs, is necessary for ensuring a better prognosis.
There is no existing study that examines COVID-19 severity prediction models that provides an explicit solution for the delivery of optimal triage using threshold modification that accounts for limited resource availability. We conducted a DES on our F-L + model to examine its discrimination thresholds that are usable in an adaptive manner across various patient influx scenarios and the related ICU availability. Our simulations reveal that applying the optimal thresholds will minimize the mortality rate of each patient influx scenario. Our hypothesis is supported by the significant differences found in mortality rates between the J-index and our optimized thresholds when applied to the expected influx volumes. This observation implies the potential or our model to substantially reduce COVID-19 mortality rates through its appropriate adjusting of triage thresholds.
A limitation of our study was its incorporation of a single, national cohort of Asian ethnicity, which impacts our findings’ generalizability. An external validation using a more multi-ethnic population is thus needed to determine if a similar discrimination performance occurs among other ethnic groups. However, to ensure our model’s robustness, we implemented 10-fold cross-validation with additional confirmation using the hold-out cohort. Another limitation was that the triage threshold was evaluated using a simulation. Simulations do not yield concrete answers, nor are they able to assess all kinds of potential situations [30].