In the present observational pilot study, the mROX showed slightly better but statistically insignificant predictive accuracy for HFNC failure than the ROX index of respective time points. These findings are in line with Roca O et al.’s analysis, where they also found that the diagnostic accuracies of ROX index with SpO2/FiO2 and PaO2/FiO2 (mROX index) were statistically indifferent at two, six, and 12-hours [12]. Unfortunately, the efficacy of both ROX and mROX was only fair to good, and none of the indexes at baseline or six-hour could achieve a sensitivity and specificity even near 90%. The maximum sensitivity was achieved by mROX at six hours (i.e., 87.55%) at a cut-off value of 4.05, but the specificity was only 63%. While a screening tool does not require high specificity, high sensitivity is desirable to detect most true positive cases. In that context, ROX at six hours and mROX at baseline and six hours appear to have the potential as they all have a sensitivity of > 70%. Nonetheless, for a screening test to be clinically helpful, both sensitivity and specificity are essential, and a criterion based on sensitivity + specificity of 1.5 and above is considered [13]. The only screening test that qualified our pilot cohort's criteria was mROX at six hours. The sensitivity and specificity of the ROX index in our study were similar to the other contemporary studies and meta-analyses [14, 15].
The present study hypothesis was based on the clinicopathological features of COVID-19 pneumonia and ARDS. As respiratory rate and oxygenation are altered in COVID-19 AHRF patients compared to non-COVID-19 AHRF patients, we assumed that the ROX and mROX index may have altered efficacy in predicting HFNC O2 therapy in COVID-19 patients. The failure to find mROX significantly better as a predictor might be explained by the need for FiO2 during HFNC therapy. These patients are mostly moderate-to-severely ill and usually do not require 100% FiO2. At the relatively lower FiO2, SpO2/FiO2, and PaO2/FiO2 correlate well even in COVID-19 patients [16]. As the need for FiO2 increases toward 100% in patients receiving HFNC O2 therapy, the chances of HFNC O2 therapy failure also increase.
In their landmark study, Roca O et al. [4] reported ROX index at 12 hours as having the best prediction accuracy (area under ROC 0.74 [95% confidence interval, 0.64–0.84]) with the best cut-off point for ROX index estimated to be 4.88. Vega et al. [17] validated the utility of the ROX index as a predictor of HFNC failure for COVID-19 patients and reported almost similar results to Roca et al. The 12-hour ROX index was found to be the best predictor of intubation with an AUC of 0.7916 [CI 95% 0.6905–0.8927] and the best threshold was 5.99 [Specificity 96% Sensitivity 62%]. Even the multivariate analysis to determine the predictive factors for HFNC failure by Lun CT et al. [18] found ROX index at 12has one of the significant entities. However, the time-point for best prediction is not similar. Ferrer et al. [19] found the ROX index as a good predictor in their observational study. They reported ROX index at 24has the best predictor of success (AUC 0.826, 95%CI 0.593-1.00) with a cut-off point of 5.35 (Sensitivity 0.91, Specificity 0.79, PPV 0.92, NPP 0.79). On the other hand, Suliman LA et al. [20], enrolling COVID-19 patients, found that ROX on day-1 was a significant predictor of intubation through regression analysis.
In their systematic review and meta-analysis, Prakash et al. [21] assessed the ROX index as a predictor of HFNC failure in COVID-19 patients with AHRF; eight retrospective studies (n = 1301 patients) were considered for analysis. The meta-analysis yielded a summary area under the curve (sAUC) 0.81 (95% CI, 0.77–0.84) with sensitivity of 0.70 (95% CI, 0.59–0.80) and specificity of 0.79 (95% CI, 0.67–0.88) of ROX index for predicting HNFC failure. The positive and negative likelihood ratio was 3.0 (95% CI, 2.2–5.3) and 0.37 (95% CI, 0.28–0.50), respectively, and was strongly associated with good predictive accuracy (diagnostic odds ratio 9, 95% CI, 5–16). Similarly, the systematic review and meta-analysis by Zhou et al. [22] for ROX index as a predictor of HFNC outcome in pneumonia patients with AHRF found good predictive performance for successful HFNC weaning in patients with an area under ROC of 0.81 (95% CI 0.77–0.84), a pooled sensitivity and specificity of 0.71 (95% CI 0.64–0.78), and0.78 (95% CI 0.70–0.84) respectively. The analyses also suggested that the ROX index was a reliable predictor of HFNC success in patients with COVID-19 pneumonia. The area under the curve analysis of our cohort for both ROX and mROX index at admission and six hours were also in the same line of these studies.
Although most of the study using ROX has stressed the HFNC failure, Suliman LA et al. [20] studied the ROX and mROX for predicting intubation of COVID-19 patients. At the time of intubation, the ROX and mROX median (min-max) values were 3.88 (3.33–6.09) and 5 (3.14–5.52), respectively. In contrast to Suliman LA et al.’s [20] study, 11 out of 13 failure patients were initially tried on Bilevel pressure support non-invasive ventilation before intubation and invasive mechanical ventilation. Nevertheless, the entire but one patient ultimately required IVM. The mROX and ROX values at six hours and 12 hours in our failure cohort were within the range of the study Suliman LA et al.’s [20] study, which required intubation, indicating the efficiency of the indexes for not only predicting HFNC failure but also predicting intubation. Rochwerg et al. [23] performed a systematic review and meta-analysis to evaluate the safety and efficacy of HFNC in patients with acute hypoxemic respiratory failure. Nine randomized controlled trials were included (n = 2093 patients) and analyzed. They reported that HFNC usage reduced the need for invasive mechanical ventilation compared to conventional oxygen therapy (RR 0.85[95% CI 0.74–0.99 with low certainty).
Further, the use of HFNC reduced the need for escalation of therapy, although it did not affect ICU and hospital length of stay. Notably, delaying intubation leads to increased mortality. In this context, using any of the ROX or mROX indexes to predict failure and intubation might be an excellent clinical practice despite not having excellent sensitivity and specificity and other limitations [24].
The other intriguing observational finding is the ROX or mROX values trend. Persistent lower values and decreasing trends over the 12 or 24h were the characteristics of the cohort who failed the HFNC. The findings also corroborate with the data presented by Suliman LA et al. [20], where the cohort requiring intubation also showed decreasing trend of the ROX value over the 72-hours.
However, our study is limited because it was an observational study conducted in a single center and a pilot study. The decision to convert artificial respiratory support from HFNC to NIV (BiPAP) or IMV was at the treating physician's discretion, and no written or strict protocol was followed. We intended to identify the better screener at the earliest and therefore stressed primarily within six-hour of the HFNC application. Although the patients were followed until they were either discharged or in heavenly abode, limited data deterred us from analyzing at delayed time points. Sedation is another factor that impacts the tolerance and respiratory rate during any non-invasive artificial respiratory support. It bears importance in the context that the sedation requirements of COVID-19 patients are very high and variable [25]. We did not monitor the sedation levels and drugs used for the same, which might have a minor impact on our results.