Literature search results
A total of 371 literatures were searched, including 260 from PubMed, 65 from EMBASE, and 46 from the Cochrane Library databases, 3 from other sources (such as abstracts from conferences). There were 279 articles left after excluding the reduplicated articles. After reviewing the abstracts, some articles were excluded, including systematic reviews, case reports, and coverage not matched, and 17 articles remained, which we read in full. A total of 8 studies were excluded due to the lack of data required for analysis, whereas 9 articles met the full inclusion criteria finally. The detailed PRISMA flowchart is depicted in Figure 1.
Characteristics of the enrolled trials
The detailed baseline characteristics are demonstrated in Tables 1 and 2. All the included trials were published between 2016 and 2021. Four studies were conducted in Europe[12-15], three in Asia[10, 11, 18], and the remaining two in USA[9] and Africa[19]. Four studies were multiple-center [12, 13, 15, 19] and the other five were single-center studies[9-11, 14, 18]. For the types of diseases requiring HFNC use, three studies were ARF[10, 14, 15], four were COVID-19[9, 11, 18, 19], and two were pneumonia[12, 13]. The total number of patients was 1933, of which 745 were failure cases. Among these nine studies, five were prospective observational cohort studies[9, 10, 12, 13, 19], three were conducted retrospectively[11, 14, 18], and only one was RCT[15]. The acquisition time of ROX index after applying HFNC varied in different studies; six studies[9-12, 15, 19] reported 6 h as the acquisition time, one study[13] chose 12 h, another study chose 4 h[18], and the remaining study measured ROX index before each attempt to separate from HFNC[14]. Five studies[9, 10, 12, 13, 15, 18] defined HFNC failure as the subsequent need for invasive MV, while Ming Hu et al.[11] defined HFNC failure as the need for NIV or IMV and/or death, 1 study[19] defined HFNC failure as the need for MV or death, another study[14] defined HFNC failure as requiring HFNC resumption, NIV initiation, intubation, or death. The cut-off values of ROX index ranged from 2.7 to 9.2, and while three studies[12, 13, 15] reported 4.88 as the best cut-off value, other researchers found the values of 3.66[9], 5.55[11], 5.8[10], 9.2[14], 2.7[19], 5.31[18] in each study, and the sensitivity and specificity of ROX index to predict HFNC failure were from 50% to 84%.
Results of methodological quality evaluation
After careful evaluation of the methodological quality of all nine enrolled studies, we found that seven studies[9, 10, 12, 13, 15, 18, 19] had a low risk of bias in patient selection, three trials[12, 13, 15] had a low risk of bias in index test, all the included studies exhibited low risk of bias in the item of reference standard, and eight studies[9-13, 15, 18, 19] had low risk in flow and timing. To sum up, the included seven studies were of good quality (Table 3).
The meta-analysis of the ability of ROX index to predict HFNC failure
Nine trials with 1933 patients assessed the ability of ROX index in predicting HFNC outcome, and there was statistically significant heterogeneity in the sensitivity (I2 = 84.97%), specificity (I2 = 86.95%), PLR (I2 = 33.93%), NLR (I2 = 74.64%), diagnostic score (I2 = 41.76%) and DOR (I2 = 99.91%). Therefore, we used a random-effect model to conduct this meta-analysis. Pooling all the enrolled studies, the sensitivity, specificity, PLR, NLR, diagnostic score, and DOR of ROX index in predicting the HFNC failure are depicted in Figure 2. The pooled estimates of ROX index in predicting HFNC failure were as follows: sensitivity, 0.67 (95% CI 0.57-0.76); specificity, 0.72 (95% CI 0.65-0.78); PLR, 2.4 (95% CI 2.0-2.8); NLR, 0.46 (95% CI 0.37-0.58); diagnostic score, 1.65(95% CI 1.37-1.93); and DOR, 5.0 (95% CI 4.0-7.0). We also conducted the summary receiver operating characteristic (SROC) plot to evaluate the predicting accuracy of ROX index (Figure 3), and the area under the curve (AUC) was 0.75 (95% CI 0.71-0.79), implying that ROX index could predict HFNC failure.
Subgroup analyses for the predicting value of ROX stratified by different conditions
To determine heterogeneity between studies, we conducted subgroup analyses based on the types of diseases correlated with HFNC use, the areas of enrolled studies, and the acquisition time of ROX index (Table 4). For the types of diseases correlated with HFNC use, there were four trials reported data on COVID-19, with pooled sensitivity and specificity of 0.71 (0.56-0.82) and 0.73 (0.63-0.81), respectively, while lower sensitivity (0.63, 0.48-0.75) and specificity (0.71, 0.60-0.80) were found among other diseases. In the subgroup of areas of enrolled studies, the sensitivity and specificity in predicting HFNC failure were 0.64(0.46-0.79), 0.70 (0.57-0.80) in Europe, and 0.69 (0.56-0.79), 0.74 (0.65-0.81) in the other areas. The pooled results of six enrolled studies indicated that summary sensitivity and specificity were 0.67 (0.54-0.78) and 0.70 (0.60-0.78) when acquisition time was 6 h after receiving HFNC. However, good performance of ROX index was found in settings with other acquisition times, with sensitivity and specificity of 0.73 (0.67-0.79) and 0.74 (0.70-0.78), respectively.
Publication bias analysis
We performed Deeks’ funnel plot to evaluate the potential publication bias, and the funnel plot demonstrated no statistically significant publication bias in this meta‐analysis (P=0.74) (Figure 4).