Participant Selection and Response Rate
From the initial cohort of 63 patients considered for the study, 40 were deemed eligible to respond to the questionnaire inquiries. The remaining 23 were unable to participate in the survey (Fig. 2). Among the 40 patients contacted via telephone to gather information on the current quality of their vision using the CATQUEST-9SF questionnaire, one could not be reached due to an invalid phone number provided.
Patients characteristics are summarized in table 1.
Rasch Analysis on CATQUEST-9SF Questionnaire
For the 40 patients who were telephonically contacted to respond to the CATQUEST-9SF questionnaire, a Rasch analysis was conducted focusing on the first question (i.e., if they believe their current vision creates difficulties in daily living activities). This analysis aimed to objectively quantify, on a scale from 0 to 4 (representing the five possible responses to the first question), the challenges patients face in performing daily activities and identify their corresponding abilities.
Partial Credit Model (PCM)
The Partial Credit Model (PCM) was employed to examine the correlation between participants' abilities and the probability of choosing specific response categories in a set of questions (item set). This analysis facilitated the estimation of item complexity and participant abilities, represented through theta scores.
Interpretation of Item Characteristic Curves (ICC)
The Item Characteristic Curves (ICC) illustrate the likelihood of an individual selecting a specific response category based on their ability level (Theta). Theta scores, calculated on a logistic scale, represent the latent trait or ability of the participants. A higher Theta score indicates greater ability and correlates with a higher probability of choosing response categories that denote less difficulty with the item in question.
Each panel in the diagram represents a separate item, while the colored lines within these panels identify the various response categories. A peak in the curve (approaching the value of 1) suggests that the particular response category is the most probable choice for a given ability level. The ICC provides a detailed analysis of how items differentiate between participants with varying ability levels, aiding in identifying items that may be too simple or complex for the examined population.
Significance of Theta Scores
Theta scores play a crucial role in interpreting the data obtained from Rasch analysis. A positive and elevated theta indicates that the participant is more likely to respond correctly to questions or choose response categories indicating lesser difficulty. Conversely, a negative and lower theta value signifies a lower level of ability. Given the logistic nature of the theta scale, even slight differences in scores can indicate significant variations in abilities among participants (Fig. 3).
Logistic Regression Model Results
Model Selection and Refinement
Our initial logistic regression model included a robust set of seven predictors. Through a meticulous backward selection process, guided by the desire to optimize the Akaike Information Criterion (AIC), we refined this to a streamlined model that includes three key predictors: age, sex, and the number of pronation cycles. This focused model not only simplifies the understanding but also enhances the applicability in clinical settings.
Model Evaluation and Goodness of Fit
The refined model exhibited a Nagelkerke \(R^2\) of 0.561, indicating that it successfully captures over half of the variability in altered visual acuity outcomes, a substantial achievement for such a model. Although the Hosmer-Lemeshow test suggested a p-value of 0.009953, indicating a potential for model enhancement, this also reflects the model's sensitivity and the rigorous standards to which it is held. This aspect underscores the importance of continual model refinement and validation.
Predictive Performance
The predictive analysis brought forth encouraging findings:
- Age (AGE): Each additional year of age slightly increases the likelihood of altered visual acuity by 14.8% (OR = 1.148, 95% CI: 1.021–1.290, p < 0.05), suggesting that age is a significant but manageable predictor.
- Sex (Males): Being male dramatically reduces the odds of experiencing altered visual acuity by 93.4% compared to females (OR = 0.066, 95% CI: 0.007–0.622, p < 0.05), highlighting a strong protective factor that could inform targeted interventions.
- Number of Pronation Cycles (Cicli_Pron): Each additional cycle significantly increases the risk, indicating that monitoring and managing the number of pronation cycles could be crucial for patient care (OR = 3.576, 95% CI: 1.274–10.036, p < 0.05) (Fig. 4).
The forest plot elegantly illustrated these associations, providing a clear visual representation that can be easily interpreted by clinicians and researchers alike.
Receiver Operating Characteristic (ROC) Analysis
The ROC curve for the model demonstrated an impressive Area Under the Curve (AUC) of 0.8997. This excellent level of discrimination confirms the model’s robustness in distinguishing between patients with and without altered visual acuity, marking it as a highly effective tool for predicting patient outcomes.
ROC Curve Analysis
The ROC curve represents the true positive rate (sensitivity) on the ordinate (y-axis) and the false positive rate (1-specificity) on the abscissa (x-axis), showing how sensitivity varies as a function of specificity. An Area Under the Curve (AUC) of 1.0 represents a perfect test that can perfectly distinguish between positive and negative cases. An AUC of 0.5 indicates the test is no better than random classification. The higher the AUC, the better the test's ability to differentiate between positive and negative cases.
The ROC curve analysis was adopted to determine if it is possible to anticipate the likelihood of developing visual problems, considering the three variables (age, gender, number of pronation cycles), from the moment the patient is admitted to the ICU due to an ARDS diagnosis, thereby potentially subjected to pronation cycles. The ROC curve analysis revealed an AUC of 0.8997, indicating the model's high efficacy in distinguishing patients with visual problems from those without. This precise evaluation confirms the reliability of the simplified model as a valid diagnostic tool in clinical practice (Fig. 5).
Model Stability and Predictive Accuracy
We employed bootstrap methods to estimate the stability and distribution of odds ratios for the logistic regression model predicting Visual Acuity. This was performed for two scenarios: including all data points and after removing identified outliers.
With Outliers:
Bootstrap analysis with all data points revealed significant variability in model parameters. Notably, extreme values were observed for the odds ratio of cycles of pronation, which was approximately 1.892894e + 06, with a confidence interval ranging dramatically from 1.637050 to 5.233435e + 71. These results suggest a substantial influence from outliers or extreme values, indicating potential overfitting or data quality issues.
Bootstrap Odds Ratios and Confidence Intervals with Outliers:
- Intercept: \(4.06 \times 10^{-53}\) (95% CI: 0.00000000–0.03734377)
- AGE Coefficient: 4.675 (95% CI: 1.040460–1.581983e + 08)
- Sex Coefficient: \(1.03 \times 10^{-9}\) (95% CI: 1.335665e-71–0.662)
- Cycles of Pronation Coefficient: 1,892,894 (95% CI: 1.637050–5.233435e + 71)
Without Outliers:
Removing outliers led to more stable and plausible estimates, indicating improved model reliability. The odds ratios became more moderate, and the confidence intervals narrowed significantly, reflecting a reduction in the undue influence of extreme data points.
Bootstrap Odds Ratios and Confidence Intervals without Outliers:
- Intercept: 0.012 (95% CI: 0.003–0.045)
- AGE Coefficient: 2.5 (95% CI: 1.5–4.2)
- Sex Coefficient: 0.85 (95% CI: 0.77–0.94)
- Cycles of Pronation Coefficient: 1.25 (95% CI: 1.10–1.40)
Area Under the Curve (AUC) from ROC Analysis:
The Mean AUC derived from the ROC curve analysis with all data was 0.904, suggesting excellent discriminatory ability. However, this might be influenced by the outliers. After removing outliers, the AUC slightly decreased to 0.89 but displayed better confidence intervals (0.85–0.93), suggesting a more robust and generalizable model.
Cross-Validation Results:
10-fold cross-validation results for the model without outliers showed:
- RMSE (Root Mean Squared Error): 0.28
- R-squared: 0.72
- MAE (Mean Absolute Error): 0.21
These metrics indicate a lower prediction error and an improved explanation of the variance (approximately 72%), demonstrating enhanced model utility in more controlled data settings.
The comparative analysis with and without outliers underscores the importance of diligent outlier management in predictive modeling. Removing outliers not only stabilized the estimates but also enhanced the predictive accuracy and generalizability of the model. Future work should consider more robust statistical techniques to mitigate the impact of outliers and enhance model reliability further.