3.1 OS of gastric cancer patients between PNI-positive and PNI-negative patients
Figure 1 shows the flow chart of this study. A total of 2,311 gastric cancer patients who attended Xijing Hospital in 2017–2018 were investigated to determine the relationship between PNI and patients' OS using the Kaplan-Meier method (Fig. 2). The results indicated that OS of PNI-positive patients was significantly lower than that of PNI-negative patients, and PNI was identified as an independent risk factor affecting patients' prognosis.
3.2 Clinical data
The study utilized medical records from January 2016 to December 2020 as primary data. Outliers were removed using the Isolated Forest method in sklearn, an algorithm that uses a binary tree to split the data and randomly selects samples and features for split point selection. After processing, 2080 cases remained, of which 413 had negative PNI values and 1667 had positive values. The data was oversampling using the SMOTE algorithm and normalized through linear transformation to map the values to a range of 0 to 1.
Besides, data asymmetry was addressed using Skewness (Supplementary Table, 1). Its absolute value reflects the degree of asymmetry in the distribution. Higher absolute values indicate greater Skewness.
Subsequently, 2080 patients were divided into a training group (n = 1664) and a validation group (n = 416). There was no statistically significant difference in the incidence of PNI between the two groups. Additionally, there were no statistically significant differences in clinical characteristics, including gender, age, Alpha-Fetal Protein (AFP), carcinoembryonic antigen (CEA), carbohydrate antigen 19 − 9 (CA199), and other laboratory markers (Supplementary Table 2 ).
Finally, Pearson correlation coefficient (Pearson correlation coefficient) was calculated to study the correlation between the variables (Supplementary Figure, 2). The coefficient measures the degree of linear correlation between the variables, with the larger the value, the stronger the correlation.
3.3 screening machine learning models and model training and tuning
ML algorithms offer the ability to learn features from data and identify complex patterns that may be overlooked by traditional models [18, 19]. To select the best model, we compare the following 11 ML models: LR (Logistic Regression); KNN (K Neighbors Classifier); RF (Random Forest Classifier); LDA (Linear Discriminant Analysis); MLP (MLP Classifier); SVM (SVC); DT (Decision Tree Classifier); X-Tree (Extra Trees Classifier); GB (Gradient Boosting Classifier); Bagging (Bagging Classifier); AdaBoost (Adaptive Boosting). Classification task training was performed, and balanced F Score as well as AUC curves were used for comparison in order to select the best performing model.
The results show that all individual models showed the best predictive performance in the internal validation set, with AUC around 0.9 (Figure, 3). And the X-tree model showed superior performance in both ten-fold cross-validation and balanced F Score score, (Supplementary Table, 3). So finally chose the X-Tree algorithm for classification prediction.
3.4 Feature importance and model performance
To optimize the model's performance while reducing the use of clinical indicators,The minimum Akaike’s information criterion (AIC) was used to select the optimal model parameters for assessing the risk of PNI and a total of ten predictors including Infiltration depth, Tumor volume, Blood type, Hematocrit, HB, Sodium, PLT, pathological type, RBC, TC. Based on Shapely additive interpretation approach, which provides insight into the importance of each variable in the interpretation of the data. the above clinical features were selected for use in building the final model.
Further, the model was fine-tuned using GridsearchCV and the optimal Hyperparameter combination was verified by 10-fold cross-validation. The results indicate that the model with 10 features yields good prediction results (Fig. 4a).
Besides, The test set's AUC score, precision, and recall were 0.97, 0.93, and 0.84, respectively. To evaluate our prediction model, we used an intuitive and concise confusion matrix (Fig. 4b), MCC, and DOR. The results showed that the MCC score was 0.78 and the DOR score was 75, indicating that our model has good predictive performance.
Shapely Additive Explanation (SHAP) values are based on a game-theoretic interpretation approach that treats all features as "contributors" and provides both global and local interpretability[20]. Global interpretability shows the positive or negative contribution of each predicted value to the target variable, while local interpretability enhances transparency by providing each observation with its own set of SHAP values[21].
In order to identify the main features that contributed to the prediction, this study measured the positive or negative effect of each feature on the model's prediction of PNI using SHAP values.(Fig. 5A, 5B), To facilitate the clinical application of the model, we further developed a concise and accessible nomogram based on 10 crucial features that can be used to accurately assess the PNI status of GC patients (Fig. 6).