A machine learning model for predicting the three-year survival status of patients with hypopharyngeal squamous cell carcinoma using multiple parameters

Abstract Objective This study aimed to establish a model for predicting the three-year survival status of patients with hypopharyngeal squamous cell carcinoma using artificial intelligence algorithms. Method Data from 295 patients with hypopharyngeal squamous cell carcinoma were analysed retrospectively. Training sets comprised 70 per cent of the data and test sets the remaining 30 per cent. A total of 22 clinical parameters were included as training features. In total, 12 different types of machine learning algorithms were used for model construction. Accuracy, sensitivity, specificity, area under the receiver operating characteristic curve and Cohen's kappa co-efficient were used to evaluate model performance. Results The XGBoost algorithm achieved the best model performance. Accuracy, sensitivity, specificity, area under the receiver operating characteristic curve and kappa value of the model were 80.9 per cent, 92.6 per cent, 62.9 per cent, 77.7 per cent and 58.1 per cent, respectively. Conclusion This study successfully identified a machine learning model for predicting three-year survival status for patients with hypopharyngeal squamous cell carcinoma that can offer a new prognostic evaluation method for the clinical treatment of these patients.


Introduction
Hypopharyngeal carcinoma is relatively rare, accounting for approximately 3 per cent of all head and neck tumours and 7 per cent of all upper respiratory tract tumours. 1,2 The site of onset of hypopharyngeal cancer is concealed, and early symptoms are not typical. Most patients are at an advanced stage (stages III-IV) at the time of diagnosis. More than 90 per cent of hypopharyngeal cancers are squamous cell carcinomas (SCCs), and the vast majority of patients are male. 3 These patients usually have a long-term history of smoking and drinking. In terms of treatment, patients with early stage hypopharyngeal SCC (stage I-II, which makes up a small proportion of all patients with hypopharyngeal SCC) can be treated by surgery, whether endoscopic transoral laser microsurgery, open partial hypopharyngectomy or partial laryngectomy, supplemented with radiotherapy or chemotherapy. 4 Comprehensive treatment can be selected for advanced-stage hypopharyngeal SCC, including surgery, pre-operative and post-operative chemotherapy, and/or radiotherapy.
Despite advances in medicine and oncology in treating other types of head and neck cancer, the prognosis of patients with hypopharyngeal SCC is poor, with a 5-year survival rate of 30-35 per cent and 3-year survival rate of 43-57 per cent. 1,5 Without any treatment, usually less than 20 per cent of patients with hypopharyngeal SCC survive for more than 12 months, and only a few survive for more than 2 years. For patients with advanced hypopharyngeal SCC (stages III-IV), the overall survival rate has only improved slightly in the past few years. 6 Some researchers have pointed out that it is not ideal to formulate treatment methods and predict the prognosis of patients using only tumournode-metastasis (TNM) stage. Patients should be comprehensively evaluated using TNM stage in combination with specific variables related to patient survival. 7 In recent years, artificial intelligence (AI), including machine learning and deep learning, has played an important role in the diagnosis, treatment and prognosis of various types of tumours, including head and neck tumours. [8][9][10] The machine learning algorithms most commonly used include 11 K-nearest neighbour, support vector machine, decision tree, random forest, linear discriminant analysis, AdaBoost, XGBoost and CATBoost. Different algorithms exhibit different model performance for different datasets.
This study aimed to establish a multiparameter AI model for predicting the 3-year survival status of patients with hypopharyngeal SCC using machine learning algorithms and 22 parameters. This model may provide a new method for evaluating the prognosis in regard to the clinical treatment of patients with hypopharyngeal SCC.

Data acquisition
This retrospective study was approved by the institutional review board of Beijing Tongren Hospital (approval number: TRECKY2016-025). Clinical data were collected from 295 patients with hypopharyngeal SCC who underwent surgery at our hospital between December 2004 and December 2015. It included basic clinical information and type of treatment at the time of diagnosis. The follow-up date was 31 December 2018.
Smoking history was defined as smoking more than 10 cigarettes per day for more than 10 years, and drinking history was defined as alcohol consumption at least 1 time per week for more than 10 years. Among all the data, 70 per cent were set as training sets and the other 30 per cent were set as test sets. The procedure used for assigning patient data to the training or test sets was to apply the Sklearn machine learning library based on Python programming language. The train_test_split module was assigned, and the random seed was set to 0.

Feature selection
This was a multiparameter AI model constructed study. We approached feature selection for this study from a clinical perspective. All clinical data that might be related to the prognosis of patients with hypopharyngeal SCC were chosen as the input features of the model. These included: the patient's age, sex, smoking history, drinking history, presence of basic disease, TNM stage and clinical stage at the time of diagnosis; whether radiotherapy or chemotherapy was performed before and after the operation; pathological differentiation; and whether transoral laser microsurgery, partial laryngectomy, partial hypopharyngeal resection, total laryngectomy, total hypopharyngeal resection and skin flap repair were performed. Because our hospital routinely performs immunohistochemical analyses of p53 and Ki-67 proteins in the pathological sections of patients with head and neck tumours, the levels of expression of these proteins were also included in the model training. In summary, we included 22 clinical parameters for model training and compared model performance with and without p53 or Ki-67 proteins.

Model establishment
We used the XGBoost algorithm for model establishment, and the fine-tuning details of the parameters for XGBoost are shown in Figure 1 in the supplementary material, available on The Journal of Laryngology & Otology website. The XGBoost classifier is a machine learning method recently defined by Chen et al. and is characterised by combining several weak learning algorithms into a single strong learning algorithm to obtain high-performance results. 12 In addition, other common machine learning algorithms were also established to obtain the best model performance.
The differences between the model results obtained using the XGBoost algorithm and those obtained using other machine learning algorithms were compared and recorded. These algorithms included support vector machine, random forest, logistic regression, K-nearest neighbour, linear discriminant analysis, multinomial Naive Bayes, decision tree, AdaBoost, multilayer perceptron, Light GBM and CATBoost.

Model evaluation and statistical analysis
Common parameters, such as accuracy, specificity, sensitivity, receiver operating characteristic curve and kappa value were used for model evaluation. The area under the receiver operating characteristic curve was calculated. For statistical analysis, SPSS ® statistical software (version 22.0) was used. The difference in the three-year survival status of hypopharyngeal SCC patients with different characteristics was analysed using the chi-squared test.

Demographic data from enrolled patients
Data from 295 patients with hypopharyngeal SCC were included in this study. The age range was 33-86 years (mean and standard deviation, 59.7 ± 10.1 years), and the follow-up time was 6-126 months (median follow-up time was 52 months). A comparison of demographic data and three-year survival status of patients with hypopharyngeal SCC using different clinical characteristics is presented in Table 1.
Among the 22 clinical features, 9 showed no statistically significant correlation with the 3-year survival status of patients with hypopharyngeal SCC. These parameters were sex, lymph node metastasis, distant metastasis, basic disease, smoking history, drinking history, partial laryngectomy, partial hypopharyngeal resection and flap repair. Considering that these characteristics were also important clinical parameters, even though they had no statistically significant effect on the three-year survival status of patients with hypopharyngeal SCC, we still used them as input characteristics for the model.
The other 13 characteristics were significantly associated with the 3-year survival status of patients with hypopharyngeal SCC. Patients with the following conditions were more likely to die after 3 years: age equal to or more than 60 years, T 3-4 disease, pathology poorly differentiated, positive p53 and Ki-67 protein expression, advanced hypopharyngeal SCC (stages III-IV), no radiotherapy or chemotherapy performed before or after surgery, and no history of transoral laser microsurgery, total laryngectomy or total hypopharyngeal resection.

Performance of XGBoost and other algorithms
The performance of each algorithm is listed in Table 2. The model constructed using the XGBoost algorithm had the best accuracy (80.9 per cent), sensitivity (92.6 per cent), specificity (62.9 per cent), area under the receiver operating characteristic curve (77.7 per cent) and kappa value (58.1 per cent). The confusion matrix and receiver operating characteristic curve are shown in Figures 1 and 2, respectively, and the feature importance of the XGBoost model is shown in Figure 3. It should be noted that because the number of patients with hypopharyngeal SCC with distant metastases was too small, its role in the model was not significant, so it is not shown in the feature importance diagram. However, because it is a very important clinical parameter, it was retained as an input feature in the model.

Performance of XGBoost with or without p53 or Ki-67
We also compared the performance of the model established by the XGBoost algorithm with or without p53 or Ki-67 protein expressed in hypopharyngeal SCC tissue to further explore the role of the two proteins in the model (Table 3). We found that if p53 was not included, the accuracy fell to 79.8 per cent and if Ki-67 was not included, the accuracy decreased to 75.3 per cent. If both p53 and Ki-67 were not included, the accuracy fell to 77.5 per cent. Thus, to ensure a good model performance that can predict the three-year survival status of patients with hypopharyngeal SCC, the expression of p53 and Ki-67 should be included as features in the model.

Discussion
The establishment of a 3-year or 5-year survival prediction model for different tumour types can provide a reference for clinicians to predict the prognosis of patients undergoing tumour treatment. Delen et al. used three different machine learning algorithms to establish a prediction model for survival status in breast cancer. 13 Gong et al. also used various machine learning algorithms to establish a model for predicting fiveyear survival status for oesophageal cancer, providing a reference for the prediction of prognosis for that disease. 14 However, an AI model for predicting the survival status of patients with hypopharyngeal SCC has not yet been reported. This is the first report of the construction of an AI model for the prediction of the three-year survival status of patients with hypopharyngeal SCC. The model was constructed based on 22 clinical parameters, using 12 types of machine learning algorithms, and XGBoost was chosen as the optimum algorithm. The performance of the model is relatively satisfactory and  The clinical features in this study could be divided into three main groups. The first was a basic information group, which included age, sex, smoking, alcohol consumption and basic disease at diagnosis. According to the analysis of feature importance for the prediction model, we found that age was one of the most important features, which was in agreement with the findings of Gong et al. for oesophageal cancer. 14 In fact, we found that patients with hypopharyngeal SCC who were younger than 60 years had a lower probability of death 3 years after diagnosis than patients with hypopharyngeal SCC who were older than 60 years. The remaining four features in the first group were of lower importance, particularly sex, which was not ranked in the feature importance diagram (Figure 3).
The second group was a diagnosis-related group and included the TNM and clinical stages, pathological information, and expression of p53 and Ki-67 proteins. The T-stage and clinical stage are well-known factors for predicting survival 1 and were also important features for prediction of survival status in this study (Figure 3). Patients with advanced hypopharyngeal SCC are more likely to have a lower probability of survival than those with early stage disease. 1 Although we did not find any statistically significant difference in the survival status of patients with or without cervical lymph  node metastasis after three years, it can be seen from the importance of characteristics that this parameter remains an important feature for predicting the survival status of patients with hypopharyngeal SCC after three years (ranking sixth; Figure 3). This may be because the importance of the characteristics cannot be completely established using statistically significant differences based on machine learning algorithms. Different machine learning algorithms may have unique   We found that patients with poorly differentiated pathological states, or those with p53 and Ki-67 expression in cancer tissues had a significantly lower probability of survival. Patients with poorly differentiated hypopharyngeal SCC have poor survival outcome and this has been well studied. 1,4 The gene p53 is well studied and plays an important role in various tumours. The positivity rate of p53 in this study was 42.7 per cent, which is in accordance with the results of other studies (34-81 per cent). 15,16 Ki-67 is a tumour proliferation marker that is also upregulated in hypopharyngeal SCC. Ki-67 levels are significantly associated with survival outcome in hypopharyngeal SCC. 17 We also found that if the model was constructed without p53 or Ki-67, its performance would decline; therefore, both p53 and Ki-67 are important features to predict survival status of patients with hypopharyngeal SCC three years after diagnosis. The third group was a treatment-related group. The treatment of hypopharyngeal SCC mainly includes surgery, chemotherapy and radiotherapy. In this study, 10 treatmentrelated parameters were included as the training features. We found that patients who had received pre-operative or postoperative radiotherapy or chemotherapy had a better probability of survival than those who had not. Other studies have confirmed the positive value of both chemotherapy and radiotherapy in the prognosis of patients with hypopharyngeal SCC. 4,18,19 We also found that patients who had a history that included transoral laser microsurgery, total laryngectomy or total hypopharyngeal resection surgery had better survival outcomes than those who did not. There were no significant differences in survival between patients with hypopharyngeal SCC who had or had not undergone partial laryngectomy, partial hypopharyngeal resection or flap reconstruction surgery. This may be because surgeons performing total resection of the larynx and hypopharynx may make a more thorough tumour cut. The reason why patients who had undergone transoral laser microsurgery surgery had a better outcome may be because most of these patients were usually in the early stage of hypopharyngeal SCC.
This study had the following limitations: (1) the sample size was relatively small, and the data were obtained from a single medical centre, so the general applicability of the model should be further tested; (2) this model is only applicable to the prediction of the three-year survival status of patients with hypopharyngeal SCC and does not predict the five-year survival status because the proportion of patients who are still alive after five years is much lower than that after three years, so the data distribution would be too unbalanced during training. Therefore, we predicted only the three-year survival status of patients with hypopharyngeal SCC.

Conclusion
In summary, we used the XGBoost algorithm with 22 clinical parameters to establish a model for predicting the three-year survival status of patients with hypopharyngeal SCC and to analyse the role of each parameter in model construction. Model performance was relatively satisfactory and can offer clinicians a new option for predicting the survival status of patients with hypopharyngeal SCC.
Supplementary material. The supplementary material for this article can be found at https://doi.org/10.1017/S0022215123000063.