A novel machine learning-based radiomic model for diagnosing high bleeding risk esophageal varices in cirrhotic patients

To develop and validate a novel machine learning-based radiomic model (RM) for diagnosing high bleeding risk esophageal varices (HREV) in patients with cirrhosis. A total of 796 qualified participants were enrolled. In training cohort, 218 cirrhotic patients with mild esophageal varices (EV) and 240 with HREV RM were included to training and internal validation groups. Additionally, 159 and 340 cirrhotic patients with mild EV and HREV RM, respectively, were used for external validation. Interesting regions of liver, spleen, and esophagus were labeled on the portal venous-phase enhanced CT images. RM was assessed by area under the receiver operating characteristic curves (AUROC), sensitivity, specificity, calibration and decision curve analysis (DCA). The AUROCs for mild EV RM in training and internal validation were 0.943 and 0.732, sensitivity and specificity were 0.863, 0.773 and 0.763, 0.763, respectively. The AUROC, sensitivity, and specificity were 0.654, 0.773 and 0.632, respectively, in external validation. Interestingly, the AUROCs for HREV RM in training and internal validation were 0.983 and 0.834, sensitivity and specificity were 0.948, 0.916 and 0.977, 0.969, respectively. The related AUROC, sensitivity and specificity were 0.736, 0.690 and 0.762 in external validation. Calibration and DCA indicated RM had good performance. Compared with Baveno VI and its expanded criteria, HREV RM had a higher accuracy and net reclassification improvements that were as high as 49.0% and 32.8%. The present study developed a novel non-invasive RM for diagnosing HREV in cirrhotic patients with high accuracy. However, this RM still needs to be validated by a large multi-center cohort.


Introduction
The esophageal varices(EV) bleeding(EVB) is one of the lethal complications of liver cirrhosis with portal hypertension.The 6-week mortality of acute EVB reaches more than 15% [1].The 3-and 5-year cumulative mortality are 24.7% and 47.2% in cirrhotic patients by endoscopic or/and medicine therapy for secondary prophylaxis after rst episode [2].Therefore, it is very important to diagnosing EV in cirrhotic patients, especially moderate-severe EV, namely high bleeding risk esophageal varices(HREV) needed treatment.Currently, the endoscopy is still "gold standard" for evaluation degree of EV.Guidelines recommend that all patients should undergo endoscopy once cirrhosis is con rmed in patients with chronic liver diseases and surveillance EV every from 1 to 3 years [3,4].However, endoscopy is an invasive procedure and may bring complications such as bleeding, cardiac tear, and discomfort.Hepatic vein pressure gradient (HVPG), as an alternative indicator of portal pressure, indicates an increased bleeding risk of EV when HVPG is more than 12 mmHg.However, HVPG measurement is not only an invasive operation but also requires skilled operators to perform [5].Thus, it is important to develop a non-invasive method to screen EV and diagnose HREV to avoid unnecessary endoscopy [6,7].Based on the meta-analysis of observational studies recently, it suggested that computed tomography (CT) imaging, a non-invasive diagnostic tool, is a good choice for evaluation of EV in cirrhotic patients [8].However, contrast-enhanced CT images have a large amount of features that cannot be completely distinguished and interpreted by doctors.Through arti cial intelligence (AI) technology, namely machine learning algorithm, we can analyze unordered and underlying pathophysiological features [9].Therefore, the aim of this study is to develop and validate a novel radiomic model (RM) for diagnosing HREV in cirrhotic patients based on machine learning algorithm with contrast-enhanced CT images.

Study design
This is a retrospective single center multi-cohort study.The inclusion criteria were: (1) age18-85 years old, (2) patients diagnosed with cirrhosis, con rmed by histopathology and/or typical symptoms and signs of laboratory and imaging ndings, including abdominal ultrasonography [10,11], (3) patients who underwent endoscopy and contrast-enhanced CT scan, the interval time between CT scan and endoscopy should be within 3 months.The exclusion criteria were: (1) patients with non-cirrhotic portal hypertension, (2) patients who previously underwent one of the following therapies: transjugular intrahepatic portosystemic shunt, splenectomy, partial splenic embolization, or endoscopic variceal therapy (endoscopic varicel ligation and injection sclerotherapy), (3) patients diagnosed with primary or metastatic hepatocellular carcinoma or other tumors, (4) pregnant or lactation patients.The study protocol was performed in compliance with the Declaration of Helsinki and approved by the Ethics Committees of Beijing You'an Hospital Capital Medical University (Approved number: 2019-074).Signed informed consent was obtained from each patient for using samples, materials and publication.

Subjects
Total 796 quali ed participants were enrolled for study.In training cohort, 391 patients with cirrhosis due to any cause were enrolled between January 2016 to December 2017 for.Among them, patients with mild EV (mild EV, n=120), moderate-severe EV (HREV, n=162) and without EV (non EV, n=109) were included.109 patients were randomly selected from mild EV group to match patients without EV, 120 patients were randomly selected from HREV group to match patients with mild EV to balance the number of cases.Ultimately, 218 (non EV and mild EV) and 240 (mild EV and HREV) patients were separated into the training and internal validation of mild EV and HREV RM (Figure 1a).Additionally, in external validation cohort, 405 cirrhotic patients with hepatitis B from January 2018 to December 2019 were consecutively enrolled, including mild EV(n=94) or HREV (n=246) or non EV(n=65).159 (non EV and mild EV) and 340 (mild EV and HREV) patients were respectively used for external validation of mild EV and HREV RM (Figure 1a).

Clinical data and Biochemistry
Clinical data included sex, age, laboratory test results, diagnosis, underlying etiology, and complications of cirrhosis/portal hypertension from the electronic medical record.The etiologies of viral cirrhosis were hepatitis B(serum HBsAg positive) ,hepatitis C(serum anti-HCV and HCV-RNA positive).Cirrhosis due to alcoholic, autoimmune and metabolism-related fatty liver diseases were identi ed according to guideline [12,13,14].Platelets counts (PLT) were analyzed with XE-5000 (Sysmex, Kobe, Japan).

EV evaluation by endoscopy
The exible GI endoscope (GIF-CV290 or GIF-CV260, Olympus, Japan) was used.The endoscopy was carried out by the experienced gastrointestinal physician and reports were consistently reviewed by two hepatologists to classify the degree of EV as follows [15]: mild EV: linear or slightly tortuous EV(≤ 3 mm diameter) without red signs (Figure 2a); moderate EV: mild EV with red signs or EV with snake or tortuous bulges (3-10 mm diameter) without red signs (Figure 2b); severe EV: medium EV with red signs, or beadlike, or nodular form or EV(>10 mm), regardless of whether there is a red sign or not (Figure 2c).The moderate-severe EV was regarded as HREV in need of treatment [16].

Contrast-enhanced CT image acquisition
All enrolled patients had abdominal contrast-enhanced abdominal CT using 64-slice CT scanner (GE Lightspeed VCT, GE Healthcare, USA).The following parameters were used-collimation, 0.625 mm; tube voltage, 120 kVp; tube current, 380 mA; slice thickness, 5 mm; pitch, 1.375.All patients received an intravenous, Iopromide (iodine concentration, 370 mg/mL) 100 ml via an antecubital vein at a rate of 3 ml/s using a high-pressure injector.All the patients underwent triple-phase contrast-enhanced scans.

Radiomic features extraction
All radiomic data and development of RM were programmed in the Python 3.5 based on Ubuntu 16.04.4.Firstly, Regions of interest (ROI) were created manually on the portal vein phase of contrastenhanced CT images (DICOM data).Two hepatologists carefully identi ed tissues in application(AIMED, Version 2.2.3, Beijing Blot Info & Tech Co. Ltd, China) on each slice and labeled liver, spleen and esophagus as ROI (Figure 2d).Since the lesion parts of EV mostly occured within 5 cm esophagus above the cardia [17], ROI of esophagus were labeled in this area.If there was a different opinion on choosing ROI, it will be decided after the two hepatologists review together.3D reconstruction with marching cubes algorithm and radiomic features extraction were performed on the marked ROI areas reconstructed as 1×1×1mm voxel.The above reconstruction process could reduce the difference of image pixel spacing and form 3D ROI.Four groups of radiomic features were extracted from the segmented ROI including textural features, statistical features, wavelet features and histogram of oriented gradient (HOG) features.
Finally, the evaluator were blinded on the patient's endoscopic results in advance.

Dimension reduction
The extraction of imaging features produced a lot of data, which will reduce the e ciency of models development.To simplify the data analysis, we used principal components analysis (PCA) to make dimension reduction.The process is as following.First, invalid image omics features (such as in nite value, null value, features with variance of 0, etc.) are removed, the remaining features are standardized and a covariance matrix was created to obtain the eigenvalues and eigenvectors; Equation is as follows: Xi'= (Xi -Xmin) / (Xmax -Xmin).Xi': The image features of the patient standardized in this dimension.Xi: The image features of the patient non-standardized in this dimension.Xmax: The maximum value of this dimension.Xmin: The minimum value of this dimension.Then, sorting the eigenvalues from large to small, selecting the principal components, nding their corresponding eigenvectors, projecting the original dimension data onto the selected eigenvectors, the dimensions of the original data characteristics were reduced, and applying the selected dimension data to represent the original data for processing and analysis .

Machine learning algorithm and development of RM
We tested several classi cation algorithms diagnosing of mild EV or HREV in training cohort.Finally, support vector machine (SVM) with highest AUC and ACC was selected to establish the model.The basic purpose of SVM is to nd the best separating hyperplane on the feature space so that the interval between positive and negative samples in the training cohort is maximized.The aim of this algorithm is to minimize the function value: is regularization term, which is used to reduce the in uence of over tting.The equation is the essential of algorithm to nd hyperplane for classi cation.5-fold cross-validation were applied to evaluate models and optimize parameters Figure 1b.The participants were randomly divided into 5 parts, one part was used as training and development of RM, and the other 4 parts was used as internal validation.Five models could be obtained and e ciency evaluation was re ected in the form of average value.The RM with minimum mean squared error (MSE) in 5-fold internal cross-validation was selected as the optimization.After retraining optimal RM with all participants, the nal RM was obtained which used for external validation.
E cacy evaluation of RM AUROC, sensitivity, speci city, accuracy were calculated as indicators to evaluate the discrimination of RM.Calibration curve re ects the agreement between predictions from the models and observed outcomes, and decision curve analysis (DCA) was applied to determine the clinical usefulness of the RM net bene ts at different threshold probabilities.Baveno VI consensus proposed that cirrhotic patients with Baveno VI criteria extended the threshold to LSM< 25kpa and PLT>110*10

Clinical characteristics
The ow chart of quali ed participants were shown in Fig. 1a-b  .002.001ALB(g/L) ^35.

Evaluation Of Rm
The average AUROC of RM for mild EV in the training was 0.943 (Fig. 3a), with an average sensitivity 0.863, speci city 0.763 and accuracy 0.841.When applied to the internal validation, the average AUROC was 0.732 (Fig. 3b), with an average sensitivity of 0.773, speci city of 0.763 and accuracy of 0.705.When applied to the external validation, the AUROC dropped to 0.654 (Fig. 3c), with a sensitivity of 0.773, speci city of 0.632 and accuracy of 0.641(Table 2).Importantly, the average AUROC of the RM for HREV in the training was 0.983 (Fig. 3d), with an average sensitivity 0.948, speci city 0.977 and accuracy 0.965.When applied to the internal validation, the average AUROC was 0.834 (Fig. 3e), sensitivity 0.916, speci city 0.969 and accuracy 0.947.When applied to the external validation, the AUROC reached 0.736 (Fig. 3f), with sensitivity 0.690, speci city 0.762 and accuracy 0.743(Table 2).The calibration curve and DCA results of the RM were shown in Fig. 4a-f respectively.The DCA showed relatively good performance of the RM for diagnosing of mild EV and HREV.The net bene t was calculated by deducting the proportion of all patients who were false positive from the proportion who were true positive.Patients could bene t by using the proposed RM for diagnosing of HREV.
In external validation cohort, we also evaluated the e cacy of diagnosing of mild EV and HREV patients compared with Baveno VI and its expanded criteria.The accuracy, sensitivity and speci city were 0.629, 0.813, 0.149 and 0.647, 0.752, 0.372 of Baveno VI and expanded Baveno VI criteria, respectively.The NRI of RM were 49.0% and 32.8% compared with Baveno VI and expanded Baveno VI criteria respectivrly.

Discussion
Currently, doppler ultrasound, transient elastography, MRI and contrast-enhanced CT are all commonly used as non-invasive imaging methods for evaluation of portal hypertension in cirrhotic patients.Doppler ultrasonography is easy to operate and can show changes in velocity and direction of portal vein blood ow; But it is susceptible to intestinal gas, and the accuracy is affected by operator's experience.Transient elastography is widely used for diagnosis of early liver cirrhosis by measuring the stiffness of the liver and spleen.However, it cannot be applied to patients with large ascites, and the accuracy is reduced in obese patients [19].MRI and magnetic resonance elastography can also be applied to evaluate EV with high cost-effectiveness [20,21].Interestingly, contrast-enhanced CT can be used to diagnose vascular diseases with high sensitivity and speci city [22].liver, spleen, and portal vein system are clearly shown in the portal vein phase by three-dimensional(3D) reconstruction.Xie et al used CT to explore the risk of rst EV bleeding in cirrhotic patients found that the total area of EV had signi cance for prediction bleeding [6].The AUROC was 0.82 with cutoff value of 1.03 cm 2 .Han et al also used spleen hemodynamics (spleen iodine concentration and spleen volume) in dual-energy enhanced CT to predict EV in cirrhotic patients [7].The AUROC for the detection of high-risk EV was 0.87 (95% CI: 0.77-0.94).However, There are no evidences of external validation above-mentioned studies.
Our study was to develop and validate the RM for identi cation of HREV in cirrhotic patients based on machine learning with contrast-enhanced CT imagine.Finally, 163 and 177 radiomic features were respectively included in mild EV and HREV RM.In the training, the average AUROC of RM for mild EV and HREV were 0.983 and 0.943.As well, the average AUROC in the internal validation were 0.834 and 0.732.
It is suggested that RM were valuable in the diagnosing varices, especially identi cation of mild EV and HREV.Moreover, we enrolled 159 and 340 hepatitis B cirrhotic patients for external validation.The AUROC of RM for mild EV dropped to 0.654.However, the AUROC of RM for HREV reached 0.736, which showed highly identi cation ability.Moreover, DCA and calibration curve exhibited good performance.It showed that RM has good value in screening HREV.
Baveno VI recommends that patients with a liver stiffness < 20 kPa of transient elastography and PLT > 150×10 9 /L have a very low risk of varices requiring treatment and can avoid endoscopy[16].Sousa et al found this criteria had a high sensitivity but low speci city [23].It suggested that, in clinical practice, there are still many patients had an unnecessary endoscopy.Salvador lauched expanded Baveno VI criteria for sparing more endoscopies [18].But in our study, the expanded criteria with same problem of high sensitivity and low speci city as Baveno VI criteria were observed.The accuracy was not better than RM, yet.According to NRI, it exhibited the distinguishing e ciency of RM for diagnosing HREV had improved by 49.0% and 32.8% compared with Baveno VI and expanded Baveno VI criteria respectively.In conclusion, It showed that the value of the machine learning-based RM, as a novel non-invasive tool for diagnosing HREV in cirrhotic patients with highly accurate, surpassed previous non-invasive diagnostic tools.
Our study had the following limitations: First, the external validation cohort were only enrolled the cirrhotic patients with hepatitis B, and it was unable to determine the identi cation ability of HREV in cirrhotic patients with other etiologies.Second, in this study, a small samples and single center subjects were studied for external validation.Therefore, a larger and multi-center cohort for external validation are needed in the future.

Figures Figure 1
Figures

Figure 2 The
Figure 2

Figure 3
Figure 3 9/L[18].In other words, people who exceed these criteria are at risk of EV bleeding and should be screened with gastroscopy.We would evaluate the e cacy of RM diagnosing HREV by accuracy and net reclassi cation improvement (NRI) in external validation cohort compared with Baveno VI, expanded Baveno VI criteria respectively.The mean and standard deviation(SD) were expressed for normally distributed quantitative data.The median and inter-quartile range were expressed in non-normally distributed data.Normally distributed data matrixes were compared by the independent Student's t test, while non-normally distributed data matrixes were compared by Mann-Whitney U test.Frequencies and proportions were used to summarize enumeration data with Chi-square test.All authors had access to the study data and reviewed and approved the nal manuscript.
. The clinical characteristics of enrolled patients in training and external validation cohort were summarized in Table 1 in detail.

Table 2
Performance of RMs for diagnosing of mild EV and HREV * The average value of AUROC, sensitivity, speci city, accuracy in 5-fold cross-validation.Abbreviation HREV high bleeding risk esophageal varices, AUROC area under the receiver operating characteristic curves.