Fig. 1 presents the schematic workflow of the proposed DL model based on PET/CT for pathological grading of patients with PDAC, which consists of multiple processing stages. Below, we elaborate the details related to the workflow, starting with study population, image labeling, to model construction, and finally model testing.
Study population
Patients who underwent pancreatic surgery at the PLA General Hospital from January 2016 to September 2021 and obtained pathological confirmation of PDAC were collected and included in the study according to the inclusion and exclusion criteria, and 370 patients were finally included.
Inclusion criteria: (i) PDAC was pathologically confirmed by radical pancreatic resection; (ii) PDAC was confirmed by pathological biopsy of non-radical pancreatic surgery; (iii) PET/CT of the pancreas was performed within 1 month before surgery. Exclusion criteria: (i) patients had adjuvant treatment such as radiotherapy, chemotherapy and intervention before surgery; (ii) PET/CT images were of poor quality (tumour and borders could not be distinguished with the naked eye or there were artifacts interfering) and could not be used to analyze patients; (iii) other malignant tumours were combined; (iv) there was no significant FDG uptake at the tumour site (SUVmax < 2.5); (v) pathological findings and images could not correspond. Clinical data such as a patient’s age, gender, preoperative CA199 level, tumour location, tumour size (long and short diameter) on PET/CT images, and SUVmax values were also collected.
PET-CT image labeling process
Supplementary Method 1.1 provides detailed information about the PET/CT scanning protocol. The regions of abnormal 18F-FDG uptake on PET and density abnormality on CT are localized as the lesion region as follows. After the PET/CT image fusion is completed, two experienced PET/CT diagnostic physicians use 3D slicers (version 5.1.0, https://www.slicer.org) software with a threshold of 40% SUVmax to draw out the ROI (Region of Interest) of the target lesion, and all discrepancies are confirmed through discussion. All examination images are interpreted by two senior nuclear medicine specialists (with more than 5 years of PET interpretation experience), including the location of tumor lesion, its relationship with surrounding tissue, the presence of lymph node metastasis, the presence of distant metastasis, and the performance under different sequences.
Constructing the lesion segmentation model
The whole process of building the deep model for lesion segmentation is shown in Figure 1A. 100 cases of annotated PET/CT images of pancreatic cancer were input into the segmentation model for training.
The PET-CT images were first pre-processed: a. Window width and window level (350, 40) were applied to intercept the gray value; b. Each pair of 3D CT series (512*512*HCT) and 3D PET series (of size 96*96*HPET, 128*128*HPET, 168*168*HPET, 170*170*HPET) were uniformly resized to 256*256*HPET; c. For each slice, the gray scale was normalized [0,1]; d. 3 PET slices and 3 CT slices centered around the corresponding location form an 6-channel input to the model.
Model construction: The 6-channel input has a PET part and a CT part, each fed into a 2D-Unet network (Fig.S1.) branch with no shared parameters. The feature vectors of the two 2D-Unets are then concatenated to pass through convolutions, which output the final lesion segmentation mask. The learning rate was set to 1 × 10-5 and the parameters were updated using the Adam optimiser.
Post-processing the segmentation result: a. A pre-trained model of organ segmentation was added to nnUnet16 to provide a coarse segmentation of the abdominal organs. The predicted segmentation of pancreatic organ was then fused with the tumour segmentation results to enhance the lesion segmentation performance; b. Medical image analysis techniques including erosion, expansion, and SUVmax 40% threshold segmentation are further applied to obtain the final lesion segmentation results.
Building PDAC pathological grade classification models
Due to the low prevalence of pathological samples with extreme pathological differentiation grades in the clinic, the grades with few samples were merged in this study and all samples were set to two predictive labels: low grade or high grade. Highly, moderate-highly, and moderately differentiated pathologies were defined as low grade; undifferentiated, lowly, and moderate-lowly differentiated were defined as high grade (Fig. S2.). This is similar to the classification method of Wasif and Rochefort et al.17,18
According to the segmentation result, the lesion regions were cropped out of the 3D data of PET, CT, and segmentation Mask, respectively, and three aligned copies of size 64*64*16 (length*width*height) were obtained. The CT data was intercepted with a window width and window level (350, 40) and normalized to [0,1], and the PET data was normalized to [0,1]. The cropped PET, CT, and Mask were concatenated in the channel dimension to obtain a tensor of size 3*64*64*16 (number_of_channels*length*width*height). The tensor was fed into a Unet3D-based Encoder to extract image feature vectors as shown in Figure 1B.
Cases with clinical data missing ratios greater than 20% were excluded from our study. A total of 21 clinical variables were collected to build predictive models based on clinical experience and literature reports. Subsequently, the individual clinical data was analysed for significance using the Random Forest method (Fig.S3.). Eleven important clinical characteristics including age, BMI, SUVmax, ALT, AST, total bilirubin, direct bilirubin, blood glucose, CEA, CA125 and CA199 were kept. Finally, the clinical data feature vectors were extracted using the MLP through the multilayer perceptron. The part was shown in Figure 1C-D.
Both image features or clinical data features can be used to obtain prediction results for their respective modalities through the fully connected (FC) layer. To obtain better prediction performance, we replaced the last FC layer with a TMC (Trusted Multi-view Classification) 19 to integrate image features and clinical data features and constructed a PET/CT + Clinical data model. TMC is a new multi-view classification algorithm that dynamically integrates different views at an evidence level to promote classification reliability by considering evidence from each view(Supplementary Method 1.2). The learning rate was set to 1×10-5 and the parameters in the feature extractor were updated using the Adam optimiser.
Seven-fold cross validation for model testing
We used a 7-fold cross-validation to better evaluate the generalization ability of the model. This is shown in Figure 2. We divided 370 patients into 7 folds, of which 5 folds were the training set for the model in each training round, 1 fold was the internal validation set and 1 fold was used as the test set to test the final performance of the model. The next round was trained by changing the order of the training, validation and test folds. The final model is obtained by averaging the results of the 7 folds.
Statistical analysis
The clinical data was statistically processed using SPSS 22.0 statistical analysis software: normally distributed measures were expressed as x±s and comparisons between groups were made using the Student-t test. Skewed measures were expressed as median (range) and comparisons of count data were made using the 𝑋^2 test or Fisher's exact probability method.
Dice score was used to evaluate the pancreatic lesion segmentation model. Accuracy, sensitivity, and specificity of the test dataset results were calculated using receiver operator characteristic curve (ROC) for the classification models. P values less than 0.05 were considered statistically significant.