Machine Learning-Based Prediction of Invisible Intraprostatic Prostate Cancer Lesions on 68Ga-PSMA-11 PET/CT in Patients with Primary Prostate Cancer

Purpose 68 Ga-PSMA PET/CT has high specicity and sensitivity for the detection of both intraprostatic tumor focal lesions and metastasis. However, approximately 10% of primary prostate cancer are invisible on PSMA-PET (exhibit no or minimal uptake). In this work, we investigated whether machine learning-based radiomics models derived from PSMA-PET images could predict invisible intraprostatic lesions on 68 Ga-PSMA-11 PET in patients with primary prostate cancer. Methods In this retrospective study, patients with or without prostate cancer who underwent 68 Ga-PSMA PET/CT and presented negative on PSMA-PET image at either of two different institutions were included: institution 1 (between 2017 to 2020) for the training set and institution 2 (between 2019 to 2020) for the external test set. Three random forest (RF) models were built using selected features extract from standard PET images, delayed PET images, and both standard and delayed PET images. Then, subsequent 10-fold cross-validation was performed. In the test phase, the three RF models and PSA density (PSAD, cut-off value: 0.15ng/ml/ml) were tested with the external test set. The area under the receiver operating characteristic curve (AUC) was calculated for the models and PSAD. The AUCs of the radiomics model and PSAD were compared. 0.510, 0.813). The AUCs of the three radiomics models were higher than that of PSAD (0.903, 0.856 and 0.925 vs 0.662, respectively; P = .007, P = .045 and P = .005, respectively). Conclusion Random forest models developed by 68 Ga-PSMA-11 PET-based radiomics features were proven useful for accurate prediction of invisible intraprostatic lesion on 68 Ga-PSMA-11 PET in patients with primary prostate cancer and showed better diagnostic performance compared with PSAD.


Introduction
Prostate cancer (PCa) is the second leading cause of cancer death in men [1].The worldwide incidence rates signi cantly increased during the last decade, most likely due to the wider application of prostatespeci c antigen (PSA) screening [1].Early detection of primary disease and its metastases is highly relevant in terms of prognosis and therapy management.Systematic transrectal ultrasound-guided biopsy (TRUS-GB) is widely used for the diagnosis of PCa in men who present with an elevated serum PSA [2,3].However, this approach is responsible for the underdetection of clinically signi cant PCa due to potential sampling error and interobserver variability; also, this method leads to the overdetection of clinically insigni cant cancer [4][5][6].Moreover, it can cause procedure-related complications, including bleeding, pain and infection [2,3].
Imaging methods can help determine which men with elevated PSA continue to undergo biopsy, which may reduce unnecessary biopsies and improve diagnostic accuracy [2,3].MRI is a well-established, highly accurate imaging modality.Combining prostate-speci c antigen density (PSAD) with the MRI PI-RADS score may help de ne patients who need biopsy [3].Ahmed et al. [2] reported that using multiparameter MRI (mpMRI) might allow 27% of patients to avoid a primary biopsy and diagnose 5% fewer clinically insigni cant cancers.However, mpMRI is known to have low speci city in very low-risk patients and poor sensitivity in small intraprostatic tumors [3,7,8].Panebianco et al. [9] presented that small foci and disease within the central gland may not be easily identi ed by mpMRI, and up to 35% of clinically insigni cant PCas may be invisible to mpMRI. 68Ga prostate-speci c membrane antigen positron emission tomography/computed tomography ( 68 Ga-PSMA PET/CT), as a new imaging method based on molecular analysis rather than morphological or physiological analysis, has a higher speci city and sensitivity for detection of both intraprostatic tumor focal lesions and metastasis compared with MRI, as well as of tumor foci to characterize the bilateral and multifocal disease [10][11][12].Recently, 68 Ga-PSMA PET/CT showed satisfactory results to aid decision-making by con rming or eliminating the further need for biopsies [12][13][14].However, according to the literature, approximately 10% of primary PCa are negative on PSMA-PET (exhibit no or minimal uptake) despite high PSA levels [15][16][17][18][19], which may be due to small lesion size or lesion con guration [15].In literature by Zamboglou et al. [15], their result indicated that the incidence of invisible PCa lesions was not correlated with patients' basic clinical parameters or basic SUV parameters derived from gross tumor volume PET(GTV-PET), and 40% of invisible lesions were clinically signi cant (ISUP>1) PCa.How to deal with prostate cancer patients with negative ndings on PSMA-PET images remains a challenge.Thus, there is a need for simple, noninvasive, and accurate methods of detecting invisible intraprostatic lesions on 68 Ga-PSMA-11 PET in patients with primary prostate cancer.
Radiomics analysis, in which large numbers of features are extracted from images, including signal intensity, histogram-based features, textural features and transform lter-based features, has recently gained attention as a promising method for tumor biology characterization, prognosis prediction, and assisting in decision making [20,21].Compared with previous ways that process medical images as pictures for visual inspection, radiomics introduces a new way to mine the information contained in the medical images [22].Recently, radiomics combined with machine learning in MRI and PET/CT demonstrated the potential feasibility for prostate cancer detection, risk assessment and prognosis prediction [15,[23][24][25][26][27][28].Therefore, we hypothesized that machine learning models based on radiomics features extracted from 68 Ga-PSMA-11 PET images might allow a comprehensive assessment of imaging features associated with invisible intraprostatic lesions and improve prediction ability.
Therefore, in light of the potential of combining PET/CT imaging and machine learning-based radiomics analysis, the purpose of our study was to develop and validate radiomics models for predicting invisible intraprostatic lesions on 68 Ga-PSMA-11 PET in patients with primary prostate cancer.

Materials And Methods
The institutional review board approved this retrospective study, and the requirement to obtain informed consent was waived.

Study Population
We retrospectively analyzed 306 patients who underwent 68 Ga-PSMA-11 PET/CT between 2017 and 2020 at institutions 1 (The third a liated hospital of Sun Yat-Sen University, Guangzhou, China) and 132 patients who underwent 68 Ga-PSMA-11 PET/CT between 2019 and 2020 at institution 2 (Sun Yat-Sen memorial hospital, Guangzhou, China).Two nuclear radiologists ( N.Y.J and Y.Z, with more than 10 years of experience at prostate PET/CT interpretation), using Horos software (v4.0.0, https://www.horosproject.org),independently evaluated the image data and included negative PSMA-PET image cases according to previously described criteria: the prostate without focal radiotracer uptake higher than that of surrounding prostate tissue in more than one image slice on standard and delayed PET images (for an example of a patient without visualization of the primary PCa tumor, see Figure 1) [15,18,24,29].Any disagreement was resolved by consensus.To avoid recall bias, these two nuclear radiologists did not participate in the PET analysis session.Inclusion criteria were: (1) prostate cancers or non-PCas were con rmed by biopsy or postoperative pathology, non-PCas with the initial biopsy were further con rmed by follow-up for at least 6 months by PSA screening, MRI, second biopsy, or transurethral resection prostate (TURP) with the histopathologic examination; prostate cancers were pathology con rmed diagnosis of prostate adenocarcinoma.(2) a time-interval between 68 Ga-PSMA-11 PET/CT scans and histopathological examination less than 14 days.The exclusion criteria included the following: (1) history of other malignancies; (2) 68 Ga-PSMA PET/CT examination was performed after anti-tumor pharmacotherapy or surgery; (3) incomplete clinical, laboratory, pathological, or imaging data unavailable or with poor quality.
Patients identi ed at institutions 1 and 2 were assigned to the training and external test sets, respectively (Figure 2). 68Ga-PSMA PET/CT acquisition and image reconstruction 68 Ga-PSMA PET/CT images were obtained using a PET/ CT scanner (Discovery Elite, GE Healthcare, Milwaukee, WI) at institution 1 and a Biograph mCT PET/CT scanner (Siemens Healthcare, Erlangen, Germany) at institution 2, according to standard clinical scanning protocols.Images were acquired after intravenous administering of 1.8-2.2MBq/kg 68 Ga-PSMA-11.A diagnostic and non-enhanced CT scan (120 kVp, 120-150mAs) was performed rst using a slice thickness of 3.0 mm.Then, whole-body PET imaging (2-4 min per bed position, 6-7 beds) from skull to the midthigh was conducted at one hour(standard), and pelvic PET imaging (2-4 min per bed position, 1-2 beds) from the superior anterior spine to thighs was conducted at 2-2.5 hours(delayed) after intravenous injection, respectively.PET acquisition was performed in a three-dimensional (3D) model, and data were corrected with attenuation, scatter, random coincidences and decay correction.PET images were attenuated-corrected using CT data.
At institution 1, PET reconstruction was performed using an ordered subset expectation maximization (OSEM) and setting a spectrum of parameters, for instance, VUE Point FX module (GE healthcare), 3 iterations, 24 subsets, matrix 192×192, slice thickness of 3.27mm, pixel size 3.65×3.65×3.27mm 3 with a lter (6.4mm), and all necessary correction methods including attenuation and scatter correction.At institution 2, acquired PET images were reconstructed using a TrueX+TOF (ultraHD-PET) with 3 iterations and 21 subsets, and a Gaussian lter of 5.0mm into an image matrix size of 200×200 with a voxel size of 4.1×4.1×3.0mm 3 , and all necessary correction methods including attenuation and scatter correction.Subsequently, PET images were converted into SUV units by normalizing the activity concentration to the dosage of 68 Ga-PSMA-11 injected and the patient body weight after decay correction.

Development of the Radiomics Models
Tissue segmentation.PET images and axial CT images were loaded into ITK-SNAP software, version 3.8.0(open source, http://www.itksnap.org)for segmentation [30].The masks of prostatic gland volume were re ned by two nuclear radiologists (Z.L.Y and S.Q.H, with 5 and 3 years of experience, respectively) who were blind to the clinical data.Three-dimensional ROIs of the left and right half-prostate glands were manually segmented on PSMA-PET images under consideration of the corresponding CT scan (Figure 3).Radiomics feature extraction. 68Ga-PSMA-11 PET-based radiomics features were derived by using the Pyradiomics package (version 3.0.1,https://github.com/Radiomics/pyradiomics)for Python (version 3.6.1).A total of 3562 quantitative features (1781 features for each phase) were extracted from the prostate ROIs on standard and delayed PET images, respectively.For each phase PET image, the 1781 features consisted of 107 features (including 14 morphology features, 18 original rst-order features, 16 gray level size zone matrix (GLSZM) features, 14 gray level dependence matrix (GLDM) features, 5 neighboring gray level dependence matrix (NGLDM) features, 24 gray level co-occurrence matrix (GLCM) features, 16 gray level run length matrix (GLRLM) features) were calculated on the original images; 744 features were calculated on the wavelet-ltered images; 465 features were calculated on the Laplacian of Gaussian (LoG)-ltered images; 186 features on the square-ltered images; 93 features on the gradientltered images; 93 features on the exponential-ltered images; 93 features on the logarithm-ltered images.
Feature Selection and Radiomics model development.In our study, the number of radiomics features was large, while the number of cases was relatively small.To mitigate model over tting and potentially improve generalizability, the classic minimum redundancy maximum relevance (mRMR) method (https://github.com/smazzanti/mrmr)was used to select the 10 most useful predictive features with R 2 difference.This algorithm ensures the selection of highly relevant features to actual classes while controlling for the redundancy within the selected features.The mRMR algorithm has been proven effective in both radiomics and genomics studies, where a small subset of features must be chosen out of the thousands of features [24,26,31].Then, we constructed a machine learning framework in Python 3.6.1 using Scikit-learn library 0.23.2 (https://scikit-learn.org/stable/whats_new/v0.23.html#version-0-23-2).The random forest (RF) machine learning model was applied using the selected features.The hyperparameters of the RF models were optimized within each training set using the GridSearchCV (Scikit-learn library 0.23.2) method with 10-fold cross-validation iteration.The average AUC score, the average precision score, the average recall score, the average accuracy score and the average F1 score obtained after 10-fold cross-validation were provided to evaluate the performance of the trained RF models for the training set.The hyperparameters are provided in Supplementary Table 1.The overall work ow of the radiomics model development is presented in Figure 3.

External test and Comparison of different machine learning models and PSA Density
The three trained RF models were applied to the external test set.The performance of the models was evaluated by the receiver operator characteristic curve (ROC) analysis.The best cut-off point for each model was estimated by using the Youden index to calculate the sensitivity, speci city, positive predictive value (PPV) and negative predictive value (NPV).
Prostate-speci c antigen density (PSAD) was selected as the single factor for comparison with the radiomics model.PSAD was calculated as total PSA (ng/ml) divided by prostate volume (ml).Based on a previous literature review [3], the PSAD cut-off value was set at 0.15ng/ml/ml.In this study, the prostate volume was calculated by CT-based prolate ellipsoid formula (three diameters measured directly on the CT images, volume = width × height × length × π/6) [32,33].

Statistical Analysis
The differences regarding the demographic data between the training and external test sets were assessed.Statistical analyses were performed with the R statistical software (version 4.0.5;R Foundation for Statistical Computing).Continuous variables were tested by Student's T-test or Mann-Whitney U test between two groups.The Chi-square test or Fisher's exact test, where appropriate, was applied for comparing categorical variables.The AUCs of the radiomics models were compared with that of the PSAD using the DeLong method [34].

Development and Testing of the Radiomics Model based on Standard and Delayed PET images
To develop the RF model based on standard and delayed PET images, we used both the 10 most useful predictive features of standard PET images and the 10 most useful predictive features of delayed PET images selected by the mRMR method.
The performance of trained radiomics RF models based on standard PET images, delayed PET images, and both standard and delayed PET images from 10-fold cross-validation were summarized in Table 2.
The AUCs of radiomics RF model based on standard PET images, delayed PET images, and both standard and delayed PET images were higher than that of PSAD (0.903, 0.856 and 0.925 vs 0.662, respectively; P = .007,P = .045and P = .005,respectively).
The diagnostic performance of the three radiomics models and the use of PSAD (cut-off value, 0.15ng/ml/ml) in the external test set were summarized in Table 3.The ROC curves of the three radiomics models and PSAD were shown in Figure 5.

Discussion
In this study, we developed three radiomics RF models (based on standard PET images, delayed PET images, and both standard and delayed PET images, respectively) by allowing for the prediction of invisible intraprostatic lesions on 68 Ga-PSMA-11 PET in patients with primary prostate cancer.The three radiomics RF models developed by using the training cohort data enabled the discrimination in the external test cohort with areas under the curve of 0.856-0.925and accuracies of 79.4%-87.0%.Although there were no differences among the AUCs of the three radiomics RF models, the RF model based on both standard and delayed PET images demonstrated excellent diagnostic performance with a test set (AUC, 0.925; sensitivity, 85.0%; speci city, 88.5%; accuracy, 87.0%; PPV, 85.0%; NPV, 88.5%).In addition, the three radiomics RF models showed better diagnostic performance than PSAD ( P< 0.05).These ndings suggest that predictive machine learning-based radiomics models could assist in predicting invisible intraprostatic lesions on 68 Ga-PSMA-11 PET in patients with primary prostate cancer.
Previous studies evaluated the feasibility of radiomics models based on T2-weight images, diffusionweighted images and ADC images for differentiating non-cancerous prostate from prostate cancer and histopathological prediction grade of PCa [23,25,26,35], as well as 68 Ga-PSMA-11 PET images [15].Wibmer et al. [23] suggested that radiomics features extracted from T2WI and ADC maps were helpful for differentiating non-cancerous and cancerous prostate tissue, and tumor Energy and Entropy on ADC maps correlated with Gleason score.In recent literature, Xu et al. [35] developed a prediction model using biparametric MRI radiomics features that showed high preoperative diagnostic performance (AUC=0.92) in discriminating between benign and malignant prostate lesions.However, Bonekamp et al. [25] demonstrated that radiomics machine learning had comparable but not better performance than mean ADC assessment differentiation of benign versus malignant prostate lesions.Although these methods yielded promising results, the subjects were all based on visually visible lesions on MRI.The above research results may not be applicable to the prediction of small, invisible intraprostatic cancer lesions.Moreover, none of these studies were validated in separate external test data sets, leaving the generalizability of their results unproven.Gong et al. [26] constructed radiomics models by delineating the whole prostate gland on T2-weight images and diffusion-weighted images, which can noninvasively identify high-grade PCa before the operation.However, this study only analyzed patients with obvious prostate cancer lesions on MRI, and a separate external test data set was also lacked.In a recent report by Zamboglou et al. [15], PSMA-PET-derived radiomics features derived from 68 Ga-PSMA-11 PET images were recently reported helpful in visually unknown PCa detection (sensitivities ≥ 0.8 in the validation cohort).In their study, non-PCa tissue was de ned as the subtraction volumes between the prostatic gland and PCa tumors in 68 Ga-PSMA-11 PET based on pathological tissue slices from PCa patients.However, radiomics features need to be further studied whether differences exist between the prostate tissue in the non-tumor area of prostate cancer patients and the prostate tissue in non-tumor patients.In addition, unlike our training set that included patients with and without PCa, the authors extracted and analyzed radiomics features from a training set with a small sample size (n=20), and only patients with PCa were enrolled.The accuracies of the two selected radiomics features (local binary pattern (LBP) sizezone non-uniformity normalized and LBP small-area emphasis) for prediction visually undetectable intraprostatic tumor lesions in 68 Ga-PSMA-11 PET images (area under the curve were 0.80 and 0.83 in half prostate level analysis, respectively) were slightly lower than ours, although a head-to-head comparison is needed to compare these results.
Among the 20 radiomics features selected using the mRMR method for developing RF models, the original_ rstorder_MeanAbsoluteDeviation feature extracted from standard PET images was the only feature directly related to the standard intake value (SUV).In this study, original_ rstorder_MeanAbsoluteDeviation was the average distance of all image values mean array, which can be used to assess the variability of metabolic activity [36].The original_ rstorder_MeanAbsoluteDeviation feature in the prostate cancer subset was higher than that in the non-PCa subset in both cohorts in our study, which may re ect that the untaken of 68 Ga-PSMA-11 in the prostate gland with cancerous tissue was more nonuniformity than the non-malignant prostate.Previous studies suggested that the deviation of SUV was helpful for differentiating solitary lung adenocarcinoma from tuberculosis [37], in agreement with our results.Five skewness and kurtosis features applied with Wavelet, Gradient, Squareroot and Exponential transform lters (wavelet_HHH_ rstorder_Kurtosis, wavelet_LHL_ rstorder_Skewness, gradient_ rstorder_Median, squareroot_ rstorder_Skewness, and exponential_ rstorder_Skewness) extracted from delayed PET images were also selected for modeling.Skewness and kurtosis are more sophisticated histogram-based features, which can be used to describe heterogeneity [38].Previous literature had shown promising results using skewness and kurtosis features for differentiating different types of malignant tumors and predicting response to therapy in various tumor types [39,40].Mean values of these features in the prostate cancer subset were lower than the non-PCa subset.This similar nding may also re ect more heterogeneity untaken of 68 Ga-PSMA-11 in prostate with cancerous tissue in delayed PET images.The remaining 14 texture radiomics features extracted from standard and delayed PET images for modeling include 3 Gray-Level Cooccurrence Matrix (GLCM) features, 5 Gray-Level Size Zone Matrix (GLSZM) features, 2 Gray-Level Run-length Matrix(GLRLM) features and 4 Gray-Level Dependence Matrix (GLDM) features associated four different lters Wavelet, LoG, Squareroot and Logarithm transform lters.These texture features calculate the nonuniformity of the gray levels or the length of the homogeneous zones, re ecting the heterogeneity within the delineated area.Kuess et al. [41] indicated that GLCM-based features extracted from mpMRI data could be used to differentiate between tumor and healthy tissue in the peripheral zone of the prostate.Sun et al. [27] reported that GLCM, GLSZM and GLRLM-based features extracted from T2-weighting images were helpful for the strati cation of prostate tumor aggressiveness.In a study by Mohamed et al. [42], GLDM-based features extracted from ultrasound images were bene cial for prostate cancer diagnosis and aiding biopsy planning.Our results are consistent with those of the studies mentioned above.However, in the above studies, the radiomics features were extracted from MR or ultrasound images, which contained less information on the expression of speci c PSMA molecules closely related to prostate cancer compared with 68 Ga-PSMA PET images.Moreover, the value of radiomics features based on the whole prostate in differentiating with or without cancer components has not been studied.In this study, GLSZM, GLCM and GLDM-based features were the most important features of the three radiomics RF models (Supplementary Table 2), respectively, which may support that prostate gland with cancer component were more heterogeneous than those without malignancy on both standard and delayed PET images.PSAD, a well-established predictor of clinically signi cant PCa, has been showing to be a useful predictor for selecting patients who need to undergo mpMRI and to optimize the diagnostic pathway for patients presenting with negative mpMRI (PI-RADS 1-2) [3,43,44].Combining PSAD with the PI-RADS score may help de ne patients who need biopsy [3].In this study, the diagnostic performance of PSAD in the external test sets was additionally compared to radiomics RF models.In the external test set of our study, the AUC of PSAD was 0.662, which slightly underperformed in comparison with a previous report (AUC: 0.717) [45].Compared to PSAD, the three radiomics RF models (AUCs: 0.856-0.925)developed in this study showed better diagnostic e ciency.Furthermore, half-glandular segmentation was used for delineation.Previous radiomics studies of prostate cancer mainly extracted and analyzed features based on delineating the tumor [23,25,26,35].However, when we encounter smaller invisible cancerous lesions in the prostate, using this method will lead to a considerable challenge in determining the interest to be delineated.Recently, Gong et al. [26] developed radiomics models based on delineated the whole prostate gland and showed good encouraging performances for noninvasively distinguishing high-grade PCa.Zamboglou et al. [15] proved that radiomics features derived from 68 Ga-PSMA-11 PET images based on half-glandular segmentation were helpful for predicting invisible PCa lesions.The above results demonstrated the feasibility of using segmentation methods based on half-glandular or full-glandular segmentation.In addition, compared with tumor segmentation, half-glandular or full-glandular segmentation may be more convenient for applying automatic or semi-automatic segmentation [46].
Our study has some limitations: (1) Although this study was a two-centers design, the relatively small sample size in both the training set and external test set may compromise the model's generalization ability and affect its sensitivity and speci city.A multicenter study with a big sample size to validate the ability of the radiomics model in this study should be applied in further research.(2) This study is retrospective with possible selection bias.The proportion of PCa patients enrolled with high-grade (GS >7) PCa was more than those with lower-grade (GS ≤7) PCa, which may cause the model to potentially overestimate the risk of patients with low-grade (GS ≤7) PCa.(3) To calculated the PSAD value, we measured the diameters of the prostate gland on abdomen CT images.Although CT had better spatial resolution than MRI and ultrasound images, previous studies mainly used MRI and ultrasound to estimate the volume of the prostate gland [32,33].

Conclusion
In conclusion, random forest models developed by 68 Ga-PSMA-11 PET-based radiomics features showed good predictive performance.Radiomics machine learning models may be a promising non-invasive method for accurate prediction of invisible intraprostatic lesion on 68   and a PSA value of 15.66 ng/ml.On fused PET/CT images and PET images alone, no focal radiotracer uptake is higher than that of surrounding prostate tissue.Supplementary Files Development and Testing of the Radiomics RF Model based on Standard PET images A total of 1781 radiomics features were extracted from the prostate ROIs on standard PET images.The 10 most useful predictive features of standard PET images selected by the mMRM method were one original rst order features, 3 wavelet-based features, 4 Laplacian of Gaussian-based features, one squareroot-based feature and one logarithm-based feature.A cluster heatmap of these radiomics features was generated by means of agglomerative hierarchical clustering to visualize the differences between PCa and non-PCa groups and the association between the found clusters of subjects and features.(Figure4.a).
Development and Testing of the Radiomics Model based on Delayed PET imagesA total of 1781 radiomics features were extracted from the prostate ROIs on delayed PET images.The 10 most useful predictive features of delayed PET images selected by the mRMR method were 5 waveletbased, one Laplacian of Gaussian-based features, one square-based feature, one exponential-based feature, one logarithm-based feature and one gradient-based feature.A cluster heatmap of these radiomics features was generated by means of agglomerative hierarchical clustering to visualize the differences between PCa and non-PCa groups and the association between the found clusters of subjects and features.(Figure 4.b).

Figures Figure 1
Figures

Figure 2 Flow
Figure 2

Figure 3 Work
Figure 3

Table 2 .
Ga-PSMA-11 PET in patients with primary prostate cancer.Rough P values were calculated using the Student's test between patients in the training and external test sets.ξ Rough P values were calculated using the Mann-Whitney U test between patients in the training set and external test set.§ Rough P values were calculated using the Chi-square test between PCa patients in the training set and external test set.Performance of Trained RF models with 10-fold Cross-validation in Training Sets PCa = prostate cancer; tPSA = total prostate speci c antigen; PSAD = prostate speci c antigen density.*