In this present study, we created a pipeline using transfer learning on a deep convolutional neural network to predict clear and eosinophilic (including mixed) phenotypes from the manually extracted highest-grade regions of whole-slide images. We revealed that the high AI score group, representing the predicted eosinophilic phenotype, significantly correlated with adverse pathological factors and gene signatures related to the responsiveness to checkpoint blockade. Our AI model detected clear and eosinophilic phenotypes with high prediction in the validation cohort (AUC = 0.929) and stratified patients’ outcomes in the TCGA-KIRC cohort (p = 0.008).
It can be challenging to define clear and eosinophilic phenotype on H&E-stained slides because the preparation of the pathological specimens and staining methods which often vary among institutions. In this current work, we successfully validated the clinical significance of clear or eosinophilic histological phenotypes, shown in our prior single-center study [4], via transfer learning using multi-institutional H&E images with color variations for clear and eosinophilic phenotypes from the TCGA-KIRC dataset. Furthermore, we confirmed that clear and eosinophilic phenotypes, predicted by our AI model, correlated with clinicopathological characteristics and gene signatures associated with angiogenesis and cancer immunity and were in accordance with the results of the urologic pathologist’s manually assessed histological phenotypes. Our findings suggest that identifying clear or eosinophilic phenotypes may facilitate prognostic predictions and treatment selections.
To the best of our knowledge, this is the first histology-based AI model using H&E images to predict clear or eosinophilic phenotypes, correlated with gene signatures related to current treatment options and prognosis. Although the application of AI for diagnosis and prognostic predictions in renal cancers has been recently developed [9], previous AI algorithms were mainly focused on distinguishing the histological subtypes [10, 22], stages [23, 24], survival [25, 26], and nuclear grades [27, 28] of ccRCC. Recently an AI prediction model based on gene signatures has been developed using RNA sequencing data from the TCGA dataset and may be helpful in treatment and surveillance decisions [23, 29]. However, a molecular-based AI model is far from being implemented in routine clinical practices.
Recently, Chen et al. trained a deep convolutional neural network on H&E histology images available from the TCGA to classify EMT molecular classifications of ccRCC, characterized by distinctive genomes, metabolic states, and immune components [20]. Their approach distinguished the prediction of epithelial or mesenchymal subtypes in H&E slides and suggested that patients in the mesenchymal subtype might respond better to checkpoint blockades combined with angiogenetic therapies [20]. Similar to their study, our research group previously reported that mixed or eosinophilic phenotypes of ccRCCs showed upregulation of EMT gene signatures, a higher prevalence of sarcomatoid/rhabdoid features, and poor responsiveness to angiogenetic therapy [4].
While clear and eosinophilic phenotypes may be analogous to epithelial and mesenchymal subtypes in terms of underlying genomes and treatment response, histological features of epithelial and mesenchymal subtypes, corresponding to eosinophilic and clear cytoplasmic features, respectively, were the opposite in our results. Nevertheless, we believe that eosinophilic phenotypic areas tend to dedifferentiate, such as EMT, according to previous findings that eosinophilic phenotypic areas molecularly and immunohistochemically displayed a lower differentiation compared to clear phenotypic areas [4, 5]. We showed that ccRCC with eosinophilic phenotypes assessed by both a pathologist and the AI model significantly correlated with adverse pathological factors such as higher WHO/ISUP grades, vascularity-based architectural categories including sarcomatoid/rhabdoid components, and ccB gene signatures, consisting of genes related to regulating EMT.
We demonstrated that manually assessed histological phenotypes and the AI model predicted phenotypes were stratified with patients’ outcomes (p < 0001 and p = 0.008, respectively). However, the predictive ability of manually assessed phenotypes outperformed the ability of the AI model. We found some discrepancies between the assessments by the pathologist and our AI model (Supplementary Figs. 2A-2E). We found that small-sized clear neoplastic cells with pale reticular pink cytoplasm were sometimes identified as an eosinophilic phenotype, resulting in overestimating the clear phenotype. (Supplementary Fig. 2A) and the AI model included various components of the tumor microenvironment such as tumor vasculature, stromal, and tumor-associated immune cells into tumor components (Supplementary Fig. 2B). We also discovered that when the class activation mapping of AI scores corresponded to the H&E images, discordances between manually assessed histological phenotypes and AI scores were observed because of the defined cut-off value (Supplementary Figs. 2C-2E). Unlike the quantitative AI scores, clear, mixed, and eosinophilic phenotypes were assessed according to the WHO/ISUP grading and is determined based on the highest grade within a single high-power field of the tumor [15].
Our study had some limitations. First, we manually selected the highest-grade area for transfer learning according to the previously described method [4]. Second, we applied the AI model to the highest-grade area, not whole-slide images. Third, the cut-off of low or high AI scores was defined clear or mixed/eosinophilic phenotypes assessed by a pathologist. Especially, tumors with mixed phenotypes included various quantities of eosinophilic phenotypic areas tended to show discordances between manual and automatic methods. Fourth, the validation set did not include prognostic information because this study set was for confirming the correlation between histological features and gene signatures. Fifth, gene sets included in the custom gene panel examined in the present study were used in our previous studies [4, 14]. Lastly, we did not apply our AI model to treatment responsiveness. Despite these limitations, we showed meaningful correlations among our AI model, histological features, gene signatures, and patients’ outcomes. Although trained deep learning algorithms, referred to as black-box algorithms [30], cannot be easily explained, larger and more complex neural networks should be employed in future studies to solve the discordances between the pathologist’s assessment and the predictions from the AI model.
In conclusion, we successfully developed an AI model for predicting clear or eosinophilic histological phenotypes, which correlated with clinicopathological characteristics, gene signatures, and prognosis.