Experiment 1: DCIS vs. normal
Distinguishing non-invasive cancer from normal mammary glands
First, we attempted to distinguish non-invasive cancer and normal mammary glands to test the AI-based diagnosis system. The differential diagnosis between the normal mammary gland and non-invasive cancer / DCIS is essential, but not difficult for pathologists. Therefore, to distinguish them is indispensable for our system. The CNN analysis successfully distinguished DCIS from normal mammary glands (Fig. 2, AUC score: 0.9902, precision: 0.935, and recall: 0.953). These results suited this system's requirement, and thus, we applied the analysis to all other benign preneoplastic and neoplastic lesions.
Experiment 2: DCIS vs. comedo DCIS
Distinguishing DCIS and comedo DCIS pattern
In case of DCIS lesions, the comedo DCIS pattern that has necrosis at the center of DCIS has a more aggressive phenotype. Comedo DCIS pattern is a risk factor for recurrence post breast-conserving surgery [23,24], and is also correlated with the expression of poor prognostic markers [25,26]. The comedo DCIS pattern is rare and usually mixed with other DCIS lesions, leading to its omission from screening. Therefore, accurately identifying the comedo DCIS pattern is vital for pathological analysis. The AUC score in the CNN analysis was very high (0.9942). Moreover, the precision and recall were 0.972 and 0.954, respectively (Fig. 3).
Experiment 3: CCLs vs. normal
Distinguishing CCLs and normal mammary glands
CCLs include columnar cell hyperplasia and FEA. FEA has a neoplastic gene alteration than that in normal mammary glands [27-29]. Additionally, CCLs, including columnar cell hyperplasia, have similar neoplastic genetic alterations [30,31]. At present, CCL cases need to be followed up closely [32-34]. Therefore, distinguishing CCLs from normal mammary glands is important for breast biopsy tissue. Thus, we distinguished CCLs from the normal mammary glands using our system. The AUC was 0.9786, and for normal vs. CCLs, the precision and recall were 0.935 and 0.953, respectively (Fig. 4).
Experiment 4: DCIS vs UDH
Distinguishing hyperplastic lesion from DCIS
Next, we focused on the hyperplastic lesions in the mammary glands. Usual ductal hyperplasia (UDH) is a common lesion in the mammary tissue. However, dense proliferation of the mammary gland epithelium resembles DCIS, and additional immunohistochemical examination is needed for the differential diagnosis between DCIS and UDH [35,36]. To simplify this differential diagnosis process, we attempted to distinguish between DCIS and UDH using the machine learning system. The morphological structure was very similar between DCIS and UDH (Fig. 5), but the AUC score was 1.000, and the precision and recall were 1.000 and 1.000, respectively (Fig. 5).
Experiment 5: Distinguishing all 5 lesions
Finally, we attempted to simultaneously distinguish all five lesions (normal mammary glands, CCL, UDH, DCIS, and comedo DCIS) to test our system's applicability for supporting daily pathological diagnoses. The average precision and recall between the five lesions were 0.923 (0.863-0.973) and 0.927 (0.880-0.991), respectively (Fig. 6). Thus, these results are similar to those of individual comparisons performed in experiments 1-4.
Experiment 6:
Feedback from machine learning-based analysis to the practical microscopic findings
From these results, we speculated that the machine learning-based evaluation criteria/process of each lesion may differ from that of human pathologists, especially from the UDH vs. DCIS analysis. Therefore, we used Gradient-weighted Class Activation Mapping (Grad-CAM) data, to visualize the important regions reflecting results of the CNN analysis and identify newer morphological characteristics to support the pathological diagnosis. According to the Grad-CAM results, although the current pathological diagnosis only focuses on the morphology of the epithelial structure, the AI-analysis concentrated more on stromal tissue (Fig. 7). The stromal tissue ratio in the whole weighted area (red and yellow colored area) was very high in normal mammary glands than that in others (Fig. 7). The percentage of average epithelial lesions and stromal lesions were 70.5% and 29.5% in DCIS, 27.4% and 72.6% in normal, 59.3% and 40.7% in UDH, 57.8% and 42.2% in CCL, and 68.9% and 31.1% in comedo DCIS, respectively (Fig. 7). From the statistical analysis, the ratio of stromal tissue in the whole weighted area was significantly higher in the normal mammary glands than that in all other lesions (Fig. 7). Moreover, the stromal tissue ratio in the UDH and CCL was significantly higher than that in DCIS (Fig. 7).