AI-Driven Cervical Cancer Cytological Diagnosis Solution based on Large Scale Data Collections and Annotations: A Multi-centre Clinical Validation

doi:10.21203/rs.3.rs-3225479/v1

Download PDF

Article

AI-Driven Cervical Cancer Cytological Diagnosis Solution based on Large Scale Data Collections and Annotations: A Multi-centre Clinical Validation

https://doi.org/10.21203/rs.3.rs-3225479/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Cervical cancer is a major health concern for women worldwide, and cervical cytology screening is a widely used and effective technique for early detection. In this study, we built a large-scale database of digital WSIs from 49 hospitals in China, comprising of 76,614 WSIs with 3,435,463 cell-level annotations by 26 cytopathologists using manual and semi-automatic approaches. A novel AI diagnostic system called CCA-DIAG was developed for cervical cancer screening based on a hybrid machine learning framework, which is capable of efficient WSI-level classification for various sedimentations. Our results of multi-center validation show that the system can make classifications at the WSI-level with high sensitivity (ASCUS+:0.89, LSIL+:0.99) for diverse sedimentations and significantly improve the time efficiency of cytopathologists by approximately 4 times. These findings suggest that CCA-DIAG is a promising tool for cervical cancer screening and could potentially improve diagnosis accuracy and efficiency in clinical practice.

Biological sciences/Computational biology and bioinformatics/Image processing

Biological sciences/Computational biology and bioinformatics/Machine learning

Biological sciences/Computational biology and bioinformatics/Computational models

Biological sciences/Biological techniques/Bioinformatics

Biological sciences/Cancer/Cancer screening

Cervical Cancer Screening

Artificial Intelligence

Deep Learning

Convolutional Neural Network

Whole Slide Imaging

Multicenter clinical validation.

Cervical cancer (CCa) is the fourth most common cancer in women worldwide¹. In 2020, there were about 604,127 women diagnosed with CCa worldwide, and 341,831 died of the disease². CCa has a long period of precancerous lesions, namely cervical intraepithelial neoplasia, which forms invasive carcinoma when the lesions break through the subepithelial basement membrane and invade the mesenchymal layer ³. It is a highly preventable disease if patients can be stratified during precancerous lesions through regular screening and given proper treatment ⁴. Cervical cytology is the most widely used and an effective and well-accepted non-invasive technique for CCa screening⁵. However, the accuracy of cervical cytology diagnosis is highly dependent on the training, experience, and attentiveness of cytopathologists. Large amount of cytology slides needs to be assessed to detect potential malignancy. One single liquid-based cytology slide often contains tens of thousands of cells, and the cytopathologists need to examine large areas on the slides and make accurate diagnoses. Cytopathologists are usually overloaded and the reading and diagnostic time for each slide is limited given a large number of daily cases. This reality may lead to result in misdiagnosis ^{6, 7}. An automatic diagnostic system is desired to assist cytopathologists in improving the accuracy and efficiency of CCa screening and avoid preventable morbidity and mortality.

With the rapid development of digital Whole Slide Imaging (WSI) and Artificial Intelligence (AI), computer aided diagnosis (CAD) has been applied to cervical cytology and achieved promising results^8–12. The common pipelines for automated analysis of cervical cell often include cell detection and segmentation, followed by abnormality classification^13–16. Traditional Machine Learning (ML) methods focus on the segmentation of nuclei^{17, 18}, cytoplasm or clumps and classifying the cell types using hand-crafted features¹⁹, including Support Vector Machine (SVM)²⁰, Artificial Neural Networks (ANN)²¹and Unsupervised Learning Classification methods^{22, 23}. The efficacy of conventional ML methods is contingent upon various factors, such as the segmentation techniques, parameter optimization, the quality of dataset, the extracted and selected features, etc. Nevertheless, despite above considerations, these techniques have demonstrated remarkable results, particularly when working with limited data.

Deep Learning (DL) typically necessitates a substantial amount of training data to achieve optimal outcomes, yet it can overcome the challenges encountered by conventional ML. Despite the advantages of DL method, several challenges remain unsolved. One of such challenges is the requirement of enormous amounts of high-quality annotated data for the development of models, which is crucial for attaining high model performance ²⁴. A few databases are publicly available, including Herlev²⁵, ISBI14 ²⁶, ISBI15 ²⁷, and CERVIX93 ²⁸, which often have limited sample sizes for large-scale validation ²⁹. For instance, the Herlev benchmark dataset contains 917 cells, of which 675 are abnormal and 242 are normal. Additionally, manual annotation of large-scale datasets is prohibitively expensive as cytopathologists need to dedicate significant amounts of time and effort to annotation task in addition to their heavy daily clinical workload.

In practical applications, the cervical cytology AI system needs to provide a WSI-level classification that considers all abnormal cells across the entire slide. Existing methods based on Convolutional Neural Networks (CNNs) have primarily focused on identifying local lesion cells³⁰, thus the challenge of WSI-level analysis has not been adequately addressed^{31, 32}. One of the key challenges in implementing AI-based CCa screening solutions is the generalization of the model in real-world scenario. The potential variations coming from sample preparation, staining and image scanning quality are key considerations of model generalization. For example, sample preparation methods, such as membrane-based sedimentation, natural sedimentation and centrifugal sedimentation, have be reported to impact on diagnostic model performance³³. Developing a robust CCa screening model with improved generalization capability requires a large-scale database that accounts for the aforementioned natural variations across multiple hospitals³³.

Although AI/DL models have shown promising results in CCa and other cancer screening/diagnosis^34–36, the adoption of AI-based screening solution in hospital is still poor. Several bottlenecks hinder the implementation of the AI-based CCa screening solution in the hospitals. One such bottleneck is the lack of high-quality database collected from diverse regions with real-world variations. The datasets collected from previous studies are limited to a few hospitals, which can lead to poorer performance of developed AI models in other clinical centres. To further address the above hurdles, we collected a large number of specimens from different geological regions and hospitals. Based on our large-scale database and annotations, we have developed an AI-assisted CCa screening system, named CCA-DIAG. An overview of our databased and proposed AI-assisted system is depicted in Fig. 1, which comprises abnormal cell detection, ASCUS nucleus-cell segmentation, cell classification, and WSI classification. We posit that the integration of DL and ML techniques in distinct scenarios will facilitate the extraction of thorough and profound features, resulting in outstanding performance. Ultimately, we validated our proposed AI-assisted system initially on an in-house dataset, followed by multi-centre validation in 14 clinical centres located in diverse regions of China, via a large-scale retrospective and prospective study.

Large geographical scale specimen collection, slides preparation and image acquisition from different geological multi-center for retrospective studies

To ensure the robustness of the WSI-level classification model and account for real world data variations, we collected a total of 174,748 specimens from 49 medical centres across six distinct regions, as illustrated in Fig. 1A. After applying quality control criteria, 172,665 slides were retained for further analysis, of which 76,614 were used to develop the AI system. These exclusion criteria included slides with faded/bleached smears that could not be accurately interpreted, slides with insufficient cells for interpretation, and slides that were excessively dirty. Supplementary Sections I-II provide additional explanations for the cell density detection and blur detection methods that applied to the exclusion criteria. In this work, each slide corresponds to one unique patient. Our data incorporated three common sedimentation methods (Fig. 1B), including natural sedimentation, membrane-based sedimentation and centrifugal sedimentation. We have also implemented manual and semi-automatic annotation strategies as shown in Fig. 1D to expedite the annotation of extensive datasets and minimize the workload of cytopathologists. This led to a total of 3,435,463 cell-level annotations shown Fig. 1C. All slides were scanned at 20X magnification (0.5µm/pixel) using 46 high performance slide scanners, such as KF-PRO-400 & KF-PRO-005 from Ningbo Konfoong Biotech International Co., Ltd.

Overview of AI-assisted CCA-DIAG system design and its optimization

The proposed AI-assisted CCA-DIAG system consists of four modules as illustrated in Fig. 1E: (1) Cell detection model to identify the locations and categories of abnormal cells; (2) Cell classification model to improve the accuracy of true positives and minimize false positives for various classifications of the abnormal cells detected in (1); (3) ASCUS nucleus-cell segmentation model to segment ASCUS cells returned by the abnormal classification model; (4) XGBoost classifiers³⁷ to classify the WSI into different categories according to cell-level features extracted from the three above mentioned models. The categories on WSI-level includes six types of Squamous intraepithelial lesions: NLIM, ASCUS, LSIL, ASCH, HSIL, AGC, and two types of microbial infections: Trichomonad and Candida. Specially, for squamous intraepithelial lesions, ASCUS, LSIL, ASCH, HSIL and AGC were grouped as ASCUS+, while LSIL, ASCH, HSIL, AGC were grouped as LSIL+. The WSIs dataset were assigned in a 8:2 ratio to the cell-level analysis dataset, and the WSI classification dataset based on the clinical diagnostic categories. The number of annotations used for the development of different models were illustrated in Fig. 1C.

Semi-automatic annotation: Cytopathologist-AI interaction greatly increase the efficiency of large-scale annotations

The development of CCa screening system often requires a large amount of high-quality annotated data. To accelerate data curation, we developed an annotation platform for cytology and designed a two-phase annotation implementation as shown in Fig. 1D. The details of our two-phase annotation strategy are given in Fig. 2.

In phase 1, a preliminary abnormal cell detection model and a preliminary ASCUS nucleus segmentation was developed based on 37, 942 WSIs with manual annotations of 375, 055 abnormal cells and 3,271 nucleus of ASCUS single cells. In phase 2, by applying the two preliminary models to another 24, 339 WSIs exhaustively using sliding window approach, a large amount of pseudo annotations, including abnormal cells of different categories and nucleus of ASCUS cells were generated, which were subsequently reviewed and confirmed by 19 cytopathologists before including them to enlarge the training dataset. Additionally, a new category, NILM (including changes in flora suggesting bacterial vaginosis, Actinomyces, Herpes simplex virus, cytomegalovirus), was added and annotated during semi-automatic annotation.

In total, 1,223,374 abnormal cells of different categories and 97,221 nuclei of ASCUS were returned by preliminary abnormal cell detection model and ASCUS nucleus segmentation model respectively. After cytopathologists’ review, 31, 826 nuclei of ASCUS were confirmed as being correct. Details of semi-automatic annotation results of abnormal cells are shown in Supplementary Section VIII. Many of the abnormal cells identified by the preliminary abnormal cell detection model were corrected as false positive (NILM), demonstrating the necessity of developing an abnormal cell classification model which further classifies the abnormal cells to eliminate the potential false positive. As for annotation efficiency of cytopathologists, by comparing the total amount of annotation together with mean of annotation time of fully manual annotation and semi-automatic annotation respectively, we found that semi-automatic is approximately 2 times much faster, as shown in Fig. 3A.These findings indicate the semi-automatic annotations assisted by the preliminary models can generate large amount of high-quality annotated data while easing the annotation burden of cytopathologists significantly.

Abnormal Cell detection model optimization and model performance improvement by copy-paste method

The preliminary models of abnormal cell detection were trained on a fully manual annotations in phase 1 and were applied to generate larger number of annotations for further training in phase 2. For the preliminary abnormal cell detection model, we evaluated the average precision (AP) values for each category in the test set and 3 different backbone structures (resnet 50, resnet 101 and Swin-transformer-base ) were compared, as shown in Fig. 3B The optimal backbone network model is Swin-transformer-base, which reaches highest AP value in all categories (AP of ACUS = 0.527, ASCH = 0.408, LSIL = 0.564, HSIL = 0.439, AGC = 0.610, Trichomonas = 0.481, Candida = 0.663) and mean average precision (mAP) at 0.503 compared to other backbone network models (mAP of resnet 50 = 0.397, resnet 101 = 0.408). Therefore, it was selected as the preliminary model of abnormal cell detection to be used for semi-automatic annotation in phase 2.

By applying the copy-paste method using the abnormal cells obtained from semi-automatic annotation, a final abnormal cell detection was trained on a larger set of abnormal cells using the same structures of Swin-transformer-base. Results shows that, comparing to the performance of preliminary model, the AP scores of all categories increased (AP of ACUS = 0.556, ASCH = 0.508, LSIL = 0.571, HSIL = 0.571, AGC = 0.689, Trichomonas = 0.593, Candida = 0.663) and mAP at 0.593 after applying copy-paste method (Enhanced Swin-transformer-base indicated in red line) in phase 2, as shown in Fig. 3B. We also performed binary detection for evaluation of abnormal cells in each class. Figure 3C illustrates the AUC obtained from the ROC curve for abnormal cell detection using the enhanced Swin-transformer-base (using the copy-paste method) as the overall result of discriminative performance showed to be excellent (AUC > 0.8).

Abnormal cell classification model can effectively eliminate the false positive cells

The abnormal cell classification model was developed using the annotated abnormal cell obtained from semi-automatic annotation in phase 2. The main objectives of the abnormal cell classification model were to further classify the abnormal cell identified by abnormal cell detection model and eliminate the potential false positive. We performed F1-score evaluations comparing four different backbones (EfficientNet-B2, EfficientNet-B3, convnext tiny and convnext small) to understand the performance summary of precision and recall. As indicated by the results of the F1 value in Fig. 3D, convnext_small is the worst, while efficientnet-b2 performs slightly better than the others. Therefore, it is selected as the final model. Some representative identified abnormal cells are shown in Fig. 3E. Confusion matrix of abnormal cell classification in Fig. 3F shows that the abnormal cell classification model can identify positive cells effectively. We can observe that the classification ability of this model for relatively similar abnormal cell types is still good. For example, in the ASCUS cell group, compared with LSIL 13.90% and NILM 20.22%, the successful classification rate of ASCUS cells is 60.84%. In the LSIL cell group, compared with ASCUS 22.04%, the successful classification rate of LSIL cells was 64.78%. LSIL cells were easily distinguishable from NILM (9.44%) compared to ASCUS cells. The summary of the confusion matrix is that it is difficult for cytologists to distinguish cells such as ASCUS and NILM, and our model has a good ability to distinguish them.

ASCUS nucleus-cell segmentation: model performances are significantly increased after incorporating large amount of semi-automatic annotations

In phase 1, the preliminary nucleus segmentation model was developed on 3,271 ASCUS nucleus manually annotated by cytopathologists. In phase 2, annotations of 31,826 ASCUS cells and their nucleus were collected through semi-automatic annotations, Otsu segmentation and manual adjustment.

For preliminary nucleus segmentation model, the mean intersection over union (IOU) and Dice score of ASCUS nucleus is 0.85 and 0.92 respectively. In the final model, the IOU of nucleus increased to 0.94 and Dice score increased to 0.95, as shown in Fig. 4A. Comparing the performances of preliminary and final models, there were significant improvements after merging large amount of data obtained from semi-automatic annotation into training set. These results demonstrate the necessity and advantage of using larger data set for development of Deep Learning models. The IOU and Dice Score of ASCUS cell segmentation is 0.92 and 0.96 respectively. The results indicate that the final model can segment ASCUS cell and nucleus accurately. Examples of ASCUS cell and nucleus segmentation results are demonstrated in Fig. 4B. The final nucleus and cell segmentation model contributed to the copy-paste method used in the development of final abnormal cell detection model by segmenting the abnormal cells in the bounding box.

WSI-level classification using extracted features

Taken together, the three models provide comprehensive analysis of abnormality on cell level, which will contribute to the WSI-level classification. By extracting discriminative features on cell level, classifiers using different Machine Learning algorithms (mean AUC of Linear = 0.8, Support Vector Machine = 0.918, Random Forest = 0.857, Logistic regression = 0.688, XGboost = 0.945, and Catboost = 0.929) were trained for three types of WSIs (the natural sedimentation method, the membrane-based method and centrifugal sedimentation method). We evaluated and compared the performances of different models on differentiating ASCUS + vs. NLIM in test sets. As demonstrated in Fig. 4C, the XGBoost classifier reaches the highest AUC for all 3 types of WSIs (mean AUC is 0.945). Its AUC for WSIs prepared using the Natural Sedimentation method is 0.998, followed by 0.93 for Membrane-based Sedimentation method and 0.908 for Centrifugal sedimentation method, which shows high accuracy on screening of ASCUS + cases. The AUC of identifying ASCUS + and LSIL + using XGBoost is demonstrated in Fig. 4D. Considering that the identification of ASCUS might be ambiguous and vague in clinical practice, we excluded ASCUS from the dataset and assess the performance of LSIL + vs. NLIM only. For the Natural Sedimentation method, the AUC is 0.996, which is similar to the performance of classifying ASCUS+. However, compared to ASCUS + vs. NLIM, the AUC of LSIL + vs. NLIM classification on WSIs prepared using the Membrane-based method and Centrifugal sedimentation method is increased by 0.035 and 0.057, respectively. This finding suggests that for classification on WSI prepared by Natural Sedimentation method, using the proposed AI system to perform classification between ASCUS + and NLIM is preferred, while for the other two methods, performing LSIL + vs. NLIM classification is more secure. As for efficiency, the average processing speed of the complete workflow as demonstrated in Fig. 1D is 5.02–6.99 mm² /s, as shown in Fig. 4E.

Clinical Validation: Independent Test Samples from Xiangya Hospital

In the clinical validation of 2,059 cervical liquid-based cytology images at Xiangya Hospital, we first evaluate the model performance based on cytology results as the gold standard. The findings showed that the sensitivity of the AI-assisted diagnosis system was 78%, specificity was 69.81%, and accuracy rate was 73.68%. In lesion categories above ASC-US (ASC-US+), the sensitivity reached 95.08%, specificity was 69.81%, and accuracy rate was 77.89%. The AUC of AI system classifying LSIL + is 91.6% and ASCUS + is 82.1%, as shown in Fig. 5A. We observed that AI-assisted demonstrated the highest sensitivity at 97.6% compared to Cytologist (78.5%) and slightly increase the specificity from 83.4–84.1%, as in Fig. 5B. In addition, we also compared our results with histopathological results as the gold standard. From 2,059 cases, we finally obtained 1,088 cases for analysis by excluding cases with incomplete data. Compared with the gold standard of histopathological results, the sensitivity of cytologists to LSIL + was 78.45%, the specificity was 83.48%, the false negative rate was 21.55%, and the accuracy was 82.17%; the sensitivity of AI to LSIL + was 98.23%, and the specificity was 83.48%. The accuracy was 56.70%, the false negative rate was 1.76%, and the accuracy was 67.46%. The sensitivity of AI-assisted doctor diagnosis for LSIL + was 98.23%, the specificity was 84.10%, and the accuracy was 87.78% as illustrated in Fig. 5C. The results show a substantial increase in the percentage improvement with AI-assisted assessment as shown in Fig. 5D. It is noteworthy that the sensitivity of diagnosis increased by 19% after cytopathologists were assisted by the AI systems. Additionally, efficiency of different types of efficiency were also assessed in Xiangya hospital. Results show that AI-assistance diagnosis is about 3.69 times faster than manual examination, as demonstrated in Fig. 5E. These findings indicate that the CCA-DIAG model can not only improve the quality of diagnosis, but also significantly increase the efficiency, which will greatly ease the burden of cytopathologists.

Multi-center clinical validation: a retrospective study and a prospective study

In order to further validate the performance of the proposed CCA-DIAG AI models, a retrospective clinical validation was conducted using 8,438 samples collected from 10 hospitals in China, including Xiangya Hospital. The comparison between the WSI-level classification results of AI-assisted diagnosis and the ground truth was used to assess the model performance, as shown in Fig. 6A. The results showed that in all 10 centres, the sensitivity of LSIL + was nearly 100%, while the specificity (also known as the sensitivity of NILM) was above 60% regardless of the preparation method, as demonstrated in Fig. 6B. These findings suggest that the AI model detected almost all LSIL + cases and generated few false positives, indicating its ability to generalize to samples prepared in different hospitals. Additionally, the sensitivity of ASCUS + was above 0.9 in 5 centres, demonstrating the model's strong performance in detecting ASCUS+.

The prospective clinical validation of the proposed AI-assisted system was conducted in two hospitals over a period of 6 months, during which 106,439 samples were collected, as demonstrated in Fig. 6C. The assessment time of AI-assisted diagnosis were recorded in two hospitals, and it was found that there was no significant difference in assessment time between these hospitals. This result was comparable to the AI-assisted diagnosis in the prospective study, indicating the consistency of the efficiency of AI-assisted diagnosis, as shown in Fig. 6D. Quantitative analysis indicates a notable enhancement in the efficiency of AI-assisted assessment, with a 2.93X increase for MI1 (6 cytologists) and a 3.56X increase for MI2 (4 cytologists) compared to manual assessment under the microscope over a period of 6 months. This signifies a reduction in cytologists' workload, surpassing the World Health Organization's recommended limit of 100 assessments per day as illustrated in Fig. 6E. Furthermore, there is potential for improvement in clinical turnaround time.

This study introduces the CCA-DIAG system, which leverages deep learning and large-scale WSIs for effective cancer detection. A comprehensive database was constructed, comprising 76,614 patient-wise WSIs and cell-level 3,435,463 annotations from 49 hospitals across 6 regions of China. The proposed system was found to be highly sensitive and had a low false negative rate during clinical validations. The cell detection framework employed the Swin-transformer feature extraction method, which outperforms traditional convolutional neural networks in feature extraction. Additionally, various image enhancement techniques were implemented during training to improve the model generalization, and a cell nucleus segmentation model was used to investigate features of cell nucleus changes. For WSI-level classification, a combined classifier was designed and trained on large-scale annotated data. This architecture effectively distinguished precancerous lesions from microbial infections and enhanced the predictive power of WSI-level classification.

Cytology professionals are primarily concerned with avoiding missed positive cases when utilizing AI for screening. Therefore, it is crucial to achieve a balance between high sensitivity and specificity to facilitate efficient screening. Our clinical verification results indicate that our AI diagnostic system exhibits an overall specificity of 70% and a sensitivity of 100% for positive lesions such as ASCUS + in natural sedimentation, over 92.12% in membrane sedimentation, and over 90% in centrifugal sedimentation. Despite a specificity of 75% in natural sedimentation of positive lesions such as LSIL+, the sensitivity level remained at 100%, enabling the system to accurately identify all HSIL + lesions. Our prospective and retrospective studies also revealed consistent results with independent clinical verification of 106,439 and 8,438 cases, respectively.

Upon further examination, it was observed that the majority of false-negative cases belonged to the ASCUS category. However, it should be noted that ASCUS classification in cervical cytology diagnosis is characterized by low reproducibility and a lack of clear boundaries with negative cases. Additionally, a significant proportion of ASCUS cases have negative biopsy results³⁸. It is important to highlight that the lack of annotated data for normal cell morphology often leads to erroneous interpretations, resulting in false positive readings. For instance, some normal squamous cells with darker nuclear staining and impurities have been misinterpreted as lesions. Furthermore, normal cervical glandular epithelial cells share similarities with high nucleoplasmic ratio cells, deep-stained nuclei, and disturbed polarity of HSIL lesion cells at certain specific angles, making it challenging even for experienced cytopathologists to distinguish between them. To address this issue, our future annotation processes will incorporate detailed information on the various levels of normal squamous epithelial cells, diverse forms of cervical columnar epithelium, red blood cells, neutrophils, and impurities that may be present in normal samples. Additionally, glandular cell lesions pose significant diagnostic challenges in cervical cytology, and their occurrence is less than 1% of the screened population³⁹.

Our study demonstrates that the implementation of an AI system leads to a significant enhancement in the sensitivity of diagnoses made by cytopathologists. The CCA-DIAG system has been shown to be approximately 3.69 times faster than manual inspection and yields an average diagnosis time of 20 seconds. Importantly, our findings indicate that the AI-assisted diagnostic times are reproducible and consistent in independent samples of our prospective and retrospective studies. In addition, we validated our AI prediction result by comparison with the pathologist's manual diagnostic results. However, cervical cytology is a screening tool to assess the risk of cervical lesions, and the diagnostic results are limited by factors such as sampling, preparation techniques, the pathologist's experience, and the limitations of the test itself. Thus, the cytopathology findings do not fully and truly reflect the patient's lesions. Only histopathological findings are the gold standard for the extent of cervical lesions. To further validate the efficacy of our AI model, perhaps a direct comparison with histological results is warranted.

The development and progression of CCa is the result of a combination of factors, e.g., age, HPV infection status, sum of different risk factors or clinical profile of the patient. Improvement of the diagnostic accuracy of AI is possible with the integration of multi-modal information, such age, chief complaint, results of physical examination, ultrasound and imaging results, HPV tests, vaginal micro-ecology, colposcopy results, and pathological histology results with WSI annotation data^{40, 41}. Furthermore, as our data collection process progresses, we anticipate the opportunity to conduct more comprehensive studies and classify additional rare categories.

In this paper, our study successfully sourced images from multiple centres and built a large database of 3,435,462 high-quality annotations extracted from the WSI of 76,614 CCa patients. Based on this large-scale data, we have constructed a novel cervical cytology artificial intelligence screening system, which has been reviewed for efficient WSI grade classification and has been validated for a variety of preparation methods and scenarios. Finally, the preliminary results of this work based on the evaluation of multi-center clinical trials based on large databases also show that our cervical cytology artificial intelligence model is expected to become a diagnostic aid for cytopathologists and help accelerate and popularize CCa screening.

Ethics Approval and consent of participate

The experimental design and methods were performed in accordance with the relevant guidelines and regulations reviewed by the Ethics Committee of Xiangya Hospital, Central South University (IRB number 201611619).

Statistical analysis

A paired t-test was used to compare the speed performance of the assessment between the cytologist and the AI-assisted cytologist. All statistical analyses were performed in MATLAB 2022a (Mathworks Inc., USA) and its statistical toolbox.

Consent for Publication

All authors of this work agreed to publish with Nature Medicine once accepted.

Availability of the data and material

This paper was produced using no publicly available image data as it is constrained by personal information protection, patient privacy regulation, and medical institutional data regulatory policies, etc. The size of our research data is too big to be stored in public repositories. However, the authors have made every effort to make the available of these resources publicly available such as software methods and the supporting information to reproduce technical pipeline, analyses, and results. All data supporting the finding of this work are available unconditionally for accredited scientific researchers for the purpose of reproducing the results and/or further academic and AI related research activities from the primary corresponding author [email protected] upon request within 10 working days.

Disclosure/Competing Interest:

The authors declare no potential competing interests.

Authors Contribution

FZ, YZ, DP and WY contributed to the conceptualization of the study. WY, KO, KG and YZ supervised the research, contributed to project coordination and administration. WY, YZ, FZ and DP contributed to the development or design of methodology. FZ, PL, QW, KY, BC, WY and WC contributed to the data curation and investigation. LW and BH contributed to resources and supervision. WY, KO, QW and BC contributed to experiment design, development of abnormal cell detection model and integration of the AI system. QW contributed to the development of ASCUS nucleus-cell segmentation model. CH contributed to the development of abnormal cell classification model and WSI-level classifier. KO, XH and LL prepared the manuscript with the contribution of all other authors. ZY contributed to the preparation of supplementary. KO, HL, JJ, JZ, DP, SX and WY contributed to the review and revision of the manuscript..

CCa

Cervical cancer

CCA-DIAG

Cervical cancer Diagnosis System

WSI

Whole Slide Image

Digital Pathology

Artificial Intelligence

CNN

Convolutional Neural Network.

Machine Learning

Deep Learning

HPV

Human Papilloma Virus

Ethics Approval and consent of participate

Statistical analysis

Consent for Publication

All authors of this work agreed to publish with Nature Medicine once accepted.

Availability of the data and material

Disclosure/Competing Interest:

The authors declare no potential competing interests.

Authors Contribution

Funding

This research is funded by Xiangya Hospital of Central South University Clinical Research Grant (Grant No. 2016L03), and the General Program of the Natural Science Foundation of Hunan Province (Grant No. 2023JJ30969). This project is jointly supported by A*STAR BII research core funding and BMRC Applied and Translational Research (ATR) Grant.

Ginsburg, O. et al. The global burden of women's cancers: a grand challenge in global health. Lancet 389, 847–860 (2017).
Sung, H. et al. Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries. CA Cancer J Clin 71, 209–249 (2021).
Ostor, A.G. Natural history of cervical intraepithelial neoplasia: a critical review. Int J Gynecol Pathol 12, 186–192 (1993).
Pimple, S.A. & Mishra, G.A. Global strategies for cervical cancer prevention and screening. Minerva Ginecol 71, 313–320 (2019).
Marth, C. et al. Cervical cancer: ESMO Clinical Practice Guidelines for diagnosis, treatment and follow-up. Ann Oncol 28, iv72-iv83 (2017).
Nanda, K. et al. Accuracy of the Papanicolaou test in screening for and follow-up of cervical cytologic abnormalities: a systematic review. Ann Intern Med 132, 810–819 (2000).
Fahey, M.T., Irwig, L. & Macaskill, P. Meta-analysis of Pap test accuracy. Am J Epidemiol 141, 680–689 (1995).
Chankong, T., Theera-Umpon, N. & Auephanwiriyakul, S. Automatic cervical cell segmentation and classification in Pap smears. Comput Methods Programs Biomed 113, 539–556 (2014).
Jusman, Y., Ng, S.C. & Abu Osman, N.A. Intelligent screening systems for cervical cancer. ScientificWorldJournal 2014, 810368 (2014).
Roberts, J.M. et al. A three-armed trial of the ThinPrep Imaging System. Diagn Cytopathol 35, 96–102 (2007).
Halford, J.A. et al. Comparison of the sensitivity of conventional cytology and the ThinPrep Imaging System for 1,083 biopsy confirmed high-grade squamous lesions. Diagn Cytopathol 38, 318–326 (2010).
Kitchener, H.C. et al. MAVARIC - a comparison of automation-assisted and manual cervical screening: a randomised controlled trial. Health Technol Assess 15, iii-iv, ix-xi, 1-170 (2011).
A, M.E.P., A, C.N. & B, A.C. Combining shape, texture and intensity features for cell nuclei extraction in Pap smear images. Pattern Recognition Letters 32, 838–853 (2011).
Tareef, A. et al. Multi-pass Fast Watershed for Accurate Segmentation of Overlapping Cervical Cells. IEEE Transactions on Medical Imaging, 1–1 (2018).
Zhao, L. et al. Automatic cytoplasm and nuclei segmentation for color cervical smear image using an efficient gap-search MRF. Computers in Biology and Medicine 71, 46–56 (2016).
Shinde, S., Kalbhor, M. & Wajire, P. DeepCyto: a hybrid framework for cervical cancer classification by using deep feature fusion of cytology images. Math Biosci Eng 19, 6415–6434 (2022).
Plissiti, M.E. & Nikou, C. Overlapping cell nuclei segmentation using a spatially adaptive active physical model. IEEE Trans Image Process 21, 4568–4580 (2012).
Jung, C. & Kim, C. Segmenting clustered nuclei using H-minima transform-based marker extraction and contour parameterization. IEEE Trans Biomed Eng 57, 2600–2604 (2010).
Chen, Y.F. et al. Semi-automatic segmentation and classification of Pap smear cells. IEEE J Biomed Health Inform 18, 94–108 (2014).
Mariarputham, E.J. & Stephen, A. Nominated texture based cervical cancer classification. Comput Math Methods Med 2015, 586928 (2015).
Mat-Isa, N.A., Mashor, M.Y. & Othman, N.H. An automated cervical pre-cancerous diagnostic system. Artif Intell Med 42, 1–11 (2008).
Marinakis, Y., Marinaki, M. & Dounias, G. Particle swarm optimization for pap-smear diagnosis. Expert Systems with Applications 35, 1645–1656 (2008).
Marinakis, Y., Dounias, G. & Jantzen, J. Pap smear diagnosis using a hybrid intelligent scheme focusing on genetic algorithm based feature selection and nearest neighbor classification. Comput Biol Med 39, 69–78 (2009).
Zhang, L. et al. DeepPap: Deep Convolutional Networks for Cervical Cell Classification. IEEE J Biomed Health Inform 21, 1633–1643 (2017).
Jantzen, J., Norup, J., Dounias, G. & Bjerregaard, B. Pap-smear Benchmark Data For Pattern Classification. nisis (2005).
Zhi, L., Carneiro, G. & Bradley, A.P. An improved joint optimization of multiple level set functions for the segmentation of overlapping cervical cells. IEEE Trans Image Process 24, 1261–1272 (2015).
Lu, Z. et al. Evaluation of Three Algorithms for the Segmentation of Overlapping Cervical Cells. IEEE J Biomed Health Inform 21, 441–450 (2017).
Phoulady, H.A. & Mouton, P.R. A New Cervical Cytology Dataset for Nucleus Detection and Image Classification (Cervix93) and Methods for Cervical Nucleus Detection. (2018).
Madabhushi, A. & Lee, G. Image analysis and machine learning in digital pathology: Challenges and opportunities. Med Image Anal 33, 170–175 (2016).
Lin, H., Hu, Y., Chen, S., Yao, J. & Zhang, L. Fine-Grained Classification of Cervical Cells Using Morphological and Appearance Based Convolutional Neural Networks. IEEE Access 7, 71541–71549 (2019).
Lin, H. et al. Dual-path network with synergistic grouping loss and evidence driven risk stratification for whole slide cervical image analysis. Med Image Anal 69, 101955 (2021).
Cheng, S. et al. Robust whole slide image analysis for cervical cancer screening using deep learning. Nat Commun 12, 5639 (2021).
Zhu, X. et al. Hybrid AI-assistive diagnostic model permits rapid TBS classification of cervical liquid-based thin-layer cell smears. Nat Commun 12, 3541 (2021).
Zhao, M., Wu, A., Song, J., Sun, X. & Dong, N. Automatic screening of cervical cells using block image processing. Biomed Eng Online 15, 14 (2016).
Esteva, A. et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature 542, 115–118 (2017).
Wang, W. & Wang, Y. Deep Learning-Based Modified YOLACT Algorithm on Magnetic Resonance Imaging Images for Screening Common and Difficult Samples of Breast Cancer. Diagnostics (Basel) 13 (2023).
Chen, T. & Guestrin, C. in Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining 785–794 (2016).
Pan, Q.J. et al. Pooled analysis of the performance of liquid-based cytology in population-based cervical cancer screening studies in China. Cancer Cytopathol 121, 473–482 (2013).
Wang, Z. et al. Diagnostic Yield and Performance of a Large Population-Based Cervical Cancer Screening Program in High-Risk Rural China. J Cancer 11, 4000–4006 (2020).
Tao, X., Huang, X., Kim, E., Long, L.R. & Antani, S. in Conference on computer-aided diagnosis (2015).
Song, D. et al. Multimodal entity coreference for cervical dysplasia diagnosis. IEEE Trans Med Imaging 34, 229–245 (2015).
Nayar, R. & Wilbur, D.C. The Bethesda system for reporting cervical cytology: definitions, criteria, and explanatory notes. (Springer, 2015).
Cai, Z. & Vasconcelos, N. in IEEE transactions on pattern analysis and machine intelligence, Vol. 43 1483–1498 (2019).
Chen, K., Wang, J., Pang, J., Cao, Y., Xiong, Y., Li, X., Sun, S., Feng, W., Liu, Z., Xu, J., Zhang, Z., Cheng, D., Zhu, C., Cheng, T., Zhao, Q., Li, B., Lu, X., Zhu, R., Wu, Y., Dai, J., Wang, J., Shi, J., Ouyang, W., Loy, C.C., & Lin, D. (arXiv, 2020).
Lin, H. et al. Dual-path network with synergistic grouping loss and evidence driven risk stratification for whole slide cervical image analysis. Medical Image Analysis 69, 101955 (2021).

There is NO Competing Interest.

Supplementaryfile20230731.docx

Download PDF

Version 1

posted

You are reading this latest preprint version

AI-Driven Cervical Cancer Cytological Diagnosis Solution based on Large Scale Data Collections and Annotations: A Multi-centre Clinical Validation

Status:

Version 1

Abstract

Figures

INTRODUCTION

RESULTS

Overview of AI-assisted CCA-DIAG system design and its optimization

Semi-automatic annotation: Cytopathologist-AI interaction greatly increase the efficiency of large-scale annotations

Abnormal Cell detection model optimization and model performance improvement by copy-paste method

Abnormal cell classification model can effectively eliminate the false positive cells

ASCUS nucleus-cell segmentation: model performances are significantly increased after incorporating large amount of semi-automatic annotations

WSI-level classification using extracted features

Clinical Validation: Independent Test Samples from Xiangya Hospital

Multi-center clinical validation: a retrospective study and a prospective study

DISCUSSION

Online Methods

Abbreviations

Declarations

References

Additional Declarations

Supplementary Files

Status:

Version 1