Uveal melanoma (UM) and choroidal nevi are melanocytic choroidal tumors that share similar clinical characteristics, presenting a formidable challenge in differentiating them1,2. It is however imperative to distinguish these two lesions for appropriate management. UM is a malignant tumor originating from the melanocytes of the uveal tract and is the most common primary intraocular malignancy in adults3,4. Approximately half of all UM patients develop distant metastasis during the course of their disease progression5. Choroidal nevi, although benign in nature, may exhibit features resembling UM and can rarely undergo malignant transformation6. Erroneous classification of these benign tumors as malignant lesions can lead to unwarranted treatments, such as radiation therapy or enucleation. Conversely, the misdiagnosis of UM as a benign lesion may have serious consequences, including greater vision impairment, a higher risk of metastasis and death7.
Biopsy, followed by histological investigation is considered the gold standard for tumor diagnosis8,9. However, in the context of ocular tumors, this gold standard approach is not applicable due to the potential risk of unnecessary ocular biopsy for benign tumors10,11. As a result, alternative diagnostic methods and imaging modalities are favored to avoid the risks associated with invasive procedures and to ensure the safety and integrity of the ocular structures. Prominent among these are fundus photography, fluorescein angiography (FA), optical coherence tomography (OCT), and ultrasonography (US). These imaging techniques are widely employed in the diagnosis and differentiation of UM and choroidal nevi, as they offer valuable insights into the anatomical and clinical characteristics of these ocular lesions. These methods, however, are not widely available at all eye clinics and often require an evaluation by an ocular oncologist to make a conclusive diagnosis. Moreover, the overlapping characteristics between these tumors exacerbate the diagnostic complexity, leading to inconclusive results and delays in appropriate management decisions.
Given these challenges, machine learning and deep learning offer promising solutions for automating the detection and classification of these tumors. These techniques have been successfully implemented in ophthalmology for the automated diagnosis of age-related macular degeneration (AMD)12,13, diabetic retinopathy (DR)14–16, sickle cell retinopathy17,18 and glaucoma19–21. However, there is limited utilization of machine learning and deep learning in the diagnosis of ocular tumors like UM and choroidal nevi22. Large amounts of data are often required for the development of accurate and reliable machine learning and deep learning models for automated disease diagnosis. However, due to the relative rarity of ocular tumors, gathering sufficient data for this purpose becomes challenging. This scarcity of available data could be a contributing factor to the limited research on this subject matter. Nonetheless, the promise shown by machine learning and deep learning techniques in other ophthalmic applications sparks considerable potential for diagnosis and classification in ocular oncology.
One approach that holds promise in overcoming the challenge of limited data is transfer learning. Transfer learning is a training method that leverages the pre-trained weights of a convolutional neural network (CNN) on a large and diverse dataset and fine-tunes specific layers to adapt them for a new and more specific task. By utilizing the knowledge learned from the initial training on the broader dataset, transfer learning aims to optimize the weights of the CNN for the targeted task, enabling the model to excel in specialized applications, even with limited amounts of data available. This technique has proven to be highly effective in various applications in ophthalmology23,24. Some prior studies have also explored the combination of transfer learning with data fusion in ophthalmology16,25–27.
Data fusion is a powerful strategy widely employed to enhance the performance of deep learning models. This technique involves integrating data from multiple sources or modalities and has applications in several fields. Notably, in ophthalmology, data fusion has been employed both using the same imaging modality 16,27,28 or different imaging modalities25,26 for the automated diagnosis of retinal diseases. By fusing diverse data streams, the deep learning model gains access to a broader spectrum of information, allowing it to capture intricate patterns and correlations that may be challenging to discern using individual sources. There are three main strategies employed in the fusion of data for deep learning applications: early, intermediate, and late. In the early fusion strategy, the data is combined before any processing is done while in the intermediate fusion strategy, representations of the individual data are merged at an intermediate stage of the network. Late fusion, also known as decision-level fusion, combines predictions from individual modalities at the final layer of the network.
In this study, we explored the feasibility of employing a transfer learning approach along with data fusion strategies for automated classification of UM and choroidal nevi. To enhance the accuracy of classification, we incorporated early, intermediate, and late fusion strategies on distinct color channel images. This approach aimed to harness the unique information offered by each color channel – with red potentially providing crucial tumor-related information, while blue and green channels potentially contributing insights into features such as drusen, orange pigment and subretinal fluid. Our objective was to determine the most effective color fusion strategy for accurately differentiating UM and choroidal nevi.