Inter-observer agreement among specialists in the diagnosis of Oral Potentially Malignant Disorders and Oral Cancer using Store-and-Forward technology

Oral Cancer is one of the most common causes of morbidity and mortality. Screening and mobile Health (mHealth) based approach facilitates remote early detection of Oral cancer in a resource-constrained settings. The emerging eHealth technology has aided specialist reach to rural areas enabling remote monitoring and triaging to downstage Oral cancer. Though the diagnostic accuracy of the remote specialist has been evaluated, there are no studies evaluating the consistency among the remote specialists, to the best of our knowledge. The purpose of the study was to evaluate the interobserver agreement between the specialists through telemedicine systems in real-world settings using store and forward technology. Two remote specialists independently diagnosed the clinical images from image repositories, and the diagnostic accuracy was compared with onsite specialist and histopathological diagnosis when available. Moderate agreement (k = 0.682) between two remote specialists and (k = 0.629) between the onsite specialist and two remote specialists in diagnosing oral lesions. The sensitivity and specificity of remote specialist 1 were 92.7% and 83.3%, whereas remote specialist 2 was 95.8% and 60%, respectively, compared to histopathology. The store and forward technology and telecare can be effective tools in triaging and surveillance of patients.


Introduction
Cancer of the lip and oral cavity is the 16th most common cancer worldwide, with over 377,713 new incident cases and 177,757 deaths annually. India is the second country with the highest number of oral cancer cases [1] . India accounts for about 100,000 incident cases accounting for nearly one-fourth of the overall burden, making oral cancer a leading cause of death among men [2,3] .
The stage of the disease at diagnosis is the prime determinant of the treatment outcome of the patients [4] . The 5-year survival rate for localized cancers is 54.3-60.2%, while it is as low as 3.1-3.3% in advanced stages [5] . Detection at an advanced stage lowers the chances of a cure, decreases the quality of life, and results in considerable cost to the patient [3] . In India, 70% of the cases are reported in the advanced stages (American Joint Committee on Cancer, Stage III-IV), with ve-year survival rates of around only 20% [2] . One important cause is the lack of access to an oral cancer specialist [5] . Moreover, early-stage cancers are often asymptomatic, decreasing the chances of the patients seeking the medical attention they need [3] .
Oral cancer is preceded by a group of visible mucosal lesions called Oral Potentially Malignant Disorders (OPMDs) that exhibit oral epithelial dysplasia [6] . Early detection and surveillance of such OPMDs have the potential of not only decreasing the incidence but also improving the survival of those who develop oral cancer. Conventional Screening provides an excellent opportunity for early detection and surveillance [3] .
Some of the drawbacks, however, are ine cient data management, poor follow-up after the screening, lack of knowledge and practice of dentists to recognize and diagnose OPMDs, and a breakdown or delay in communication with the specialist [7] . Hence, the continuous challenge of effective specialist-patient

Methodology
The prospective double-blinded study was conducted to evaluate the diagnostic e cacy of two remote specialists in diagnosing OPMDs and Oral cancer and also to determine the interobserver agreement of the two remote specialists in store and forward technology and mHealth telemedicine. The Institutional review board approved the study (KIDS/IEC/Nov-2018/18). The onsite specialist performed the conventional visual examination of the participants and provided a speci c diagnosis. The lesions were biopsied based on an onsite specialist recommendation. The FHPs collected the clinical images using a smart-phone based mHealth application. The deidenti ed image data was stored in the image repositories of KLE Institute of Dental Sciences, Bengaluru. These clinical images (n = 822) accrued from the image repository were evaluated by two remote specialists. The remote diagnosis was performed based on the image data without information on associated risk factors such as age, gender, habit history, and underlying systemic disease/ drug history. The remote specialists were blinded and independently provided a diagnosis for each image under four categories as follows-Diagnostic criteria and diagnosis: The diagnostic accuracy of the remote specialists was compared with the onsite specialist, who we have considered as the reference standard.
A total of (n = 102) biopsies were performed based on onsite specialist recommendation, which was considered the gold standard. The two onsite, remote specialists' diagnosis was compared with histopathology. The diagnostic accuracy in terms of sensitivity, speci city, positive predictive, and negative predictive values were determined, and the interobserver agreement was estimated using Cohen's kappa. Cohen suggested the Kappa result be interpreted as follows: values ≤ 0 as indicating no agreement and 0.01-0.20 as none to slight, 0.21-0.40 as fair, 0.41-0.60 as moderate, 0.61-0.80 as substantial, and 0.81-1.00 as almost perfect agreement [10] .

Results
A total of (n = 822) images were included in the study to evaluate the interobserver agreement between the two remote specialists. Based on the onsite specialist diagnosis who is considered the reference standard, category 1 included (n = 228), category 2 (n = 147), category 3 (n = 326), and category 4 (n = 121) images.
Concordance between the two remote specialists and the Onsite specialist- Table 1 shows a substantial agreement (k = 0.682) between the two remote specialists in the diagnosis of clinical images. A moderate agreement (k = 0.603) was observed between the onsite specialist and remote specialist 1 in diagnosing oral lesions ( Table 2) with sensitivity and speci city of 83.1% and 90.6% respectively. The sensitivity and speci city between the onsite specialist and remote specialist 2 was 79% and 82.6% respectively with moderate agreement (k = 0.605) in diagnosis of oral lesions as shown in Table 3. Table 4 shows the concordance between the onsite specialist and the two remote specialists in diagnosing oral lesions with a moderate agreement (k = 0.629). The diagnostic accuracy and concordance between the onsite specialist, remote Specialist 1, remote Specialist 2 are presented in Table 5.     Concordance between the Histopathology, two remote specialists, and Onsite specialist-A total of (n = 102) biopsies were obtained to compare the e cacy of the onsite specialist, remote specialist 1, and 2 with histopathology (the gold standard). Table 6 lists the diagnostic accuracy and concordance between histopathology, onsite, remote 1, and 2. The onsite specialist showed a sensitivity of 94.8% and speci city of 83.3% for diagnosing OPMDs and oral cancer with histopathology as the gold standard. remote specialist 1 showed sensitivity and speci city of 92.7% and 83.3%, in comparison, remote specialist 2 showed 95.8% sensitivity but a lower speci city of 60%.

Discussion
A sustainable technology-based practice in healthcare is ever impressive. One such concept of ideal technology is telemedicine. Not long ago, the global pandemic crisis highlighted the need for telecommunication technology for communication strategies between the doctor and the patient, which has further addressed the need to evaluate existing facilities and the reliability of telehealth in reducing cancer-speci c morbidity and mortality. The main aim of telehealth technology is to provide access to health care in rural areas where the need for the early diagnosis of OPMDs and oral cancer is high but has limited access to healthcare.
The mHealth-based remote oral cancer surveillance program adopted to aid in remote early detection of oral cancer by primary care dental practitioners in a resource-constrained setting has already been demonstrated [5] . Store and forward technology, could eliminate the need for an onsite consultation in 50% of cases. In two isolated studies, the agreement between the onsite and the remote specialist was 97% and 92.7% in identifying oral lesions. Following this, the remote specialists formulated the diagnosis and recommendation for referrals [5 , 9] . In another study, the remote specialists showed a sensitivity and speci city of 95% and 84% in diagnosing OPMD and oral cancer, respectively, when compared to the onsite specialist [11] .
In our study, we evaluated the accuracy and interobserver agreement between remote specialists involved in a store-and-forward technology and mHealth telemedicine technology. The remote diagnosis used images obtained by low-skilled FHWs in low-income and limited infrastructure settings. The data collected by the FHWs included patient demographics, medical history, habit history, and photographs of the oral lesions. The remote diagnosis was done based on the morphology of the lesions (unstructured data) without any clinical data or patient risk factors, to maintain the uniformity of the data input for both machine learning and remote specialist. However, in regular telemedicine module, the remote specialist has an information on structured data (patient demographic details that includes the de-identi ed data, and habit history) and unstructured data (clinical images).
A study compared the diagnostic accuracy of remote versus onsite specialist visits using a novel, lowcost telehealth platform consisting of a smartphone-based, remote intraoral camera and custom software application found that on-site diagnosis showed high sensitivity (94%) and 69.2% speci city when compared with histopathological diagnosis, which did not signi cantly differ from the accuracy of remote specialist (sensitivity: 94%; speci city: 62.5%) [12] . Our study showed a speci city of 83.3% by the remote specialist 1, but a 60% speci city by the remote specialist 2, the reason being more years of experience in telediagnosis for the former as compared to the latter.
The variability in the diagnosis provided by the remote specialists may also be due to misdiagnosis or overdiagnosis. The prede ned clinical data and images of the lesions collected at the remote location might not be su cient for the remote specialist to arrive at a diagnosis. In such a scenario, a diagnosis may be given based on perception and intuition rather than analytical thought, a human cognitive factor, which might enhance the incidence of misdiagnoses [13] . A lesion with typical clinical features such as oral leukoplakia or oral lichen planus is easier to identify in an image than a case of oral submucous brosis and other malignant lesions (such as salivary gland tumors), which requires a thorough physical examination by the onsite specialist. The low speci city of the remote specialist, when compared to the histologic con rmation, may be due to the overdiagnosis of the lesions. The remote specialists were calibrated to over-diagnose in regular work ow, and the remote specialist diagnosis was provisional. Without the usual inspection and palpation of the lesion, the remote diagnosis relies on the clinical data and morphology of the lesions. It has been shown that early borderline lesions may be categorized as malignant to avoid the consequences of the misdiagnosis of more aggressive cancers [14] . Though not all misdiagnoses result in harm, in reality, the malignant transformation of OPMDs and the aggressiveness of OSCCs is highly variable and unpredictable, and the relative contribution of overdiagnosis bias remains to be elucidated across populations [15] .
Assessing and addressing concerns with respect to mHealth and store and forward telemedicine is a critical step towards fully integrating telemedicine in everyday clinical and outreach programs. All of this work is done in collaboration with community healthcare workers and general dentists. The patients, specialist of the practitioner need not be available at the same time, hence this model is convenient and improves e ciency, reduces travel and waiting time of patients, second opinions quickly obtained, and specialists report quickly obtained are some of the bene ts of the store and forward technology. Limitations, such as the requirement of a good internet connection with both download and upload speeds at the remote location, overdiagnosis by the specialist, and privacy and security: protecting individual health records pose signi cant challenges [16] . The diagnosis of the remote specialists depends on the quality of the photographs captured and the data collected from the remote location. The oral cavity is more di cult to diagnose in a teleconsultation because they are more di cult to photograph. FHPs should be trained in identifying the lesion and ensure proper focus on the lesion to get sharper images for diagnosis [9] . When the quality of the image is nondiagnosable, smartphones with AI-driven applications may be deployed to alert the FHWs to retake the photograph. In this study, the FHWs were intimated by the remote specialists to retake the photograph. Store and forward technology is an asynchronous service model that relies on data and information sharing outside of real-time consultations.

Conclusion
Store and forward technology can be an effective tool in triaging patients, surveillance and can strengthen the healthcare system in low-and middle-income countries. Training of remote specialists is recommended for telediagnosis to improve their e ciency. Despite limitations, telemedicine allows specialized clinicians to reach out to a larger number of cases than the geographic distance would permit. It also improves the process by avoiding delays in diagnosing oral cancer. The image repository also serves as an important tool for documenting visual changes over time.   Photograph of malignant proliferative growth on left buccal mucosa