A novel smartphone App to support the clinical practice of pediatric ophthalmology and strabismus: the validation of visual acuity tests

This study is aimed at describing the TreC Oculistica novel smartphone App that facilitated the clinical practice of pediatric ophthalmology and strabismus during the COVID-19 pandemic and at reporting on the validation of visual acuity tests in a home setting. The Trec Oculistica smartphone App was prescribed to eligible patients at the Pediatric Ophthalmology and Strabismus Clinic, Ophthalmology Unit of Rovereto Hospital, between September 2020 and March 2022. Four key indicators were identified for monitoring visual and visuo-motor functions remotely: visual acuity, ocular motility, head posture, and color vision. Clinicians selected few mobile applications (iOS, Android) and printable materials within the Trec Oculistica App: the Snellen Chart Visual Acuity App, the 9Gaze App, the eyeTilt App, the Color Blind test App, the LEA Symbols pdf, and the Snellen Chart pdf. All patients, aged 4 and older, were screened at home for visual acuity at 3 m and later in the clinic (LEA Symbols cabinet or Snellen computerized optotype). The 9Gaze, the eyeTilt, and the Color Blind test Apps were only recommended to a subset of patients based on clinical suspicion or diagnosis. The Wilcoxon signed rank sum test and weighted Cohen’s kappa coefficient were applied to compare pairs of scores from different settings. The Trec Oculistica App was downloaded and activated by 97 patients or their caregiver. 40 patients were tested at home using the 9Gaze App, 7 used the eyeTilt App, and 11 used the Color-Blind test App. Families reported that all the Apps were easy and intuitive to use; clinicians reported that measurements were reliable. 82 eyes of 41 patients (mean age 5.2 years, SD ± 0.4, range 4.4–6.1) were tested for visual acuity using the self-administered LEA Symbols pdf. 92 eyes of 46 patients (mean age 11.6 years, SD ± 5.2, range 6–35) were evaluated using the self-administered Snellen Chart Visual Acuity App or the Snellen Chart pdf. Home median visual acuity score was statistically different from that registered in clinical setting for both the LEA Symbols pdf (P-value = 0.0074) and the Snellen Chart App and pdf (P-value = 0.0001). The strength of agreement was 0.12 (slight) for the LEA Symbols pdf, 0.50 (moderate) for the Snellen Chart Visual Acuity App, and 0.69 (substantial) for the Snellen Chart pdf. Conclusion: The novel TreC Oculistica smartphone App was a useful tool for facilitating the clinical practice of pediatric ophthalmology and strabismus during the COVID-19 pandemic. In the follow-up of strabismus patients and patients with suspected inherited retinal diseases, the 9Gaze, eyeTilt, and Color Blind test applications were deemed to be intuitive and easy to use by families and were considered reliable by clinicians. In a home setting, visual acuity tested by means of Snellen Charts was moderately congruent with the in-office examination. On the contrary, agreement was poor in younger children tested with the LEA Symbols pdf. What is Known: • Teleophthalmology enables clinicians to evaluate patients’ ocular diseases remotely and various tools are helpful for screening, follow-ups and treatment. • Smartphones can currently be used to obtain ocular images and vision measurements of patients’ eyes and this information can be shared with the ophthalmologist for further evaluations and medical management (mhealth). What is New: • Smartphone Apps can be successfully used in a hybrid teleophthalmology service concerning first visits and follow-ups. • Apps and printable materials are easy, intuitive to use for patients and also reliable for clinicians. What is Known: • Teleophthalmology enables clinicians to evaluate patients’ ocular diseases remotely and various tools are helpful for screening, follow-ups and treatment. • Smartphones can currently be used to obtain ocular images and vision measurements of patients’ eyes and this information can be shared with the ophthalmologist for further evaluations and medical management (mhealth). What is New: • Smartphone Apps can be successfully used in a hybrid teleophthalmology service concerning first visits and follow-ups. • Apps and printable materials are easy, intuitive to use for patients and also reliable for clinicians.


Introduction
Starting from March 2020, the American Academy of Ophthalmology (AAO) published updated recommendations for reducing the risk of SARS-CoV-2 transmission in clinics, including a questionnaire to identify patients with possible exposure to SARS-CoV-2, guidelines on providing elective ophthalmic care (for example, expanded use of telehealth services), protocols when scheduling or seeing patients, and environmental cleaning and disinfection recommendations [1].Healthcare systems around the world have been forced to rapidly include tele-health and digital innovations in daily activities.
Teleophthalmology enables clinicians to evaluate patients' ocular diseases remotely, thereby producing positive effects, such as facilitating a more efficient and equitable distribution of limited healthcare resources and triaging prior to patients' arrival at hospital to avoid unnecessary visits and COVID 19 exposure risks.Video consultations in combination with innovative digital solutions can even improve patients' monitoring, avoiding unnecessary recourse to the clinic and maximizing the quality of consultations [2].The majority of teleophthalmology services adopt asynchronous methods (store-and-forward of images or measures), while others use a hybrid of real-time and store-and-forward methods and fewer still use synchronous methods (video conferencing) [3,4].There are various tools that can be used in teleophthalmology for screening, following, and treating patients with different ocular diseases.Smartphones and medical applications (Apps) spread rapidly in both developed and developing countries, in rural and remote areas as well as in central clinics.
This study is aimed at describing the TreC Oculistica novel smartphone App that facilitated the clinical practice of pediatric ophthalmology and strabismus during the COVID-19 pandemic and at reporting on the validation of visual acuity tests in a home setting.

Setting/venue
This study was performed in line with the principles of the Declaration of Helsinki.Informed consent was obtained from all individual adult participants (or children's parents/guardians) included in the study.
Trec Oculistica is a teleophthalmology system created to cope with the COVID-19 pandemic [5].It is aimed at promoting a cooperative patient-doctor relationship.Designed by a multidisciplinary team, it consists of a clinician web interface (clinical dashboard) and a patient mobile application (App).Clinicians can prescribe to outpatients with the App (Online Resource 1) and assign them some preliminary measurements to be self-collected before the televisit.The App conveys the clinician's requests (i.e., measurements) and eases the share of the collected information in a secure digital environment, promoting a new health care workflow.In the time window between prescription of measurements and actual televisit, patients can communicate with the healthcare staff through a dedicated chat.
The initiative has been created within the Competence Center on Digital Health TrentinoSalute4.0,involving the local healthcare trust Azienda Provinciale per i Servizi Sanitari (APSS), the local Government Provincia Autonoma di Trento (PAT), and the research institute Fondazione Bruno Kessler (FBK).The main objective was to empower patients and involve them in self-collecting relevant eyesight information to share with clinicians by means of digital material and smartphone applications made available by the dedicated TreC Oculistica App.In this innovative protocol, in person-visits occurred only when needed, while follow-up visits and periodic checks could be performed remotely, allowing a better use of the physical hospital spaces while saving time and money for patients, particularly those living in decentralized areas.
The Trec Oculistica smartphone App was prescribed to eligible patients at the Pediatric Ophthalmology and Strabismus Clinic, Ophthalmology Unit of Rovereto Hospital, between September 2020 and March 2022.The inclusion criteria were as follows: to have a smartphone and be able to download free Apps.The exclusion criteria were as follows: 4 years of age or younger and uncooperative patients.

Material selection
Four key indicators (investigated in most of the ophthalmological visits, both in first visits and follow-ups, and for both pediatric and adult patients) were identified for the monitoring of visual and visuo-motor functions: visual acuity, ocular motility, head posture, and color vision.These functions could be easily tested at home by the patient, with the support of a family member (caregiver), using either digital applications or printable materials.Clinicians identified a set of mobile applications and printable materials.The selection of the digital Apps was based on the following criteria: free, availability for both Android and iOS, more widespread use, standardization of the execution, and medical App.Similarly, the criteria for the printable materials were as follows: free, more widespread use, and standardization of the execution.Written tutorials were created for each tool, either App or printable material, providing a detailed explanation of the correct use, guiding in the acquisition of the required measurements.

Apps
Visual Acuity Charts App, Fonlow (iOS and Android) (Fig. 1) (Online Resource 2) This App displays optically precise optotypes of eye charts to measure distant visual acuity: supported and chosen optotypes-Snellen for adults and older children; chosen test distance-3 m; suggested notation-decimal; test modes-manual or autoplay [6].
9Gaze App, See Vision LLC (iOS and Android) (Online Resource 3) A quick way to document eye motility and strabismus in the nine cardinal positions of gaze.The App provides an on-screen guide for eye positioning to assist with image consistency through all nine photographs.9Gaze App automatically creates a composite image of the nine photographs for easy exporting [7,8].
eyeTilt App, See Vision LLC (iOS and Android) (Fig. 2) (Online Resource 4) It is a digital goniometer to measure head tilts and turns.
It is used for a quick way to determine how many degrees someone's head is tilted or turned.
The following are the features: on-screen guide for placement of eyes and/or nose, on-screen display of degree of tilt or of turn, and having no need to take any pictures.The degree of tilt and turn is displayed in real time.

Colorblind Eye Exam Test App, Yohann Taieb (iOS), App Color
Blind test, healthcare4mobile (Android) (Fig. 3) (Online Resource 5) It is a color blindness test to find out color deficiency, especially in the red and green tones.
Color blindness tests are mostly applicable to: -Color vision tests at various institutions.
-Testing children or adults at home to identify color deficiency or color blindness.

Printable materials
LEA Symbols pdf, LEA-Test Ltd. (Fig. 4) The   The Wilcoxon signed rank sum test and weighted Cohen's kappa coefficient were applied to compare pairs of scores from different settings.

Results
The Trec Oculistica App was downloaded and activated by 97 patients (40 males and 57 females) or their caregiver.Their complaints were visual acuity defect, strabismus, abnormal head posture, impaired color vision, and drooping eyelid.The final diagnoses were strabismus (esotropia, exotropia, hypetropia: 24 patients), strabismus post-surgery (16 patients), refractive errors (25 patients), amblyopia (16 patients), retinitis pigmentosa (2 patients), cone dysfunction syndrome (1 patient), optic disc drusen (1 patient), congenital cataracts (2 patients), congenital ptosis (1 patient), and normal eye examination (9 patients).40 patients were tested at home using the 9Gaze App, 7 used the eyeTilt App, and 11 patients used the Color Blind test App.Families reported that all the tested Apps were easy and intuitive to use; clinicians reported that measurements were reliable.82 eyes of 41 patients (mean age 5.2 years, SD ± 0.4, range 4.4-6.1)were tested for visual acuity using the selfadministered LEA Symbols pdf; 92 eyes of 46 patients (mean age 11.6 years, SD ± 5.2, range 6-35) were evaluated using the self-administered Snellen Chart Visual Acuity App or the Snellen Chart pdf.
Home median visual acuity score was statistically different from that registered in clinical settings for both the LEA Symbols pdf (P-value 0.0074) and the Snellen Chart App and pdf (P-value 0.0001).
The strength of agreement was 0.12 (slight) for the LEA Symbols pdf, 0.50 (moderate) for the Snellen Chart Visual Acuity App, and 0.69 (substantial) for the Snellen Chart pdf (Table 1).

Discussion
Smartphones can currently be used to obtain ocular images and vision measurements of patients' eyes, and this information can be shared with the ophthalmologist for further evaluations and medical management (mhealth).In recent years, the number of mobile applications, both targeting lay people and clinicians, increased, while alternative patient-clinician communication channels also spread.In such a new setting, technology can be used to acquire and share information and empower patients, involving them in a cooperative interaction with the clinician.Mobile technologies also provide a new framework for the digital connectivity of ophthalmic diagnostic devices for eye-care professionals, supporting real time decision-making and streamlining diagnostic processes [12,13].9 Gaze, eyeTilt, and Color Blind Test Apps were successfully used in our hybrid (real-time and store-andforward) teleophthalmology service concerning first visits and follow-ups (for example: strabismus post-op).Applications and printable materials were considered easy and intuitive to use for parents and adult patients; clinicians evaluated the outputs provided through the App as reliable.
To our knowledge, this is the first attempt to validate visual acuity tests in a home setting and within a teleophthalmology service.
In older children, the Snellen Charts (App and pdf) scores were moderately congruent with the in-office visual acuity tests, but the agreement was substantial for the Snellen Charts pdf due to a better standardization and performance in cooperative children.On the contrary, agreement was lower in younger children tested with the LEA Symbols pdf.In a home setting, a high number of false positives was related to children's poor cooperation with parents; a few false negatives were associated to a wrong standardization of the test and dense amblyopia in one eye.Tutorials were reported as extremely useful for understanding and correctly performing the specific test.In a follow-up qualitative survey, families reported that they would have preferred video tutorials over written tutorials.
The study has been conducted on a limited number of patients that accepted to download the Trec Oculistica App.The novelty of digital health solutions might have impacted the availability of specific subset of patients in approaching this new opportunity.Among the families that did not want to try the Trec Oculistica App, two groups emerged: (1) families that found the protocol complicated and time-consuming and (2) families that doubted on their child's collaboration.
In conclusion, the novel TreC Oculistica smartphone App was a useful tool for facilitating the clinical practice of pediatric ophthalmology and strabismus during the COVID-19 pandemic.In the follow-up of strabismus patients and patients with suspected inherited retinal diseases, the 9Gaze, eyeTilt, and Color Blind test Apps were deemed easy to use and intuitive by the families and reliable by clinicians.In a home setting, visual acuity tested by means of Snellen Charts was moderately congruent with the in-office examination.By contrast, agreement was poor in younger children tested with the LEA Symbols pdf.
Further evaluations, analyses, and experiences are needed to provide a solid methodological framework, both to widen the clinical fields of use in teleophthalmology and to prove the reliability of self-administered tests involving patients in the management of their own health.
World Health Organization (WHO) and the National Academy of Sciences Committee on Vision have made recommendations about the optotype choice and the arrangement on visual acuity testing charts.Optotypes should be clear and standardized, with similar characteristics (e.g. each line should contain five optotypes), and should not reflect a cultural bias.Spacing between the optotypes should be proportional: the horizontal spacing between individual optotypes should be equal to the size of the optotype, and the vertical spacing between lines should be the height of the optotypes in the lower line.Optotype sizes should generally be presented in 0.1 logMAR decrements.This arrangement leads to an inverted pyramid design for wall charts.Visual acuity testing charts used with children that meet these recommendations include LEA Symbols.Test distance is 3 m [9, 10].

Table 1
Statistical analysis applied to compare pairs of scores of visual acuity from different settings The Wilcoxon signed rank sum test is the non-parametric version of a paired samples t-test (P-values ≤ 0.05 indicate significant differences between medians) a Kappa values ≤ 0 indicate no agreement, 0.01-0.20 none to slight agreement, 0.21-0.40fair agreement, 0.41-0.60moderate agreement, 0.61-0.80substantial agreement, and 0.81-1.00almost perfect agreement *