The Impact of Delivering High-Quality Cataract Surgical Mentorship Through Distance Wet Laboratory Courses on Cataract Surgical Competency of Second and Final Year Residents.

Background: This study aimed to assess the acceptability and effectiveness of training second and nal-year residents, at the Regional Institute of Ophthalmology, a tertiary-level ophthalmic training center in Trujillo, Peru, in phacoemulsication cataract surgery through structured distance surgical mentorship wet lab courses. Methods: Delivered three ve-week distance surgical mentorship wet lab courses, administered through Cybersight, Orbis International’s telemedicine platform. Weekly lectures and demonstrations addressed specic steps in phacoemulsication surgery. Each lecture had two accompanying wet lab assignments, which residents completed and recorded in their institution’s wet lab and uploaded to Cybersight for grading. Competency was assessed through the anonymous grading of pre- and post-training surgical simulation videos, masked as to which videos were recorded before and after training, using a standardized competency rubric adapted from the International Council of Ophthalmology’s Ophthalmology Surgical Competency Assessment Rubric (ICO-OSCAR). Day one best-corrected post-operative visual acuity (BVCA) was assessed in the operative eye on the initial consecutive 4-6 surgeries conducted by the residents. An anonymous satisfaction survey was administered to trainees’ post-course. Results: In total, 21 second and nal-year residents participated in the courses, submitting a total of 210 surgical videos. Trainees’ average competency score (scale of 0-32) increased 6.95 (95%CI [4.28, 9.62], SD=5.01, p<0.0001, two sample t-test) from 19.3 (pre-training, 95%CI [17.2, 21.5], SD=4.04) to 26.3 (post-training, 95%CI [24.2, 28.3], SD=3.93). Among 100 post-training resident surgeries, visual acuity for 92 (92%) was ≥ 20/60, meeting the World Health Organization’s criterion for good cataract surgical quality. Conclusions: Structured distance wet lab courses in phacoemulsication resulted in signicantly improved cataract surgical skills. This model could be applicable to locations where there are obstacles to traditional in-person wet lab training

growing and aging population in many high-income countries is growing faster than the number of ophthalmologists. These aging populations will put pressure on current ophthalmic service providers globally, resulting in challenges in training enough doctors in the coming years. 1 While training is crucial to meeting the growing need for surgical services in eye care, [vi] the majority of residents in many settings, including those at high volume and well-staffed teaching hospitals, are not receiving adequate hands-on surgical training to ensure competency in cataract surgery, [vii] with some receiving none at all. [viii] , [ix] One approach to improved surgical skills and the delivery of hands-on training has been surgical simulation training via a wet/dry lab. Surgical simulation can reduce the learning curve of di cult surgical techniques, accelerate the rate for trainees to achieve surgical competency, and improve patient safety.[x] However, there are challenges to the delivery of surgical simulation training reported by both low/middle-income and high-income countries alike, [xi] , [xii] including the absence of structured simulation training programs and the lack of trained instructors or personnel. [xiii] The current paper reports on a novel and scalable model for cataract surgical training: distance wet labs. Data are presented on masked, anonymous assessments of surgical videos before and after training, to determine whether this approach could lead to objectively measurable improvements in surgical competency.

Methods
The La Libertad region of Peru is the third most populous in the country, with 1,617,050 inhabitants, 50% of whom live in the capital city of Trujillo. The majority of heath care workers in the Peru Northern zone (population approximately 4.5 million) are based in Trujillo.
[i] The regional population is aging rapidly due to a reduction in the fertility rate and rising life expectancy, [ii] increasing the number at risk for cataract blindness. Some 58% of the estimated 150,000 blind persons in Peru have lost sight from unoperated cataract, and cataract surgical coverage is only 63%. [iii] In 2017, Orbis International partnered with the Instituto Regional De Oftalmología Javier Servat Univazo (IRO) in Trujillo to pilot new methods of delivering cataract surgical training. IRO is staffed by 25 ophthalmologists and 17 residents, serves as the main eye care referral center for La Libertad and the Northern Peru Zone and is the second largest training institute in ophthalmology in the country which accepts international residents from throughout Latin America. In 2016, the hospital had 135,969 outpatient visits, and performed 6,841 surgeries, of which, 2,254 were cataract surgeries.
[iv] IRO was also the rst teaching hospital in Latin America to receive the International Council of Ophthalmology Accreditation as an "Advanced Surgical Training Program." Despite the fact that IRO is a busy hospital with a large faculty, residents were not receiving adequate cataract surgical training in phacoemulsi cation. Some of the reasons shared by the faculty were: high clinic volume keeping both faculty and residents occupied, the majority of senior faculty are unavailable in the afternoons, when they work in their private sector to supplement low government wages (~$1,000/month) and therefore not readily available for training, and residents only start practicing phacoemulsi cation in their nal year, limiting the number of cases they receive. These same issues are seen in other large hospitals, where consultants invest minimal time in training residents and busy clinics keep residents occupied, leaving little additional time for wet lab training. [v] As part of this partnership, Orbis and IRO began delivering structured distance wet lab courses in phacoemulsi cation for residents to improve their cataract surgical skills. Orbis partnered with Emory University in Atlanta, USA, and IRO to deliver three distance wet lab surgical courses targeting residents in their second or nal years of training. The rst course was held in January-February 2017 for ve residents, the second course in October-November 2017 for 7 residents and the third course in October 2018 for 9 residents. A structured course was developed to mimic the in-person training delivered to Emory ophthalmology residents. The training was administered by a mentor from Emory Eye Center (YK) and a course director at IRO (RA), and was delivered through Cybersight, Orbis' telemedicine and learning platform, which is hosted on a mobile responsive learning management system (LMS). The LMS allows for curriculum creation, interactivity between learners and mentors, task assignment, and the grading and tracking of learners' progress and results. Ethics approval was obtained from the Institutional Review Board at IRO, written consent was obtained from the leadership of IRO and the tenets of the Declaration of Helsinki were followed throughout.
The residency program at IRO introduces phacoemulsi cation surgery in the nal year of training.
Second year residents practice manual small incision cataract surgery or extracapsular cataract extraction. Therefore, nal year residents were selected for the rst and second distance wet lab courses.
For the third course, second year residents were also included, as IRO faculty decided they wanted residents to participate in the course twice, once in the second and once in their nal year of residency. All residents were new to phacoemulsi cation surgery when enrolled in the course, having no prior experience performing such cases in the operating room.
Prior to the course, an orientation was given to residents to familiarize them with Cybersight, including information on registering for and accessing the course; recording and uploading assignments; initiating online communication with the mentor; and obtaining other available open source educational materials. Remote testing of the internet speed and bandwidth in Trujillo was completed by the information technology staff at IRO and Orbis. A minimum Internet connection of 3Mbps upload and download is desirable for real-time video and uploading recorded wet lab video assignments, similar to the requirements for watching YouTube videos. However, it is possible to use systems that can adapt to lower bandwidth/slower internet connections, which Orbis has deployed in other project locations.
The course was designed to run for ve weeks, with each week addressing speci c, sequential steps in phacoemulsi cation surgery ( Table 1). The mentor delivered weekly lectures and demonstrations by live video conference. Conferences were interactive, with opportunities for questions, quizzing of students and repetition of demonstrations when requested. Each lecture had two accompanying wet lab assignments, for a total of 10 assignments. Residents had four days to practice, complete and record their assignments on video in the IRO wet lab. Practice was self-paced and determined by the resident. Assignments were conducted on either animal eyes, Kitaro® Wet Lab kit (FCI Ophthalmics, Marsh eld Hills, MA., USA) simulation eyes or Philips Studio arti cial eyes (Bristol, UK). With the rst course, sessions 1 and 2 were performed with animal eyes and the remaining three sessions with model eyes. In the second and third course, all sessions were done with model eyes with the exception of session one. The OSSCAR (Table 2) has 16 steps, compared to the ICO-OSCAR, which has 20 steps. Steps not appropriate for or considered too advanced for the wet lab setting were removed, including step 1 on draping, step 15 on minimizing eye rolling and corneal distortion, step 18 on intraocular spatial awareness and step 19 on iris protection. Other steps were combined, such as step 7 and 8 on use of the phacoemulsi cation probe and second instrument; and some steps were broken down into two parts, such as step 11 on chopping and phacoemulsi cation of the nucleus. All 16 steps of the OSSCAR were covered in the course. The OSSCAR has three possible grades (0-2) for each surgical step, corresponding respectively to Novice, Advanced Beginner and Competent (total grading scale 0 -32). The ICO-OSCAR by comparison has four grades (2-5), corresponding to Novice, Beginner, Advanced Beginner and Competent (total grading scale 0 -100). A passing score of 1 on the 0-2-point scale ("Advanced Beginner") for each surgical step was necessary to continue to the next step; anything less required the resident to redo and resubmit the assignment. Residents could access their grades and communicate directly with the course mentor via the Cybersight platform, receiving unstructured, informal feedback in an ongoing manner for each assignment and on their overall performance.
Upon completion of the course, the course director and residency director received a comprehensive digital report generated by Cybersight on each resident's weekly performance, including their grades for each assignment. Residents who successfully completed the course received a certi cate of completion, automatically generated by Cybersight.
Orbis tracked the surgical outcomes of rst consecutive 4-6 phacoemulsi cation surgeries conducted by participating residents after they completed the course. Orbis also retrospectively collected the surgical outcomes for the rst 4-6 surgeries performed by senior year residents in the two years prior (2015-2016) to the rst distance wet lab surgical course, to measure the training impact on residents' surgical outcomes Nurse technicians measured and recorded the best-corrected day one post-operative visual acuity in the operative eye. We did not exclude results of patients with systemic or ocular comorbidities but took the rst cases in sequential order for each resident performing cataract surgery using phacoemulsi cation after training. This re ects practice patterns at IRO where residents perform many surgeries on cases of patients with comorbidities.
Also, a self -administered course satisfaction survey developed by the investigators was shared with all resident participants to capture their feedback on course content, format, acceptability and selfassessment of improvements in their skill. Learners rated their level of agreement on a 5-point Likert scale, with 1 representing strongly disagree and 5 representing strongly agree. The survey was administered anonymously, using the Typeform platform (Typeform, Barcelona, Spain).
In addition, adjustments were made following the piloting of the rst course, and the second and third courses (which enrolled 16 residents) included two further assignments, asking residents to upload three pre and three post-training cataract surgical simulation videos made using model eyes and covering all steps of the OSSCAR. Pre-course videos were uploaded within the week prior to beginning the course and post-course videos within one week after course completion. These videos were sent for anonymous grading using the OSSCAR competency rubric. The grader (RJ), an experienced ophthalmic surgeon, was masked as to which videos were recorded before and after training.

Statistical methods
The difference between pre and post-course video overall grading was analyzed using the two-tailed paired sample t-test. To evaluate the impact of the training course on speci c surgical steps assessed using OSSCAR, the Wilcoxon signed-rank tests were performed for each of the 16 MSIC surgical steps pre and post course. The training impact on the quality of surgical outcomes pre and post course introduction, was analysed using a two-proportion test. All statistical analyses were done in R version 3.6.1 (2019-07-05) (The R Foundation; Vienna, Austria).

Results
We did not nd signi cant differences in the performance of second-and third-year residents participating in the remote wet labs and as such, have combined their results. In total, 21 residents participating in the two courses, submitted a total of 210 surgical videos. A score of ≥1 was achieved on 205 (97.6%) assignments on the rst submission, and 21 certi cates of completion (100%) were awarded. A post-course satisfaction survey reported that 100% of residents would recommend the training to others, and all con rmed a desire to participate in similar future distance surgical mentorship wet labs.
Visual acuity was measured for 100 post-training resident phacoemulsi cation surgeries (one secondyear resident has not yet performed phacoemulsi cation cases due to being on an outreach rotation), of which 92 (92%) had day one post-operative best corrected VA ≥20/60, meeting the WHO criterion for cataract surgical quality.
We analyzed patient outcomes of the rst phacoemulsi cation surgeries (N = 44) performed by residents (N=9) in 2015 and 2016, prior to when the distance surgical wet lab training began and found that 75% (33) had a day one post-operative BCVA ≥20/60. This represents a 22.7% relative improvement in surgical outcomes, (an absolute increase of 17% (95% CI 1.5% -32.5%, P=0.012, two-proportions test)) for second and nal year residents having taken the course over nal year residents in the previous two years.
Sixteen residents each uploaded three pre and three post-training videos for anonymous grading following the second and third course, for a total of 96 videos. Among these, 95 (99.0%) were graded, due to the low image quality of one pre-course video. Tele-education or distance learning is widely used in academic training today. By 2006-07, 61% of US higher education institutions offered some form of online courses. 10 It is likely that this number has continued to rise, as online education provides greater cost-e ciency and allows access to learners in remote locations. 10 There is evidence of better performance among students who engage in ipped classrooms or blended-learning, de ned as approaches combining in-person and online activities. A group of medical students at Zhongshan Ophthalmic Center who received online lectures and resources during their ophthalmology clerkships performed better on subsequent examinations and reported improved motivation and understanding of the material in comparison to a group who attended traditional classroom-based lectures. 11 The University of Miami Miller School of Medicine integrated ipped courses for students of ophthalmology, after which students performed better in examinations than in four previous semesters where typical teaching methods were applied.[i] Additionally, ophthalmologists recruited in Brazil, Mexico and the Philippines who were trained in the diagnosis of retinopathy of prematurity through a structured tele-education course showed signi cantly-improved diagnostic accuracy and reliability, and reported a preference for tele-education to traditional teaching methods. 12 Wet lab education is also a common teaching method and found in many current ophthalmic residency programs. An ophthalmologist experiences the highest complication rates during the rst 60 -80 surgical cases, 13,14,22 while evidence demonstrates that structured cataract surgical simulation courses can reduce the complication rates in these initial operations. 16 Simulation training of micro-surgical skills has shown improved performance outcomes.
[ii] Training using model eyes has shown improvements in both the accuracy and speed for surgically naïve residents in performing capsulorhexis and corneal sutures[iii] and improvements in corneal suturing were also found with experienced cornea surgeons. [iv] Combining these two effective teaching models by delivering wet lab training through distance learning, the program described in the current paper was able to realize improved performance outcomes.
Despite the effectiveness of simulation training, ophthalmic residents in low and middle-income countries, as well as high-income countries have inadequate simulation surgical courses.
[v] , [vi] Some of the reasons reported were lack of structured or poorly organized simulation training programs; lack of simulation facilities; lack of trained instructors or personnel, disinterest in training residents from consultants, particularly for residents assigned to peripheral hospitals rather than teaching hospitals for clinical practice; high volume of residents due to government mandates, and outpacing the faculty available for teaching. [vii] The model presented in this paper suggests that this approach is an effective way to improve the cataract surgical skills of trainees. Further, this model could provide a viable alternative method for delivering wet lab cataract surgical training in locations where some of the reported challenges to wet lab training, such as structured programs, limited faculty, and lack of access to training institutions, undermine the delivery of in-person structured wet lab courses.
Another consideration is the recent Corona virus disease (COVID-19) pandemic, which resulted in many ophthalmology departments closing or severely reducing their patient volume. This coupled with the need to practice social distancing, left many residents with limited access to traditional in-person hands-on training and clinical learning opportunities, essentially disrupting surgical training at a global level. [viii] During COVID-19, many medical training institutions were forced to rethink how they delivered education, and explore the use of virtual teaching, distance learning and simulation, to meet the training needs of their learners.
[ix] Distance wet lab courses also offer the opportunity to provide continuity of training, respecting the need for social distancing, during disruptive events such as COVID-19. The University of California, San Francisco recently delivered a distance wet lab in corneal suturing, in which faculty and residents reported the session equally and/or more effective than previous in-person wet lab training in corneal suturing. Continuous Curvilinear Capsulorhexis (CCC), as the most complicated stages, which also had the lowest completion rates (66.7% and 74.4% respectively). Followed by, irrigation/aspiration and IOL insertion, with completion rates of 78.4%, and 83.1%, respectively. Similar results have been reported from mentors (evaluated) and trainees (self-assessed/reported), with phacoemulsi cation and CCC consistently reported as the most challenging steps to learn.
[xii] ,[xiii], [xiv] Our analysis showed that residents most signi cantly improved in lens insertion, rotation and nal position of IOL, irrigation and aspiration technique with adequate removal of cortex and capsulorhexis-formation and circular completion. Five additional steps showed signi cant improvements (p<0.05) including nucleus cracking or chopping. This indicates that the distance wet lab training aligns to improvements in intermediate to challenging surgical steps.

Conclusions
Positive user feedback and the signi cant improvement in surgical performance based on masked assessment of videos by an experienced grader (RJ) suggest that distance wet lab mentorship courses can be an effective training tool in this setting. Further work is needed in other settings to con rm these results.

Declarations
Ethics approval and consent to participate: Ethics approval was obtained from the Institutional Review Board at IRO, written consent was obtained from the leadership of IRO and the tenets of the Declaration of Helsinki were followed throughout. Consent for publication: All participants consented to the research study and publication of research results.