Training Visual Pattern Recognition in Ophthalmology Using a Perceptual and Adaptive Learning Module

Background: To evaluate the e�cacy of a perceptual and adaptive learning module (PALM) for teaching the identi�cation of �ve optic nerve �ndings. Methods: In this prospective trial at a single academic institution, second-through fourth-year medical students were randomized, by class, to the PALM or a video didactic lecture. The PALM presented the learner with a short classi�cation task consisting of an optic nerve image with all �ve possible �ndings. Learner accuracy and response time were used to guide the sequencing of successive classi�cation tasks. Mastery for each �nding was de�ned as three consecutive correct identi�cations within a target response time of eight seconds. The lecture intervention was a narrated video designed to simulate the related part of a traditional medical school didactic lecture. Accuracy and �uency on a pre-test, post-test, and delayed test (one month after the learning intervention) were compared within and between groups using the t-test. Cohen’s d was used to estimate effect size. Results: Eighty-three students participated. Accuracy and �uency improved signi�cantly (P < 0.001) from pre-test to immediate post-test for both the PALM (accuracy d = 2.94, �uency d = 3.39) and lecture (accuracy d = 2.32, �uency d = 1.06). At the delayed test, PALM performance remained signi�cantly (P < 0.001) greater than the pre-test in both accuracy (d = 0.89) and �uency (d = 1.16), while lecture performance remained greater in accuracy only (d = 0.44, P = 0.02). The PALM induced greater improvements than the lecture in both accuracy and �uency on the immediate post-test and delayed test. Conclusions: The PALM is an effective modality for training identi�cation of common optic nerve �ndings.


Background
Visual pattern recognition is central to ophthalmology; a substantial portion of ophthalmic diagnosis involves direct visualization of the affected tissue.2][3] Unfortunately, curricular time constraints in undergraduate medical training have marginalized ophthalmology teaching. 4,5 eover, the development of pattern recognition depends on non-declarative learning mechanisms that are not well-addressed by lectures and most other conventional teaching methods. 6Innovative learning strategies, including e-learning, have been proposed to increase meaningful exposure to ophthalmology. 7][10][11][12][13] Condensed immersion experiences involving brief lectures and active participation with ophthalmic examinations have similarly demonstrated e cient advances in knowledge and technical skills. 14,15 elf-directed, 16 team-based, 3 and problem-based 17,18 learning strategies have also shown e cacy in undergraduate ophthalmology education.
Perceptual and adaptive learning modules (PALMs) are a novel online application of advanced learning theories which are especially relevant to the instruction of visual content.PALMs require the learner to classify large numbers of unique exemplars into categories to facilitate perceptual learning, broadly de ned as experience-induced improvements in the extraction of information. 6,19 hey also use the learner's accuracy and uency (accuracy with little hesitation) to sequence and space subsequent category presentations, enhancing focus on areas of weaker learning (using adaptive response timebased sequencing, ARTS). 20Feedback provided after each classi cation task further strengthens learning.PALMs have been employed for improving expertise in several areas of medical teaching, 21 including dermatology, 22 histopathology, 23 transesophageal echocardiography (TEE), 24 and electrocardiography (ECG). 25We developed a PALM for teaching the discrimination of ve common optic nerve ndings (glaucomatous changes, swelling, pallor, myopia, and normal nerve) and examined its effectiveness, compared to a video didactic lecture, in promoting optic nerve diagnostic skills among medical students in a prospective randomized trial.We hypothesized that the PALM would result in greater improvements than the lecture in diagnostic accuracy and uency.To our knowledge, this is the rst publication of a PALM in ophthalmology.

Study Design
This was a prospective, double-masked, and randomized trial of second-through fourth-year medical students at a single academic institution.Approval from the University of California San Francisco (UCSF) Institutional Review Board and Committee for Human Subjects Research was obtained.The study was conducted in accordance with the tenets of the Declaration of Helsinki.First-year students were excluded to ensure all participants had a basic understanding of medical knowledge and anatomy.

Procedure
A recruitment email was sent to UCSF medical students in the second-, third-, and fourth-year classes.Once con dentiality was assured and electronic consent was obtained, participants enrolled via an online survey using their name, email address, and year in medical school.After a one-week enrollment period, participants were randomized, by class, to the PALM or lecture, and each was given a unique code for anonymization which they used for the study.Participants were masked to the intervention as the recruitment materials referred only to an online learning module for teaching diagnosis of optic nerve diseases.Both interventions used the same online site (med.insightlt.com).
A 10-day study period commenced in which participants completed a pre-intervention questionnaire, the assigned module (including pre-and post-tests), and a post-intervention questionnaire.Participants completed a delayed test one month after the start of the study.Those completing all elements of the study received a $25 electronic gift card.Those scoring in the top 10 percent (within each group) received an additional $15 card.These incentives were stated in the original recruitment email to encourage motivated performance.

PALM Intervention
A 50-second 'primer' video, consisting of an image of each diagnosis and textual description displayed for 10 seconds, preceded the PALM to expose participants to basic optic nerve anatomy and the key features for each disease category.For example, a single image of a swollen nerve was shown with the description 'blurred disc margins, hemorrhages, obscured blood vessels.'The PALM incorporated 28 unique images for each of the ve optic nerve categories (140 images total), obtained from the UCSF institutional image repository.Given the possibility for comorbid optic nerve pathologies, images were carefully selected by investigators (SR and DA) to ensure a single, unambiguous diagnosis for each image.
In the PALM, an image was shown with all ve possible diagnoses listed in a multiple-choice format.The learner selected a diagnosis and the correct choice was immediately indicated along with notation of the incorrect choice if one had been chosen.The response time (RT) was shown for correct answers.Examples of PALM trials and feedback are in Fig. 1.The PALM automatically scored the category as incorrect and transitioned to the feedback after 16 seconds without a response.
In PALMs, both learner accuracy and RT are used to estimate learning strength and guide the sequencing of subsequent category presentation.A minimum spacing number is also assigned so that subsequent exemplars of the same category are not presented too closely in sequence.For the current PALM, the minimum spacing was three intervening trials.When a category was identi ed incorrectly, a new exemplar of that category was presented following this minimum spacing.More intervening problems were presented before category repetition for accurately identi ed categories and shorter RTs.A target RT of eight seconds was assigned to denote a uent response for accurate answer choices.An RT of eight seconds or fewer was judged by expert opinion to be su cient for an experienced viewer to classify the optic nerve pathology in a photograph.Therefore, this threshold was used as a target RT below which a learner was considered uent for the classi cation task.Mastery of a category was de ned as uent responses on three consecutive presentations of that category.Mastered categories were retired from presentation; however, additional exemplars from those categories were used as llers (and not scored) once there were no longer three unmastered categories to ful ll the minimum spacing requirement.

Lecture Intervention
The lecture intervention was a 4.3-minute narrated video designed to simulate a traditional medical school didactic lecture.The lecture was written by authors TA, DA, and SR, and narrated by TA.The lecture introduced optic nerve anatomy and function and then elaborated on the visual features for each disease.For example, for a swollen optic nerve, a verbal description of common causes was followed by a characterization of visual features of disc margin blurring, blood vessel obscuration, dilated capillaries, and hemorrhages.Animated arrows highlighted each of these features.

Performance Assessments
Assessments were administered before the learning interventions (pre-test), immediately after the learning interventions (post-test), and one month after the learning interventions (delayed test).Images from each of the diagnostic categories were presented in a multiple-choice format with no feedback and no adaptive sequencing.Three different test versions were developed, each incorporating two unique images per category, none of the images having been included in the PALM or lecture.To normalize for possible differences in di culty between the three test versions, the versions were counterbalanced at each stage of testing.In addition, each participant received a different test version for their pre-test, posttest, and delayed test.

Experience Questionnaires
Pre-and post-intervention questionnaires surveyed participant con dence diagnosing optic nerve diseases and satisfaction with the learning module.Prior involvement in ophthalmology activities and intended residency specialization were also obtained.All survey items used a 5-point Likert scale.A freetext response solicited additional feedback.The surveys are included as Supplement 1.

Statistical Analysis
Data were analyzed using VassarStats.com.Statistical signi cance was set at P < 0.05 (two-tailed).The primary outcomes were changes in accuracy and uency from the pre-test to the immediate post-test and delayed test, within each group and between groups as assessed by paired and independent t-tests, respectively.Chi-square was used to compare participant characteristics, con dence, and satisfaction between groups, and the Bonferroni adjustment was applied for multiple comparisons.Cohen's d was used to estimate effect size. 26Inductive thematic analysis was applied to qualitative feedback.One investigator (TA) developed a preliminary codebook through immersion and crystallization, 27 then re ned the codebook into a list of themes used to formally code all responses.

Participants
Eighty-nine students completed the pre-intervention questionnaire (43 PALM and 46 lecture), 83 (89%) of whom completed all components of the study through the delayed test (40 PALM and 43 lecture).Most (83%) were beginning their second or third year of medical school.A minority (23%) had prior ophthalmology experience, most commonly clinical shadowing.Most (89%) felt uncon dent to very uncon dent diagnosing optic nerve diseases using images.Students nished the PALM in a median time of 8.5 minutes when including the 50 second primer video.) with persistently large effect sizes.Retention at one month for the lecture group was signi cant only for accuracy and with a moderate effect size (34 ± 2%, P = 0.020, d = 0.44).
The median duration of the PALM was 8.5 minutes (including the 50 second primer video).The lecture duration was 4.3 minutes.In order to examine whether a greater increase in performance could be observed for the PALM group after spending approximately 4.3 minutes on the PALM, we examined the PALM accuracy and uency data of this group for the 10 problems immediately following those encountered within the rst 4.3 minutes and compared it to the accuracy and uency for lecture participants on the 10-item post-test.Improvement in accuracy and uency for PALM participants on this 10-problem performance data remained signi cantly greater than those for lecture participants on the post-test, with large effect sizes (mean PALM 10-problem vs. lecture post-test accuracy: 70 ± 3% vs. 53 ± 2%, P < 0.0001, d = 0.99; uency: 65 ± 3% vs. 38 ± 2%, P < 0.0001, d = 1.42).

Participant Experience
Both interventions resulted in a statistically signi cant increase in participant con dence but the magnitude of the improvement was greater for the PALM group (P = 0.008), as shown in Fig. 3.The PALM outperformed the lecture in overall satisfaction (P = 0.007), effectiveness (P = 0.016), and enjoyability (P = 0.035), and stimulated greater interest in similar modules (P = 0.034), as shown in Fig. 4. Most PALM participants (78%) provided feedback on their experience.Participants described the PALM as a fun learning tool and appreciated its ability to support pattern recognition.Some PALM participants remarked that additional instruction on the pathophysiology of optic nerve ndings would have been helpful.Representative comments, organized by theme, are shown in Supplement 2.

Discussion
Expert clinicians often describe the evolution of their mastery as "experience." 23Indeed, medical training spans a decade or more, from medical school to residency and fellowship.However, expanding medical education requirements have marginalized ophthalmology teaching, necessitating more e cient ways of delivering educational content. 7Moreover, it is becoming increasingly clear that clinician "experience" relates to crucial but somewhat neglected aspects of learning.In particular, perceptual learning, an implicit and gradual learning process, advances via many classi cation episodes rather than through explicit verbal or procedural instruction, and produces pattern recognition and intuitive knowledge that is often not verbalizable. 6This study demonstrates signi cant improvements in diagnostic accuracy, uency, and con dence following a PALM for optic nerve disease pattern recognition that required, on average, eight minutes to complete.Learning was durable over time, despite the absence of further engagement with this material, and superior to an intervention modeled on a traditional lecture.This is the rst study to validate the e cacy of a PALM in ophthalmology and the rst in any medical specialty to compare the PALM to a more traditional, lecture-based learning intervention.
The rapid learning improvements observed in this study are consistent with those of PALMs in other medical specialties.A histopathology PALM for medical students requiring 11 to 17 minutes of engagement demonstrated large increases in accuracy and uency sustained at six weeks, with more e cient learning observed among students familiar with the PALM format. 23Similarly robust learning followed dermatology PALMs for medical students and ECG PALMs for medical students and emergency medicine residents, with an even longer follow-up time of 12 months. 22,25 mproved diagnostic expertise following PALMs for TEE 24 interpretation have also been shown among anesthesiology residents to be maintained over a six-month period, regardless of baseline pro ciency.This study is unique in that, unlike the subject matter in previous PALMs, ophthalmology is not a core component of the medical curriculum.
The optic nerve PALM resulted in substantial learning for medical students at all levels of training despite minimal to no prior exposure to such content.PALMs are likely applicable to a broad swathe of ophthalmic pathology, from retinal disease to cup-to-disc size estimation.
In contrast to the conventional "lecture and test" approach, 28 perceptual and adaptive techniques offer learners a more active learning experience.Learners practice using real-world images and performance is measured throughout -rather than at the end of -the learning process to provide feedback in realtime. 21While trainees are generally able to follow and easily recognize patterns highlighted in a presentation, they classically encounter di culty when a new image is encountered independently. 23ence, the development of expertise requires a re ned "information extraction system, not a storehouse of memorized instances." 29This system, which follows different laws of learning from declarative memory, requires exposure to many relevant examples. 23,30 ALMs allow visualization of the variations in presentation of many diseases that learners would otherwise encounter gradually over months to years of training, likely explaining the large and reliable improvements in diagnostic automaticity observed in this study and others.Additionally, frequent testing coupled with feedback, integrated into the learning process, have been shown to promote stronger, more durable learning, 31 even when the content is new and not previously studied. 32layed test scores declined substantially for both groups, with the PALM maintaining superiority by only a moderate margin, based on effect sizes, and the differences in both accuracy and uency hovering very close to the 0.05 cut-off for statistical signi cance.As the majority of medical students in this study were not immersed in clinical ophthalmology, they likely lacked the continued exposure required to consolidate and retain the learning.Active re-learning and spaced repetition, particularly in expanding time intervals, have been shown to improve long-term retention. 32,33 he practical application of this PALM would be among medical students or residents prior to or during a clinical rotation, wherein continued exposure to ophthalmic pathology would be expected to improve long-term retention.
Many participants in this study -in both intervention groups -appreciated either form of exposure to ophthalmology.Indeed, medical students and general medical practitioners perceive their undergraduate training in ophthalmology to be inadequate and desire more teaching. 34The widespread marginalization of ophthalmology teaching has rami cations even for students pursuing non-ophthalmic specialization.
Eye-related complaints comprise up to three percent of all primary care o ce visits, 35 a proportion certainly to increase given the aging population, 2 yet general medical practitioners lack con dence and minimum competency for managing ophthalmic problems. 5PALMs may offer an effective and e cient modality for such instruction.For the medical student and general provider population, we recommend combining the PALM with foundational content either via a supplemental lecture or, preferably, integration into the PALM.
There are a few limitations to review.This PALM is designed to require a minimum of three intervening trials before a particular diagnosis could be repeated.Since this study included ve diagnostic categories, retired categories were incorporated to maintain the minimum spacing requirement as soon as one category had been retired.Thus, adaptive techniques were not fully applied and participants inevitably reviewed already mastered categories (although these were strictly " ller" and not included in any sequencing or mastery criteria).More robust learning and retention may have been observed in a PALM with additional categories.A direct comparison between the PALM and lecture was also limited by the shorter duration of the lecture relative to the PALM.However, a post-hoc analysis simulating a post-test occurring contemporaneously between the two groups suggests that most of the short-term learning occurred in the rst few minutes of the PALM.

Conclusions
Diagnosis of ophthalmic disease requires substantial use of visual pattern recognition.The PALM was effective for training identi cation of common optic nerve ndings, and may have further applications in ophthalmic education for medical students, residents, and non-ophthalmology providers.Increase in diagnostic con dence before and after an optic nerve PALM or lecture.The PALM group tended to experience greater increases in con dence compared to the lecture group (P = 0.008).

Figure 1
Abbreviations PALM (perceptual and adaptive learning module)

Figure 2 Effect
Figure 2