Evaluating Undergraduate Learning in Ophthalmology Rotation: A Cross-Sectional Study Using Virtual Questionnaires

doi:10.21203/rs.3.rs-3636736/v1

Download PDF

Research Article

Evaluating Undergraduate Learning in Ophthalmology Rotation: A Cross-Sectional Study Using Virtual Questionnaires

https://doi.org/10.21203/rs.3.rs-3636736/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Background

Traditional ophthalmology education in medical schools is inadequate for nonophthalmologist residents in primary outpatient care. This article focuses on measuring ophthalmology learning in a clinical rotation of a Brazilian medical school that implemented a two-week ophthalmology rotation, with one week dedicated to online theoretical classes and the following week dedicated to outpatient practices. Therefore, our objective was to evaluate the modality, method, and established hours to determine their effectiveness in the training of generalist physicians.

Methods

This was a cross-sectional and quantitative study. All students eligible for the rotation were given a questionnaire before and after the rotation to assess their knowledge, understanding, and problem-solving abilities in the field. Each questionnaire contained 11 theme-paired questions developed according to the International Council of Ophthalmology curriculum and guidelines. The primary outcome was improvement in scores between the pre- and postrotation tests. Researchers defined 30% or higher as satisfactory. Data were anonymously analyzed using STATA BE 17 software. Questionnaire reliability was measured using Cronbach's alpha.

Results

In this study, 95 students were found to be eligible, and 37 of them answered both the pre- and postrotation tests correctly, constituting 38,94% of the eligible population. The average response time for the postrotation test was 34.04 ± 31.22 minutes, while the prerotation test took an average of 04:35 minutes, and the postrotation test took 07:02 minutes on average. Only 13 students (34%) showed a minimum improvement of 30% in their scores after the rotation. The average number of correct answers on the prerotation test was 6.756 ± 2.087, and on the postrotation test, it was 7.567 ± 2.128, with a mean difference of -0.81 ± 1.912 between them. There was no significant difference between the pretest and posttest scores.

Conclusions

Medical students’ learning in the ophthalmology rotation was not satisfactory. Therefore, it is essential to consider the implementation of more comprehensive and integrated educational approaches to enhance students' ophthalmological knowledge with an entirely in-person modality and increased practical hours.

Ophthalmology rotation

questionnaire

medicine

education

educational assessment

Preparing medical students to provide ophthalmological care has been a challenge in medical education, with a documented decrease in instruction in this area over the decades.(1) In 2014, a study revealed that only 18% of medical schools needed participation in a clinical ophthalmology rotation, representing a significant decline of 68% compared to 2000.(2) Several factors have been identified as responsible for this negative trend, including increasingly overloaded curricula, greater competition among medical specialties for teaching time, lack of support from ophthalmology educators, and the reluctance of curriculum committees to recognize the utility of a mandatory ophthalmology rotation.(3, 4) Consequently, medical graduates have been shown to be ill prepared to handle their patients' ophthalmological complaints.

The International Council of Ophthalmology (ICO) proposes a competency-based approach to medical education that emphasizes knowledge of ocular diseases, anatomy, and ocular manifestations of systemic diseases, as well as the essential clinical skills to assess visual acuity, visual fields, extraocular movement, and other indicators of ocular health.(5) However, despite these existing guidelines, many medical students still experience discomfort regarding their level of theoretical and practical ophthalmological knowledge.(6, 7) This lack of exposure to ophthalmology may lead some students to dismiss the possibility of pursuing this specialty as a career. Furthermore, physicians who graduate from medical school without adequate experience in ophthalmology may not fully grasp the importance of ocular health and the breadth of ophthalmological practice as generalist doctors.

Although the need for formal ophthalmology training in medical schools is evident, available educational tools are scarce to meet this demand. No publications detailing ophthalmology curriculum content specifically designed for medical students in their clinical years have been found in MedEdPORTAL. However, Waning et al. described a highly successful 10-day ophthalmological rotation procedure. This experience was based on pre- and posttest data, demonstrating the feasibility of an ophthalmology rotation for students in their clinical stage.(8)

Given this gap in ophthalmological educational content, our proposal is to evaluate the ophthalmology rotation at a medical school in Brazil, with the aim of assessing the effectiveness of the teaching model employed during this rotation.

Population

This cross-sectional and quantitative study selected fourth-year medical students who participated in an ophthalmology rotation consisting of both theoretical and practical outpatient activities lasting two weeks with a workload of 23 hours between June 2022 and December 2022 at an Ophthalmological Hospital. None of the students had any prior formal exposure to ophthalmology. The participants formed a single group and were given a questionnaire after approval of this project by the Research Ethics Committee of the Bahiana School of Medicine. Seventy-nine students were eligible to respond to the questionnaire, but only 37 answered both the pre- and postrotation tests, which constituted the sample for the statistical analysis.

Questionnaire and Outcomes

To measure learning, two questionnaires were administered before and after rotation to assess knowledge, understanding, and problem-solving abilities in ophthalmology. Each questionnaire consisted of 11 questions with a final score ranging from 0 to 11. The questions were developed according to the curriculum matrix of the component, the guidelines of the International Council of Ophthalmology for Ophthalmological Education in undergraduate programs, and input from medical students, retina specialists, cornea specialists, and retina fellows. A pilot questionnaire consisting of 40 questions was administered, and after analyzing the pilot results, 22 questions were selected. The questions were categorized into themes and divided equally between the pre- and postrotation tests. The content covered by the questions included basic anatomy and physiology, anterior chamber anatomy and physiology, extraocular muscle anatomy and physiology, retinal anatomy and physiology, and red eye. Although the content was the same for the pre- and postrotation exams, the questions were slightly different but equivalent.

The primary outcome of the study was the improvement in the number of correct answers when comparing the scores of the pre- and postrotation tests. Prior to the rotation, the same individuals who developed the questions stated that a result equal to or above 30% would be considered satisfactory, considering the difficulty of the questions. All data were analyzed anonymously, and the study results were presented in an aggregated manner, preserving students' identities.

Statistical Analysis

STATA BE 17 software (StataCorp LLC, College Station, TX, USA) was used for the statistical analysis. The Shapiro‒Wilk test was used to test the normality of the data. Cronbach's alpha was used to measure the reliability and internal consistency of the questionnaire. Although there is no definitive agreement on the cutoff for Cronbach's alpha, it is conventionally accepted among test designers that a Cronbach's alpha coefficient of > = 0.6 is satisfactory for exploratory research.(9) The comparison of the pre- and postrotation test results for numerical variables that meet the criteria of normality will be performed using the paired samples t test.

Ethical aspects: The anonymity of the research participants was preserved in accordance with the principles of beneficence and nonmaleficence. The ethics committee of the Bahiana School of Medicine No. 5,768,569 approved this project.

A total of 95 individuals were eligible to participate in this study, but only 44 underwent the prerotation test. Of these, 7 did not complete the postrotation test after concluding their rotation. Therefore, 37 participants answered both questionnaires, representing 38,94% of the eligible population. The internal consistency (Cronbach's alpha) was 0.7409.

The average response time for the postrotation test was 34.04 ± 31.22 minutes. Additionally, the average time to complete the prerotation test was 04:35 min, whereas the average time for the postrotation test was 07:02 min.

Among the participants, two individuals achieved the maximum score. One participant obtained the full score in both the pre- and postrotation tests, whereas the other participant reached it only in the postrotation test. However, only 34% of the participants (13 students) improved their scores by at least 30% (Table 1 – Field in submission upload).

Table 1

Correct answers before and after Ophthalmology rotation, and time to complete post-rotation test (days).
N	BEF	AFT	DAYs
1	8	5	6
2	5	2	6
3	9	6	100
4	8	10	100
5	10	8	26
6	4	4	27
7	5	6	28
8	5	7	28
9	8	9	32
10	5	5	32
11	3	7	117
12	7	7	42
13	5	3	42
14	7	10	120
15	5	7	17
16	7	10	52
17	6	9	3
18	10	11	17
19	11	11	5
20	5	8	22
21	5	8	23
22	8	7	60
23	7	8	63
24	8	7	28
25	10	9	0
26	6	7	3
27	4	6	3
28	9	9	3
29	6	9	18
30	6	8	20
31	10	10	12
32	8	10	40
33	8	10	13
34	6	8	31
35	3	6	33
36	5	6	48
37	8	7	39
N – Number of people; BEF – Before rotation; AFT – After roration; Days - time to complete post-rotation test in days

The mean score on the prerotation test was 6.756 ± 2.087 with a 95% confidence interval of [6.06–7.45], and on the postrotation test, it was 7.567 ± 2.128 with a 95% confidence interval of [6.85–8.27]. The mean difference in the number of correct answers was − 0.81 ± 1.912 with a 95% confidence interval of [-1.44 - -0.17] (Table 2). Additionally, only 34% of the participants (13 students) improved their scores by at least 30%.

Table 2

Results of pre- and postrotation tests.
	N	MEAN	SD	IC 95%
BEF	37	6.75	02.08	6,06–7,45
AFT	37	7.56	2.12	6,85–8,27
DIFF	37	-0.81	1.91	-1,44 – -0,17
N - number of students; SD - standard deviation; CI - confidence interval; BEF – mean before rotation; AFT – mean after rotation; DIFF – mean difference between after and before rotations

The mean score on the postrotation test before 30 days was 7.187 ± 2.34 with a 95% confidence interval of [5.93–8.43], and after 30 days, it was 7.43 ± 2.03 with a 95% confidence interval of [6.35–8.52]. In this case, the mean difference in the number of correct answers was − 0.25 ± 2.79 with a 95% confidence interval of [-1.73–1.23]. Therefore, there was no statistically significant difference between the participants who completed the test before or 30 days after the end of the rotation.

Undergraduate medical education aims to train generalist physicians with the ability to diagnose and treat diseases, including ocular conditions, either through direct patient care or by referring them to the appropriate specialty.(10) Most medical students lack confidence in performing ophthalmological examinations(6, 7), but competence in the fundamentals of ophthalmology is essential for safely evaluating patients with ocular complaints in primary care, emergency departments, neurology, and other nonophthalmological settings.(11, 12) Therefore, this study evaluates the effectiveness of an ophthalmology teaching model including one week of online theoretical classes followed by another week of in-person outpatient practices and in-person classes containing 23 hours of instruction.

Ophthalmology remains an area of significant relevance in the field of medicine. However, a study conducted in the United States showed that its role in medical school curricula has been systematically eroded over the past four decades.(2, 13) This scenario is not different in Brazil. Currently, the teaching of ophthalmology to medical students has been gradually reduced and, in some cases, eliminated from the curriculum in Brazil. Curriculum revisions have emphasized that case-based teaching, small-group learning, and problem-based learning techniques are valuable tools for enhancing students' learning capacity. However, these methods require significant time to achieve satisfactory integration, and new teaching methods must be tested to provide better student rotations.(14) This raises the question of how to optimize ophthalmology education within a limited timeframe. The medical program is already packed, so strategies are needed to maximize the effectiveness of ophthalmology teaching.(15)

Despite the limited time dedicated to ophthalmology, all medical school graduates are expected to have basic knowledge about ophthalmic diseases and ocular manifestations of systemic diseases, such as diabetes mellitus and systemic arterial hypertension. Therefore, the ophthalmology rotation is an essential component of comprehensive medical education.(2)

Previous studies on ophthalmology rotations and students' knowledge gains have shown that overall student performance was satisfactory, although some students had unsatisfactory performance.(16–18) However, despite the postrotation test taking longer than the prerotation test, which may indicate more extraordinary dedication and attention to the second questionnaire, there was no significant difference between the two groups' pre- and postrotation test scores. This indicates that the level of knowledge before the course was similar in both groups. The same occurred for the difference in responses between the postrotation tests conducted before and 30 days after the completion of the rotation, with no significant difference between the two groups.

Therefore, although the majority of participants obtained a higher score on the postrotation test, the average increase was relatively small, and only one-third of the students showed significant improvement. This result may indicate that a two-week rotation, with one week in a virtual format, may not be sufficient to provide substantial learning in ophthalmology, especially considering that students had no prior exposure to the discipline. Other studies have reported the need for more comprehensive and integrated educational approaches to ophthalmology. Some institutions have adopted team-based learning modules and intensive courses to enhance student knowledge in this area. However, the duration of these programs can vary considerably, and it is essential to consider the extent of exposure time needed to achieve more significant results in ophthalmic learning.(19)

As an increasing number of institutions seek to shorten the preclinical period, ophthalmology will face the challenge of creatively reimagining its curriculum, as well as other specialties being condensed within the same timeframe. However, ophthalmology has already demonstrated its ability to adapt, compensating for the loss in the traditional curriculum through innovative approaches to teaching and learning. Many institutions have adopted extracurricular activities for medical students, such as academic leagues (89%), mentorship programs (80%), career fairs (77%), and community outreach or service experience (75%). These initiatives broaden student horizons and provide opportunities to acquire knowledge and skills beyond the formal academic curriculum. The coronavirus pandemic has recently driven the development of virtual electives, allowing content to be delivered remotely to medical students. This emergency adaptation shows that new educational approaches can be effective and beneficial for medical training, including ophthalmology.(20)

Therefore, ophthalmology needs to continue exploring and implementing innovative approaches in teaching, leveraging technology, and learning opportunities outside the traditional classroom environment.

Limitations

The present study addressed the use of an online questionnaire to evaluate ophthalmology rotation, but it had several limitations. While gains in practical skills are a central issue in clinical practice, the present study only analyzed gains in theoretical knowledge. Additionally, we did not assess long-term knowledge retention, as this was not one of the study objectives. Furthermore, the limited number of evaluated students represents another potential weakness. Further investigations are recommended, as the questionnaire may bring benefits to the learning and assessment process and could significantly improve the overall ophthalmology education provided to medical students during the rotation.

Our study indicates the need for improvement in ophthalmology teaching during ophthalmology rotation and a more effective method. While some students achieved performance at the expected level for ophthalmological content and clinical skills, the data suggest that the duration of rotation may be insufficient to provide substantial learning in ophthalmology, especially for students without prior exposure to the discipline. It is important to consider implementing more comprehensive and integrated educational approaches to enhance students' ophthalmological knowledge.

ICO

International Council of Ophthalmology

Ethics approval and consent to participate: This project was approved by the ethics committee of the Bahiana School of Medicine No. 5,768,569.

Consent for publication: Informed consent was obtained from all participants or their legal guardians, ensuring their understanding and agreement to participate.

Availability of data and materials: The datasets used and analyzed during the current study are available from the corresponding author upon reasonable request, and data are provided within the manuscript or supplementary information files.

Competing Interests: The authors declare that they have no competing interests

Funding: Not applicable

Author’s Contribution: A.F., D.A., J.C. and M.S. designed the study, developed the questionnaire, and oversaw the implementation of the two-week ophthalmology rotation, which included online theoretical classes and outpatient practices. D.A. and A.F. analyzed relevant data, while L.G. and A.F. played a key role in statistical analysis using STATA BE 17 software.

R.T. and R.O. contributed to the literature review section, while J.G., D.P. and M.S. supervised the research process, offered guidance in the interpretation of results and critically reviewed the manuscript for content, scientific rigor, and ethical standards. A.F. and D.A. wrote the main manuscript text and L.G. prepared tables 1-2. All authors reviewed the manuscript.

Acknowledgments: Not applicable

Footnotes: Not applicable

Succar T, Grigg J, Beaver HA, Lee AG. A systematic review of best practices in teaching ophthalmology to medical students. Surv Ophthalmol. 2016;61(1):83–94.
Shah M, Knoch D, Waxman E. The state of ophthalmology medical student education in the United States and Canada, 2012 through 2013. Ophthalmology. 2014;121(6):1160–3.
Albert DM, Bartley GB. A proposal to improve ophthalmic education in medical schools. Ophthalmology. 2014;121(6):1157–9.
Quillen DA, Harper RA, Haik BG. Medical student education in ophthalmology: crisis and opportunity. Ophthalmology. 2005;112(11):1867–8.
Tso MO, Goldberg MF, Lee AG, Selvarajah S, Parrish RK, Zagorski Z. An international strategic plan to preserve and restore vision: four curricula of ophthalmic education. Am J Ophthalmol. 2007;143(5):859–65.
Gupta RR, Lam WC. Medical students' self-confidence in performing direct ophthalmoscopy in clinical training. Can J Ophthalmol. 2006;41(2):169–74.
Cobbs L, Tsui E, Haberman I, Kim E, Sperber L, Wu M, et al. Student Perceptions of the Ophthalmology Curriculum in Medical School. J Acad Ophthalmol. 2018;10:e79–e82.
Waring GO, Harris R, Walters RF, Berris C. Clinical ophthalmology instruction for medical students. Surv Ophthalmol. 1977;22(2):106–12.
Silva C, Souza M, Silva Filho R, Medeiros L, Criado P. E-learning program for medical students in dermatology. Clinics (São Paulo, Brazil). 2011;66:619 – 22.
De Biase CLCL. Atenção Primária à Saúde-Oftalmologia: Percepção Discente Sobre Competências na Formação Médica. Revista Portal: Saúde e Sociedade. 2018;3(2):750–66.
Chan TY, Rai AS, Lee E, Glicksman JT, Hutnik CM. Needs assessment of ophthalmology education for primary care physicians in training: comparison with the International Council of Ophthalmology recommendations. Clin Ophthalmol. 2011;5:311–9.
Clarkson JG. Training in ophthalmology is critical for all physicians. Arch Ophthalmol. 2003;121(9):1327.
Moxon NR, Goyal A, Giaconi JA, Rosenberg JB, Graubart EB, Waxman EL, et al. The State of Ophthalmology Medical Student Education in the United States: An Update. Ophthalmology. 2020;127(11):1451–3.
Succar T, Grigg J, Beaver HA, Lee AG. Advancing ophthalmology medical student education: International insights and strategies for enhanced teaching. Surv Ophthalmol. 2020;65(2):263–71.
Succar T, McCluskey P, Grigg J. Enhancing Medical Student Education by Implementing a Competency-Based Ophthalmology Curriculum. Asia Pac J Ophthalmol (Phila). 2017;6(1):59–63.
Bowers EMR, Perzia B, Enzor R, Clinger O, Yadav S, Commiskey PW, et al. A Required Ophthalmology Rotation: Providing Medical Students with a Foundation in Eye-Related Diagnoses and Management. MedEdPORTAL. 2021;17:11100.
Noble J, Somal K, Gill HS, Lam WC. An analysis of undergraduate ophthalmology training in Canada. Can J Ophthalmol. 2009;44(5):513–8.
Patil NS, Bal M, Khan Y. Medical student competence in ophthalmology assessed using the Objective Standardized Clinical Examination. Indian J Ophthalmol. 2023;71(5):2218–21.
Kivell TL, Doyle SK, Madden RH, Mitchell TL, Sims EL. An interactive method for teaching anatomy of the human eye for medical students in ophthalmology clinical rotations. Anat Sci Educ. 2009;2(4):173–8.
Wilcha RJ. Effectiveness of Virtual Medical Teaching During the COVID-19 Crisis: Systematic Review. JMIR Med Educ. 2020;6(2):e20963.

No competing interests reported.

Download PDF

Version 1

posted

You are reading this latest preprint version

Evaluating Undergraduate Learning in Ophthalmology Rotation: A Cross-Sectional Study Using Virtual Questionnaires

Status:

Version 1

Abstract

Background

Methods

Results

Conclusions

Background

Methods

Population

Questionnaire and Outcomes

Statistical Analysis

Results

Discussion

Limitations

Conclusions

Abbreviations

Declarations

References

Additional Declarations

Status:

Version 1