Improving Diagnostic Ability and Confidence In Otoscopy Through A Web Based Learning Platform: A Prospective Interventional Study


 Background

Otoscopy is an important clinical skill to master in the undergraduate medical curriculum. This study investigated the effect of a new web-based learning platform in improving diagnostic ability and confidence in otoscopy among final year medical undergraduates in Universiti Kebangsaan Malaysia Medical Centre (UKMMC).
Methods

A new web-based learning platform for otoscopy was introduced to final year medical undergraduates who have completed a three-week posting in otorhinolaryngology. A total of 45 subjects who participated in the learning platform were included in this prospective interventional study. All of them answered questionnaires on level of confidence (LC) and completed online questions on diagnostic ability (DA) at three different timelines during the study: prior to intervention (T1), after intervention (T2) and 4 months following the intervention (T3).
Results

LC scores were significantly higher in between T1 and other timelines measured: T1 vs T2 and T1 vs T3 respectively (p < 0.05). DA scores were significantly higher in between T1 and other timelines measured: T1 vs T2 and T1 vs T3 respectively (p < 0.05). LC and DA scores continued to improve at T3 with scores significantly higher when compared with scores at T2 (p < 0.05).
Conclusion

Subjects’ level of confidence in performing otoscopy and diagnostic ability showed sustained improvements at 4 months with this intervention. We conclude that otoscopy web based learning platform complements the teaching and learning of otoscopy among final year medical undergraduates.


Abstract Background
Otoscopy is an important clinical skill to master in the undergraduate medical curriculum. This study investigated the effect of a new web-based learning platform in improving diagnostic ability and con dence in otoscopy among nal year medical undergraduates in Universiti Kebangsaan Malaysia Medical Centre (UKMMC).

Methods
A new web-based learning platform for otoscopy was introduced to nal year medical undergraduates who have completed a three-week posting in otorhinolaryngology. A total of 45 subjects who participated in the learning platform were included in this prospective interventional study. All of them answered questionnaires on level of con dence (LC) and completed online questions on diagnostic ability (DA) at three different timelines during the study: prior to intervention (T1), after intervention (T2) and 4 months following the intervention (T3).

Results
LC scores were signi cantly higher in between T1 and other timelines measured: T1 vs T2 and T1 vs T3 respectively (p < 0.05). DA scores were signi cantly higher in between T1 and other timelines measured: T1 vs T2 and T1 vs T3 respectively (p < 0.05). LC and DA scores continued to improve at T3 with scores signi cantly higher when compared with scores at T2 (p < 0.05).

Conclusion
Subjects' level of con dence in performing otoscopy and diagnostic ability showed sustained improvements at 4 months with this intervention. We conclude that otoscopy web based learning platform complements the teaching and learning of otoscopy among nal year medical undergraduates.

Background
Diseases of the ear are common in general practice, making otoscopy an important skill for a general practitioner. However, teaching and learning of otoscopic examination are often inadequate at both the undergraduate and postgraduate levels. As a result, the diagnostic accuracy of a healthcare practitioner, even in diagnosing a common otologic condition such as acute otitis media remains questionable. In a systematic review of the literature from 1996 to 2003 by Blomgren et al, the diagnostic accuracy of acute otitis media ranged between 38-50% among pediatricians from various states in the United States [1].
Although similar studies have not been carried out in Malaysia, we feel that this problem is rampant in our current setting. Failure to accurately diagnose this condition will result in over or under diagnosis leading to improper treatment, development of complications, and promotion of antimicrobial resistance [1].
In the current undergraduate medical curriculum of Universiti Kebangsaan Malaysia (UKM), training of otoscopy skill was provided in a short, three-week posting in the otorhinolaryngology department. The posting took place in the fourth undergraduate year. During the posting, students are exposed to lectures on otoscopy in a large group, one-hour training in a small group of 6-8 students and direct observation of patient experiences in the clinic and the ward. Based on the recommendations outlined by the Association of American Medical Colleges, the current curriculum lacked directed independent learning, which forms a major component in the acquisition of a clinical skill [2].
The purpose of this study was to investigate the level of con dence and diagnostic ability in otoscopy among nal year undergraduate medical students UKM. Speci cally, we sought to assess the effect of a new web-based learning platform on the level of con dence and diagnostic ability in otoscopy prior intervention (T1), after the intervention (T2) and four months following the intervention (T3).
In this study, we developed a web-based learning platform to enhance the otoscopy skills among the medical undergraduate students of UKM. It was made available as mobile content to allow accessibility on students' personal devices. All nal year undergraduate medical students were invited to participate in the study. The effectiveness of learning otoscopy through this platform was assessed using online questionnaires on the level of con dence in the otoscopic examination and one best answer questions on diseases of the ear canal and tympanic membrane to test the diagnostic ability. All subjects were sampled at three different timelines: prior to using the web-based learning platform (T1), directly after using the web-based learning platform (T2) and four months after using the web-based learning platform (T3).

Methods
A prospective interventional study was conducted among nal year medical undergraduates of UKM between January 2019 and August 2019. Participation in the study was voluntary and written informed consent was obtained from each subject. This study was approved by the Institutional Ethical Review Board of Universiti Kebangsaan Malaysia Medical Centre (FF-2019-324) along with the research support grant on 28th of June 2019. One hundred and twenty-two nal year medical undergraduate students consented to the study. Subjects that did not complete any of the outcome measures were excluded from the study. A total of 77 subjects were excluded due to incomplete data, making the nal study population 45 subjects. The sample size needed to exclude the null hypothesis calculated from the previous study by Fisher and P eiderer (1992) yielded 31 subjects to obtain a study power of 80%. [3] Developing web-based learning on otoscopic examination A web-based learning platform was developed using a free online account on the Schoology™ (Schoology Inc. New York) application. The learning platform included an educational video on otoscopic examination and an online forum moderated by an otorhinolaryngologist to answer questions regarding the otoscopic examination. The educational video was developed by the researchers and consisted of real case scenarios and examinations on real patients. Students are able to view, zoom, pause, and rewind the video at any segment to fully experience this examination. Otoscopic examination videos from real patients were provided in the educational video, differentiating abnormal and normal tympanic membrane to heighten the subjects' learning experience. All otoscopic images were captured using Horus Digital Otoscope ® (Miis, Taiwan) to simulate real-life experiences with the traditional otoscope. The educational video, which lasted for less than 6 minutes, showed the correct technique of otoscopic examination, emphasizing the important steps and common mistakes while performing otoscopy.

Intervention
All subjects in the study underwent the same intervention which included: online learning through educational videos, full accessibility to materials on the online discussion forum moderated by an otorhinolaryngologist and continued feedback on common diseases of the ear. All questions posed by the subjects on the online forum was answered within 24 hours by an experienced otorhinolaryngologist .

Study Outcome Measures
The study used two self-administered, online questionnaires to assess the level of con dence in the otoscopic examination and single best answer questions on diseases of the ear canal and tympanic membrane to test the diagnostic ability. Online questionnaires were used to assess the study outcome measures. These questionnaires were available on the web-based learning platform, delivered as assignments, time-bound relative to the time when the subjects rst complete all the learning activities provided.
The study subjects were instructed to complete all the questionnaires at three different timelines during the study: prior intervention (T1), after the intervention (T2) and four months following the intervention (T3). Reminders were sent through text messages via WhatsApp Messenger™ (WhatsApp Inc. California) and Schoology™ (Schoology Inc. New York) applications to complete the online questionnaires within the study duration.
Questionnaire Assessing the Level of Con dence The subjects' self-perceived con dence was evaluated using a questionnaire to assess the subjects' level of con dence in the otoscopic examination at T1, T2, and T3. This study used a similar questionnaire as Kaf et al, in which a Likert scale of 1-5, low to high, respectively was used for the level of con dence evaluation. [4] The parameter was assessed at six different variables; (1) Level of con dence in identifying the external auditory canal, (2) Level of con dence in distinguishing normal and abnormal external auditory canal, (3) Level of con dence in identifying the tympanic membrane, (4) Level of con dence in distinguishing normal and abnormal tympanic membrane, (5) Level of con dence in practicing otoscopic examination correctly and (6) Level of con dence to diagnose common pathologies affecting the external auditory canal and the tympanic membrane. Scores for every variable ranged from 1-5, with a total score, ranging from 6 to 30 marks. The level of con dence for the respective ear anatomical sites; external auditory canal and tympanic membrane were analysed by taking the total scores from variables (1) and (2) for external auditory canal, and variables (3) and (4) for tympanic membrane, with a total score, ranging from 2 to 10 marks. The level of con dence was assessed at three different timelines; T1, T2 and T3.

Questions Assessing Diagnostic Ability
To measure subjects' diagnostic ability in common ear pathologies, 25 one best answer questions on common diseases of the ear was constructed. These questions underwent content validation by an expert panel, which consisted of one otology consultant with clinical experience of more than 15 years and three otorhinolaryngologists with clinical experience for more than 5 years. All the images used in these questions were taken from real patients using the Horus Digital Otoscope ® (Miis, Taiwan). Following content validation, these questions were uploaded into the Schoology™ (Schoology Inc. New York) application in an online questionnaire format. The questions composed of 8 common otological diseases (3 questions per disease) of the external and middle ear, and 1 question on the normal ear. A total of 6 out of 25 questions assessed the subjects' diagnostic ability on diseases of the external auditory canal. On the other hand, the remaining 19 questions assessed the subjects' diagnostic ability on diseases of the tympanic membrane. Each correct answer was given one mark and each wrong answer was given zero marks. The total score of the diagnostic ability on external auditory canal diseases ranged from 0 to 6 marks, while the total score for tympanic membrane ranged from 0 to 19 marks. The diagnostic ability was assessed according to the three different timelines; T1, T2, and T3.

Statistical Analysis
Descriptive statistical analysis on the demographic characteristics; gender, ethnic, and age were analysed by using percentage and frequency. Continuous data; the level of con dence and diagnostic ability were analysed by comparing means and standard deviation between the three different timelines: prior intervention (T1), after the intervention (T2) and four months following the intervention (T3). Paired ttests with con dence interval 95% were used to compare means between 3 pairs of data: prior intervention (T1) and after intervention (T2), prior intervention (T1) and four months following the intervention (T3), and after intervention (T2), and four months following the intervention (T3). The two study parameters were analysed respectively across the time of sampling and also according to ear anatomical sites (external auditory canal and tympanic membrane). A p-value of ≤ 0.05 was considered to be signi cant in two-tailed tests. All of the analysis was conducted using SPSS version 23.0 (SPSS Inc., Chicago, IL).

Demographic Characteristics
One-hundred and twenty-two nal year medical undergraduate students participated in this prospective interventional study. 45 out of 122 (36.9%) subjects completed all the study variables prior to intervention (T1), directly after the intervention (T2) and four months following the intervention (T3). All 45 subjects were recruited for this study. 77 out of 122 (63.1%) subjects were dropped out due to incomplete data. The levels of con dence of the subjects were assessed at three different timelines during the study: prior intervention (T1), after the intervention (T2) and four months following the intervention (T3). T-test was used to compare means between 3 pairs of data: prior intervention (T1) and after the intervention (T2), prior intervention (T1) and four months following the intervention (T3), and after the intervention (T2) and four months following the intervention (T3). Total scores for the level of con dence was calculated for all the study subjects and subsequently analysed.
The mean of total scores for level of con dence showed signi cant improvement from T1 at 20.87 (2.920) to T2 at 24.98 (3.265). T-test showed a signi cant difference in the distribution of the mean of total scores for con dence level between T1 and T2 (p < 0.05). Similarly, the mean of total scores for the level of con dence showed signi cant improvement from T1 at 20.87 (2.920) to T3 at 26.44 (3.094). Ttest showed a signi cant difference in the distribution of the mean of total scores for con dence level between T1 and T3 (p < 0.05). Additionally, the mean of total scores for the level of con dence showed signi cant improvement from T2 at 24.98 (3.265) to T3 at 26.44 (3.094). T-test showed a signi cant difference in the distribution of the mean of total scores for con dence level between T2 and T3 (p < 0.05). [ Table 2]

Diagnostic Ability
The diagnostic ability of the subjects was assessed at three different timelines during the study: prior intervention (T1), after the intervention (T2) and four months following the intervention (T3). T-test was used to compare means between 3 pairs of data: prior intervention (T1) and after intervention (T2), prior intervention (T1), and four months following the intervention (T3), and after intervention (T2) and four months following the intervention (T3  Two out of 6 questions assessed subjects' level of con dence in identi cation and examination of the external auditory canal during otoscopy. Another 2 questions assessed the subjects' level of con dence in the identi cation and examination of the tympanic membrane during otoscopy. Each question was assessed by a numerical scale of 0-5, with a score of 5 representing a high level of con dence. To analyse the level of con dence for each ear anatomical sites (external auditory canal and tympanic membrane) in the otoscopic examination, T-test was used to compare means between three different timelines during the study: prior intervention (T1), after the intervention (T2) and four months following the intervention (T3). T-test was used to compare means between 3 pairs of data: prior intervention (T1) and after the intervention (T2), prior intervention (T1) and four months following the intervention (T3), and after intervention (T2) and four months following the intervention (T3), Total scores for level of con dence was calculated according to anatomical sites (external auditory canal and tympanic membrane) separately for all the study subjects and subsequently analysed.  021). T-test showed a signi cant difference in the distribution of the mean of total scores for con dence level between T1 and T3 (p < 0.05). Additionally, the mean of total scores for level of con dence in tympanic membrane showed signi cant improvement from T2 at 8.33 (1.297) to T3 at 8.84 (1.021). T-test showed a signi cant difference in the distribution of the mean of total scores for con dence level between T2 and T3 (p < 0.05). [ Table 3]

Diagnostic Ability
Six out of 25 questions that assessed the subjects' diagnostic ability in diseases of the external auditory canal. Meanwhile, the remaining 19 questions assessed the subjects' diagnostic ability in diseases of the tympanic membrane. Each correct answer was given one mark and total marks for diagnostic ability in diseases of the external auditory canal and the tympanic membrane was calculated separately for all subjects.
To analyse the diagnostic ability in diseases of the external auditory canal and tympanic membrane, Ttest was used to compare means between three different timelines during the study: prior intervention (T1), after the intervention (T2) and four months following the intervention (T3). T-test was used to compare means between 3 pairs of data: prior intervention (T1) and after intervention (T2), prior intervention (T1) and four months following the intervention (T3), and after intervention (T2), and four months following the intervention (T3).

Discussion
Otoscopic examination is crucial to diagnose otologic conditions, which are common in general practice.
[5] However, studies have shown that teaching and learning of otoscopic examination are often inadequate at both the undergraduate and postgraduate levels. [3] Nowadays, with the rapid progression in multimedia technology, it has become easier to use web-based platforms for teaching and learning.
Web-based teaching can offer many potential advantages over the traditional method of teaching [6]. Web-based teaching can allow students to learn using graphic images, sound and video, accessing content anywhere else other than class, interactive learning by quiz or feedback, and to revisit the content for revision. Web-based teaching is also a low-cost teaching method with wide accessibility [7].
Grundman et al. concluded that visual and audio was able to augment the learning of a particular examination skill. In their study, the students' examination skills and diagnostic ability showed improvement using web-based learning compared to revising printed lecture notes [6]. In another study in 2016 by Stepniak et al, web-based learning was found to be effective in teaching otoscopy for undergraduate medical students [8]. However, this study utilised a high end, and expensive simulator with a very limited follow-up period of only one week. In this study, the long term effectiveness of learning otoscopy through a newly developed online platform was assessed using online questionnaires on the level of con dence in the otoscopic examination and one best answer questions on diseases of the ear canal and tympanic membrane to test the diagnostic ability.

Level of Con dence
Student acquisition of knowledge and skills in otoscopy can be measured via students' self-evaluation opportunities. An example of such evaluation is self-reporting of perceived con dence level in performing the various tasks in the otoscopic examination. This study utilized such methods using a 5-point Likert scale to determine the level of con dence in the otoscopic examination. A similar evaluation was used by Kaf et al in their interventional study of thirty-two rst-year undergraduate audiology students [4]. On the other hand, Swamy et. al. in 2014, assessed their subjects using a 4-point Likert scale in an interventional study on a simulated model (SimMan) to improve the con dence level in performing otoscopy [9]. We feel that utilising a 5-point Likert scale is better compared to a 4-point scale to observe differences in subjects' level of con dence across the different sampling points in the study.
This study demonstrated a signi cant increment in the subjects' perceived level of con dence in performing otoscopy following the intervention. This increment is sustained at even four months following the study intervention, showing that a web-based learning platform may be an effective addition in teaching and learning of otoscopic examination.  [4,9,10].
Comparing the subjects' self perceived level of con dence in external auditory canal and the tympanic membrane, we found that the nal year medical undergraduate students were less con dent in the identi cation and abnormalities of the tympanic membrane compared to the external auditory canal. This nding concurred with a similar study by You et al, who mentioned that it was challenging to assess tympanic membrane due to its location and inaccessibility compared to external auditory canal [10]. This was re ected in our results where the mean level of con dence score to identify the conditions in tympanic membrane is slightly lower at T3 (8.84) than the external auditory canal (9.09).

Diagnostic Ability
Students' diagnostic ability can be measured by using questions with marks. The questions which may include normal ear and common otology pathology of the external auditory canal and tympanic showing the normal anatomical structure [11,12]. Each correct answer was given one mark and the total marks attributed to the students' diagnostic ability [4,11,12]. Thus, in this study, we determined the diagnostic ability by providing 25 validated one best answer questions given by a panel of experts. The questions composed of 8 common otology pathology (3 questions per pathology) of the external ear, middle ear, and 1 question about the normal ear. Each correct answer was given one mark.
There was a signi cant improvement in diagnostic ability on both the external auditory canal and the tympanic membrane. In line with the previous study conducted by Moberly et. al, the diagnostic ability of otoscopic examination increased after the intervention [12]. The long term retentions for diagnostic ability was also measured 4 months following the intervention. Similar to the level of con dence, the diagnostic ability in otoscopic examination among students was sustained at high scores over a period of four months. The results showed a signi cant difference in diagnostic ability scores when compared 4 months following intervention (5.20) to the baseline (4.38). The results obtained were similar to a previous study conducted by Dastjerei et al who concluded that a multimedia program on teaching empirical science among second-grade students showed better retention in learning 3 months following the intervention compared to traditional methods [13]. The authors felt that this sustained improvement in the diagnostic ability scores was mainly attributed to the accessibility of the teaching materials, focused learning of core skills in the educational video, and the ability to provide feedback to learners.
However, the diagnostic ability for the external auditory canal was not signi cant when comparing after the intervention (5.29) and 4 months following the intervention (5.20). This is because both mean values were already very good and near the maximum score of 6.
This study has some limitations to be acknowledged including the high rate of dropouts due to incomplete data. The researchers, however, managed to achieve the targeted sample size to obtain a considerably good study power of 80%. We recommend further studies to utilise the various platforms including WhatsApp Messenger™ (WhatsApp Inc. California) and Schoology™ (Schoology Inc. New York) to send reminders so that subjects are able to complete the online questionnaires within the study duration. Additionally, this study did not have a control group without intervention to compare the study outcome measures. A randomized, prospective interventional study is needed to con rm our study ndings.

Novelty of the study
This study developed a web-based learning platform for the medical undergraduates of Universiti Kebangsaan Malaysia which is tailored the needs of the current curriculum of the university as well as the local setting. Adapting a web-based learning platform developed by the Western countries may not only be costly, but may lack linguistic and cultural adaptations unique to the learner and their future patients. This study demonstrated long term positive effects on students' level of con dence and diagnostic ability which was not demonstrated in previously published interventions in teaching and learning of the otoscopic examination. [7,10,11] A web-based learning platform may offer many potential advantages in the current teaching method and should be incorporated in current teaching method to improve the level of con dence and diagnostic ability in otoscopic examination among the undergraduate students.
How will this paper make a difference in medical education practices?
This study sheds light on more effective ways to train future doctors in the otoscopic examination. This skill is important to master before they graduate especially as they are preparing themselves to diagnose otologic conditions and managing these diseases independently in general practice. Failure in teaching and learning of otoscopic examination will contribute towards future problems such as misdiagnosis, antimicrobial resistance and, even morbid complications. It is appealing to the general practice as a signi cant proportion of patient presents with ear complain, the most prevalent being, otitis media with effusion which is reported to be as high as 18.3% among preschoolers. [14] Conclusion A web-based learning platform for otoscopy is an effective tool that complements the teaching and learning of otoscopy among nal year medical undergraduates. There are sustained improvements at 4 months in the level of con dence in performing otoscopy and diagnostic ability among the subjects following this intervention. This accessible and mobile learning platform may be a valuable addition to the undergraduate medical curriculum.

Abbreviations
Not applicable

Declarations
Ethics approval and consent to participate: The study was approved by the institutional ethical review board for human research of University Kebangsaan Malaysia within which the study was undertaken (IRB code of approval: FF-2019-324). All study participants provided a written consent to participate in the study. Participation is voluntary and all students who ful lled the inclusion criteria were invited to participate.

Consent for publication:
Not applicable Availability of data and material: The dataset supporting the conclusions of this article is available in the corresponding author's repository at https://docs.google.com/spreadsheets/d/1qWoryXTjRENSEpRLM6PKy5YoaZblasrp3eTuBdzEUos/edit?
usp=sharing. Link to the dataset will be activated following a formal request made to the corresponding author after the manuscript has been published for duration of ve years to protect the con dentiality of the research participants, according to the institutional requirements.