Improving Diagnostic Ability And Condence In Otoscopy Through A Supplementary Web Based Learning Platform: A Prospective Interventional Study

Background Otoscopy is an important clinical skill to master in the undergraduate medical curriculum. This study investigated the effect of a supplementary web-based learning platform in improving diagnostic ability and condence in otoscopy among nal year medical undergraduates in Universiti Kebangsaan Malaysia Medical Centre (UKMMC). Methods A supplementary otoscopy training using a web-based learning platform was introduced to nal year medical undergraduates who have completed a three-week posting in otorhinolaryngology. A total of 45 subjects who participated in the learning platform were included in this prospective interventional study. All of them answered questionnaires on level of condence (LC) and completed online questions on diagnostic ability (DA) at three different timelines during the study: prior to intervention (T1), after intervention (T2) and 4 months following the intervention (T3). Results LC scores were signicantly higher across the timelines measured: T1 vs T2 vs T3 respectively (p<0.05). DA scores were signicantly higher across the timelines measured: T1 vs T2 vs T3 respectively (p<0.05). Both LC and DA scores showed sustained improvements at 4 months post intervention across both external auditory canal and tympanic membrane anatomical subsites. Conclusion Subjects’ level of condence in performing otoscopy and diagnostic ability showed sustained improvements at 4 months with this intervention. We conclude that supplementary otoscopy training using a web based learning platform complements the teaching and learning of otoscopy among nal year medical undergraduates.

Results LC scores were signi cantly higher across the timelines measured: T1 vs T2 vs T3 respectively (p<0.05). DA scores were signi cantly higher across the timelines measured: T1 vs T2 vs T3 respectively (p<0.05). Both LC and DA scores showed sustained improvements at 4 months post intervention across both external auditory canal and tympanic membrane anatomical subsites.
Conclusion Subjects' level of con dence in performing otoscopy and diagnostic ability showed sustained improvements at 4 months with this intervention. We conclude that supplementary otoscopy training using a web based learning platform complements the teaching and learning of otoscopy among nal year medical undergraduates.

Background
Diseases of the ear are common in general practice, making otoscopy an important skill for a general practitioner. However, teaching and learning of otoscopic examination are often inadequate at both the undergraduate and postgraduate levels. As a result, the diagnostic accuracy of a healthcare practitioner, even in diagnosing a common otologic condition such as acute otitis media remains questionable. In a systematic review of the literature from 1996 to 2003 by Blomgren et al, the diagnostic accuracy of acute otitis media ranged between 38% to 50% among pediatricians from various states in the United States (1). Although similar studies have not been carried out in Malaysia, we feel that this problem is rampant in our current setting. Failure to accurately diagnose this condition will result in over or under diagnosis leading to improper treatment, development of complications, and promotion of antimicrobial resistance (1).
In the current undergraduate medical curriculum of Universiti Kebangsaan Malaysia (UKM), otoscopy training was provided in a formal, three-week posting in the otorhinolaryngology department. This takes place in the fourth undergraduate year. During this short learning duration, students listen to lectures on otologic conditions in a large group, receive one-hour face to face training in a small group of 6-8 students and later directly observe patient experiences in the clinic and the ward. The current curriculum lacked directed independent learning, which forms a major contributor in the acquisition of a clinical skill in undergraduate medical training. The authors observed that, continued teaching and learning of otoscopy examination was negligible outside this formal, three week posting. This lack of reinforcement may contribute to reduced con dence and pro ciency in otoscopy examination as the student progresses to the nal, fth year of the program.
The purpose of this study was to develop a web-based learning platform to provide supplementary otoscopy training to the nal year undergraduate medical students. This learning platform will include educational resources on otologic diseases, instructional videos and a discussion forum to enhance learning of otoscopic examination. We would then sought to investigate the effect of using this supplementary web-based learning platform on the level of con dence and diagnostic ability in otoscopy prior intervention (T1), after the intervention (T2) and four months following the intervention (T3).
The supplementary web-based learning platform was made available as a mobile content to allow accessibility on students' personal devices. All fth year undergraduate medical students were invited to participate in the study. The effectiveness of learning otoscopy through this platform was assessed using online questionnaires on the level of con dence in the otoscopic examination and one best answer questions on diseases of the ear canal and tympanic membrane to test the diagnostic ability. All subjects were sampled at three different timelines: prior to using the web-based learning platform (T1), directly after using the web-based learning platform (T2) and four months after using the web-based learning platform (T3).

Methods
A prospective interventional study was conducted among nal year medical undergraduates of UKM between January 2019 and August 2019. Participation in the study was voluntary and written informed consent was obtained from each subject. This study was approved by the Institutional Ethical Review Board of Universiti Kebangsaan Malaysia Medical Centre (FF-2019-324) along with the research support grant on 28 th of June 2019. The sample size needed to exclude the null hypothesis calculated from the previous study by Fisher and P eiderer (1992) yielded 31 subjects to obtain a study power of 80%. (2) One hundred and twenty-two nal year medical undergraduate students consented to the study. Subjects that did not complete any of the outcome measures were excluded from the study. A total of 77 subjects were excluded due to incomplete data, making the nal study population, 45 subjects. Out of the 77 subjects who were excluded from the study, 12 subjects were unable to complete the baseline parameters, 30 subjects were lost to follow-up at T2 and the remainder 35 subjects were lost to follow-up at T3. All the excluded subjects were interviewed and none had problems with accessibility or disliked the online platform. All 122 subjects continued to use the platform after completion of the study.
Developing a web-based learning platform on otoscopic examination A web-based learning platform was developed using a free online account on the Schoology™ (Schoology Inc. New York) platform. The URL address for the otoscopy course used was https://app.schoology.com/course/1993729423. A unique access code was given to all the study subjects to ensure selective enrolment to the undergraduate otoscopy online course . The learning platform included four lecture notes on otologic diseases and audiological tests, an educational instructional video on otoscopic examination and a discussion forum moderated by an otorhinolaryngologist to answer questions regarding the otoscopic examination ( Figure 1). The educational video was developed by the researchers and consisted of real case scenarios and examinations on real patients. Students are able to view, zoom, pause, and rewind the video at any segment to fully experience this examination. Otoscopic examination videos from real patients were provided, differentiating abnormal and normal tympanic membrane to heighten the subjects' learning experience. All otoscopic images were captured using Horus Digital Otoscope ® (Miis, Taiwan) to simulate real-life experiences with the traditional hand-held otoscope. The educational video, of less than 6 minutes duration, showed the correct technique of otoscopic examination, emphasizing the important steps and common mistakes while performing otoscopy.

Intervention
All subjects in the study underwent the same intervention which included: online learning through materials provided in the course inclusive of lecture notes, educational video, and full accessibility to materials on the online discussion forum moderated by an otorhinolaryngologist and continued feedback on common diseases of the ear. All questions posed by the subjects on the online forum were answered within 24 hours by an experienced otorhinolaryngologist ( Figure 2).

Study Outcome Measures
The study utilised two self-administered, online questionnaires to assess the level of con dence in the otoscopic examination and single best answer questions on diseases of the ear canal and tympanic membrane to test the diagnostic ability. Online questionnaires were used to assess the study outcome measures. These questionnaires were available on the web-based learning platform, delivered as assignments, time-bound relative to the time when the subjects rst complete all the learning activities provided.
The study subjects were instructed to complete all the questionnaires at three different timelines during the study: prior intervention (T1), after the intervention (T2) and four months following the intervention (T3). Reminders were sent through text messages via WhatsApp Messenger™ (WhatsApp Inc. California) and Schoology™ (Schoology Inc. New York) applications so that subjects can complete the online questionnaires within the study duration.
Questionnaire Assessing the Level of Con dence The subjects' self-perceived con dence was evaluated using a questionnaire to assess the subjects' level of con dence in the otoscopic examination at T1, T2, and T3. This study used a similar questionnaire as Kaf et al, in which a Likert scale of 1-5, low to high, respectively was used for the level of con dence evaluation.(3) The parameter was assessed at six different variables; (1) Level of con dence in identifying the external auditory canal, (2) Level of con dence in distinguishing normal and abnormal external auditory canal, (3) Level of con dence in identifying the tympanic membrane, (4) Level of con dence in distinguishing normal and abnormal tympanic membrane, (5) Level of con dence in practicing otoscopic examination correctly and (6) Level of con dence to diagnose common pathologies affecting the external auditory canal and the tympanic membrane. Scores for every variable ranged from 1-5, with a total score, ranging from 6 to 30 marks. The level of con dence for the respective ear anatomical sites; external auditory canal and tympanic membrane were analysed by taking the total scores from variables (1) and (2) for external auditory canal, and variables (3) and (4) for tympanic membrane, with a total score, ranging from 2 to 10 marks. The level of con dence was assessed at three different timelines; T1, T2 and T3.

Questions Assessing Diagnostic Ability
To measure subjects' diagnostic ability in common ear pathologies, 25 one best answer questions on common diseases of the ear were constructed. These questions underwent content validation by an expert panel, which consisted of one otology consultant with clinical experience of more than 15 years and three otorhinolaryngologists with clinical experience for more than 5 years. All the images used in these questions were taken from real patients using the Horus Digital Otoscope ® (Miis, Taiwan). Following content validation, these questions were uploaded into the Schoology™ (Schoology Inc. New York) application in an online questionnaire format. The questions composed of 8 common otological diseases (3 questions per disease) of the external and middle ear, and 1 question on the normal ear. A total of 6 out of 25 questions assessed the subjects' diagnostic ability on diseases of the external auditory canal. On the other hand, the remaining 19 questions assessed the subjects' diagnostic ability on diseases of the tympanic membrane. Each correct answer was given one mark and each wrong answer was given zero marks. The total score of the diagnostic ability on external auditory canal diseases ranged from 0 to 6 marks, while the total score for tympanic membrane ranged from 0 to 19 marks. The diagnostic ability was assessed according to the three different timelines; T1, T2, and T3.

Statistical Analysis
Descriptive statistical analysis on the demographic characteristics; gender, ethnic, and age were analysed by using percentage and frequency. Continuous data; the level of con dence and diagnostic ability were analysed by comparing means and standard deviation between the three different timelines: prior intervention (T1), after the intervention (T2) and four months following the intervention (T3). One-way ANOVA test with con dence interval 95% was used to compare means between 3 set of data: prior intervention (T1), after intervention (T2), and four months following the intervention (T3). The two study parameters were analysed respectively across the time of sampling and also according to ear anatomical sites (external auditory canal and tympanic membrane). A p-value of ≤0.05 was considered to be signi cant in two-tailed tests. All of the analysis was conducted using SPSS version 23.0 (SPSS Inc., Chicago, IL).

Demographic Characteristics
One-hundred and twenty-two nal year medical undergraduate students participated in this prospective interventional study. 45 out of 122 (36.9%) subjects completed all the study variables prior to intervention (T1), directly after the intervention (T2) and four months following the intervention (T3). All 45 subjects were recruited for this study. 77 out of 122 (63.1%) subjects were dropped out due to incomplete data.
Of the 45 subjects recruited, equal proportions were males (42.2%) and females (57.8%) with a p-value of 0.1482. The ethnic group distribution were signi cantly different, where majoritywere Malays (66.7%), and Chinese (24.4%). Age of the subjects ranged from 24 to 26 years of age, with the majority of the subjects aged 24 years old (68.9%). [ Table 1] The levels of con dence of the subjects were assessed at three different timelines during the study: prior intervention (T1), after the intervention (T2) and four months following the intervention (T3). One-Way ANOVA test was used to compare means of total score for level of con dence between the three timelines. :. The mean of total scores for level of con dence showed signi cant improvement from T1 at 20.87 (2.920) to T2 at 24.98 (3.265) and T3 at 26.44 (3.094). A signi cant difference in the distribution of the mean of total scores for con dence level was demonstrated with a p-value of <0.05. [ Table 2] Diagnostic Ability The diagnostic ability of the subjects was assessed at three different timelines during the study: prior intervention (T1), after the intervention (T2) and four months following the intervention (T3). One-Way ANOVA test was used to compare means of total score for diagnostic ability between the three timelines.  Two out of six questions assessed subjects' level of con dence in identi cation and examination of the external auditory canal during otoscopy. Another 2 questions assessed the subjects' level of con dence in the identi cation and examination of the tympanic membrane during otoscopy. Each question was assessed using a numerical scale of 0-5, with a score of 5 representing a high level of con dence.
To analyse the level of con dence for each ear anatomical sites (external auditory canal and tympanic membrane) in otoscopic examination, One-way ANOVA test was used to compare means between three different timelines during the study: prior intervention (T1), after the intervention (T2) and four months following the intervention (T3). Total scores for level of con dence were calculated according to anatomical sites (external auditory canal and tympanic membrane) separately for all the study subjects and subsequently analysed.  Table 3]

Diagnostic Ability
Six out of 25 questions assessed the subjects' diagnostic ability in diseases of the external auditory canal. Meanwhile, the remaining 19 questions assessed the subjects' diagnostic ability in diseases of the tympanic membrane. Each correct answer was given one mark and total marks for diagnostic ability in diseases of the external auditory canal and the tympanic membrane was calculated separately for all subjects.
To analyse the diagnostic ability in diseases of the external auditory canal and tympanic membrane, Oneway ANOVA test was used to compare means between three different timelines during the study: prior intervention (T1), after the intervention (T2) and four months following the intervention (T3).

Discussion
Otoscopic examination is crucial to diagnose otologic conditions, which are common in general practice. (4) However, studies have shown that teaching and learning of otoscopic examination are often inadequate at both the undergraduate and postgraduate levels.(2) Nowadays, with the rapid progression in multimedia technology, it has become easier to use web-based platforms for teaching and learning. Web-based teaching can offer many potential advantages over the traditional method of teaching (5). Web-based teaching can allow students to learn using graphic images, sound and video, accessing content anywhere else other than class, interactive learning by quiz or feedback, and to revisit the content for revision. Web-based teaching is also a low-cost teaching method with wide accessibility (6). Grundman et al. concluded that visual and audio was able to augment the learning of a particular examination skill. In their study, the students' examination skills and diagnostic ability showed improvement using web-based learning compared to revising printed lecture notes (5). In another study in 2016 by Stepniak et al, web-based learning was found to be effective in teaching otoscopy for undergraduate medical students (7). However, this study utilised a high end, and expensive simulator with a very limited follow-up period of only one week. In this study, the long term effectiveness of learning otoscopy through a newly developed online platform was assessed using online questionnaires on the level of con dence in the otoscopic examination and one best answer questions on diseases of the ear canal and tympanic membrane to test the diagnostic ability.

Level of Con dence
Student acquisition of knowledge and skills in otoscopy can be measured via students' self-evaluation opportunities. An example of such evaluation is self-reporting of perceived con dence level in performing the various tasks in the otoscopic examination. This study utilized such methods using a 5-point Likert scale to determine the level of con dence in the otoscopic examination. A similar evaluation was used by Kaf et al in their interventional study of thirty-two rst-year undergraduate audiology students (3). On the other hand, Swamy et. al. in 2014, assessed their subjects using a 4-point Likert scale in an interventional study on a simulated model (SimMan) to improve the con dence level in performing otoscopy (8). We feel that utilising a 5-point Likert scale is better compared to a 4-point scale to observe differences in subjects' level of con dence across the different sampling points in the study.
This study demonstrated a signi cant increment in the subjects' perceived level of con dence in performing otoscopy following the intervention. This increment is sustained at even four months following the study intervention, showing that a web-based learning platform may be an effective addition in teaching and learning of otoscopic examination. To the best of the authors' knowledge, no  (3,8,9) Comparing the subjects' self perceived level of con dence in external auditory canal and the tympanic membrane, we found that the nal year medical undergraduate students were less con dent in the identi cation and abnormalities of the tympanic membrane compared to the external auditory canal. This nding concurred with a similar study by You et al, who mentioned that it was challenging to assess tympanic membrane due to its location and inaccessibility compared to external auditory canal (9). This was re ected in our results where the mean level of con dence score to identify the conditions in tympanic membrane is slightly lower at T3 (8.84) than the external auditory canal (9.09).

Diagnostic Ability
Students' diagnostic ability can be measured by using questions with marks. The questions may include normal ear and common diseases of the external auditory canal and tympanic membrane, but must be validated by experts. The diagnostic ability of study subjects was similarly explored by previous authors  (10,11). Each correct answer was given one mark and the total marks attributed to the students' diagnostic ability (3). Thus, in this study, we determined the diagnostic ability by providing 25 validated one best answer questions given by a panel of experts. The questions composed of 8 common diseases (3 questions per pathology) of the external ear, middle ear, and 1 question about the normal ear. Each correct answer was given one mark.
A signi cant improvement in diagnostic ability scores on both the external auditory canal and the tympanic membrane was demonstrated in the present study. In line with previous study by Moberly et. al, the diagnostic ability of otoscopic examination increased after the intervention (11). The long term retention for diagnostic ability was also measured 4 months following the intervention. Similar to the level of con dence, the diagnostic ability in otoscopic examination among students was sustained at high scores over a period of four months. The results showed a signi cant difference in diagnostic ability scores when compared 4 months following intervention (5.20) to the baseline (4.38). The results obtained were similar to a previous study conducted by Dastjerdi et al who concluded that a multimedia program on teaching empirical science among second-grade students showed better retention in learning 3 months following the intervention compared to traditional methods (12). We postulate that this sustained improvement in the diagnostic ability scores is explained by the Kolbe model of experiential learning where lectures and small group training in the fourth undergraduate year provides concrete experience on otoscopy examination.(13) Patient experiences and case scenarios encountered during and outside of the formal posting may allow re ective observation, abstract conceptualisation and active experimentation, but lack of discussion and feedback mechanism may deter learners from re ecting and progressing on the experience ladder. Using the online discussion forum, timely feedback was given, juxtaposed with instructional content related to the educational materials provided in the platform itself.
This way, students are actively learning, which can be more effective for the student. This timely, immediate feedback mechanism provides reassurance and con rms progress among students, more so if it is made public and readily available. (14) This study has limitations to be acknowledged including the high rate of dropouts due to incomplete data. The researchers, however, managed to achieve the targeted sample size to obtain a considerably good study power of 80%. We recommend further studies to rigorously send reminders using more platforms including phone calls and provide more incentives so that subjects are able to complete the study outcome measures. All the excluded subjects were individually interviewed via a phone call to identify the potential factors contributing to their poor compliance to the study. Audio recording of these interviews were anonymised and analysed using thematic analysis. The main reason mentioned was not enough time to adhere to the given study time lines (70.84%) and did not notice reminders given to complete the questionnaires given (29.16%). Direct questions were asked regarding accessibility, ease of navigation and whether excluded subjects will continue to access the online platform despite being excluded. All excluded subjects agreed that the materials provided were accessible, it was easy to navigate the web-based platform and they continued to use the learning platform despite being excluded. Additionally, this study did not have a control group without intervention to compare the study outcome measures. A future randomized, prospective interventional study is needed to con rm and consolidate our study ndings.

Novelty of the study
This study developed a supplementary web-based learning platform for the medical undergraduates of Universiti Kebangsaan Malaysia which is tailored towards the needs of the current curriculum of the university as well as the local setting. Adapting a web-based learning platform developed by the Western countries may not only be costly, but may lack linguistic and cultural adaptations unique to the learner and their future patients. This study demonstrated long term positive effects on students' level of con dence and diagnostic ability which was not demonstrated in previously published interventions in teaching and learning of the otoscopic examination.(9, 10) A web-based learning platform may offer many potential advantages in the current teaching method and should be incorporated to improve the level of con dence and diagnostic ability in otoscopic examination among the undergraduate students.
Learning on otologic condition was provided by content experts in otorhinolaryngology in a formal threeweek posting. Study subjects assimilate the previously learnt knowledge with the patients' clinical condition after reviewing the educational video to improve their understanding of the otoscopic examination. The online discussion provided a platform for further discussion, inquiries and feedback.
Furthermore, the exibility of viewing the video repeatedly allowed discussion among peers. Thus, students were able to identify, plan, and react to the information during the discussion. Timely feedback by an experienced otorhinolaryngologist will construct new knowledge and understanding to achieve the intended learning outcomes.
The above process is related to social constructivist theory which emphasizes the role of social contexts of learning, and the knowledge which is mutually built and constructed. According to Vygotsky (15), social interaction between learners and other knowledgeable persons contributes to learning through the development of the cognitive domain (i.e., language, thought, and reasoning). Another important concept highlighted by Vygotsky is the role of Zone Proximal Development (ZPD) in which the students were able to solve their patients' problems due to the opportunity they had to communicate and collaborate with one another through online discussion with peers and feedback information from the content expert. Hence, two-way online interaction enabled students to identify and justify their weaknesses and strengths based on their existing knowledge.
How will this paper make a difference in medical education practices?
This study sheds light on more effective ways to train future doctors in the otoscopic examination. This skill is important to master before they graduate especially as they are preparing themselves to diagnose otologic conditions and manage these diseases independently in general practice. Failure in teaching and learning of otoscopic examination will contribute towards future problems such as misdiagnosis, antimicrobial resistance and, even morbid complications. It is appealing to the general practice as a signi cant proportion of patient presents with ear complain, the most prevalent being, otitis media with effusion which is reported to be as high as 18.3% among preschoolers. (16) In the light of the coronavirus pandemic where distant learning is becoming the norm, this paper provides more evidence on the effectiveness of using a web-based learning platform to supplement clinical teaching in medical education practices.

Conclusion
A web-based learning platform for otoscopy is an effective tool that complements the teaching and learning of otoscopy among nal year medical undergraduates. There are sustained improvements at 4 months in the level of con dence in performing otoscopy and diagnostic ability among the subjects following this intervention. This accessible and mobile learning platform may be a valuable addition to the undergraduate medical curriculum.

Declarations
Ethics approval and consent to participate: The study was approved by the institutional ethical review board for human research of University Kebangsaan Malaysia within which the study was undertaken (IRB code of approval: FF-2019-324). All study participants provided a written consent to participate in the study. Participation is voluntary and all students who ful lled the inclusion criteria were invited to participate.

Consent for publication: Not applicable
Availability of data and material: The dataset supporting the conclusions of this article is available in the corresponding author's repository at https://docs.google.com/spreadsheets/d/1qWoryXTjRENSEpRLM6PKy5YoaZblasrp3eTuBdzEUos/edit?
usp=sharing. Link to the dataset will be activated following a formal request made to the corresponding author after the manuscript has been published for duration of ve years to protect the con dentiality of the research participants, according to the institutional requirements.