Evaluation Of Online Development Examinations Of Infectious Diseases And Clinical Microbiology (IDCM) Expertise Students

Background: Formative examinations are required to monitor the development of research assistants in the field of Infectious Diseases and Clinical Microbiology. However, the preparation, conduct and evaluation of these exams also require especial features and experience. In this study, it is aimed to review the experiences obtained in the evaluation exams of research assistants conducted by the education commission (EMEK) in the field of IDCM specialization. Methods: There are two online exams per year for research assistants in the IDGM field. In this study, the examinations between 2014-2018 were evaluated. The validity and reliability analyzes for each exam were performed and the difficulty and discrimination indexes of the questions were calculated. The topics with the least correct answers were determined. Results: Over the years, an increase in the number of research assistants was observed. The internal consistency coefficient of the exams was the lowest of the cronbach alpha value of 0,69 and the highest of 0,92. The mean score of the exam was over 70. Some of the questions needed to be reviewed and corrected. Although clinically rare diseases and basic microbiology questions have been answered less frequently, the rate of responding to diseases that are frequently seen in the clinic was higher. Conclusion: Online examinations facilitate broad-based measurement and evaluation at national level. With the increase of our online exam experience, we found that there are still parts to be improved.

Formative evaluation is an intermittent assessment and evaluation while training is in progress. The aim is to determine the learning deficiencies of the learner (2)(3)(4)(5).
Online formative assessments in medical education have been increasing in recent years with the aim of measuring the knowledge of research assistants according to a national norm, determining minimum national standards in the specialization training program, and measuring the quality of teaching under the training program. The appreciation of the online exams is due to its encouragement of self-learning in medical education. Self-learning students can assess their work themselves, identify their strengths and weaknesses. Students who participate in the formative examinations are more successful in the grades-based exams. In this study, it is aimed to evaluate the learning objectives and examinations, and to report the experience obtained in the research assistant exams of EMEK.

Exam Method
The date and time of the exam will be announced on the internet in the platforms where every research assistant can be informed and the applications are taken online. Research assistants taking IDCM education in Turkey are participating in this voluntary examination. The research assistants who are applying for the test attend the exam by logging into the system with their own passwords in the clinic where they receive their education. The examiners are appointed by the responsible institutions of education. Assistant examinations are held twice a year with six-month intervals.
The scope and distribution of the questions are determined according to TUKMOS curriculum subjects and the questions are prepared by EMEK executive board in the form of multiple choice test format (5 choices). All questions are reviewed by EMEK executive board and questions are validated. The exams which are made online from a central system and consist of 50 questions, have a 1 hour duration.
Each question is 2 points, maximum exam score is 100 points. As this is an exam with formative purpose development, no passing grade has been determined. The exams are held at lunchtime. This timeframe appears to be a reasonable period that will not only disrupt the clinical work but also make the organization difficult.

After Exam Feedback
Feedback is given to each center after the exam. The result of the examination is not given to any person other than the research assistant himself and the education supervisor of the institution he / she is studying. Through the examinations, it is aimed to evaluate the professional knowledge development of the expertise students according to years and to evaluate the educational institutions' deficiencies according to these results.

Exam Statistics
The numerical data of the exams were obtained by reviewing retrospective data records. The data obtained were tabulated. Data were compiled as MS excel data.

Reliability and Validity of Exams
Cronbach's alpha reliability coefficient was calculated statistically. Student's T test was used to compare the means for countable values, chi-square and Fisher's corrected chi-square tests were used to compare the qualitative values. P <0.05 was considered significant.

Results
It was found that the number of research assistants who wanted to test themselves in EMEK exams increased gradually (Figure 1). In particular, the participation rate of the 1st year research assistants was found to be quite high ( Figure 2). The distribution of the research assistants, the mean grade point of the exams and the cronbach's alpha values are presented in Table 1. The cronbach alpha value of the exams was found to be lowest at 0,69 and highest at 0,92 (Table 1). In each exam period as a result of the exam analysis the most and least correctly answered questions were found and presented in Table 2.
The discriminant rate of the exam questions was 16% in 400 questions. 84% of the questions had to be corrected before being added to the question bank ( Table 3). The mean difficulty index of the exams was found to be at least 69 and at most 84.7, therefore it is determined to be in the easy question class. Table 3 presents the discrimination and mean difficulty index ratios of the questions according to the exams. Table 4  When the seniority of the students taking the exam is examined, it was observed that the first year research assistants participated in the exam in the most intense way and it was noticed that the number of research assistants who took the exam in the following years decreased according to seniority. This may be due to different reasons. This may be due to the fact that the first year assistants are more willing to test themselves and are in the clinic. Last-year assistants in the period of rotation or thesis preparation may be less motivated.
The online evaluation systems have many positive contributions to the education process. When appropriate infrastructure is provided to the web environment, it is accessible and applicable from anywhere. Students' achievements in exams can be seen instantly in the system (2). Online exams allow the creation of a question bank. The students who prepare the exam keep their knowledge upto-date and follow the subjects they will prepare for the exam carefully. Exams are done through the network, thus paper and publication expenditures are also eliminated (2,16). It takes a long time for classical examinations to be conducted and evaluated by academics. Thanks to online exams, trainers also save time. In addition, faculty members can look at the response rates of the questions and focus on subjects in that students' knowledge is insufficient, so that these subjects can be better understood. Individualization of the exam, instant feedback, quick scoring, and no limitations in terms of time and place are the advantages of the online exam (2,17). As a matter of fact, EMEK exams were conducted with one hour duration and many research assistants were evaluated online.
An exam is expected to be reliable, valid, useful and contributing to education. The fact that the margin of error in the exam is low means that it is a very reliable test (5). The measurement method should be valid and reliable (18,19). Cronbach's alpha reliability coefficient is mostly used to measure the reliability of a test. The higher the coefficient, the higher the internal consistency. For reliable tests, this coefficient is expected to be 0.80 or higher. It has also been reported that 0.70 or more can be accepted for formative exams (20). When EMEK examinations are considered, it is seen that the best value in reliability is taken in the first exam and the most negative measurement is taken in May 2017 exam. The high margin of error in this period is attributed to both the reduction of motivation in the examiners and the technical disadvantages in the preparation of the questions and in the implementation of the online exam. However, the reliability coefficient of the EMEK research assistant development examinations, which were made for formative purposes, was found to be quite satisfactory ( Table 1).
The discrimination index is an index used to separate those who do not have the characteristic to be measured by the test. Questions with high levels of discriminative features increase the reliability of the test. The discriminative feature of a question with a discriminant index of 0.40 or higher is defined as very high. The discriminativity of a question with index 0.30-0.39 is not complete, so it needs to be reviewed. The question with a 0.20-0.29 index should be corrected and developed, and should not be reused without any changes. 0.19 index of a question is low, the question must be removed from the test and not transferred to the question bank (20). In an exam made for the evaluation of research assistants, the expected feature of the questions is to distinguish the student who does not know the subject matter (20). When we look at the most of the EMEK examinations, although the discrimination rate is sufficient in some questions, it is necessary to review many questions ( Table 2). In the examinations, the question rate which can distinguish the student who does not know the subject with the relevant subject is found to be 16%. However, the percentage of questions that should not be used after reviewing is 43% in total. 41% of the exam questions should be reviewed and included in the question bank after making the necessary corrections. These data suggest that it is necessary to receive specific training for measurement and evaluation during question preparation.
In the analysis of exams, the investigation and interpretation of why the difficulty of the questions may have been caused is a gain of the evaluation (2,17,20). Table 4 presents the subjects in which the students have difficulty in answering correctly and have lack of knowledge. The remarkable result in this table is that less common or rare infections and basic medical science questions are answered less accurately. This will be the educational gain of the units with particular attention to these topics in the training process.
In the online applications, problems such as the emergence of unexpected technical problems, infrastructure and hardware dependence, security problems, students' use of different computers and especially requirement of having a computer are the limitations of online exams. No matter how good the network is, there may be some problems such as power failure or voltage change, lack of features of the computer being used. In addition, although online exams are strictly assigned to supervisors, cheating is much easier than classical exams. For this reason, the security of the system, (the user codes must not be created on the systems) communication security, difficulties in identification of the user and cheating affect the reliability of the online examination negatively (2,17).
It is the limitation of the EMEK examinations that the number of questions in categorized subject titles are not determined by taking into consideration the learning objectives of the IDCM.

Conclusions
As a result, the EMEK research assistants examinations conducted twice a year contribute to the evaluation of the training of the research assistants, but the examinations have aspects to be developed. Online exams make it easy to measure. With the online exams to be made, it will be easier to monitor the learning progress of the research assistant by monitoring the cognitive development of each assistant and reviewing the training programs.     Figure 1 Number of students taking EMEK exams over the years The number of students taking exams according to their years being as an assistant