Perspectives and Attitudes of Emergency Medicine Educators Towards E-Assessment Technologies

Background: Distance assessment and evaluation(E-assessment) has recently regained popularity with the widespread use of distance education due to the pandemic. Emergency medicine education has developed theoretical and practical models on online learning and accelerated the studies on E-assessment of competencies on emergency care. While the educatiors and trainees are promptly adapting to the distance learning platforms, several factors can affect these adoptions to e-assessment models. Our study aimed to nd these factors affecting the perspectives and attitudes on the practice among emergency medicine educators. Results: A survey was conducted on ninety participants of emergency medicine educators regarding self-ecacy, self-condence, and attitudes towards E-assessment technologies. There was a statistically signicant difference between computer use skills and scores, especially regarding self-condence (p=0.02) and self-ecacy (p=0.01). We found no signicant difference in terms of attitude (p=0.877). We found that those with very good computer skills had higher self-condence and self-ecacy for distance exams. The participants' computer use skills affected self-condence and self-ecacy for e-assessments. There was a signicant difference in self-condence (p=0.000) and self-ecacy (p=0.001) between those with and without e-assessment experience. Furthermore, the thought that it is an effective assessment and evaluation. Regarding screen sharing, 44.4% of the participants have ethical concerns; 30% were not concerned about this issue, and 25.6% were partially concerned. There was no statistically signicant difference in ethical anxiety regarding e-assessment experiencing (p=0.914). Conclusion: The prociency in using computers, experience in teaching, and the need for training on e-assessments are the key factors that affect e-assessments in emergency medicine education. the


Introduction
Distance education delivers education to teachers and learners geographically distant from each other using different technologies (satellite, video, audio, graphics, computer, multimedia technology) [1,2]. The global epidemic of coronavirus disease (COVID- 19), which started in the Wuhan province of China in December 2019 and affected the whole world and declared a pandemic by the World Health Organization [3], has also accelerated the increasing trend of remote / internet-based training [4].
In addition to distance education, face-to-face assessment and evaluation exams have also shared the same consequence, bringing the necessity of remote assessment and evaluation to the agenda. Electronic examination technologies (E-assessment/Web-based assessment) allow examinations to be carried out electronically through the web/internet [5,6]. These methods allow the exams for distant learners according to traditional exam methods. Besides, being fully automated systems also have advantages such as increasing the validity of the evaluation, increasing the reliability of scoring and test results with deep analysis, providing fast feedback, thus ensuring the rapid correction of errors, marking, moderation, data storage, and management, and therefore workload reduction [5]. Conceptual framework Self-con dence is de ned as "con dence in oneself and one's powers and abilities which is affected by the experience of being successful and will affect the person's perception of achievement [7][8][9]. Selfe cacy is de ned as almost nearly but a perception of achieving speci c tasks or situations [10]. This concept is created in the social cognitive theory by Alber Bandura [11,12]. Self-e cacy is different from self-con dence in managing organizational skills, but self-e cacy is a determinant for self-con dence. In the educational framework [13,14], regarding the social cognitive theory, self-e cacy connects the con dence of educators and learners to participate in activities to achieve the goals [13]. Besides, the effectiveness of education is related to educators' self-e cacy [15][16][17][18]. However, little is known about self-e cacy and self-con dence relation with attitudes towards assessment and measurement technologies. One of the studies shows that attitudes are grown with self-e cacy, which increased selfcon dence [19].
Users' technological readiness and attitudes are important factors that may affect the proliferation of eassessments. In the literature, readiness in distance education is de ned as the individual's ability to use e-learning resources and multimedia technology to improve the quality of learning [20]. Concepts related to e-learning readiness include self-e cacy, internet self-e cacy, online communication self-e cacy, selfdirected learning, learner control, and motivation for e-learning [20][21][22][23]. Preparation of the exams and the administration of the exams require time and self-e cacy in this regard [5,6,21,[21][22][23].
Distance Education, Assessment and Measurement in Emergency Medicine Education: Pre-and post-graduate emergency medicine education has also risen, supported by free open access medical education sites (FOAMed), podcasts, and web-based seminars. Especially during the pandemic period, it has become widespread that emergency medicine residency training and clerkship training for medical students moved to the distance education platforms. In this case, summative assessments of the students or residents who completed the theoretical training were brought to the agenda. However, there is no suggestion for distance online education and assessment in the curriculum of emergency medicine education both in Europe and in our country.
This study aimed to evaluate the experiences, perspectives, self-con dence, self-e cacy, attitudes, educational needs, and concerns of emergency medicine educators using distance assessment and evaluation technologies. In this way, it is planned to identify emergency medicine educators' opinions on distance assessment and evaluation technologies (E-Assessment), their needs about technologies, and ethical and anxiety issues and offer solutions. Besides, it is thought to guide assessment and evaluation studies to develop quickly applicable e-assessment platforms for emergency medicine educators.

Materials And Methods
The study protocol for involving emergency medicine educators was in accordance to guidelines of institutional ethical guidelines and Declaration of Helsinki. This prospective cross-sectional qualitative study was held between 01-31 October 2020 after the approval of the Institutional Research Ethics Committee.
In this qualitative study, educators of emergency medicine residency training programs with different academic ranks (Professor, Associate Professor, Assistant Professor, Lecturer, and Specialist) work in an emergency department accredited by the national authority for emergency medicine residency training. The educators who have been engaged in education and training activities and who have completed emergency medicine residency training in emergency medicine were included. Emergency Medicine residents, educators in departments other than emergency medicine residency training, and nonemergency physicians working in emergency medicine training programs were not included in the study.
The educators with experiences of e-assessment in other educational activities (not relevant with emergency medicine education) and those who did not give consent were excluded from the study.
Since there is no face-to-face meeting opportunity due to the pandemic, an online questionnaire was created. The data were obtained through semistructured/structured questionnaire questions prepared on web and mobile-based platforms, with the questionnaire delivered to the participants via e-mail and academic, social groups. Questionnaire questions in studies [4,[22][23][24][25][26] where reliability and validity analyzes were applied were modi ed and developed for emergency medicine educators. A 7-point Likert scale was used in the questionnaire.
To evaluate whether the questionnaire form was clear and understandable, 17 emergency medicine educators with different academic ranks were pilot tested, and minor changes were made according to the feedback.
Secondary endpoint: Do emergency medicine educators' attitudes, self-e cacy, and self-con dence in distance assessment and evaluation technologies (E-Assessment) differ according to their experiences?
Statistical Analysis: Numerical variables were represented as mean±standard deviation or median (min-max). Categorical variables were presented as absolute values and percentages. Kolmogorov-Smirnov and Shapiro-Wilk tests were used to evaluate the distributions. Independent groups were assessed using the independent ttest and the Mann-Whitney U test, and the Kruskal-Wallis test for comparisons of more than two groups. The relationship between categorical variables was assessed using the chi-square test. Spearman test was used in correlation analysis according to the distribution of the data. The statistical signi cance level was set as p <0.05. Statistical analysis of the data was performed with IBM SPSS version 23.

Reliability Analysis:
The reliability of the questionnaire was calculated by the Internal Consistency test Cronbach's Alpha coe cient. The reliability coe cient was calculated separately for Self-E cacy, Self-Con dence, and Attitude. The Cronbach's alpha coe cient was 0.70 and above for all measurements, and the questionnaire was considered reliable. The reliability analysis for the questions created in the questionnaire is shown in Table 1. Kaiser-Meyer-Olkin and Bartlett tests of self-e cacy, self-con dence, and attitude-related factors have been performed; the Varimax rotated factor analysis method was performed in the test and presented in Table 2.

Results
At the time of the study, 99 emergency medicine residency training programs were authorized by the Ministry of Health's Board of Medical Specialties to provide emergency medicine specialty training in our country. Undergraduate medical education was also provided in 71 of these institutions. During this period, the number of emergency medicine educators who met the inclusion criteria was about 400. The questionnaire was shared through the national emergency medicine specialty association, through social media groups, and within clinics. Five of the 95 responses did not agree to participate at the beginning of the questionnaire. Thus the data of 90 participants were included in the analysis.

Computer Skill Levels
In terms of computer use, 51.1% (n=46) participants are at a good level; 21 of them (23.3%) are very good; 21 participants (23.3%) stated that they used a medium-level computer. Only 2 participants stated that they were at a baseline level on computer use.

Distance Education Experiences:
94.4% of the participants stated that they have been teaching by distance education in their clinics. Distance education was not provided in ve centers. It was determined that the delivery of emergency medicine residency training through distance education started in 7-11 months with a rate of 56.7% and in 0-6 months with a rate of 36.7%. The target audience of emergency medicine education was mostly emergency medicine residents (83.3%; n=75), emergency medicine specialists (44.4%; n=40), later medical students, and other health personnel. It was determined that 75.6% of the distance learning experience was also in congresses/symposiums and conference activities and 65.6% at the emergency department lectures. The most frequently used distance education platforms were Zoom Client for Meetings (77.8%) and Microsoft Teams (41.1%).

E-Assessment Experiences:
The proportion of participants who experienced an e-assessment was 47.8% (n = 43). While the rate of taking part in the distance exams was 77.4% (n=14), it was found that these educators mostly worked in preparing questions (58.1%; n=25) and evaluating the results (30.2%; n=13). The number of those with supervision experience was found to be 6 (14%).
It was determined that the distance exams were mainly used for medical school students' clerkship exams (83.7%; n=36), 34.9% were used for seniority exams of emergency medicine students, and 20.9% were used for the nal examination in emergency medicine residency training. It was determined that the online (53.5%) and o ine (34.9%) tests with multiple-choice questions were the most frequently used remote exam type in clinics. It was observed that the examination duration was mainly between 0-60 minutes (83.7%; n=36). 95.3% of the exams were made accessible from anywhere.
Platforms, where examination evaluations (grading) mainly were calculated automatically by the eassessment platform (74.4%), were at the forefront. 69.8% of the participants with remote test experience stated that audio and video recordings were not taken during the exams, 32.6% stated that only video recordings were taken, and 18.6% of them stated that only audio recordings were taken. The rate of invigilation in online theoretical exams was 44.2% (n=19). The rate of using a question bank/pool in specialty training in medicine was 48.8%, and all participants (n=21) using this pool could also use this pool for remote examinations.

Advantages
Participants believe that the most signi cant advantages of taking the exams are as follows: The participants thought that 72.2% (n=65) of multiple-choice questions and 53.3% (n=48) of multiple true/false question tests were the most suitable exam type for the distance exam. According to the purposes of the exams, the participants believed that the types of summative assessment (56.7%), diagnostic assessment (made at the entrance to the program) (50%), and formative assessment (47.8%) types were more suitable for holding remote examinations.

Concerns
Those who thought that an effective assessment and evaluation could not be done with e-assessment were 13.3% (n=12), 48.9% (n=44) who thought it could be done partially, and 37.8% (n=34) who thought it could be done absolutely.

Security and Ethical Concerns
The rate of ethical concern about video and audio recording about remote exams were found to be 48.9%. The rate of those who were not concerned about this was 26.9%, and those who were partially concerned were 24.4%. Regarding screen sharing, 44.4% of the participants have ethical concerns; 30% were not concerned about this issue, and 25.6% were partially concerned. There was no statistically signi cant difference in ethical anxiety between those with and without e-assessment experience (p=0.914).
Age, experience in emergency medicine was not signi cantly associated with ethical concerns on video and voice recording or screen sharing (p>0.05).

Need for Education
Although there was no statistically signi cant difference between those with and without e-assessment experience (p>0.05), the participants in both groups thought themselves, students, other colleagues, and administrators needed to receive training on e-assessment.
Factors affecting the Self-Con dence, Self-E cacy and Attitude Scores No statistically signi cant correlation was found between age and self-con dence, self-e cacy, and attitude scores (p>0.05) (Table3). There was no signi cant difference among genders in terms of attitude (p=0.117), self-con dence (p=0.052), and self-e cacy (p=0.224).There was no statistically signi cant difference between the academic ranks in terms of scores (self-con dence = 0.170; self-e cacy = 0.772; attitude = 0.respectively).There was no statistically signi cant relationship found between scores and experience as a specialist and trainer (p>0.05) ( Table 3). No signi cant difference was found among the institutions regarding scores (self-con dence = 0.158; self-e cacy = 0.169; attitude = 0.828). While Age, gender, institution, experience groups did not differ regarding scores. There was a statistically signi cant difference between computer use skills and scores, especially in terms of self-con dence (p=0.02) and self-e cacy (p=0.01), but no signi cant difference was found in terms of attitude (p=0.877). Medium, good, and very good levels are statistically higher than baseline levels regarding self-con dence and self-e cacy. There was no difference between medium and good levels. However, very good levels are showed higher self-con dence and self-e cacy scores than medium levels. Also, no statistically signi cant difference was seen between very good and good levels of computer use regarding self-con dence and self-e cacy. Computer skills were found to affect selfcon dence and self-e cacy for e-assessments.
In terms of computer self-e cacy, the median level of problem-solving without technical support during use was 5. There was a correlation between the scores and computer self-e cacy (p<0.05) (Table3). Computer self-e cacy was poorly correlated with attitude (rho=0.324) but positively and signi cantly correlated with self-e cacy(rho=0.463) and self-con dence (rho=0.403).
There was a signi cant difference in self-con dence (p=0.000) and self-e cacy (p=0.001) between those with and without e-assessment experience. Self-con dence and self-e cacy of those with e-assessment experience were found to be higher than non-experienced participants. Experience of e-assessment made a statistically signi cant difference on self-con dence and self-e cacy.
Self-con dence (p=0.005), self-e cacy (p=0.001), and attitude (p=0.000) were found to be statistically signi cantly higher in those who thought e-assessment is effective. Although no signi cant difference between gender (p=0.518) or academic rank (p=327) groups, perception of effectivity showed a positive relationship between the increase in self-con dence, self-e cacy, and attitude of e-assessment and the thought that it was a practical assessment and evaluation.
The participants who do not have ethical concerns on video and voice recordings have signi cantly higher self-e cacy scores (p=0.002). Screen sharing concerns also have a statistically signi cant association with self-e cacy (p=0.028) and self-con dence (p=0.027), in which participants without concerns on screen sharing were more self-e cient and self-con dent.
The summary of the results can be visualized in Figure 1.

Discussion
Our study aimed to evaluate emergency medicine educators' self-e cacy, self-con dence, and attitudes on e-assessments during the COVID-19 pandemic. The study revealed that 93.4% of the distance education experiences started during the COVID-19 pandemic. During this period, when social distance gained importance, the duration of experience of the study participants was determined to be less than one year.
Electronic assessment technology in higher education institutions has been rapidly gaining acceptance in the last 20 years. E-evaluation is implemented in various ways as electronic reports, electronic portfolios, blogs and forums, electronic exams. There are two tools used for e-assessment purposes: Learning management systems used to provide teaching, learning, and assessment in educational institutions and large-scale computer-based assessment technologies designed for assessment purposes only [27].
Despite the development of a large number of e-assessment technologies and their advantages, it is clear that these technologies did not receive intense attention, as they were not required until the COVID-19 pandemic.
When considering the studies on e-assessment, it is seen that the educators are generally familiar with computer technology and prefer e-assessments to paper and pencil-based ones [28]. Likewise, we found that high computer use skills, computer use self-e cacy are correlated with high self-con dence, selfe cacy, and attitudes towards e-assessments.
Educators nd computer-based exams favorable. It has been stated that women and higher academic ranked educators are more positive about these exams than the less experienced educators [29]. Although the perception of effectivity was not statistically associated with gender or academic rank in our study, women and lower academic ranked educators found e-assessment more effective for assessments than men and higher academic ranked educators. We thought the distribution of the women in higher academic ranked educators might cause this result. Self-con dence, self-e cacy, and attitude to the eassessment also increased the belief that the e-assessments were effective. This situation shows a positive relationship between the increase in self-con dence, self-e cacy, and attitude of e-assessment and the thought of an effective assessment and evaluation.
A positive correlation was found between the use of information and communication technologies and the educators' competence and the importance of educators' volunteerism and compliance for eassessments. Besides, it has been found that there is an inverse correlation between the use of information and communication technologies and age, teaching experience [30,31]. We found no statistical correlation with age, teaching experience, and attitudes. That may be because of the acute necessity of using e-learning and e-assessment platforms due to pandemic circumstances that did not give a chance to discuss the preferences of educators. It has also been reported that time is needed for educators to understand the functioning of e-assessments, but the experience is the key to use [32]. In our study, the experience is limited with a one-year pandemic period; we believe that the increase in experience had a strong effect on disseminating the e-assessment, which produces self-con dence and self-e cacy levels of educators.
In general, there are various advantages and disadvantages of online e-assessments [33]. Advantages: There is no need to print any exam documents before the exam.
There is an option to sort questions to prevent duplication randomly.
The results of the exam can be obtained immediately after the test is completed.
Archiving of questions, past results, and student pro les can be done in one place.
It is possible to evaluate a student several times and create additional exams based on their wrong answers in previous exams.

Disadvantages:
One computer is required for each student in the class.
There is the possibility of cheating, which is prevented by designing the computer room. The possibility of accessing other computer resources should be disabled.
Checks should be made just before the assessment to ensure that each computer is operational and connected to the internet.
If many students take the exam simultaneously, powerful servers are needed to process the amount of information recorded in the system. Problems will arise in case of power failure or server failure.
Training of educators using the system requires time and workload.
The most frequently used method is online and o ine exams with multiple-choice questions, and the exam types they consider the most appropriate types are also exams with multiple-choice and multiple true/false questions. However, there is no ideal type of method to be used in e-assessments in emergency clinical practice.
It has been observed that the concerns about video audio recordings and screen sharing do not decrease with experience. Besides, the participants thought that they and colleagues in the education environment should need training. The pandemic may have also led to an acute and unplanned occurrence of elearning and e-assessment processes to ensure continuity of education. This situation may have affected the participants' self-con dence and caused an increase in the need for training.
Due to the COVID-19 process, studies on attitude, readiness, and self-e cacy towards distance education, especially for medical students, are available in the literature [33][34][35]. It is thought that the self-e cacy and readiness of academic staff affect the success of distance education [35,36]. It is stated that as the experience in distance education increases, the effectiveness increases at the same rate [35]. Although our study was not an effectiveness study, it was found that experience affects self-con dence and selfe cacy. Studies on distance assessment and evaluation are limited. Although speci c criteria are tried to ensure the standardization and control of assessment and evaluation technologies, there is no study in the literature on the opinions, concerns, perspectives, and readiness of educators for assessment and evaluation in emergency medicine residency training.
In a study evaluating the effectiveness of an emergency medicine education program that integrates webbased learning into classroom sessions, the success and satisfaction of students who completed more than 75% of the web-based module were found to be statistically higher [36]. It has been stated in the studies that the educators are not comfortable with how to include students in the online environment, causing di culties in encouraging participation and an increase in the need for education [35].

Limitations:
Our study has a few limitations. The number of participants is only one-fourth of the total number of educators. As well as an opportunity for distance education and e-assessment, we think that the pandemic period affected the survey participation rates due to the workload in emergency departments.
Considering that the distance education experiences are less than one year, the distance assessment and evaluation experiences are only half of these experienced educators. The participation may be less since the survey was made at the beginning of the adaptation to distance learning.
In our study, experiencing the e-assessments with multiple-choice questions might affect their selfcon dence, self-e cacy, attitude, and anxiety levels. Further studies should also include a comparison of the types of e-assessments applied. Although this study's population includes the experiences of the eassessments given in the hospitals accredited with emergency medicine residency training, it does not include the exams experienced or taken remotely, such as board exams and certi cation exams. This situation can also be taken into consideration in further studies.

Declarations
Ethics approval and consent to participate: The study conducted after the approval of the Local Clinical Research Ethics Committee(Bakirkoy Dr Sadi Konuk Training and Research Hospital; 2020/372-24/08/2020). The participants are asked to participate at the beginning of the survey and recorded their consent to participate.

Consent for publication: NA
Availability of data and materials: The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
Competing interests: The authors have no con ict of interest.

Funding: None
Authors' contributions: G.BB. and B.I wrote the main manuscript text with supervision and revision of the drafts of C.O. All authors contributed data collection and analyze the results.