Exploring the cognitions of Iranian university TEFL teachers and students of the efficacy of online assessment practices in the wake of Covid 19

doi:10.21203/rs.3.rs-1523197/v1

Download PDF

Research Article

Exploring the cognitions of Iranian university TEFL teachers and students of the efficacy of online assessment practices in the wake of Covid 19

https://doi.org/10.21203/rs.3.rs-1523197/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

As a deadly pandemic, COVID 19 has not only presented public health challenges, it has also disrupted education all over the world. To prevent the spread of the virus, like many other countries, the Iranian universities were closed and converted their system into distance education. Since the educational centers have already delivered two years of on-line education, there is a need to address the quality of online assessment as part of the attempt to gauge the accountability of online pedagogical practices. Thus the current study aimed to explore the perceptions of Iranian English as a Foreign Language (EFL) university teachers and students on the implementation of online assessment (OLA) during the pandemic. As such, 212 Teaching English as a Foreign Language teachers and 325 postgraduate students completed the Teachers’ Views Towards Implementing Online Assessment Survey developed by the researchers. The findings of the current study suggest that on the whole, both Iranian EFL teachers and students favor the web-based form of assessment. In addition, there were similarities in perceptions of the teachers and students regarding familiarity with OLA and its feasibility, OLA contribution in improving learning and feedback processes, the worth and value of implementing OLA. However, the findings also highlighted differences in the inadequacies of OLA.

EFL teachers

Online assessment

Covid-19

challenges

The unnoticed extensive closures of educational institutions from late 2019 onwards were a logical reaction to the burgeoning COVID-19 pandemic. Although the unprecedented conditions seem to be ameliorated now, however, the majority of schools and higher education institutes are still physically closed which cast serious doubts on the accountability issues of distance education as far as the assessment techniques and procedures are concerned. One obvious reason for the nationwide schools and universities lockdowns in Murphy’s (2020) terms had to do with the idea that the education systems were viewed as threats to securitization at the time of Coronavirus pandemic. On the face of the fact that online assessment has been extensively utilized in education systems universally, there are still concerns about the reliability of the assessment tools and the validity of the students’ scores on such tests. The enigma hinges around the idea if current online assessment techniques yield critical information about students’ academic performance and achievement.

Opposed to the dynamic interaction between the teachers and students in real classes, distance education inherently deprives the teachers from genuinely interacting with the students (Dhawan, 2020). That is, the possibility of immediate on-task counselling and feedback on teachers’ part is considerably minimized in online education. Things get further exacerbated when it comes to online assessment where teachers can not directly observe the ways in which the learners engage in and interact with various test forms and items. Taking logistic elements of online assessment such as internet access/speed and technology literacy for granted, there are still assessment-specific issues that are more likely to influence students’ performance (Martin, 2020). Clarity of test instructions, time constraints, response formats, and cheating are among such issues. Under such circumstances, there are also apparent discrepancies between stakeholders, teachers, and students’ views on the most practical and pertinent online assessment procedures (Gamage, de Silva, & Gunawardhana, 2020). Consequently, data can be gathered from stakeholders, teachers, and students to obtain useful information towards fulfilling students’ assessment-oriented needs without which unrelenting disruptions will most allegedly aggravate until learners can return back to the classroom.

A good specimen of distance education system in Iran, with its root in the Electronic branch of Islamic Azad University (IAU), is the virtual education system known as Vadana. IAU is a non-governmental university and the world's sixth-largest university. It has been set up in order to integrate and improve the quality level of distance and online education at the IAU. With the inception of Covid 19, the first wide scale online exams in the form of semester-end summative tests were administered throughout the country in all IAU branches. In general, implementing online courses has had its advantages as well as challenges. Since this was literally the first large-scale trial in the country, it seems necessary to explore university teachers’ perceptions of web-based assessment in general and EFL university teachers in particular in the wake of COVID 19 as to the effectiveness of the online assessment. Thus, the present research grew out of a desire to investigate EFL teachers’ perceptions of the efficacy of online assessment in IAU.

Since its inception, the COVID-19 pandemic has affected education system worldwide and it has resulted in the partial or full closure of schools and universities across the globe. To counterbalance the negative effects of the contagious disease, education has changed dramatically and with the rise of e-learning, teaching is undertaken on digital platforms remotely. With this sudden shift away from the physical classrooms in many countries across the world, the situation, as Murphy (2020) argues, turned to a situation called ‘emergency e-learning.’ The term is defined as “the temporary shift of instructional delivery to an alternate delivery mode due to crisis circumstances” (p. 6). In such a situation, accustomed to face-to-face classes, Iranian teachers have to totally depend on online courses as their only choice to teach, as such they have faced tough days (Badrkhani, 2021). Iranian educational institutions and universities launched online education systems and changed classes from face-to-face to online courses.

As a platform for online learning, the Learning Management System (LMS) which had already been introduced to the context of Iranian educational system in 1990 in a very small scale (Mahmoudi-Dehaki et al., 2021) was embraced by majority of universities (Ebadi, et al., 2020). One such system, Vadana, is used by IAU. As a private university, Islamic Azad University is among the largest universities in the world which is scattered mainly inside Iran. IAU has updated and improved its electronic infrastructure and systems with a large number of students and educational units scattered throughout the country and in different cities to facilitate educational and student affairs.

Despite the fact that the situation is far from ideal, IAU was better prepared to transition to the online learning environment, as it had already launched its Electronic Campus and students from the country and abroad had been studying through online courses. Despite the benefits, such as accessibility to the course from anywhere at any time, asynchronous discussions with teachers and classmates, immediate feedback on tests, and so on (Ahmady et al., 2020), online learning is fairly different from the one in face-to-face courses due to its particular nature. Although it seems that it is not likely to predict the exact time of the end of pandemic (Ebadi et al., 2020), if properly utilized, such an opportunity may turn into a valuable asset to the educational systems and for students to get involved in learning and knowledge sharing if timely decisions are made.

A very important point is the assessment of what students have learned in the online classroom system. Tests provide critical data in both face to face courses and online courses to achieve two definite goals: to secure accountable teaching and learning and to guide decisions on subsequent academic measures (Jimenez, 2020). As she argued, the data required to meet these purposes come from three major types of assessment, which yield different yardsticks. Diagnostic tests are administered at the beginning of the academic year to determine learners’ starting levels and help the learners themselves as well as the teachers access appropriate graded content. Formative tests offer teachers touchpoints throughout the academic year to help them tailor their attempts to enhance learning. Finally, year-end summative tests, as Jimenez (2020) contended, provide information on how successfully the learners progressed toward preordained goals and academic standards.

Additionally, not unlike traditional teaching and assessment that is primarily teacher-centered, the ultimate aim of online teaching and assessment practices, as a more student-centered form of education, is academic attainment and accountability. Despite the wide range of possibilities online learning offers for presenting and sharing teaching materials that facilitate the learning process, it still proffers practical concerns when it comes to assessing students’ performance (Zakaryia, Khaled, & Omar, 2021). While evaluation is fairly exclusively carried out by the teachers in traditional assessment and its efficacy draws substantially on teacher’s knowledge and skills, it is primarily performed with the application of tools and systems that mediate between the students and the tests. Hence, students’ performance on online tests is influenced not only by their competency in the subject matter, but also by their nimble control of these tools and systems as a facet of test methods. The situation further exacerbates once assessment security is taken into account as a determining factor in remote assessment. Assessment security, as Gamage, de Silva, and Gunawardhana (2020) argued, aims at securing assessment practices against issues such as cheating, and on identifying any instance of cheating that might have occurred. Gamage, de Silva, and Gunawardhana (2020) pointed out that a vital element in assessment design is to safeguard the idea that “assessments enable students to demonstrate their learning practically” (p. 5). It seems that one of the big challenges in front of the teachers is how to prevent cheating on online exams.

While previous studies have mainly focused on the efficacy of online teaching courses (e.g. Abdel-Rahim, 2021; Coman, et al., 2020), research specifically targeting online assessment is scarce. Meccawy, Meccawy, and Alsobhi (2021) explored the views of students and faculty members towards online assessment. As they reported, there is need for the problem of cheating in online courses. Such a measure includes “raising student awareness and ethics, training teachers to detect cheating methods, and institutions activating their code of practice and applying severe sanctions on those who engage in such practices” (p.1). Verhoef, Yolandi, and Coetser (2021) studied academic integrity of university students during COVID 19. As the researchers found, due to pandemic-related issues the students were dishonest, however, there were some other reasons for their dishonesty like feeling overwhelmed and stressed and struggling with technology lack of time management.

Ghanbari and Nowroozi (2021) conducted a study in the context of Iran to examine 20 Iranian English language teachers’ views towards the online assessment challenges during COVID-19. They carried out semi-structured retrospective interviews with the teachers at Persian Gulf university at different intervals throughout an academic course. The findings of this study pointed to idea that after the move to online assessment, the teachers primarily encountered acute instructional, technical, administrative, and affective barriers; however, they could gear their actions to the demands of the new situation as the course went on.

Statement of the problem

As Iranian universities began to address the Covid—19 pandemics in 2020, the inclination towards replacing exams with online assessment hastened. Based on the researchers’ experience, the fast-paced transition at scale to online exams which was for the time of emergency did not meet the guidelines necessary for administering university exams. Concerns for academic integrity from one hand and the possible negative washback effect encouraged the researchers to investigate TEFL teachers’ and postgraduate students’ cognitions on the circumstances.

Apart from the benefits such as immediate feedback ( Bonham et al., 2000), improving the quality of student learning ( Saricoban, 2013) and more frequent administration of assessments ( (Bull & McKenna, 2004), such a rapid transition to remote assessment could give rise to various concerns such as technical issues, students’ technological incompetence and academic integrity (Khan & Khan, 2018; Guangul, et al, 2020; Zamani et al., 2015), online proctoring (Sullivan 2016), test being limited to objective-type formats like multiple choice (Bull & McKenna ,2004).

Although these studies assessed the efficacy of online assessment from the viewpoint of either teachers or students, literature is scarce when it comes to the comparison of the perceptions of teachers and students especially in the wake of Covid-19 pandemic. Accordingly, we formulated the following hypotheses:

H₀1: Iranian TEFL instructors do not have a positive attitude towards the efficacy of online assessment during COVID 19.

H₀2: Iranian TEFL postgraduate students do not have a positive attitude towards the efficacy of online assessment during COVID 19.

RQ3: There is not any significant difference between Iranian TEFL instructors and TEFL postgraduate students’ perceptions about the efficacy of online assessment during COVID 19.

The aim of the present descriptive study was to examine the cognitions of Iranian university TEFL teachers and students of the efficacy of online assessment practices in the wake of Covid 19. The data were collected through a questionnaire and analyzed quantitatively.

Sampling and sample size

As for the participants, we recruited 325 postgraduate students including TEFL M.A. and Ph.D. students. All students were selected on the basis of purposive sampling technique (Teddlie & Yu, 2007) since we believed that they could provide us with reliable information. They comprised 114 males (35%) and 211 females (65%). The selection criteria for 212 TEFL teachers were made based on their experience of teaching two executive terms after the inception of COVID 19 via online courses, having Ph.D. degree and having the experience of teaching TEFL at both BA and MA levels for more than 10 years. There were 143 males (67%) and 69 females (33%). For the second phase of the study, 23 teachers agreed to be interviewed over the phone.

Instruments:

The efficacy of online assessment questionnaire

Based on the related literature (Attia, 2014; Sharadgah & Sa’di, 2020), we developed a scale to measure the efficacy of online assessment questionnaire in TEFL courses (Appendix). The questionnaire includes 19 items and four subscales: (1) familiarity with OLA and its feasibility (1–4), (2) OLA contribution in improving learning and feedback processes (5–8), (3) the worth and value of implementing OLA (0–14), and (4) shortages of OLA (15–19). In addition, a 5-point Likert scale ranging from strongly agree to strongly disagree to identify both TEFL teachers’ and students’ perceptions toward online assessment.

Development of the instrument - the efficacy of online assessment questionnaire (EOAQ) was initially undertaken in two phases. Phase 1 - Based on the literature research, a list of 35 items was compiled and distributed for pre-validation review and comments to five expert researchers in the TEFL, and fifteen postgraduate students of TEFL at a state university in Iran. We asked the experts and the students provide a critical review of the items regarding the clarity, and fitness. Phase 2- After taking experts and students’ comments into consideration, the remaining 27 items were subjected to exploratory factor analysis (EFA). The first results of Kaiser-Meyer-Olkin (KMO) 0.61 was not satisfactory. After we eliminated 8 items that suffered low levels of factor loading (below 0.50), we ran EFA again, so that the KMO raised to 0.79. Bartlett’s Test of Sphericity was also statistically significant (χ2 (476) = 5772.33, p = 0.000 < 0.05). A principal component factor analysis with Varimax rotation revealed that all remaining items had acceptable loadings. As such, the final draft of the scale included 19 items. A 5-point Likert scale was used ranging from 1 indicating “not at all”, 2 “to a small scale”, 3 “to some extent”, 4 “to a moderate extent, and 5 “to a great extent”. As the last step, to make sure whether the items elicited consistent responses, the reliability which was estimated based on Cronbach α found to be adequate (.86).

Procedure

The EOAQ was distributed among 325 TEFL students. Because of the Covid-19 lockdown, we did not have direct access to TEFL teachers and students; therefore, we developed the scale and gathered them via Google Forms. Of 376 questionnaires, 336 copies were returned (a response rate of 89.3). Of 250 questionnaires sent to TEFL teachers, 212 copies were returned (a response rate of 84.8). It is noteworthy that the researchers informed teachers and students that their information and participation would remain confidential and that their participation was voluntary. It took about 2 months to gather the data.

Data analysis

The data gathered from the questionnaire was analyzed both descriptively and inferentially. To examine the first and second research hypotheses, the means calculated for both the students and the teachers were compared with those of related populations to see if there existed any significant differences. To examine the third research hypothesis, the four subscales of the questionnaires were considered as dependent variables and having the assumptions met, MANOVA was utilized

The first research hypothesis considered the perceptions of Iranian TEFL instructors towards the efficacy of online assessment during COVID 19. To analyze the data on subscales, the score of each participant on all the items of each subscale was added up so that the possible minimum and maximum scores for each participant on the first three subscales would be between 4 and 20 respectively while for the last subscale (with five items) it would be between 5 and 25.

Table 1

Descriptive statistics on the subscales of the questionnaire administered to EFL teachers
	N	Range	Minimum	Maximum	Mean		Std. Deviation	Variance	Skewness
	Statistic	Statistic	Statistic	Statistic	Statistic	Std. Error	Statistic	Statistic	Statistic	Std. Error
Subscale A	52	8.00	8.00	16.00	12.7885	.34006	2.45218	6.013	− .583	.330
Subscale B	52	16.00	4.00	20.00	11.8269	.50017	3.60675	13.009	− .193	.330
Subscale C	52	16.00	4.00	20.00	13.2692	.57612	4.15445	17.259	− .521	.330
Subscale D	52	13.00	12.00	25.00	19.2308	.37313	2.69069	7.240	− .262	.330

To examine the first research hypothesis, four one sample t-tests were run. Before running a t-test, however, it was also necessary to verify the normal distribution of the data in the questionnaire. The results of skewness tests conducted for this purpose proved the normality of the data collected in this part since the calculated values for the subscales in the teachers’ questionnaire ranged between 1.96 and − 1.96, hence the data were normally distributed. The H₀ at this point assumes that there is no significant difference between the calculated means and the average (10 for the first three subscales and 12.5 for the last subscale). To testify the null hypothesis, four one-sample t-test conducted. Table 2 shows the results of one-sample t-tests for all the subscales.

Table 2

One sample t-test results on the subscales of the questionnaire administered to EFL teachers
	t	df	Sig. (2-tailed)	Mean Difference	95% Confidence Interval of the Difference
	t	df	Sig. (2-tailed)	Mean Difference	Lower	Upper
Subscale A	8.200	51	.000	2.78846	2.1058	3.4712
Subscale B	3.653	51	.001	1.82692	.8228	2.8310
Subscale C	5.675	51	.000	3.26923	2.1126	4.4258
Subscale D	18.039	51	.000	6.73077	5.9817	7.4799

The calculated means for the variables were compared to the desired means, and since the reported significance values (.000 ) are all less than the significance level of .05, the H₀ is not maintained; hence, it is concluded that the means calculated for both factors are significantly different from the average. In other words, it can be inferred that teachers held significantly positive attitudes towards the use of online assessment in English classes.

The second null hypothesis suggests that Iranian TEFL postgraduate students do not have a positive attitude towards the efficacy of online assessment during COVID 19. To analyze the data on subscales, the same procedure, as followed for teachers, was considered for students. So the students' scores on each item of the same subscale were added up. Table 3 shows the descriptive statistics for the students on all the subscales.

Table 3

Descriptive statistics on the subscales of the questionnaire administered to EFL learners
	N	Minimum	Maximum	Mean		Std. Deviation	Variance	Skewness
	N	Minimum	Maximum	Statistic	Std. Error	Statistic	Statistic	Statistic	Std. Error
Subscale A	103	8.00	17.00	13.3107	.22334	2.26668	5.138	− .534	.238
Subscale B	103	4.00	20.00	12.6311	.36400	3.69417	13.647	− .641	.238
Subscale C	103	6.00	19.00	13.3301	.28723	2.91510	8.498	− .429	.238
Subscale D	103	11.00	24.00	18.0485	.26589	2.69851	7.282	− .129	.238

To examine the second research hypothesis, four one sample t-tests were run. The results of skewness tests conducted for this purpose proved the normality of the data collected in this part since the calculated values for the subscales in the teachers’ questionnaire ranged between 1.96 and − 1.96, the data were normally distributed

Table 4

One sample t-test results on the subscales of the questionnaire administered to EFL learners
	t	df	Sig. (2-tailed)	Mean Difference	95% Confidence Interval of the Difference
	t	df	Sig. (2-tailed)	Mean Difference	Lower	Upper
Subscale A	14.823	102	.000	3.31068	2.8677	3.7537
Subscale B	7.228	102	.000	2.63107	1.9091	3.3531
Subscale C	11.594	102	.000	3.33010	2.7604	3.8998
Subscale D	20.868	102	.000	5.54854	5.0211	6.0759

Table 4 presents the test results of One-Sample t-Test, with mean differences, t values, degrees of freedom, and two tailed significances of these tests. The means for the subscales were compared to the desired means, and since the reported significance values (.000 ) are all less than the significance level of .05, the H₀ is not maintained; hence, it is concluded that the means calculated for both factors are significantly different from the average. In other words, it can be inferred that students held significantly positive attitudes towards the use of online assessment in English classes.

The third research hypothesis suggests that there is not any significant difference between Iranian TEFL instructors and TEFL postgraduate students’ perceptions about the efficacy of online assessment during COVID 19. To test the hypothesis, since the four subscales were considered as dependent variables, MANOVA was utilized. The descriptive statistics as presented in Table 5 shows the mean scores and standard deviations for both of the groups. It is worth mentioning that the participants' scores on each item of the same subscale have been added up as done in the previous data analysis procedures.

Table 5

Descriptive statistics on the subscales of the questionnaire administered to EFL teachers and learners
	GROUPS	Mean	Std. Deviation	N
Subscale A	TEACHERS	12.7885	2.45218	52
Subscale A	STUDENTS	13.3107	2.26668	103
Subscale B	TEACHERS	11.8269	3.60675	52
Subscale B	STUDENTS	12.6311	3.69417	103
Subscale C	TEACHERS	13.2692	4.15445	52
Subscale C	STUDENTS	13.3301	2.91510	103
Subscale D	TEACHERS	19.2308	2.69069	52
Subscale D	STUDENTS	18.0485	2.69851	103

The test of equality of covariances (Table 6) indicate that the observed covariance matrices of the dependent variables are equal across groups.

Table 6

The test of equality of covariances
Box's M	19.062
F	1.843
df1	10
df2	50314.384
Sig.	.088

Results of tests of between subjects demonstrated significant interaction effects for groups and subscale D. Table 7 shows that there is no significant difference between the groups in the first three subscales while there exists a significant difference in subscale D as the sig. is less than (0.05) as the alpha decision level.

Table 7

Results of tests of between subjects effect
Source	Dependent Variable	Type III Sum of Squares	df	Mean Square	F	Sig.	Partial Eta Squared	Noncent. Parameter
GROUPS	Subscale A	9.424	1	9.424	1.736	.190	.011	1.736
	Subscale B	22.345	1	22.345	1.663	.199	.011	1.663
	Subscale C	1.228	1	1.228	.061	.916	.000	.011
	Subscale D	48.296	1	48.296	6.645	.011	.042	6.645

The COVID-19 pandemic has not only posed unprecedented physical and psychological, social and economic challenges, it has also cast serious doubts on the efficiency of the long-established and time-honored educational orthodoxies on a global scale. The first research hypothesis of the present study specifically addressed the perceptions of Iranian TEFL instructors towards the efficacy of online assessment during COVID 19 the findings from which indicated that teachers held significantly positive attitudes towards the use of online assessment in English classes. Partially in line with the present study, in a study of pre-service English teachers’ views regarding web-based assessment, Öz (2014) found that although EFL teachers did not seem to fully appreciate the use of online assessment and displayed some lack of interest to use this form of assessment in their future classes, they showed a positive attitude towards ease of use of online assessment testing. In addition, they did not favor a shift to a fully web-based form of assessment and the majority preferred an online assessment in combination with paper-based assessment.

The findings from the first research hypothesis supports those in Ghanbari and Nowroozi (2021) which point to the participating teachers’ ability to adjust their practice to the peculiarities of the pandemic. Although their participants initially appeared to hold uncertain attitudes towards successful administration of online assessment at the outset of the pandemic, but they could eventually overcome the challenges toward the end of the academic course.

The results from the first research hypothesis do not seem to support the findings from Hedayati and Marandi (2014) in which they investigated Iranian EFL teachers perceptions toward implementing computer-assisted language learning (CALL) with a primary focus on technology integration in Iranian EFL classes. Based on their results, Iranian EFL teachers referred to teacher, facility, and learner constraints as the three main obstacles in implementing CALL in language classrooms. One reason for such an incongruence in the results can be merely attributed to the very outbreak of the COVID-19. Under the pandemic-induced circumstances, language teachers felt an insistent demand to incorporate online assessment techniques into their practices and as a corollary of implementing remote classes, they began to improve their technology skills and restructure their attitudes towards CALL in general and online assessment in particular.

The first research hypothesis results are also in line with the results from Guangul, Suhail, Khalit, and Khidhir (2020) which examined the challenges of online assessment during COVID-19 pandemic in Middle East College. The perceptions of 50 faculty members were collected and analyzed through questionnaires. The participants held positive attitudes towards online assessment and viewed academic dishonesty and infrastructure as the main inhibitors to online assessment among other factors and suggested several suggestions to minimize academic dishonesty.

Turning to the second research hypothesis that investigated the perceptions of Iranian TEFL postgraduate students towards the efficacy of online assessment during COVID 19, we found that students held significantly positive attitudes towards the use of online assessment in English classes. In a study, Rezaei (2020) studied methods of online tests and as reported they include face-to-face exam, virtual written exam, virtual oral exam, oral question and answer, virtual presentations, electronic portfolios and formative evaluation. As the researcher reported, participants believed that the online assessment should not be limited to a particular type of assessment.

The results of the second hypothesis corroborate those in Tümen Akyıldız (2020) partially. The participating college students in this study carried out in Turkey believed that the pandemic education has had both drawbacks such as constraints in interaction and face-to-face communication which direct the students to isolation and the load of assignments on the one hand and advantages like flexibility of time and place, students’ active participation in their own learning and comfort in exams on the other. The final advantage, i.e., comfort in exams, marks the interface between Tümen Akyıldız’s (2020) results and the present study’s finding where the participants in both studies held positive attitudes toward OLA.

The findings from the second research hypothesis are not in line with those in Khan and Khan (2019) in which students did not show positive attitudes towards OLA. In particular, they did not recognize the need for online assessment, were technologically incompetent, viewed online assessment as restrictive for certain subjects.

Azimi, Zamani, and Soleimani (2016, p. 214) explored Iranian and Russian students’ viewpoints regarding the quality of assessment. As the authors reported, Iranian students considered "providing sufficient information about the final exam conditions, the type of scoring, time and place of exam”, “diversity in methods of assessing and supervising the work of the students" and "offering regular feedback to the learner with the aim of solving the learning problems “as important factors which promotes quality of online assessment while Russian students regarded "fight against fraud and plagiarism, the quality of pre-test to assess students’ pre-knowledge and consistent of evaluation methods with educational content " as the main three factors that are important in the quality of assessment.

Zamani Parhizi, and Kaviani (2015) aimed to identify challenges of evaluation in virtual college from the view point of students. Based on the results, the greatest challenge for the students was technical problem. Khan et al (2021) who sought the challenges of assessment during COVID 19 found that among other findings, quality of Internet, prior knowledge of ICT, and family income were among the factors which affected the quality of assessment. Özden, Ertürk, and Sanli (2004) also investigated students’ perceptions of online assessment and reported that the based on the students’ views most noticeable features of the online assessment system were “immediate feedback, randomized question order, item analysis of the questions, and obtaining the scores immediately after the exam” (p.89). Overall, participants agreed on the effectiveness of the online assessment system.

The findings from the second research hypothesis are also in line with those in Attia’s (2014) study which examined the postgraduates’ perceptions toward OLA they had taken. Attia found substantially positive perceptions toward OLA. In addition, he identified gender and ICTs (information and communication technologies) familiarity as two main factors to significantly influence postgraduate students’ responses.

Concerning the third research hypothesis which investigated the differences between Iranian TEFL instructors and TEFL postgraduate students’ perceptions about the efficacy of online assessment during COVID 19, we found that except in the last scale, shortages of OLA, there is no difference between both groups. The findings from the third research hypothesis corroborate mostly the results from several similar investigations (Öz, 2014; Rezaii, 2020; Wahid & Faroq, 2020;). In Öz’s (2014) study, the participants exhibited a positive attitude towards ease of use of web-based testing for their courses, but they did not prefer a shift to an entirely web-based form of assessment.

The results of the analysis of instructors and students’ perception on the challenges and solutions to the assessment of students’ performance during the pandemic pointed to the proximity of their views which seem to be congruent with the present study’s results. Specifically, eclectic assessment encompassing several forms of tests, electronic portfolio assessment, and oral online tests were among the different assessment techniques the participants suggested for online assessment of students’ performance. In Wahid and Faroq’s (2020) study, the teachers held positive views on the administration of online exams from home. In sum, both teachers and students held positive views towards online testing. This is against the researchers’ expectations since as the COVID-19 pandemic spread in the country, government suspended education activities temporarily and in a short time full-online classes named emergency e-learning replaced face-to-face education (Murphy, 2020). Perhaps, it is likely that although the virtual courses did not perform at their best, the online courses served as a suitable panacea for face to face courses.

In designing education systems in general and online educational techniques and resources in particular, instructors and students’ viewpoints must be taken into account to ensure dynamism, motivation, and interaction. The two main agents in online education systems, i.e., instructors and students, need to be convinced of the efficacy of the transition from long-established well-trodden educational system to uncertain and untrodden online assessment before they embrace it voluntarily. Inasmuch as the results of this study are brought to the fore, training courses and workshops on online technology and resources have to be incorporated into foreign language teacher education and in-service teacher training courses and curricula through which instructors can get familiar with and boost their understanding of implementing online assessment techniques. Such transitions can focus on low stake assessments at their first attempts and gradually move toward high stake assessments.

Despite the COVID-19 personal and online classes technological challenges for TEFL instructors and students, they still held positive attitudes towards OLA and believed that it can be used to determine if students have achieved the educational aims. The COVID-19 pandemic has become a reality in our life for some time and it continues to upset the normal rhythm of our communities. Within status quo, it is incumbent upon the education systems to continue to warrant accountability. To assure accountability, they need to integrate rigorous and reliable online assessment adds-in into the current virtual class software and applications.

Education system authorities and teachers need an extensive array of information about their students in order to deliver geared assessment. Supporting teachers in developing restructured online assessments that accommodate a wide range of student and contextual factors is a necessary first step if educational systems including those of universities are ever to compensate for the unprecedented learning gaps. As the results of the present study disclosed, both instructors and students held positive attitudes towards the efficacy of online assessments. Prior to the administration of online tests, as a corollary of this finding, education leaders and teachers have to ensure learners’ engagement and facilitative conditions for taking the test. This is not be an easy feat, as a number of factors such as internet access and speed are not in the hands of the teachers, but it is necessary to address the drawbacks associated with OLA as echoed by several studies (Ahmady, Shahbazi, & Heidari, 2020; Badrkhani, 2021; Guangul, Suhail, Khalit, & Khidhir, 2020; Khan, Kamal, Illiyan, Asif, 2021).

This study was constrained on several grounds. Sampling for the study was restricted to purposive procedure due to the aims of the study. The sample might not have represented the entire population fully. Due to its qualitative nature, the findings are not generalizable to the population. The use of a convenience sample and relative similarity of participants within each focus group means that the views reflected cannot represent the entire university students’ population. The interviews, because of the pandemic, were run on the phone and this mode of interaction might have impacted the participants’ responses. In the times of COVID-19, every one of us experienced certain levels of apprehension and consequently the participants’ responses might have been influenced by the contiguity of the context. Therefore, further investigation can examine teachers and students’ views on online assessment after the pandemic will have come to its end.

In order to make online assessment more gratifying for students and teachers, the results of this study imply that the interaction patterns between them have to be dynamic, meaningful, and personalized. Moreover, both teachers and students’ interests and concerns should be addressed to successfully incorporate online assessments into TEFL courses. To render online assessment efficient and enduring, teachers are encouraged to use inspiring and interactive techniques to engage students during online assessment. It is incumbent upon teacher education programs to equip the teachers with skills to apply sophisticated software and assessment tools during online assessment.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Competing interests

The authors declare that they have no competing interests.

Funding

The authors received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Authors’ Contributions

The authors collaboratively conceived of the presented idea. M.F. wrote an initial draft of the study. He also wrote the discussion section. H.GH. conceptualized data analysis procedure. He did the statistical analyses and wrote the results. Y.R. did the literature review and wrote the introduction and conclusion. All authors read specific drafts of the manuscript at different times. All authors gathered the data through questionnaire/ interview, discussed the results, commented on the manuscript and checked the citations, tables, and references in terms of APA formatting. Y.R., M.F. and H.GH. contributed to the design and implementation of the research, to the analysis of the results and to the writing of the manuscript.

Acknowledgements

We would like to thank the university TEFL teachers and students who agreed to take part in the present study and responded to questionnaire items and interview questions.

Abdel-Rahim, H. (2021). The effectiveness of online teaching and learning tools: Students’ perceptions of usefulness in an upper-level accounting course. Learning and Teaching, 14(3), 52-69.
Ahmady, S., Shahbazi, S. & Heidari, M. (2020). Transition to virtual learning during the Coronavirus disease–2019 crisis in Iran: Opportunity or challenge? Disaster Medicine and Public Health Preparedness, 14(3), E11-E12. doi:10.1017/dmp.2020.142
Attia, M. A. (2014). Postgraduate students’ perceptions toward online assessment: The case of the faculty of education, Umm Al-Qura university. In A. W., Wiseman, N. H., Alromi, & S. Alshumrani (Eds.), Education for a Knowledge Society in Persian Gulf countries (International Perspectives on Education and Society, Vol. 24) (pp. 151-173). Emerald Group Publishing Limited, Bingley.
Azimi, S. A., Zamani., B. E., & Soleimani, N. (2016). Investigating the quality of assessment system of students’ performance in virtual education in Iran and Russia. Educ Strategy Med Sci., 9(3), 213-223.
Badrkhani, P. (2021). How a catastrophic situation turns into an exceptional opportunity: Covid-19 pandemic in Iran and challenges of online education for new English language educators. Interactive Learning Environments. doi.org/10.1080/10494820.2021.1956545.
Bonham, S. W., Titus, A., Beichner, R. J., & Martin, L. (2000) Education research using web-based assessment systems. Journal of Research on Computing in Education, 33(1), 28-44.
Bull, J., & McKenna, C. (2004). Blueprint for computer-assisted assessment. London: Routledge Falmer.
Coman, C., Tîru, L. G., Mesesan-Schmitz, L., Stanciu, C., & Bularca, M. C. (2020). Online teaching and learning in higher education during the coronavirus pandemic: Students’ perspective. Sustainability, 12. doi:10.3390/su122410367.
Dhawan S. (2020). Online learning: A panacea in the time of COVID-19 crisis. Journal of Educational Technology Systems, 49(1), 5–22.
Ebadi, S., Khazaie, S., & Bashiri, S. (2020) Technology acceptance of NAVID learning management system in the Iranian medical English courses under the COVID-19 Pandemic. Journal of English language Teaching and Learning,12(26),401-433.
Gamage, K. A. A., de Silva, E. K., & Gunawardhana, N. (2020). Online delivery and assessment during COVID-19: Safeguarding academic integrity. Education Sciences, 10(11), 301; doi:10.3390/educsci10110301
Ghanbari, N., & Nowroozi, S. (2021). The practice of online assessment in an EFL context amidst COVID-19 pandemic: Views from teachers. Language Testing in Asia, 11, 27. https://doi.org/10.1186/s40468-021-00143-4
Guangul, F. M., Suhail, A. H., Khalit, M. I., & Khidhir, B. A. (2020). Challenges of remote assessment in higher education in the context of COVID-19:A case study of Middle East College. Educational Assessment, Evaluation and Accountability, 32, 519-535.
Jimenez, L. (2020, September 10). Student assessment during COVID-19. https://www.americanprogress.org/article/student-assessment-covid-19/
Khan, H., & Khan, A. K. (2019). Online assessments: Exploring perspectives of university students. Education and Information Technologies, 24, 661-677.
Khan, M. A., Kamal, T., Illiyan, A., Asif, M. (2021). School students’ perception and challenges towards online classes during COVID-19 pandemic in India: An econometric analysis. Sustainability, 13, 4786.
Martin, A. (2020). How to optimize online learning in the age of coronavirus (COVID-19): A 5-point guide for educators. https://www.researchgate.net/publication/339944395_How_to_Optimize_Online_Learning_in_the_Age_of_Coronavirus_COVID-19_A_5-Point_Guide_for_Educators
Meccawy, Z., Meccawy, M. & Alsobhi, A. (2021). Assessment in ‘survival mode’: Student and faculty perceptions of online assessment practices in HE during Covid-19 pandemic. International Journal for Educational Integrity,17, 1-24.
Murphy, M. P. (2020). COVID-19 and emergency e-learning: Consequences of the securitization of higher education for post-pandemic pedagogy. Contemporary Security Policy, 41(3), 1-14.
Öz, H. (2014). Pre-service English teachers’ perceptions of web-based assessment in a pedagogical content knowledge course. Procedia-Social and Behavioral Sciences,141, 45-58.
Özden, M. Y., Ertürk, I., & Sanli, R.(2004). Students’ perceptions of online assessment: A case study. Journal of Distant Education, 19(2),77-92.
Rezaii, A. M. (2020). Students performance during COVID-19: Challenges and solutions. Educational Psychology Quarterly, 16(55), 179-214. [Persian] doi: 10.22054/jep.2020.52660.3012
Richardson, J. T. (1996). Handbook of qualitative research methods for psychology and the social sciences. Leicester: BPS Books.
Saricoban, A. (2013). Prospective and regular ELT teachers’ digital empowerment and self-efficacy. Porta Linguarum, 20, 77-87.
Sharadgah, T. A., & Sa’di, R. A. (2020). Preparedness of institutions of higher education for assessment in virtual learning environments during the COVID-19 lockdown: Evidence of bona fide challenges and pragmatic solutions. Journal of Information Technology Education: Research, 19, 755-774.
Sullivan, D. P. (2016). An integrated approach to preempt cheating on asynchronous, objective, online assessments in graduate business classes. Online Learning, 20(3),195–209.
Teddlie, C., & Yu, F. (2007). Mixed methods sampling. Journal of Mixed Methods Research, 1(1), 77 – 100.
Tümen Akyıldız, S. (2020). College students’ views on the pandemic distance education: A focus group discussion. International Journal of Technology in Education and Science (IJTES), 4(4), 322-334.
Verhoef, A., & Coetser, Y. (2021). Academic integrity of university students during emergency remote online assessment: An exploration of student voices. Transformation in Higher Education, 6. doi:https://doi.org/10.4102/the.v6i0.132
Wahid, R., & Farooq, O. (2020). Online exams in the time of COVID-19: Quality parameters. International Journal of Social Sciences & Educational Studies, 7(4), 13-21.
Zakaryia, A., Khaled, M., & Omar, A. M. (2021). Faculty’s and students’ perceptions of online learning during COVID-19. Frontiers in Education, 6, doi: 10.3389/feduc.2021.638470.
Zamani, B., Parhizi, R., Kaviani, H. (2015). Identify challenges of evaluating students' academic performance e-courses. Technology of Education Journal (TEJ), 9(2), 105-112. doi: 10.22061/tej.2015.305

The efficacy of online assessment

Strongly disagree

Disagree

Undecided

Agree

Strongly Agree

Mean

familiarity with OLA and its feasibility

I think OLA can be a good alternative to traditional

paper-based exams

(21.6

(58.8)

(17.6)

(2)

1.94

OLA is compatible to assess all lessons in TEFL

(22.9)

(62.1)

(14.4)

(.7)

Teachers have sufficient experience in conducting online assessment

(20.3)

(62.7)

(13.7)

(3.3)

Universities were not ready for OLA due to the abrupt and rapid transition to e-learning under the COVID-19 pandemic

(24.8)

(63.4)

(11.1)

(.7)

OLA contribution in improving learning and feedback processes

OLA improves the quality of learning

(22.2)

(62.1)

(13.7)

(2)

1.93

OLA motivates students to learn

(26.8)

(60.1)

(10.5)

(2.6)

OLA effectively assesses students’ level of knowledge

(21.6)

(64.7)

(11.1)

(2.6)

Immediate feedback from OLA could help students learn

(22.2)

(62.7)

(13.1)

(2)

the worth and value

of implementing OLA

OLA is more interesting than paper-based exams

(6.5)

(36.6)

(13.7)

(21.6)

2.85

OLA can be time-saving

(12.4)

(28.8)

(13.7)

(23.5)

(21.6)

OLA is economical

(9.8)

(36.6)

(7.8)

(27.5)

(18.3)

OLA is objective and fair when assessing students’ performance.

(15.7)

(33.3)

(5.2)

(26.1)

(19.6)

OLA is more rapidly accessible to remote students than paper-based exams.

(9.8)

(37.3)

(7.8)

(24.2)

(20.9)

OLA assess students’ knowledge more objectivity than paper-based exams

(36.6)

(58.8)

(2.6)

(2)

shortages of OLA

Students who take OLA, are more anxious

than the time they take paper-based exams

(11.1)

(53.6)

(31.4)

(3.9)

2.33

Cheating and plagiarism are easier in OLAs

than paper-based exams

(7.2)

(43.8)

(44.4)

(4.6)

Not having adequate proficiency in dealing with ICTs affects students’ performance in OLAs

(5.9)

(59.5)

(31.4)

(3.3)

students’ lack of online facilities and resources negatively affects’ the results of the tests

(8.5)

(54.2)

(31.4)

(5.9)

The need to adapt the tests to suit online hardware and software capabilities takes a lot of time and effort.

(10.5)

(54.9)

(34.6)

No competing interests reported.

Download PDF

Version 1

posted

You are reading this latest preprint version

Exploring the cognitions of Iranian university TEFL teachers and students of the efficacy of online assessment practices in the wake of Covid 19

Status:

Version 1

Abstract

1. Introduction

2. Literature Review

3. Methodology

4. Results

5. Discussion

6. Conclusions, Limitations And Implications

Declarations

References

Appendix

Additional Declarations

Status:

Version 1