Comparison of the statistics and item analysis of multiple-choice questions between free and tuition-paying admission


 Background: The purpose of this pilot was to compare the multiple-choice test statistics of medical and dental students' exams between free and tuition-paying.Methods: This descriptive-analytical study was conducted at Golestan University of Medical Sciences in Iran in 2020. The study population included students of medicine and dentistry. A total of 56 exams were selected in two student groups of free and tuition-paying admission in the physiology course. The results of quantitative evaluation of tests were used as the data of this study. The variables included difficulty index, discrimination index, the degree of difficulty, score variance, and Kuder-Richardson correlation coefficient. Results: There were 32 medical and 24 dentistry exams. The cumulative total number of questions in these exams was 437 and 330 multiple choice questions, respectively. The number of medical students participating in the free-tuition and paying-tuition admissions was 1336 and 1076, and for dental students, these numbers were 395 and 235, respectively. There were no significant differences in normalized adjusted exams scores between two admission groups in both medical and dentistry tests. The mean of discrimination index in the free-tuition group was higher than in the paying-tuition group. The interaction between the type of admission and the field of study was significant for the discrimination index. This difference was more in tuition-free dental students than tuition-free medical students and tuition-paying dental students. Conclusion: The type of student admission has no significant effect on student assessments in multiple-choice exams in matched educational conditions.


Introduction
Educational researches have indicated the impact of student admission policies on educational performance, both academic and non-academic [1][2][3] . In terms of paying tuition, student admission policies apply in medical schools for various reasons. Some of these reasons include providing services in rural regions 3 , the number of students admittances 4 , the provision of primary care physicians, and educational justice 5 . The impact of tuition on educational performance is one of the main concerns of educational policymakers in most universities.
In Iran, universities are divided into two major groups in terms of eld of activity. The rst groups are universities that work in the eld of medicine and its related elds and are known as medical universities. These universities are managed according to the rules and regulations of the Ministry of Health and Medical Education. The second groups are universities in the elds of engineering, art, literature, and other non-medical elds. These universities are managed according to the rules and regulations of the Ministry of Science, Research and Technology. In Iran, the universities can be categorized into two major groups, governmental and non-governmental, in terms of ownership and funding. Accordingly, student admission also includes free tuition and paying tuition. Free tuition is only at public universities and admission with tuition is at public and non-public universities. Admission without paying tuition and using free education has many applicants, and the competition between the candidates is erce and requires a high score in the entrance exam. If the applicant is not accepted in the free tuition government admission, the next option is to apply for admission by paying tuition.
In the case of medicine and dentistry, the limited capacity of universities and a large number of applicants for them cause intense competition even in admission with paying tuition. Of course, the average score level in the entrance exam of those admitted without tuition is always higher than those admitted with tuition. This fact in some cases has led to bias and misjudgment about the ability and educational performance of paying-tuition students.
Various studies have been conducted about the educational performance of paying-tuition and free-tuition students. Some studies state that admission without tuition does not affect educational performance 6,7 and others point to the effect of tuition on low academic performance 8,9 . In most of these studies, the comparison criteria in the two groups of paying-tuition and free-tuition students were not fully matched or could not be implemented. Comparing the grade point average, scores of different courses, scores of a simple course in different semesters, and other similar methods without su cient matching of the two groups can be accompanied by bias and error. The present study was conducted in terms of a maximum matching in 56 exams in terms of the course title, instructor, sources of the test, educational content, the volume of course content, time of the exam, number of questions, and holding the test timing at the same time in two independent groups of paying-tuition and free-tuition students. The purpose of this study was to compare the item analysis of multiple-choice questions between paying-tuition and free-tuition students in the basic science stage.

Methods
The present study was a descriptive-analytical study that was conducted cross-sectional in 2020 at Golestan University of Medical Sciences in Iran. The study population included students of medicine and dentistry. Medical and dental students have nearly identical curriculums and educational programs in their basic science stage. This stage includes the rst four semesters. A total of fty-six physiology course exams were selected in two groups of free-tuition and paying-tuition students in basic sciences stages of medicine or dentistry. After matching the conditions including the matching of the exam features (number of questions, date of exam, exam time, volume of content, topics, and resources) and the instructors matching, the results of quantitative evaluation of exams were used as the data of this study. Matching was not considered in terms of the location and order of the correct answers of multiple-choice questions, the number of students participating in each exam, and other student-related conditions such as admission quota, gender, or grade point average due to limitations and impossibility. It should be noted that at the time of the exams, none of the teachers, learners, facilitators, and colleagues had any information about the presentation of this study, and the idea of this research was developed long after the exams and their quantitative evaluation report.
For increasing the power of the study and provide a proper sample size, we included both medical and dental students' exams. During the basic science stage, which consists of the rst four semesters, the medical and dental students have almost the same educational programs and very similar curriculums. The physiology course of medical and dental students is also very comparable to each other. Therefore, we assumed considering both medical and dental exams' data had no bias on results.
The data were collected based on the o cial test evaluation results of the Education Development O ce and the o cial reports of the education o ce of the medical and dental school and the International Campus, which are the exams evaluators and exams organizers of medical and dental students for both free-tuition and paying-tuition, respectively. The items in these reports for each exams included twenty-one parameters, which are the title, date, correct answer score, incorrect answer score, no answer score, passing score, number of options, number of questions, number of candidates, di culty index, discrimination index, number of the item with the degree of di culty, number of the item with the ability to discriminate, mean, Score variance, standard deviation, highest scores, lowest scores, Kuder-Richardson correlation coe cient, and standard error of measurement. The number of questions in different exams was not equal. Therefore, we used adjusted exams scores for comparison between two admission groups in both medical and dentistry exams. The number of questions and the maximum score of the exam were different among these fty-six exams. Therefore, they must be normalized for statistical analysis. Because the twenty is routinely considered as the maximum exam score in Iran's educational system, this value was considered as the maximum exam score and all adjustments were normalized to it.
In this study, we classi ed the di culty index of each question into ve categories: simple, moderate, hard, very hard, and extremely hard. We also classi ed the discrimination index of each question in for categories: negative, poor, fair, and good.
A Chi-square test was used to compare qualitative variables. The Kolmogorov-Smirnov test was used to test the normality of quantitative data. Tables and graphs were used to describe the variables. Independent t-test and Mann-Whitney test were used to compare quantitative variables between the two independent groups. Two-way univariate analysis of variance was used to investigate the interaction of independent variables on quantitative variables. Statistical package for the social sciences (SPSS) software version 22 was used for statistical analysis. This study was approved by the ethical committee and the institutional review boards of the Golestan University of Medical Sciences (code: IR.GOUMS.REC.1397.159).

Results
In this study, the item analysis of multiple-choice questions for students of medicine and dentistry between free-tuition and paying-tuition courses was compared. There were thirty-two medical and twenty-for dentistry exams. The number of exams in each of free tuition and paying tuition courses in the students of medicine were sixteen exams and in the students of dentistry were twelve exams. The total number of exams' questions in the free-tuition and paying-tuition courses of the medical students was 437 questions and in the dental students was 330 questions. The number of medical students participating in the free-tuition and paying-tuition courses was 1336 and 1076 students, respectively, and the number of dental students was 395 and 235 students, respectively.
The number of questions based on the di culty index and discrimination index and their relation to di culty and discrimination level is shown in Table 1. According to the ndings, the number of questions with simple and hard di culty in the free-tuition group was more than the paying-tuition group. Also, the number of questions with poor and good discrimination index in the free-tuition group was more than the paying-tuition group. Regarding the di culty index, there were no exams in the simple and extremely hard level. There were nineteen and thirty-ve exams in the moderate and hard class, respectively. The ndings show that the discrimination index of the exams was fair (thirteen for both free and paying-tuition admission) and good ( fteen for free and fourteen for paying-tuition admission). There were no negative or poor exams for differentiating ability. Table 2 shows the results of the independent t-test to compare the di culty index, discrimination index, Kuder-Richardson correlation coe cient, and standard error of measurement between two groups of freetuition and paying-tuition students. According to the ndings, the mean discrimination index of questions in free-tuition students was higher than paying-tuition students, and this difference is statistically signi cant (p = 0.048). The mean di culty index, Kuder-Richardson correlation coe cient, and the standard error of measurement of the questions were not different in the two groups of free-tuition and paying-tuition students. Table 3 shows the results of the independent t-test to compare the di culty index, discrimination index, Kuder-Richardson correlation coe cient, and standard error of measurement of the questions between two independent medical and dental students' groups. The nding showed that the di culty index had not differed between the two groups of medical and dental students. According to the results of the Levin test (F = 6.738), the signi cance level was lower than 0.05 (p = 0.012), so for the di culty index, the variance of medical and dental groups was not equal. Discrimination index, Kuder-Richardson correlation coe cient, and standard error of measurement of the questions were not different in the two groups of medical and dental students.
Two-way univariate analysis of variance was used to show the interaction between the type of student admission (free and paying-tuition) and the eld (medicine and dentistry) on the two variables (the degree of di culty and discrimination index of the questions). (Table 4) The interaction between the type of admission and the eld was not signi cant on the di culty index of questions (p = 0.906) but was signi cant on the discrimination index of the questions (p = 0.0001). Therefore, the mean discrimination index of questions in four groups including the tuition-free medical, tuition-free dental, tuition-paying medical and tuition-paying dental groups was not the same. This was shown in gure 1. The type of admission in the medical and dental groups was different so that the mean of the discrimination index of questions in tuition-free dental students was more than in tuition-paying students in that eld.

Discussion
This study was conducted to compare the item analysis of multiple-choice questions among medical and dental students in two groups of tuition-free and tuition-paying students in a maximum matched condition. The results of the present study generally showed that the item analysis of multiple-choice questions among the free-tuition and paying-tuition students was the same. Also, the results showed that the item analysis of multiple-choice questions was the same among students in medicine and dentistry.
The results showed that only the differentiation index of questions between free-tuition and paying tuition students was signi cant. As Taib and Yusoff (2014) pointed, multiple-choice questions can distinguish between good and poor students 10 . According to the results of this study, the mean differentiation index of questions in free-tuition students was higher than paying-tuition students, which means that multiplechoice questions in free-tuition students were more differentiated between good and poor students than paying-tuition students. Although the number of exams' questions was the same for the free and paying tuition students, the variable of student numbers was not. The number of free-tuition and paying tuition students was 1731 and 1311, respectively.
In this study, nearly 50% of the questions were considered hard in terms of di culty. This case indicates that the questions were designed appropriately in the sense of di culty. Given that the quantitative evaluation of the questions was done under the same conditions, the quality of questions has no relation with the admission type. Nematbakhsh (2015) points out that students' admission type for free or tuitionpaying is not an important issue, but how the student graduates are necessary 11 . Therefore, we conclude that the students' evaluation methods, whether free or with tuition, should be such that it leads to the promotion of education quality.
According to the results, the mean, standard deviation, standard error of measurement, and Kuder-Richardson correlation coe cient of the exams' showed no statistical signi cance in the two groups of free and paying tuition students in medicine and dentistry. It seems that the type of student admission and their eld do not affect the exam's results.
The statistical results to examine the interaction between the type of admission and the eld in the di culty index showed that the mean di culty index was the same in both admission type and eld groups. Despite examining the interaction between the discrimination indexes of questions with these variables, the results showed that the mean of discrimination index was different between the groups of admission type and eld. This index was higher in tuition-free dental students than tuition-free medical students and tuition-paying dental students.
Two limitations need to be considered for interpretation and future research. First, in this study, the educational objectives in different disciplines were not examined. Although the type of content of the educational program is considered in both medical and dental groups, it is suggested that in another study, other variables such as educational goals in different elds be examined. Second, this study was conducted in one course, which limited the generalizability of the results. Thus, this limitation might be considered during the interpretation of the result.

Conclusion
The type of student admission has no signi cant effect on students' assessments in multiple choice exam in matched educational conditions. Figure 1 mean of discrimination index of questions for four groups of admission type by the eld: tuition-free medical, tuition-free dental, tuition-paying medical and tuition-paying dental groups.