Comparison of the Statistics and Item Analysis of Multiple-Choice Questions of Medical and Dental Students' Tests Between Free Admission and Tuition-Paying Admission.

doi:10.21203/rs.3.rs-650668/v1

Download PDF

Research Article

Comparison of the Statistics and Item Analysis of Multiple-Choice Questions of Medical and Dental Students' Tests Between Free Admission and Tuition-Paying Admission.

https://doi.org/10.21203/rs.3.rs-650668/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this older preprint version

Read the latest preprint version →

Background: The type of student admission may affect their educational performance. The purpose of this pilot was to compare the multiple-choice test statistics of medical and dental students' tests between free admission and tuition-paying admission under maximum matched conditions.

Methods: This descriptive-analytical study was conducted at Golestan University of Medical Sciences in Iran in 2020. The study population included students of medicine and dentistry. A total of 56 tests were selected in two student groups of free admission and tuition-paying admission in basic sciences stages in the physiology course. The matched factors included test features consist of the number of questions, date of the test, test time, the volume of content, topics, and resources, and the instructors. The results of quantitative evaluation of tests were used as the data of this study. There were 21 items for each test including difficulty index, discrimination index, the degree of difficulty, score variance, and Kuder-Richardson correlation coefficient.

Results: There were 16 medical and 14 dentistry tests with 437 and 330 questions respectively. The number of medical students participating in the free-tuition and paying-tuition admissions was 1336 and 1076, and for dental students, these numbers were 395 and 235 respectively. There were no significant differences in normalized adjusted test scores between two admission groups in both medical and dentistry tests. The discrimination index of tests was fair or good, and there were no cases of the negative or poor index. The mean of discrimination index in the free-tuition group was higher than in the paying-tuition group (p = 0.048). The interaction between the type of admission and the field was significant for the discrimination index (p = 0.0001). This difference was more in tuition-free dental students than tuition-free medical students and tuition-paying dental students. The mean difficulty index and the Kuder-Richardson correlation coefficient had no differences in the two groups.

Conclusions: The type of student admission has no significant effect on students’ assessments in multiple choice tests in matched educational conditions.

Internal Medicine

Educational Philosophy and Theory

Test

Student

Tuition

Free admission

Medical

Dentistry

Question analysis

Multiple Choice Question (MCQ)

Educational researches have indicated the impact of student admission policies on educational performance, both academic and non-academic (1–3). In terms of paying tuition, student admission policies apply in medical schools for various reasons. Some of these reasons include providing services in rural regions (3), the number of student admittances (4), the provision of primary care physicians, and educational justice (5). The impact of tuition on educational performance is one of the main concerns of educational policymakers in most universities.

In Iran, universities are divided into two major groups in terms of field of activity. The first groups are universities that work in the field of medicine and related fields and are known as medical universities. These universities are managed according to the rules and regulations of the Ministry of Health and Medical Education. The second group is universities in the fields of engineering, art, literature, and other non-medical fields. These universities are managed according to the rules and regulations of the Ministry of Science, Research and Technology. In Iran, universities can be divided into two major groups, government and non-government, in terms of ownership and funding. Accordingly, student admission also includes free tuition and paying tuition. Free tuition is only at public universities and admission with tuition is at public and non-public universities. Admission without paying tuition and using free education has many applicants, and the competition between the candidates is fierce and requires a high score in the entrance exam. If the applicant is not accepted in the free tuition government admission, the next option is to apply for admission by paying tuition.

In the case of medicine, the limited capacity of universities and a large number of applicants for medical students cause intense competition even in admission with paying tuition. Of course, the average score level in the entrance exam of those admitted without tuition is always higher than those admitted with tuition. This fact in some cases has led to bias and misjudgment about the ability and educational performance of paying-tuition students.

Various studies have been conducted about the educational performance of paying-tuition and free-tuition students. Some studies state that admission without tuition does not affect educational performance (6, 7) and others point to the effect of tuition on low academic performance (8, 9). In most of these studies, the comparison criteria in the two groups of students were not fully matched or could not be implemented. Comparing the grade point average, test scores of different courses, test scores of a simple course in different semesters, and other similar methods without sufficient matching of the two groups can be accompanied by bias and error. The present study was conducted in terms of a maximum matching in 56 tests in terms of the course title, instructor, sources of the test, educational content, the volume of course content, time of the test, number of test questions, and holding the test timing at the same time in two independent groups of paying-tuition and free-tuition students. The purpose of this study was to compare the item analysis of multiple-choice questions between paying-tuition and free-tuition medical and dental students in the basic science stage.

The present study was a descriptive-analytical study that was conducted cross-sectional in 2020 at Golestan University of Medical Sciences in Iran. The study population included students of medicine and dentistry. A total of 56 tests were selected in two groups of free-tuition and paying-tuition students in basic sciences stages in the physiology course. After matching the conditions including the matching of the test features (number of questions, date of test, test time, volume of content, topics, and resources) and the instructors matching, the results of quantitative evaluation of tests were used as the data of this study. Matching was not considered in terms of the location of the correct option, the number of students participating in each test, and other student-related conditions such as admission quota, gender, or grade point average due to limitations and impossibility. It should be noted that at the time of the tests, none of the teachers, learners, facilitators, and colleagues had any information about the presentation of this study, and the idea of this research was developed long after the test and their quantitative evaluation report. The data were collected based on the official test evaluation results of the Education Development Office and the official reports of the education office of the medical and dental school and the International Campus, which are the test evaluators and test organizers of medical and dental students for both free-tuition and paying-tuition, respectively. The items in these reports for each test included 21 parameters, which are the title, date, correct answer score, incorrect answer score, no answer score, passing score, number of options, number of questions, number of candidates, difficulty index, discrimination index, number of the item with the degree of difficulty, number of the item with the ability to discriminate, mean, number of 1, 2, 3, 4, 5, 6 choice questions, Score variance, standard deviation, highest scores, lowest scores, Kuder-Richardson correlation coefficient, and standard error of measurement. The number of questions in different tests was not equal. Therefore, we used adjusted test scores for comparison between two admission groups in both medical and dentistry tests. The 20 was considered as the maximum test score and all adjustments were normalized to it. In this study, we classified the difficulty index and discrimination index of the tests for different ranges of difficulty level and the discrimination index, as shown in Table 1.

Table 1

difficulty index and discrimination index and their relation to difficulty and discrimination level
Index	Value	Classification
Difficulty Index	Difficulty Index > 0.85 0.85 > = Difficulty Index > 0.65 0.65 > = Difficulty Index > 0.45 0.45 > = Difficulty Index > 0.25 0.25 > = Difficulty Index	Simple Moderate Hard Very hard Extremely hard
Discrimination Index	Discrimination Index < 0 0 < = Discrimination Index < 0.1 0.1 <= Discrimination Index < 0.3 0.3 < = Discrimination Index	Negative Poor Fair Good

A Chi-square test was used to compare qualitative variables. The Kolmogorov-Smirnov test was used to test the normality of quantitative data. Tables and graphs were used to describe the variables. Independent t-test and Mann-Whitney test were used to compare quantitative variables between the two independent groups. Two-way univariate analysis of variance was used to investigate the interaction of independent variables on quantitative variables. Statistical package for the social sciences (SPSS) software version 22 was used for statistical analysis. This study was approved by the ethical committee and the institutional review boards of the Golestan University of Medical Sciences (code: IR.GOUMS.REC.1397.159).

In this study, the item analysis of multiple-choice questions for students of medicine and dentistry between free-tuition and paying-tuition courses was compared. The number of tests in free tuition and paying tuition courses in the students of medicine was 16 tests and in the dental students, there were 12 tests. The number of test questions in the free-tuition and paying-tuition courses of the medical students was 437 tests and in the dental students was 330 tests. The number of medical students participating in the free-tuition and paying-tuition courses was 1336 and 1076 students, respectively, and the number of dental students was 395 and 235 students, respectively. The normalized adjusted test scores between two admission groups in both medical and dentistry tests were shown in Fig. 1.

The number of questions based on the difficulty index and discrimination index and their relation to difficulty and discrimination level is shown in Table 2. According to the findings, the number of questions with simple and hard difficulty in the free-tuition group was more than the paying-tuition group. Also, the number of questions with poor and good discrimination index in the free-tuition group was more than the paying-tuition group.

Table 2

Frequency of number of questions based on the degree of difficulty and the ability to differentiate according to the field of study and the admission type
		Admission type*field of study						Total (%)
		Free-tuition			Paying-tuition
		Medical (%)	Dental (%)	Total (%)	Medical (%)	Dental (%)	Total (%)
Difficulty index	Simple	100 (22.9)	53(16.1)	153(19.9)	85(19.4)	46(13.9)	131(17.1)	284(18.6)
	Moderate	136(31.1)	103(31.2)	239(31.2)	150(34.3)	111(33.6)	261(34)	500(32.6)
	Hard	98(22.4)	84(25.4)	182(23.7)	93(21.3)	77(23.3)	170(22.2)	352(22.9)
	Very hard	76(17.4)	61(18.5)	137(17.9)	81(18.5)	63(19.1)	144(18.8)	281(18.3)
	Extremely hard	27(6.2)	29(8.8)	56(7.3)	28(6.5)	33(10.1)	61(7.9)	117(7.6)
	Total	437(100)	330(100)	767(100)	437(100)	330(100)	767(100)	1534(100)
Discrimination index	Negative	11(2.5)	24(7.3)	35(4.6)	16(3.8)	40(12.2)	56(7.5)	91(6)
	Poor	55(12.6)	22(6.7)	77(10.1)	37(8.9)	36(10.9)	73(9.8)	150(9.9)
	Fair	138(31.6)	75(22.7)	213(27.8)	129(30.9)	116(35.3)	245(32.8)	458(30.3)
	Good	232(53.2)	209(63.3)	441(57.6)	235(36.3)	137(41.6)	372(49.9)	813(53.8)
	Total	436(100)	330(100)	766(100)	417(100)	329(100)	746(100)	1512(100)

Table 3 shows the frequency of difficulty index and discrimination index of the tests according to the categories by free-tuition and paying-tuition groups. No tests were in the moderate and extremely hard level and were often in the hard class. The findings show that the discrimination index of the tests was fair and good. There were no negative or poor tests for differentiating ability.

Table 3

Frequency of difficulty index and the discrimination index of tests according to the type of admission
		Admission type		Total (%)
		Free-tuition (%)	Paying-tuition (%)	Total (%)
Difficulty index	Simple	0(0)	0(0)	0(0)
	Moderate	10(35.7)	9(33.3)	19(34.6)
	Hard	17(60.7)	18(66.7)	35(63.6)
	Very hard	1(3.6)	0(0)	1(1.8)
	Extremely hard	0(0)	0(0)	0(0)
Discrimination index	Negative	0(0)	0(0)	0(0)
	Poor	0(0)	0(0)	0(0)
	Fair	13(46.4)	13(48.1)	26(47.3)
	good	15(53.6)	14(51.9)	29(52.7)

Table 4 shows the results of the independent t-test to compare the difficulty index, discrimination index, Kuder-Richardson correlation coefficient, and standard error of measurement between two groups of free-tuition and paying-tuition students.

Table 4

Independent t-test results to compare test variables in two groups of free-tuition and paying-tuition students
	Group	Mean	Std. deviation	t value	Degree of freedom	P value	Lower	upper
Difficulty index	Free-tuition	0.624	0.105	0.185	53	0.854	-0.049716	0.059796
Difficulty index	Paying-tuition	0.619	0.097	0.185	53	0.854	-0.049716	0.059796
Discrimination index	Free-tuition	0.327	0.060	2.023	53	0.048	0.000307	0.072524
Discrimination index	Paying-tuition	0.290	0.073	2.023	53	0.048	0.000307	0.072524
Kuder-Richardson correlation coefficient	Free-tuition	0.610	0.129	0.601	54	0.550	0.039013	-0.054751
Kuder-Richardson correlation coefficient	Paying-tuition	0.586	0.161	0.601	54	0.550	0.039013	-0.054751
standard error of measurement	Free-tuition	0.610	0.129	0.602	54	0.550	0.039143	-0.054905
standard error of measurement	Paying-tuition	0.587	0.162	0.602	54	0.550	0.039143	-0.054905

According to the findings, the mean discrimination index of tests in free-tuition students was higher than paying-tuition students, and this difference is statistically significant (p = 0.048). The mean difficulty index, Kuder-Richardson correlation coefficient, and the standard error of measurement of the tests are not different in the two groups of free-tuition and paying-tuition students.

Table 5 shows the results of the independent t-test to compare the difficulty index, discrimination index, Kuder-Richardson correlation coefficient, and standard error of measurement of the tests between two independent medical and dental students’ groups.

Table 5

Independent t-test to compare test variables in two groups of medical and dental students
	Group	Mean	Std. deviation	t value	Degree of freedom	P value	Lower	upper
Difficulty index	Medical	0.641	0.095	1.701	53	0.095	-0.008215	0.099899
Difficulty index	Dental	0.595	0.103	1.701	53	0.095	-0.008215	0.099899
Discrimination index	Medical	0.324	0.052	1.802	34.184	0.080	-0.004516	0.075494
Discrimination index	Dental	0.288	0.083	1.802	34.184	0.080	-0.004516	0.075494
Kuder-Richardson correlation coefficient	Medical	0.593	0.133	0.601	54	0.550	-0.054751	0.101680
Kuder-Richardson correlation coefficient	Dental	0.605	0.163	0.601	54	0.550	-0.054751	0.101680
standard error of measurement	Medical	0.593	0.133	0.602	54	0.550	-0.054905	0.102048
standard error of measurement	Dental	0.605	0.163	0.602	54	0.550	-0.054905	0.102048

The finding showed that the difficulty index had not differed between the two groups of medical and dental students. According to the results of the Levin test (F = 6.738), the significance level was lower than 0.05 (p = 0.012), so for the difficulty index, the variance of medical and dental groups was not equal. Discrimination index, Kuder-Richardson correlation coefficient, and standard error of measurement of the tests were not different in the two groups of medical and dental students.

Two-way univariate analysis of variance was used to show the interaction between the type of student admission (free-tuition and paying-tuition) and the field (medicine and dentistry) on the two variables (the degree of difficulty and discrimination index of the tests). (Table 6)

Table 6

Univariate two-way analysis of variance to examine the interaction between the type of admission and the field in the difficulty index and discrimination index of tests
		df	Mean square	f	P value
Difficulty index	Admission type	1	0.000	0.040	0.843
	Field	1	0.028	2.790	0.101
	Admission type*field	1	0.000	0.014	0.906
	error	51	0.010
Discrimination index	Admission type	1	0.031	9.724	0.003
	Field	1	0.019	6.092	0.017
	Admission type*field	1	0.057	18.117	0.0001
	error	51	0.003

The interaction between the type of admission and the field was not significant on the difficulty index of the tests (p = 0.906) but was significant on the discrimination index of the tests (p = 0.0001). Therefore, the mean discrimination index of tests in four groups including the tuition-free medical, tuition-free dental, tuition-paying medical and tuition-paying dental groups was not the same. This was shown in Fig. 1. The type of admission in the medical and dental groups was different so that the mean of the discrimination index of tests in tuition-free dental students was more than in tuition-paying students in that field.

This study was conducted to compare the item analysis of multiple-choice questions among medical and dental students in two groups of tuition-free and tuition-paying students in a maximum matched condition. The results of the present study generally showed that the item analysis of multiple-choice questions among the free-tuition and paying-tuition students was the same. Also, the results showed that the item analysis of multiple-choice questions was the same among students in medicine and dentistry.

The results showed that only the differentiation index of tests between free-tuition and paying tuition students was significant. As Taib and Yusoff (2014) pointed, multiple-choice questions can distinguish between good and poor students (10). According to the results of this study, the mean differentiation index of tests in free-tuition students was higher than paying-tuition students, which means that multiple-choice questions in free-tuition students were more differentiated between good and poor students than paying-tuition students. Although the number of test questions was the same for the free and paying tuition students, the variable of student numbers may be affected. The number of free-tuition and paying tuition students was 1731 and 1311, respectively.

In this study, nearly 50% of the questions were considered hard in terms of difficulty. This case indicates that the tests were designed appropriately in the sense of difficulty. Given that the quantitative evaluation of the questions was done under the same conditions, the quality of questions has no relation with the admission type. Nematbakhsh (2015) points out that students' admission type for free or tuition-paying is not an important issue, but how the student graduates are necessary (11). Therefore, we conclude that the students' evaluation methods, whether free or with tuition, should be such that it leads to the promotion of education quality.

According to the results, the mean, standard deviation, standard error of measurement, and Kuder-Richardson correlation coefficient of the tests showed no statistical significance in the two groups of free and paying tuition students in medicine and dentistry. It seems that the type of student admission and their field do not affect the test results.

The statistical results to examine the interaction between the type of admission and the field in the difficulty index showed that the mean difficulty index was the same in both admission type and field groups. Despite examining the interaction between the differentiation indexes of tests with these variables, the results showed that the mean differentiation index was different between the groups of admission type and field. Thus, this difference was higher in tuition-free dental students than tuition-free medical students and tuition-paying dental students.

Two limitations need to be considered for interpretation and future research. First, in this study, the educational objectives in different disciplines were not examined. Although the type of content of the educational program is considered in both medical and dental groups, it is suggested that in another study, other variables such as educational goals in different fields be examined. Second, this study was conducted in one course, which limited the generalizability of the results. Thus, this limitation might be considered during the interpretation of the result.

The type of student admission has no significant effect on students’ assessments in multiple choice tests in matched educational conditions.

MCQ: Multiple choice questions

SPSS: Statistical package for the social sciences

CI: Confidence intervals

Std: Standard

df: Degree of freedom

Ethics approval and consent to participate

This observational study was confirmed by the institutional review board standards at the Golestan University of Medical Sciences, Gorgan, Iran. No: IR.GOUMS.REC.1397.159. No informed consent was obtained from participants because the publication of the results of this study was anonymous and the need for informed consent was deemed unnecessary according to enactment No: 1397.159 of the review board of Golestan University of Medical Sciences.

Consent for publication

Not applicable.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Competing interests

The authors declare that they have no competing interests.

Funding

This work was financially supported by the grant No.269687 from Golestan University of Medical Sciences. The funding agency had no role in the study design; in the collection, analysis and interpretation of data; in the writing of the report; and in the decision to submit the article for publication.

Authors’ contributions

SMH and MHK designed the study. SMH, HS and RR collected data. SMH and MHK analyzed data. MHK and SMH prepared the manuscript. All authors read and approved the final manuscript.

5- Acknowledgments

This work was done in the Golestan University of Medical Sciences. Sincere thanks are given to all students who participated in the study.

Source of financial support or funding: This work was financially supported by the grant No.269687 from Golestan University of Medical Sciences.

The authors declare no conflict of interest financial or otherwise.

This observational study was confirmed by the institutional review board standards at the Golestan University of Medical Sciences, Gorgan, Iran. No: IR.GOUMS.REC.1397.159.

Author details

Seyed Mehran Hosseini, MD. PhD, Associate Professor of Medical Physiology, Department of Physiology, School of Medicine, Neuroscience Research Center Golestan University of Medical Sciences, Gorgan, Golestan, Iran. E-mail: [email protected] ORCID number: 0000-0002-4783-7428

Reza Rahmati, PhD, Assistant Professor of Medical Physiology, Department of Physiology, School of Medicine, Golestan University of Medical Sciences, Gorgan, Golestan, Iran. E-mail: [email protected] ORCID number: 0000-0003-4900-1373

Hamid Sepehry, PhD, Associate Professor of Medical Physiology, Department of Physiology, School of Medicine, Neuroscience Research Center Golestan University of Medical Sciences, Gorgan, Golestan, Iran. E-mail: [email protected] ORCID number: 0000-0003-0319-3867

Mahdi Habibi-koolaee, PhD, Assistant Professor of Medical Informatics, Educational Development Center, Golestan University of Medical Sciences, Gorgan, Golestan, Iran. E-mail: [email protected] ORCID number: 0000-0001-6681-8137

All authors confirm that this work is original and also declare that the article was not published in part or whole, simultaneously in any other journal or proceedings and is not currently under consideration for publication elsewhere. All authors of this manuscript accept the paper in its present condition and are agree with what is expressed in it.

Schripsema NR, van Trigt AM, Borleffs JCC, Cohen-Schotanus J. Selection and study performance: comparing three admission processes within one medical school. Medical Education. 2014;48(12):1201-10.
Yang EB. Comparison of competency level of medical, non-medical students and its relevance for admission policy. Korean J Med Educ. 2018;30(3):219-27.
Matsumoto M, Inoue K, Kajii E. Characteristics of medical students with rural origin: Implications for selective admission policies. Health Policy. 2008;87(2):194-202.
Gallagher J, Calvert A, Niven V, Cabot L. Do high tuition fees make a difference? Characteristics of applicants to UK medical and dental schools before and after the introduction of high tuition fees in 2012. British dental journal. 2017;222(3):181-90.
Thomas B. Free Medical School Tuition: Will It Accomplish Its Goals? JAMA. 2019;321(2):143-4.
CMJ S. Paying tuition and academic performance of students at the Zagreb University School of Medicine. Croat Med J. 2001;42:74-8.
Janković SM, Tufegdzić BI, Vuković MV, Folić M. Paying tuition and academic performance of students at Kragujevac University School of Medicine. Croatian medical journal. 2002;43(1):63-6.
Atuahene F. The impact of “tuition-paying” policy on retention and graduation rates at the University of Ghana. Journal of College Student Retention: Research, Theory & Practice. 2012;14(1):37-65.
Bunce L, Baird A, Jones SE. The student-as-consumer approach in higher education and its effects on academic performance. Studies in Higher Education. 2017;42(11):1958-78.
Taib F, Yusoff MSB. Difficulty index, discrimination index, sensitivity and specificity of long case and multiple choice questions to predict medical students’ examination performance. Journal of Taibah University Medical Sciences. 2014;9(2):110-4.
Nematbakhsh M. Assess a lesson in both public and tuition admission. Iranian Journal of Medical Education. 2015;15(0):189-91.

No competing interests reported.

Download PDF

Version 1

posted

You are reading this older preprint version

Read the latest preprint version →

Comparison of the Statistics and Item Analysis of Multiple-Choice Questions of Medical and Dental Students' Tests Between Free Admission and Tuition-Paying Admission.

Status:

Version 1

Abstract

Figures

Background

Methods

Results

Discussion

Conclusion

Abbreviations

Declarations

References

Additional Declarations

Status:

Version 1