Item analysis of midterm pediatrics multiple choice exams of college of Medicine KKU: a step in evaluation and improvement of assessment

doi:10.21203/rs.3.rs-1994965/v1

Download PDF

Research Article

Item analysis of midterm pediatrics multiple choice exams of college of Medicine KKU: a step in evaluation and improvement of assessment

https://doi.org/10.21203/rs.3.rs-1994965/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Objectives: To evaluate the detailed indices of the analysis of the items of the written multiple choice questions (MCQs) exams in the Department of Child Health, Faculty of Medicine over the last four academic years (1439-1443 AH) and to construct the outlines of a plan for improving the upcoming written MCQs exams.

Methods: This was retrospective cross sectional study on the item analysis of the exams in the MBBS course of pediatrics-2 for both boys and girls group in the Child Health Department, Faculty of Medicine, King Khalid University, Saudi Arabia for the midterm MCQs exams in the years of 1439,1440,1441,1442 and the first semester of girl group in 1443. The total number of items studied in these 16 exams were 643 items. The data was obtained constitute the difficulty, discrimination, point biserial reliability and distractor analysis of each of the exams items. The data was tabulated and the statistical significance determined for some variables in the analysis.

Results: Students’ scores: were as follows: A scored by 7.3% of students, B by 21.8%, C by 33%, D by21.4% and F by 16.5%. Difficulty index: considering a difficulty index of 80% as easy item, of 30% or less as difficult item and that between 30% and 80% of moderate difficult; we obtain 3 categories of items: difficult percentage is 6.6% of the total items, moderate difficulty represent 53.4% and easy items 40% of the total items. There was significant statistical correlation (p ≤ 0.05) when these difficulty levels compared through over the exam years.

Conclusions: Departmental exam committee needs to work comprehensively to improve the difficulty of the exams towards moderate intermediate class also the quality of the questions need extensive work on refining the distractors and revision of the correctness and the suitability of a considerable number of items.

Item analysis

multiple choice questions

Difficulty index

Discrimination index

Distractors

Non-functional distractor

Reliability

Consolidating the concept of evaluating the MCQs item analysis
Providing a solid base for student post-test feedback
Use in post-test consideration of omission of items with poor indices
Activation of the role of the departmental exam committee in assessments quality
Selection of well-constructed questions for the development of the questions bank

Item analysis is the process involving evaluation of the individual student responses to the test items (questions) so as to assess and determine the quality of those questions and of the whole test. Item analysis is particularly valuable in improving items that will be needed for use again in later examinations, but also can be used to eliminate ambiguous or misleading items in a single test administration (Scorepak 2015). In the item analysis; statistical techniques are used for selecting and rejecting the items of the test on the basis of their difficulty value and discrimination power. Item analysis helps educators to get active feedback from students’ performance and identify areas which require more emphasis, reinforcement or an alteration in teaching methodology perhaps using other learning facilities (Annthakrishnan 2000 p.131-137) Item analysis is also important for increasing instructors’ skills in test construction, and identifying specific areas of course content which need greater emphasis or clarity. Item analysis enables identifying good MCQs based on difficulty index (DIF I) or facility value (FV) or P-value, discrimination index (DI), and distractor effectiveness (DE). High quality MCQs, require well-written alternative answers (Tarrant M et al. 2009). Point Bi-serial correlation (PBS) is yet another important parameter which gives information about how consistent an item with the whole test is. PBS helps in indicating items which are not testing the same domain and construct as the remaining part of the test and hence aids in improving the validity and reliability of the test. Assessing the quality of items used in a test can assess test. However, the reliability coefficient and standard error of measurement help to evaluate the performance of the whole test. Reliability tells us whether a test is likely to yield the same results if administered to the same group of test-takers multiple times. The most frequently reported internal consistency reliability estimates are the K-R20 and Cronbach's alpha. The indices test the quality of items (Gajjar et al. 2014, p.17-20; Patel 2017, p. 22-29).Multiple choice questions is preferred over other tools of assessment because of its objectivity in assessment, comparability in different settings, wide coverage of subject, and minimization of assessor’s bias. Designing good MCQs is a complex, challenging, and time-consuming process. Having constructed and assessed, MCQs need to be tested for the standard or quality. Item analysis examines the student responses to individual test items (MCQs) to assess the quality of those items and test. (Singh et al.2009, p. 70-77). Quality educational assessment is a purposed process with student feedback assessment analysis assist in this objective (Hervas 2020). Item analysis is the process of collecting, summarizing, and using information from student's responses to assess the quality of multiple-choice questions (MCQs) (Rao et al. 2016). The best assessment method must meet five criteria which include reliability, validity, acceptability, feasibility and educational impacts on learning and practice (Van der V. and leuten 2000)

Quantitative item analysis is a statistical technique meant to know about the test items and item concerned on the basis of three numerical indicators:

Difficulty index or factor (DIF I): if test item was easy or difficult for the specific grot-JP of students. Acceptable item difficulty is how many exam takers answered the item correct. There is no a set number; If the intent is a mastery item, a difficulty level between 0.80 and 1.00 is acceptable. If the intent is a discriminating question, a range of 0.30 to 0.70 is generally acceptable items with DI below 0.3 generally need revision (Zurawski1998)
Discrimination index (DI): how well item discriminated between high and low scorers in the test.This value is based on the top 27% scorers (HA) and bottom 27% scorers (LA) of the class on the exam. Computed by subtracting the number of successes by the low group on the item from the number of successes by the high group, and divide this difference by the size of the class. Ranges from -1.0 to +1.0; the closer to 0.0 indicates no discrimination among high- and low-performing students. Achieving closer to 1.0 discrimination index is optimal. A discrimination index of 0.3 or greater is considered highly discriminating, with no need of item revision. With a discrimination index of less than 0.3 usually the item need revision or elimination (Ermie 2017).
Distractor analysis or efficiency (DE): If all the alternatives functioned as intended with equal distribution in choosing the wrong keys. (Ermie 2017). The quality of MCQs items depend on these 3 indices (Zurawski1998).
Point biserial; generally a value of 0.2 and above is considered to have high correlation and positive association with overall performance on the assessment (i: e, acceptable ranges are 0.2 or above 0.2) lower levels are acceptable for mastery; and 0.3 or higher are best for discriminating questions A positive point biserial indicates that those scoring high on the total exam (Ermie 2017).
Reliability coefficient is the extent to which the test is likely to produce consistent scores and this implies the study of inter-correlations between the test items, the test size (more items more reliability) and test content. ranges from zero (no reliability) to 1.00(perfect reliability) but practically the range is usually from 0.5 to 0.9 the bigger this ratio the more .This Kuder-Richardson Formula 20, often abbreviated KR-20, is used to measure the internal consistency reliability of the test and is influenced by number of items in the exam (Zurawski1998; Ermie 2017).

The objectives of this study: were to evaluate the detailed indices of the analysis of the items of the written multiple choice questions (MCQs) exams in the department of child health in the Faculty of Medicine over the last four academic years (1439-1443 AH) and to construct the outlines of a plan for improving the upcoming written MCQs exams in the department based on the indicators of the analysis.

This was a retrospective cross sectional study performed by collecting the data regarding the item analysis of all midterm written MCQs exams in the MBBS course of pediatrics -2 for both boys and girls group . The data was obtained from the examination department and constitute the difficulty, discrimination, point biserial reliability and distractor analysis of each of the exams items for the whole years of 1439,1440,1441,1442 and the first semester of girl group in 1443. The indices of the exams analysis were automatically calculated electronically and registered by the correcting scanner machine. The total number of items studied in these 16 exams were 643 items. All of the items were included in the study and the results were tabulated and the different statistical correlations were computed using spss program and taking the significant level when the p-value is less than 0.05. The reliability of the exams were also determined according to KR-20 method.

The ethical approval for this study was given by the King Khalid University committee of research ethics on 17.2.2022 by the approval number (ECM#2022-607). There is no disclosure of any personal student information in this study.

No conflict of interest in the study

Financial support is nil

In this study 16 MCQs exams were evaluated for the item analysis of the major indicators of the exams quality .Of each of these exams 4 modules containing similar questions were prepared and taken by the students as part of their evaluation in pediatrics course in level 11 .These exams were constituted of 643 items or questions with average number of items of 40 per one exam .The total number of students who performed these exams were 1002 students .There were 7 groups of boys composed of 593 students (59% of the total students) and 9 groups of girls composed of 409 students (41% of the total number of the students) . The average number of students per one boy group is 84, while in girl group is 45 per one group.

1. Grades and scores of the student groups :The student scores A if he or she correctly answers 90% or more of the items , B between 80% to 90% correct, C between 70% to 80% correct , D between 60% to 70% and F if below 60% . The distribution of the students according to this score was as follows: A scored by 7.3% of students, B by 21.8%, C by 33%, D by 21.4% and F by 16.5%

(Table -1). These score values in general were of very comparable distribution between boys and girls with no significant statistical difference (p-value= 0.276). Scores of the first semester and second semester groups when compared there was a significant statistical difference (p=0.01)

2. Difficulty evaluation of the exams: Overall when considering a difficulty index(DIF I) of 80% as easy item, of 30% or less as difficult item and that between 30% and 80% of moderate difficult ; we obtain 3 categories of items : difficult percentage is 6.6% of the total items, moderate difficulty represent 53.4% and easy items 40% of the total items. There was significant statistical correlation (p ≤ 0.05) when these difficulty levels compared over the exam years. In this same group of items as the recommendation of some educators and to increase the standard levels of our tests we chose to consider a difficulty index of 50 % to 60% as ideal, of 30% to 49% and 61% to 70% as acceptable, of less than 30% as very difficult item and of more than 70% as very easy item; the findings in our exam series were as follows only 11% of the items were of ideal difficulty index and 21.4% were of acceptable difficulty; so that the total of these 2 categories of reasonable difficulty constitute 32.4%. Items which were very easy constitute 61% which seems higher than the intended goal, while very difficult items were 6.6% which is not high. Difficulty indices of the studied items for boys and girls exams and the statistical correlations are shown in tables -2 and -3 respectively

3. Discrimination index(DI) : in this study an item with discrimination of 0.4 or above was considered as a very good discriminating item ; good discriminating item if the DI between 0.3 to0.39, moderately discriminating if between 0.2 to 0.29, poorly discriminating if between 0.1 to0.19 and not discriminating if below 0.1.The average discrimination of the items in the study according to this scale was as follows ; very good DI in 51.5%,good DI in 11.3%, moderate DI in 9.4% ,poor DI in 2.7%, unacceptable DI in 25.1% . Therefore acceptable discrimination of the items in this study was 72.2% with 27.8% of the items being of poor or no discriminating function. Discrimination indices of the studied items for boys and girls exams and the statistical correlations are shown in tables -2 and -3 respectively.

4. Point biserial index and reliability: as an indicator of exam discrimination and internal consistency and also for validity and reliability; the point biserial of the items of the exams was of good performance of more than 0.15 in average of 69% of the items and not performing well in 31% indicating necessary revision of the correctness of the keys in this group. Reliability detection according to point biserial was compared statistically between the girl and boys groups and the difference was significant (p=0.002) (Table -4). The average KR-20 calculated for the whole exams was about 71%. KR-20 for the years (1439-1441AH). The reliability according to this index is shown in figure-3 and the statistical association of reliability compared to the different exam years and also to the gender of the students was not significant.

5. Non-functioning distractors percentage: Percentages of zero non-functioning distractors (ANFD), 1NFD (BNFD), 2NFD (CNFD) and 3 NFD (DNFD) of the items of the exam series are shown in Figure -1for male groups and Figure -2 for female groups. These different 4 categories of distractor indices range from 20% to 35% in males and from 20% to 30% in female groups and the levels in the exams as compared over the last 4 years was not statistically significant (Tables-2 and 3)

MCQ form useful assessment tool in measuring factual recall and if carefully constructed can test higher order of thinking skills which is very important for a medical graduate. The method of assessment should be regularly evaluated. Developing an appropriate assessment strategy is a key part in curriculum development. It is important to evaluate MCQ items to see how effective they are in assessing the knowledge of students (Gajjar et al. 2014). This the most widely used test format in health sciences today. The efficiency of MCQs as an efficient tool for evaluation depend mainly on their quality which is best assessed by item and test analysis (Poulomi et al. 2015, p. 47–52)

Item analysis is especially valuable in improving items which will be used again in later tests, but it can also be used to eliminate ambiguous or misleading items in a single test administration. In addition, item analysis is valuable for increasing instructors’ skills in test construction, and identifying specific areas of course content which need greater emphasis or clarity. Separate item analyses can be requested for each raw score. (Pande et al. 2013, p. 45–50). Such statistics must always be interpreted in the context of the type of test given and the individuals being tested (Lord 1952, p. 181–194)

According to the results of this study many aspects of our MCQs exams in the department of pediatrics require further deep study and posttest re-evaluation as for consideration in the same test or the future use of the items. Optimizing the different important item analysis indices is an essential aim of medical exams constructors. A study by Poulomi et al. concluded that items with average difficulty, high discrimination and functional distractors are the best to be incorporated in the exams (Poulomi et al. 2015, p. 47–52). Avery similar conclusion was reached to by the study performed by Durgesh et al (Durgesh et al. 2017, p. 5351–5355) Determination of the item analysis indices can -to certain extent- differ according to the cognitive levels of Bloom taxonomy; Serpil et al in their study found that difficulty and discrimination were more associated with the remembering and understanding levels than with the applying level (Serpil 2016, p.16–24). In any case, all indices should be considered together before making decisions or revisions (Suskie 2017).

Comparable and similar results to this study were also reported by other authors when they evaluated the MCQs tests in their courses. Tarik Al Shaibani et al concluded that the mean DIF- I, DI and DE were in the acceptable ranges. A high percentage of items was easy, and a high percentage of distractors were NFDs. These distractors need to be revised to improve the DIF-I, DI and DE parameters. The reliability of the exams was acceptable (Al Shaibani et al. 2021, p. 471–476). Vrunda Kolter et al study results were as such: difficulty index of analysed MCQs ranged from 6.25% (lowest) to 90.6% (Highest) & discriminative index ranged from 0.0 (lowest) to 0.63 (Highest). Total 65% items were in acceptable range of difficulty level (‘p’ value 30–70%) & 10% items were very difficult which later discussed with students. Discrimination index of 60% items was excellent (d value > 0.35). No item had Negative discriminative power. About 47.5% items had100% Distracter Efficiency (DE) whereas 7.5% items had 0% DE (Vrunda 2015, p.320–326). These two studies have figures of indices of item analysis which may resemble or differ from our figures which indicate the need of individualization of the analysis of every institutional exam and looking for underlying factors of the analysis result. Many other similar studies in medical schools in the Gulph or other regional parts obtain useful results and indicators for improvement of their MCQs exams ; for example the studies done by Rao C et al, Prashant E. et al. and Deena Kheyami et al. (Rao et al. 2016; Prashant et al. 2016 Deena et al. 2018). Gujarat et al in their detailed study with objectives close to this study in assessing the analysis indices stated that out of 50 items, 24 had “good to excellent” DIF I (31–60%) and 15 had “good to excellent” DI (> 0.25). Mean DE was 88.6% considered as ideal/ acceptable and non-functional distractors (NFD) were only 11.4%. Mean DI was 0.14. Poor DI (< 0.15) with negative DI in 10 items indicates poor preparedness of students and some issues with framing of at least some of the MCQs. Increased proportion of NFDs (incorrect alternatives selected by < 5% students) in an item decrease DE and makes it easier. There were 15 items with 17 NFDs, while rest items did not have any NFD with mean DE of 100% compared with this present study there was no negative DI which is a positive finding in our setting. Actually they proposed the cause for negative DI in their sample that it can be wrong key, ambiguous framing of question or generalized poor preparation of students as was the case in their study where the overall score of their students was very poor (0–33/100) and none of them secured passing marks. Items with negative DI are not only useless, but actually serve to decrease the validity of the test (Gajjar et al. 2014)

Grades and scores of the student groups in the present study: the curve of grade distribution is acceptable as most of the students lie in the central part of the curve giving somewhat a normal distribution; however there is tendency of F results to double the A scores which is a source of some annoyance. The difference between the student scores in the first compared to the second terms may be attributable to the different choice level of students when they are initially accepted to the college as students with higher levels at entrance were those of the first term.

Difficulty evaluation of the exams: to calculate the Index of Difficulty we need three parameters. They are a) number of higher achievers (HA), b) number of lower achievers (LA) and c) total number of respondents (N) (M. Senthil et al. 2020, p. 80–85). At the end of the Item Analysis report, test items are listed according their degrees of difficulty (easy, medium, and hard) and discrimination (good, fair, and poor). These distributions provide a quick overview of the test, and can be used to identify items which are not performing well and which can perhaps be improved or discarded with this high percentage of very easy items in our midterm exams of 61% measures are needed to improve the difficulty levels of the items (although the difficulty is lower than average in some of the groups). Part of the problem may be attributed to the repetition of certain questions in a way or another and part may be due to possession of the students of large question banks with consumption of many questions used by the department. This necessitates renewal of the questions bank in the departmental exam committee and more effort of the faculty members to re –edit or replace the questions. Ideal difficulty levels for 4 response multiple-choice items in terms of discrimination potential is approximately 74%. This is highly recommended to improve the levels of the exams (Lord 1952, p. 181–194). There are instances when the value of DI can be 60% will result into to inflated scores and a decline in motivation. Too easy items should be placed either at the start of the test or better to be removed altogether, similarly difficult items should be reviewed for possible confusing language, areas of controversies, or even an incorrect key (Gajjar et al. 2014). A study by Kishore K.et al concluded that even if a single distracter is non-functioning or poor, it would seriously affect the psychometric properties and reliability of the test paper in a 3-option format. The present data clearly provides evidence that the items with three non-functioning distracters have serious psychometric inadequacy (Deepak et al. 2015, p. 428–435). Ideally, if the distractors are properly designed, it should lead to LA group selecting the incorrect options more often than the HA group (M. Senthil et al. 2020, p. 80–85). In their review of functioning and non-functioning distractors in 514 four option MCQs assessments, Tarrant et al; found that only 13.8% of all items had three functioning distractors and just over 70% had only one or two functioning distractors (Tarrant M et al. 2009).

A reliability coefficient between 70% and 80% is good for classroom tests Good for a classroom test; in the range of most. There are probably a few items which could be improved. While that of level 60% to70% is somewhat low. This test needs to be supplemented by other measures (e.g., more tests) to determine grades. There are probably some items which could be improved. Between 50% and60% there is a need to revise the whole test (Scorepak 2015). Factors affecting the reliability of the test are multiple and include the length of the test with improved reliability with more number of questions, proportion of correct and incorrect responses, item difficulty as very easy and very difficult items do not discriminate properly and also number and individual factors regarding the examinee (Suskie 2017). Thus through item analysis, the instructors can improve their skill in constructing valid MCQs in the future. In addition it also directs the curriculum administrators to identify specific areas of the course content that needs revision or further clarification as evidenced by poor mastery of the subject (M. Senthil et al. 2020, p. 80–85). In a study by Shikha S. et al; after the analysis of their MCQs test and considering the DIFI, DI and DE together, 23.33% items were validated for MCQ Bank, 56.67% items would be re-validated after revision and modification, and 20% items were discarded(Shikha et al. 2016, p. 263–269).

Based on the findings of this study that the percentage of easy items approaching that of those with average difficulty, non-discriminating items in one fourth of the tests, average reliability just over 70% and average of 3 NFD of about 24%; it is of vital importance to the exam committee in the department to get benefits of these item analysis indicators to improve the quality and educational and assessment yield of the written MCQs exams. The followings are suggested practical steps for the committee towards that goal

Faculty member extensive training on proper construction of the stems, keys and distractors of the MCQs in a detailed professional manner to improve the pre-test preparation of the exam and improve the sense of item adjusting regarding the most important indices at the expected desired level.

Committee exam meetings to revise every item in context of the ideal MCQs construction guidelines with efforts of all of the members to have their inputs in putting the best possible items

The post-test meeting for discussion of the exam item analysis is a neglected important part of the job. With consensus of the committee some poor items can be omitted form the correction and some of the good items can be selected to enrich the departmental questions bank for reuse in the future exams; with rejection of items with improper indices as not accepted for developing the departmental bank items.

Conflict of interest:

No conflict of interest is claimed by this study

Funding:

None

Al Shaibani T, Fuad A, Deifalla AH, Jaradat A. Item Analysis of type "A" Multiple Choice Questions from a multidisciplinary Units assessment in a Problem Based Curriculum Bahrain Medical Bulletin, 2021; 43(2); 471-476
Ananthakrishnan N. Item Analysis validation and banking of MCQs. In: Medical Education Principles and Practices. 2nd ed. Ananthkrishnan N, Sethuraman KR, kumar S. Alumini Association of National Teacher Training Centre, JIPMER, Pondichery, India.2000; p,131-137.
Deena Kheyami, Ahmed Jaradat, Tareq Al-Shibani, and Fuad A. AliItem Analysis of Multiple Choice Questions at the Department of Paediatrics, Arabian Gulf University, Manama, Bahrain,Sultan Qaboos Univ Med J. 2018; 18(1): e68–e74.
Deepak KK, Al-Umran KU, Al-Sheikh MH, Adkoli BV, Al-Rubaish A. Psychometrics of Multiple Choice Questions with Non-Functioning Distracters : Implications to Medical Education Indian J Physiol Pharmacol 2015; 59(4); 428-435
Durgesh Prasad Sahoo, Rakesh Singh ; Item and distracter analysis of multiple choice questions (MCQs) from a preliminary examination of undergraduate medical students ; International Journal of Research in Medical Sciences Sahoo DP et al. Int J Res Med Sci. 2017; 12: 5351-5355
Ermie E. Psychometrics 101: Know what your assessment data is telling you. Slides presented at the Exam Soft Assessment Conference 2017, Denver, Colorado. 2017.
Gajjar S, Sharma R, Kumar P, Rana M. Item and test analysis to identify quality Multiple Choice Questions (MCQs) from an assessment of medical students of Ahmedabad, Gujarat. Indian journal of Community Medicine 2014; 39:17-20.
Hervas, Gabriel. Features and qualities of educational assessment. 24. 8-10. Center for Teaching and Learning International Christian University. 2020.
Lord, FM. The Relationship of the Reliability of Multiple-Choice Test to the Distribution of Item Difficulties, Psychometrika, 1952; 18, 181-194
Pande SS, Pande SR, Parate VR, Nikam AP, Agrekar SH. Correlation between difficulty and discrimination indices of MCQ’s informative exam in physiology. South East Asian J Med Educ 2013; 7:45-50.
Patel R M;Use of item analysis to improve quality of multiple choice questions in ΙΙ MBBS. J Educ Technol Health Sci 2017; 4(1): 22-29.
Poulomi Mukherjee ,Saibendu Kumar Lahiri Analysis of Multiple Choice Questions (MCQs): Item and Test Statistics from an assessment in a medical college of Kolkata, West Bengal IOSR Journal of Dental and Medical Sciences (IOSR-JDMS) 2015; 14(12): 47-52.
Prashant E. Natekar, Fatima De Souza; Item Analysis (Postvalidation) of MCQs in Anatomy for the First MBBS Course. Indian Journal of Anatomy, 2016; 5(1): 29-32.
Rao Chandrika & Prasad HL, Kishan & Sajitha, Kaliyath & Permi, Harish & Shetty, Jayaprakash. Item analysis of multiple choice questions: Assessing an assessment tool in medical students. International Journal of Educational and Psychological Researches. 2016; 2. 201. 10.4103/2395-2296.189670.
Scorepak®: Item analysis. Available from: www.washington.edu./oea/score1/htm. [Last Accessed on 13 September 2015].
Serpil Kocdar, Nejdet Kardag, Murat Doğan Şahin;Analysis of the Difficulty and Discrimination Indices of Multiple-Choice Questions According to Cognitive Levels in an Open and Distance Learning Context; Turkish online Journal of Educational Technology. 2016; 15(4): 16-24
Shikha Saxena, P. C. Srivastava, Ayaz k. Mallick, H. S. Joshi, Biswajit das; Item analysis: An unmatched tool for validating MCQs in Medical Education. Indian Journal of Basic and Applied Medical Research, 2016; 5(4): 263-269.
Singh T, Gupta P, Singh D. Test and Item analysis. In: Principles of Medical Education. 3rd ed. New Delhi: Jaypee Brothers Medical Publishers (P) Ltd; 2009;70-77.
Suskie, L. Making Multiple Choice Tests More Effective. Schreyer Institute for Teaching Excellence. The Pennsylvania State University. 2017.
Tarrant M, Ware J, Mohammed AM. An assessment of functioning and nonfunctioning distractors in multiple choice questions: a descriptive analysis. BMC Medical Education 2009; 9:1-8.
Van der , Vleuten C. ; Validity of final examinations in undergraduate medical training. BMJ; 2000; 221 7270)
Velou MS, Ahila E. Refine the multiple choice questions tool with item analysis. IAIM, 2020; 7(8): 80-85.
Vrunda Kolte. Item analysis of Multiple Choice Questions in Physiology examination; Indian Journal of Basic and Applied Medical Research, 2015; 4(4): 320-326.
Zurawski RM. Making the most of exams: procedures for item analysis- The Notional Teaching & Learning Forum- 1998; 7(6): http:// www ntlf.com/html/pã/9811/v7n6smpLp6f

Table 1

Sscores of the students in the study group

Girl group n =409	Boys group n = 593	Combined students group n = 1002	Score
8.7%	8.2%	7.3%	A
26.4%	24%	21.8%	B
29.5%	31.2%	33%	C
20.2%	20.8%	21.4%	D
15.2%	15.8%	16.5%	F
100%	100%	100%	Total %

Table 2

the different important item analysis of the exams in boys groups years (1439-1442 H) and their statistical significance ratio

P value	1442		1441		1440		1439
	2nd semester	1st semester	2nd semester	1st semester	2nd semester	1st semester	2nd semester	1st semester
Difficulty index
P < 0.05	16.9%	9.1%		10.6%	13.4%	7.5%	2.2%	7.5%	Ideal
P < 0.05	34.4%	18.2%		23.3%	23.3%	27.5%	2.2%	18.3%	Acceptable
P < 0.05	0.6%	1.1%		6.7%	0%	10.8%	11.1%	20%	Very difficult
P < 0.05	48.1%	71.6%		59.4%	63.3%	54.2%	84.4%	54.2%	Very easy
Discrimination index
N. S	58.1%	55.7%		49.4%	64.4%	57.5%	51.1%	52.5%	Very good test
P < 0.05	12.5%	1.7%		6.7%	4.5%	8.4%	15.6%	2.5%	Good test
N. S	19.4%	27.9%		22.2%	21.1%	19.1%	8.9%	15%	Moderately discriminating
P < 0.05	0%	0%		5%	0%	1.7%	17.8%	17.5%	Not discriminating (marginal item)
N. S	10%	14.7%		16.7%	10%	13.3%	6.7%	12.5%	Poor quotation
Distractor index
N. S	23.3%	27.6%		24.5%	22.7%	27.8%		25.8%	A NFD
N. S	26%	27.3%		21.3%	34.6%	26.3%		23.1%	B NFD
N. S	27.9%	19.8%		23.8%	21.1%	22.2%		25.8%	C NFD
N. S	22.8%	25.3%		30.4%	21.6%	23.7%		25.3%	D NFD

Table 3

the different important item analysis of the exams in girls groups years (1439-1442 H) and their statistical significance ratio

P value	1443	1442		1441		1440		1439
	1st semester	2nd semester	1st semester	2nd semester	1st semester	2nd semester	1st semester	2nd semester	1st semester
Difficulty index
N. S	13.8%	5.6%	12.8%	8.5%	16.7%	8.9%	16.7%	15.5%	10%	Ideal
N. S	11.3%	15.6%	27.2%	16.5%	21.1%	26.7%	24.2%	24.5%	28.3%	Acceptable
N. S	6.3%	6.3%	6.1%	5.1%	6.7%	4.4%	8.3%	4.4%	8.3%	Very difficult
N. S	68.7%	72.5%	53.9%	69.9%	55.5%	60.0%	50.8%	55.6%	53.4%	Very easy
Discrimination index
N. S	45.6%	41.2%	62.2%	33%	46.1%	51.7%	50.0%	60.0%	50.0%	Very good test
P < 0.05	27.5%	0%	16.7%	35.8%	17.8%	7.7%	15.0%	0%	6.7%	Good test
P < 0.05	0%	0%	0%	0%	0%	0%	6.7%	0%	7.5%	Moderately discriminating
N. S	0%	0%	0%	0%	0%	0%	0%	0%	0%	Not discriminating (marginal item)
N. S	26.9%	58.8%	21.1%	31.2%	36.1%	40.6%	28.3%	40%	35.8%	Poor quotation
Distractor index
N. S	26.2%	23.3%	22.8%	25.1%	21.7%	27.9%	22.1%	25.0%	24.5%	A NFD
N. S	22.6%	25.8%	24.4%	21.5%	28.9%	21.9%	27.3%	24.7%	24.0%	B NFD
N. S	25.8%	26.0%	30.0%	30.2%	25.1%	24.7%	29.0%	25.3%	28.6%	C NFD
N. S	25.4%	24.9%	22.8%	23.2%	24.3%	25.5%	21.6%	25.0%	22.9%	D NFD

Table 4

Point biserial indicators of exam reliability in the item series with comparison between boys and girls group (p = 0.002)

1443 Sem1	1442 Sem2	1442 Sem1	1441 Sem2	1441 Sem1	1440 Sem2	1440 Sem1	1439 Sem2	1439 sem1	interpretation	p. biserial	sex
Not included	92.5	90.3	MCQs Exam Not done	75.5	88.9	61.8		66.7%	Performing well	(≥ 0.15)	Boys
Not included	7.5	9.7	MCQs Exam Not done	24.5	10.1	38.2		33.3%	Should be examined for a possible incorrect key	(< 0.15)
100%	100%	100%	100%	100%	100%	100%	100%	100%	Total
70.6	42.5	81.7	69.3	63.9	66.7	66.7	55.6	51%	Performing well	(≥ 0.15)	Girls
29.4	57.5	18.3	30.7	26.1	33.3	33.3	44.4	49%	Should be examined for a possible incorrect key	(< 0.15)
100%	100%	100%	100%	100%	100%	100%	100%	100%	Total

No competing interests reported.

Download PDF

Version 1

posted

You are reading this latest preprint version

Item analysis of midterm pediatrics multiple choice exams of college of Medicine KKU: a step in evaluation and improvement of assessment

Status:

Version 1

Abstract

Figures

Practice points

Introduction

Methodology

Results

Discussion

Conclusions And Recommendations:

Declarations

References

Tables

Additional Declarations

Status:

Version 1