This preprint is under consideration at BMC Medical Education. A preprint is a preliminary version of a manuscript that has not completed peer review at a journal. Research Square does not conduct peer review prior to posting preprints. The posting of a preprint on this server should not be interpreted as an endorsement of its validity or suitability for dissemination as established information or for guiding clinical practice.
Learn more about In Review
Research article
Evaluation of the Effect of Items’ Format and Type on Psychometric Properties of Sixth Year Pharmacy Students Clinical Clerkship Assessment Items
Suha Almuhaissen
University of Jordan
Corresponding Author
ORCiD: https://orcid.org/0000-0002-5177-1222
License:
This work is licensed under a CC BY 4.0 License. Read Full License
Background
Examinations are the traditional assessment tools. In addition to measurement of learning, exams are used to guide the improvement of academic programs. The current study attempted to evaluate the quality of assessment items of sixth year clinical clerkships examinations as a function of assessment items format and type/structure and to assess the effect of the number of response choices on the characteristics of MCQs as assessment items.
Methods
A total of 173 assessment items used in the examinations of sixth year clinical clerkships of a PharmD program were included. Items were classified as case based or noncase based and as MCQs or open-ended. The psychometric characteristics of the items were studied as a function of the Bloom’s levels addressed, item format, and number of choices in MCQs.
Results
Items addressing analysis skills were more difficult. No differences were found between case based and noncase based items in terms of their difficulty, with a slightly better discrimination in the latter . Open-ended items were easier, yet more discriminative. MCQs with higher number of options were easier. Open-ended questions were significantly more discriminative in comparison to MCQs as case based items while they were more discriminative as noncase based items.
Conclusion
Item formats, structure, and number of options in MCQs significantly affected the psychometric properties of the studied items. Noncase based items and open-ended items were easier and more discriminative than case based items and MCQs, respectively. Examination items should be prepared considering the above characteristics to improve their psychometric properties and maximize their usefulness.
Keywords
Assessment Items, Clinical Clerkships, Difficulty Index, Discriminating Index, Point Biserial
Badges
Prescreen
This manuscript passed our Prescreen assessment
Complete author information
Appropriate declaration statements
Potential risks to human health
Prescreen
Peer Review Timeline
CURRENT STATUS: Under Review
Version 3
Posted 19 May, 2020
No community comments so far
Review #1 received
Received 26 May, 2020
Reviewer #1 agreed
On 14 May, 2020
Editor assigned
On 07 May, 2020
Reviewers invited
Invitations sent on 07 May, 2020
Editor invited
On 06 May, 2020
Submission checks complete
On 22 Nov, 2019
No comments provided
Editorial decision: Major revision
On 07 Apr, 2020
Review #1 received
Received 31 Mar, 2020
Reviewer #1 agreed
On 16 Mar, 2020
Reviewers invited
Invitations sent on 13 Mar, 2020
Editor assigned
On 20 Feb, 2020
Submission checks complete
On 19 Feb, 2020
Editor invited
On 19 Feb, 2020
No comments provided
Editorial decision: Major revision
On 05 Jan, 2020
Review #3 received
Received 03 Jan, 2020
Review #2 received
Received 26 Dec, 2019
Review #1 received
Received 19 Dec, 2019
Reviewer #3 agreed
On 16 Dec, 2019
Reviewer #2 agreed
On 12 Dec, 2019
Reviewer #1 agreed
On 04 Dec, 2019
Editor assigned
On 01 Dec, 2019
Reviewers invited
Invitations sent on 01 Dec, 2019
Submission checks complete
On 23 Nov, 2019
Editor invited
On 22 Nov, 2019
First submitted
On 20 Nov, 2019
Metrics
Comments: 0
PDF Downloads: ...
HTML Views: ...
Subject Areas
Internal Medicine
Learn more about our company.
This preprint is under consideration at BMC Medical Education. A preprint is a preliminary version of a manuscript that has not completed peer review at a journal. Research Square does not conduct peer review prior to posting preprints. The posting of a preprint on this server should not be interpreted as an endorsement of its validity or suitability for dissemination as established information or for guiding clinical practice.
Learn more about In Review
Research article
Evaluation of the Effect of Items’ Format and Type on Psychometric Properties of Sixth Year Pharmacy Students Clinical Clerkship Assessment Items
Suha Almuhaissen
University of Jordan
Corresponding Author
ORCiD: https://orcid.org/0000-0002-5177-1222
Badges
Prescreen
This manuscript passed our Prescreen assessment
Complete author information
Appropriate declaration statements
Potential risks to human health
Prescreen
Peer Review Timeline
CURRENT STATUS: Under Review
Version 3
Posted 19 May, 2020
No community comments so far
Review #1 received
Received 26 May, 2020
Reviewer #1 agreed
On 14 May, 2020
Editor assigned
On 07 May, 2020
Reviewers invited
Invitations sent on 07 May, 2020
Editor invited
On 06 May, 2020
Submission checks complete
On 22 Nov, 2019
No comments provided
Editorial decision: Major revision
On 07 Apr, 2020
Review #1 received
Received 31 Mar, 2020
Reviewer #1 agreed
On 16 Mar, 2020
Reviewers invited
Invitations sent on 13 Mar, 2020
Editor assigned
On 20 Feb, 2020
Submission checks complete
On 19 Feb, 2020
Editor invited
On 19 Feb, 2020
No comments provided
Editorial decision: Major revision
On 05 Jan, 2020
Review #3 received
Received 03 Jan, 2020
Review #2 received
Received 26 Dec, 2019
Review #1 received
Received 19 Dec, 2019
Reviewer #3 agreed
On 16 Dec, 2019
Reviewer #2 agreed
On 12 Dec, 2019
Reviewer #1 agreed
On 04 Dec, 2019
Editor assigned
On 01 Dec, 2019
Reviewers invited
Invitations sent on 01 Dec, 2019
Submission checks complete
On 23 Nov, 2019
Editor invited
On 22 Nov, 2019
First submitted
On 20 Nov, 2019
Metrics
Comments: 0
PDF Downloads: ...
HTML Views: ...
Subject Areas
Internal Medicine
License:
This work is licensed under a CC BY 4.0 License. Read Full License
Background
Examinations are the traditional assessment tools. In addition to measurement of learning, exams are used to guide the improvement of academic programs. The current study attempted to evaluate the quality of assessment items of sixth year clinical clerkships examinations as a function of assessment items format and type/structure and to assess the effect of the number of response choices on the characteristics of MCQs as assessment items.
Methods
A total of 173 assessment items used in the examinations of sixth year clinical clerkships of a PharmD program were included. Items were classified as case based or noncase based and as MCQs or open-ended. The psychometric characteristics of the items were studied as a function of the Bloom’s levels addressed, item format, and number of choices in MCQs.
Results
Items addressing analysis skills were more difficult. No differences were found between case based and noncase based items in terms of their difficulty, with a slightly better discrimination in the latter . Open-ended items were easier, yet more discriminative. MCQs with higher number of options were easier. Open-ended questions were significantly more discriminative in comparison to MCQs as case based items while they were more discriminative as noncase based items.
Conclusion
Item formats, structure, and number of options in MCQs significantly affected the psychometric properties of the studied items. Noncase based items and open-ended items were easier and more discriminative than case based items and MCQs, respectively. Examination items should be prepared considering the above characteristics to improve their psychometric properties and maximize their usefulness.