The student learning assessment process should permit students to identify their difficulties in order to help them overcome their shortcomings. This process should also allow teachers to use student performance as a tool to improve instruction to meet their students’ needs [1]. In this context, there are three ways of evaluating the teaching and learning process [2]. The first is the diagnosis, or pre-test, performed to obtain the cognitive dimensions of learning that students possess before starting an educational process, and this defines whether students have the basic knowledge and cognitive ability that are necessary to pursue new learning [2].
The second form of assessment is summative and is applied at the end of the subject or course of study to identify whether students have acquired the skills necessary for the new stages of learning [3]. Summative assessment is traditionally performed by means of tests or discursive questions [3].
The third kind of evaluation is formative, is carried out during a teaching period, and includes the teacher’s feedback on the student’s performance [4]. It aims to verify whether students understand each stage of the educational process by identifying the difficulties and learning possibilities in the cognitive construction of knowledge throughout the undergraduate program [5]. This is the evaluation stage in which instructors can learn from students’ learning results that can help them make changes to their own teaching strategies.
In the medical field, the Progress Test (PT) is a formative assessment tool focused on clinical reasoning, which seeks to identify the student’s progress throughout the undergraduate program [6, 7]. This assessment allows students to have the necessary feedback about their progress, while identifying the difficulties and providing the educational institution with input for the evaluation of the curricular model used [8-10]. The questions are drafted in accordance with and in the context of medical practice, and their purpose is to evaluate whether students meet the expected levels of knowledge for the practice of medicine and to assess their ability to apply complex principles of reasoning, reflection, and judgment in resolving the questions, thereby avoiding the sole measurement of memorized information [11]. The PT, as a mechanism for improving student performance, establishes new teaching and self-evaluation strategies. It is widely recognized and utilized internationally by medical schools with different teaching and learning methodologies [4, 12].
This instrument is an important complement for the assessment of students in different teaching and learning methodologies, from the perspective of a longitudinal approach. The PT is performed at fixed intervals, ranging from one to four times per year [6, 7]. It focuses on the evaluation of students’ knowledge and competencies over the years and is conducted at the same time by students of different schooling levels. Moreover, the PT can also be used for summative purposes [6, 7].
The PT was produced through a collaborative and multicentric approach, intended to be administered by medical schools worldwide. Through simultaneous application in different partner schools, the PT promoted increased efficiency and reduced test administration costs [6, 7]. When the PT is performed multicentrically—in a consortium of several medical schools from the same or different countries—an extensive database of evaluations is created [6-8]. Database consortia such as these provide larger datasets from which educators can detect patterns and create new instructional methodologies to meet students’ needs. These robust databases can be used to provide students with important feedback regarding their strengths and weaknesses in terms of their knowledge [8, 9]. The partnership among schools has contributed to the development of high-quality questions because question writers, reviewers, and test administrators can share their expertise to develop a more comprehensive set of questions [11].
The number of questions used in the PT depends on the consortia designing and implementing them. The Dutch and German consortia, among others, include 200 questions. Others, such as the Canadian consortia, use 180; 120 questions are included on the PT in the United Kingdom. The number of questions varies according to the frequency with which the tests are administered, and, since they are very short, they may underrepresent the curriculum contents, thereby reducing the content validity of the PT [6, 7]. In Brazil, these tests generally consist of 120 multiple-choice questions, each with four alternative answers of which only one is correct. The total resolution time is four hours [13].
However, after completing the course of study, graduates worldwide are faced with summative assessments of a classificatory nature; that is, assessments that enable the successful students to acquire the right to practice the profession or to fill a vacancy in the selection processes, such as Medical Residency (MR) [14]. MR, as a required postgraduate program for physicians, unfolds with in-service training, and is the most important model for the training of specialists [15]. It is considered the gold standard of medical specialization education, directed toward the development of medical skills and professional abilities in a qualified work environment [16].
In Brazil, the program accredited by the National Commission on Medical Residency provides the doctor who performs the training with the title of specialist in the area in which he or she has worked [17]. The average annual workload is 2,880 hours, for a number of years that varies according to the resident’s specialty. There are more than one hundred accredited programs, with some allowing direct access, while others require previous training [15, 16]. As a large number of recent graduates seek to join MR, there are relatively few vacancies in reputable institutions in comparison to the number of trainees [18, 19]. In certain fields, including neurosurgery, dermatology, ophthalmology, and imaging diagnosis [18], openings for the position are highly coveted by candidates. In other areas, there are more vacancies offered than are filled by demand from graduates, such as family and community medicine [19].
Thus, when considering MR as the primary training program for specialized medical practice, one must recognize the fact that, over the years, there has been an increase in competition in the selection processes in general. Therefore, it is crucial to verify during the medical course of study whether there is any indication of particular student performance areas or scores that correlate with successful performance in the selection process for MR.
Moreover, considering the current importance of the PT in the pedagogical context of medical schools and the test’s characteristics, from the form of construction to its application, it is necessary to determine the relationship between these two evaluations. One evaluation is formative, as the PTs are performed throughout the medical course of study, and the other is summative, focusing on the selection processes for MR, and both evaluate fundamental concepts at the conclusion of the medical course. Therefore, the objective of this study is to identify whether student performance in the PT of their final year of study is related to performance in the MR selection process. Specifically, we focus on the possible correlation between students’ PT performance and their performance on the multiple-choice test (MC) leading to selection in MR.