Development of teaching quality evaluation questionnaire for preclinical courses: an exploratory factor analysis

Background Course quality assessment contributes to evaluate teaching effectiveness and to improve student learning. Several course quality assessment questionnaires have been carried out associated with teaching quality evaluation of medical education. However, little is known about views of medical students regarding the quality of preclinical courses. To ll this gap, we aimed to develop a novel multi-dimension instrument for assessing teaching quality with the perception of medical students in preclinical courses. Methods The original Teaching Quality Evaluation Questionnaire (TQEQ) containing seven dimensions: course contents, teaching abilities, teaching methods, teaching attitudes, learning outcomes, teacher characteristics, student subjection, consisting of 47 items was formed according to literature reviews and group design. We sent the original questionnaires with items in a random order to medical students of Shanghai Jiao Tong University School of Medicine. After collecting the valid questionnaires, the exploratory factor analysis was conducted to assess construct validity and Cronbach’s alpha coecient was used for evaluating internal consistency reliability of the questionnaires. Results In total, 590/646 (91.3%) of participants completed the questionnaire regarding preclinical course evaluation. The exploratory factor analysis yielded seven common factors, learning outcomes, teaching attitudes, student subjection, teaching abilities, teaching methods, teacher characteristics, teaching interactions, consisting of 39 items explained 58.449% of total variance and the factor loading value was above 0.4. In addition, Cronbach’s alpha coecients ranged from 0.669 to 0.914. Conclusion This study provides a new, validated and useful instrument for measuring teaching effectiveness of preclinical courses based on the views of medical students. It is feasible for use in medical schools. questionnaire contributes to facilitate teaching quality of preclinical courses and student learning outcome. This study provides a new, validated and useful instrument for measuring teaching effectiveness of preclinical courses based on the views of medical students. The Exploratory factor analysis of the new teaching quality evaluation scale demonstrated seven factors consisting of 39 items: learning outcomes, teaching attitudes, student subjection, teaching abilities, teaching methods, teacher characteristics and teaching interactions. This new scale is based on the views of medical students. It is feasible for use in medical schools to evaluate the preclinical courses.


Background
Medical education must meet high standards as medical students carry more responsibility in the future.
In order to ensure the effectiveness of the medical courses, course evaluation is an integral of medical education [1][2].
Rating scale is emerging as an important determinant in the course evaluation of medical education. For instance, the Course Experience Questionnaire (CEQ), containing ve main factors: good teaching, clear goals and standards, appropriate workload, appropriate assessment and generic skills [3], has been demonstrated validity in the medical education setting during preclinical courses [4].The Student Evaluation of Educational Quality (SEEQ) was widely used in higher education institution in United States [5]. Thus, a good quality of preclinical course assessment could also help teachers to improve teaching effectiveness and outcomes.
Traditional lecture-based learning (LBL) still remains the mainly method in the process of preclinical teaching in medical schools in China, though, a growing number of medical schools have adopted the popularity teaching methods in medical courses, such as problem-based learning (PBL) [6], team-based learning (TBL) [7]. So far, although many studies have carried out different questionnaires associated with teaching quality evaluation of medical education, such as clinical teaching [8], outpatient teaching [9], bedsides teaching [10], PBL [11], there lacks views of medical students regarding the quality of preclinical courses.
There is evidence that student perceptions are more predictive of student learning outcomes than other methods such as external observations or teachers' subjective perceptions of their own teaching behavior [12]. The reliability and validity of student ratings is, to a certain degree, compromised because student evaluations usually re ect their expectations about the teacher, despite of a bigger expenditure [12].
Therefore, in this study, we develop and validate a novel questionnaire covering seven dimensions from the view of medical students, hoping to have an accurate and profound understanding of our students and help teachers and faculties to improve the preclinical courses.

Item generation and scale development
The items were yield through literature reviews and group design. It was arranged to contain seven dimensions and each dimension was designed to cover 5-to 7-items. Each item was measured by a vepoint Likert-type scale, with 1 represents strongly disagree, 2 represents disagree, 3 represents neutral, 4 represents agree, 5 represents strongly agree. Ultimately, an original version of The Teaching Quality Evaluation Questionnaire, consisting of 47 items, was generated. On the top of the questionnaire, it included the investigators information, such as class, grade, and major. Each item of the questionnaire was in a random order when sent to the participants (Table 1).

Participants
We investigated students whose major were nutrition, clinical medicine, pediatrician and public health in Shanghai Jiao Tong University School of Medicine. The inclusion criteria for participants were having the compulsory curriculum at grade 2 to grade 4.

Data analysis
We used SPSS 22.0 to analyze the data. Item scores was calculated as Mean ± standard deviations (SD). Exploratory factor analysis (EFA) was performed to assess the construct validity, while Cronbach's alpha coe cient was applied to examine the internal consistency.

Exploratory Factor Analysis
The number of respondents between ve to ten participants per item is essential for EFA [13]. The Kaiser-Meyer-Olkin (KMO) measure and Bartlett's test of sphericity were applied to explore the sampling adequacy of factor analysis. Then, EFA using principal component analysis with varimax rotation was conducted to determine the factor structure of the items. Factor with an eigenvalue higher than one was considered for extraction [13]. Factor loadings, equal to or greater than 0.40, were selected [14].

Internal consistency
Cronbach's alpha coe cient was conducted to measure the consistency of each item. The satis ed Cronbach's alpha coe cient was more than or equal to 0.70.

Ethics approval
This study was approved by the Ethical Committee of Shanghai Jiao Tong University School of Medicine.
Study participants han signed informed consent and all data were collected anonymously.

Missing values and descriptive statistics
In total, 646 medical students participated in this study, and 616 surveys were returned. We screened out and eliminated the samples having missing, outliner or invalid values by descriptive statistics. Totally, there were 590 valid and completely questionnaires. The response rate was 91.3%.

Structural validity
The KMO measure of sampling adequacy was 0.946 and the Bartlett's test of sphericity was signi cant (χ2 = 1081.43, p < 0.001), indicating the data were satis ed for factor analysis [15]. Seven common factors, eigenvalues greater than 1, were extracted using principal component analysis with varimax rotation (Fig.1). The seven-factor solution accounted for 58.449% of the variance ( Table 2). The items were removed if the factor loading was lower than 0.40. The rst factor (F1) was named learning outcomes, which involved thirteen items; the second factor (F2) was named teaching attitudes, which involved ve items; the third factor (F3) was named student subjection, which involved four items; and the fourth factor (F4) was named teaching abilities, which involved ve items; the fth factor (F5) was named teaching methods, which involved ve items; the sixth factor (F6) was named teacher characteristics, which involved four items; the seventh factor (F7) was named teaching interactions, which involved three items.

Internal consistency reliability
The Cronbach's alpha coe cient of the new 39-item scale was 0.941, and the Cronbach's alpha coe cients of the subscales raged from 0.669 to 0.914 (Table 3), indicating excellent internal consistency and stability of the instrument.

Discussion
In this study, we described the development of a new teaching quality evaluation questionnaire (TQEQ) of preclinical courses from the perceptive of medical students with the purpose of improving the effectiveness of preclinical courses in medical schools.
Reliability, validity e ciency and feasibility assessment of the evaluation program is necessary in order to provide an accurate, fair and reliable assessment of teaching quality [16]. We send the original questionnaire whose 47 items are in random to the participants, in order to reduce the impact of effects of subjective consciousness. The EFA of the TQEQ demonstrated seven factors consisting of 39 items: learning outcomes, teaching attitudes, student subjection, teaching abilities, teaching methods, teacher characteristics and teaching interactions, which included the four main aspects of teaching quality program: structure, process, teacher characteristics, and outcomes [17]. Besides, the value of Cronbach's alpha was exceeded 0.70, indicating an acceptable and good internal consistency reliability of the instrument [18]. Thus, this new instrument may well be used to assess teaching quality for medical schools.
A good questionnaire not only contributes to measure the effectiveness of teaching, but also helps to provide a well informal feedback for the teachers and faculties. In this study, interestingly, we found that the common factor "learning outcomes' weighted most heavily among the seven factors. Moreover, ve of ten highest value items, "Teachers use heuristic teaching method to inspire students to think" (q3, 4.35 ± 0.82), which 86.8% agreed and only 2.9% disagreed; "Teachers are good at cultivating students' ability of subject thinking rather than indoctrinating knowledge blindly " (q1, 4.49 ± 0.72), which 90.3% agreed and only 1.2% disagreed; "This course will train students to solve the problems" (q20, 4.29 ± 0.82), which 84.2% agreed and only 2.7% disagreed;"Students will have a good command of key points after the course" (q6, 4.4 ± 0.78), which 86.8% agreed and only 2.0% disagreed;"Teachers have excellent theoretical knowledge of their specialty, and appropriate application in teaching " (q21, 4.44 ± 0.71), which 89.7% agreed and only 1.4% disagreed, are in this common factor. In the other words, whether the course will improve students' abilities, such as heuristic thinking, solving problems, is important for medical students. This is consistent with the previous studies that three types of outcomes need to take in consideration when determining the e cacy of medical education programs: education, clinical career, and environmental outcomes [19]. So, it is critical for teachers to focus on how to cultivate and elevate students' comprehensive abilities rather than teaching literally.
In addition, the highest value items "Teachers are familiar with the teaching content and able to sort out the key points" (q42, 4.66 ± 0.63), 93.5% agreed and only 1.0% disagreed; "Teachers are well prepared for the course " (q4, 4.60 ± 0.65), 93.8% agreed and only 1.4% disagreed; "Teachers are good at organizing the process of class " (q32, 4.48 ± 0.69), 90.0% agreed and 10.0% disagreed, are in the common factor, "teaching abilities", revealing that teacher's performance is the most important for medical students. Thus, in the LBL model, it is essential for teachers to make good preparation and have a good charge of the course.
Moreover, the other two high value items "Teachers encourage students to discuss and ask questions " (q41, 4.57 ± 0.69), 91.7% agreed and only 1.0% disagreed; "Teachers are patient with students' questions " (q40, 4.32 ± 0.84), 83.4% agreed and only 2.7% disagreed, are in the common factor, "teaching interactions", indicating that students rely on teachers and hope teachers could give positive feedback to them.
Among the seven common factors, the common factor "teacher characteristics", containing the four items "Teachers are young and nearly have no generation gap with students" (q43, 3.2 ± 1.07), 38.4% agreed and 22.0% disagreed; "Teachers are humorous" (q45, 3.39 ± 1.16), 47.9% agreed and 21.0% disagreed; "Teachers are good-looking and attractive" (q16, 3.35 ± 1.19), 46.4% agreed and 22.5% disagreed; "The evaluation of teacher related to the course di culty" (q15, 3.64 ± 1.20), 60% agreed and 18.1% disagreed, seems not so essential to the medical student. This was further validated that teachers should exert their strong efforts to prepare their courses and improve their abilities, while they still need to pay attention to their words and actions.
However, this study still has some limitations. First, the course evaluation was administered at the middle of a curriculum; however, some studies suggested that continuous assessment during the courses brings up higher reliability ratings rather than the retrospective evaluation [1,20]. Second, the questionnaire is collected from the view of students, but it should be also accessible to teachers. The teaching planning, teaching methods, and teaching skills may more suitable for teachers themselves to evaluate. Thus, both students and teachers are critical in the process of the assessment of preclinical courses. In addition, the data were collected in a single medical school; therefore, application of the questionnaire in other medical schools is needed to validate the effectiveness of the evaluation system.
In conclusion, TQEQ was demonstrated to be a valid and credible instrument for assessing teaching quality of medical school. Future studies needs to explore whether feedback acquired from this questionnaire contributes to facilitate teaching quality of preclinical courses and student learning outcome.

Conclusion
This study provides a new, validated and useful instrument for measuring teaching effectiveness of preclinical courses based on the views of medical students. The Exploratory factor analysis of the new teaching quality evaluation scale demonstrated seven factors consisting of 39 items: learning outcomes, teaching attitudes, student subjection, teaching abilities, teaching methods, teacher characteristics and teaching interactions. This new scale is based on the views of medical students. It is feasible for use in medical schools to evaluate the preclinical courses. Study participants han signed informed consent and all data were collected anonymously.

Consent for publication
Not applicable.

Availability of data and materials
The datasets used and/or analysed during the current study available from the corresponding author on reasonable request.

Competing interests
The authors declare that they have no competing interests.

Funding
There was no funding for this study.
Authors' contributions YLQ, NNF designed the study. PH, XFY collected the data, did the statistical analysis and wrote the rst draft of the manuscript. HYH contributed to the data collection and statistics. YLQ, NNF contributed to the review of the manuscript. All authors read and approved the nal manuscript.  The class will train students to solve the problems q20 4 Students wants to improve their competitiveness in the talent market q39 5 Teacher choose the appropriate textbook for the class q34 6 The class can reflect the progress and dynamics of disciplines in domestic and abroad q8 7 The evaluation of the teacher related to the class difficulty q15 Teaching abilities 8 The teacher will keep the class lively and interactive, and the participation of students is in high level q17 9 The teacher are familiar with teaching content and able to sort out the key point q42 10 Teachers have excellent theoretical knowledge of their specialty, and appropriate application in teaching q21 11 Teachers use heuristic teaching method to inspire students to think q3 12 The teacher is in high spirits during teaching q24 13 The words of the teacher are exact q37 14 Teachers are good at cultivating the students' ability of subject thinking rather than indoctrinating knowledge blindly The students are patient with difficulties and able to find solutions 46 The students have the ability to establish links and integrate with relevant courses quickly q11 47 The students are interested in the class q26  Figure 1 Scree plot for TQEQ