Performance of Graduation Examination of Standardized Residency Training in Zhejiang Province, China


 Background

Standardized residency training is an important part of post-graduation medical education for medical students in China, and the graduation examination is a necessary part to test the effect of residency training and an important measure to guarantee the quality of residency training. This study analyzed the graduation examination results of standardized residency training in Zhejiang Province in 2020.
Methods

This study calculated the pass rates for the practical skills examination and written examination for 5555 examinees and analyzed the relationship between examinees' demographic characteristics and pass rates.
Results

The pass rates for practical skills examination and written examination were 91.8% and 96.5%, respectively. Gender, age, education, and type of organizations were significantly associated with the pass rates. Examinees who retook the examination were less likely to pass the practical skills examination (AOR = 0.266, 95% CI = 0.189-0.377) and the written examination (AOR = 0.043, 95% CI = 0.025-0.074) than examinees who took the examination for the first time. If an examinee did not pass the practical skills examination initially, he or she was less likely to pass the written examination (AOR = 0.172, 95% CI = 0.120-0.249)
Conclusion

The pass rate of graduation examination of standardized residency training was not only influenced by the individual characteristics of the examinees but also related to the level of training hospitals and lead teachers. Therefore, it is necessary to take comprehensive measures to further improve the system of standardized residency training.


Abstract Background
Standardized residency training is an important part of post-graduation medical education for medical students in China, and the graduation examination is a necessary part to test the effect of residency training and an important measure to guarantee the quality of residency training. This study analyzed the graduation examination results of standardized residency training in Zhejiang Province in 2020.

Methods
This study calculated the pass rates for the practical skills examination and written examination for 5555 examinees and analyzed the relationship between examinees' demographic characteristics and pass rates.

Results
The pass rates for practical skills examination and written examination were 91.8% and 96.5%, respectively. Gender, age, education, and type of organizations were signi cantly associated with the pass rates. Examinees who retook the examination were less likely to pass the practical skills examination (AOR = 0.266, 95% CI = 0.189-0.377) and the written examination (AOR = 0.043, 95% CI = 0.025-0.074) than examinees who took the examination for the rst time. If an examinee did not pass the practical skills examination initially, he or she was less likely to pass the written examination (AOR = 0.172, 95% CI = 0.120-0.249)

Conclusion
The pass rate of graduation examination of standardized residency training was not only in uenced by the individual characteristics of the examinees but also related to the level of training hospitals and lead teachers. Therefore, it is necessary to take comprehensive measures to further improve the system of standardized residency training.

Background
Many countries have established residency training systems to standardize the quality of doctors [1] .Internationally, residency training is considered the only way for doctors to be quali ed to practice independently [2,3] . China's standardized residency training began in 1921, when the Peking Union Medical College was founded and introduced the "John Hopkins model " [4,5] . In 2013, seven government ministries jointly issued the "Guidance on the Establishment of a Standardized Residency Training System", marking the o cial launch of residency training in China [6,7] . The guideline proposed that by 2020, a standardized residency training system will be basically established, and all new clinicians entering medical positions with bachelor's degrees and above will receive standardized residency training.
In 2018, 88,000 western medicine residents and 21,000 Chinese medicine residents were enrolled in standardized training nationwide [8] . The China Health Statistical Yearbook 2020 shows that the total number of physicians reached 3.867 million by the end of 2019, of which 32.6 percent and 5.9 percent had undergraduate and postgraduate degrees, respectively. Standardized residency training is mandatory and takes up to 3 years depending on the educational degree of the participants [9] . For those who have obtained postgraduate medical degrees and physicians who are already engaged in clinical medical work, the training hospitals will determine the speci c time and content in accordance with the training standards, taking into account their clinical experience and practical ability. If they do not complete the training tasks within the required time or fail the examination, the training period can be extended for a maximum of 3years. The assessment of standardized residency training is divided into two parts: process assessment and graduation examination, of which the process assessment is a dynamic evaluation involving medical ethics, attendance, completion of training targets, and learning participation, while the graduation examination is a measure of the overall effect of training, including practical skills examination and written examination. The written examination is organized by the state uniformly, using computerized question-answering methods, focusing on the assessment of examinees' level of public basic knowledge and professional theoretical knowledge, and the practical skills examination is organized by each province. obtained a certi cate of eligibility [10] . The clinical practical skills examination in Zhejiang province is conducted in objective structured clinical examinations (OSCE) pattern at 6 stations, including medical data interpreting, patient interviewing(medical history taking, physical examination), medical documents writing( rst course of disease record, complete medical record), clinical reasoning and decision making, basic life supportive skills(CPR, endotracheal intubation) and specialized procedures performing. Only after passing both the written examination and the practical skills examination can examinees get the certi cate of completion. If examinees fail to pass one of the two examinations, the result of the passed project will be kept for 3 years and examinees have to pass the failed project by a make-up exam.
Meanwhile, if one of the clinical practice skill examinations fails, the whole clinical practice skill examination will be deemed as failed. The purpose of this study is to analyze the pass rate of standardized residency training and its in uencing factors in Zhejiang Province and to provide a reference basis for further improving the training models and assessment mechanisms.

Data source and study population
This study used data from the database of graduation examination of standardized residency training in Zhejiang Province in 2020, provided by the Zhejiang Medical Service Management Evaluation Center. The graduation examination included two parts: the practical skills examination held from August 4 to 7, 2020 and the written examination held on August 16, 2020. This database contained information about 5555 examines who taken the graduation examination, including name, ID number, gender, date of birth, ethnic group, educational level, whether to take the examination for the rst time, examinees' organization, types of examination taken, examination results, etc. All 5555 examines were included in this study.

Outcomes
According to the cut-off passing score, the original examination scores of practical skills examination and written examination were recoded as dichotomous variables of 1=pass and 0=fail. The cut-off passing score of the written examination was 400. For the practical skills examination, the cut-off passing score of medical history taking, physical examination, rst course of disease record, complete medical record, clinical reasoning and decision making, CPR, endotracheal intubation, and specialized procedures performing was 80, while the cut-off passing score of medical data interpreting was 60. If an examinee failed one of the 9 projects, the entire practical skills examination was considered a failure.

Statistical analysis
Descriptive statistics were presented as frequency and percentage for the categorical variables. The Pearson Chi-square tests and Fisher's exact tests were used to compare pass rates by examinees' characteristics. The multivariable logistic regression analysis was performed to identify the association between sociodemographic variables and pass rates of practical skills examination and written examination. All variables of bivariate analysis were included in the regression models. Since the practical skills examination was held before the written examination, we included the results of the practical skills examination as an independent variable in the regression model of the written examination. Adjusted odds ratios (AOR) with their corresponding 95% con dence intervals (CI) from regression analysis were presented. The two-tailed p<0.05 was considered to be statistically signi cant. All analyses were conducted using R 3.6.1 for windows.  (Table 1).

Pass rates
Twenty-nine examinees missed the practical skills examination and 24 missed the written examination. In 2020, 5030(91.8%) of the 5477 examinees passed the practical skills examination, and 5156(96.5%) of 5345 examinees passed the written examination, yielding an overall pass rate of 89.5%.
Pass rates by examinee characteristics Table 2 presents the results of bivariate analysis of examinees' characteristics relative to the pass rates of practical skills examination and written examination. There were signi cant differences in the pass rates of the two examinations in terms of gender, age, educational level, whether to take the examination for the rst time, and examinee' organization type. Females reported signi cantly higher pass rates than males in the practical skills examination (93.4% vs 90.1%, p<0.001) and written examination (97.3% vs 95.5%, p<0.001). Those who took the examination for the rst time had signi cantly higher pass rates than those who retook the examination in the practical skills examination (92.6% vs 70.2%, p<0.001) and written examination (97.2% vs 49.4%, p<0.001). Compared with other age groups, the youngest group(age<25) had the highest pass rates in the practical skills examination (92.8%, p=0.002) and written examination (99.3%, p<0.001). In terms of educational level, examinees with a postgraduate degree had the highest pass rates in the practical skills examination (92.3%, p=0.006) and written examination (98.9%, p<0.001). Compared with examinees from other institutions, examinees from primary health care institutions and second-level hospitals had the lowest pass rates in the practical skills examination (87.5%, p<0.001) and written examination (93.9%, p<0.001), respectively.
Factors associated with pass rates The multivariable logistic regression analysis results are shown in Table 3. Females were 1.526 (95% CI=1.251-1.865) times more likely than males to the pass practical skills examination, and 1.645 (95% CI=1.188-2.295) times more likely to pass the written examination. Compared with those who took the examinations for the rst time, examinees who retook the examinations were less likely to pass the practical skills examination (AOR = 0.266, 95% CI = 0.189-0.377) and written examination (AOR = 0.043, 95% CI = 0.025-0.074). Examinees aged 30 and above were less likely to pass the practical skills examination than those under the age of 25(AOR = 0.519, 95% CI = 0.276-0.952). Compared with examinees under the age of 25, examinees aged between 25 and 30 (AOR = 0.214, 95% CI = 0.033-0.800), and those aged 30 and above (AOR = 0.077, 95% CI= 0.012-0.298) were less likely to pass the written examination. When we included all the independent variables in the regression analysis, the results of educational level in the practical skills examination contradicted the results of the bivariate analysis. After excluding the variable of institution type, examinees with an undergraduate degree and a graduate degree were 1.561(p=0.013,95% CI=1.087-2.198) times and 1.874(p=0.006,95% CI=1.189-2.938) times more likely to pass the practical skills examination than those with a junior college degree, respectively. The results were close to the results of univariate logistic regression of education level, which suggested that the variable of institution type may have a suppressor effect on the education level. Examinees with a postgraduate degree were 5.829 (95% CI=2.068-17.272) times more likely to pass the written examination than those with a junior college degree. Examinees from primary health care institutions were less likely to pass the practical skills examination than those from tertiary hospitals (AOR = 0.524, 95% CI = 0.383-0.719). Compared with those from tertiary hospitals, examinees from second-level hospitals were less likely to pass the written examination (AOR = 0.585, 95% CI = 0.391-0.872), and examinees from colleges and universities were 2.867 (95% CI = 1.403-6.656) times more likely to pass the written examination. In addition, if an examinee failed the practical skills examination at rst, he or she was less likely to pass the written examination (AOR = 0.172, 95% CI = 0.120-0.249).

Discussion
This study showed that the pass rate of the practical skills examination was lower than that of the written examination, which may be related to several reasons: rst, the length of practical skills training is lower than that of theoretical knowledge learning; second, the pass rate of practical skills examination is indeed in uenced by the degree of attention paid by municipalities, the training level of training hospitals, and the teachers who lead them. Therefore, clinical practice skills should always be a top priority of training [11] . The pass rate of written examination in Zhejiang Province was 96.5%, ranking 2nd in the country, higher than that of Guizhou Province in western China [12] . Since the content of the practical skills examination is not uniform across provinces, it is not appropriate to compare pass rates.
This study found that female examinees had higher pass rates than male examinees in the two examinations. However, some other studies did not nd differences in evaluation results by gender [13][14][15] .
In addition, gender bias may have an impact on evaluation results [16] . Examinees aged 30 and above had the lowest pass rate in the clinical practical skill examination and written examination, probably because older examinees may have certain clinical work experience and have developed certain personal work habits prior to their standardized training, and therefore have di culty adapting and changing to the new training requirements. In addition, with age, the ability of the aging motor cortex to respond to training will decline [17] , and that can be translated into an age-dependent decline in the ability to acquire new motor skills [18] . Examinees with a graduate degree were more likely to pass the written examination than examinees with a junior college degree, a result consistent with the ndings of a previous study in Guangdong Province [19] . Moreover, several other studies have shown that examinees with graduate degrees have higher pass rates on both practical skills examination and written examination than those with undergraduate degrees [12,20] . The higher pass rate for examinees with postgraduate degrees may be related to their more solid theoretical knowledge and more opportunities for clinical training. As the content of the written examination is nationally standardized, it is worth further thinking and exploring how to achieve homogeneous training requirements for examinees with different quali cations. Those who took both examinations for the rst time had signi cantly higher pass rates than those who took the examinations again. If an examinee does not pass the practical skills examination at the outset, his or her likelihood of passing the written exam will decline. This may be because the new policy changes the examination from twice a year to once a year and if examinees fail the examination they will not be able to obtain their degree within a year, which will affect their employment and application for a PhD, so examinees who resit the examination may be more nervous and anxious. Examinees from primary health care institutions were less likely to pass the practical skills examination than those from the tertiary hospitals, which may be partly due to the fact that advanced medical equipment and skilled doctors were mainly concentrated in tertiary hospitals [21][22][23] , and partly because most examinees from primary health care institutions were assistant general practitioners with only two years of training, resulting in insu cient exercise of clinical practice skills. Examinees from colleges and universities had a higher pass rate in written examination compared to those from tertiary hospitals, mainly because most of the examinees from colleges and universities were graduating full-time postgraduate students whose theoretical knowledge had been consolidated and strengthened by three years of study.
The main objective of residency training in China is to standardize and homogenize physicians training [24] . However, the lack of a well-established competency framework for residency training and the great heterogeneity of physicians in terms of age, marriage, level of education, and type of institution in which they work results in a mixed quality of training [24,25] . Therefore, there is a need to standardize residency recruitment criteria and assessment programs nationwide, and to provide differentiated and tiered training based on different competencies and specialties. In addition, the differences in training conditions, faculty strength and management systems due to the different levels, sizes, regions and nature of specialties of training hospitals also have a greater impact on the overall quality of residency training, of which the level and responsibility of the lead teachers are particularly important [26] . Studies have shown that the overall satisfaction of Chinese residents was low, especially in terms of treatment subsidies, training methods, the level of lead teachers, and facilities and equipment [27][28][29] . In order to improve the standardized residency training system, the following measures should be taken: 1)strengthen cooperation and exchange between different regions and levels of training hospitals, promote the orderly mobility of doctors, and narrow the gap in faculty strength between hospitals; 2) make full use of information technology to develop online courses and promote resource sharing; 3) optimize the selection criteria for lead teachers, and strengthen training and assessment of lead teachers; 4) strengthen the management and assessment of training hospitals and establish an exit mechanism; 5) improve the remuneration of residents and increase satisfaction with training.
Our study is subject to several limitations. First, this study only analyzed the pass rates of the written and practical skills examinations and did not compare the performance of different modules of the practical skills examination. Second, some studies have shown that marital status has a signi cant impact on resident performance [24,30] , however, this study did not include marital status as an independent variable. Third, this study only analyzed data for the year 2020 and no time series analysis was conducted, thus the reliability of the ndings is somewhat affected.

Conclusion
The overall pass rate of standardized residency training in Zhejiang Province was relatively high, and some demographic characteristics of examinees, the level of training hospitals and faculty are important