Perception of The Use of The DOPS In Occupational Therapy PGY Training In Taiwan: The Perspectives of Trainees And Supervisors

doi:10.21203/rs.3.rs-1245634/v1

Download PDF

Research Article

Perception of The Use of The DOPS In Occupational Therapy PGY Training In Taiwan: The Perspectives of Trainees And Supervisors

https://doi.org/10.21203/rs.3.rs-1245634/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Background: Professional competence of young occupational therapists is enhanced through on-site resources, hands-on training and outcome-based education. The Direct Observation of Procedural Skills (DOPS) is a work-based assessment to evaluate professional knowledge, skills and attitude in occupational therapy (OT) clinical training. The perspectives of OT fieldwork teachers and trainees regarding the DOPS are important, but rarely investigated. This study aimed to investigate the perspectives of OT educators and trainees in using DOPS and their discrepancy for OT post-graduate year (PGY) training.

Methods: An online survey was distributed to OT educators and trainees who participated in OT PGY training. The survey comprised five sections: demographic information, the practicality in using the DOPS in clinical settings, the ease of rating the DOPS, and advantages and disadvantages of the DOPS. The responses were based on a 5-point Likert scale. A score of 4 or 5 indicated that the responder agreed/simple or strongly agreed/very simple with the statement. A total of 86 and 41 surveys from OT teachers and trainees were returned respectively, and 64 and 30 from OT teachers and trainees who used DOPS were analyzed.

Results: Over 70 percent of OT educators and trainees agreed that clinical resources (i.e., time, personnel support and cooperation, feedback and training) were sufficient when using the DOPS. Most OT educators and trainees agreed that DOPS matched with OT training goals, benefited the OT competence training and had a fair, objective and consistent scoring system. A significant higher percentage of OT trainees felt stressful in DOPS assessment than that of trainers. Significant higher percentages of OT trainees strongly agreed that DOPS was a practical and appropriate assessment for OT competence training, and had a fair, objective and consistent scoring system than those of OT teachers. There were no significant differences between teachers and trainees regarding easiness of rating DOPS items.

Conclusion: Most OT educators and trainees agreed that DOPS was a practical and appropriate assessment for the OT PGY training.

Educational Philosophy and Theory

Internal Medicine

Occupational therapy

clinical performance

DOPS

Clinical education is the most important part of health professional education [1] and also one of the strategies for developing continuing competence in occupational therapy (OT) [2]. The postgraduate year (PGY) training program for general medicine was implemented by the Taiwanese government in 2003 [3] and for other health care professions in 2007 including OT[4], pharmacy [5] and nursing [6] to address a need for improved professional training and the quality of health care services. The programs are implemented by the Joint Commission of Taiwan under the supervision and instruction of the relevant professional associations to equip PGY trainees with (1) professional knowledge, the ability to provide quality care, evidence-based skills, and resource management ability; (2) patient-centered, comprehensive and holistic perspectives and skills; (3) professional ethical reasoning and communication skills; and (4) the ability to work in a team [7–8].

Assessment of clinical competence is a significant part of the training for young occupational therapists. Objective and systematic assessment allows both supervisors and trainees to be aware of the training objectives and monitor the progress. The assessment of practical performance in clinical conditions (does) is ranked as the highest level in Miller's hierarchical model for the assessment of clinical competence [9]. Direct Observation of Procedural Skills (DOPS) is a workplace-based assessment tool specifically designed to assess practical skills; it requires the assessor to directly observe a trainee conducting a procedure, document evaluation in a checklist and give feedbacks to the trainee based on objective findings [10]. Several studies have reported supervisor’s and trainee’s experiences with the DOPS in health professional training such as medical, nursing and veterinary students and showed that both examiners and examinees were very satisfied with the DOPS[10–11]. However, the feasibility and user experience of the DOPS for OT PGY training has not been examined. We investigated the perception of OT supervisors and trainees on the DOPS in Taiwan for PGY training. Therefore, the aims of our study were to investigate (1) the experience of using the DOPS in clinical settings; (2) the ease of rating clinical performance with DOPS and (3) advantages and disadvantages of the DOPS for OT PGY training.

Procedures

Data were collected from January 2017 to December 2018. Based on the database of the Ministry of Health and Welfare, Taiwan, OT chiefs in charge of OT PGY training programs in 2017 were invited to participate in this study. The online questionnaire was sent by e-mail along with regular postal mail to participants. Participant’s signed informed consent to involve in this study, and their data coding and analysis were anonymized.

Instrument

This questionnaire was developed by two senior occupational therapists (Y. C. Liu and P. C. Huang), based on the research questions and literature of medical education with work-based assessment in 2015[11–14]. Five experts in OT PGY training were invited to assess and discussed the questionnaire in a focus group and revised this questionnaire accordingly. The second group of another five external experts in OT fieldwork education and research were invited to revise and finalized the questionnaire.

The final questionnaire had four sections. The first section included basic sociodemographic information of the participants, including job position, age, gender, highest educational level, teaching/working years, specialty area, status of hospital, and service location. The second section included 21 items related to experiences with using DOPS (i.e., practicality, subjective opinion, cover domain and scoring system) and rated on a 5-point Likert scale. The third section assess trainer’s and trainee’s perception of ease of using the DOPS in rating the clinical performance of PGY OT trainees with a 5-point Likert scale. The forth part included the advantages and disadvantages of the DOPS for the evaluation of OT PGY trainees.

Statistical Analyses

Demographic characteristics and distribution of participants’ responses in perception of DOPS were presented with descriptive statistics (i.e., frequency, percentage, mean and standard deviation). Comparisons between supervisors and trainees in terms of experience of using DOPS, ease of rating the DOPS, and advantages and disadvantages of DOPS were assessed with The χ² and t tests. SPSS 18.0 was utilized to analyze data (SPSS, Chicago, IL, USA).

Completed questionnaires were returned by 86 supervisors and 41 trainees of the 95 hospitals (individuals) who received the invitational letter and survey by e-mails and regular mail for a 90.5% return rate. Data from 64 supervisors and 30 trainees who had used the DOPS and completed surveys were included in the analysis. The demographic characteristics of the participants are presented in Table 1. Significantly more female, being younger and lower levels of education were noted in the trainee group than in the supervisor group. There were no differences in terms of specialty areas and classification of hospital in the two groups.

Most respondents reported that completing one DOPS spent at least 11 minutes for direct observation (11-20 min: teacher 26.6%, n =17, trainee 29.0%, n = 9; 21-30 min: teacher 34.4%, n = 22, trainee 38.7%, n = 12; 31-40 min: teacher 15.6%, n = 10, trainee 12.9%, n = 4). Most respondents (teacher 96.9%, n = 62, trainee 96.8%, n = 30) had feedbacks after direct observation of DOPS, and about half of the feedbacks took

5 to 10 minutes (teacher 53.1%, n = 34, trainee 48.4%, n = 15). There were no differences in terms of time for direction observation (X² (7) = 2.29, p = .94) and for feedbacks (X² (5) = 8.87, p = .11) between teachers and trainees.

A significantly higher percentage (70%, n = 21) of trainees who received DOPS training was found than that (56.3%, n = 36) of teachers (X²(2) = 16.27, p < .0001). There was no significant difference (X² (3) = 4.12, p = .25) in terms of discussion of DOPS rating criteria among teachers before formal assessment between teachers (75%, n = 48) and trainees (90%, n = 27).

Regarding experience with DOPS, over 70% of trainees (range: 72.4% to 93.3%) and supervisors (range: 71.3% to 85.9%) reported “strongly agree” or “agree” with most items except for teacher’s rating of sufficient teachers (67.2%), assessing clinical reasoning ability (62.6%), and consistency in scoring standard (61.0%) and both teacher’s and trainee’s rating of stressfulness in assessment (17.2% and 41.4% respectively) and need further training (54.7% and 55.1% respectively). In addition, less than 10% of trainees (range: 0 to 6.8%) and supervisors (range: 0 to 9.4%) reported “strongly disagree” or “disagree” with most items except for teacher’s rating of need further training (20.6%) and both trainee’s and teacher’s rating of stressfulness in assessment (10.3% and 23.5% respectively). The use experience of DOPS of the trainees and supervisors and their comparisons are presented in Table 2.

Results of X² tests indicated there were significant differences between trainees and teachers on most items of experiences of using DOPS for OT PGY training except for items of low cost, better than traditional teaching assessment, fairness, objectiveness, and consistency in scoring standard. Significant higher percentages of “strongly agree” were noted in trainee’s perception (range: 20.7% to 41.4%) than in supervisor’s perception (range: 1.6% to 12.5%).

Most respondents reported “simple” (trainee range: 10.7% to 34.5%; teacher range: 23.0% to 35.0%) or “neutral” (trainee range: 17.2% to 53.6%; teacher range: 27.9% to 47.5%) in the easiness of rating most of the DOPS in clinical settings. Results of X² tests indicated there were no significant differences between trainees and teachers on all items of easiness of rating DOPS for OT PGY training program. The easiness of rating DOPS as reported by the trainees and supervisors and their comparisons are presented in Table 3.

The advantages of the DOPS assessment tool were ranked as follows: direct observation of clinical performance (trainee: 96.6%, supervisor: 96.8%), immediacy of feedbacks (trainee: 75.0%, supervisor: 76.2%), flexibility (trainee: 75.9%, supervisor: 76.2%), finishing assessment in a short period of time (trainee: 69.0%, supervisor: 69.8%), no additional funding or resource burden (trainee: 62.1, supervisor: 55.6%), and no space restriction (trainee: 58.6%, supervisor: 55.6%). Having a valid and reliable scoring system may be an issue for DOPS in the assessment of clinical competence of OT PGY training because over 75% of respondents reported subjectivity as one of the disadvantages. The results of X² tests found no differences on perceptions of advantages and disadvantages of DOPS between trainees and supervisors. Table 4 presents the comparison of advantages and disadvantages between trainees and supervisors.

To the best of our knowledge, this is one of the few studies to examine the feasibility of DOPS and explore the use experience, advantages and disadvantages for OT PGY training programs by including both perspectives of trainees and supervisors. The results found that the DOPS tool was a practical measurement tool to assess the competence of OT PGY trainees.

Time for implementing DOPS and providing feedbacks is a significant factor for feasibility and acceptability [11]. Some studies [15–16] found lack of time or being seen as an additional workload in administrating DOPS and giving feedbacks in physician training. This may reflect busy clinical hours of the physicians and time constraints for using DOPS in the clinical training of medical doctors. In contrast to findings of the above studies [11, 15–16], about 90% of the respondents in this study spent at least 11 minutes for rating DOPS and half of the respondents reported to have at least 6-10 minutes for feedbacks. Most of the trainees (86.7% agree or strongly agree) and supervisors (75% agree or strongly agree) agreed time for implementation of DOPS was sufficient for OT PCY training. In addition, over 75% trainees and supervisors agree that immediate feedback was one of the advantages of using DOPS in OT PGY training.

Training of rating DOPS is another significant issue for consideration in the assessment for OT PGY training. Half of the trainees (55.1%) and supervisors (54.7%) claimed that they need further training in using DOPS. Proper implementation is one of the significant factor related to the education impacts of administrating DOPS [17] and relies on sufficient training of using assessment tools such as DOPS.

About 40% of supervisors (35.9% neural, 1.6%disagree, and 1.6% strongly disagree) did not support consistency in scoring standard of rating DOPS in this study. Inter-case variation may affect the reliability [11]. Training is needed not only on the procedure of administrating DOPS, but also on scoring training on different patients or conditions. In addition, a study suggested rating of trainees’ performance by at least three supervisors observing at least two procedures each to achieve good reliability [11].

Consistent with previous studies [10–11] that some trainees (41.4% agree or strongly agree) felt that taking DOPS is a stressful experience. However, trainee’s satisfaction with DOPS is high and even higher than the supervisor’s. It is possible that attending a work-based examination such as DOPS is stressful, but the trainees appreciate this experience as learning and positive though stressful experience.

Most OT educators and trainees agreed that DOPS is a practical and appropriate assessment for the OT PGY training. Sufficient time for rating DOPS and feedback about performance are essential to learning effects. Training in using DOPS is necessary and should include rating criteria with different conditions and patients with same procedures.

OT: occupational therapy, DOPS: Direct Observation of Procedural Skills PGY: post-graduate year

Ethics approval and consent to participate

The study protocol has been reviewed and approved by the Ethical Committees for Human Research of the National Cheng Kung University Hospital in Taiwan. The study procedures, risks, and benefits will be explained to all participants, and participants will sign an informed consent before enrolling in this study. Patients will be participating in this study as volunteers and are free to leave the study without any reason. All experiments were performed in accordance with relevant guidelines and regulations.

Consent for publication

Not applicable

Availability of data and materials

The datasets generated and analyzed during the current study are not publicly available due to participant confidentiality, but are available from the corresponding author on reasonable request.

Competing interests

The authors declare that they have no competing interests.

Funding

This study was supported in part by grants from National Cheng Kung University Hospital (NCKUH-10607022) and the Ministry of Science and Technology (Grant Number MOST 108-2314-B-002-165-MY3) in Taiwan.

Authors' contributions

YCLiu, PCH, and KCL designed this study and the questionnaire, YCLiu, YCLee and IHL collected and analyzed data, and YCLiu and YCLee interpreted the results and drafted the manuscript. All authors read and approved the final manuscript.

Acknowledgements

The authors wish to thank colleagues of the Department of Psychiatry, National Cheng Kung University Medical College, for their valuable comments on this work.

1. American Occupational Therapy Association. Occupational therapy fieldwork education: Value and purpose. American Journal of Occupational Therapy. 2016;70:7012410060.

2. American Occupational Therapy Association. Standards of practice for occupational therapy. American Journal of Occupational Therapy. 2021;75:7513410050.

3. Hsu PW, Hsieh MJ, Fu RH, Huang JL, Liao MC, Lee ST. Comparing the outcomes of different postgraduate year training programs in Taiwan. Biomed J. 2015;38(6):544-9.

4. Liu YC, Huang, P. C., Lee, Y. C., Chu, C. L., Hsu, W. Y., Chen, P. S., & Lin, K. C. The use of the Mini-CEX in occupational therapy postgraduate year training programs in Taiwan. Journal of Medical Education. 2021;25:49-60.

5. Wei CJ, Lu, T. H., Chien, S. C., Huang, W. T., Liu, Y. P., Chan, C. Y., & Chiu, C. H. The development and use of a pharmacistspecific Mini-CEX for postgraduate year trainees in Taiwan. 2019;19:165-72.

6. Yin YC. The two-year post graduate training program for nurses: Implementation status and personal perspectives. Hu Li Za Zhi. 2013;60(3):11–6.

7. Joint Commission of Taiwan. Guidelines for the 2-year Occupational Therapist Training Course. Joint Commission of Taiwan; 2018.

8. Taiwan Occupational Therapy Association. Core competence. Taiwan Occupational Therapy Association; 2019.

9. Miller GE. The assessment of clinical skills/competence/performance. Acad Med. 1990;65(9 Suppl):S63-7.

10. Erfani Khanghahi M, Ebadi Fard Azar F. Direct observation of procedural skills (DOPS) evaluation method: Systematic review of evidence. Med J Islam Repub Iran. 2018;32:45.

11. Naeem N. Validity, reliability, feasibility, acceptability and educational impact of direct observation of procedural skills (DOPS). J Coll Physicians Surg Pak. 2013;23(1):77-82.

12. Bindal N, Goodyear H, Bindal T, Wall D. DOPS assessment: a study to evaluate the experience and opinions of trainees and assessors. Med Teach. 2013;35(6):e1230-4.

13. Hayashi J, Christmas C, Durso SC. Educational outcomes from a novel house call curriculum for internal medicine residents: report of a 3-year experience. J Am Geriatr Soc. 2011;59(7):1340-9.

14. Hoseini BL, Jafarnejad F, Mazloum SR, Foroughipour M. Practical Experience of the Mini-CEX in Undergraduate Trainees. Procedia - Social and Behavioral Sciences. 2013;83(4):803-7.

15. Wilkinson JR, Crossley JG, Wragg A, Mills P, Cowan G, Wade W. Implementing workplace-based assessment across the medical specialties in the United Kingdom. Med Educ. 2008;42(4):364-73.

16. Miller A, Archer J. Impact of workplace based assessment on doctors' education and performance: a systematic review. BMJ. 2010;341:c5064.

17. Lorwald AC, Lahner FM, Nouns ZM, Berendonk C, Norcini J, Greif R, et al. The educational impact of Mini-Clinical Evaluation Exercise (Mini-CEX) and Direct Observation of Procedural Skills (DOPS) and its association with implementation: A systematic review and meta-analysis. PLoS One. 2018;13(6):e0198009.

Table1. Demographic characteristics of survey participants using DOPS for OT PGY training

		Training teacher (n = 64)		Trainee (n = 30)
		Frequency	Percentage	Frequency	Percentage	X²(df)	p
Gender	Male	28	43.8%	7	23.3%	3.64 (1)	.06
	Female	36	56.3%	23	76.6%
Age	20 -25 years	0	0	23	76.7%	71.84 (4)	<.001
	26 -30 years	11	17.2%	6	20.0%
	31 -35 years	14	21.9%	1	3.3%
	36 -45 years	25	39.1%	0	0
	Over 46 years	14	21.9%	0	0
Education	Junior college	0	0	2	6.7%	11.62(2)	.003
	University	36	56.3%	24	80.0%
	Master and above	28	43.8%	4	13.3%
Teaching years	Under 5years	12	18.8%
	6 -10 years	19	29.7%
	11 -15 years	14	21.9%
	16 -20 years	11	17.2%
	Over 20 years	8	12.5%
Specialty area	Physical	21	32.8%	7	23.3%	2.74 (3)	.43
	Pediatric	3	4.7%	0	0
	Physical and Pediatric	21	32.8%	13	43.3%
	Mental health	19	29.7%	10	33.3%
Hospital	Medical center	20	31.3%	11	36.7%	.90 (3)	.82
	Regional teaching hospital	29	45.3%	14	46.7%
	District teaching hospital	10	15.6%	4	13.3%
	Psychiatric teaching hospital	5	7.8%	1	3.3%

Table 2. Comparison between teachers and trainees in the practicality in using the DOPS in clinical settings

Questionnaire Items		Strongly agree	Agree	Neutral	Disagree	Strongly disagree	Mean (SD)	X²(df)	t(df)
Used at predictable time	Teacher (n = 64)	6 (9.4%)	45 (70.3%)	13 (20.3%)	0	0	3.89 (.54)	15.23 (3)**	2.19 (43)*
Used at predictable time	Trainee (n = 30)	12 (40.0%)	14 (46.7%)	3 (10.0%)	1 (3.3%)	0	4.23 (.77)
Sufficient observation time	Teacher (n = 64)	5 (7.8%)	43 (67.2%)	14 (21.9%)	2 (3.1%)	0	3.80 (.62)	12.56 (3)**	2.72 (92)**
Sufficient observation time	Trainee (n = 30)	11 (36.7%)	15 (50.0%)	3 (10.0%)	1 (3.3%)	0	4.20 (.76)
Immediacy of feedback	Teacher (n = 64)	8 (12.5%)	44 (68.8%)	12 (18.8%)	0	0	3.94 (.56)	12.68 (3)**	1.88 (41)
Immediacy of feedback	Trainee (n = 29)	12 (41.4%)	13 (44.8%)	3 (10.3%)	1 (3.4%)	0	.24 (.79)
Sufficient feedback time	Teacher (n = 63)	6 (9.5%)	39 (61.9%)	17 (27.0%)	1 (1.6%)	0	3.79 (.63)	11.46 (3)**	2.64 (48)*
Sufficient feedback time	Trainee (n = 29)	10 (34.5%)	16 (55.2%)	2 (6.9%)	1 (3.4%)	0	4.21 (.73)
Good cooperation among supervisors, trainees, peers	Teacher (n = 63)	6 (9.5%)	40 (63.5%)	16 (25.4%)	1 (1.6%)	0	3.81 (.62)	10.07 (3)*	2.43 (90)*
Good cooperation among supervisors, trainees, peers	Trainee (n = 29)	10 (34.5%)	15 (51.7%)	3 (10.3%)	1 (3.4%)	0	4.17 (.76)
Sufficient teachers	Teacher (n = 64)	5 (7.8%)	38 (59.4%)	18 (28.1%)	3 (4.7%)	0	3.70 (.68)	10.73 (3)*	1.99 (91)
Sufficient teachers	Trainee (n = 29)	10 (34.5%)	11 (37.9%)	7 (24.1%)	1 (3.4%)	0	4.03 (.87)
Stressfulness in assessment	Teacher (n = 64)	1 (1.6%)	10 (15.6%)	38 (59.4%)	12 (18.8%)	3 (4.7%)	2.91 (.77)	12.67 (4)*	3.04 (45)**
Stressfulness in assessment	Trainee (n = 29)	6 (20.7%)	6 (20.7%)	14 (48.3%)	3 (10.3%)	0	3.52 (.95)
Assess the trainee's learning status	Teacher (n = 63)	4 (6.3%)	41 (65.1%)	16 (25.4%)	2 (3.2%)	0	3.75 (.62)	8.03 (3)*	1.91 (90)
Assess the trainee's learning status	Trainee (n = 29)	8 (27.6%)	15 (51.7%)	5 (17.2%)	1 (3.4%)	0	4.03 (.78)
Low cost	Teacher (n = 64)	9 (14.1%)	40 (62.5%)	14 (21.9%)	0	1 (1.6%)	3.88 (.70)	6.81 (4)	.51 (91)
Low cost	Trainee (n = 29)	9 (31.0%)	13 (44.8%)	5 (17.2%)	1 (3.4%)	1 (3.4%)	3.97 (.98)
Overall implementability	Teacher (n = 64)	7 (10.9%)	47 (73.4%)	10 (15.6%)	0	0	3.95 (.52)	8.27 (3)*	1.21 (41)
Overall implementability	Trainee (n = 29)	9 (31.0%)	16 (55.2%)	3 (10.3%)	1 (3.4%)	0	4.14 (.74)
Familiar with the DOPS	Teacher (n = 64)	6 (9.4%)	39 (60.9%)	17 (26.6%)	1 (1.6%)	1 (1.6%)	3.75 (.71)	11.83 (4)*	3.46 (92)**
Familiar with the DOPS	Trainee (n = 30)	10 (33.3%)	18 (60.0%)	2 (6.7%)	0	0	4.27 (.58)
Need further training	Teacher (n = 64)	4 (6.3%)	31 (48.4%)	23 (35.9%)	5 (7.8%)	1 (1.6%)	3.50 (.80)	11.39 (4)*	.07 (38)
Need further training	Trainee (n = 29)	7 (24.1%)	9 (31.0%)	7 (24.1%)	3 (10.3%)	3 (10.3%)	3.48 (1.27)
Better than traditional teaching assessment	Teacher (n = 63)	8 (12.7%)	39 (61.9%)	14 (22.2%)	2 (3.2%)	0	3.84 (.68)	7.22 (3)	2.75 (90)**
Better than traditional teaching assessment	Trainee (n = 29)	9 (31.0%)	18 (62.1%)	2 (6.9%)	0	0	4.24 (.58)
Use as a teaching assessment tool	Teacher (n = 64)	8 (12.5%)	47 (73.4%)	9 (14.1%)	0	0	3.98 (.52)	6.49 (2)*	2.29 (49)*
Use as a teaching assessment tool	Trainee (n = 29)	10 (34.5%)	17 (58.6%)	2 (6.9%)	0	0	4.28 (.59)
Matching with OT’s training goals	Teacher (n = 64)	4 (6.3%)	42 (65.6%)	17 (26.6%)	0	1 (1.6%)	3.75 (.64)	11.01 (3)*	2.92 (91)**
Matching with OT’s training goals	Trainee (n = 29)	9 (31.0%)	16 (55.2%)	4 (13.8%)	0	0	4.17 (.66)
Assessing clinical /communication skills	Teacher (n = 64)	6 (9.4%)	44 (68.8%)	13 (20.3%)	1 (1.6%)	0	3.86 (.59)	8.89 (3)*	2.94 (90)**
Assessing clinical /communication skills	Trainee (n = 28)	9 (32.1%)	17 (60.7%)	2 (7.1%)	0	0	4.25 (.59)
Assessing clinical reasoning ability	Teacher (n = 64)	4 (6.3%)	36 (56.3%)	19 (29.7%)	5 (7.8%)	0	3.61 (.73)	11.37 (3)*	3.37 (91)**
Assessing clinical reasoning ability	Trainee (n = 29)	8 (27.6%)	17 (58.6%)	4 (13.8%)	0	0	4.14 (.64)
Assessing standard procedural ability	Teacher (n = 64)	7 (10.9%)	44 (68.8%)	11 (17.2%)	2 (3.1%)	0	3.88 (.63)	10.28 (3)*	2.24 (91)*
Assessing standard procedural ability	Trainee (n = 29)	11 (37.9%)	13 (44.8%)	5 (17.2%)	0	0	4.21 (.73)
Fairness	Teacher (n = 64)	5 (7.8%)	42 (65.6%)	16 (25.0%)	1 (1.6%)	0	3.80 (.60)	5.26 (3)	1.98 (91)
Fairness	Trainee (n = 29)	7 (24.1%)	17 (58.6%)	5 (17.2%)	0	0	4.07 (.65)
Objectiveness	Teacher (n = 64)	5 (7.8%)	41 (64.1%)	17 (26.6%)	1 (1.6%)	0	3.78 (.60)	3.06 (3)	1.62 (91)
Objectiveness	Trainee (n = 29)	5 (17.2%)	19 (65.5%)	5 (17.2%)	0	0	4.00 (.60)
Consistency in scoring standard	Teacher (n = 64)	3 (4.7%)	36 (56.3%)	23 (35.9%)	1 (1.6%)	1 (1.6%)	3.61 (.68)	7.05 (4)	2.34 (54)*
Consistency in scoring standard	Trainee (n = 29)	6 (20.7%)	16 (55.2%)	7 (24.1%)	0	0	3.97 (.68)

* p < .05, ** p < .01

Table 3. Comparison between teachers and trainees in the easiness of rating the DOPS in clinical settings

Questionnaire Items		Very simple	Simple	Neutral	Difficult	Very difficult	Mean (SD)	X²(df)	t(df)
Demonstrates understanding of indications, relevant anatomy, technique of procedure	Teacher (n = 59)	11 (18.6%)	15 (25.4%)	20 (33.9%)	10 (16.9%)	3 (5.1%)	2.64 (1.13)	2.05 (4)	1.01 (86)
	Trainee (n = 29)	3 (10.3%)	7 (24.1%)	10 (34.5%)	8 (27.6%)	1 (3.4%)	2.90 (1.05)
Obtains informed consent	Teacher (n = 61)	27 (44.3%)	15 (24.6%)	17 (27.9%)	1 (1.6%)	1 (1.6%)	1.92 (.97)	6.15 (4)	1.19 (88)
Obtains informed consent	Trainee (n = 29)	11 (37.9%)	8 (27.6%)	5 (17.2%)	3 (10.3%)	2 (6.9%)	2.21 (1.26)
Demonstrates appropriate preparation pre-procedure	Teacher (n = 60)	11 (18.3%)	21 (35.0%)	21 (35.0%)	5 (8.3%)	2 (3.3%)	2.43 (1.00)	1.56 (4)	.07 (87)
Demonstrates appropriate preparation pre-procedure	Trainee (n = 29)	5 (17.2%)	10 (34.5%)	10 (34.5%)	4 (13.8%)	0	2.45 (.95)
Appropriate analgesia or safe sedation	Teacher (n = 57)	8 (14.0%)	14 (24.6%)	25 (43.9%)	6 (10.5%)	4 (7.0%)	2.72 (107)	1.53 (4)	.13 (83)
Appropriate analgesia or safe sedation	Trainee (n = 28)	3 (10.7%)	8 (28.6%)	11 (39.3%)	5 (17.9%)	1 (3.6%)	2.75 (1.01)
Technical ability	Teacher (n = 61)	2 (3.3%)	17 (27.9%)	24 (39.3%)	16 (26.2%)	2 (3.3%)	2.98 (.90)	2.60 (4)	.08 (87)
Technical ability	Trainee (n = 28)	1 (3.6%)	5 (17.9%)	15 (53.6%)	7 (25.0%)	0	3.00 (.77)
Aseptic technique	Teacher (n = 59)	14 (23.7%)	16 (27.1%)	21 (35.6%)	5 (8.5%)	3 (5.1%)	2.44 (1.10)	4.39 (4)	-.61 (85)
Aseptic technique	Trainee (n = 28)	10 (35.7%)	4 (14.3%)	10 (35.7%)	4 (14.3%)	0	2.29 (1.12)
Seeks help where appropriate	Teacher (n = 60)	3 (5.0%)	14 (23.3%)	27 (45.0%)	14 (23.3%)	2 (3.3%)	2.97 (.90)	7.22 (4)	-1.19 (44)
Seeks help where appropriate	Trainee (n = 28)	6 (21.4%)	5 (17.9%)	9 (32.1%)	8 (28.6%)	0	2.68 (1.12)
Post procedure management plan	Teacher (n = 61)	4 (6.6%)	14 (23.0%)	25 (41.0%)	15 (24.6%)	3 (4.9%)	2.98 (.98)	2.68 (4)	.92 (87)
Post procedure management plan	Trainee (n = 28)	1 (3.6%)	3 (10.7%)	15 (53.6%)	8 (28.6%)	1 (3.6%)	3.18 (.82)
Communications skills	Teacher (n = 59)	8 (13.6%)	14 (23.7%)	22 (37.3%)	13 (22.0%)	2 (3.4%)	2.78 (1.05)	1.73 (4)	1.07 (85)
Communications skills	Trainee (n = 28)	3 (10.7%)	4 (14.3%)	11 (39.3%)	9 (32.1%)	1 (3.6%)	3.04 (1.04)
Consideration for patient/professionalism	Teacher (n = 60)	5 (8.3%)	15 (25.0%)	23 (38.3%)	12 (20.0%)	5 (8.3%)	2.95 (1.06)	1.42 (4)	-.67 (86)
Consideration for patient/professionalism	Trainee (n = 28)	4 (14.3%)	6 (21.4%)	11 (39.3%)	6 (21.4%)	1 (3.6%)	2.79 (1.07)
Overall clinical competence performing procedure	Teacher (n = 59)	3 (5.1%)	16 (27.1%)	28 (47.5%)	12 (20.3%)	0	2.83 (.81)	6.91 (4)	1.57 (85)
Overall clinical competence performing procedure	Trainee (n = 28)	2 (7.1%)	3 (10.7%)	14 (50.0%)	7 (25.0%)	2 (7.1%)	3.14 (.97)

Table 4. Advantages and disadvantages of DOPS reported by trainees and supervisors

Advantages		Yes	No	X²(df)
Direct observation of clinical performance	Trainees (n = 29)	28 (96.6%)	1 (3.4%)	.95 (1)
Direct observation of clinical performance	Supervisors (n = 63)	61 (96.8%)	2 (3.2%)
No space restriction	Trainees (n = 29)	17 (58.6%)	12 (41.4%)	.78 (1)
No space restriction	Supervisors (n = 63)	35 (55.6%)	28 (44.4%)
Finishing assessment in a short period of time	Trainees (n = 29)	20 (69.0%)	9 (31.0%)	.93 (1)
Finishing assessment in a short period of time	Supervisors (n = 63)	44 (69.8%)	19 (30.2%)
Flexibility	Trainees (n = 29)	22 (75.9%)	7 (24.1%)	.97 (1)
Flexibility	Supervisors (n = 63)	48 (76.2%)	15 (23.8%)
Consistent scoring criteria among supervisors	Trainees (n = 29)	14 (48.3%)	15 (51.7%)	.13 (1)
Consistent scoring criteria among supervisors	Supervisors (n = 63)	41 (65.1%)	22 (34.9%)
Consistent scoring criteria among OT and other health care professionals	Trainees (n = 29)	10 (34.5%)	19 (65.5%)	.91 (1)
	Supervisors (n = 63)	21 (33.3%)	42 (66.7%)
Immediacy of feedbacks	Trainees (n = 29)	22 (75.9%)	7 (24.1%)	.97 (1)
Immediacy of feedbacks	Supervisors (n = 63)	48 (76.2%)	15 (23.8%)
Reliable and valid rating of clinical performance	Trainees (n = 29)	14 (48.3%)	15 (51.7%)	.73 (1)
Reliable and valid rating of clinical performance	Supervisors (n = 63)	28 (44.4%)	35 (55.6%)
No additional funding or resource burden	Trainees (n = 29)	18 (62.1%)	11 (37.9%)	.56 (1)
No additional funding or resource burden	Supervisors (n = 63)	35 (55.6%)	28 (44.4%)
Disadvantages		Yes	No
Assessing only parts of clinical skills	Trainees (n = 29)	5 (17.2%)	24 (82.8%)	.08 (1)
Assessing only parts of clinical skills	Supervisors (n = 63)	22 (34.9%)	41 (65.1%)
Subjectivity	Trainees (n = 29)	22 (75.9%)	7 (24.1%)	.11 (1)
Subjectivity	Supervisors (n = 63)	56 (88.9%)	7 (11.1%)
Low patient acceptance	Trainees (n = 29)	4 (13.8%)	25 (86.2%)	.95 (1)
Low patient acceptance	Supervisors (n = 63)	9 (14.3%)	54 (85.7%)
Not adaptable to greater variability of clinical situations	Trainees (n = 29)	16 (55.2%)	13 (44.8%)	.70 (1)
Not adaptable to greater variability of clinical situations	Supervisors (n = 63)	32 (50.8%)	31 (49.2%)
Time restriction in clinical setting	Trainees (n = 29)	3 (10.3%)	26 (89.7%)	.70 (1)
Time restriction in clinical setting	Supervisors (n = 63)	5 (7.9%)	58 (92.1%)

No competing interests reported.

Download PDF

Version 1

posted

You are reading this latest preprint version

Perception of The Use of The DOPS In Occupational Therapy PGY Training In Taiwan: The Perspectives of Trainees And Supervisors

Status:

Version 1

Abstract

Literature Review

Methods

Procedures

Instrument

Statistical Analyses

Results

Discussion

Conclusion

List Of Abbreviations

Declarations

References

Tables

Additional Declarations

Status:

Version 1