Moving from Basic to More Advanced: Is there an Increase Over Time in the Complexity of Teacher Questions and Student Responses in Case Based Clinical Seminars?

doi:10.21203/rs.3.rs-1110860/v2

Download PDF

Research Article

Moving from Basic to More Advanced: Is there an Increase Over Time in the Complexity of Teacher Questions and Student Responses in Case Based Clinical Seminars?

https://doi.org/10.21203/rs.3.rs-1110860/v2

This work is licensed under a CC BY 4.0 License

Version 2

posted

You are reading this latest preprint version

Introduction:

Case-based group discussions (CBGD) are a specific, interaction-focused form of case based medical education dedicated to foster medical students’ skills of applying basic biomedical knowledge to patient cases. In such seminars, teacher question-student response patters are a crucial aspect. We empirically investigate to which degree clinical teachers progress from more basic, knowledge-oriented towards more advanced, elaboration-oriented questions in CBGD-seminars.

Methods

We video recorded 32 case-based seminars in the areas of surgery and internal medicine. 21 different clinical teachers had taught and 398 medical students had attended these seminars. We coded closed-reproductive and open-elaborative teacher questions as well as reproductive and elaborative student responses to these questions. Interrater reliability was satisfactory. For determining trends regarding the teacher questions / student responses, we compared eight equally long time-segments per seminar.

Results

Overall, clinical teachers posed more closed-reproductive than open-elaborative questions to students and the latter gave more reproductive than elaborative responses. Regarding the frequencies of these forms of teacher questions / student responses, we found no significant differences over time.

Discussion

We found that clinical teachers did not deliberately modify the types of questions they posed over time to push their students towards more elaborative responses. Generally, this raises the critical question to which degree promising didactic concepts are actually put into clinical teaching practice. In our sample, we observed substantial heterogeneity regarding the variables in focus of our study. We conclude that medical education researchers should more thoroughly take different starting conditions and teaching approaches of different teachers into account.

Educational Psychology

Teacher questions

Case based learning

Video study

Student elaboration

Case based didactic approaches are essential in the context of medical education (1, 2). Case based courses are characterized by the application of basic biomedical knowledge to authentic patient-cases; thereby, clinical reasoning and decision making are performed along key-phases of clinical case management, like clinical examination, diagnosis and therapy. In the present study, we focus upon case-based group discussions (CBGD), a specific form of case-based learning. In CBGD, a clinical teacher guides students through the phases of clinical case management by asking questions, explaining subject matter and encouraging discussion with and among students (3-5). Thereby, the clinical teacher (and not the students, as in other case base formats) is the primary source of teaching. Still, the goal of CBGD is in line with key aspects of case based medical education in general, i.e., to bridge the gap between basic biomedical knowledge and clinical work, to invite students to actively contribute to class and, in this way, to prepare them for their role as clinicians.

In context of CBGD, a challenge for clinical teachers is to adjust the level of complexity of subject matter and, specifically, of questions they pose to students’ level of knowledge (6, 7). If a teachers’ questions are too difficult, students will feel overburdened; if questions are too easy to answer, they will get bored. Both situations can lead to students’ feeling frustrated and detached from class. Moreover, as CBGD is supposed to bridge theory and clinical practice and prepare students for the latter, students should not only demonstrate that they possess relevant factual knowledge, but get opportunities to apply knowledge to clinically relevant problems and decisions and engage in clinical reasoning.

Some empirical studies give insights into the kinds of questions clinical teachers ask students in CBGD (8) and other medical teaching formats (9). However, these studies do not incorporate process-oriented aspects of teaching (10), specifically whether and how clinical teachers progressively modify their questions over time towards higher complexity in course of a teaching session. The progression from lower to higher teaching complexity and learner autonomy is a relevant didactic principle in medical education (11); moreover, it is an essential element of widely accepted theoretical models, e.g. the concepts of entrustable professional activities (12) and learners’ zone of proximal development (13). The former model suggests that with growing professional experience, young physicians perform increasingly complex clinical tasks more and more autonomously. The CBGD-seminars in focus of our study are supposed to prepare medical students for entering clinical work environments. Hence, it is consequent for clinical teachers to apply the idea of progressively increasing degrees of complexity and freedom of argumentation also in this context. The latter model hinted to above, learners’ zone of proximal development, represents a more learning-theoretical perspective on this didactic strategy: Learning theorists argue that for learning to be effective, the complexity of problems learners are confronted with should be slightly outside of their zone of actual development (13). In this way, learners are not overwhelmed, but are challenged to increase their understanding, e.g., by discovering new ways to apply their knowledge. To put this idea into practice, (clinical) teachers need to identify learners’ zone of actual development (e.g. through less complex, primarily fact-oriented questions) and, from there, move into learners’ zone of proximal development. Applied to CBGD, both concepts suggest that it is didactically reasonable if teachers initially pose basic questions (lower difficulty, focus on facts) and progress towards more advanced questions (higher difficulty, demanding transfer), in order to urge medical students to apply knowledge and explore new connections (14)

As was foreshadowed, whether clinical teachers apply this basic didactic strategy in context of CBGD has not yet been empirically investigated. This lack of empirical insight is a critical cavity in the literature for at least two reasons: First, an existing study from surgical education shows that clinical teachers do not modify questions they pose according to learners pre-existing knowledge (9). In order to estimate whether this is a more widespread problem in medical education, more evidence from other medical disciplines, phases of medical education and didactic contexts is necessary. Second, researchers from empirical educational research have argued that analyzing features of dialogue is an essential step towards an in-depth understanding of what makes teaching and learning effective in many didactic contexts (15). In addition, this kind of a base-line analysis of dialogic instructional patterns provides a fruitful basis for designing didactic interventions and professional training. Up to date, we could identify only one existing study from medical education which adopts such a perspective with respect to CBGD (8). We see this as a substantial gap in the literature, given the fact that literally all descriptions of case based medical education stress that the interaction between students and clinical teachers is one of its key characteristics (1, 16). Existing research shows that it is difficult for teachers to validly judge their own teaching because they tend to overestimate their performance (17). We hence argue that a video-analytic methodological approach inspired by research on dialogic teaching is promising to gain more objective and more detailed insights into CBGD as a form of case based medical education. On this basis, we address the following research questions: In context of CBGD seminars, 1) do clinical teachers move from posing more basic, reproductive to more complex, deliberative questions? and, 2) does the amount of deliberative student responses increase over time as compared to reproductive student responses?

Sample and recording procedure

During winter-semester 2016/17, we videotaped 32 case based clinical seminars (16 in internal medicine and 16 in surgery) taught by 21 different clinical teachers (we filmed two teachers five times, three teachers two times and 16 teachers once). In each seminar, a patient-case from the respective clinical discipline was discussed. Three seminars had to be excluded from further analyses due to technical problems. The clinical teachers in our sample were 38 years old (SD = 6.3) and had worked in their profession since 10 years (SD = 6.7) on average. In the present sample, only three teachers were female and 16 were male. On average, 15 students took part in each seminar (SD = 2.5, Min = 10, Max = 20); they were 24 years old (SD = 3.0) and were in their 8th semester of medical studies (SD = 0.9). All students had been informed about the study in advance. In case students did not agree to be video recorded, they were offered to change into another seminar. In case they did not want to be videotaped when their seminar started, they were offered to be seated outside the angle of the camera. To collect coherent data, we developed a standardized procedure of videotaping the seminars (18) and trained all researchers involved in the study. The average duration of a seminar was 83 minutes (Min = 62, Max = 104, SD = 10.9).

Ethical considerations

Out study was approved by the ethics committee of the TUM rechts der Isar university hospital (Application code 400/16 S). All teachers and students were informed about the study prior to the seminar per email. Out of the teachers, one physician refused to take part, so we did not record the respective seminar. Results based upon analyses from the dataset collected in context of the present study were already published in (8) and (19). Please see (8) for more details on the study methodology.

Instruments and variables

To collect demographic information, we distributed a questionnaire to all study participants. All other data analyzed in this study are based upon video-analyses. For this purpose, a hierarchical categorical scheme was created based upon published rating schemes from previous video studies (18, 20). Here, we focus upon the prevalence of two combinations of codes describing teacher questions, i.e., closed-reproductive questions and open-elaborative questions. Closed-reproductive questions have only very few (or even only one) right answer(s). They are often focused upon basic knowledge which students should be familiar with, e.g. from preclinical or clinical courses. An example for such a question from the video-study is “Which lab values do you need to look at in order to determine a liver problem?”. This question focuses information which is relevant also beyond the specific case discussed in the seminar. Moreover, we focus open-elaborative questions posed by clinical teachers. Such questions do not have a specific number of correct answers, but allow for differentiated, deliberative answers. Accordingly, they do not focus upon specific information, but respondents need to apply their knowledge in a reflective and evaluative way. An example of an open-elaborative question from our video study is “Given the age and background of the patient, which therapy would you recommend?”. Focusing closed-reproductive and open-elaborative questions allows us to analyze whether clinical teachers in case based seminars actually move from posing more basic (i.e., closed-reproductive) questions towards questions which impose higher cognitive demands upon students, like open-elaborative questions. We excluded the other two possible combinations (open-reproductive and closed-reasoning) of these categories from our analyses, mainly because they do not as consequently represent a rather narrow focus on facts and their reproduction vs. an orientation towards open reflection.

Regarding student answers to teacher questions, we also focus on reproductive and elaborative responses: Reproductive answers are mostly brief and contain very limited information, such as one piece of basic knowledge. Not seldomly, reproductive responses consisted only of one single word. In contrast, elaborative answers are more lengthy student contributions that contain considerations relevant for clinical decision making, like descriptions of cause-effect relationships, weighting of arguments or clinical reasoning processes.

Coding process

In the following, we describe the coding process with focus upon the variables analyzed in the present study (See 8 for a more comprehensive description). Coding was done in three rounds using the software Mangold Interact (21). In coding round one, the raters trained using the categorical scheme with videos from a pilot study which were not part of the main study sample until a satisfactory, overall interrater-reliability (Cohen’s Kappa > .80) was achieved. In round two, the main study videos were segmented to determine speakers (clinical teacher vs. student/s) and other surface level aspects (e.g. media use). In round three, further codes were applied to the video material to categorize the content of the verbal contributions, specifically regarding types of teacher questions / student responses. Thereby, the main study videos were divided between four coders. To ensure good reliability, two videos were analyzed by all four coders, resulting in a satisfactory IRR-value of Cohen’s Kappa = 0.68 (Gwet 2012).

Statistical analyses

The distribution of continuous data is presented by mean, range and standard deviation. Qualitative data is described by absolute and relative frequencies. For analyzing frequencies of teacher questions and student responses over time, and to standardize seminars of different length, we used deciles to divide seminars into ten equally spaced time intervals. If a seminar lasted 90 minutes, for instance, the video was divided into 10 intervals of 9 minutes each; if a seminar lasted 80 minutes, each interval was 8 minutes long. For each seminar and time interval, we then estimated the relative frequency of question / response types and illustrated their distribution across seminars by boxplots. A possible non-linear time trend in the respective median values was assessed by linear regression models with an orthogonal polynomial of degree 2. For each seminar, we excluded the first and the last time interval from all further analyses. This was because all seminars started and ended with clinical teachers clarifying organizational matters, like whether all participants were present or whether participants needed their signature to confirm seminar attendance, etc. All analyses reported here were done with the software R 4.0.3.

We could identify 511 teacher questions which were coded as closed-reproductive and 424 open-elaborative teacher questions in our video material. On average, teachers posed 22 (Min = 2, Max = 95) closed-reproductive and 19 (Min = 4, Max = 50) open-elaborative questions per seminar. Regarding student responses, we identified 577 reproductive and 434 elaborative student responses. On average, students gave 26 (Min = 6, Max = 102) reproductive responses and 21 (Min = 3, Max = 59) elaborative responses per seminar.

In figures 1 - 4, we report the frequencies of teacher questions / student responses in case-based seminars over time. Figure 1 shows the frequency of closed-reproductive questions posed by clinical teachers. We observed that clinical teachers posed such questions in a constant fashion during their seminars, with only minor differences between the time intervals (cf. figure 1). Accordingly, no significant differences between time intervals emerged here.

As is apparent from figure 2, clinical teachers posed open-elaborative questions with a lower overall frequency than closed-reproductive questions (cf. figure 1). However, we did not observe a trend of such questions being posed more frequently towards the end of the case based clinical seminars. In contrast, the mean values indicate a slowly declining trend regarding such questions, but we did not detect statistically significant differences between different time intervals.

Regarding student responses to clinical teacher questions, frequencies of reproductive responses also indicated a slowly declining trend of such statements being made across the duration of clinical seminars. However, no substantial differences between time intervals were found.

Finally, regarding elaborative student responses, we observed a positive trend in course of the seminars which, however, was also not statistically significant. In tendency, this results hints towards an increase in students’ elaboration about the clinical management of specific patient cases.

In the present study, we have studied case-based group discussions as a form of case-based clinical teaching. Our research questions were whether in course of the seminars, clinical teachers move from posing closed, fact oriented towards open, elaboration-oriented questions. Moreover, we have investigated how frequencies of reproductive and elaborative responses given by students develop over time across the clinical seminars. These research questions were analyzed based upon video data collected in case-based clinical seminars and analyzed by multiple raters with good reliability. For determining trends regarding the teacher questions / student responses, we compared eight equally long time-segments per seminar. Thereby, we detected no statistically significant differences regarding any of the variables. However, an increase over time regarding elaborative student contributions was apparent.

Conceptual descriptions of case-based didactic approaches in medical education stress that these are useful to bridge the gap between theory and practice, to teach clinical reasoning to advanced medical students and to help those acquire clinically relevant problem-solving skills (1, 16). However, given the results of our study, it is not apparent that the clinical teachers in the seminars managed to meet these goals in their instructional practice. Our data tentatively indicates that students contributed elaborative statements to class with a positive, however non-significant trend towards the end of the case-based seminars. At the same time, clinical teachers upheld a constant rate of closed, reproduction-oriented questions until the final stage of the seminars. Hence, it is not apparent that clinical teachers modified the types of questions they posed to push their students to elaborate, for instance, upon pros and cons of different therapeutic strategies. As other authors have formulated, “reliance on lower level questions diminishes the learner’s need to synthesize and formulate higher level answers” (9, p.544). In that sense, our results suggest that the case based seminars in our study were dominated by an informal, teacher-initiated division of labor between students and clinical teachers: Seemingly, the former were primarily responsible for providing facts, giving keywords and throwing brief ideas into the discussion, while the latter were adopting responsibility of weighing and evaluating these for their relevance in clinical practice. However, as is apparent from figures 1-4, substantial variance was apparent in our data. This points towards great heterogeneity between the seminars analyzed our study – which, overall, makes it difficult to identify statistically significant effects. For future research as well as for didactic interventions, this suggests that individual differences regarding starting conditions and teaching approaches between different teachers should more strongly be taken into account (22).

In the introduction, we have drawn upon the concepts of entrustable professional activities (12) and learners’ zone of proximal development (13) to substantiate the importance of moving from basic to more advanced questions. Our results suggest that clinical teachers did not deliberately attempt to push towards students’ zone of proximal development and did not progressively entrust a leading role to students in undertaking the more complex steps in analyzing the case at hand. In most cases, clinical teachers are much better able to analyze patient cases than their students based on their clinical experience and knowledge. However, in context of case-based teaching, they need to deliberately “suppress” this ability. Our study suggests that this is challenging for clinical teachers.

We argue that our results advance research on case-based didactic approaches in medical education, specifically of CBGD. They highlight that researchers should put more emphasis on investigating how teachers and learners interact in CBGD, as well as in other, dialogue-focused formats in medical education (23). Evidence from school settings shows that productive, learning oriented instructional discourse has measurable positive effects, e.g. regarding learning outcomes (24-26). With respect to medical education, such effects are just as plausible, but have rarely been deliberately investigated. Results which underline this conjecture come from studies on pimping or prodding (27-29). Through this form of direct questioning of medical students by clinical teachers, those deliberately attempt to disclose students’ knowledge gaps and potentially humiliate them in front of their peers. The fact that there are dozens of scholarly publications on this issue underscores that dialogic instructional practices – also critical ones! – are essential aspects of medical education. Video-based studies on teacher-student interaction in medical education are hence promising, on the one hand, to more precisely describe and contextualize such phenomena; on the other hand, as a valuable basis for conceptualizing didactic interventions dedicated to improve dialogic teaching practices (30, 31).

We finally address limitations of the present study and, on this basis, draw conclusions on future research. At first, the present study provides some hints towards more student-centered and effective interaction-based medical education. However, one limitation of our research approach surely is that we did not directly analyze student-teacher interaction patterns as they dynamically unfold during teaching. Instead, we counted all clinical teacher / student utterances relating to specific question / response categories and statistically explored their distribution over time. From a research strategic point of view, this is a promising first step; however, our study does not indicate why clinical teachers did not pose more open, elaboration-oriented questions. A second point relates to the fact that we empirically investigated CBGD as one specific form of case-based clinical teaching. Other, potentially more widely practiced forms of case-based teaching include elements which give learners a more active role and higher responsibility over prolonged periods of time. Based on the present results, we argue that this surely is one way to prevent clinical teachers from playing a too dominant role during the more complex, reasoning-oriented phases of case-based seminars.

Funding: This research received no financial support from any funding agency in the public, commercial, or not-for-profit sectors.

Conflict of interest: None.

1. Thistlethwaite JE, Davies D, Ekeocha S, et al. The effectiveness of case-based learning in health professional education. A BEME systematic review: BEME Guide No. 23. Med Teach. 2012;34:e421-44.

2. McLean SF. Case-based learning and its application in medical and health-care fields: a review of worldwide literature. J Med Educ Curric. 2016;3:39-49.

3. Koles P, Nelson S, Stolfi A, Parmelee D, DeStephen D. Active learning in a year 2 pathology curriculum. Med educ. 2005;39:1045-55.

4. Ten Eyck RP, Tews M, Ballester JM. Improved medical student satisfaction and test performance with a simulation-based emergency medicine curriculum: a randomized controlled trial. Ann Emerg Med. 2009;54:684-91.

5. Fatmi M, Hartling L, Hillier T, Campbell S, Oswald AE. The effectiveness of team-based learning on learning outcomes in health professions education: BEME Guide No. 30. Med teach. 2013;35:e1608-e24.

6. Tofade T, Elsner J, Haines ST. Best practice strategies for effective use of questions as a teaching tool. Am J Pharm Educ. 2013;77:1-15.

7. Long M, Blankenburg R, Butani L. Questioning as a teaching tool. Pediatrics. 2015;135:406-8.

8. Gartmeier M, Pfurtscheller T, Hapfelmeier A, et al. Teacher questions and student responses in case-based learning: outcomes of a video study in medical education. BMC Med Educ. 2019;19:1-13.

9. Barrett M, Magas CP, Gruppen LD, Dedhia PH, Sandhu G. It's worth the wait: optimizing questioning methods for effective intraoperative teaching. ANZ J Surg. 2017;87:541-6.

10. Bolhuis S. Towards process-oriented teaching for self-directed lifelong learning: a multidimensional perspective. Learn Instr. 2003;13:327-47.

11. Sandhu G, Magas CP, Robinson AB, Scally CP, Minter RM. Progressive entrustment to achieve resident autonomy in the operating room: a national qualitative study with general surgery faculty and residents. Ann Surg. 2017;265:1134-40.

12. Ten Cate O. Nuts and bolts of entrustable professional activities. J Grad Med Educ. 2013;5:157-8.

13. Harland T. Vygotsky's zone of proximal development and problem-based learning: Linking a theoretical concept with practice through action research. Teach High Educ. 2003;8:263-72.

14. Swanwick T, Forrest K, O’Brien BC. Understanding medical education: evidence, theory and practice. Oxford: Wiley Blackwell; 2019.

15. Mercer N, Wegerif R, Major L. The Routledge international handbook of research on dialogic education. Abington: Routledge; 2019.

16. Irby DM. Three exemplary models of case-based teaching. Acad Med. 1994;69:947-53.

17. Clausen M. Unterrichtsqualität: Eine Frage der Perspektive?. Münster: Waxmann; 2002.

18. Seidel T, Prenzel M, Kobarg M, editors. How to run a video study: Technical report of the IPN video study. Münster: Waxmann; 2005.

19. Häusler J, Grünewald M, Pfurtscheller T, Seidel T, Berberat PO, Gartmeier M. Too much time or not enough? An observational study of teacher wait time in case-based medical education seminars. Teach Learn Med. 2021 (Paper submitted for publication).

20. Johannes C, Fendler J, Hoppert A, Seidel T. Projekt LehreLernen (2008-2010). Dokumentation der Erhebungsinstrumente. Münster: Monsenstein und Vannerdat; 2011.

21. Mangold PT. Softwareunterstützte Verhaltensbeobachtung in klinischer Arbeit und Forschung. Horsch A, Lehmann T, editors. Bildverarbeitung für die Medizin 2000 Algorithmen - Systeme – Anwendungen. Berlin: Springer; 2000:444-8.

22. Schindler A-K, Seidel T, Böheim R, et al. Acknowledging teachers’ individual starting conditions and zones of development in the course of professional development. Teach Teach Educ. 2021;100:1-17.

23. Walton JM, Steinert Y. Patterns of interaction during rounds: implications for work‐based learning. Medic Educ. 2010;44:550-8.

24. Böheim R, Schnitzler K, Gröschner A, Weil M, Knogler M, Schindler A-K, et al. How changes in teachers' dialogic discourse practice relate to changes in students' activation, motivation and cognitive engagement. Learn Cult Soc Interact. 2021;28:100450.

25. Pehmer A-K, Gröschner A, Seidel T. How teacher professional development regarding classroom dialogue affects students' higher-order learning. Teach Teach Educ. 2015;47:108-19.

26. Smart JB, Marshall JC. Interactions between classroom discourse, teacher questioning, and student cognitive engagement in middle school science. J Sci Teacher Educ. 2013;24:249-67.

27. Pylman S, Ward A. 12 tips for effective questioning in medical education. Med teach. 2020;42:1330-6.

28. Goebel EA, Cristancho SM, Driman DK. Pimping in residency: the emotional roller-coaster of a pedagogical method–a qualitative study using interviews and rich picture drawings. Teach Learn Medicine. 2019;31:497-505.

29. Chen DR, Priest KC. Pimping: a tradition of gendered disempowerment. BMC Med Educ. 2019;19:1-3.

30. Jacobs J, Borko H, Koellner K. The power of video as a tool for professional development and research: Examples from the Problem-Solving Cycle. In: Janik T Seidel T, editors. The power of video studies in investigating teaching and learning in the classroom. 2009:259-73.

31. Gröschner A, Seidel T, Kiemer K, Pehmer A-K. Through the lens of teacher professional development components: the ‘Dialogic Video Cycle’ as an innovative program to foster classroom dialogue. Prof Dev Educ. 2014;41:729–56.

Download PDF

Version 2

posted

You are reading this latest preprint version

Moving from Basic to More Advanced: Is there an Increase Over Time in the Complexity of Teacher Questions and Student Responses in Case Based Clinical Seminars?

Status:

Version 2

Abstract

Introduction:

Methods

Results

Discussion

Figures

Introduction

Methods

Results

Discussion

Declarations

References

Status:

Version 2