Settings
Jichi Medical University (JMU) is a private medical university founded in 1972 in Japan. The curriculum of JMU complies with the standardized model core curriculum outlining fundamental learning contents for undergraduate medical education in Japan [27]. While the pre-clinical curriculum is partly integrated, it remains mostly stepwise: Before Year 4 to 6 (Y4-6) clinical clerkship, students mainly learn clinical medicine in traditional didactic lectures, and their progress is assessed through end-of-unit tests. JMU students study liberal arts in the first and second trimesters of Y1. Lectures and experiments in basic medicine also begin in Y1 second trimester. Clinical medicine lectures start from Y2 second trimester. Each basic and clinical medicine class is capped by end-of-unit tests where lecturers create test items based on lecture content. Before Y3 end, students finish almost all subject lectures in basic and clinical medicine.
Y3 has seven sessions of one-day hybrid PBL, each divided into four segments: morning case discussion for the formulation of self-study objectives, self-study period for research on objectives and preparation for afternoon discussion, afternoon discussion including within-group information sharing, and a 60-minute wrap-up lecture from a specialist. In every PBL, faculty facilitators assess each student’s opinion statements, cooperative attitudes, and appropriateness of self-study using a 3-point scale. However, the assessment does not affect grade point average, and students never receive individual feedback.
Currently, students read the introductory part of a clinical scenario before PBL and preview morning discussion content. For example:
‘A 56-year-old man came to your hospital because he had 20-min anterior chest oppression after breakfast this morning. Please find clinical problems or possible differential diagnoses as best you can.’
The full story containing clinical history and findings is provided on the date of the PBL session.
Because of prevailing teacher-centered education culture at this university, implementing large-scale curriculum reform towards full PBL-based implementation is difficult. Students attend only seven one-day hybrid PBL sessions a year with the rest of the Y3 curriculum lecture-based. This partial use of the PBL concept does not guarantee its full efficacy [14].
PIF-oriented PBL
The core component of the PIF-oriented PBL is the pre-PBL communication platform composed of an online instruction video and the essay format. The instruction video aimed at encouraging students to articulate their future image as an independent medical professional tackling patient problems via life-long learning. The essay format meant to provide pre-PBL in-depth communication between students and their future role models (Figure 1).
After watching the instruction video and reading the PBL scenario introductory, students were asked to answer three essay questions:
Q1: Please recall as much knowledge as possible you have about this case.
Q2: Please formulate your future professional images, and articulate how useful this PBL case-study would be to you as a doctor responsible for this case.
Q3. Based on your answer for Q2, please articulate how you will optimize your self-study for this case to make this opportunity most meaningful.
Aside from these questions, students were asked to submit the professional identity essay (PIE) proposed by Kalet et al. [28] three times. PIE is useful for helping learners articulate their values and norms about medical professionalism, and teachers provide feedback by referring to rubrics based on Kegan’s constructive developmental theory [28, 29]. This study used a Japanese version of PIE (PIE-J) as a reference for mentors’ feedback on Q1-3 and as an assessment tool for students’ PIF levels (see also Instruments).
Those materials were provided on a Moodle online learning management system. Through the Moodle platform, eligible JMU-graduate mentors provided feedback on Q1 to Q3 by simultaneously referring to each student’s PIE. As a rule, the mentor feedback did not contain hints for the PBL scenario in order to avoid teacher-centered instruction. In this study, six JMU graduates with clinical experience of 18-37 years were chosen as feedback providers. All of them have agreed to received intensive training for appropriate PIE use and feedback on Q1-3 before the study.
We hypothesized Moodle-based pre-PBL learning would raise student awareness of their future professional image and the relevance of PBL scenarios for their future professional selves. We also proposed that a clearer image of their future professional selves and relevance of PBL contents would encourage them to apply learning strategies encouraged by thoughtful mentor comments. Overall, we expected that the pre-PBL PIF-oriented education format would promote PIF and accordingly improve SRL.
Participants and design
We used a quantitative analysis for RQ1 to RQ3 and a mixed explanatory method to address RQ4 regarding causes of PIF or SRL improvement. We used this method because we expected qualitative analysis would illuminate the mechanisms for PIF or SRL changes in this research context; i.e., qualitative data could help explain quantitative results [30]
1) Quantitative Approach: RQ1-RQ3
A randomized controlled crossover trial was designed for the quantitative arm. All JMU 2019 Y3 students (n = 124) were invited to participate in this research. Eventually, 112 agreed were randomly divided into two groups: Group A (n = 56, female=18, male=38, mean age 21.5y±0.7) and Group B (n = 56, female=11, male=45, mean age 21.7y±1.0). Group A used Moodle-based PIF-oriented format before the second to fourth PBL, while group B did before the fifth to seventh PBLs in 2019. Both groups conducted the six one-day PBLs in the same manner on PBL dates, and SRL and PIF levels were compared between the two groups (Figure 2). Group A and B did not mix in the PBL group session. We hypothesized that PIF and SRL levels should improve in parallel, i.e., Group A in the first half of the research period, with Group B becoming equivalent to Group A in the second half.
2) Qualitative Approach: RQ 4
Following quantitative analysis results, we explored potential reasons for PIF improvement using responses from the PIE-J from 112 students (see also Instruments). The analysis was conducted in March and April of 2020.
Instruments
1) PIF data collection
For quantitative analysis, the PIF levels for norms and values of professionalism were measured using PIE, an essay-based measurement tool with 9 question items. Referring to Dr. Kalet and colleagues’ rubric [28] based on Kegan’s identity stage [28, 29], assessors chose learners’ professional identity levels from Stage II to II/III, III, III/IV, IV, IV/V, and V. The PIF measurement by PIE has been validated in undergraduate settings [28, 31]. In this study, we used a Japanese version of the PIE form and rubric (PIE-J) originally in English. Back translation between English and Japanese was conducted by the main author (YM, Japanese) and an American professor living in Japan literate in both English and Japanese (AJL). PIE stages from all students were assessed by two authors (YM & MN) by in-depth discussions following the rubric until full agreement was reached.
Responses to all PIE-J questions were investigated for the qualitative analysis. However, we especially focused on PIE-Q9 ‘Reflect on your experiences in medical school or in the community that have been critical in fostering change in your understanding of what it means to be a professional/to be a physician’ to illuminate change processes in PIE stages.
2) SRL data collection
Learners’ SRL levels were measured by a Japanese-language version of the Motivated Strategies for Learning Questionnaire (MSLQ-J) [32] in the quantitative analysis reported to be useful in measuring SRL in undergraduate medical education [19, 33, 34]. MSLQ is composed of 81 items with seven-point Likert scales which quantify levels of nine types of learning strategies (rehearsal: R, elaboration: ELA, organization: O, critical thinking: CT, metacognitive self-regulation: MSR, time and study environment: TaSE, effort regulation: ER, peer learning: PL, and help-seeking: HS), and six variables of motivation states (intrinsic goal orientation: IGO, extrinsic goal orientation: EGO, task value: TV, control of learning beliefs: CBaL, self-efficacy for learning and performance: SEfLaP, and test anxiety: TA). All 81 items were translated into Japanese and back-translated by the main author (YM) and an American professor (AJL).
Analysis
1) Quantitative Approach
Effects of treatment and time (fixed effects) on MSLQ-J scores and PIE-J stages were tested and estimated (RQs 1-2) using two-level regression analysis (upper level: participants; lower level: occasion) in the Open Source statistical package jamovi (version 1.2.9) [35]. Treatment and time were treated as fixed effects (estimated with full informed maximum likelihood), and participant-level random intercept served as a random effect (estimated with restricted maximum likelihood). For the first measurement of all scales, the two groups were treated as one because the first measurement took place before any treatment (see Chapter 15 in [36] for a detailed explanation of this model and the rationale behind it). Marginal R2, a multilevel equivalent of the R2-statistic commonly used for traditional linear regression models, was used to estimate the effects of time and treatment (values of around 0.01, 0.06 and 0.14 represented small, medium and large effects). The Bayesian Information Criterion (BIC) was used to determine which of the time-effect-only and the time-and-treatment-effect model is to be preferred (i.e., the model with the smallest BIC) [36]. Correlations between MSLQ-J scores and PIF-J stages (RQ3) were analyzed and visualized using network analysis in the Open Source statistical package JASP (version 0.12.1.0) [37].
2) Qualitative Approach
From a constructivist paradigm in which ‘reality’ is subjective and context-specific, and multiple truths are constructed by and between people [38], qualitative data from PIE-J were analyzed using constructivist thematic analysis. We coded anonymized transcripts of the Japanese essays following the six phases proposed by Braun and Clarke [39]. Coding was conducted by the two Japanese researchers (YM and MN). YM was the lead author, who engaged in the development of PIF-oriented PBL and had experienced qualitative studies relevant to SRL. MN was chosen to conduct coding because he is not involved in the JMU curriculum but had experienced qualitative studies relevant to SRL. The PIE statements were thoroughly read and analyzed using an inductive coding approach until agreement on coding was achieved between the pair. In the inductive coding approach, we referred to the PIF stages measured in the quantitative approach. We intensively analyzed statements from those with increased PIE-J stages or intentionally compared statements between those with and without PIF improvement to illuminate meaningful codes and themes.