IB exists in healthcare providers,39 can increase throughout medical training,40 and may be a modifiable factor in reducing health disparities due to racism.41 The IB education literature is characterized by variable approaches to curricular design and evaluation, learner outcomes assessed, and quality of reporting.12,23,24,27,28 If IB education is to lead to long-term changes in provider knowledge, skills, and attitudes, then a rigorous approach to curricular development is necessary. In this national modified Delphi study, a diverse group of subject matter experts in IB and health disparities with established relationships with NPM learners and staff reached consensus for curricular goals, learning objectives, education methods/strategies, and educator principles that will lay the foundation for a targeted neonatal medicine IB and health disparities curriculum.
Delineation of educational goals is critical to rigorous curriculum development for two reasons. First, goals should reflect the desired learning outcomes, and thus, they emphasize the development of certain knowledge, skills, or attitudes. This has profound implications on the appropriateness of subsequent learner assessment and curriculum evaluations. For example, curricula with goals oriented to provider behavioral change (e.g., communication strategies) align well with the assessment of simulated and real provider behaviors, as well as the impact of these behaviors on patients. Secondly, goals inform which learning objectives are required, and subsequently, which educational strategies will be appropriate to achieve the learning objectives.
In our Delphi process, experts reached consensus on eight overarching goals. These goals emphasize self-awareness of bias (G2 and G3), knowledge of the multiple components of modern health disparities (G1, G4, G5 and G7), and strategies for provider behavioral change (G6 and G8). The goals developed in this modified Delphi study reflect consensus opinion that IB education must not stop at acknowledgement and awareness of one’s IBs, but that such education must also provide learners with opportunities to practice bias mitigation and interventions when bias is recognized. This is similar to IB frameworks within broader HPE contexts imploring educators and institutions to “move beyond concepts toward applications.”21
The educational goals reaching consensus gave rise to numerous cognitive, social and behaviorist-oriented objectives that align with multiple levels of knowledge, as described by Miller.33 These objectives encompassed concepts related to bias and the self (O1-5), the impact of race in our society (O6-11), power and privilege (O12-16), and clinician skills and behaviors (O17-23). Significant attention was given to the action verbs within the objectives by both the study team and the Delphi experts to ensure objectives were specific, measurable, achievable, and realistic, which are critical to inform appropriate learning and assessment strategies. Because of the nature of the topics, many cognitively oriented objectives (with verbs such as “describe,” “identify,” and “reflect”) were found to be appropriate by the expert panel. These objectives support the foundational level of knowledge (in Miller’s pyramid, that a learner “knows”) and lend themselves well to written assessments.33 Although foundational and necessary, this knowledge level is not sufficient for the ultimate goal of reducing provider IB and subsequently reducing health disparities.41 Additional objectives reaching consensus support application and demonstration of knowledge, specifically in the development and practice of skills to recognize and mitigate IB (O17-23). These skills (with verbs such as “develop,” “articulate,” and “apply”) allow for learning at the level of Miller’s “shows” and “does”33 and lend themselves well to oral, performance, and workplace-based assessments.
Though experts reached very high levels of agreement for goals and objectives that closely aligned with several established IB educational frameworks, the educational methods, strategies, and guiding principles were more contentious. Some educational approaches advocated in the literature were rejected by the expert panel, showing that not all strategies espoused in the literature are recommended as practical and useful by experts. Eight educational methods/strategies and eight educator principles proposed in the literature failed to reach consensus amongst this expert group (see Appendix 1). For example, experts disagreed that this curriculum should incorporate mindfulness training, implore learners to increase their motivation to be fair, use fantasy characters to teach about stigma, or use exercises based on public exposure of privilege. Experts also had divergent opinions about the utility of the Implicit Association Test (IAT)37 and did not reach consensus that the IAT should be a required component of the curriculum. Experts did not agree that taking an IAT leads to changes in learner self-perception, a finding that Gonzalez and colleagues have also reported.12 Experts also did not reach consensus on the value of the IAT as an assessment tool for long-term learning nor as a program evaluation tool. This finding aligns with the guidance from Project Implicit which specifically states that the IAT does not meet reliability standards for measurement and discourages its use as an assessment tool within pre-post research designs unless a control group is also used.37 Nonetheless, experts reached consensus that the IAT has a role as a “catalytic reagent” for inciting self-awareness though cautioned its use in isolation, as knowledge of one's biases alone may lead to discounting of the findings or even hostility in the learner.
Experts encouraged educational strategies such as reflection, storytelling and teach back for learning about IB. Strongly endorsed bias mitigation strategies (with > 90% agreement) included stereotype replacement, counter-stereotyping, individuation, perspective taking, and increasing opportunities for contact. Experts also reached consensus for all five proposed bias-response strategies, with the highest levels of percent agreement for the Step Up/Step Back method (100%)24 and the Active-Bystander model (93%).42 The Step Up/Step Back model encourages participants to actively decide whether to speak out for those unable to or to support others to speak out for themselves, whereas the Active Bystander model focuses on recognition of the bystander role, inhibitors and facilitators of action, and a variety of intervention techniques. This suggests that learners may benefit from learning multiple practical strategies for attending to bias.
Finally, the consensus on educator principles suggest that educators, facilitators, and learners all shape the learning environment during IB and health disparities education. Learners approach bias education from a variety of backgrounds and with unique life experiences and educators may benefit from conducting local needs assessments to determine their own learners needs, baseline knowledge, skills, and attitudes. In contrast to many curricula in which goals and objectives are to be completed at the end of a finite period, experts agreed that personal and professional growth in IB topics are lifelong pursuits that cannot be addressed in a single session, module, or calendar year. Experts valued individual goal setting and encouraged educators to recognize that learners may progress in their knowledge and skill at different rates. Thus, the ideal curriculum would allow for local educators to determine the relative emphasis, frequency, and duration of topics. Experts encouraged facilitators to vigilantly monitor for power imbalances and adverse effects on learners (such as marginalization, adding to the minority tax, shame, or guilt), and discouraged educators from presenting cultural “menus” and perpetuating stereotypes. Finally, though both “safe” and “brave” spaces are advocated for in the literature,24, 29, 43 experts did not agree that one approach was superior to another; they instead left the choice to the local facilitator.
This study has limitations related to the modified Delphi process. The final product of consensus building methodologies relies on the level of expertise of the expert pool.44 Expert attrition during the Delphi study was moderate. We also chose an a priori level for consensus as 70% agreement and did not force experts to rank order items in order of importance, both of which may increase the number of final curricular components and make the curriculum longer or more complicated than desired. Yet, the number of items reaching high levels of expert consensus highlights the complexity of this topic, and it is unlikely that any one curriculum would be able to incorporate all these elements. Instead, the compiled lists provide educators a scholarly initial direction to bias education while still allowing them to tailor the curriculum to their local learners, resources, and contexts.
In summary, a rigorous approach to a neonatology specific IB and health disparities curriculum is a key step towards promoting health equity in the NICU. Delphi experts reached consensus for numerous curricular components, including goals and objectives, educational strategies, and learning environment considerations, for such a curriculum. These curricular components should be considered when creating IB focused educational materials for NICU providers. Future studies are needed to evaluate the proposed methods of delivering this curriculum and its effects on patient and family outcomes.