Acquisition of Behavioral Skills after Manikin-Based Simulation of Neonatal Resuscitation by Fellows in Neonatology

Abstract Objective  Behavioral performance of health professionals is essential for adequate patient care. This study aimed to assess the behavioral skills of fellows in neonatology before and after a simulation training program on neonatal resuscitation. Study Design  From March 2019 to February 2020, a prospective cohort with 12 second-year fellows in neonatology were evaluated during three training cycles (16 hours each) in manikin-based simulation of neonatal resuscitation with standardized scenarios. Each cycle lasted 1 month, followed by a 3-month interval. One video-recorded scenario of approximately 10 minutes was performed for each fellow at the beginning and at the end of each training cycle. Therefore, each fellow was recorded six times, before and after each one of three training cycles. Anxiety of the fellows was assessed by the Beck Anxiety Inventory applied before the first training cycle. The videos were independently analyzed in a random order by three trained facilitators using the Behavioral Assessment Tool. The behavioral performance was evaluated by repeated measures of analysis of variance adjusted for anxiety and for previous experience in neonatal resuscitation. Results  Fellows' overall behavioral performance improved comparing the moment before the first training and after the second training. The specific skills, such as communication with the team, delegation of tasks, allocation of attention, use of information, use of resources, and professional posture, showed a significant improvement after the second month of training. No further gains were noted with the third training cycle. Anxiety was observed in 42% of the fellows and its presence worsened their behavioral performance. Conclusion  An improvement in behavioral performance was observed, comparing the moment before the first training and after the second training, without further gains after the third training. It is worth noting the important role of anxiety as a modulator of acquisition and retention of behavioral skills in health professionals in training. Key Points Simulation training should improve technical and behavioral skills of providers Behavioral skills improve after a first cycle of training, but not after a repeat cycle Anxiety modulates trainees' behavioral performance

Under-five mortality is a public health issue. 1 Worldwide, neonatal mortality represented 47% of deaths among children under 5 years of age in 2019 2 and intrapartumrelated events contributed with around one-third of these deaths. 3Neonatal resuscitation training could reduce 30% of neonatal deaths due to intrapartum-related events, especially in countries where peripartum care is insufficient. 4o perform neonatal resuscitation, health care professionals require cognitive, technical, and behavioral skills to provide adequate assistance to the newborn at birth. 5 In critical situations, behavioral performance may influence clinical outcomes.According to the Joint Commission, 72% of reported sentinel events in North American hospitals in 2004 were related to the lack of communication among the members of the teams that provided assistance to the mother and to the newborn infant. 6Again according to the Joint Commission, from 2004 to 2014, 79% of the root causes for perinatal events were related to human factors. 7eams working in the delivery room must be trained to carry out the necessary procedures and the coordinated action with effective communication between its members, providing quality care and patient safety. 8The International Liaison Committee on Resuscitation (ILCOR) recommends that students and professionals who work in newborn assistance in the delivery room must be trained in their technical skills more than once a year, but there is insufficient evidence to determine the exact time between trainings for the better retention of knowledge by the students. 9Realistic simulation is a promising educational methodology to train technical and nontechnical skills in neonatology, as it is student centered and focused on previous experience-based learning. 10,11It allows participants' emotional involvement, and it makes the experience as accurate as real life, providing training in leadership, communication, problem solving, and teamwork in a safe and controlled environment. 12he acquisition of behavioral skills with simulation training in neonatology has not been extensively studied.These skills were analyzed by LeFlore et al, comparing self-directed learning with instructor-modeled learning.The authors validated the Behavioral Assessment Tool to be used in the neonatal resuscitation scenarios and they found that, with the learning "modeled" by an instructor, the behavioral performance of the students was better. 13,14However, it is not known what the best ways are to design simulation training to improve both technical and behavioral skills of providers.In this context, this study aimed to assess behavioral skills of neonatology fellows in scenarios of resuscitation at birth before and after three training cycles with realistic simulation.

Materials and Methods
This was a single-center, prospective cohort study that included 12 second-year neonatal fellows from Escola Paulista de Medicina, Federal University of São Paulo, divided in four groups of three fellows each.This study was conducted from March 2019 to February 2020.The study was approved by the Ethics Committee on Research of Universidade Federal de São Paulo (approval number: 3.128.093).
Inclusion criteria were neonatal fellows from the mentioned institution who agreed to participate through the consent form and image use form; they had to be certified by the Neonatal Resuscitation Program of Brazilian Society of Pediatrics that provides an 8-hour theoretical and practical training on initial steps, positive pressure ventilation with mask, tracheal intubation, chest compressions, and use of medication.Also, they should be present in at least 70% of the curricular sessions of the institutional training of newborn care with realistic simulation provided by the fellowship program.
During the second year of neonatal fellowship, the fellows attend a manikin-based simulation training for three nonconsecutive months.Each month of training is followed by 3 months of clinical activities, with a total of 3 months of simulation training during the year.Two weekly meetings are held: one in situ at the obstetric ward, and one at the Laboratory of Realistic Neonatal Simulation.In each of the 120-minute meetings, fellows participate in two to three scenarios of neonatal care in the delivery room.Over the course of the 3-month training, fellows are subjected to approximately 60 different scenarios, focusing on technical skills (positive pressure ventilation, orotracheal intubation, chest compressions, and umbilical catheterization), preparation of materials/equipment in the delivery room, anamnesis and communication with the mother, communication with the obstetric team, and behavioral skills with an emphasis on teamwork and leadership.All simulated clinical scenarios are videotaped.Clinical scenarios are elaborated by the facilitators, according to the objectives to be discussed.The scenarios performed throughout the month are randomly distributed and may include more than one technical skill, but all fellows participate in the same scenarios until the end of the training.After each scenario, a debriefing is performed to review the activities of the simulated scenario.
In this study, each training block had three fellows who participated together in the same standardized scenarios, each at a time being the leader of the resuscitation scenario.Although working together, each fellow was individually evaluated regarding the performance during the scenario that he/she had the leadership role.The other nine fellows of the Neonatal Fellowship Program were in clinical activities.On the first and last training days of each cycle, three scenarios, followed by a debriefing, were performed, so that each fellow could be a leader in one of these scenarios.Six videos were analyzed for each fellow, at the beginning and at the end of each of the three cycles of training (►Fig. 1).The signed consent guarantee that these videos will be discarded 10 years after the end of the study.
The videos were evaluated by three independent facilitators blinded to the training cycle and moment that the film was made.It was not possible to carry out the evaluation completely blindly, as the facilitators knew that the students are being evaluated.All the video analyses were performed after recording all six sessions from all fellows, after they have completed their Neonatal Fellowship Program.The evaluators were unaware of the sequence of training for each student.The randomization of the video sequence offered to each evaluator was done by the Random application (Developer: Mireia L. Ortola for IoS system, 2019).The three evaluators were female, with a median age of 50 years (range: 42-50 years) who had been practicing neonatology for a median of 25 years (range: 10-25 years); all of them had taken the Realist Simulation Instructor Training Course promoted by Laerdal Medical (Brazil), and all of them were trained in behavioral assessment by the coordinator of the Simulation Laboratory.
Behavioral performance was assessed in the videos using the "Behavioral Assessment Tool" (BAT).This tool had its internal consistence measured in several studies, with a Cronbach's α coefficient varying from 0.92 to 0.97, [14][15][16][17][18] but its validation is cited by Anderson et al 16 as unpublished.The BAT has 10 variables related to behavioral skills and, to score of each one, the evaluator must choose from 0 to 4, ranging from poor to excellent or from inexperienced to expert.The analyzed behaviors were knowledge of the environment, anticipation and planning of possible problems, assumption of the leadership role, communication with other team members, workload distribution and delegation of responsibility, allocation of attention, use of all information, use of resources, recognition of limitations and request for help at the appropriate time, professional behavior and interpersonal skills, as well as the total score of the scale.
To adjust the results for possible confounders, the neonatal fellows were asked about the number of newly born infants that they participated in the delivery room care during the second fellowship year.Also, the Beck Anxiety Inventory (BAI) was applied to evaluate the anxiety before the beginning of the initial training.BAI is a validated tool to evaluate anxiety symptoms in a Likert's scale from 0 to 4 points.The final score classifies the anxiety as absent (0-10 points), mild (11-19), moderate (20-30), and severe (31-63). 19,20The written version of BAI was given to the fellows to be completed in the simulation setting or at home.
Fellows estimated their confidence during newborn care in the delivery room by a questionnaire adapted from Lee et al. 21The fellows answered the questionnaire in the first and last training day, estimating their self-confidence in four clinical situations as follows: (1) newborn infant with 35 weeks of gestational age (GA) born by vaginal delivery (situation 1); (2) newborn infant with 33 weeks of GA born by cesarean section due to a hypertensive disorder of pregnancy (situation 2); (3) neonate with 40 weeks of GA with meconium stained amniotic fluid (situation 3); and (4) neonate with 26 weeks of GA born by cesarean section due to intrauterine growth restriction (situation 4).The answers were graduated by a Likert's scale from 0 to 9, as 1 being "uncomfortable" and 9 "very comfortable." A sample size calculation was made by estimating a 30% improvement in the main outcome for the study, the global behavioral score assessed by BAT, with a standard deviation of BAT scores of 0.35, based on a pilot study.An α error of 5% for a sample power of 80% was considered.According to these parameters, the participation of 12 fellows was needed.
The agreement between the facilitators for the BAT scale evaluated in the videos was analyzed using Cronbach's α. 22 For the final score of each student in each video analyzed, the average of the scores obtained by the three evaluators in each of the items of the BAT and the total score were considered.The behavioral performance at the beginning and at the end of each of the 3 months of simulation training, for each of the items of the BAT scale and for the total score, was compared by repeated measures of analysis of variance (RM-ANOVA), and the differences were assessed by Bonferroni's post hoc test. 23The RM-ANOVA for each variable of the BAT and for its total score was adjusted for number of neonates assisted in the delivery room during the second-year fellowship and for the score of the BAI.Thus, the analysis by RM-ANOVA included the main effects "Training (first, second, or third month of training)," "Moment (beginning and end for each month of training)," and the interaction between "Training" and "Moment," adjusted for "Anxiety" (BAI 10 or !11) and for the number of neonates received in the delivery room during the study year.

Results
Demographic data of the 12 second-year neonatal fellows are shown in ►Table 1.The fellows, who had an average age of 28.5 years, were mostly women, and had assisted around 100 neonates in the delivery room in each year of the Neonatal Fellowship Program.Before the beginning of the training cycle, the assessment of the presence of anxiety by the BAI showed that 41.7% of the fellows had some degree of anxiety (►Table 1).
Regarding self-confidence in the four situations evaluated, there was an increase in the average confidence of the fellows from the initial moment of the simulation to the final moment.However, only in situations 2 and 4, where the degree of complexity was higher, the difference was significant (►Table 2).
As for the agreement among the three evaluators of the BAT score, the Cronbach's α for each behavioral item ranged from 0.635 (domain: resource utilization) to 0.801 (domain: leadership assumption), and it was 0.769 for the total score, showing a good agreement. 24he mean score measured by the three independent evaluators for each item of the BAT scale individually and for the whole score, at the six time points, are shown in ►Fig.2A and B, respectively.There was an increase in the scores of all skills after the first training, with a decrease after the 3-month interval until the beginning of the second training.There was again an increase in the behavioral The results of the BAT were analyzed by RM-ANOVA for the main effects "training" " (first, second.or third cycle of training) and "moment" (before and after each training) and the interaction between training and moment, adjusted by "presence of anxiety (BAI > 10)" and "number of newborns received by the fellows in the year of the study."The main effect "moment" was significant for the following BAT domains: communication with the team, delegation of tasks, allocation of attention, use of information, use of resources, professional attitude, and overall score of behavioral performance (►Fig. 2).There was an interaction between training and moment only for one domain that recognizes limitations and call for help early enough (p < 0.001).The presence of anxiety modified the performance in the following domains: knowledge of the environment (p ¼ 0.013), allocation of attention (p ¼ 0.032), use of resources (p ¼ 0.013), asking for help in appropriate time (p < 0.001), and professional behavior (p ¼ 0.014), as well as in the total score of the scale (p ¼ 0.031).The post hoc analyses of the differences with the Bonferroni's test are shown in ►Table 3.

Discussion
In this prospective cohort study of second-year neonatal fellows, a 4.5-point significant improvement in the BAT total score was observed between the initial time prior to any training and the moment after the second training, with no additional gains at the end of the third training.The presence of mild or moderate anxiety was an independent variable, decreasing the behavioral performance of the studied trainees.The self-confidence of neonatal fellows in assisting newborn infants in the delivery room increased after the three training cycles, especially in more complex clinical situations.Some hypotheses have been raised for the lack of consistent improvement of the behavioral performance of neonatal fellows at the end of the last training.The trainees were not  inexperienced in Realistic Simulation before the training because they had already participated in the Neonatal Resuscitation Program of the Brazilian Society of Pediatric.Another issue to be considered is the fact that the third training cycle took place in the last months of fellowship, while most trainees already have a certain degree of fatigue, anxiety, and loss of focus.Also, highlighting problems in the teaching model offered by this simulation program, the repetition of scenarios on the same topic, with only minor changes, may have contributed to the lack of interest in the training.Therefore, a change in our teaching program has been elaborated after the results of this study, offering more diverse and challenging scenarios during the last months of training.
For young adults' learning, novelty and challenge are important drivers. 25One of adult learning phases is the perception of a gap in their knowledge, and for this to happen, there needs to be a challenge which can be internal or external, stimulated or not by the facilitator. 24This process may have been minimized by the previous experience with the Neonatal Resuscitation Program and by the repetition of scenarios.It is also a worth-mentioning fact that the group of fellows was always the same, and it was possible to observe that some fellows, by the end of the year, were used to work together and were no longer concerned about establishing a communication in a closed loop.
Although the final result of the training was not as expected, it is interesting to observe the curve of acquisition, retention, and loss of behavioral skills exhibited by the fellows in this study (►Fig. 2).There was a downward trend in behavioral performance between the end of each training cycle and the beginning of the next, showing that behavioral skill retention remains only for a few months.Patel et al evaluated cognitive and technical skills in North American Pediatric residents.The authors showed that retention of technical skills starts to drop after 2 to 3 months of training. 26Matterson and colleagues showed a reduction in retention of technical skills after 4 months of Realistic Simulation training in neonatal resuscitation. 27Our data reinforce the need for periodic retraining in neonatal resuscitation for health care teams responsible for neonatal resuscitation to maintain previously acquired skills.
Regarding the specific behavioral skills, there was significant improvement between the beginning of the training and the end of the second training in the following skills: communication with the team, delegation of tasks, allocation of attention, use of information, use of resources, professional attitude, and the overall score of behavioral performance.Of the four behavioral domains that did not change with training (knowing the work environment, anticipating problems, taking the lead, and asking for help at an appropriate time), the first two may have been influenced by the lack of a longer filming time before the actual scenario.Also, not all scenarios allowed the evaluation of the behavior "request for help."Therefore, in some videos, the evaluators noted this behavior as "not evaluated," reducing the number of videos analyzed and decreasing the sample power.However, the absence of improvement in the leadership performance deserves special attention.The fact that the group of trainees was always the same and may have influenced the teamwork, with the observation in some videos of a "shared" leadership, with two trainees providing commands.It was also noted that some fellows started the scenario as leaders, but lost the leadership, and another fellow assumed this role.The same finding was shown by Sawyer et al in 2014.In their study, pediatrics and family medicine residents received three simulation trainings on neonatal resuscitation and 15 teams of two residents completed all three training over a period of 9 months.A total of 45 resuscitation videos, 15 from each of the three simulation sessions, were reviewed and scored by two blinded reviewers using the BAT.Although BAT global score improved from the first to third session, assumption of leadership did not change throughout the simulation trainings. 17n the present study, approximately 42% of fellows had some degree of anxiety, according to the BAI, and its presence significantly modified the trainees' performance in the following domains of the BAT: knowledge of the environment, allocation of attention, use of resources, request for help in appropriate time and professional behavior, as well as in the total BAT score.Al-Ghareeb et al studied 33 nursing students to assess the influence of anxiety on their performance in a Realistic Simulation scenario and students with higher levels of anxiety showed worse technical, cognitive, and behavioral performance. 28The data obtained here and by Al-Ghareeb et al suggest that attention to the mental health of professionals in training is critical for a full benefit of programs aimed at the acquisition of technical and behavioral skills in areas in which the need for decisions and actions at a rapid and consistent pace is decisive in the prognosis of patients.
In our study, there was an increase in the average confidence of the fellows when comparing the moment before any training in simulation with the moment after all training cycles for all clinical situations evaluated.However, only in situations of higher medical complexity, the difference was significant.This result cannot be attributed only to the experience provided by the realistic simulation training, but it occurred in the context of completing the whole Neonatal Fellowship Program developed in the university hospital.The role of exposure to realistic simulation in the confidence of health professional is controversial.A systematic review with 24 studies, published in 2012, suggested that there is no robust evidence to confirm that high-fidelity realistic simulation improves students' self-confidence. 29owever, in another systematic review published in 2019, the self-confidence of nursing students after Realistic Simulation training was evaluated in 29 studies and the results indicated that this teaching technique improved students' self-confidence. 30verall, this study contributes to building knowledge about the acquisition and retention of behavioral skills of health professionals after realistic simulation training.The cohort design over 1 year, with three cycles of 1 month of training, followed by 3 months of "rest" (►Fig. 1),allowing us to verify the acquisition and retention of behavioral skills over time and not only the short-term effect of a single training session.After this study and because of the lack of improvement in behavioral performance after the third training cycle, the simulation teaching group changed the curriculum for the following groups of fellows with a reduction of cycles and a more diverse number of scenarios, but the results of these changes still need to be evaluated.It is worth noting the important role of anxiety as a modulator of acquisition and retention of behavioral skills in health professionals in training.

Limitations
Despite these findings, it is necessary to point out some limitations of the study.The sample of fellows was small from a single center and chosen by convenience, nevertheless with a sample calculation showing sufficient power and with a prospective design that followed the fellows' performance throughout the year.It is important to consider that the video evaluators were present in some scenarios during data collection and knew the fellows.To minimize the bias of the unblinded evaluation, the randomization of the order of the videos analyzed by each evaluator decreased the subjectivity of the evaluation of each fellow individually.Another important limitation was the lack of a systematic assessment, in this study, about suggestions of facilitators and learners on how to optimize the training and avoid the lack of improvement in the third cycle of training.Maybe, by confronting both views, with a qualitative analysis, we could have identified opportunities to improve this learning experience.

Conclusion
In conclusion, an improvement in the behavioral skills of neonatal fellows in delivery room newborn care scenarios was observed between the time before any training and after the second training cycle, with no benefit after the final training cycle.Reported self-confidence in handling complex cases increased between the beginning and the end of the realistic simulation training, and anxiety presence was a modulator of trainees' behavioral performance.

Note
This manuscript is being submitted only to BMC Medical Education, and it will not be submitted elsewhere while under consideration.All authors read, reviewed, and approved the final version of the manuscript.

Ethical Approval
The research protocol was approved by the Ethics Committee of Federal University of São Paulo (Unifesp-Escola Paulista de Medicina), approval number: 3.128.093.All methods were performed in accordance with the precepts of the Declaration of Helsinki and the Nuremberg Code, respecting the Standards for Research involving human beings of the National Health Council.An informed written consent was obtained from all participants of the study.

Fig. 1
Fig. 1 Distribution of videos and training cycles.

Fig. 2
Fig. 2 Acquisition of behavioral skills during the training cycles according to the BAT scale.(A) Mean scores of each learning curve of each behavioral domains.Repeated measures ANOVA showed significant differences for the main effect "moment" for communication with the team (p ¼ 0.010), delegation of tasks (p ¼ 0.013), Allocation of attention (p < 0.001), use of information (p ¼ 0.004), use of resources (p ¼ 0.041), Professional attitude (p ¼ 0.025).(B) Mean AE standard error of the total score (repeated measures ANOVA for the main effect for "moment": p ¼ 0.009).ANOVA, analysis of variance; BAT, Behavioral Assessment Tool.

Table 1
General characteristics of the neonatal fellows a Median (range).b 8-hour theoretical-practical courses that are offered by the Neonatal Resuscitation Program of the Brazilian Society of Pediatrics.American Journal of Perinatology Vol.41 No. 8/2024 © 2022.Thieme.All rights reserved.Behavioral Skills after Realistic Simulation Training Guerreiro et al. 1097 This document was downloaded for personal use only.Unauthorized distribution is strictly prohibited.performance score for each of the individual domains after the second training.After this training, there was a decrease in performance until the beginning of the third training, without further improvements.

Table 2
Residents' self-confidence for managing clinical situations related to the neonatal care soon after delivery Before first training After last training p-Value : GA, gestational age.Note: Values are expressed as mean AE standard deviation (minimum-maximum). Abbreviation

Table 3
Differences observed for the main effect "Moment" according to the post hoc Bonferroni's test for each domain of BAT and for the whole scaleBehavioral Skills after Realistic Simulation Training Guerreiro et al. 1099This document was downloaded for personal use only.Unauthorized distribution is strictly prohibited.