In this prospective cohort study of second-year neonatal fellows, a 4.5-points significant improvement in the BAT total score was observed between the initial time prior to any training and the moment after the second training, with no additional gains at the end of the third training. The presence of mild or moderate anxiety was an independent variable, decreasing the behavioral performance of the studied trainees. The self-confidence of neonatal fellows in assisting newborn infants in the delivery room increased after the three training cycles, especially in more complex clinical situations.
Some hypotheses have been raised for the lack of consistent improvement of the behavioral performance of neonatal fellows at the end of the last training. The trainees were not inexperienced in Realistic Simulation before the training because they had already participated in the Neonatal Resuscitation Program of the Brazilian Society of Pediatric. Another issue to be considered is the fact that the third training cycle took place in the last months of fellowship, when most trainees already have a certain degree of fatigue, anxiety, and loss of focus. Also, highlighting problems in the teaching model offered by this simulation program, the repetition of scenarios on the same topic, with only minor changes, may have contributed to the lack of interest in the training. Therefore, a change in our teaching program has been elaborated after the results of this study, offering more diverse and challenging scenarios during the last months of training.
For young adults’ learning, novelty and challenge are important drivers.25 One of adult learning phases is the perception of a gap in their knowledge, and for this to happen, there needs to be a challenge, which can be internal or external, stimulated or not by the facilitator.24 This process may have been minimized by the previous experience with the Neonatal Resuscitation Program and by the repetition of scenarios. It is also worth mentioning the fact that the group of fellows were always the same, and it was possible to observe that some fellows, by the end of the year, were used to work together and were no longer concerned about establishing a communication in a closed loop.
Although the final result of the training was not as expected, it is interesting to observe the curve of acquisition, retention and loss of behavioral skills exhibited by the fellows in this study (Figure 2). There was a downward trend in behavioral performance between the end of each training cycle and the beginning of the next, showing that behavioral skill retention remains only for a few months. Patel et al. evaluated cognitive and technical skills in North American Pediatric residents. The authors showed that retention of technical skills starts to drop after 2-3 months of training.26 Matterson and colleagues showed a reduction in retention of technical skills after 4 months of Realistic Simulation training in neonatal resuscitation.27 Our data reinforce the need for periodic retraining in neonatal resuscitation for healthcare teams responsible for neonatal resuscitation to maintain previously acquired skills.
Regarding the specific behavioral skills, there was significant improvement between the beginning of the training and the end of the second training in the following skills: communication with the team, delegation of tasks, allocation of attention, use of information, use of resources, professional attitude, and the overall score of behavioral performance. Of the four behavioral domains that did not change with training (knowing the work environment, anticipating problems, taking the lead, and asking for help at an appropriate time), the first two may have been influenced by the lack of a longer filming time before the actual scenario. Also, not all scenarios allowed the evaluation of the behavior “request for help". Therefore, in some videos, the evaluators noted this behavior as "not evaluated", reducing the number of videos analyzed and decreasing the sample power. However, the absence of improvement in the leadership performance deserves special attention. The fact that the group of trainees was always the same may have influenced the teamwork, with the observation in some videos of a "shared" leadership, with two trainees providing commands. It was also noted that some fellows started the scenario as leaders, but lost the leadership, and another fellow assumed this role. The same finding was shown by Sawyer et al, in 2014. In their study, Pediatrics and Family Medicine residents received three simulation trainings on neonatal resuscitation and 15 teams of two residents completed all three training over a period of 9 months. A total of 45 resuscitation videos, 15 from each of the three simulation sessions, were reviewed and scored by two blinded reviewers using the BAT. Although BAT global score improved from the first to third session, assumption of leadership did not change throughout the simulation trainings.17
In the present study, approximately 42% of fellows had some degree of anxiety, according to the Beck Anxiety Inventory, and its presence significantly modified the trainees’ performance in the following domains of the BAT: knowledge of the environment, allocation of attention, use of resources, request for help in appropriate time and professional behavior, as well as in the total BAT score. Al-Ghareeb et al. studied 33 nursing students to assess the influence of anxiety on their performance in a Realistic Simulation scenario and students with higher levels of anxiety showed worse technical, cognitive, and behavioral performance.28 The data obtained here and by Al-Ghareeb et al. suggest that attention to the mental health of professionals in training is critical for a full benefit of programs aimed at the acquisition of technical and behavioral skills in areas in which the need for decisions and actions at a rapid and consistent pace is decisive in the prognosis of patients.
In our study, there was an increase in the average confidence of the fellows when comparing the moment before any training in simulation with the moment after all training cycles for all clinical situations evaluated. However, only in situations of higher medical complexity, the difference was significant. This result cannot be attributed only to the experience provided by the realistic simulation training, but it occurred in the context of completing the whole Neonatal Fellowship Program developed in the university hospital. The role of exposure to realistic simulation in the confidence of health professional is controversial. A systematic review with 24 studies, published in 2012, suggested that there is no robust evidence to confirm that high-fidelity realistic simulation improves students' self-confidence.29 However, in another systematic review published in 2019, the self-confidence of nursing students after Realistic Simulation training was evaluated in 29 studies and the results indicated that this teaching technique improved students' self-confidence.30
Overall, this study contributes to building knowledge about the acquisition and retention of behavioral skills of health professionals after realistic simulation training. The cohort design over one year, with three cycles of one month of training, followed by three months of "rest" (Figure 1), allowed us to verify the acquisition and retention of behavioral skills over time and not only the short-term effect of a single training session. It is worth noting the important role of anxiety as a modulator of acquisition and retention of behavioral skills in health professionals in training.
Limitations
Despite these findings, it is necessary to point out some limitations of the study. The sample of fellows was small, from a single center and chosen by convenience, nevertheless with a sample calculation showing sufficient power and with a prospective design that followed the fellows' performance throughout the year. It is important to consider that the video evaluators were present in some scenarios during data collection and knew the fellows. To minimize the bias of the unblinded evaluation, the randomization of the order of the videos analyzed by each evaluator decreased the subjectivity of the evaluation of each fellow individually. Another important limitation was the lack of a systematic assessment, in this study, about suggestions of facilitators and learners on how to optimize the training and avoid the lack of improvement in the third cycle of training. Maybe, by confronting both views, with a qualitative analysis, we could have identified opportunities to improve this learning experience.