Assessment of Novice Learners in a Novel High-delity Simulation-Based Extracorporeal Membrane Oxygenation Education Program

Background: This study aimed to design a suitable integrated extracorporeal membrane oxygenation (ECMO) curriculum and assessed the impact of this a high-delity simulation-based education module on novice learners. Methods: An ECMO training curriculum was developed by well-trained and experienced ECMO experts that incorporated simulation modules to train multidisciplinary health care professionals. Pre- and post-participation questionnaires were used to determine the effects on the knowledge, ability, and condence level of the participants. A ve-point Likert scale was used to assess the participants regarding the ECMO practical test. Results: The study enrolled 37 participants (10 doctors, 27nurses). 67.6% (25/37) of trainees didn’t have ECMO experience. All two questionnaires were completed by 37 participants. A ve-point Likert scale on practical test was nished by 4 ECMO experts. All participants passed the written and practical tests. All the responding participants thought the curriculum was useful in improving their perception of their overall knowledge and their ability to perform the required critical performance criteria on simulated ECMO. 91.9% (34/37) trainees thought that they invested more in active learning during the whole course; 60% (22/37) of them achieved a good level of technical evaluation. On the other hand, 67.6% (25/37) of the trainees have shown a good self-condence in the priming course; however, 43.2% (16/27) trainees still needed to strengthen their overall competence in ECMO management. Conclusions: The integration of traditional teaching and high-delity simulation teaching can effectively improve the ECMO monitoring level and management skills of novice learners, but whether this training mode can be successfully transferred to the clinical eld needs further research and conrmation.


Introduction
As the last effective treatment for life-threatening severe acute respiratory distress syndrome (ARDS) patients, extracorporeal membrane oxygenation (ECMO) is the most complex and di cult technology, and demands the most delicate management and monitoring in the intensive care unit (ICU), especially during the COVID-19 pandemic. To establish a standard and effective ECMO program and make health care professionals to master this technology, we developed a novel ECMO training curriculum integrating traditional teaching and high-delity simulation to train multidisciplinary novice nurses and doctors at the participants were planned to be divided into 3 groups (12-13 per group), and each group received the 4day identical training course. It would be taken 12 days for all trainees to complete the training 2.Training equipment The major training equipment included 2 sets of ECMO equipment: PLS set system (MAQUET, Germany) and HLS Module Advanced system (MAQUET, Germany)), high-delity simulation system (CALIFIA simulation system (Biomed Simulation, Inc., USA)).

3.Course design
According to the "ELSO guidelines for training and continuing education" framework [1] ,, the curriculum consisted of didactic course, priming ECMO circuit, water-drills practice, and emergency ECMO scenarios performed with high-delity simulation system. (Figure 1).
The ECMO training course was designed over a 4-day period. The rst day was didactic lectures, followed by a full day of priming training, simulation training took place at the third day. And the practical examination was arranged on the last day.

Didactics
According to the "ELSO guidelines for training and continuing education" framework [1] , purpose-designed basic ECMO theory focused on the following content: the characteristics of pathophysiology and hemodynamics in severe ARDS patients with ECMO, cannulation, indications and contraindications for severe ARDS patients , management and weaning circuit.

Construction and development high-delity simulation scenarios
The platform for simulated situational teaching was constructed at the simulated ICU lab in the clinical teaching center of the First A liated Hospital, SYSU. By remodeling an adult cardio-pulmonary resuscitation (CPR) model, a mannequin model, a ECMO and venovenous ECMO circuit, the "patient module" of the CAFILIA system, and the CAFILIA simulation operating system were connected with a monitoring instrument to construct the ECMO simulation. In order to re ect immediate changes to vital parameters for the ECMO scenarios, the simulator system was operated remotely via CAFILIA central control console and completed of setting preprogrammed. The vital signs and lab data were displayed on a monitor which have been preprogrammed already and in accordance with the given scenario.
To make the scenarios more authentic and optimize the clinical experience , ve scenarios were designed by the experienced and well-trained ECMO experts (Supplemental table 1). The team or the individual trainee should notice the abnormal vital sign and lab data from the monitor, check the human model and ECMO circuit, the ECMO machine also should be take into consideration. Simulation scenario was controlled by the clinical teaching center colleagues and was synchronized with the interventions. Also, we had a medical volunteer assist with any practical aspects and provide additional information for the context of the scene. The volunteer received a half-day of training before the training course started.
12-13 participants were divided into 4 teams, with at least 1 physician in each team. Extensive debrie ng followed each scenario.

4.Assessment and Evaluation
At the end of the course, the participants were required to past a 60min written test. This test was comprised of 50 multiple choice questions (MCQ) based on the didactic course. In the practical test, each team selected a simulated scenario randomly from Supplemental table 1 and the whole exam process was videotaped.
Four ECMO well-trained and experienced ICU experts judged the performance for each participant during the practical test with predesigned speci c trigger events and expected interventions. These four experts were blinded to each other's judgements. They used a 5-point Likert scale to assess the participants' performance. The score for each evaluation item ranged from 1 to 5, with 1 being the lowest and 5 being the best; the total score was 50 (table 2).
Pre-simulation test questionnaire was an evaluation of trainees for the authenticity of the simulation circumstance (Supplemental Table 2), and post-simulation test questionnaire was formulated based on ELSO's training guidelines and the teaching objectives of the course, which were also the trainees' selfjudgments (Supplemental Table 3). Each question ranged from 1 to 10, with 1 being the poorest and 10 being the best.
The 5-point Likert scale and the questionnaires were formulated based on the "Guidelines for training and continuing education" [1] and "The Mayo High Performance Teamwork Scale" [2] .

Teaching Objective
The primary outcome of this study was to evaluate whether integration both traditional teaching and high delity simulation teaching can improve the technical and non-technical skills of novice health workers , enable them to accurately recognize the emergencies , and solve the problems effectively.
The secondary objective was to determine if the training can improve novices' con dence and nontechnical behaviors when facing complicated ECMO management.
Statistical analysis SPSS (IBM Statistical Package for Social Sciences v21.0; Chicago) was used for the statistical analysis. Continuous variables are expressed as the median and interquartile ranges (IQR 25-75%). Discrete variables are expressed as percentages.

1.Participants' characteristics
A total of 38 medical professionals participated in the training course. Except one nurse did not nish the course for unknown reason, the other 37 completed, of whom 10 were doctors and 27 were nurses. Most of the trainees came from emergency, general surgery, ICU and anesthesiology departments, aged between 27 to 36 years (mean, 29 years). 17% of the doctors had more than 10 years of clinical experience, and 44.5% of the nurses had 5-10 years of clinical experience. Of all the trainees, 67.6% without any previous experience about ECMO management. For their expectations of the course, 1 (1/37, 2.7%, a nurse) hoped to learn the basic principles of ECMO; 9 (9/37, 24.3%) wanted to master priming procedure, and 27 (27/37, 73%) hoped to completely grasp ECMO priming, management, and monitoring ( Table 1).

2.Evaluation of ECMO didactic course
The ECMO theory assessment was a written test. All trainees passed it, with the average score of 90 (range, 75 to 95).

3.Evaluation of the ECMO teaching courses
3.1Evaluation of simulation scenarios' authenticity by trainees As shown in Supplemental Table2, The trainees evaluated the authenticity of the simulations regarding four aspects: the constructed space for the simulations, equipment (ECMO, circuit connection, monitor screen, mechanical ventilation setting, and drug pump), development of clinical scenario, and teacher demonstration. The rating ranged from "poor" to "medium", "good", and "excellent". Most trainees rated the authenticity of the simulations as excellent; only 1 trainee regarded the "authenticity of the human model" as medium, and 3 trainees regarded the "authenticity of the simulated medications" as medium.
For the teacher demonstration, only 1 trainee regarded the demonstration as not very useful.

Evaluation of the trainees' simulation test by experts
The evaluation of the trainees' simulation test included two aspects (table2): a technical evaluation and a behavioral (non-technical) evaluation, with a total of 10 items and a total score of 50.
Regarding the technical evaluation (Table 2), the technical evaluation results showed that 60% (22/37) of the trainees can reach a good level; 59.5% (22/37) of them can give good suggestions for optimizing the division of labor during the priming process, and can complete their own priming tasks; after a rapid assessment of the clinical environment, 73% (27/37) of the participants can respond quickly and correctly; in addition, 83.8% (31/37) of them can make a pre-judgment in an emergency situation and can give solutions; at the same time, 70.3% (26/37) of the subjects were good at nding problems and can ask questions effectively.
For the non-technical evaluation (Table 2), trainees' team leadership abilities needed improvement. Over half (20/37, 54.1%) just reached the standard level. If there was an emergent situation, 40.5% of the trainees were easily distracted by alarms and other environmental factors, and could not effectively allocate their attention. Regarding the use of available human resources, there was not much difference between those who just reached the standard (17/37, 45.9%) and those who performed well (20/37, 54.1%).

Simulation training feedback by trainees
34 trainees (91.9%) agreed that they took initiative learning more during the entire simulation training.
In Supplemental Table 3,54% (16/37) trainees believed that they can independently complete the monitoring tasks during the ECMO operation (score ≥ 8 points); 51.3% (19/37) of them believed that they were better at predicting and handling emergency situations Completion (score ≥ 8 points). In terms of overall competence, although all trainees thought they could compete with the ECMO technology, most of them gave themselves a low evaluation score. 43.2% scored at 6, only 1 gave a score of 9, and none gave a score of 10.

Discussion
In this study, we developed a 4-day training integration course that combined traditional teaching and high-delity simulations. According to results of the post-training questionnaires and experts' evaluations, the designed curriculum was successful. In addition, we believe that it is feasible to improve novices' ECMO technical and behavioral skills through integration courses within 4-day in the intensive care environment.
In this study, 60% trainees reached the "good" technical level. When met the ECMO emergency environment, more than 70% participants could judge it accurately and solve the problem rapidly. ECMO, as the most di cult extracorporeal life support technology, has high requirements for health care professionals and strict standardizes for the training courses' quality. Therefore, the integration teaching mode is the rst choice for ECMO training.
To optimize the training course, our ECMO curriculum integrated with didactic, water-drills, teacher practice demonstration, simulation training, and case joint drills, from easy to di cult, from simple to complex, by individual to the team cooperation. Hope to make the novice learners grasp theoretical knowledge and improve their technical skills.
ECMO training has its own characteristics. It is totally different from other extracorporeal life support training. It requires learners to carry out a lot of practical operation training and repeated practice. Therefore, this curriculum decomposed water-drills and high-delity simulation training separately, and participants were assigned from individual to teamwork to ensure that they had a chance to receive a large number of practical exercises during the whole course. The preliminary results of the course showed that this method of decomposition training helped novice learners to overcome the fear of di culty gradually and make them invest more active learning. Therefore, after the training, about 92% of the trainees were able to take a proactive approach and complete the training.
The key elements for the authenticity of high-delity simulation teaching is multi-dimensional construction for the scenes, supplemented by a highly simulated simulation scenario teaching plan which is closely integrated with clinical practice. And that can make the ECMO training of novice health care professionals more e cient.
In addition, the course has another key component: water drills and the design of a simulation teaching program. All the cases were drafted and designed after discussion by doctors who have senior professional titles and at least 5 years of independent management experience with ECMO cases. Hence, the course can provide high-quality simulated teaching scenarios that are related to the ECMO emergencies in ICU. These novel training mode and simulation curriculum can provide enough opportunities to the learners which they can apply new knowledge and use their past clinical experiences to solve problems by repeating a large number of practical exercises.
The authenticity and practicality of training cases are the essence of ECMO training courses. High-delity simulation training is an ideal platform for developing ECMO training [3] . In this study, the simulation course developed scenarios through various aspects including cases, environment (ICU setting), medicine, equipment (ventilator, monitor, blood gas test), speci c mannequin, etc., so that novice learners can be as immersive as possible. To our satisfaction, the trainees, in general, regarded the authenticity of the constructed scenes for clinical cases as "good", and some regarded them as "excellent" (Supplemental Table 2).
Anderson et al. [4] found that during ECMO emergencies, trainees who had received simulation training made fewer mistakes. Similarly, Burkhart et al. [5] showed that simulation training improved the con dence of trainees in using ECMO. The results of this study are in agreement with previous studies. We found that the trainees who took our training course improved regarding both technical and nontechnical skills. The latter evaluation showed that the sense of teamwork increased; 63% of the trainees could effectively communicate with each other and deal with clinical situations through asking for assistance. Besides, the self-con dence and competence of the trainees have been continuously improved in the end of the course.

Limitations In The Design Of The Course
Although we proved that through this integration training course, beginners can effectively acquire knowledge and improve self-con dence, the time restraint prevented us from setting up training applying clinical practice. Although after completing the training, 1 trainee participated in a case of ECMO rescue in ICU, and performed priming and managed the case independently for over 24 hours, the actual training effects of the remaining 36 trainees still require long-term follow-up.

Conclusion
Using professional knowledge of medical education and critical care medicine, we successfully designed and provided a 4-day ECMO course. Our results showed that traditional teaching integrated with novel high-delity simulations improved the professional knowledge, con dence, and teamwork cooperation of the novice health worker professionals. Whether this training method can be transferred to clinical eld needs validation from further research. The study protocol and study methodology was approved by ethic committees in the First A liated Hospital of Sun Yat-sen University (GUANGZHOU, CHINA). All participants gave a written informed consent.

Consent for publication
The manuscript has been read and approved by all the authors, and each author believes that the manuscript represents honest work.
Availability of data materials The data described in this Data note see Supplemental Tables.

Competing interests
None of the authors has any nancial or non nancial competing interests to declare.

Funding
Not applicable.
Authors' contributions L TONG and WJ WH were involved in the writing of the manuscript and in the interpretation of the results.
YP ZHU and L CAO Participated in data acquisition and statistical analysis. YH LI and N ZHANG were involved in the data collection process and the study coordination. CJ CHANG and M KUANG Participated in study conception and design, data interpretation, and critical revisions of the manuscript.  CCSupplementaltables.docx