Construct Validity of Simulation Model for Training the Ultrasound-Guided Nerve Block and Catheter Placement

Background: Continuous ultrasound-guided peripheral nerve block is a widely practiced technique that can be a challenge for young anesthesiologists. We developed a new simulation model for learning perineural catheter placement. This study aims to test the validity of the model and investigate the learning process of residents on this model. Methods: The simulation model was constructed with partial animal tissue and physical materials. Ultrasound-guided simulation of perineural catheter placement was then performed. Twenty-eight anesthesiologists representing novices, intermediates and experts were required to perform 30 trials within one single day. Practice trials were video-recorded to extract data of task time and performance errors. Construct validity were assessed. Results: Practicing on the simulation model, experts were able to achieve superior task performance, i.e., shorter task time (P < 0.001) and fewer performance errors (P < 0.001), than intermediates and novices. Learning curve recorded from young residents suggested that they were not able to reach the prociency level as experts within one training day. Conclusions: This hybrid simulation model was capable of training the skills required for continuous ultrasound-guided peripheral nerve block. Future studies under this model need to concern extending the duration of the training phase for residents.


Background
Continuous peripheral nerve block techniques have been shown to provide superior analgesia, reduce opioid requirements, and improve functional outcomes after surgery [1]. To achieve long-lasting regional pain control, the anesthetic agents need to be precisely delivered to the selected nerve branch by a needle or a catheter. The outcome of regional anaesthesia is determined by the accuracy of the needle or the catheter placement [2]. Performing regional anaesthesia is not always straightforward as patients may present anatomic variation from case to case. Ultrasound has been increasingly used for guiding peripheral nerve blockade and perineural catheter placement in recent decades [3]. For anesthesiologists in training, the chance of missing target is unfortunately high at the beginning of their career; su cient practices is required before they reach a satisfactory level in performing this vital technique [4]. We believe simulation can play an important role in training the required skills of continuous ultrasoundguided peripheral nerve block.
To date, several physicians and educators have reported their effort of ultrasound-guided peripheral nerve block training using simulators, ranging from low-to high-delity models. Live animals or cadavers can be expensive [5]. Some use synthetic tissues or virtual models to reduce the training cost. However, models may not represent tissue properties and anatomic realities of human patients [6]. Moreover, published reports often merely provide descriptions of the simulator and training protocol [7,8].
Simulation models are used for teaching procedural steps [9], dexterities [10], target identi cation [11], and needle guidance [12]. Only a few have reported the validity tests results from a simulation-based program that is designed for ultrasound-guided peripheral nerve block [4].
We have developed a hybrid simulation model, where partial animal tissue is combined with physical task trainers to create a high-delity but low-cost solution for training the ultrasound-guided peripheral nerve block with emphasis on the perineural catheter placement. We rst provided a description of the simulation model. Then we reported the construct validity of this model for training the ultrasound-guided peripheral nerve block and the perineural catheter placement skills. At last, we presented the learning curve when we applied this model to a group of young anesthesiology residents.
Construct validity is regarded as one of the most important aspects of simulator education as it determines whether the simulator can differentiate between expert and novice for training the appropriate skills [13]. If our hybrid simulation model can be effectively teach ultrasound-guided peripheral nerve block and the perineural catheter placement skills, then experienced anesthesiologist should achieve a signi cant higher score than novices. Speci cally, we hypothesize that our attending physician will complete the task with shorter time and precision than young anesthesiologists in training.

Methods
The study was conducted in the Second A liated Hospital Zhejiang University School of Medicine (SAHZU). Research protocol and methods were reviewed and approved by the Committee on Research Ethics on October 1 st of 2019 (IR 2019-321). Written consent was obtained from each participant before they entered the study.

Participants
Twenty-eight anesthesiologists, varied in three levels of training (novices, intermediates, and experts), were recruited as participants of this study from SAHZU, a tertiary hospital located in Hangzhou of China. The novices (n=14) were freshman residents without any previous experience in performing ultrasoundguided peripheral nerve block. The intermediates (n=7) included residents who had limited experience (2-30 cases) in ultrasound-guided peripheral nerve block. The experts (n=7) included attending anesthesiologists with more than ve years' experience in regional anaesthesia (each over 300 cases of ultrasound-guided peripheral nerve block). A pre-test questionnaire was distributed to obtain demographic data as well as participants' amount of ultrasound-guided peripheral nerve block performed.

Construction of simulation model
A piece of fresh pork belly (10 × 10 × 3 cm) with fascia was purchased from the local grocery store for fabricating the simulator. A silicone tube (0.8 × 15 cm) was lled with ultrasound coupling gel to simulate a blood vessel. Another silicone tube (0.5 × 15 cm) was lled with 5 barium laments and coupling gel to simulate a nerve branch. Both silicone tubes were clamped then placed in parallel on the fascia side of the pork (Fig. 1A, B). The animal tissue was folded to enclose the two silicone tubes. Finally placing the nished model (10 × 5 × 6 cm) onto a surgical table.
Ultrasound Ultrasound data and images were acquired with an ultrasound machine (Sonosite MicroMaxx, SonoSite Inc., Bothell, WA, USA), equipped with a 6-13 MHz linear probe. The depth of tissue detection was set to 3.3cm and the value of gain was adjusted to obtain a perceivably clearer image.

Tasks and procedure
The instructor showed each participant a complete trial on the hybrid simulation model, demonstrating the correct step of placing a regional block needle and catheter (21-G, 100-mm Stimuplex®, B. Braun, Melsungen AG, Germany) using out-of-plane technique. The correct steps included navigating the needle into the correct anatomic location between two silicone tubes and porcine fascia in the target, injecting 1 ml saline until a "donut sign" has been con rmed. Subsequently, the exible catheter was threaded into the target space, followed by withdrawal of the needle and then a bolus of saline. The step were adopted from the recommendation of the Peripheral Nerve Block Ultrasound Training Guideline for continuous nerve block.
After watching the demonstration performed by the instructor, the participants were allowed to make rst attempt when they understood how to use the simulation model. Each participant performed a total of 30 trials, given a 1-minute break after every six trials (one block) to prevent hand fatigue. The performance was recorded by videos and used for assessment.

Assessment
The task time was calculated from the moment when the needle inserting into the model (start time) to the moment of injecting saline through the catheter under ultrasound (end time). A trial was successful if the trainees were able to correctly place the nerve block catheter between the silicone tubes and porcine fascia. The attempt was considered a failure if the needle punctured the silicone tube, if the nerve block catheter reached the space outside of the fascia, or if the trainee failed to complete the task within the allotted time of 120 seconds.
Quality-compromising behaviors (QCBs) of each trial was marked for counting the performance errors.

Results
Ultrasonographic image of Simulation model When ultrasound was applied to the simulation model, silicone tubes and barium laments provided an echo image similar to that of an artery and a nerve branch ( Fig. 2A). Each performer was required to move the probe along the short axis of silicone tubes to nd the tip of the needle (Fig. 2B). After rapid injecting of 5 ml normal saline, an anechoic region of uid around the needle tip (a.k.a. the donut sign) was seen (Fig. 2C). A nerve block catheter could be inserted through the needle to the correct position near the target nerve (Fig. 2D).
Descriptive Analysis Table 2 reports demographic data for each participant, including age, gender, previous experience, and con dence of ultrasound-guided procedures (self-reported). Previous cases on regional anesthesia 0 18 ± 16 368 ± 176 Con dence in ultrasound # Low Low High # Self-report of con dence in ultrasound-guided procedures performed Before performing the quantitative analysis, we would like rst present the novices' task completion time aggregation to demostrate their learning curve over 30 practice trials (Figure 3). We noticed that novices clearly improved their performance over time, as revealed in the reduction in task duration. It was shown learning becomes more consistent over 28 repetitions for novices. However, there was a apparent time discrepancy between novices and experts, showing that novices could not reach the experts' skill level after 30 practice trials, a full day of training.

Quantitative Analysis on Task Time
Outputs from 3 x 5 Mixed ANOVA on task time were displayed in Figure 4. Task time displayed a signi cant difference (P < 0.001) among different anesthesiologist groups (Figure 4, left). Post-hoc analysis revealed that differences were presented between experts and novices (P < 0.001), experts and intermediates (P < 0.001), but not between novices and intermediates (P = 0.164). ANOVA also disclosed a signi cant difference (P < 0.001) on the training blocks (Figure 4 right). However, we did not found any interaction between the groups and training blocks (P = 0.768, Table 3 bottom). There was a total of 828 QCBs marked by the supervisors in 840 trials done by the 21 residents and 7 attendings. The common errors included poor catheter placement (QCB7, 280 times), failure to recognize catheter tip (QCB5, 174 times), needle tip advancement without ultrasound visualization (QCB1, 128 times), malposition of the target on the screen (QCB4, 118 times), improper depth of needle tip advancement (QCB2, 92 times), accidentally pricking "vessel" and "nerve" (QCB3, 26 times), and inappropriate focus of attention on hands (QCB8, 10 times). None of the residents showed poor needle handling (QCB6, 0 times).
A signi cant difference of performance errors between the three anesthesiologist group was found (P < 0.001, Figure 5 left), speci cally, differences were presented between experts and novices (P < 0.001), experts and intermediates (P < 0.001), but not between novices and intermediates (P = 0.327). The difference between ve training blocks was examined and it showed that practice signi cantly reduced errors recorded from all participants (P < 0.001, Figure 5 right).

Discussion
In this study, we successfully constructed a hybrid, high-delity but low-cost simulator for training required skills required for performing the ultrasound-guided peripheral nerve block with an emphasis on the perineural catheter placement procedure. The hybrid model was made partially with a piece of pork, emulating tactile feedback similar to human tissues. By carefully selecting materials enclosed in the tissue, it was possible to obtain visual feedback very similar to human arteries and nerve branches when scanned by ultrasound. This custom designed model ts the de nition of high-delity because it provides a high level of tactile and visual impression for ultrasound-guided peripheral nerve block as performed in patients [14].
The rst goal of this project was to prove the model can teach the skills and ability faithfully, i.e., construct validity. Evidences presented in Figure 4 and Figure 5 demonstrates that the model is able to separate anesthesiologists with different levels of training. That means residents practicing with this model should gradually build up skills towards experts.
Besides its low cost, our model has the advantage of providing close-to-real ultrasound imaging on detailed anatomy, including blood vessels, nerves, and the donut sign after injecting saline to the anaesthesia site. Barium laments are taken from surgical sponges, where their original purpose is for xray detection. When placed into a silicone tube and lled with gel, which creates a comparable image of nerve branches. Filling silicone tubes with gel improves the ultrasonic images without obvious acoustic shadowing, which often impedes to observation of needle tip positioning, affecting precise dose control of local anesthetics to the target nerve. The silicone tube has elastic properties similar to blood vessels, superior than the transfusion tube used in Scott's model [15].
A potential bene t of using simulation for residents to learn the ultrasound-guided peripheral nerve block skills is the stress-free environment for skill honing. Residents claim that they feel less stress and fatigue when practicing using simulator. Overload stresses and mental fatigue are problems for learning skills of regional anaesthesia [16]. Although the trainees were given time for short rests in-between our study, many of them felt they did not need to take a break between training sessions. The increased engagement to simulator practice was observed in our groups of residents.
The second goal of this project was to describe the learning curve of residents when practicing on our model. Figure 1 has shown novices clearly improved their performance over time. However, novices were not reaching to the skill level of experts. We believe the main reason was the setting of training protocol. In current study, residents were asked to perform 30 trials within one day. This was a limitation. We may observe an improved learning outcome if the training was extended over several days.
Another limitation of this study was the selection of intermediate group.
In the current setting, the training level of residents in the intermediate group was near to novices'; therefore, we were not able to record a clear skills discrepancy between these two groups. If we recruited more senior residents, we might collect stronger evidence to support construct validity. The third limitation was the setting of QCB criteria. In this study, we did not include some undesirable actions into QCBs, such as the catheter extending beyond the fascia behind the "nerve", and re-entering the space around the nerve after entering the muscles. Additionally, we did not blind trainees to the evaluators when counting errors; potential bias might exist. Measures such as obfuscating the face of trainees on the videos will help to enhance the quality of data in our future study.

Conclusions
We presented a hybrid simulation model made by partial animal tissue and physical materials that provided high-delity for training continuous ultrasound guidance nerve block skills. Evidence for the construct validity of using this model for training the ultrasound-guided peripheral nerve block skills was discernible from measures of task time and performance errors. Learning curve recorded from young residents practicing over 30 trials suggested that they were not able to reach the pro ciency level as experts within one training day. Future studies under this model will be needed to optimize the ultrasoundguided peripheral nerve block training protocol by extending the duration of the training phase for residents.
Abbreviations QCB: Quality-compromising behaviors, SPSS: Statistical package for social sciences Declarations Ethics approval and consent to participate   Statistical outputs on the interaction between training group and practice trial (Left: task time, Right: performance errors)