The Impact of the Fidelity of Simulation on Medical Undergraduate Education: A Meta-Analysis


 Background: With the development of science and technology, simulation-based education has also developed rapidly. However, whether the fidelity level of simulators has a positive correlation with medical students' learning outcomes is controversial. This study aims to compare the theoretical knowledge, skill performance and confidence of undergraduate medical students through meta-analysis according to the fidelity level of the simulator.Methods: Two researchers independently searched the PubMed database, the Cochrane Library, and the Embase database through October 20, 2020, to retrieve articles on the differences in effectiveness between high-fidelity simulators and low-fidelity simulators in undergraduate medical education. The Cochrane risk of bias tool was used to evaluate all included literature. Quantitative meta-analysis of the included literature was performed with Review Manager 5.3.Results: Fifteen studies met the inclusion criteria, 11 of which were meta-analysed. Meta-analysis showed whether there were differences in students’ theoretical knowledge [standardized mean difference -0.51; 95% CI -1.30～0.29，P=0.21], skill performance [standardized mean difference -0.26; 95% CI -0.87～0.35, P = 0.40], and confidence [standardized mean difference 2.53; 95% CI -1.05～6.10, P = 0.17]: there were no significant differences between high-fidelity simulators and low-fidelity simulators.Conclusions: In medical undergraduate education, students who experience low-fidelity simulator training are not inferior to students who learn from high-fidelity simulators in their theoretical knowledge, skill performance, or confidence.


Background
Since its birth in the 1960s and with the rapid development of human model simulators, simulation-based teaching has been integrated into most clinical courses [1]. Moreover, many experiments have fully demonstrated the positive effects of simulation-based theoretical knowledge and clinical skills [2,3]. Lack of clinical practice is a common problem in undergraduate medical education. Training and teaching based on simulation can provide medical students with practical application experience. Training and teaching based on simulation is an ideal teaching mode to provide medical students with theoretical knowledge and hands-on practice by constructing simulation environments instead of real scenes; this approach is considered to have great development potential.
With the development of modern technology and advances in synthetic materials technology, current simulators are able to provide a very realistic environment, reproducing realistic changes and providing feedback. High-delity simulation (HFS) can be used for training and to immerse users in complex and realistic scenarios by providing realistic feedback. However, some simulators with limited functions can only provide a speci c simulation environment and cannot provide all realistic feedback; this is called low-delity simulation (LFS).
From personal experience, it seems that there is a positive correlation between the delity of the simulator and the learning performance of medical students, but several studies have found that LFS is no less effective at improving knowledge and skills than HFS [4]. Studies have found that compared with LFS, HFS can not only fail to better than LFS in improve students' abilities in terms of knowledge and skills but can also cause them to have blind con dence and seriously overestimate their abilities. This is an undesirable outcome because one of the most common cognitive biases that leads to clinical diagnosis errors is overcon dence [5]. Therefore, to explore the effect of simulator delity on undergraduate medical education, a meta-analysis based on existing studies is necessary.
The purpose of this meta analysis was to objectively evaluate the impact of using high-delity simulators or low-delity simulators on the theoretical knowledge, skill performance and con dence of undergraduate medical students.

Search strategy
Literature retrieval: Two researchers (Y.H. and X.C.) independently searched PubMed, the Cochrane Library, and Embase online and collected randomized controlled studies on HFS and LFS in medical undergraduate education; the date range was January 1, 1995, to October 20, 2020. The online search was supplemented by a manual search and follow-up search, and the authors were asked for the full text and original data. Search keywords: ("high patient simulators" or "high delity simulation") and ("low delity simulation" or "static" or "low patient simulators) and ("medical education" or "undergraduate education").

Data screening and extraction
Inclusion criteria: 1. Type of study: randomized controlled study; 2. Research objects: Students receiving undergraduate medical education; 3. Intervention measures: A high-delity simulator was used in the experimental group and a low-delity simulator was used in the control group. Speci c information on the simulator was mentioned in the paper, and the sizes of the experimental group and control group were clear. 4. Outcome indicators: theoretical knowledge, skill performance and con dence.
Study selection: All possible eligible study titles were screened by two independent reviewers (Y.H. and X.C.), not excluding abstracts and full text. After ltering based on the titles of the articles, both reviewers reviewed the remaining papers and identi ed articles that met the inclusion criteria. Differences between the two reviewers were resolved by discussion or by arbitration by the third investigator. We de ne the delity of the simulator as the physical properties of the simulator. High-delity simulators are "those that provide physical examination, display vital signs, physiological responses, intervention (through a computer interface) and allow certain operations to be performed on them (endotracheal intubation, intravenous intubation, face mask, etc.)." Low-delity simulators, on the other hand, are "static models that are otherwise limited by these capabilities" [6]. Out of 4,568 potentially relevant articles, the selection based on titles and abstracts resulted in 270 relevant studies. After reviewing the papers, 36 studies were retained. The 36 full-text articles were systematically reviewed to con rm their eligibility (Fig. 1).
Grading the evidence Data collection: two independent reviewers (Y.H. and X.C.) used spreadsheet data extraction to extract the results of the randomized controlled trials. The evidence was evaluated according to the Cochrane bias risk tool, and discussion or arbitration by a third researcher was used to settle differences in the evaluation. If the articles reported uncertain data or had missing data, the author was contacted to obtain the missing details so that enough original data could be obtained for the meta-analysis.

Endpoints
The main end point is to determine that low-delity simulators are not inferior to high-delity simulators in terms of the theoretical knowledge and skill performance of medical undergraduate students. The secondary endpoint is to determine whether there is a difference in the con dence of participants between those taught with low-delity simulators and those taught with high-delity simulators.

Statistical analyses
The data were analysed using a combination of quantitative and qualitative evidence. Results from tests that occurred at similar time points (such as at the end of a performance skills course) were reported using the standardized mean difference (SMD) to allow direct comparison [7]. We used random effects meta-analysis to quantify the results of more than one study. The SMD, 95% con dence interval and statistical heterogeneity were calculated by inputting the data into Review Manager 5.3 (Nordic Cochrane Centre, The Cochrane Collaboration, Copenhagen, Denmark). We quanti ed the inconsistencies between studies, and for the analysis of three or more studies, using I 2 statistics, we determined the percentage of variability due to chance. I 2 > 50% indicates the existence of high inconsistency or heterogeneity, so a random effects model was chosen [7]. Funnel plot was used to observe publication bias.

Results
Using the above search strategy, we identi ed 4,568 potentially relevant studies. Fifteen of the studies met the criteria. The characteristics of the 15 studies included in this meta-analysis are listed in Table 1. Random assignment was used in all the studies. Thirteen studies compared high-delity simulators with low-delity simulators, and two studies compared high-delity simulators with static simulators. Figure 2 summarizes the risk of bias assessment. Eight studies did not describe the method of random allocation [8][9][10][11][12][13][14][15]. Only three studies detailed the generation of allocation. Four studies [5,16,17] used a blinded method, while one study [5,[16][17][18] clearly indicated that no blinding was used [19]. Six studies blinded the evaluators [5, 8-10, 17, 18]; participants in one study were randomly assigned, but the researchers who assessed their scores were not [20]. One study had risk of bias due to missing results [13]. There was a low risk of publication bias in all the studies. Visual inspection of the funnel plots indicated asymmetry. The asymmetry of the funnel plots may be due to an insu cient number of trials (leading to a sma effect) and signi cant statistical heterogeneity.

Outcomes
Six studies [5,8,10,14,20,21] quantitatively compared the mastery of theoretical knowledge of undergraduate medical students exposed to HFS and LFS.
There was high heterogeneity among the studies (I 2 = 93%, P < 0.00001), and a random effects model was used. The results showed that there was no signi cant relation between the delity of the simulator and the mastery of theoretical knowledge of medical undergraduates [WMD and its 95% CI were − 0.51 (-1.30 ~ 0.29), P = 0.21] (Fig. 3). King and Reising [12] compared the effectiveness of static simulation and HFS in the teaching of advanced heart life support guidelines. The results showed that there was no signi cant difference in theoretical knowledge between the static simulation group and the HFS group (P = 0.1455, a = 0.05). It seems that the funnel plot is asymmetrical (Fig. 4).
Four studies [8,10,18,20] quantitatively compared the professional skill performance of undergraduate medical students taught via HFS and LFS. There was high heterogeneity among the studies (I 2 = 71%, P = 0.02), and a random effects model was used. The results showed that there was no signi cant relation between the delity degree of the simulator and the degree of professional skills mastery of the medical undergraduates [WMD and its 95% CI were − 0.26 (-0.87 ~ 0.35), P = 0.40] (Fig. 5). Four studies obtained positive results: Banaszek et al. [16], King et al. [12], Mills et al. [13], and Moccy et al. [19] The results showed that the clinical skills performance of students in the HFS group after receiving HFS teaching was signi cantly different from that of students in the LFS group (P < 0.05). The funnel plot is symmetric (Fig. 6).
Three studies [11,15,18] quantitatively compared participants' con dence after receiving HFS and LFS teaching and training. There was high heterogeneity among the studies (I 2 = 98%, P < 0.00001), and a random effects model was used. The results showed that there was no statistically signi cant effect of simulator delity on the con dence of medical undergraduates [WMD and its 95% CI were 2.53 (-1.05 ~ 6.10), P = 0.17] (Fig. 7). The funnel plot is asymmetrical (Fig. 8).

Discussion
This meta-analysis provides data for evidence-based education by comprehensively analysing undergraduate medical education under different backgrounds and types of simulation. In this study, it was found that there was no signi cant difference between HFS and LFS in terms of students' theoretical knowledge, clinical skills or improved con dence.
Both high-delity and low-delity simulators can improve students' theoretical knowledge and clinical skills [20]. There is no difference in theoretical knowledge and clinical skills, and similar results have been reported in many studies [8,20,21]. These results may suggest that the teaching effect of LFS can be equivalent to that of HFS in medical undergraduate education. However, some studies have found that high-delity simulators are superior to low-delity simulators in improving students' clinical skills [12,14].
In terms of economy, the cost and price of high-delity simulators are much higher than those of low-delity simulators [22]. This leads to the low cost performance of high-delity simulators in medical undergraduate education. Second, the majority of students from both groups had strong positive expectations of the value of HFS. Before the course, only the majority of the HFS group adhered to this belief, while many participants in the LFS group changed their views and did not consider LFS training to be inferior [19]. This suggested that LFS training did not discourage participants but rather made them more con dent.
As the simulation level increases, the cognitive burden of inexperienced students also increases, and the complexity of the working environment will distract students' attention, leading to low learning e ciency and even lack of knowledge [23]. Some literature also suggests that students will feel pressured by highdelity simulators because of the highly simulated environment they create. However, students who have basic knowledge of clinical skills can re ne their performance by entering the "deep" simulated environment of high-delity simulators [13]. It is completely feasible to conduct low-edlity simulations for students with little experience [21]. This has great educational value.
The limitations of this paper are as follows. First, the participants in our included studies were undergraduate medical students whose speci c training and education levels may have in uenced the outcomes in a way that is different from the way medical professionals are assessed. Second, the high heterogeneity of this study may be due to the heterogeneity of the intervention measures and measurement schemes across the included studies. This article includes only research published in English. There are too few studies based on quantitative analysis, and the funnel plots seem asymmetric. Therefore, more clinical studies are needed to determine the relation between the delity of the simulator and medical undergraduate learning outcomes.

Conclusions
According to the results of this study, there is no positive relation between education outcomes and the delity of the simulator. This nding may be associated with the education level of our sample, and whether it is replicable in professional medical personnel needs further research. Forest plot of the in uence of HFS and LFS on the skill performance.

Figure 7
Forest plot of the in uence of HFS and LFS on the con dence.

Supplementary Files
This is a list of supplementary les associated with this preprint. Click to download. PRISMA2009checklist.doc