Study characteristics
A total of 2053 related articles were obtained during the initial inspection, and after a layer-by-layer screening, 21 RCTs were finally included [11-31], which included 1683 nursing students. The literature screening process and results are shown in Figure 1. The publication years of the included literature ranges from 2006 to 2020, and five of the studies are conducted in Turkey, five in Jordan, two in the UK, two in China, and two in Singapore. The United States, Brazil, France, Portugal, and Japan have only one article each. The research objects are mainly nursing students from different grades, mainly students in the third grade whose ages range from 10.29 years to 33.00 years. The sample sizes of the studies range from 31 to 146. The detailed information of the 21 studies is shown in Table 1.
Table 1. Characteristics of the included studies.
Study
|
Country
|
Design
|
Comparison
|
Samples
|
Sample source
|
Age
(EG/CG)
|
Kahraman et al., 2019
|
Turkey
|
quasiexperimental
|
Pre-intervention test vs.Post intervention test / simulation-based training group vs.standard training group
|
72
|
third-year nursing students
|
21.91±0.86
|
Basak et al., 2019
|
Turkey
|
RCT
|
simulation-based training group vs.standard training group
|
71
|
second-year nursing students
|
19.50±0.57
|
Tamaki et al., 2019
|
Japan
|
RCT
|
Pre-intervention test vs.Post intervention test / simulation-based training group vs.standard training group
|
38
|
third-year nursing students
|
21.30±0.62/21.00±0.00
|
Pamela et al., 2013
|
America
|
pretest-posttest two-group randomized experimental
|
Pre-intervention test vs.Post intervention test / simulation-based training group vs.standard training group
|
79
|
nursing students
|
/
|
Alinier et al ., 2006
|
UK
|
pretest-posttest two-group randomized experimental
|
Pre-intervention test vs.Post intervention test / simulation-based training group vs.standard training group
|
99
|
second-year nursing students
|
29.3±7.5/33.0±8.4
|
JU et al., 2019
|
China
|
RCT
|
Pre-intervention test vs.Post intervention test / simulation-based training group vs.standard training group
|
132
|
first-year nursing students
|
/
|
Stayt et al., 2015
|
UK
|
two centre phase II single RCT
|
Pre-intervention test vs.Post intervention test / simulation-based training group vs.standard training group
|
98
|
nursing students
|
26.3±7.0/29.5±8.4
|
José et al., 2019
|
Portugal
|
RCT
|
Pre-intervention test vs.Post intervention test / simulation-based training group vs.standard training group
|
42
|
nursing students
|
19.29±0.46/20.29±2.19
|
AlAmrani et al., 2017
|
Turkey
|
RCT
|
Pre-intervention test vs.Post intervention test / simulation-based training group vs.standard training group
|
72
|
first-year nursing students
|
19.69±1.23/20.22±2.17
|
Loai, 2020
|
Jordan
|
RCT(pre-test-post-test)
|
Pre-intervention test vs.Post intervention test / simulation-based training group vs.standard training group
|
76
|
nursing students
|
20.36±0.54/20.47±0.55
|
Aloush, 2019
|
Jordan
|
two-arm RCT with pre-post tests
|
Pre-intervention test vs.Post intervention test / simulation-based training group vs.standard training group
|
131
|
fourth-year nursing students
|
21.0±2.1
|
Loai, 2016
|
Jordan
|
pretest-posttest RCT
|
Pre-intervention test vs.Post intervention test / simulation-based training group vs.standard training group
|
84
|
nursing students
|
19.60±1.02/19.10±1.05
|
Ahmad et al., 2015
|
Jordan
|
pretest-posttest RCT
|
Pre-intervention test vs.Post intervention test / simulation-based training group vs.standard training group
|
91
|
nursing students
|
20.40±0.98
|
Ahmad et al., 2014
|
Jordan
|
two-arm RCT
|
Pre-intervention test vs.Post intervention test / simulation-based training group vs.standard training group
|
90
|
nursing students
|
20.1±1.9/19.6±1.6
|
Banu et al., 2019
|
Turkey
|
RCT
|
Pre-intervention test vs.Post intervention test / simulation-based training group vs.standard training group
|
59
|
first-year nursing students
|
/
|
Ayse et al.,2019
|
Turkey
|
mixed-methods
|
Pre-intervention test vs.Post intervention test / simulation-based training group vs.standard training group
|
107
|
third-year nursing students
|
20.77±1.22
|
Li et al., 2019
|
China
|
pretest-posttest two-group randomized experimental
|
Pre-intervention test vs.Post intervention test / simulation-based training group vs.standard training group
|
100
|
nursing students
|
20.60±0.78/20.2±1.00
|
Sok et al., 2011
|
Singapore
|
pretest-posttest RCT
|
Pre-intervention test vs.Post intervention test / simulation-based training group vs.standard training group
|
31
|
third-year nursing students
|
21.45±1.55
|
Sok et al., 2012
|
Singapore
|
pretest-posttest RCT
|
Pre-intervention test vs.Post intervention test / simulation-based training group vs.standard training group
|
31
|
third-year nursing students
|
21.47±1.43/21.44±1.41
|
Antonia et al.,2020
|
France
|
prospective multicenter study
|
simulation-based training group vs.standard training group
|
146
|
second-year nursing students
|
24.0±6.4/25.0±6.5
|
Raphael et al., 2020
|
Brazil
|
pretest-posttest RCT
|
Pre-intervention test vs.Post intervention test / simulation-based training group vs.standard training group
|
34
|
nursing students
|
22.3
|
Description of the intervention in the studies
All interventions in the study are simulation tutorials or trainings, and 13 articles [13, 14, 16-19, 21-26, 31] articles use scenario simulation teaching, which creates hospital-like treatment environments for students. One of the articles [28] uses simulator training, that is, high-level simulators, such as a robotic arm, is used in helping nurse students learn blood pressure measurement techniques. Seven of the articles [11, 12, 15, 20, 27, 29, 30] use a combination of simulators and simulation scenarios for education. The intervention times of most studies are short, mostly 1–2 hours, and only six studies have an intervention period of more than 1 week. The PICO information of 21 studies is shown in Table 2.
Table 2. PICO of the included studies.
Study
|
Samples(EG/CG)
|
Male/Female(EG)
|
Male/Female(CG)
|
Intervention
|
Duration
|
Control
|
Outcome measures
|
Kahraman et al., 2019
|
36/36
|
6/30
|
3/33
|
Scenario simulation
|
1w
|
standard training
|
①
|
Basak et al., 2019
|
35/36
|
0/35
|
0/36
|
Scenario simulation and simulator
|
1w
|
standard training and curriculum lecture
|
③
|
Tamaki et al., 2019
|
20/18
|
0/20
|
1/17
|
Scenario simulation
|
80min
|
standard training
|
①③
|
Pamela et al., 2013
|
40/39
|
0/40
|
0/39
|
Scenario simulation
|
90min
|
standard training
|
①
|
Alinier et al ., 2006
|
49/50
|
7/42
|
9/41
|
Scenario simulation
|
6h
|
standard training
|
②③
|
JU et al., 2019
|
66/66
|
12/54
|
14/52
|
Scenario simulation
|
8w
|
standard training
|
①④
|
Stayt et al., 2015
|
50/48
|
5/45
|
6/42
|
Scenario simulation
|
1h
|
standard training
|
①
|
José et al., 2019
|
21/21
|
0/21
|
2/19
|
Scenario simulation and simulator
|
2m
|
standard training
|
①②④⑤
|
AlAmrani et al., 2017
|
36/36
|
12/24
|
14/22
|
Scenario simulation and simulator
|
2w
|
standard training
|
①⑥
|
Loai, 2020
|
38/38
|
16/22
|
14/24
|
Scenario simulation
|
3m
|
standard training
|
①③
|
Aloush, 2019
|
65/66
|
59/72
|
Scenario simulation
|
6h
|
standard training
|
①
|
Loai, 2016
|
42/42
|
21/21
|
20/22
|
Scenario simulation
|
3h20min
|
standard training
|
②③
|
Ahmad et al., 2015
|
47/44
|
/
|
Scenario simulation and simulator
|
40min
|
standard training
|
①②
|
Ahmad et al., 2014
|
45/45
|
10/35
|
9/36
|
Scenario simulation and simulator
|
4h
|
standard training
|
①②
|
Banu et al., 2019
|
32/27
|
/
|
simulator
|
2d
|
standard training
|
③④⑥
|
Ayse et al.,2019
|
53/54
|
7/46
|
7/47
|
Scenario simulation and simulator
|
4h
|
standard training
|
③
|
Li et al., 2019
|
50/50
|
23/27
|
21/29
|
Scenario simulation
|
2w
|
standard training
|
①②③
|
Sok et al., 2011
|
15/16
|
9/22
|
Scenario simulation
|
6h
|
standard training
|
①
|
Sok et al., 2012
|
15/16
|
1/14
|
1/15
|
Scenario simulation
|
6h
|
standard training
|
①③
|
Antonia et al.,2020
|
73/73
|
14/59
|
8/65
|
Scenario simulation and serious game
|
2h
|
standard training
|
⑤
|
Raphael et al., 2020
|
17/17
|
7/27
|
Scenario simulation
|
40h
|
standard training
|
①②
|
Abbreviations: ①Test scores;②Knowledge retention;③Self-confidence;④Self-efficacy;⑤Satisfaction;⑥Anxiety.
Quality of the studies
The proportion of items assessed for bias risk in the RCTs is shown in Figure 2, and the traffic light chart is shown in Figure 3. The RCT bias risk assessment tool ROB2.0 was used in risk assessment. The risk of bias is shown in Figures 2 and 3. Seven trials (33.33%) have a low risk of bias for the randomization process. All trials have low risk of bias for deviations from intended interventions, missing outcome data, measurement of the outcome, and selection of reported result. Seven trials (33.33%) have low risk of bias for overall bias. All the studies included in this article describe the random process in detail, and the main reason for the high risk of bias in the randomization process is that most of these studies do not mention the process of allocation concealment. A small number of studies use single-blind or double-blind studies, and the research subjects and data analysts are blinded. However, the intervention measures of the research mainly focus on courses and training. The blinding method is unsuitable for most research subjects and interveners. Therefore, we believe that the lack of blinding will not affect the measurement of final outcome indicators.
Meta-analysis for outcome measures
In the included studies, a meta-analysis was conducted on the performance and mentality of nursing students after intervention. The data evaluated in each area is summarized below.
Test scores
Fifteen RCTs [12-14, 16-22, 25-27, 30, 31] consisting of 1114 nursing students were included. The analysis results of the random-effects model show that the test scores of the simulation-based education group is higher than those of the control group (SMD = 1.46, 95% CI = 1.02–1.90, P < 0.00001; Figure 4).
Knowledge retention
Seven RCTs [20, 22-25, 27, 30] consisting of 507 nursing students were included. The analysis results of the random-effects model show that the degree of knowledge retention of the simulation-based education group is higher than that of the control group (SMD = 1.89, 95% CI = 0.76–2.87, P = 0.0008; Figure 5).
Self-confidence
Nine RCTs [11, 13, 15, 23-26, 28, 31] consisting of 665 nursing students were included. The analysis results of the random-effects model show that the self-confidence of the simulation-based education group is higher than that of the control group (SMD = 1.19, 95% CI = 0.48–1.90, P = 0.001; Figure 6).
Self-efficacy
Three RCTs [21, 28, 30] consisting of 233 nursing students were included. The analysis results of the random-effects model show that the self-efficacy of the simulation-based education group is higher than that of the control group (SMD = 0.24, 95% CI = −0.27–0.74, P = 0.36; Figure 7).
Satisfaction
Two RCTs [29, 30] consisting of 188 nursing students were included. The analysis results of the random-effects model show that the satisfaction of the simulation-based education group is higher than that of the control group (SMD = 0.86, 95% CI = 0.13–1.60, P = 0.02; Figure 8).
Anxiety
Two RCTs [12, 28] consisting of 131 nursing students were included. The analysis results of the random-effects model show that the anxiety of the simulation-based education group is lower than that of the control group (SMD = −0.36, 95% CI = −1.69–0.96, P = 0.59; Figure 9).
Subgroup analysis
The meta-analysis results show that I2>50% of many results indicates high heterogeneity between the studies. To determine the source of heterogeneity, we conducted a subgroup analysis based on the length of the intervention. The short-term intervention included six studies [12, 14, 21, 25, 26, 30], and the main intervention time was within a few hours or 2 days. The long-term intervention included 15 studies [11, 13, 15-20, 22-24, 27-29, 31], and the intervention time ranged from 1 week to 3 months. Given that few related studies involve knowledge retention, self-confidence, self-efficacy, satisfaction, and anxiety, we conducted a subgroup analysis on the index of assessment performance. The results are as follows.
Test scores
After the studies were grouped according to time, the heterogeneity of the meta-analysis results was significantly reduced. Thus, a random-effects model was used for analysis. After subgroup analysis, the total I2=90, the heterogeneity is still high, indicating that the main source of heterogeneity has no relationship with intervention measures. In fact, 20 studies [11-27, 29-31] adopt scenario simulation teaching, and the detailed settings of the scenario simulation of each study may be the main source of heterogeneity. Six studies were included [12, 14, 21, 25, 26, 30] in the long-time group. The long-time intervention group is significantly better than the control group in terms of reducing test scores (MD = 1.44, 95% CI = 0.62, 2.26, P = 0.0006). Nine studies [13, 16-20, 22, 27, 31] were included in the short-time group. Short-time intervention significantly reduces test scores (MD = 1.46, 95% CI = 0.95, 1.97, P < 0.00001). The results are shown in Figure 10.
GRADEpro evidence assessment
In the evaluation of GRADEpro’s level of evidence, the meta-analysis results have three pieces of intermediate evidence: one piece of low-level evidence and two pieces of very low-level evidence. The overall level of evidence is low. The main reasons for degradation are inconsistency and imprecision. In terms of inconsistency, most indicators are statistically heterogeneous, that is, I2 > 50%, and they are all downgraded to one level. Two main problems were determined in terms of imprecision. On the one hand, the sample sizes of some indicators do not meet the optimal sample size, that is, the total sample size of the total continuous variable is less than 400. On the other hand, the effect sizes of some indicators cross the invalid line, that is, P ≥ 0.05. Through downgrade treatment, both types of indicators have been downgraded two times. No downgrade treatment was performed on the other three aspects of grade evaluation. In terms of limitations, some studies do not use blinding, and the allocation concealment report is insufficient, although they are not downgraded because blinding and allocation concealment have little effect on the experimental results. In terms of indirectness, although the intervention measures are not completely consistent (different types of simulation scenarios), no significant difference was found between PICO and the research question, and thus it is not downgraded. In terms of publication bias, the literature search in this study is comprehensive, the included studies do not involve commercial interests, and no clear evidence of a risk of bias is found. Hence, publication bias will not be downgraded. See Table 3 for details.
Table 3. Evidence assessment of outcomes.
Outcome indicators (number of studies)
|
Sample size(T/C)
|
limitation
|
Inconsistency
|
Indirectness
|
Imprecision
|
Publication bias
|
Effect size
|
Quality of evidence
|
Test scores(15)
|
560/554
|
0
|
-1
|
0
|
0
|
0
|
SMD=1.46,95%CI (1.02~1.90)
|
⊕⊕⊕Ο B(middle-level)
|
Knowledge retention(7)
|
254/253
|
0
|
-1
|
0
|
0
|
0
|
SMD=1.81,95%CI (0.76~2.87)
|
⊕⊕⊕Ο B(middle-level)
|
Self-confidence(9)
|
334/331
|
0
|
-1
|
0
|
0
|
0
|
SMD=1.19,95%CI (0.48~1.90)
|
⊕⊕⊕Ο B(middle-level)
|
Self-efficacy(3)
|
119/114
|
0
|
-1
|
0
|
-2
|
0
|
SMD=0.24,95%CI (-0.27~0.74)
|
⊕ΟΟΟ D(Very low-level)
|
Satisfaction(2)
|
94/94
|
0
|
-1
|
0
|
-1
|
0
|
SMD=0.86,95%CI (0.13~1.60)
|
⊕⊕ΟΟ C(Low-level)
|
Anxiety(2)
|
68/63
|
0
|
-1
|
0
|
-2
|
0
|
SMD=-0.36,95%CI (-1.69~0.96)
|
⊕ΟΟΟ D(Very low-level)
|
Abbreviations: Reason for deduction: Inconsistency: I2>50%; Imprecision: continuous variable, the total sample size of the two groups of research subjects is less than 400 cases or the confidence interval crosses the invalid line.