Comparing formative and summative simulation-based assessment in undergraduate nursing students: nursing competency acquisition and clinical simulation satisfaction

Background

Clinical simulation methodology has increased exponentially over the last few years and has gained acceptance in nursing education. Simulation-based education (SBE) is considered as an effective educational methodology for nursing students to achieve the competencies needed for their professional future [1–4]. In addition, simulation-based educational programs have demonstrated to be more useful than traditional teaching methodologies [3, 5]. As a result, most nursing faculties are integrating this methodology into their study plans [6]. SBE has the potential to shorten the learning curve for students, increase the fusion between theoretical knowledge and clinical practice, establish deficient areas in students, develop communication and technical skills acquisition, improve patient safety, standardise the curriculum and teaching contents, and offer observations of real-time clinical decision making [4, 5, 7, 8].

SBE offers an excellent opportunity to perform not only observed competency-based teaching, but also the assessment of these competencies. Simulated-based assessment (SBA) is aimed at evaluating various professional skills, including knowledge, technical and clinical skills, communication, and decision-making; as well as higher-order competencies such as patient safety and teamwork skills [1–3, 9]. Compared with other traditional assessment methods (i.e. written or oral test), SBA offers the opportunity to evaluate the actual performance in an environment similar to the ‘real’ clinical practice, assess multidimensional professional competencies, and present standard clinical scenarios to all students [1–3, 9].

The main SBA strategies are formative and summative evaluation. Formative evaluation is conducted to establish students’ progression during the course [10]. This evaluation strategy is helpful to educators in improving students’ deficient areas and testing their knowledge [11]. Employing this evaluation strategy, educators give students feedback about their performance. Subsequently, students self-reflect to evaluate their learning and determine their deficient areas. In this sense, formative evaluation includes an ideal phase to achieve the purposes of strategy: the debriefing [12]. International Nursing Association for Clinical Simulation and Learning (INACSL) defines debriefing as a reflective process immediately following the simulation-based experience where ‘participants explore their emotions and question, reflect, and provide feedback to one another’. Its aim is ‘to move toward assimilation and accommodation to transfer learning to future situations’ [13]. Therefore, debriefing is a basic component for learning to be effective after the simulation [14, 15]. Furthermore, MAES^© (according to its Spanish initials of self-learning methodology in simulated environments) is a clinical simulation methodology created to perform formative evaluations [16]. MAES© allows evaluating specifically nursing competencies acquired by several nursing students at the same time. This methodology meets all the requirements defined by the INACSL Standards of Best Practice: Simulation^SM [17] and it has proven to be more effective than traditional simulation-based learning [18].

In contrast, summative evaluation is used to establish the learning outcomes achieved by students at the end of the course [10]. This evaluation strategy is helpful to educators in evaluating students’ learning, the competencies acquired by them and their academic achievement [11]. This assessment is essential in the education process to determine readiness and competence for certification and accreditation [9, 19]. Accordingly, Objective Structured Clinical Examination (OSCE) is commonly conducted in SBA as a summative evaluation to evaluate students' clinical competence [20]. Consequently, OSCE has been used by educational institutions as a valid and reliable method of assessment. OSCE most commonly consists of a ‘round-robin’ of multiple short testing stations, in each of which students must demonstrate defined clinical competencies, while educators evaluate their performance according to predetermined criteria using a standardized marking scheme, such as checklists. Students must rotate through these stations where educators assess students’ performance in clinical examination, technical skills, clinical judgment and decision-making skill during the nursing process [20, 21]. This strategy of summative evaluation incorporates actors performing as simulated patients. Therefore, OSCE allows assessing students’ clinical competence in a real-life simulated clinical environment. After simulated scenarios, this evaluation strategy provides educators with an opportunity to give students constructive feedback according to their achieved results in the checklist [9, 19–21].

Despite both evaluation strategies are widely employed in SBA, there is scarce evidence about the possible differences in satisfaction with clinical simulation when nursing students are assessed using formative and summative evaluation. Therefore, the aims of our study were to evaluate the acquisition of nursing competencies through clinical simulation methodology in undergraduate nursing students, as well as to compare their satisfaction with this methodology using two strategies of SBA, such as formative and summative evaluation.

Methods

Study design and setting

A descriptive cross-sectional study using a mixed-method and analysing both quantitative and qualitative data. The study was conducted from September 2018 to May 2019 in a University Centre of Health Sciences in Madrid (Spain). This centre offers Physiotherapy and Nursing Degrees.

Participants

The study included 3rd-year undergraduate students (106 students participated in MAES^Ⓒ sessions within the subject ‘Nursing care for critical patients’) and 4th-year undergraduate students (112 students participated in OSCE sessions within the subject ‘Supervised clinical placements – Advanced level’) in Nursing Degree. It should be noted, 4th-year undergraduate students had completed all their clinical placements and they had to approve OSCE sessions to achieve their certification.

Clinical simulation sessions

To assess the clinical performance of 3rd-year undergraduate students using formative evaluation, MAES^Ⓒ sessions were conducted. This methodology consists of 6 elements [16]: Team selection and creation of group identity, voluntary choice of subject of study, establishment of baseline and programming skills to be acquired through brainstorming, design of a clinical simulation scenario in which the students practice the skills to be acquired, execution of the simulated clinical experience, and finally debriefing and presentation of the acquired skills.

Alternatively, OSCE sessions were developed to assess the clinical performance of 4th-year undergraduate students using summative evaluation. Both MAES^Ⓒ and OSCE sessions recreated critically ill patients with diagnoses of Exacerbation of Chronic Obstructive Pulmonary Disease (COPD), acute coronary syndrome haemorrhage in a postsurgical, and severe traumatic brain injury.

Critically ill patient’s role was performed by an advanced simulator mannequin (NursingAnne® by Laerdal Medical AS) in all simulated scenarios. A confederate (a health professional who acts in a simulated scenario) performed the role of a registered nurse or a physician who could help students as required. Occasionally, this confederate could perform the role of a relative of a critically ill patient. Clinical scenarios were designed by professors of the subjects with standardised scripts. Nursing students formed work teams of 2–3 students in all sessions. Specifically, each work team formed in MAES^Ⓒ sessions received a brief description of simulated scenario two months before and students had to propose 3 NIC (Nursing Interventions Classification) interventions [22], and 5 related nursing activities with each of them, to resolve the critical situation. In contrast, the critical situation was presented to each work team formed in OSCE sessions for 2 minutes before entering the simulated scenario. During all simulated experiences, professors were monitoring and controlling the simulation with a sophisticated computer program in a dedicated control room. All simulated scenarios lasted 10 minutes.

After each clinical simulated scenario was concluded, a debriefing was carried out to give students feedback about their performance. Debriefings in MAES^Ⓒ sessions were conducted according to the Gather, Analyse, and Summarise (GAS) method, a structured debriefing model developed by Phrampus and O'Donnell [23]. Additionally, MAES^Ⓒ includes an expository phase in debriefings, where the students who performed the simulated scenario establish the contributions of scientific evidence about its resolution [16]. Each debriefing lasted 20 minutes in MAES^Ⓒ sessions. In contrast, debriefings in OSCE sessions lasted 10 minutes and they were carried out according to Plus-Delta debriefing tool [24], a technique recommended when time is limited. Within these debriefings, professors communicate students the total score obtained in the appropriate checklist. Each debriefing lasted 10 minutes in OSCE sessions. After all debriefings, students completed the questionnaires to evaluate their satisfaction with clinical simulation. In OSCE sessions, students had to report their satisfaction only with the scenario performed, which took part in a series of clinical stations.

Instruments

Clinical performance

Professors assessed students’ clinical performance using checklists (‘Yes’/‘No’). In MAES^Ⓒ sessions, checklists were based on the 5 most important nursing activities included in the NIC [22] selected by nursing students. Table 1 shows the checklist of the most important NIC interventions and its related nursing activities selected by nursing students in Exacerbation of Chronic Obstructive Pulmonary Disease (COPD) simulated scenario. In contrast, checklists for evaluating OSCE sessions were based on nursing activities selected by consensus among professors, registered nurses, and clinical placement mentors. Nursing activities were divided into 5 categories: nursing assessment, clinical judgment/decision-making, clinical management/nursing care, communication/interpersonal relationships, and teamwork. Table 2 shows the checklist of nursing activities that nursing students had to perform in COPD simulated scenario. During the execution of all simulated scenarios, professors checked if the participants perform or not the nursing activities selected.

Table 1. Formative evaluation: Checklist of the most important NIC interventions and its related nursing activities (Butcher et al., 2018) selected by nursing students in Exacerbation of Chronic Obstructive Pulmonary Disease (COPD) simulated scenario.

[3350] Respiratory Monitoring	YES	NO
Monitor rate, rhythm, depth, and effort of respirations
Auscultate lung sounds after treatments to note results
Note changes in SaO₂, SvO₂, end tidal CO₂, and ABG values, as appropriate
Monitor for increased restlessness, anxiety, and air hunger
Institute respiratory therapy treatments (e.g., nebuliser), as needed
[3302] Mechanical Ventilation Management: Noninvasive	YES	NO
Monitor for conditions indicating the appropriateness of noninvasive ventilation support (e.g., acute exacerbations of COPD)
Consult with other health care personnel in selection of a noninvasive ventilator type (e.g., pressure limited [bilevel positive airway pressure], volume-cycled flow-limited, or CPAP)
Instruct the patient and family about the rationale and expected sensations associated with the use of noninvasive mechanical ventilators and devices
Place the patient in semi-Fowler position
Apply noninvasive device, assuring adequate fit and avoidance of large air leaks (take particular care with edentulous or bearded patients)
[5820] Anxiety Reduction	YES	NO
Use a calm, reassuring approach
Clearly state expectations for patient´s behaviour
Explain all procedures, including sensations likely to be experienced during the procedure
Provide factual information concerning diagnosis, treatment, and prognosis
Stay with the patient to promote safety and reduce fear

Table 2. Summative evaluation: Checklist of nursing activities performed by nursing students

in Exacerbation of Chronic Obstructive Pulmonary Disease (COPD) simulated scenario.

Nursing Assessment (20 points)	YES	NO	Points
They perform a focused respiratory exploration through appropriate pulmonary auscultation (5 points)
They recognise correctly signs and symptoms of respiratory distress, including SaO₂(5 points)
They assess correctly haemodynamic signs and symptoms (5 points)
They interpret correctly the complementary tests ordered by the physician (5 points)
Clinical Judgement and Decision-Making (20 points)	YES	NO	Points
They diagnose correctly the patient´s clinical situation (5 points)
They prioritise adequately nursing interventions (5 points)
They re-evaluate the patient according to nursing assessment (5 points)
They apply the appropriate treatment for respiratory distress at the right time (5 points)
Clinical Management and Nursing Care (30 points)	YES	NO	Points
Handwashing (2,5 points)
Use of gloves (2,5 points)
They place the patient in semi-Fowler position (2,5 points)
Proper pulse oximeter placement (2,5 points)
Proper EEG electrodes placement (2,5 points)
Proper blood pressure cuff placement (2,5 points)
They apply correctly the adequate oxygen therapy according to nursing assessment (2,5 points)
They call a physician (2,5 points)
They follow properly physician instructions (2,5 points)
They administer correctly the prescribed medication (2,5 points)
They evaluate the patient´s response to the medical treatment administered (2,5 points)
They perform correctly the complementary test ordered by the physician (2,5 points)
COMMUNICATION AND INTERPERSONAL RELATIONSHIPS (15 points)	YES	NO	Points
They introduce themselves to the patient (3 points)
They reduce the patient´s anxiety (3 points)
They show empathy, active listening and respect when they communicate with the patient and/or family (3 points)
Appropriate communication with the physician (3 points)
Appropriate communication among team members (3 points)
Teamwork (15 points)	YES	NO	Points
Appropriate coordination among team members and they demonstrate an effective teamwork

Clinical simulation satisfaction

To determine satisfaction with clinical simulation perceived by nursing students, the Satisfaction Scale Questionnaire with High-Fidelity Clinical Simulation [25] was used after each clinical simulation session. This questionnaire consists of 33 items with a 5-point Likert scale ranging from ‘strongly disagree’ to ‘totally agree’. These items are divided into 8 scales: simulation utility, characteristics of cases and applications, communication, self-reflection on performance, increased self-confidence, relation between theory and practice, facilities and equipment and negative aspects of simulation. Cronbach’s α values for each scale ranged from .914 to .918 and total scale presents satisfactory internal consistency (Cronbach’s α value = .920). This questionnaire includes a final question about any opinion or suggestion that participating students wish to reflect after the simulation experience.

Data Analysis

Quantitative data were analysed using IBM SPSS Statistics version 24.0 software for Windows (IBM Corp., Armonk, NY, USA). Descriptive statistics were calculated to interpret the results obtained in demographic data, clinical performance, and satisfaction with clinical simulation. T-test was used to compare satisfaction with clinical simulation in both evaluation strategies. Cohen's d was calculated to analyse the effect size for t-tests. Statistical tests were two-sided (α = 0.05), so the statistical significance was set at 0.05. Subsequently, all students’ opinions and comments were analysed using the ATLAS-ti version 8.0 software (Scientific Software Development GmbH, Berlin, Germany). All the information contained in these qualitative data were stored, managed, classified and organized through this software. All the reiterated words, sentences or ideas were grouped into themes using a thematic analysis [26]. It should be noted that the students’ opinions and comments were preceded by the letter ‘S’ (student) and numerically labelled.

Results

A total of 218 nursing students participated in the study (106 students were trained through MAES^♥ sessions, whereas 112 students were assessed through OSCE sessions). The age of students ranged from 20 to 43 years (mean = 23.28; SD = 4.376). Most students were women (n = 184; 84.4%).

In formative evaluation, professors checked 93.2% of students selected adequately both NIC interventions and its related nursing activities for the resolution of the clinical simulated scenario. Subsequently, these professors checked 85.6% of students, who participated in each simulated scenario, performed the nursing activities previously selected by them. In summative evaluation, students obtained total scores ranged from 65 to 95 points (mean = 7.43; SD = .408).

Descriptive data for each scale of satisfaction with clinical simulation questionnaire, t-test, and effect sizes (d) of differences between two evaluation strategies are shown in Table 3. Statistically significant differences were found between two evaluation strategies for all scales of satisfaction with clinical simulation questionnaire. Students´ satisfaction with clinical simulation was higher for all scales of the questionnaire when they were assessed using formative evaluation, including the ‘negative aspects of simulation’ scale, where they students perceived fewer negative aspects. The effect size of these differences was large (including the total score of the questionnaire) (Cohen’s d values > .8), except for ‘facilities and equipment’ scale, which effect size was medium (Cohen´s d value > .5) [27].

Table 3
Descriptive data, t-test and effect sizes (d) of differences between two evaluation strategies for scales of clinical simulation satisfaction (n = 218).
Nursing Assessment (20 points)	YES	NO	Points
They perform a focused respiratory exploration through appropriate pulmonary auscultation (5 points)
They recognise correctly signs and symptoms of respiratory distress, including SaO₂ (5 points)
They assess correctly haemodynamic signs and symptoms (5 points)
They interpret correctly the complementary tests ordered by the physician (5 points)
Clinical Judgement and Decision-Making (20 points)	YES	NO	Points
They diagnose correctly the patient´s clinical situation (5 points)
They prioritise adequately nursing interventions (5 points)
They re-evaluate the patient according to nursing assessment (5 points)
They apply the appropriate treatment for respiratory distress at the right time (5 points)
Clinical Management and Nursing Care (30 points)	YES	NO	Points
Handwashing (2,5 points)
Use of gloves (2,5 points)
They place the patient in semi-Fowler position (2,5 points)
Proper pulse oximeter placement (2,5 points)
Proper EEG electrodes placement (2,5 points)
Proper blood pressure cuff placement (2,5 points)
They apply correctly the adequate oxygen therapy according to nursing assessment (2,5 points)
They call a physician (2,5 points)
They follow properly physician instructions (2,5 points)
They administer correctly the prescribed medication (2,5 points)
They evaluate the patient´s response to the medical treatment administered (2,5 points)
They perform correctly the complementary test ordered by the physician (2,5 points)
COMMUNICATION AND INTERPERSONAL RELATIONSHIPS (15 points)	YES	NO	Points
They introduce themselves to the patient (3 points)
They reduce the patient´s anxiety (3 points)
They show empathy, active listening and respect when they communicate with the patient and/or family (3 points)
Appropriate communication with the physician (3 points)
Appropriate communication among team members (3 points)
Teamwork (15 points)	YES	NO	Points
Appropriate coordination among team members and they demonstrate an effective teamwork
	TOTAL

Table 4 shows specifically descriptive data, t-test, and effect sizes (d) of differences between both evaluation strategies for each item of clinical simulation satisfaction questionnaire. Statistically significant differences were found between two evaluation strategies for all items of the questionnaire, except for items ‘I have improved communication with the family’, ‘I have improved communication with the patient’, and ‘I lost calm during any of the cases’. Students´ satisfaction with clinical simulation was higher in formative evaluation sessions for most items, except for item ‘simulation has made me more aware/worried about clinical practice’, where students informed being more aware and worried in summative evaluation sessions. Most effect sizes of these differences were small or medium (Cohen´s d values ranged from .238 to .709) [27]. The largest effect sizes of these differences were obtained for items ‘timing for each simulation case has been adequate’ (d = 1.107), ‘overall satisfaction of sessions’ (d = .953), and ‘simulation has made me more aware/worried about clinical practice’ (d = − .947). In contrast, the smallest effect sizes of these differences were obtained for items ‘simulation allows us to plan the patient care effectively’ (d = .238) and ‘the degree of cases difficulty was appropriate to my knowledge’ (d = .257).

Table 4. Descriptive data, t-test and effect sizes (d) of differences between two evaluation strategies for each item of clinical simulation satisfaction questionnaire (n=218).
Scale	Formative evaluation (n = 106)	Summative evaluation (n = 112)	F_(1,217)	Sig.	Effect size (d)
Scale	Mean (SD)	Mean (SD)	F_(1,217)	Sig.	Effect size (d)
Simulation utility	56.59 (5.584)	52.67 (10.109)	21.71	.001	3.925
Characteristics of cases and application	18.57 (1.487)	16.74 (2.690)	27.84	.000	1.825
Communication	14.36 (1.244)	12.98 (2.379)	42.13	.000	1.376
Self-reflection on performance	14.28 (1.119)	12.73 (2.438)	35.84	.000	1.551
Increased self-confidence	13.72 (1.378)	11.71 (3.071)	42.87	.000	2.003
Relation between theory and practice	13.78 (1.345)	11.71 (2.447)	41.43	.000	2.069
Facilities and equipment	12.20 (1.775)	11.58 (2.225)	4.29	.024	.618
Negative aspects of simulation	3.73 (1.231)	4.77 (.849)	12.09	.000	− .947
Total score	147.23 (9.977)	134.61 (21.955)	35.10	.000	12.619

In addition, participating students provided 74 opinions or suggestions expressed through short comments. Most students’ comments were related to 3 main themes after the thematic analysis: utility of clinical simulation methodology (S45: ‘it has been a useful activity and it helped us to recognize our mistakes and fixing knowledge’, S94: ‘to link theory to practice is essential’), to spend more time on this methodology (S113: ‘I would ask for more practices of this type‘, S178: ‘I feel very happy, but it should be done more frequently’), and its integration into other subjects (S21: ‘I consider this activity should be implemented in more subjects’, S64: ‘I wish there were more simulations in more subjects’). Finally, students´ comments about summative evaluation sessions included other 2 main themes related to: limited time of simulation experience (S134: ‘time is short’, S197: ‘there is no time to perform activities and assess properly’) and students´ anxiety (S123: ‘I was very nervous because people were evaluating me around’, S187: ‘I was more nervous than in a real situation’).

Discussion

The most significant results obtained in our study are the nursing competency acquisition through clinical simulation by nursing students and the different level of their satisfaction with this methodology depending on the evaluation strategy employed.

Firstly, professors in this study verified most students acquired the nursing competencies to resolve each clinical situation. This result confirms the findings in other studies that have demonstrated nursing competency acquisition by nursing students through clinical simulation [28, 29], and specifically nursing competencies related to critical patient management [8, 30].

Secondly, students’ satisfaction assessed using both evaluation strategies could be considered high in most items of the questionnaire, regarding their mean scores. The high level of satisfaction expressed by nursing students with clinical simulation obtained in this study is also congruent with empirical evidence, which confirms that this methodology is a useful tool for their learning process [5, 25, 31–34].

However, satisfaction with clinical simulation was higher when students were assessed using formative evaluation. The main students’ complaints with summative evaluation were related to reduced time for performing simulated scenarios and increased anxiety during their clinical performance. Reduced time is a frequent complaint of students in OSCE [21, 35] and clinical simulation methodology [4, 5, 9]. Professors, registered nurses, and clinical placement mentors tested all simulated scenarios and their checklist in this study. They checked the time was enough for its resolution. Another criticism of summative evaluation is increased anxiety. However, several studies have demonstrated during clinical simulation students’ anxiety increase [36, 37] and it is considered as the most disadvantage of clinical simulation [1–9]. In this sense, anxiety may influence negatively students’ learning process [36, 37]. Although the current simulation methodology can mimic the real medical environment to a great degree, it might still be questionable whether students´ performance in the testing environment really represents their true ability. Test anxiety might increase in an unfamiliar testing environment; difficulty to handle unfamiliar technology (i.e., monitor, defibrillator, or other devices that may be different than the ones used in the examinee’s specific clinical environment) or even the need to ‘act as if’ in an artificial scenario (i.e., talking to a simulator, examining a ‘patient’ knowing he/she is an actor or a mannequin) might all compromise examinees' performance. The best solution to reduce these complaints is the orientation of students to the simulated environment [9, 19–21].

Both SBA strategies allow educators evaluating students’ knowledge and apply it in a clinical setting. However, formative evaluation is identified as ‘assessment for learning’ and summative evaluation as ‘assessment of learning’ [38]. Using formative evaluation, educators' responsibility is to ensure not only what students are learning in the classroom, but also the outcomes of their learning process [39]. In this sense, formative assessment by itself is not enough to determine educational outcomes [40]. Consequently, a checklist for evaluating students’ clinical performance was included in MAES© sessions. Alternatively, educators cannot make any corrections in students’ performance using summative evaluation [39]. Gavriel [38] suggests providing students feedback in this SBA strategy. Therefore, a debriefing phase was included after each OSCE session in our study. The significance of debriefing recognised by nursing students in our study is also congruent with the most evidence found [12, 14, 15, 41]. Nursing students appreciate feedback about their performance during simulation experience and, consequently, debriefing is considered as the most rewarding phase in clinical simulation by them [4, 5, 42]. In addition, nursing students in our study expressed they could learn from their mistakes in debriefing. Learn from error is one of the most advantages of clinical simulation shown in several studies [4, 5, 43] and mistakes should be considered learning opportunities rather than there being embarrassment or punitive consequences [44].

Furthermore, nursing students who participated in our study considered the practical utility of clinical simulation as another advantage of this teaching methodology. This result is congruent with previous studies [4, 5]. Specifically, our students indicated this methodology is useful to bridge the gap between theory and practice [45, 46]. In this sense, clinical simulation has proven to reduce this gap and, consequently, it has demonstrated to shorten the gap between classrooms and clinical practices [4, 5, 45, 46]. Therefore, as this teaching methodology relates theory and practice, it helps nursing students to be prepared for their clinical practices and future careers. According to Benner’s model of skill acquisition in nursing [47], nursing students become competent nurses through this learning process, acquiring a degree of safety and clinical experience before their professional careers [48]. Although our research indicates clinical simulation is a useful methodology for the acquisition and learning process of competencies mainly related to adequate management and nursing care of critically ill patients, this acquisition and learning process could be extended to most nursing care settings and its required nursing competencies.

Limitations and future research

Although checklists employed in OSCE have been criticized for their subjective construction [9, 19–21], they were constructed with the expert consensus of nursing professors, registered nurses and clinical placement mentors. Alternatively, the self-reported questionnaire used to evaluate clinical simulation satisfaction has strong validity. All simulated scenarios were similar in OSCE and MAES sessions (same clinical situations, patients, actors and number of participating students), although the debriefing method employed after them was different. This difference was due to reduced time in OSCE sessions. Future research should combine formative and summative evaluation for assessing clinical performance of undergraduate nursing students in simulated scenarios.

References

Cant RP, Cooper SJ. Simulation-based learning in nurse education: systematic review. J Adv Nurs. 2010;66:3–15.
Chernikova O, Heitzmann N, Stadler M, Holzberger D, Seidel T, Fischer F. Simulation-Based Learning in Higher Education: A Meta-Analysis. Review of Educational Research. 2020;90:499–541.
Kim J, Park JH, Shin S. Effectiveness of simulation-based nursing education depending on fidelity: a meta-analysis. BMC Med Educ. 2016;16:152.
Ricketts B. The role of simulation for learning within pre-registration nursing education—a literature review. Nurse Educ Today. 2011;31:650–4.
Shin S, Park JH, Kim JH. Effectiveness of patient simulation in nursing education: meta-analysis. Nurse Educ Today. 2015;35:176–82.
Bagnasco A, Pagnucci N, Tolotti A, Rosa F, Torre G, Sasso L. The role of simulation in developing communication and gestural skills in medical students. BMC Med Educ. 2014;14:106.
Oh PJ, Jeon KD, Koh MS. The effects of simulation-based learning using standardized patients in nursing students: a meta-analysis. Nurse Educ Today. 2015;35:e6-e15.
Stayt LC, Merriman C, Ricketts B, Morton S, Simpson T. Recognizing and managing a deteriorating patient: a randomized controlled trial investigating the effectiveness of clinical simulation in improving clinical performance in undergraduate nursing students. J Adv Nurs. 2015;71:2563–74.
Ryall T, Judd BK, Gordon CJ. Simulation-based assessments in health professional education: a systematic review. J Multidiscip Healthc. 2016;9:69–82.
Billings DM, Halstead JA. Teaching in nursing: A guide for faculty, 4th edition. St. Louis, MO: Elsevier; 2012.
Nichols PD, Meyers JL, Burling KS. A framework for evaluating and planning assessments intended to improve student achievement. Educational measurement: issues and Practice. 2009;28:14–23.
Cant RP, Cooper SJ. The benefits of debriefing as formative feedback in nurse education. Aust J Adv Nurs. 2011;29:37–47.
INACSL Standards Committee. INACSL Standards of Best Practice: Simulation^SM: Simulation Glossary. Clin Simul Nurs. 2016;12:S39-S47.
Dufrene C, Young A. Successful debriefing-best methods to achieve positive learning outcomes: a literature review. Nurse Educ Today. 2014;34:372–6.
Levett-Jones T, Lapkin S. A systematic review of the effectiveness of simulation debriefing in health professional education. Nurse Educ Today. 2014;34:e58-e63.
Díaz JL, Leal C, García JA, Hernández E, Adánez MG, Sáez A. Self-Learning Methodology in Simulated Environments (MAES©): Elements and Characteristics. Clin Simul Nurs. 2016;12:268–74.
INACSL Standards Committee. INACSL Standards of Best Practice: Simulation^SM: Participant Evaluation. Clin Simul Nurs. 2016;12:S26-S29.
Díaz Agea JL, Megías Nicolás A, García Méndez JA, Adánez Martínez MG, Leal Costa C. Improving simulation performance through Self-Learning Methodology in Simulated Environments (MAES©). Nurse Educ Today. 2019;76:62–7.
Oermann MH, Kardong-Edgren S, Rizzolo MA. Summative Simulated-Based Assessment in Nursing Programs. J Nurs Educ. 2016;55:323–8.
Harden RM, Gleeson FA. Assessment of clinical competence using an objective structured clinical examination (OSCE). Med Educ. 1979;13:41–54.
Mitchell ML, Henderson A, Groves M, Dalton M, Nulty D. The objective structured clinical examination (OSCE): optimising its value in the undergraduate nursing curriculum. Nurse Educ Today. 2009;29:394–404.
Butcher, HK, Bulechek GM, Dochterman JMM, Wagner C. Nursing Interventions Classification (NIC), 7th edition. St. Louis, MO: Elsevier; 2018.
Phrampus PE, O’Donnell JM. Debriefing using a structured and supported approach. In AI Levine AI, JrS De Maria, AD Schwartz & AJ Sim (Eds.), The comprehensive textbook of healthcare simulation (pp. 73–84). New York, NY: Springer; 2013.
Decker S, Fey M, Sideras S, Caballero S, Rockstraw L, Boese T, Franklin AE, Gloe D, Lioce L, Sando CR, Meakim C, Borum JC. Standards of Best Practice: Simulation Standard VI: The Debriefing Process. Clin Simul Nurs. 2013;9:S26-S29.
Alconero-Camarero AR, Gualdrón-Romero A, Sarabia-Cobo CM, Martínez-Arce A. Clinical simulation as a learning tool in undergraduate nursing: Validation of a questionnaire. Nurse Educ Today. 2016;39:128–34.
Mayan M. Essentials of qualitative inquiry. Walnut Creek, CA: Left Coast Press, Inc.; 2009.
Cohen L, Manion L, Morrison K. Research methods in education, 7th edition. London: Routledge; 2011.
Lapkin S, Levett-Jones T, Bellchambers H, Fernandez R. Effectiveness of patient simulation manikins in teaching clinical reasoning skills to undergraduate nursing students: A systematic review. Clin Simul Nurs. 2010;6:207–22.
McGaghie WC, Issenberg SB, Petrusa ER, Scalese RJ. Revisiting “A critical review of simulation-based medical education research: 2003–2009’’. Med Educ. 2016;50:986–91.
Abelsson A, Bisholt B. Nurse students learning acute care by simulation - Focus on observation and debriefing. Nurse Educ Pract. 2017;24:6–13.
Bland AJ, Topping A, Wood BA. Concept analysis of simulation as a learning strategy in the education of undergraduate nursing students. Nurse Educ. Today. 2011;31:664–70.
Franklin AE, Burns P, Lee CS. Psychometric testing on the NLN Student Satisfaction and Self-Confidence in Learning, Design Scale Simulation, and Educational Practices Questionnaire using a sample of pre-licensure novice nurses. Nurse Educ Today. 2014;34:1298–304.
Levett-Jones T, McCoy M, Lapkin S, Noble D, Hoffman K, Dempsey J, Arthur C, Roche J. The development and psychometric testing of the satisfaction with simulation experience scale. Nurse Educ Today. 2011;31:705–10.
Zapko KA, Ferranto MLG, Blasiman R, Shelestak D. Evaluating best educational practices, student satisfaction, and self-confidence in simulation: A descriptive study. Nurse Educ Today. 2018;60:28–34.
Kelly MA, Mitchell ML, Henderson A, Jeffrey CA, Groves M, Nulty DD, Glover P, Knight S. OSCE best practice guidelines-applicability for nursing simulations. Adv Simul. 2016;1:10.
Cantrell ML, Meyer SL, Mosack V. Effects of Simulation on Nursing Student Stress: An Integrative Review. J Nurs Educ. 2017;56:139–44.
Nielsen B, Harder N. Causes of Student Anxiety during Simulation: What the Literature Says. Clin Simul Nurs. 2013;9:e507-e512.
Gavriel J. Assessment for learning: A wider (classroom-researched) perspective is important for formative assessment and self-directed learning in general practice. Educ Prim Care. 2013;24:93–6.
Taras M. Summative and formative assessment. Active Learning in Higher Education. 2008;9:172–82.
Wunder LL, Glymph DC, Newman J, Gonzalez V, Gonzalez JE, Groom JA. Objective structured clinical examination as an educational initiative for summative simulation competency evaluation of first-year student registered nurse anesthetists’ clinical skills. AANA J. 2014;82:419–25.
Neill MA, Wotton K. High-fidelity simulation debriefing in nursing education: A literature review. Clin Simul Nurs. 2011;7:e161-e168.
Norman J. Systematic review of the literature on simulation in nursing education. ABNF J. 2012;23:24–8.
King A, Holder MGJr, Ahmed RA. Error as allies: error management training in health professions education. BMJ Qual Saf. 2013;22:516–9.
Higgins M, Ishimaru A, Holcombe R, Fowler A. Examining organizational learning in schools: the role of psychological safety, experimentation, and leadership that reinforces learning. J Educ Change. 2012;13:67–94.
Hope A, Garside J, Prescott S. Rethinking theory and practice: Pre-registration student nurses experiences of simulation teaching and learning in the acquisition of clinical skills in preparation for practice. Nurse Educ Today. 2011;31:711–7.
Lisko SA, O’Dell V. Integration of theory and practice: Experiential learning theory and nursing education. Nurs Educ Perspect. 2010;31:106–8.
Benner P. From Novice to Expert: Excellence and Power in Clinical Nursing Practice. Menlo Park, CA: Addison-Wesley Publishing; 1984.
Nickless LJ. The use of simulation to address the acute care skills deficit in pre-registration nursing students: a clinical skill perspective. Nurse Educ Pract. 2011;11:199–205.
Declarations