Research design and samples
This study was a quasi-experimental design and was conducted at a private university with two campuses as previously described in the “Two-year RN-to-BSN program in Taiwan” section of this paper. The study was conducted from September 2015 to February 2016. There were 504 total students enrolled in the two-year RN-to-BSN program with 304 and 200 students at the L and C campuses, respectively. All 504 RN-to-BSN students were potential participants. To minimize intervention “contamination” between experimental and control participants, the students from L campus belonged to the experimental group (EG) and those from C campus belonged to the control group (CG).
Flipped classroom intervention
In the present study, a flipped classroom was implemented in an adult-health nursing course on L campus that incorporated face-to-face TBL and simulation activity and online self-directed learning (via the “e-campus” platform). The adult-health nursing course is one of the core nursing courses taught in the ADN program. The reasons why the adult-health nursing course was chosen in our two-year RN-to-BSN program curriculum was because its application and utilization play an important role in clinical practice, and the majority of our graduates’ work in medical or surgical wards. Thus, more in-depth course work from an ADN level is necessary, and this may help students have a better understanding of the cultural, economic, and social issues affecting patients.
Before the flipped classroom was implemented, we held several faculty-training sessions on L campus to ensure consistency in content and teaching materials, including quiz questions used in TBL (in-class), the simulation scenario (in-class), the assignment format (post-class), and reading materials (pre-class). The adult-health nursing course is a 36-hour, two-credit course taught during the first semester of the first academic year at both campuses. In terms of the content of the adult-health nursing course, we divided 36 hours into five blocks for five topics based on the leading causes of death in Taiwan: diabetes mellitus, chronic obstructive pulmonary disease, acute coronary syndrome, stroke, and cancer. We adapted an in-depth approach to course contents by integrating pathophysiology, physical assessments, nursing care, and psychosocial issues. Each topic was addressed in four phases.
The first phase was an online self-directed learning pre-class phase in which students reviewed assigned reading materials via the online e-campus platform. Two weeks before the course began, the instructor was asked to upload reading materials (e.g., syllabus, related papers or videos, case descriptions for simulation exercises, assignments, and reflection and evaluation forms) on e-campus. The second phase was a face-to-face TBL in-class activity.
The two- to four-hour TBL process began with an individual quiz, followed by a group discussion, and ended with an appeal or argument process. The third phase was a face-to-face simulation activity in a laboratory. There was a two- to four-hour simulation exercise related to the TBL content, with a case based on actual clinical practice. The last phase involved completing a post-class assignment on e-campus. The students were asked to submit their completed assignments, reflection reports, and course evaluations, as well as any comments they had concerning the e-campus platform. The e-campus platform also facilitated interaction, discussion, and announcements.
In the first phase, two weeks before the course began, it was optional for the instructor to upload adult-health nursing course teaching materials or videos online and the students were not required to review materials or videos before the course began. The second phase was face-to-face traditional classroom teaching during which the instructor delivered the knowledge mostly using slides. The third phase was a face-to-face simulation activity (at least 1 simulation activity). The students participated in the classroom and laboratory activities in both face-to-face phases. The last phase, the instructor competed a written evaluation of the course in the form of a written report and/or revised the course design for next semester. The students were asked to submit homework reports and evaluate the course. Figure 1 shows the flipped classroom and traditional teaching procedures.
Students’ demographic data, including age and gender were collected. To understand the learning outcomes of the flipped classroom, particularly regarding the students’ mental self-evaluation processes (also referred to as metacognitive ability) and self-directed learning skills the following instruments were used .
Self-Evaluated Core Competencies Scale
The Self-Evaluated Core Competencies Scale (SECC) includes eight core competencies stipulated by the Taiwan Nursing Accreditation Council . The SECC contains 55 items, grouped into two sections and eight subscales. The humanity/responsibility section includes four subscales: caring (6 items), ethics (9 items), accountability (7 items), and life-long learning (5 items). The cognitive/performance section also includes four subscales: communication and teamwork capability (6 items), critical thinking and reasoning (5 items), general clinical skills (9 items), and basic biomedical science (5 items). An additional three items measure overall competence, confidence conducting clinical practice, and the ability to adapt after graduation. An 8-point Likert scale, ranging from 0 (cannot assess) to 7 (excellent competence), is used to indicate the level of competency. Higher scores indicate higher levels of competency, ranging from 0 to 385 points. The SECC showed a good Cronbach’s alpha of .80. Cronbach’s alphas for the humanity/responsibility section (.81), cognitive/performance sections (.63), and eight subscales (ranging from .63 to .81) all demonstrated good internal consistency as well .
Metacognitive Inventory for Nursing Students
The Metacognitive Inventory for Nursing Students (MINS) developed by Hsu  was used to measure the association between participants’ knowledge and their awareness of their own thoughts and behaviors. The MINS includes 28 items and five subscales: self- monitoring (7 items), self-modification (7 items), self-awareness (6 items), effective learning (3 items), and problem solving (5 items). Scores are measured using a 5-point Likert scale ranging from 1 (never) to 5 (always), with higher scores indicating higher metacognitive ability (ranging from 28 to 140). The MINS demonstrated good internal consistency, and Cronbach’s alpha was .94 for the total scale and ranged from .73 to.90 for the five subscales, explaining 53.09% of the variance .
Self-Directed Learning Readiness Scale
The Self-Directed Learning Readiness Scale (SDLRS) used in this study was adapted from Tang  to examine participants’ readiness to perform self-directed learning. The SDLRS contains 36 items grouped into six subscales: effective learning (6 items), love of learning (7 items), learning motivation (5 items), active learning (9 items), independent learning (5 items), and creative learning (4 items). A 5-point Likert scale ranging from 1 (never) to 5 (most of the time) was used, with higher scores indicating higher trends of self- directed learning (total scores range from 0 to 180 points). Cronbach’s alpha was .92 for the total scale, explaining 54.33% of the variance. For each subscale, Cronbach’s alphas ranged from .70 to .88, for this study.
Flipped classroom satisfaction questionnaire
A satisfaction questionnaire was developed exclusively for this study. This questionnaire has 35 items grouped into four subscales: teacher’s teaching (14 items), course content (8 items), learning environment (10 items), and administrative service (3 items). The 35 items measuring satisfaction were scored using a scale ranging from 1 (totally disagree) to 5 (totally agree), with higher scores representing higher levels of satisfaction. In current study, the satisfaction questionnaire demonstrated a good internal consistency with Cronbach’s alphas of .98, .95, .94, .95, and .82 for the total scale, teacher’s teaching subscale, course content subscale, learning environment subscale, and administrative service subscale, respectively.
This study was approved by an institutional review board (IRB No. 104-5709C) before data collection. During the first week of the adult-health nursing course, either the principal investigator or co-investigator explained the study’s purpose, as well as the procedures regarding the distribution of the questionnaires, to potential participants. For the entire semester, the EG students on L campus received the flipped classroom method, as shown in Figure 1, while those in the CG received traditional teaching methods (see Figure 1). Before (pre-test) and after (post-test) the adult-health nursing course, students in both groups completed the SECC, MINS, and SDLRS questionnaires. The flipped classroom satisfaction questionnaire was administered only to those in the EG.
We used the generalized estimating equation (GEE) model to test the intervention’s possible effect on the SECC, MINS, and SDLRS scores. Each GEE model included a main effect of group (EG vs. CG), a main effect of time (post-test vs. pre-test), and a two-way interaction effect of group by time. The parameter estimate of the two-way interaction effect indicates group differences concerning the change from the pretest to the post-test. Data analysis was performed using SPSS 22 (IBM SPSS, Armonk, NY: IBM Corp).