To confirm the homogeneity of the data, a descriptive statistic was run in SPSS 25 among the pretest and posttest of both control and experimental groups. The mean of the control group (N = 72) increased from 47.750 (SD = 15.513) to 53.833 (SD = 14.931), while in the experimental group (N = 72) improved from 37.527 (SD = 11.105) to 46.694 (SD = 15.224). Skewness and Kurtosis values in all data ranged between + 2 and − 2 (Skewness = .299, .096, .279, .517; Kurtosis = − .840, − .657, .342, − .136), so the data were homogenous and could be analyzed using parametric tests.
Table 2
Results of Descriptive Statistics
Test | N | Min | Max | M | SD | Skewness | Kurtosis |
Value | SE | Value | SE |
C Pre- | 72 | 20.00 | 82.00 | 47.750 | 15.513 | .299 | .283 | − .840 | .559 |
C Post- | 72 | 24.00 | 86.00 | 53.833 | 14.931 | .096 | .283 | − .657 | .559 |
E Pre- | 72 | 8.00 | 66.00 | 37.527 | 11.105 | .279 | .283 | .342 | .559 |
E Post- | 72 | 20.00 | 90.00 | 46.694 | 15.224 | .517 | .283 | − .136 | .559 |
Flipped Classroom with Gamified Technology for Teaching Vocabulary
The results of Paired sample t-test in the experimental group showed a significant difference between the pretest and posttest in gamified technology classes (Sig. = .001) with a mean improvement of 6.083 (SD = 14.219, t = -3.630, df = 71). The Pearson coefficient indicated a moderate correlation between the pretest and posttest (r = .564). The effect size resulted in a medium effect between the pretest and posttest (d = .427). The results imply that gamified technology has effects in improving students’ learning outcomes with medium effect size railway polytechnic students.
Table 3
Results of Paired Sample t-Test on Experimental Group
Paired Sample Statistics | |
Test | M | N | SD | SE Mean | r | |
All Pre- | 47.750 | 72 | 15.513 | 1.828 | .564 | |
All Post- | 53.833 | 72 | 14.931 | 1.759 | |
H Pre- | 68.190 | 21 | 5.6534 | 1.233 | .153 | |
H Post- | 65.142 | 21 | 12.9548 | 2.826 | |
L Pre- | 39.333 | 51 | 7.0439 | 1.016 | .371 | |
L Post- | 49.176 | 51 | 14.6175 | 2.109 | |
Paired Sample Test | |
| Paired Differences | t | df | Sig. (2-tailed) | d |
Pair Pretest & Posttest | M | SD | SE Mean | 95% Confidence Interval of the Difference |
Lower | Upper |
All | -6.083 | 14.219 | 1.675 | -9.424 | -2.741 | -3.630 | 71 | .001 | .427 |
H | 3.047 | 13.320 | 2.906 | -3.016 | 9.111 | 1.048 | 20 | .307 | .228 |
L | -9.843 | 12.911 | 1.807 | -13.474 | -6.211 | -5.444 | 50 | .000 | .762 |
Based on proficiency level, high-proficiency learners lessened their learning outcome with a mean reduction of 3.047 (SD = 13.320, t = 1.048, df = 20). The results of paired sample t-test showed no difference between the pretest and posttest (Sig. = .509) with weak correlation (r = .153) and small effect size (d = .153). These results reveal that gamified technology has no effect on students’ learning outcomes toward high-proficiency students. On the other hand, low-proficiency learners improved their scores significantly with a mean upgrade of 9.843 (SD = 12.911, t = -5.444, df = 50). The paired sample t-test presented a significant difference between the pretest and postest (Sig. = .000) with weak correlation (r = .371) and medium effect size (d = .762). These results demonstrate that gamified technology significantly affects low-proficiency students’ learning outcomes with a medium effect size.
Flipped Classroom with Paper-Based Method for Teaching Vocabulary
A Paired sample t-test was run in the control group to know the effect of the paper-based method for vocabulary teaching in a flipped classroom setting. The results showed that the paper-based method had a significant difference in the pretest and posttest (Sig. = .000) with a mean improvement of 9.166 (SD = 16.913, t = -4.599, df = 71). The Pearson coefficient described a very low correlation between the pretest and posttest (r = .085) with medium size effect (d = .541). The results of the analysis reveal that the paper-based method has effects to improve students’ scores on vocabulary teaching in flipped classroom settings toward railway polytechnic students in general.
Table 4
Results of Paired Sample t-Test on Control Group
Paired Sample Statistics | |
Test | M | N | SD | SE Mean | r | |
All Pre- | 37.527 | 72 | 11.105 | 1.308 | .085 | |
All Post- | 46.694 | 72 | 15.224 | 1.794 | |
H Pre- | 49.583 | 24 | 7.198 | 1.469 | .040 | |
H Post- | 49.416 | 24 | 16.349 | 3.337 | |
L Pre- | 31.500 | 48 | 7.043 | 1.016 | .858 | |
L Post- | 45.333 | 48 | 14.617 | 2.109 | |
Paired Sample Test | |
| Paired Differences | t | df | Sig. (2-tailed) | d |
Pair Pretest & Posttest | M | SD | SE Mean | 95% Confidence Interval of the Difference |
Lower | Upper |
All | -9.166 | 16.913 | 1.993 | -13.141 | -5.192 | -4.599 | 71 | .000 | .541 |
H | .166 | 14.825 | 3.026 | -6.093 | 6.426 | .055 | 23 | .957 | .011 |
L | -13.833 | 16.057 | 2.317 | -18.495 | -9.170 | -5.969 | 47 | .000 | .861 |
Further, the analysis indicated different results when it was analyzed based on the students’ level of proficiency. High-proficiency learners enhanced their scores by very low points (.166), while low-proficiency learners enhanced higher points (13.833). The paper-based method had no difference on high-proficiency learners (Sig = .957, SD = 14.825, t = .055, df = 23) between the pretest and posttest with a very low correlation (r = .040) and very small effect (d = .011). On the other hand, the method had a significant difference in low-proficiency learners (Sig = .000, SD = 16.057, t = -5.969, df = 47) between the pretest and posttest with very high correlation (r = .858) and large small effect (d = .861). These results reveal that the paper-based method impacts low-proficiency learners to enhance their vocabulary scores, yet, it has no effect on high-proficiency learners.
Comparison of Flipped Classroom with Gamified Technology and Paper-based Method
Comparing the test results of experimental and control groups in one-way ANOVA showed a significant difference in students’ vocabulary learning outcomes (Sig = .000). The students got different results in vocabulary teaching using gamified technology and paper-based method. The results were also found across high-proficiency learners of both groups (Sig = .001). It implies that there was a significant difference in their vocabulary learning outcomes among high-proficiency learners. However, across low-proficiency learners of experimental and control groups showed no difference in their average scores (Sig = .172). It reveals that low-proficiency students had no different results in learning using gamified technology or paper-based method. The results were supported by Tukey HSD analysis, which described no difference among low-proficiency students from the control and experimental groups and high-proficiency students from the control group. High-proficiency learners from experimental groups had different results among other groups of learners.
Table 5
| ANOVA |
| | Sum of Squares | df | Mean Square | F | Sig. |
All | Between | 5893.489 | 3 | 1964.496 | 9.743 | .000 |
| Within | 28228.483 | 140 | 201.632 | | |
| Total | 34121.972 | 143 | | | |
H | Between | 2769.906 | 1 | 2769.906 | 12.532 | .001 |
| Within | 9504.405 | 43 | 221.033 | | |
| Total | 12274.311 | 44 | | | |
L | Between | 365.214 | 1 | 365.214 | 1.892 | .172 |
| Within | 18724.078 | 97 | 193.032 | | |
| Total | 19089.293 | 98 | | | |
Tukey HSDa,b | |
Groups | N | Subset for alpha = 0.05 | |
1 | 2 | |
L C | 48 | 45.333 | | |
L E | 51 | 49.176 | | |
H C | 24 | 49.416 | | |
H E | 21 | | 65.142 | |
Sig. | | .672 | 1.000 | |
Overall, the control group students got higher improvement (M = 9.167) than the experimental group (M = 6.083). Based on proficiency levels, the highest improvement was in low-proficiency learners in the control group (M = 13.833), followed by low-proficiency learners in the experimental group (M = 9.843). Nonetheless, there was no improvement in the high-proficiency learners’ group. The analysis showed that the results slightly deteriorated for high-proficiency students in the control group (M = − .167), and the peak decrease happened in high-proficiency students in the experimental group (M = -3.048). These results reveal that low-proficiency learners achieve higher scores in the paper-based method rather than in gamified technology during the teaching and learning process in flipped classroom settings. While high-proficiency learners could maintain their scores in the paper-based method, their average scores were decreased in gamified technology. However, high-proficiency learners still got higher results than low-proficiency students in both control (paper-based) and experimental (gamified technology) groups.
Teachers’ Perceptions
Thematic analysis was done on the results of teachers-guided reflective writing. The results were categorized into two main themes based on the treatment given to each group of students: gamified technology and paper-based method. Each theme consisted of four elements: process, effect, problem, and alternative solution. Three teachers were coded as T and a continuous number (T1, T2, and T3). At last, the results of the thematic analysis were summarized and checked with the previous findings. The details are presented below.
Flipped classroom with gamified technology
Flipped classrooms with gamified technology allowed students to study before class so they were prepared when they arrived. Further, the students could learn independently instead of creating learning habits massively, which led to active learning. Gamified technology is considered to improve learners’ enjoyment during the teaching and learning process.
T2: The impacts of this method are: students can learn independently, create learning habits massively, feel curious about new things, and create contextual experiences, and this is student-centered learning where the students involve actively in the process of learning.
Although this method was assumed to help teachers and students in vocabulary teaching and learning, some challenges arose. It was especially on the internet connection problems since the students were not allowed to bring their mobile phones to the classroom. The students depended on the Wi-Fi connection during the classroom learning process, which sometimes caused trouble. Hence, it was suggested to improve the internet connection services and give the students free access to their mobile phones to support their learning process.
T3: However, the things that matter related to digital things are the connection to the internet. Sometimes, if the internet connection is low, the students cannot access the material given. In this case, the students are also prohibited from using their cell phones, and the internet connection is only allowed in classes and certain specified places. Thus, the students could not access it anytime they wanted due to limited internet connection.
T1: As today is in the era of society 5.0, I think the students should get free access to their phones in order to support their learning experience outside the classroom, instead of improving internet connection services so that online learning materials can be easily accessible for them.
Flipped classroom with paper-based method
Since the students had difficulties accessing the material in gamified technology due to the internet connection, the teachers thought the paper-based method could be an alternative solution. However, the paper-based was a conventional method that could create students' boredom. Thus, teachers had to monitor students’ progress when implementing flipped classroom model, both in gamified technology or paper-based method.
T1: Flipped classrooms help Z-generation students learn English since they like using the internet. In the absence of the internet, paper-based material may work. These strategies are unlikely to succeed when students' motivation and self-regulation are poor. Flipped classrooms require teacher support.
Learning through the paper-based method needed extra work for both students and teachers. The students had to keep the paper which was easily broken, while the teachers were encouraged to motivate the students in order to achieve learning goals.
T2: Paper is not durable; it leads to breaking into pieces once it catches water/rain.
T3: Since the paper-based method is conventional, the teacher/lecturer needs to motivate students and analyze their progress toward learning goals.
Effective method for students with different proficiency levels
Learning in the flipped classroom setting is new and unconventional as well as challenging. Teachers have to do more preparation for the materials given to study outside the classroom before class. Having team teaching and collaboration with other English teachers might help to reduce the challenge. Yet, the students have to be motivated to have high self-regulation to learn outside the classroom to fulfill the learning goals. After teaching for 10 weeks, the teachers concluded that high-proficiency students could learn effectively with gamified technology and paper-based methods. However, gamified technology is supposed to be more effective. On the other hand, low-proficiency students were more effective in learning vocabulary in the paper-based method since internet connection problems and limited access to mobile phones became the main issues. Learning through paper would lessen their challenges as they did not need additional effort to open online materials.
T1 and T2: The game-based method seems more fun and effective for high-pro students, while low-proficiency students use paper-based methods.
T3: In my opinion, both methods could be given to high-proficiency students, but gamified technology is more challenging than paper-based. The students can use the internet and meet the new material given in different ways than usual. It will motivate them to join the method given. For low-proficiency students, the flipped classroom method is challenging and needs more effort if the students do not meet the maturity level of learning and have low self-regulation. Studying in a paper-based method will be less effort for them since it does not cause any confusion in implementing it rather than in gamified technology.