Correlation Outcomes
For the first research question, among the 440 survey responses, 234 Points achievers completed all 19 questions in the survey; therefore, their engagement scores have been sorted out for correlation analysis. The mean (SD) skills engagement, emotional engagement, participation engagement, and performance engagement scores for Points achievers were 23.94 (3.565), 19.62 (3.031), 20.74 (4.61), 7.77 (1.463), respectively (Table 2).
Table 2 Statistical analysis of mean OSE subscale (skills engagement, emotional engagement, participation engagement, and performance engagement) scores for Points achievers (n=234)
|
Mean (±SD)
|
Minimum value
|
Maximum value
|
Skills engagement (6–30)
Emotional engagement (5–25)
Participation engagement (6–30)
Performance engagement (2–10)
|
23.94
(3.565)
19.62
(3.031)
20.74
(4.61)
7.77
(1.463)
|
6.0
5.0
6.0
2.0
|
30.0
25.0
30.0
10.0
|
Of all 440 participants, only 86 who acquired at least one Badge have been sorted out for the correlation study. The mean (SD) skills engagement, emotional engagement, participation engagement, and performance engagement scores for Badge achievers were 19.62 (8.688), 15.47 (8.439), 15.98 (10.421), 5.79 (3.726), respectively (Table 3).
Table 3 Statistical analysis of mean OSE subscale (skills engagement, emotional engagement, participation engagement, and performance engagement) scores for Badge achievers (n=86)
|
Mean (±SD)
|
Minimum value
|
Maximum value
|
Skills engagement (6–30)
Emotional engagement (5–25)
Participation engagement (6–30)
Performance engagement (2–10)
|
19.62
(8.688)
15.47
(8.439)
15.98
(10.421)
5.79
(3.726)
|
0
0
0
0
|
30.0
25.0
30.0
10.0
|
Significant positive correlations between Points and the OSE score were detected (Table 4). As it shows, the coefficients between the score of Points and the four domains of the OSE (put in sequence: Skills engagement, Emotional engagement, Participation engagement, and Performance engagement) were 0.146 (p<0.05), 0.274, 0.248, and 0.293 (p<0.01), respectively.
Table 4 Spearman’s correlation coefficients between Points and OSE subscale (skills engagement, emotional engagement, participation engagement, and performance engagement) scores (n=234)
|
Skills engagement
|
Emotional engagement
|
Participation engagement
|
Performance engagement
|
Average Points
|
0.146*
|
0.274**
|
0.248**
|
0.293**
|
*Correlation is significant at the .05 level (2-tailed)
**Correlation is significant at the .01 level (2-tailed)
Meanwhile, only one significant positive correlation between Badges and four domains of the OSE was found. The coefficient between the Badges and Participation engagement was 0.225 (p<0.05), and no significant correlations have been observed between the Badges and the rest of the SOE domains (Table 5). These results have answered RQ1.1 and RQ1.2.
Table 5 Spearman’s correlation coefficients between Badges and OSE subscale (skills engagement, emotional engagement, participation engagement, and performance engagement) scores (n=86)
|
Skills engagement
|
Emotional engagement
|
Participation engagement
|
Performance engagement
|
Total Badges
|
0.094
|
0.194
|
0.225*
|
0.054
|
*Correlation is significant at the .05 level (2-tailed)
To answer RQ2, the impact of gamification on online students’ engagement was further explored by conducting a textual analysis of online students’ perceptions of gamification, which were reflected in their responses to the three open-ended questions. Examples of evidence for coding have been illustrated in Table 6. The textual analysis has triangulated the statistical results and cross-validated the research findings.
Table 6 Examples of perception regarding gamification impacts
Themes
|
Codes
|
Definitions
|
Quotes
|
Positive perceptions
|
Skills Engagement
|
What students do (i.e., staying up on readings, `listening/reading carefully)
|
“seeing an increase in my anar seeds give me motivation to keep studying and browsing through study materials”
|
Emotional Engagement
|
How connected or applicable students feel to the course/content (i.e., applying course material to their lives, really desiring to learn the material)
|
“The Anar seeds force me to think of new questions to ask that haven't been posed by other students”
|
Participation Engagement
|
Students interact with others to enjoy the content/course ((i.e., participating actively in small-group discussion forums, helping fellow students)
|
“Anar seeds, competition among learners, which can encourage them to work together and support one another”
|
Performance Engagement
|
Students’ desire/goal to succeed in the course ((i.e., getting a good grade, doing well on tests/quizzes)
|
“i love anar seeds it gives me extra credit in my class”
|
Negative perceptions
|
Skills Engagement
|
What students do (i.e., staying up on readings, listening/reading carefully)
|
“Tbh.I don’t really care about those things ,it’s all about self discipline”
|
Emotional Engagement
|
How connected or applicable students feel to the course/content (i.e., applying course material to their lives, really desiring to learn the material)
|
“No, it just give me pressure to add more words when im submitting assignment when i didnt need to”
|
Participation Engagement
|
Students interact with others to enjoy the content/course ((i.e., participating actively in small-group discussion forums, helping fellow students)
|
“While I compete with myself, I am not a competitive person in relation to others”
|
Performance Engagement
|
Students’ desire/goal to succeed in the course ((i.e., getting a good grade, doing well on tests/quizzes)
|
“I feels like this do nothing for me but if tis can be one of the grade for the certificate would be great”
|
Positive Perception of Points on Online Learners’ Engagement
Perception of Points on Skills Engagement
Serving as the extrinsic reward for online learners’ desired behaviors, such as quality posting, reflecting, and contributing to the course dialogue, Points can be effective in providing positive reinforcers for learning (Kapp, 2012). Students could accumulate Points by conducting or finishing specific online activities. Consequently, their learning behaviors are influenced or altered during the Points collection process.
Some students responded that Points motivated them to study regularly or form an active learning habit by reflecting that “seeing an increase in my Points gives me motivation to keep studying,” “It gave me a motivation to collect more therefore I’m more often on CN scrolling through notes,” and “Anar Seeds…keeps one motivated to keep studying through CN.” The application of the rewarding mechanics of Points proved to be effective in maintaining online students’ regular or consistent visits to either the online platform or their learning material, thus helping keep their online course retention rate.
Notably, a regular visit to either the online platform to check on their status or a timely review of course-related notes could be perceived as a form of self-regulation. Self-regulation, or is often thought to be the same thing as self-control, has been defined by Vohs and Baumeister (2011) as the “overriding of one action tendency in order to attain another goal” (page 3). It has been claimed that effective self-regulation can foster health-promoting behaviors (Fuhrman & Kuhl, 1998), positive psychological well-being (Baumann et al., 2005), and high job performance (Diefendorff et al., 2000). On the other hand, as Kuhl et al. (2006) pointed out, it is not easy to put self-regulation theory (SRT) into practice. The application of gamification (Anar Seeds) in promoting online students’ skills engagement, or to be more specific, the construct of self-regulation, can be viewed as one practical strategy to put SRT into practice in an online learning context.
Perception of Points on Emotional Engagement
Emotional engagement talks about online students’ endeavors about their studies, their willingness to connect what they have learned to their lives, and their desire to learn. Based on these emotional engagement indicators, specific responses have been identified as positive perceptions of Points impacts on online learners’ emotional engagement. Some students replied that Points help them “work hard” to get more seeds and increase their learning willingness, whereas they did not clarify what the “work hard” behaviors look like or to what extent their learning willingness can be. One student mentioned his desire by stating, “The Points force me to think of new questions to ask that have not been posed by other students.” In other words, Points positively impacted students’ desire to learn more by asking novel questions in their study.
One aspect that has appeared repetitively in almost one-third of all the responses concerning Points’ positive impacts on online students’ engagement is the “availability of tracking.” Nine out of 32 online students who reflected their positive perceptions about Points mentioned that Points are practical in helping them keep on track of online learning endeavors, such as their study progress, effort spent on each course, and estimation of the workload in the courses. Online students also appreciated the ease of tracking progress and the enjoyable process of tracking points accumulation; however, tracking availability did not directly relate to the emotional engagement indicators.
The theoretical implications hidden beneath these responses cannot be ignored. Participants also mentioned that seeing the accumulation of Points can help them “get an idea about my current position,” “shows the work I have done…make me happy,” and “Anar Seeds…view my power.” Seen from these perceptions, Points were found to elicit participants’ sense of accomplishment by showing them up-to-date progress to strengthen their self-efficacy. They can also provide the confidence to achieve learning goals reflected as their personal “power,” which can increase the participants’ emotional engagement and overall desire to learn more.
Perception of Points on Participation Engagement
According to Dixon (2015), participation engagement refers to online learners’ active and diversified involvement in different learning activities, including online chats, discussions, online conversations, and forums, as well as their social interactions with their peers, including the willingness to know or to help their peers. Although not many respondents mentioned specifically the impact of gamification on their participation engagement, several distinctive confirming viewpoints were revealed.
One agreed that collecting Points elicited their competitive motivation and monitored “how active I am compared to others.” From this response, it’s interesting that the gamification mechanics impacted online learners’ participation engagement not in cooperation but in competition, where the learner gained self-fulfillment by collecting enough Points to demonstrate his endeavor beyond his peers.
Of course, there are also responses mentioning both competition and cooperation. One response replied, "Points (motivated) competition among learners, which can encourage them to work together and support one another.” It seems that the collection of Points helped encourage cooperation among learners to win certain forms of competition in online learning activities, and gamification can be utilized to either motivate a personal sense of competitiveness or group efforts of active participation.
Also, one response mentioned, "When I need to increase my Anar Seeds, I always try to put a good post or poll because it gives 10 and 5 Points for each.” This is a typical example of how Points can enhance online students’ participation engagement by encouraging them to make active polls or posts to share their thoughts with others. Similarly, another response confirmed that Points can motivate learners to complete tasks and participate in activities as long as the Points are set to be a “tangible goal to work towards.”
From the above, Points are proven effective in motivating online students’ participation engagement by eliciting their sense of competition, willingness to cooperate, and desire to complete tangible tasks. What needs to be taken care of is the potential harm to participation engagement caused by either the over-indulged competitions on pure collections of Points or the inappropriate setting of Points goals, which might sabotage learners’ participation engagement and learning motivation.
Perception of Points on Performance Engagement
In OSE, only two indicators (i.e., a good grade/doing well on tests) have been used to reflect online students’ performance engagement. One response replied that she loved Points because they gave her extra credit in her class. However, she did not detail whether the extra credits will be accumulated to her grade or scores on tests and quizzes; therefore, further exploration is needed, such as semi-structured interviews of participants’ perceptions of how the Points impact their performance engagement.
Negative Perception of Points on Online Learners’ Engagement and Suggestions on Gamification Design Improvement
Only one response mentioned how negative perceptions of Points impacted their specific engagement: “The gamification features do not help with my engagement. It feels like additive work since my instructor requires a certain amount of Points to receive points in class.” Interestingly, the online learner’s psychological resistance to finishing learning tasks was aroused by the instructor but not by the gamification design or its mechanics. According to this perception, earning a certain amount of Points seems to be compulsory work requested by the instructor but not a motivating reward for the learner to earn on his or her willingness. This is a typical example when discussing gamification mechanics and SDT theory (Ryan & Deci, 2000) where engagement is closely linked to satisfying the three psychological needs: autonomy, competence, and relatedness. For this example, the learner’s autonomy has been diminished by losing control over her actions; thus, the learner is less likely to engage willingly and actively. Given this, it should be noted that the instructor can play a vital role in maximizing the gamification mechanics in engaging online students by purposefully setting up reasonable goals and, at the same time, offering students sufficient autonomy so that they can feel a sense of control over their behaviors and the tasks that are about to finish. Otherwise, the impact of gamification mechanics would decrease.
Five respondents suggested improving Points design to facilitate online learners’ engagement in learning activities. One person suggested that more Points should be awarded for high-quality content to motivate students to put forth effort on specific learning materials and enhance their emotional engagement, as indicated in Dixon’s (2015) OSE scale. Three showed their interest in getting a bonus gift or claiming a “virtual currency” after their Points reached specific numbers, or they were willing to be notified once their Points accumulation was updated. These gamification design changes might not directly impact the online learners’ specific engagement. However, there is no doubt that they can help maintain the positive attitudes of online learners toward their course retention and involvement in online activities. Also, one respondent mentioned it would be “helpful if there was more detailed information on what exactly each item is to better understand the Anar Seeds.” This response revealed the importance of making game rules explicit and explaining to students how each gamification mechanic works, as evidenced by Alomari et al.(2019) and Machajewski (2017).
Positive Perception of Badge on Online Learners’ Engagement
Positive Perception of Badge on Participation Engagement
Participation engagement emphasizes social interactions among online students with their peers and instructors during the online study, their interest and desire to know others, and their active involvement in all online discussions. One respondent mentioned, “I do enjoy the badges… I was excited about the content and meeting other like-minded individuals,” thereby showing his intention to reach out to other online students that would promote his engagement to participate in different online activities.
Another reflection confirmed the respondent’s preference for positive badge impacts on her online learning by saying that “Badges, (it) can show to others how I’m active on cn (CourseNetworking).” Interestingly, some online learners keep motivated or engaged in learning activities by showing their achievements to their peers. In contrast to what has been discussed concerning “sense of achievement” in the emotional engagement section, where online learners concentrate on their endeavors and self-fulfillment that does not necessarily rely on connection with other social members, this response indicated the significance of being “exposed” to others, a form of social connection or interaction to help increase the intrinsic motivation which is a crucial driver of engagement—aligned with the Self-Determination Theory (Ryan & Deci, 2000), which suggests that engagement is closely linked to the satisfaction of the three critical psychological needs mentioned earlier: competence, autonomy, and relatedness. Among these factors, relatedness can explain why the badge mechanics can be practical to promote participation engagement as when learners are aware of their achievement being acknowledged by the social community they are involved in, such positive social recognition can make individuals feel encouraged and motivated to engage in their academic endeavors.
Though not many responses mentioned the badge impacts on their participation engagement, the implication of badge impacts on online learners’ participation engagement is worth deeper discussion and interpretation using related learning theories. Notably, badge earning reflected the gamification mechanic impacts on learners’ participation engagement. Of all the 440 responses, 86 reflected that they had earned at least one badge during their online course-taking. 181 badges in total have been earned. The most earned badge types are: “Post of the week” (38 badges, 21% of all badges), “Great post” (27 badges, 14.9% of all badges), “Academic Integrity” (15 badges, 8% of all badges), “ePortfolio Appreciation” (15 badges, 8% of all badges), “Course of the week” (11 badges, 6% of all badges), and “Scavenger Hunt” (11 badges, 6% of all badges). Among these badges, “Post of the week” badge (this badge recognizes CN members whose post received the highest ranking based on peer ratings over the past seven days in their class), “Great post” badge (this badge recognizes CN members whose post was selected as a high-quality post by course instructors), and “Scavenger Hunt” badge (This badge recognizes a student’s participation and/or completion of the Scavenger Hunt activity) are representative badges that record the endeavor of the online learners’ positive participation in online discussions, and active interaction with peers for completing specific online tasks.
In addition, those badges that contribute to the promotion of online learners’ participation engagement, such as the “Best Participant” badge (this badge recognizes active members of a class; seven badges have been earned), “Teamwork” (this badge recognizes learners’ collaboration experience and skills, three badges have been earned) and “Community service” (this badge recognizes contributions to local and global communities, one badge has been earned), though not too many of them have been obtained, put together with badges of “Post of the week,” “Great post,” and “Scavenger Hunt,” have covered 48.1% of all the badges (n=181) that have been earned. From this result and the positive correlation between badges and online learners’ participation engagement, it can be proposed that badges designed to elicit online learners’ involvement in online activities or social interactions can effectively enhance their participation interests and engagement.
Suggestions on Gamification Design Improvement
No negative perceptions concerning badge impacts on specific learners’ engagement were received. However, ten responses suggested how to improve the badge design, which might reflect why badges did not fully contribute to online learners’ engagement. Based on these responses, two major themes have emerged regarding the learners’ expectations of badge design improvement to facilitate their engagement.
Clear Description of Badge Rules
One learner mentioned, “I do not really notice badges, so maybe that feature should be emphasized more,” indicating a lack of awareness toward the gamification elements on the CN platform. The limited impacts of badges on his engagement were not caused by the design flaw but by the ignorance of gamification components that can be applied in learning activities. Likewise, another learner suggested “include a list of all the ways you can earn badges or Points if there isn’t already,” implying that the learners desire to know more about the gamification rules before applying them. Interestingly, this suggestion echoed the suggestion about improving Points, where the learners hoped to better understand how to accumulate Anar Seeds.
Sense of Control Over Earning Badges
Other responses demonstrated learner preferences for multiple or diversified choices of badge earning, which would grant them a great sense of control over choosing specific badges to fit their needs best. For example, one learner expected to have more chances of earning badges once a week instead of passively waiting for specific badges to be issued by the instructors or the system. Interestingly, another learner wanted to be rewarded with specific badges as long as any tests or exams had been passed so that her success could be noticed and appreciated. In addition, one learner was curious whether it is possible to make the badge “redeemable” as Points, which means he could have the autonomy to “trade” them for more tangible awards. Yet another learner suggested “make a higher tier badge” to make it more engaging for competitive learners to choose from and meet their needs to obtain more challenging badges.
These suggestions can be summarized as online learners’ requests for a higher level of autonomy or a greater sense of control over their behaviors on badge earning. Once again, the findings are well in line with the SDT theory (Ryan & Deci, 2000), where engagement is closely linked to the satisfaction of autonomy, one of the three key psychological needs. The more that learners feel satisfied, the more chances there are for them to engage willingly and actively in certain goals.