Evaluation of a PlayPosit Guided Group Project’s Impact on Student Engagement in an Undergraduate Course

doi:10.21203/rs.3.rs-2744177/v1

Download PDF

Research Article

Evaluation of a PlayPosit Guided Group Project’s Impact on Student Engagement in an Undergraduate Course

https://doi.org/10.21203/rs.3.rs-2744177/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

Background

Student engagement is critical for meaningful learning and can be impacted by teaching methods, tools, and course content. Online student engagement continues to be challenging because they have fewer ways to engage with professors, peers, and the university. Our study aimed to evaluate how a PlayPosit guided group project impacted engagement in a structured online learning environment.

Methods: For this qualitative study, a PlayPosit guided group project was implemented in an online undergraduate course taught via Canvas learning management system. Engagement was evaluated using online course analytics and student’s written feedback. Basic Canvas reports were analyzed using descriptive statistics and students’ written feedback was analyzed using directed content analysis.

Results: All 29 students enrolled in the course completed the course successfully. They completed 99.3% of the PlayPosit activities with accuracy 87.6%-100% and completed the course project with scores 89%-100%. Student comments were grouped into themes related to PlayPosit, group project, and engagement. Students found PlayPosit activities helpful/beneficial, aligned with deliverables, a great learning resource, and enjoyable. Students recognized the group project for its importance in teaching them collaboration, expressed that it enabled them to learn more from each-other, was an enjoyable networking experience, and was challenging. Engagement themes included knowledge gained was applicable to the real world, the course stimulated higher-order thinking, and the course was enjoyable and kept students wanting more.

Conclusions: This study suggested that a PlayPosit guided group project was well-received by students and contributed to high engagement with the content, peers, and the professor.

PlayPosit

group project

undergraduate

online engagement

evaluation

community of inquiry

Engagement of students with their studies at the undergraduate level continues to be a topic of discussion and concern, especially in fully online courses. With online learning expanding in the United States at a greater rate than any other segment in higher education (Paulson & McCormick, 2020), maintaining student engagement in technology-mediated learning such as through an online Learning Management Systems (LMS) is even more important, but at the same time, can be difficult (Alqurashi, 2019; Henrie, Halverson & Graham, 2015). Teaching methods and technology may contribute to the level of engagement. Our study explored the combination of PlayPosit and a group project, to improve student engagement in a fully online undergraduate course.

The term student engagement can mean many things, and there is no one uniform standard definition (Christenson, Reschly & Wylie, 2013). Everett (2015) utilized a definition of engagement as the intention of the student to participate in a learning activity, although did acknowledge additional descriptors such as observable behavior related to learning content, internal cognition, embracing active learning, participation, and communication. Student engagement is associated with completion of course tasks, active thinking, talking (in synchronous meetings), and interacting with the course content, classmates, and the instructor (Dixson et al., 2017; Kehrwald, 2008; Kuh, 2003; Tsay, Kofinas, & Luo, 2018). Himmele and Himmele (2017) point out that student engagement is more than entertaining students and getting them to participate in a lesson. In order for the content to be retained, it has to matter and involve higher-order thinking. These descriptions of engagement are aligned with the Seven Principles for Good Practice in Undergraduate Education, which include student–faculty contact, cooperation among students, active learning, prompt feedback, time-on-task, high expectations, and respect for diverse talents and ways of learning (Chickering & Gamson, 1987). Given the strong relationship to student engagement, those principles are commonly applied to online learning (Chickering & Ehrman, 1996). Engaging students in online classes may not be very different from the face-to-face classroom from a principles perspective; however, the tools and processes vary and they could make a difference.

There has been an increase in the number of online programs across colleges and universities around the country (Seaman, Allen, Seaman, 2018) and the COVID pandemic also forced educational institutes to relay on online courses more than before. Online courses are often criticized for a lack of instructor-student interactions (Rockey & Saichaie, 2020) and generally, online students are found to have fewer ways to engage with their professor, peers, and the university (Platt, Raile & Yu, 2014). Educational videos are increasingly becoming the mode of delivering course content, but the main challenge is to stimulate students to watch these videos conscientiously (Scager, Boonstra, Koster, 2020). In terms of engagement with peers, the typical online assessment is a discussion board where students are asked to respond to their peers in relation to the discussion topic; online group work and collaboration is not widely practiced and when used it is not well-structured (Davidson & Katopodis, 2020; Roberts, & McInnerney, 2007; Lowes, 2014).

PlayPosit (formerly known as eduCanon) is an application used to make interactive videos, known as bulbs (Shahrokni, 2018). The application can be integrated into the LMS, requires only an active internet connection and can run on all platforms making it easier to access for both instructors and students (Shahrokni, 2018). Educators may create their own videos or extract videos from other sources, such as YouTube or Ted Talks, and convert them into an interactive and student focused mini-lessons. Educators can prepare video clips and encourage student engagement by providing context or additional information through text and images on slides, inserting questions to check for understanding, giving instant feedback on some questions, and even incorporating polling (Kung & ACERT, 2019). Instructors can also include discussion and reflection questions as students are watching the videos. PlayPosit videos may be paused at certain intervals chosen by the instructor to give students an opportunity to respond. PlayPosit settings allow for multiple playback options, including allowing students to rewind, fast forward or retake the activities. The instructor may also incorporate feedback after each question; so, students not only see whether they answered correctly but also understand the rationale.

As more students take online courses, greater efforts are placed in improving access to online content other than text. As per the American Disabilities Act (ADA) compliance, classes should be accessible and include closed captioning of video and audio material or provide a downloadable transcript of the presentation (Harris et al., 2020). PlayPosit activities make it easier to meet the ADA requirements, given that they include closed captions, and allow instructors to incorporate text, visuals, and auditory delivery of content. When using PlayPosit, instructors can manage and monitor the learning process more effectively due to analytics that are available. Analytics show how well students performed, how many times they repeated the activity, and how much time they spent viewing the videos. PlayPosit has been in use for a few years in the US and other countries. For example, PlayPosit videos were used to assess clinical reasoning skills by analyzing scenarios (Bachman, 2020), to provide an engaging bootcamp for pharmacists needing to enhance their understanding of oncology concepts (Horn, Wells & Halford, 2021), as well as to enhance engagement of elementary school children in Taiwan (Su & Chiu, 2020). A study conducted by the physics education research showed that in a lab course, students appreciated the use of video introduction over written prelab materials (Lewandowski & Pollard, 2019).

Group work has been a popular approach for learning and assessment in the traditional setting and is valued by students as an experience that enhances interaction with peers and leads to deeper interactions among students and the professor (Pal et al., 2012). Group work has been used in a more limited manner in the online environment, but when used, it has been found to build community and engagement (Williams, Cameron, and Morgan, 2012). Additionally, complex, meaningful online group projects have been found effective and contributing to better learning experiences (YunJeong, Chang & Hannafin, 2015; Brunner & Smallwood, 2017; Du, Xu, Wang, Sun & Liu, 2018). Group work is considered beneficial to student learning and engagement but online group work presents challenges related to group communication, logistics, and other administrative aspects. In recognizing such challenges, Chang and Kang (2016) suggest that instructors support groups with such operational tasks and at the same time allow them freedom to organize and control their own content and group deliverables. Davidson and Katopodis (2020) suggest building structure for group work, using real world problems, assigning prework, and providing checkpoints and a timeline for group deliverables.

Online student engagement can be elusive, and is influenced by multiple factors (Muir, et al. 2019). The age of the student, work and family responsibilities, enrollment status (Paulsen & McCormick, 2020), instructor presence in the LMS, how relevant the students felt the content to be (Muir et al., 2019), and desire to interact (O’Shea, Stone & Delahunty, 2015; Crosslin et al. 20XX) can all influence student engagement. O’Shea, Stone, and Delahunty (2015) received specific feedback from adult learners regarding the return on time-investment of interacting; if the perceived benefit was low the student directed effort elsewhere - whether it be to other aspects of the class or other work/life responsibilities. Some students valued the interaction while others viewed it as an additional burden to the online class. This was especially true if there were other assessments which amounted to a larger portion of the overall grade; student made a strategic choice to devote the time and effort to the assessment rather than interaction and engagement with classmates or faculty. This matches findings by Muir et al. (2019) in which students would sacrifice engagement with the class to focus on other graded assessments.

Multiple studies have found student engagement is associated with positive outcomes. Increased student retention, persistence, performance, and achievement (Paulsen & McCormick, 2020, Han-Huei Tsay et al 2020, Everett, 2015), avoiding feeling isolated (Crosslin, et al., 20XX), increased interaction in courses, student satisfaction, and perceived greater knowledge gained (Anne, Raile & Yu, 2014) are all suggested benefits of student engagement. Baranik, Wright, and Reburn (2017) found that students who developed a connection with even one classmate by engaging in their online course saw an increase in perceived satisfaction, classroom community, performance, and even a difference in the final grades.

Han-Huei Tsay et al (2020) indicated that student engagement represents the observable behaviors in completing a learning activity and divided further those behaviors into engagement in the process and engagement toward a result. A suggested example of process engagement would be measuring how many times a student viewed information in the LMS although it may not have resulted in completing any assignment or other gradable assessments. Distinction between process and results engagement aids in determining how to monitor or assess online engagement. For example, how many times an item in the LMS was viewed is an observable quantitative measure of behavior. Monitoring engagement can be a method to recognize students who may need help in the online environment and can be a valuable tool in evaluating the quality of a class or activity (Henrie, Halverson & Graham, 2015).

As indicated by prior research, encouraging greater engagement is essential in successful delivery instruction utilizing online LMS technology (Henrie, Halverson & Graham, 2015). Strategies for successful engagement need to be matched to the characteristics of online students (Paulsen & McCormick, 2020). Feedback from students indicates designing courses specifically for the LMS (Henrie, Halverson & Graham, 2015), rather than trying to deliver the same in-person course via video or other interface is the best approach (Crosslin, 20XX). In the online classroom, there are approaches and tools used by the faculty which can help. Being accessible, present, and positive can help students feel valued and encourage engagement (O’Shea, Stone & Delahunty, 2015, Crosslin, et al., 20XX, Dixson, et al., 2017). Dixson et al. suggests immediacy behaviors by faculty (communicative verbal or non-verbal actions) which send positive messages of liking and closeness can help in encouraging online engagement. Online nonverbal immediacy behaviors such as using figurative language, colors, images, the timelines of responses, frequency of messages, and promptness in grading and e-mail was related to students’ reports of higher online engagement (Dixson, et al. 2017). An activity combining the benefits of teaching presence and social interactions with an assessment or learning purpose may be the key to bind this student feedback regarding online LMS class needs together. This matches what Arbaugh et al. (2010) titled a Community of Inquiry (COI). The COI framework suggested the teaching presence clearly communicates what learning will occur and communicates in a way which humanizes the instructor. The social presence focuses on the learner and offers low-risk opportunities to express themselves and actively participate and interact in the class. A cognitive presence encourages students to take responsibility for their learning with activities which are meaningful in delivering the content and meeting the objectives of the class.

Implementing a graded semester long group project using the PlayPosit tool as guidance with key concepts could provide an opportunity to address multiple findings from the literature research, including greater interaction with the course content, with the instructor, and among students in a group. A project approach could be the vehicle to help students not only interact with each other but move past what Bloom’s Taxonomy refers to as lower levels of learning like remembering, understanding, and applying concepts to higher cognitive processing like analyzing, evaluating, and creating (Rohrdantz et al., 2014). PlayPosit activities could allow the instructor to monitor students’ exposure to and understanding of the key concepts. The group project could allow the instructor to not only monitor the interactions of students, but also give periodic encouragement and feedback as needed, which has also been found to be beneficial in encouraging online engagement (Lawrence et al., 2019).

Previous research on the topic of engagement has not focused on engagement that may result from the combination of PlayPosit and a group project. The current study approaches the topic of undergraduate online student engagement from a broad perspective by asking the students to share their thoughts and experiences regarding the use of PlayPosit activites, the online group work and the overall engagement with the course. With this evaluative study, we aimed to explore how a teaching method incorporating the PlayPosit tool as a guide to a group project might affect student engagement.

The study was guided by the Community of Inquiry (COI) framework, which explains the conditions for creating a community of inquiry in a virtual learning environment from a perspective of cognitive presence, social presence, and teaching presence (Garrison, Anderson, & Archer, 2000). Cognitive presence is defined as the degree to which learners are able to construct and confirm meaning through sustained communication, reflection and discourse; social presence is defined as the degree to which learners feel socially and emotionally connected with others in an online environment; and teaching presence is defined as the way course is organized and designed, facilitated, and guided in order to achieve meaningful and educationally worthwhile learning outcomes. (Garrison, Anderson, & Archer, 2000, p. 88).

Those three key elements to students learning and education experience need to be evident through unique indicators, which as per framework are categorized in a few groups (Garrison, Anderson, & Archer, 2000, p. 89). The main categories that reveal the cognitive presence include triggering events, exploration, integration, and resolution. Indicators may include exchange of information among students as well as between students and the instructor, sense of puzzlement, connecting and applying ideas. The main categories for the social presence element are emotional expression, open communication, and group cohesion. Some indicators include emotions, risk free expressions, and encouraging collaboration. Last, the main categories for teaching presence are instructional management, building understanding, and direct instruction. Indicators include defining and initiating discussion topics, sharing personal meaning, and focusing discussion.

The focus of our study was student engagement, which is associated with higher order thinking, interaction, communication, cooperation, and quality learning outcomes. Such aspects of learning are also key components of the COI theory. In researching our main question “How a PlayPosit guided group project might impact engagement in an online undergraduate course?” we used COI as the backdrop and looked for examples of indicators mentioned above in the students’ comments, as well as course work completed.

With the implementation of a PlayPosit guided group project, the goals were to improve the interaction of the students with the course content; increase the social presence of the professor through a combination of videos, live sessions, virtual office hours, and timely feedback; increase interaction among students through a small group project requirement; increase the interaction of students with the professor through live sessions and virtual office hours; and improve process engagement and result engagement in the course. Given the investment in PlayPosit, it was important to examine students’ perceptions on whether the chosen teaching strategy would help student learning and engagement. The ultimate expectation is a positive and meaningful learning experience.

This study evaluated student engagement in a 15-week undergraduate-level health information management course. The course name was Performance Improvement and the LMS used was Canvas. A semester long (15 weeks) interactive group project was assessed along with a series of PlayPosit videos which contained focused activities to guide students in the process of completing the course project. The research was approved by the Institutional Review Board at the authors institution.

2.1 Description of the Educational Intervention Teaching Strategies

In the course studied, the cognitive presence element was stimulated by the questions embedded in the PlayPosit activities, discussions, and the requirements for completing the project deliverables. Social presence was facilitated by incorporating a group project and encouraging group collaboration, as well as by providing an opportunity to discuss and ask questions during live sessions. Teaching presence was established by providing a series of PlayPosit videos (also known as bulbs) aligned with the course objectives and the group project requirements, live Zoom sessions, lecture recordings via Kaltura, interaction in the discussion area, and constructive feedback in individual and group encounters.

To evaluate how a PlayPosit guided group project might affect student engagement, we used the instrumental case study approach. In a single instrumental case, one bounded case is selected for the investigation of the research question (Creswell, 2007, p. 74). One junior-level course, Performance Improvement, was selected, given professor’s availability and timing of the study. The course ran for 15 weeks, during the Spring 2021 semester. Participants in the study were the 29 students enrolled in the selected course. The educational intervention included activities performed by the professor and the students, as shown in Fig. 1.

An overarching course project was designed in alignment with the course objectives and in a way that reflected application of the knowledge shared/acquired from week to week. Several project deliverables were due in Weeks 3, 4, 6, 8, 10, 12, and 13. The PlayPosit tool was utilized as a method to provide a greater connection to students while they participated in a meaningful activity. The course included 15 PlayPosit activities with an average length of 6.41 minutes and about 4 questions per PlayPosit. They were prepared in advance by the professor and posted in the respective weekly modules, weeks 3, 4, 6, 8, 10, 12, and 13. There were a total of 80 questions, each worth 1 point, for a total of 80 points (out of 1000 points – potential for the course). PlayPosit content addressed key concepts and included questions to check on the understanding of the concepts explained. Key concepts were a reflection of the course objectives. PlayPosit-embedded questions were aligned with the objectives, and intentionally connected to the course project deliverables.

There were also 19 supplementary (regular) videos pertaining to the course content. Professor offered 11 live lectures during the day and provided recordings in the respective modules and Medica Gallery within 24 hours of the live session. Additionally, professor offered 3 live debates in the evening. Debate sessions provided student who participated with some scenarios and challenged the students on how they would approach addressing the scenarios. Debate sessions were not recorded.

Student groups were determined by the end of Week 1, after considering students’ preferences and students’ background. An effort was made to have at least one student with some healthcare experience in each group. Professor followed up with the entire class in regard with overall instruction, discussions, and feedback in the gradebook. Additionally, the professor communicated with each group when necessary and checked the Canvas group space on a regular basis. Canvas group space ended up being used very little at the beginning of the course, mostly as a space where group members shared contact information and preferred communication methods. After the initial contact in the Canvas group space, groups continued their weekly communication by using tools of their choice, such as Google docs, Facetime, WhatsApp, phone calls, text messaging, and occasionally Zoom meetings.

2.2 Data Collection

Data used in this study includes course analytics and students’ comments. Course analytics such as student numbers, gradebook, PlayPosit “Monitor” function, and Media Gallery “Analytics” function are inherent parts of Canvas online learning platform. At the end of the course, students were asked to participate in an online discussion activity and share their thoughts on PlayPosit, group project, and overall engagement in the course. Those comments were collected for analysis. Last, communication via e-mail, text, calls, and Zoom was tracked by the professor.

2.3 Data Analysis

Analysis included descriptive statistics of the Canvas course analytics, a review of written student comments pertaining to PlayPosit, the group project, and engagement with the course, and exchange of information encounters between the professor and students. Student comments were analyzed using a directed content analysis approach (Creswell, 2007). The data analysis process included reading the student’s comments and coding them based on the words students used to describe PlayPosit, group work, and overall engagement, learning, and course experiences. Codes were aggregated into concept maps in order to group related codes into themes and show relations. The written student comment data were coded by two researchers. Less than 5% coding discrepancies were found, discussed, and resolved with 100% consensus.

3.1 Course Analytics

There were 29 students enrolled in the course studied, and all of them completed the course successfully, with an overall student score ranging from 82.1–98.3%. There were no dropouts. Completion rate for the PlayPosit activities or bulbs was 99.3%. This was calculated based on the number of students (29), number of bulbs (15), and incomplete PlayPosit activities (3). There were 810 total bulb plays/completions, which shows that in most cases, students went back to the bulb for an average completion rate of 1.86 times per bulb. The accuracy of completion varied from 87.6–100%. The course project completion rate was 100% and as per peer reviews, all group members were reported as contributors. Course project scores varied from 89–100%. In terms of communication, there were 162 student e-mails received via Canvas inbox and 77 e-mails received in Outlook, as well as an average of 6 text messages and 2–3 phone calls/Facetime/Zoom meets per week. Communication during these interactions was mostly about seeking suggestions on how to improve group dynamics and discuss ideas and approaches to some of the project requirements. It should be noted that most of the inquiries were made from the group lead. There were 9 groups with 3–4 students each. Group members were copied on the e-mails, and occasionally, they joined the phone call, Facetime discussion or a quick Zoom meeting. Attendance of weekly live session was limited to 4–6 students; however, video recordings were viewed by 22 unique viewers (students). Attendance of live sessions was not mandatory and not graded.

3.2 Students Comments

During the last week of the course, students were asked to participate in an online discussion that asked them to share their thoughts about PlayPosit, the group project, and engagement, 22 out of 29 students posted comments.

3.2.1 PlayPosit Comments. Students were asked to comment on the PlayPosit activities. All 22 students who responded to the discussion were explicit in addressing them and commented in a positive way. Some of the comments were short and some more elaborate. Overall, four themes were identified in relation to PlayPosit.

Students labeled PlayPosit activities as helpful and beneficial. The words “useful”, “beneficial”, “helped”, “helpful”, “very helpful, or “extremely helpful” were mentioned by 20 students. Additional student comments show evidence that PlayPosit activities were helpful and beneficial because they were relevant, served as a learning resource, and were seen as a different way of learning.

Students explained that PlayPosit activities were helpful because of the alignment with what they were learning and how it applied to the required project deliverables. Most of the students (19 out of 22) commented that PlayPosit activities were “informative”, “to the point”, “like a GPS”, “captured essential information”, were “related to the assignments”, “kept students up-to-date”, had “valuable information”, and “helped better understand and gain clarity on what was expected.”

PlayPosit activities were considered a learning resource by half of the students who expressed “helped understand the material on another level”, “questions tested our knowledge”, “loved retaking until understood”, “enforced readings and lectures”, “I would take a second look and take notes”, “repetition helped remember most of the information”, “forced you to think”, “understood the material on another level”, and “used PlayPosit as a resource”. Ten students explicitly pointed out that they liked the embedded tests as a form of assessment and assurance they understood. A few students appreciated the “examples” in the PlayPosit. There was also a suggestion to “continue to use (PlayPosit) for future students”. Students valued the explanations and the mini assessments.

Students appreciated that PlayPosit activities were enjoyable and different from other learning resources. Eleven students expressed that PlayPosits were “enjoyable:, “short”, “quick”, “not overbearing”, “not drawn out or boring”, “made learning fun”, “a unique way of learning”, and “kept students interested”.

3.2.2 Group Work Comments. Students were asked to share their thoughts about the group project as well as challenges associated with it. The 22 students who participated in the discussion commented on the group work and the length of responses to this question was greater than the discussion about PlayPosit or overall engagement. Four main themes were identified in relation to the group work.

The first emerging theme showed that students recognized the importance of collaboration, working with others as a team. While group work challenges were identified first, most (add number) comments ended on a positive note and students valued the opportunity to work together. Students pointed out the benefits of working together lessons learned and resemblance to real work environment, through several comments, including “where you may lack in one aspect of a project, another group member may excel in that area which lifts the group”, “worked to our benefit”, “great insight on what’s it like to work with future co-workers”, “it gave a look into settings that require engagement and collaboration”, “we must collaborate with physicians, … on real life cases”, “learned how to speak with my peers in a professional and respectful manner”, “it taught us how to manage team work”, “sneak peek as to what to expect in the real work force”, “learned from mistakes”, “opportunity to put our skills to practice”, “challenging to establish a workflow and distribution of work in a fair and equal way”, and “confusing at the beginning… then, easy to follow the routine and deliver on schedule”.

The second theme that emerged in relation to the group project was that students learned more because they worked as a group. Students expressed that they “learned from each other”, “helped learn more”, “easier to put our thoughts together and see everyone’s perspective”, “better understanding”, “It helped improve in certain areas to help us working together in the healthcare field with others”, “parts of the task I did not understand – classmates were able to come up with ideas and figure it out”, and “challenging to accept other’s opinions and ideas …BUT … definitely liked working in a group because I got to hear other perspectives on the material”.

A third theme was related to the social aspect of the group project, students enjoyed networking with each-other. Half of the students shared comments such as, “nice to see some of my peers” (given the pandemic situation), “learned about each-other”, “met new people”, “know each-other better”, “getting to know ourselves”, “my group is hardworking and communicative”, “made us closer”, “showing each other our true colors”, “listened to each other”, “enjoyed work as a team”, or “tried to be as helpful and open with my group” illustrate that despite listing challenges, students capitalized on the time together and developed an appreciation and respect for each-other.

Fourth, thirteen students identified lack of time and scheduling as the main challenge, and half of them connected that challenge to the fact that the course is online. Some examples included, “scheduling group time can be difficult”, “we all have different working schedules and learning schedules”, “challenging to find time”, and “group work was time-consuming and tiring”.

3.2.3 Overall Engagement Comments. When asked about the overall engagement with this course and the content, students comments converged on three main themes: the course was enjoyable and kept students wanting more, knowledge gained in the course was applicable to real world, and the course stimulated higher-order thinking.

Knowledge gained in the course was applicable to real world: Half of the students shared positive comments, such as, “learned a lot”, “good learning experience”, “able to communicate with my colleagues with more confidence and knowledge”, “able to use the skills and apply them to my everyday work life”, “made me think like an HIM professional”, “worth it”, “taught me a lot about myself and how to tackle certain situations”, and “prepared us for the real world challenge; so, it helped “.

Course stimulated higher-order thinking: Students’ comments such as “challenging”, “although this course was most challenging this semester, it was also my favorite”, “challenge in applying some of the measurements”, “appreciate jumping out of the comfort zone”, “understand the material on another level, “a bit confusing which required more critical thinking”, “forced you to think”, and “helped see things from a different perspective” illustrate that by completing the course activities, students developed a higher level of thinking.

Course was enjoyable and left students wanting more: Half of the students used words such as, “enjoyed”, “awesome”, “pleasant”, “great”, “favorite”, “kept me interested”, “exciting”, “successful” and “excited for what’s to come” in describing the course. Many expressed they wanted to go back to it or did not like missing parts of it. For example, some students pointed that they “missed the debate” and were “not able to attend live lectures”. Others said, “I would take a second look (at PlayPosit), and take notes”, “really engaged with the course content”, “recordings posted were incredibly beneficial”, “course set up was good”, and “PlayPosit … improved learning engagement”. This theme was also supported by the number of views and plays of the course videos; there were 132 plays in Kaltura. Additionally, PlayPosit analytics showed that many students went back to the activity for a second time.

Course analytics and themes identified from students’ comments seem to align with several aspects of engagement. There was a very high level of task completion, higher-order thinking as expressed by students and seen in their deliverables, learning what matters and applies in the real-world, as well as enjoyable interactions with the group members and the course content, all of which were associated with student engagement (Dixscon et al., 2017; Kehrwald, 2008; Kuh, 2003; Tsay, Kofinas, & Luo, 2018). Based on the interest in completing graded and ungraded tasks, the observed process engagement was high. Based on course completion and the high course scores, result engagement was also high. Overall, students expressed high satisfaction and engagement with the course, which is consistent with a study from Rajabalee and Santally (2021) who found a positive relationship between student satisfaction with a course and their level of engagement.

Themes such as alignment of PlayPosit content with deliverables, course stimulated higher-order thinking, knowledge was applicable to real world, as well as the ongoing purposeful exchange between students and the professor and among students via e-mail, text, phone call, FaceTime or Zoom meets are indicators of cognitive presence component of COI theory (Hsieh & Shannon, 2005).

Themes such as students enjoyed networking with each-other, recognized the importance of working as a group, PlayPosit activities were enjoyable, and course was enjoyable and left students wanting more are illustrations of students’ emotions and positive view of collaboration, which are indicators of social presence component of COI theory (Hsieh & Shannon, 2005). Another sign of social presence was that students felt free and at ease in reaching out to the professor and have open communication individually as a group. On a different note, the volume of student comments pertaining to the learning experience in the course under study was higher that the volume of comments for another course taken by the same group of students and taught by the same professor. The willingness to write about the course is also an indicator of emotional engagement, which supports students’ social presence.

PlayPosit increased the teaching presence of the professor by virtue of having multiple touch points with the material focused on relevant tasks. A typical course has one lecture or a couple of videos per week; the course under study had 15 bulbs in addition to the weekly lectures and/or videos. Themes such as PlayPosit was a learning resource and they aligned well with the project are also an indicator of teaching presence component of the COI theory (Hsieh & Shannon, 2005).

PlayPosit contributed to better interaction of the students with the course content, and as per their comments was helpful and beneficial, as well as interesting and attractive. Furthermore, students were more attentive to the other course videos and recordings. Videos, live lectures and/or recorded lectures are part of many courses taught in our health information management program; however, the number of views in the past or even concurrent courses has been low. To put things in perspective, without considering this a quantitative analysis, in another health information management course involving almost the same group of students and the same professor, and providing students with similar number of weekly videos and recordings (PlayPosit is not counted, since it was a graded activity and only present in the course under study), the total number of views was 50 and the total number of minutes viewed was 502, compared to the course under study which had 132 views and 2047 minutes. This could be related to the positive experience with PlayPosit and the “wanting more” attitude and is something to be explored further in the future.

This study’s purpose was to explore the potential impact of a PlayPosit guided group project on student engagement. The COI theory served as a frame for the course design and infused the course structure and processes. Results reflect that this specific design, utilizing PlayPosit as a guide to the group project was perceived very positively by the students, generated and supported engagement, and created an enjoyable and meaningful learning experience.

Designing an online course for group work remains a challenge (Theodosiou & Corbin, 2020). While there is strong agreement that group work is important in developing teamwork skills, most online professors view group work from a perspective of creating a forum where students can comment on each-other nicely and positively, be supportive, and get to know each-other a little, but not necessarily develop deep relationships among them (Wade, Cameron, Morgan & Williams, 2016). On the other side, from an engagement perspective, social presence means more than just written discussions about each-other or peer reviews of a paper. As per COI theory, it means that students create a meaningful, trusting community and develop inter-personal relationships, which most students in this course stated they achieved, because of the group project. Group projects tend to be intimidating because of the multiple requirements and group facilitation. In our case, PlayPosit helped in overcoming some of the barriers in comprehension and in breaking down the requirements, thus allowing the students to focus more on developing higher-level knowledge, critiquing others, self-reflecting and building consensus among each-other, while enjoying the experience; all of which contributed to a meaningful deeper level of engagement. Based on the findings, we believe that our research question of how a PlayPosit guided group project might affect student engagement was answered favorably. While there are different methods and tools, this is one great recipe for achieving engagement and positive overall experience with the course, that is rooted into the COI theory. With an estimated 37% of students taking online courses (U.S. Department of Education, 2019) before the COVID-19 pandemic, and post-pandemic numbers being higher, finding new tools and teaching recipes that enable greater engagement and yield positive learning outcomes is very important.

Generally, the design of the online course requires some time-investment in assuring that learning materials and assessments are well-aligned with the course objectives, are distributed somewhat evenly among the weeks, and include clear instructions and grading rubrics. For the course used in this study, an effort was made to connect the various parts of the course project in such a way that most of the parts could be completed independently; however, at certain points, students needed to look back and connect the dots. Significant time was spent in preparing the 15 bulbs. Some of them were new recordings by the professor and others were found online. An effort was made to keep the recordings short. At times, this required creating and following scripts to avoid the “fluff” and keeping track of time. Questions embedded in the bulbs were mostly self-graded (multiple choice, multiple selection, true/false, or matching) with only two out of 80 questions being free response/essay; yet, they went beyond the recall and understand levels and made students think, apply the knowledge, and analyze the information (which may require a greater upfront time investment from the professor). Another consideration was to keep the videos free of references to dates, times, order, specific weeks/modules, or even the course project; PlayPosit videos explained the concept and provided examples. Questions on the other side were occasionally related to the course project requirements, in order to make students connect what they were learning to what they need to accomplish later. Given that questions could be easily changed, this set up allows for greater flexibility in using and reusing the bulbs not only when teaching the same course but also in reviewing or referring to the same content in another course. In an effort to embrace diversity, the instructor intentionally searched for and selected videos where the presenters represented different races, genders, and ages. A strong initial bulb pool means much less work the next time the course is taught.

The intention of the study was not to generalize findings but try to better understand how the combination of PlayPosit and a group project might affect student engagement and experience. In today’s online teaching and learning environment, it may be hard to incorporate everything a professor believes is necessary for a great learning experience, such as full interactive live lectures, making students read everything, asking them to engage with their peers in a meaningful way, and focus on the learning outcomes (versus focusing on getting the desired score). Yet, it may be possible to effectively and strategically incorporate some key ingredients that will not only make student learn in a meaningful way but also perceive the course experience positively. PlayPosit promises to be a tool that can help facilitate this effort. Further studies of courses with a PlayPosit component, as well as modification of assignments utilizing PlayPosit as a guide may be helpful in better understanding the effectiveness of PlayPosit on students’ engagement and outcomes. This study involved only one class for one semester, there was no comparison group nor were pre and post intervention data assessed. There were 7 students who did not share any comments which might bias what we found in the comments. Nevertheless, the professor who taught the course and followed up with the students along with the second researcher who was not part of the health information management program completed the analysis and interpretation of written comments, which contributes to increasing validity.

This study builds upon prior online student engagement research and provides an evaluation of how a PlayPosit guided group project might contribute to student engagement and overall learning experience. By implementing a teaching approach such as PlayPosit that incorporates well-aligned assessments and targeted teaching material, it may not only be feasible to achieve meaningful high-level engagement in the online courses but also an enjoyable experience that makes students want more of it. This is just one successful recipe that can be expanded to more courses or tweaked depending on the course content, level, objectives, and expected learning outcomes.

Data Availability Statement

The data generated and/or analyzed during the current study are available from the corresponding author upon reasonable request.

Competing Interests

N/A

Funding

No funding was received for conducting this study. The authors have no relevant financial or non-financial or competing interests to declare that are relevant to the content of this article.

Acknowledgements

N/A

Alqurashi, E. (2019). Predicting student satisfaction and perceived learning within online learning environments. Distance Education, 40(1), 133–148. https://doi.org/10.1080/01587919.2018.1553562
Anne, C., Raile, A. N. W. & Yu, N. (2014). Virtually the Same?: Student Perceptions of the Equivalence of Online Classes to Face-to-Face Classes. MERLOT Journal of Online Learning and Teaching, 10(3). https://jolt.merlot.org/vol10no3/Platt_0914.pdf
Arbaugh, Bangert, A., & Cleveland-Innes, M. (2010). Subject matter effects and the Community of Inquiry (CoI) framework: An exploratory study. The Internet and Higher Education, 13(1), 37–44. https://doi.org/10.1016/j.iheduc.2009.10.006
Bachman, S. (2020). The Use of Video in the Occupational Therapy Graduate Classroom. Journal of Occupational Therapy Education, 4(3). https://doi.org/10.26681/jote.2020.040318
Baranik, L. E., Wright, N. A., &; Reburn, K. L. (2017, July 1). Mentoring relationships in online classes. Internet and Higher Education. Retrieved February 8, 2022, from https://www.learntechlib.org/p/199184/
Brunner, B. R., & Smallwood, A. M. K. (2019). Prioritizing public interest in public relations: Public interest relations. Public Relations Inquiry, 8(3), 245–264. https://doi.org/10.1177/2046147X19870275
Chang, & Kang, H. (2016). Challenges facing group work online. Distance Education, 37(1), 73–88. https://doi.org/10.1080/01587919.2016.1154781
Chickering, A. & Ehrmann, S. (1996). Implementing the Seven Principles: Technology as Lever. American Association for Higher Education Bulletin, 49, 3-6. https://sphweb.bumc.bu.edu/otlt/teachingLibrary/Technology/seven_principles.pdf
Chickering, A. W. Gamson, Z. F. (1987, Mar). Seven Principles for Good Practice in Undergraduate Education. AAHE Bulletin, 3-7.
Christenson, S. L., Reschly, A. L., & Wylie, C. Eds (2013). Research on student engagement. New York: Springer.
Creswell, J. W. (2007). Qualitative inquiry and research design. Choosing among five approaches, 2nd ed. SAGE Publications.
Crosslin, M., Benham, B., Dellinger, J., Patterson, A., Semingson, P., Spann, C., Usman, B., & Watkins, H. (n.d.). Creating online learning experiences: A brief guide to online courses from small and private to massive and open. Mavs Open Press, Arlington TX. https://uta.pressbooks.pub/onlinelearning/
Curry, L. (1991). Patterns of learning style across selected medical specialties. Educational Psychology, 11, 247–277.
Davidson, C. N. & Katopodis, C. (2020). 8 ways to improve group work online. Inside Higher Ed. https://www.insidehighered.com/advice/2020/10/28/advice-how-successfully-guide-students-group-work-online-opinion
Du, Fan, X., Xu, J., Wang, C., Sun, L., & Liu, F. (2018). Predictors for students’ self-efficacy in online collaborative groupwork. Educational Technology Research and Development, 67(4), 767–791. https://doi.org/10.1007/s11423-018-9631-9
Harris, B. N., McCarthy, P. C., Wright, A. M., Schutz, H., Boersma, K. S., Shepherd, S. L., Manning, L. A., Malisch, J. L., Ellington, R. M. (2020). From panic to pedagogy: Using online active learning to promote inclusive instruction in ecology and evolutionary biology courses. Academic practice in ecology and evolution, 12581-12612.
Dixson, M. D., Greenwell, M. R. Rogers-Stacy, C., Weister, T & Lauer, S. (2017). Nonverbal immediacy behaviors and online student engagement: bringing past instructional research into the present virtual classroom. Communication Education, 66(1), 37-53, http://dx.doi.org/10.1080/03634523.2016.1209222
Everett, D. R. (2015). Adding Value: Online Student Engagement. Information Systems Education Journal, 13 (6), ISSN: 1545-679X November
Flick, U., & Flick, U. (2014). The SAGE handbook of qualitative data analysis. SAGE.
Garrison, D. R., Anderson, T., & Archer, W. (2000). Critical inquiry in a text-based environment: Computer conferencing in higher education model. The Internet and Higher Education, 2(2-3), 87-105.
Han-Huei Tsay, C., Kofinas, A. K., Trivedi, S. K., Yang, Y. (2020). Overcoming the novelty effect in online gamified learning systems: An empirical evaluation of student engagement and performance. Journal of Computer Assisted Learning, 36, 128–146.
Henrie, C. R., Halverson, L. R. & Graham, C. R. (2015). Measuring student engagement in technology-mediated learning: A review. Computers & Education, 90, 36-53
Himmele, P., & Himmele, W. (2017). Total participation techniques: Making every student an active learner (2^nd ed.). ASCD.
Horn, B., Wells, L., & Halford, Z. (2021). Oncology boot camp: A preparatory curriculum for advanced pharmacy practice experience students. Journal of Oncology Pharmacy Practice, 1, https://doi-org.proxy.libraries.rutgers.edu/10.1177/10781552211019116
Hsieh, H. F. & Shannon, S. E. (2005, November). Three Approaches to Qualitative Content Analysis. Qualitative Health Research, 15(9), 1277-1288.
Jensen, E., & Laurie, C. (2017). An introduction to qualitative data analysis. SAGE Publications Ltd.
Kanuka, & Garrison, D. R. (2004). Cognitive presence in online learning. Journal of Computing in Higher Education, 15(2), 21–39. https://doi.org/10.1007/BF02940928
Kauffman, H. (2015). A review of predictive factors of student success in and satisfaction with online learning: Association for learning technology journal. Research in Learning Technology, 23. https://doi.org/10.3402/rlt.v23.26507
Kehrwald, B. (2008). Understanding social presence in text‐based online learning environments, Distance Education, 29(1), 89-106. DOI: 10.1080/01587910802004860
Kuh, G. D. (2003). What we’re learning about student engagement from NSSE: Benchmarks for effective educational practices. Change: The Magazine of Higher Learning, 35(2), 24-32. doi:10.1080/00091380309604090
Kung Shiao-Chuan and ACERT. (2019, March 1). PlayPosit: A Remedy for the Passive Viewing of Videos. Academic Center for Excellence in Research and Teaching. https://acert.hunter.cuny.edu/blog/playposit/2019/03/01/
Lawrence, J. & Brown, A., Redmond, P. & Basson, M. (2019). Engaging the disengaged: Exploring the use of course-specific learning analytics and nudging to enhance online student engagement. Student Success, 10, 47-58. 10.5204/ssj.v10i2.1295.
Lewandowski. H. J. & Pollard, B. (2019). Using custom interactive video prelab activities in a large introductory lab course. Physics Education Research Conference 2019 (pp. 312-317). Provo,UT: American Association of Physics Teachers.
Lowes, S. (2014). How much “group” is there in online group work? Online Learning Journal. doi: http://dx.doi.org/10.24059/olj.v18i1.373
Meyer, K. A. (2014). Student Engagement in Online Learning: What Works and Why: Student Engagement Online. ASHE Higher Education Report, 40(6), 1–114. https://doi.org/10.1002/aehe.20018
Muir, T., Milthorpe, N., Stone, C., Dyment, J., Freeman, E. & Hopwood, B. (2019). Chronicling engagement: students’ experience of online learning over time. Distance Education, 40(2), 262–277, https://doi.org/10.1080/01587919.2019.1600367
O’Shea, S., Stone, C. & Delahunty, J. (2015). “I ‘feel’ like I am at university even though I am online.” Exploring how students narrate their engagement with higher education institutions in an online learning environment. Distance Education, 36(1), 41–58. http://dx.doi.org/10.1080/01587919.2015.1019970
Pal, R., Kar, S., Zaman, F. A., Jha, D. K. & Pal, S. (2012). Assessment of impact of small group teaching among students in community medicine. Indian Journal of Community Medicine, 37(3), 170-172. DOI: 10.4103/0970-0218.99920. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3483510/
Paulsen, J. & McCormick, A. C. (2020). Reassessing Disparities in Online Learner Student Engagement in Higher Education. Educational Researcher, 49(1), 20-29. 0.3102/0013189X19898690
Rajabalee, Y.B., & Santally, M.I. (2021). Learner satisfaction, engagement and performances in an online module: Implications for institutional e-learning policy. Education and Information Technology, 26, 2623–2656. https://doi.org/10.1007/s10639-020-10375-1
Roberts, T.S. & McInnerney, J. M. (2007). Seven Problems of Online Group Learning (and Their Solutions). Educational Technology & Society, 10(4), 257-268.
Rockey Alexandria and Saichaie Kem. (2020). Developing an Ecology of Feedback in Online Courses in the Disciplines. Research in Online Literacy Education. http://www.roleolor.org/developing-an-ecology-of-feedback-in-online-courses-in-the-disciplines.html
Rohrdantz, C., Mansmann, F., North, C. & Keim, D. A. (2014). Augmenting the educational curriculum with the Visual Analytics Science and Technology Challenge: Opportunities and pitfalls. Information Visualization, 13(4), 313–325.
Ruey, S. (2010). A case study of constructivist instructional strategies for adult online learning. British Journal of Educational Technology, 41(5), 706–720. https://doi.org/10.1111/j.1467-8535.2009.00965.x
Scager, K., Boonstra, J., & Koster, M. C. (2020). Pop-Up Questions within Educational Videos: Effects on Students' Learning. Journal of Science Education and Technology, 723-724.
Seaman. J, Allen. E, Seaman. J. (2018). Grade Increase: Tracking Distance Education in the United States. https://www.bayviewanalytics.com/reports/gradeincrease.pdf
Shahrokni, S. A. (2018). PlayPosit: Using Interactive videos in Language Education. Teaching English with Technology, 105-115.
Shea, & Bidjerano, T. (2009). Community of inquiry as a theoretical framework to foster “epistemic engagement” and “cognitive presence” in online education. Computers and Education, 52(3), 543–553. https://doi.org/10.1016/j.compedu.2008.10.007
Su, C. Y & Chiu, C. H. (2020, November 8). Perceived Enjoyment and Attractiveness Influence Taiwanese Elementary School Students’ Intention to Use Interactive Video Learning. International Journal of Human-Computer Interaction, 37(6), 574–583. https://doi.org/10.1080/10447318.2020.1841423
Tsay C, Kofinas AK, Luo J. (2018). Enhancing student learning experience with technology-mediated gamification: an empirical study. Computers and Education, 121, 1-17.
Theodosiou, N. A., & Corbin, J. D. (2020). Redesign your in‐person course for online: Creating connections and promoting engagement for better learning. Ecology & Evolution (20457758), 10(22), 12561–12572. https://doi-org.proxy.libraries.rutgers.edu/10.1002/ece3.6844
U.S. Department of Education. (2019). Fast facts. Distance Learning. https://nces.ed.gov/fastfacts/display.asp?id=80
Wade, C. E., Cameron, B. A., Morgan, K., & Williams, K. C. (2016). Key components of online group projects: Faculty Perceptions. Quarterly Review of Distance Education, 17(1), 33–41.
Williams, K. & Cameron, B. & Morgan, K. (2012). Supporting online group projects. North American Colleges and Teachers of Agriculture Journal, 56(15).
YunJeong (Eunice), Chang, and Michael J. Hannafin. (2015, November 28). The Uses (and Misuses) of Collaborative Distance Education Technologies. Quarterly Review of Distance Education, 16(2). 77-92.

No competing interests reported.

Download PDF

Editorial decision: Major revision
24 May, 2023
Reviews received at journal
02 May, 2023
Reviews received at journal
23 Apr, 2023
Reviewers agreed at journal
19 Apr, 2023
Reviewers agreed at journal
16 Apr, 2023
Reviewers invited by journal
30 Mar, 2023
Editor assigned by journal
29 Mar, 2023
Submission checks completed at journal
29 Mar, 2023
First submitted to journal
27 Mar, 2023

You are reading this latest preprint version

Evaluation of a PlayPosit Guided Group Project’s Impact on Student Engagement in an Undergraduate Course

Status:

Version 1

Abstract

Background

Figures

Introduction

Literature Review

Methods

2.1 Description of the Educational Intervention Teaching Strategies

2.2 Data Collection

2.3 Data Analysis

Results

3.1 Course Analytics

3.2 Students Comments

Discussion

Considerations In Implementation

Significance And Limitations

Conclusions

Declarations

References

Additional Declarations

Status:

Version 1