Student Lecture Attendance and Its Relation to Exam Performance in the Era of Lecture Capture Technology.

Background: Students can now review on-line lecture recordings rather than attend lectures. However, there is little data on the fraction of students attending lectures, the patterns of attendance, or the ultimate learning outcomes. To study this issue, attendance data was used to compare learning outcomes of students with different patterns of attendance. Methods: A retrospective observational analysis was performed using data collected in the Medical Pharmacology course in Fall 2013 (197 students) and Fall 2014 (207 students). Attendance was monitored at 13 of 40 lectures with an audience response system in each semester. All lectures were recorded using lecture-capture, and students could attend lectures or review online recordings and PowerPoint les as desired. Exam averages of students in different attendance categories were compared, along with the frequency distribution of exam scores in the different attendance categories. Results: In the years studied, 12-14% of the students had a consistent pattern of high attendance at monitored lectures; 35-46% of the students had an erratic pattern of low to intermediate attendance, and 41-52% of the students did not attend any monitored lectures. Aggregate exam averages of students with >80% attendance were over 5 percentage points higher than those of students in other attendance categories (P<0.05 using t-tests or ANOVA). Students with >80% attendance also displayed a signicant shift in aggregate exam averages to higher ranks in frequency distribution analyses (P<0.05 using Chi-Square tests). Exam averages of students with low to intermediate attendance did not differ from students with no attendance. The scoring advantage associated with >80% lecture attendance remained evident 20 weeks after the Fall semester when students took a comprehensive pharmacology examination (P<0.05). Conclusions: On average, students with a consistent pattern of high lecture attendance had higher exam scores than students with low to intermediate or no attendance who had access to the identical course material on-line. This may reect factors related to the method of content acquisition (live vs. on-line) or differences in the learning readiness of students choosing one approach vs. another. The data suggest that curricular changes that encourage consistent lecture attendance might improve learning outcomes for some students.


Background
Faculty lectures in a classroom with students has been a traditional teaching format in medical education for generations. Faculty convey essential knowledge in a structured dialog involving words/speech, body language and visual aids (PowerPoint slides with text, charts, diagrams, photos, etc. in modern times). However, in modern times student lecture attendance is often no longer required due to lecture capture technology that records lectures and uploads the video les to online learning management systems where students can study the material whenever desired (1)(2)(3). The lack of a mandatory attendance requirement at lectures that are recorded, and approaches like the ' ipped classroom' give the impression that review of online lecture recordings enables learning as effectively as live lecture attendance. However, data comparing the learning outcomes of live lecture attendance and review of on-line lecture recordings is sparse and variable in its ndings (4)(5)(6)(7)(8)(9). Some studies found lower exam performance in students using online recordings to replace lecture attendance (4)(5)(6); while others reported similar e cacy (7)(8)(9). A 2010 meta-analysis comparing traditional in-person lectures and on-line teaching approaches (10) concluded that blended approaches with online offerings were as effective as traditional instruction. However, that report also noted that blended approaches typically bene ted from greater learning time, more instructional materials, differing pedagogies and greater collaborative opportunities than traditional lectures (10). Similar conclusions were reached in a 2013 survey of ipped classroom research (11); the ipped classroom produces an expansion of the curriculum as well as a rearrangement of curricular activities. This imbalance in learning time, educational resources and instructional approaches remains evident in virtually all studies comparing ipped classrooms to traditional lectures (12)(13)(14)(15)(16)(17)(18)(19)(20). Given such curricular advantages, it is surprising that improved academic performance is not a consistent nding in studies comparing ipped classrooms to traditional lectures (13,15,(17)(18)(19)(20). This suggests that the learning e cacy of live lecture attendance may be underestimated.
Our understanding of the learning e cacy of live lectures is greatly clouded by a lack of detailed information on student attendance. In particular, what proportion of the class attends any lectures, what is the average attendance level among those present, how variable is attendance over time, and how do these factors relate to exam performance? Most modern data on lecture attendance is largely based on student surveys with limitations in participation, accuracy, and detailed attendance data (2,8,(21)(22)(23)(24)(25)(26)(27)(28).
Moreover, studies have shown that attendance decreases over the course of a semester and from year to year (21,23,24,27,28). This complicates efforts to assess the value of live lectures relative to online offerings since students may initially attend and later switch to on-line viewing. Such attendance decreases might re ect recognition that online viewing enables learning as well as live attendance (2).
However, surveys assessing student attitudes indicate that diverse factors in uence attendance decisions (2,8,(21)(22)(23)(24)(25)(26). Thus, time of day, subject matter, other commitments, lecturer quality, lifestyle choices, distance from school, time-savings from accelerated replays and other convenience factors all prominently gure in student attendance decisions (2,8,(21)(22)(23)(24)(25)(26). Overall, the learning e cacy of live lecture attendance versus review of on-line recordings remains unresolved but is an important issue since material covered in lectures typically provides foundational knowledge that other teaching formats build upon.
The objective of the present study was to collect detailed data on the features of student attendance at voluntary lecture sessions, and then use that data to perform an analysis of the relation of student attendance to exam performance. We accomplished this with attendance data collected over two years from the Medical Pharmacology course for second year medical students at New York Medical College (Valhalla, New York, USA). In particular, an audience-response system (ARS) was used in a subset of lecture sessions to pose questions that students answered using wireless response devices (29,30). In addition to answer choices, transmitted responses identi ed students in attendance. The attendance data provided primary information on the attendance habits of students and enabled secondary analyses of the relation of attendance to exam scores in the course.

Methods
Experimental design: A retrospective observational population-based study was conducted using student data that had previously been collected in the Medical Pharmacology 1 course at New York Medical Lectures involved PowerPoint presentations in a large auditorium with equipment and software that created synchronized audio and screen-capture recordings (Camtasia Recorder, TechSmith, Okemos, Michigan, USA). Lecture recordings (including the ARS questions) were promptly posted to an online learning management system for student use. All students also had on-line access to the PowerPoint les for each lecture before and after the session.
Students were provided with advance notice of which voluntary lectures would include interactive questions to encourage attendance and alert them to bring their wireless response devices. Wireless response devices were mandatory equipment for medical students since the ARS was already used to administer required formative quizzes in 2 nd year courses. Each response device was registered to a speci c student and transmits its identifying code along with the student responses. Turning Technologies software (Turning Pont 5) automatically generated an ARS session le that was converted to Excel spreadsheets for use in the data analysis. The ARS assigned attendance points to students who answered at least 50% of the questions presented in a session. Attendance points were only used to track and calculate student attendance during the voluntary lecture sessions that were monitored; they had no bearing on the formal assessment of student performance in the course. Attendance data from monitored sessions, and data from exam result spreadsheets were compiled and analyzed to examine the relation of student attendance to exam performance.
Statistical analysis: Compiled data were analyzed using NCSS Data Analysis software (version 11; Kaysville, Utah, USA). There was no set 'a priori' format for analyzing the data. The data was subjected to a sequential series of formal statistical analyses designed to provide insight on the relation of student attendance to exam performance (see Results).
Datasets of exam performance and student attendance were initially subjected to linear regression analysis. Datasets were subsequently strati ed into groups based on different levels of attendance. Due to differences in the 2013 and 2014 course schedules, minor variations in attendance groups were used in different years (see Results). Attendance strati cations that yielded groups with equivalent exam averages were often re-strati ed into a smaller number of groups to enhance statistical power. The datasets were analyzed using ANOVA followed by multiple comparison tests (Tukey HSD test) or by ttests when only two means were analyzed. Frequency distribution analyses were performed using the Chi-squared statistic to verify that groups with signi cantly higher exam averages also displayed a signi cant shift in exam scores to higher scoring ranks. The criterion of signi cance for ANOVA, Tukey tests, t-tests, and Chi-square tests was P<0.05. The attendance data also enabled an analysis of the pattern of attendance in the different attendance groups. The high attendance group exhibited a consistent pattern of high attendance; averaging 12% variation in attendance levels in October monitored sessions compared to November and December monitored sessions. In contrast, the low to intermediate attendance groups exhibited an erratic attendance pattern; averaging 39% variation in attendance levels in the different course sections. 2014 r= 0.072, r 2 =0.005, P=0.302. For a more discriminating analysis the datasets were segregated into groups based on attendance levels. Datasets were initially strati ed into 6 groups based on different levels of attendance (100-80%, >80%<60%, >60%<40%,>40%<20%,>20%<0%, 0%) and the aggregate exams averages of the different attendance levels were calculated. This led to the identi cation of higher exam scores in the group with >80% attendance but not in any other groups (data not shown).

Results
The large number of attendance groups and the small number of students in some groups weakened the statistical power of the dataset. Thus, we switched to an attendance strati cation that gave fewer attendance groups to improve statistical power (100-83%, 67-17%, 0% in 2013 and 100-83%, 77-8% and 0% in 2014). Figure 2 (left panel) shows the average scores for students exhibiting high, intermediate to low, or no attendance during the monitored class sessions during each of the 4 exams held during the Fall 2013 semester. The group with high attendance had higher exam scores (+3 to 6 percentage points) compared to groups with intermediate to low or no attendance. This difference was signi cant for exams 1, 3 and 4, The 2013 data also indicated that students with intermediate to low attendance did not exhibit a signi cant scoring advantage relative to students with no attendance. The relatively small numbers of students exhibiting high attendance weakened the statistical power of analyses focusing on a single exam. Thus, we analyzed the aggregate exam averages of the different attendance groups. Figure 3 (upper left panel) compares the aggregate exam averages of groups of students exhibiting high, intermediate to low, or no attendance in Fall 2013. The high attendance group had a signi cantly higher aggregate exam average than the groups with either intermediate to low or no attendance (+5.9 to 5.5 percentage points). This dataset was then subjected to frequency distribution analysis to assess the relation of student attendance to relative scoring rank (Fig. 3 upper right panel). The frequency distribution analysis revealed that high attendance was associated with a signi cant shift in aggregate exam averages towards the high rank coupled with a decrease in the intermediate and low ranks. Thus, students with high attendance were 71% more likely to have an upper rank, and 50% less likely to have a low rank, compared to students with intermediate to low or no attendance. again associated with signi cantly higher aggregate exam averages (+5.4 percentage points) relative to intermediate to no attendance. Frequency distribution analysis revealed that high attendance was again associated with a shift in the number of students with averages in the higher rank coupled with a decrease in the number of students scoring in the lower rank (Fig. 3, lower right panel). Thus, students with high attendance were 60% more likely to have an upper rank, and 37% less likely to have a lower rank compared to students with intermediate to low or no attendance.
The Medical Pharmacology 1 course in the Fall semester is followed by a Medical Pharmacology 2 course in the Spring semester that concludes with a comprehensive exam in pharmacology covering material taught in both semesters. The comprehensive pharmacology subject exam is prepared by the National Board of Medical Examiners (NBME)(Philadelphia, Pennsylvania, USA) and uses questions prepared by a national committee of content experts assembled by the NBME. The NBME exams were given 20 weeks after the end of the Fall semester when the attendance data was collected, In both years the group with high lecture attendance had a higher NBME exam average than groups with intermediate to low or no attendance (+4.5 points in Spring 2014 and + 3.1 points in Spring 2015); this difference was signi cant for the Fall 2013 class (t-test P < 0.02), and approached signi cance in the Fall 2014 class (ttest P < 0.13). To increase the statistical power of the NBME exam data analysis, we transformed the NBME scores to give a mean of 55.6 for both classes, with no change in the variance of either class. The data were then pooled and analyzed. The pooled data showed that the high attendance group had a signi cantly higher NBME average (+3.8 pts.) than the intermediate to low and no attendance groups (Fig. 4, left panel). Frequency distribution analysis of NBME scores showed that high attendance was associated with a signi cant shift in scores towards the higher rank coupled with a decrease in scores in the lower rank (Fig. 4, right panel). Students with high attendance were 50% more likely to have an upper rank and were 48% less likely to have a lower rank. Thus, on average, high lecture attendance in the Fall semester was associated with increased NBME exam scores almost 5 months later.

Discussion
After the introduction of lecture capture technology, it became apparent that attendance at voluntary lecture sessions progressively declined during the semester, particularly after exam 1. This declining attendance has been widely reported (24)(25)(26)(27)(28). It was anticipated that most students attending lectures in the late October to early December would have high attendance. However, this was not true; only a small fraction of those attending had a consistent pattern of high attendance. The other attending students had an erratic pattern low to intermediate attendance.
Attendance variability has prompted others to use linear regression analysis to correlate attendance level to course outcomes such as exam performance (9,(31)(32)(33)(34). Such analyses have generally found low correlations with coe cients of determination (r 2 ) below 0.1. Although the correlations were positive, such data suggested that attendance was not a major determinant in exam performance for most students. Similar low correlations were found in the present data. However, linear regression analyses assume there is a linear relationship between two variables, and further analysis of the present datasets showed that this was not true.
When students were strati ed into groups based on attendance levels it was evident that, on average, students with high attendance consistently achieved higher exam scores than students with intermediate to low or no attendance. This was true in 7 out of the 8 in-house exams administered in Fall 2013 and 2014 and con rmed in the aggregate exam averages of Fall 2013 and Fall 2014. The scoring advantage associated with high attendance also appeared long-lasting. On average, students with high attendance displayed higher scores in a comprehensive pharmacology NBME exam given 20 weeks after the end of the Fall semester. Frequency distribution analyses demonstrated that students in the high attendance groups had a signi cant shift towards higher exam scores coupled with a decrease in lower scores. This was true for both the in-house exams and for the NBME exam.
The reasons why some students adopted a high attendance approach was not addressed but presumably re ects their preferred learning style and desire for live interactions with classmates and faculty. In this regard, it is notable that surveys of student attitudes regarding attendance indicate that many students prefer live attendance to study of recorded lectures, but factors related to lifestyle, other commitments and convenience can over-ride such preferences (7,22,24). Interestingly, students who did not attend any monitored sessions had the most consistent attendance pattern and the most specialized study approach: attendance had been abandoned in favor of online or other resources. Nonetheless, on average, this approach did not yield an exam scoring advantage. The reasons why so many students chose not to attend lectures was not addressed but surveys of student attitudes indicate that a multitude of factors play a role in such decisions, many of which relate to convenience rather than perceived learning outcome (2,7,(21)(22)(23)(24)(25)(26)(27)(28).
At this point it should be noted that there were many students in the top ranks of the class who had intermediate to low or no lecture attendance. Thus, other learning approaches worked well for some students. Conversely, high lecture attendance did not guarantee high exam scores since some students with high attendance had scores that placed them in the middle or lower ranks of the class. Nonetheless, a learning strategy that increases the chance of earning a score in the middle ranks may be highly valued by some students given the varying academic strengths and weaknesses of individuals within a class.
Although our data show that the high attendance group had increased exam performance relative to those with lower attendance, the behavioral characteristics of students with high lecture attendance might also contribute. Thus, personal attributes such as organization, discipline and motivation that may foster high attendance may also improve the learning readiness of the students. However, such attributes should not be viewed as innate character traits; they are more likely to re ect the in uence of years of learning experiences that rewarded class attendance and shaped behavior accordingly.

Conclusions
The objective of the present study was to collect detailed data on student lecture attendance and use that data to assess the relation of lecture attendance to exam performance. It was found that most students either did not attend voluntary lecture sessions or had an erratic pattern of intermediate to low attendance. On average, students with a consistent pattern of high lecture attendance achieved signi cantly higher exam scores than other students who had access to the identical material in lecture recordings and other course materials posted on-line. The bene ts of high lecture attendance also appeared long-lasting (>20 weeks). The data suggest that curricular changes that encourage consistent lecture attendance might improve learning outcomes for some students.
A limitation of this study is that it is observational and does not address causation. Thus, it is unclear if the differing learning outcomes are related to different methods of content acquisition (live vs. on-line) or differences in the learning readiness of students using different learning approaches. In addition, attendance was measured at only 13 of 40 voluntary lecture sessions. Thus, it was not possible assess how attendance before each exam was related to subsequent exam performance. Acquisition of such data would be a useful direction for further research. Availability of data and materials: The course datasets used in the study are not publicly available as they are based on exam scores and attendance data of identi able students. Datasets are available from the corresponding author on reasonable request after modi cation to remove identi ers. data collection, and editing of the manuscript. All authors have read and approved the manuscript.
Acknowledgements: The authors gratefully acknowledge the contributions of the Educational Technology staff who manage the lecture-capture and ARS software and hardware in our auditoriums and provide students and faculty with support services related to such technology. The authors acknowledge Dr. Carl Thompson (Department of Physiology) for discussions on the use of NCSS software and the statistical analyses.
Author's Information: C.A.P. has been the course director of the Medical Pharmacology courses at New York Medical College for over 20 years. He is also the course director of two advanced graduate courses and is a teaching faculty member in many other courses. He has actively encouraged, supported, and implemented diverse innovations in medical education in the courses he directs or teaches in. This includes the use of lecturecapture technology -which provides on-line lecture recordings that are a valuable learning resource for students. The medical education research of C.A.P. seeks to better understand the diverse learning approaches used by students and explore how such diversity can be more effectively assisted by versatile methods of curriculum delivery. Lecture attendance is one of many methods students use to achieve their educational goals and should neither be ignored nor over-emphasized.