Investigation of time period of activity for on-demand classes among pharmacy students during the COVID-19 pandemic

Background: The COVID-19 outbreak brought about major changes in the format in which university lectures were delivered in Japan. Most classes changed from face-to-face classes to remote education methods to prevent infection. This study investigated the relationship between lecture comprehension with on-demand classes and the use of time to complete lecture-related assignments during COVID-19. Methods The eligible participants were 139 third-year pharmacy students;they received seven pharmacotherapeutics lectures via on-demand classes from September to November 2020. We classified the participants into three groups according to the period between lecture upload to the submission of related assignments: submission on the day of upload (“early,” n = 42);the 2nd day to the day before the deadline (“intermediate,” n = 58);and on the deadline day (“late,” n = 39). In our analysis, we mainly used Spearman’s correlation coefficients to examine the correlations among end-of-semester examination scores, total assignment scores, and time of assignment submission. Results A strong positive correlation was evident between test scores and total assignment scores in the early group ( r  = 0.68, P  < 0.001) and a weak correlation in the intermediate group ( r  = 0.27, P  = 0.038), but there was no correlation in the late group. We observed a negative correlation between test scores and assignment submission time in the early group ( r = − 0.59, P  < 0.001) but not in the intermediate or late groups. Conclusions On-demand classes give university students the opportunity to choose when they wish to study. Remote education offers a certain amount of freedom and encourages students’ initiative;however, our results suggest that some students cannot make the best use of on-demand lecture characteristics and act appropriately. Therefore, while taking advantage of on-demand education during COVID-19, it is necessary to consider students’ use of time.


Results
A strong positive correlation was evident between test scores and total assignment scores in the early group (r = 0.68, P < 0.001) and a weak correlation in the intermediate group (r = 0.27, P = 0.038), but there was no correlation in the late group. We observed a negative correlation between test scores and assignment submission time in the early group (r = − 0.59, P < 0.001) but not in the intermediate or late groups.

Conclusions
On-demand classes give university students the opportunity to choose when they wish to study. Remote education offers a certain amount of freedom and encourages students' initiative; however, our results suggest that some students cannot make the best use of on-demand lecture characteristics and act appropriately. Therefore, while taking advantage of ondemand education during COVID-19, it is necessary to consider students' use of time.

Background
In Japan, the new school term begins in April. Almost all universities have adopted a system whereby the rst semester begins in April and the second in September. Until 2019, almost all universities conventionally taught using face-to-face classes. However, with the COVID-19 global pandemic since March 11, 2020 [1,2], viral transmission via droplet, contact, and aerosol infections were reportedly the cause of person-to-person spreading [3,4]. Because face-to-face classes pose the risk of infection through such viral transmission, COVID-19 has led to major changes in how university lectures have been delivered in Japan. Most university lectures have shifted from face-to-face classes to remote education [5].
On-demand education is a common remote education method, whereby lecture materials or videos are uploaded onto a platform for students to complete [6]. In that way, university students have gained a certain level of freedom to learn when and how they wanted, although this required self-directed learning [7]. However, some students may be unable to adapt to the change in educational method during COVID-19. For example, a number of students may not take remote lectures seriously: they are concerned about submitting assignments but not understanding the lecture content. In addition, being able to take lectures anytime could lead to their life rhythms becoming disturbed. Conversely, other students diligently attend on-demand lectures according to schedule. Differences in those two approaches could affect comprehension of lecture content.
In this study, we investigated the time in which pharmacy students worked on and submitted the assignments associated with on-demand classes during COVID-19. We also evaluated the relationship among the use of time to complete lecturerelated assignments, assignment scores, and end-of-semester examination results.

Subject selection
The study participants were 139 third-year university students in the Faculty of Pharmaceutical Sciences at Sojo University, Japan. The present authors were responsible for teaching both biostatistics and pharmacotherapeutics there; we selected pharmacotherapeutics for our analysis. The lectures via on-demand classes were held from September to November 2020 during the second semester. We selected pharmacotherapeutics for our analysis for the following reasons. The Japanese government declared the rst state of emergency for COVID-19 in April 2020; that state of emergency was lifted for all Japan in late May 2020. Accordingly, the rst semester at Sojo University started about 2 months later than usual. It was possible that many students had di culty adapting to the novel situation in the rst semester, so we believed it more appropriate to choose subjects during the second semester. Further, there was no declared state of emergency from September to November 2020. In addition, to allow familiarity with our lecture style via on-demand classes (described below), we applied the same system for biostatistics (seven lectures in all), which was held from July to August 2020 in the rst semester.

Lectures and assignment content
The Internet diffusion rate in Japan is reportedly over 90%; the proportion of Japanese with smartphones was about 60% and those with personal computers about 50% in 2019 [8]. Smartphone screens are small, and the devices are not suitable for students to take on-demand lectures over the long term. Thus, considering students' Internet environment at home, we did not deliver video lectures; instead, we created lecture materials in PDF format, which we converted into Microsoft PowerPoint slides. We uploaded the PDF les for each lecture to a learning management system (WebClass), which was developed for Japanese universities (Data Paci c Japan, Ltd, Tokyo, Japan). We released the PDF lecture les at 7:00 a.m. each Tuesday. In total, seven lectures were provided from September to November 2020; the lectures covered drug and alcohol dependence, palliative medicine, anemia and purpura, leukemia, hepatitis and hepatic carcinoma, hepatobiliary disease, and arteriosclerosis obliterans. Each lecture had two assignments (assignments 1 and 2); an assignment had ve multiple-choice questions (i.e., 10 questions per lecture). With each question, students had to select one from among ve answer items. The content of the two assignments for each lecture corresponded to that of the lecture itself. Accordingly, students should have been able to choose the correct answer. We did not set pass criteria for each lecture's assignment; however, students had to submit ve or more assignments for the seven lectures. Thus, if a student submitted two assignments for one lecture, we counted that as one submission.
These assignments were generated using Google Forms, which is a free application provided by Google [9]. Links to Google Forms were placed at the middle and end of each lecture. Students needed to submit answers for the two assignments within 6 days (Tuesday to Sunday) from the uploading of each lecture; an exception was the seventh lecture, which could be completed within 13 days. The day following the deadline for each lecture, we released on WebClass a PDF le that contained the answers and commentaries for the two assignments. The students had to submit their assignments within the deadline; however, they were able to view, download, and print the PDF les related to the lecture content or assignment answers at any time until the regular end-of-seminar examination. After all the lectures had been delivered, the end-ofsemester examination for pharmacotherapeutics was held on November 17, 2020. The above information is summarized in Table 1.

Statistical analysis
For each lecture, we collected information about the time of day when the students submitted their assignments, the number of hours from uploading the lecture to the assignment submission time, and the time lag (minutes) between assignments 1 and 2. For each student, we obtained the above three time-related variables for each lecture.
To undertake our analyses using those three variables, we took the median for all seven lectures as the representative value; we assumed that the variables were continuous. We also collected the total assignments scores and test scores for each student. We categorized the students into three subgroups with respect to the number of hours from uploading a lecture to when assignment 2 was submitted. The lectures were uploaded on Tuesday, and students who submitted assignment 2 that day were classed as "early"; those who did so from Wednesday to Saturday were the "intermediate" group; and those who did so on the deadline day (Sunday) were the "late" group.
For univariate analysis, we conducted one-way analysis of variance or the Kruskal-Wallis test. For continuous variables, we employed the pairwise Wilcoxon rank sum test with Bonferroni correction. We applied Spearman's correlation coe cients and regression analysis to con rm the relationship between two variables. Analyses were performed using R version 4.0.2 (R Foundation for Statistical Computing, Vienna, Austria).

Lecture characteristics
The variation in the time lag between the two assignments for each lecture was small; the score for the assignments for each lecture showed the same tendency (Table 1). For every lecture, most students submitted their assignments between 09:00 and 22:00 (Table 1). However, the number of hours between lecture upload and assignment submission varied widely ( Figure 1, Table 1). Further, the proportion of students who submitted their assignments just before the deadline increased as the lecture course progressed (Figure 2).
Comparison between lecture upload and assignment submission time As described, we categorized students into three subgroups based on the time from lecture upload to assignment submission ( Table 2). We found that about 30% belonged to the early (n = 42) or late group (n = 39). When we analyzed each group separately, the early (assignment 1, P <0.001; assignment 2; P <0.001) and intermediate (assignment 1, P = 0.002; assignment 2, P = 0.001) groups submitted their assignments early in the day compared with the late group. The test scores in those two groups were also higher than with the late group (early, P <0.001; intermediate, P <0.001); however, there were no differences in the time lag between the two assignments and the total assignment scores. Notably, almost all the students continuously worked on the two assignments ( Figure 3). When comparing the submission time between the early and intermediate groups, we found that time for assignment 1 in the early group was early in the day (P = 0.003).

Correlation analysis
For each group, we examined Spearman's correlation coe cients between the test scores and total assignment scores for the time of submitting assignment 2. We observed a strong correlation between the test scores and total assignment score (r = 0.68, P <0.001) in the early group; there was a weak correlation between those variables in the intermediate group (r = 0.27, P = 0.038), but there was no correlation in the late group (r = 0.24, P = 0.14). We con rmed a negative correlation between the test scores and time for submitting assignment 2 in the early group (r = -0.59, P <0.001). A similar tendency was evident in the case of assignment 1 (data not shown). By contrast, there was no such correlation in the intermediate and late groups. We also examined the relationship between the total assignment score and time lag in submitting the two assignments. A strong, positive correlation was evident in the early (r = 0.71, P <0.001) and late (r = 0.82, P <0.001) groups; the intermediate group manifested a moderate positive correlation (r = 0.52, P <0.001).
We applied univariate regression analysis (Figures 4 and 5, Table 3). We were able to con rm a correlation between the total assignment score (P <0.001, 95% con dence interval [CI] 0.48 to 0.94) and submission time (P <0.001, 95% CI −0.05 to −0.02) in relation to test scores in the early group. We observed a relationship between the total assignment score and time lag in submitting the two assignments in the early (P <0.

Discussion
Until 2019 at our university, subjects in the Faculty of Pharmaceutical Sciences, including pharmacotherapeutics, were taught in face-to-face lectures. Attendance con rmation took place during each lecture, and the intelligibility and effectiveness of the lecture content could be mainly veri ed with the end-of-semester examination. However, COVID-19 caused us to switch to on-demand education. In that faculty, on-demand education started in May 2020; no lectures were delivered in April 2020. When on-demand education was rst implemented for most lectures, faculty members and students reported some confusion over the unfamiliar environment. Since then, students and faculty members have become accustomed to this teaching method. Face-to-face lectures have since resumed in Japan in line with the COVID-19 infection status: appropriate measures for disease control are applied. However, in the pandemic, a complete return to face-to-face classes from on-demand education has been di cult.
With on-demand classes, in general, the presence or absence of activity is judged though log access; however, it is di cult to determine the quality of activity. Therefore, for many subjects in the Faculty of Pharmaceutical Sciences, various assignments were set to con rm on-demand class attendance and assess comprehension by each student. In this regard, faculty members had discretionary power regarding class content and assignments for which they were responsible; the style of lectures and assignments depended on each faculty member. In our case, we undertook on-demand classes using PDF les and assignments employing Google Forms. In light of the students' network environment, PDF les-not videoswere selected for teaching the on-demand classes. Students were able to view, download, and print the materials at any time. However, as a result, we were unable to grasp the cumulative access time of the lecture from the browsing history.
In the present study, we employed information related to when faculty students submitted their lecture-related assignments. Notably, we investigated the time between lecture upload and assignment submission. We observed that as the on-demand lectures progressed, that median time increased. For example, many students promptly submitted their assignments for lecture 1; however, the proportion who submitted their assignments just before the deadline increased as the lecture course progressed. On-demand classes have the bene t that students are able to learn when and how they wish; thus, this approach cannot be dismissed as bad. However, when the students submitted their assignments so close to the deadline, it is possible that that they were simply preoccupied with the assignment commitment rather than to properly understanding the lecture content.
We categorized the students into three subgroups based on the number of hours from lecture upload to assignment submission. Our thinking was as follows. If students made their assignment submission on the day of the lecture upload, they wished to adhere to the schedule regardless of the on-demand lecture format. We de ned those as the early group. If students made their submissions between the 2nd day and day before the deadline, they made the best use of the characteristics of on-demand lectures. We de ned those as the intermediate group.
If students made their submissions on the deadline day, they were mainly concerned about submitting assignments rather than understanding the lecture content.
And we de ned those as the late group.
As noted above, we were unable to determine the cumulative access time of each lecture from the browsing history; however, we calculated the time lag between assignments 1 and 2, and almost all students worked continuously on the two assignments. Accordingly, the time lag data could be used as a substitute for a student's cumulative access time for a lecture. In fact, the relationship between that time lag and total assignment score showed a strong positive correlation among all three groups. However, that correlation with the intermediate group was low compared with the early and late groups. Some students in the intermediate group may have tackled assignments 1 and 2 separately, thereby making optimal use of the on-demand lecture format.
We observed that seven students in the intermediate group submitted their assignments after midnight. It has been reported that staying up late causes deterioration in physical condition, including decreased immunity, fatigue, and disturbance of the endocrine system [10,11]. Further, staying up late or reduced sleeping time increases the risk of long-term declining academic results [12]. There are serious implications to student medical professionals (i.e., student pharmacists) being unaware of negative effects on their health and academic records through late-night studying. It is necessary to encourage appropriate time periods for students to undertake on-demand classes.
We investigated the relationship among test scores, total assignments scores, and submission times for assignment 2. The early group displayed a positive relationship between test scores and total assignment scores; however, there was a negative relationship between test scores and submission times for assignment 2. The students who completed the ondemand lectures as soon as possible were most likely very keen and followed the lectures enthusiastically. By contrast, students who worked late on the 1st day may have been overly conscious about having to submit assignments on that day rather than understanding the lecture content. Thus, it is possible that they were unable to make good use of the exibility offered with on-demand classes.
The above trends were also evident with the late group. With that group, the time of day when the students submitted their assignments for the on-demand classes was the afternoon of the deadline day. We supposed that their main focus was on submitting the assignments rather than on the lecture content. In this regard, it appeared that some late-group students seriously worked on the on-demand lectures from the perspective of total assignment scores. However, both the total assignment scores and test scores were the lowest among all three groups. We also observed no relationship between test scores and total assignment scores in the late group. Therefore, these results may also indicate that the students did not fully understood the content: they simply wanted to complete the assignments.
With the intermediate group, the students undertook the assignments at varying times compared with the other two groups.
Intermediate-groups students may be able to adapt to effectively utilizing the on-demand classes. In addition, the positive correlation between test scores and total assignment scores was weak. Further, we discerned no correlation between the test scores and submission times. However, the test scores for the intermediate group were higher than those of the late group. The interval between the deadline for the last lecture's assignment and the day of the end-of-semester examination was 9 days. Therefore, intermediate group students may have adapted successfully in using time for both the on-demand classes and in preparing for the examination.

Conclusion
On-demand education offers university students the opportunity to decide for themselves when they should study, and the system may encourage a sense of self-reliance [13]. The present research was performed at a university with a geographically limited sampling area for the study subjects, and this is a limitation of the study. However, owing to COVID-19, it is necessary to continue a certain amount of remote education. It is highly probable that most medical universities in Japan are in a similar situation. Therefore, it is necessary to analyze time factors and activities related to on-demand classes. When students show inappropriate use of time, it is important to communicate with them in that regard. When delivering education remotely during COVID-19, faculty members should monitor students' use of time with respect to ondemand classes and encourage students not to get into bad habits.  indicates students who submitted their assignments on the lecture upload day. Intermediate group indicates students who submitted their assignments between the 2nd day and the day before the deadline. Late group indicates students who submitted their assignments on the deadline day.