A Comparative Study Between Traditional Face-To-Face, Audience Response System, and A Flipped Classroom Plus Audience Responce System in Pharmacology Education.

Background: Active learning methods, audience response system (ARS) and ipped classroom (FC), have applied in higher education internationally. However, these outcomes in Japan are uncertain. This study presents a comparison of teaching methods such as the traditional face-to-face (TFF), ARS, and FC plus ARS that are used to enhance student pharmacists’ understanding of pharmacology. Methods: Sophomores at Ohu University were recruited for this study from 2017 to 2019. The TFF were performed in 2017; the ARS were performed in 2018; and the FC plus ARS were performed in 2019. Exam scores were compared between the three methods and anonymous questionnaires were also surveyed in 2018 and 2019 to compare students’ preferences. The Kruskal-Wallis test and ANOVA were carried out depending on data distributions. Categorical data were analyzed using the Chi square test and the Fisher’s exact test. The ANCOVA was performed if background information between the three cohorts was signicantly different. Results: The groupings of the participants went as follows: 49 students in 2017, 78 students in 2018, and 90 students in 2019. Their academic backgrounds of each year at the end of their freshman year were similar except for their grade point averages (GPAs). The medians of their pharmacology exam correct rates for the TFF, ARS, and FC plus ARS were 73.3%, 90.0%, and 93.3%, respectively. ANCOVA utilizing GPA as the covariance revealed that the ARS and the FC plus ARS signicantly improved exam correct rates compared with the TFF (p < 0.001). There was no signicant difference between the ARS and the FC plus ARS groups (p = 0.487). A subgroup analysis in terms of gender difference revealed that the ARS was a signicantly improved method compared to the FC plus ARS for female students (p = 0.009), whereas the ARS was not for male students (p = 0.819). Anonymous questionnaires revealed that students in the FC plus ARS lectures had good readiness, enjoyed their classes, and felt fulllment compared with the ARS lectures. Conclusions: Active learning methods including the ARS and FC plus ARS are signicantly more effective than TFF methods.


Introduction
In 2012, active learning was academically de ned by the Ministry of Education, Culture, Sports, Science and Technology (MEXT) with this statement: It is important for students with little motivation and purpose to take the attitude of independently studying. Universities have to reexamine the teaching method, such as by providing opportunities to participate in interactive classes and active activities. "Expected approaches to the university" include the development of interactive learning, the active use of teaching assistants and student assistants, and a small number of students. It illustrates the promotion of guidance and utilization of information and communication technology. (MEXT) Therefore, some universities in Japan have tried to adjust their classes based on this statement.
A ipped classroom (FC) and an audience response system (ARS) have been included in active learning through the utilization of information and communication technology. Some papers have shown positive learning performances for student pharmacists using these teaching methodologies (1)(2)(3)(4)(5)(6). Conversely, a few negative articles also have been reported (7,8). Other academic eld articles regarding the FC have been internationally reported (9)(10)(11)(12)(13). Recently, a meta-analysis for FC in comparison with traditional faceto-face lectures was published and the FC improved students' learning (14). Comprehensively, the FC seems to be a positive way of teaching for student pharmacists. In Japan, few reports on active learning written in Japanese have been published; however, most of them have reported students' preference for active learning. Only an article from Nishigawa et al. compared the FC with team-based learning (TBL) at a school of dentistry in Japan mentioned that there was no statistical difference in the examination score between the FC and TBL, and both styles are more effective than traditional style lectures and constitute valid formats for clinical dental education (15) Thus, active learning outcomes in Japan are uncertain.
Therefore, the purpose of this study was to examine which teaching method is the most effective teaching method for student pharmacists, such as traditional face-to-face (TFF) lectures, ARS lectures, and FC plus ARS (FC plus ARS) lectures.

Targeted students
This study targeted sophomore students taking part in the pharmacology course offered at Ohu University School of Pharmaceutical Sciences from 2017 to 2019. The pharmacology course consisted of 13 categories from their sophomore year to the 4 th grade. Topics lectured were taught as 4 th category in the sophomore year along with endocrine systems; moreover, this included the hypothalamus, pituitary abnormality, adrenal cortical hormone, adrenal medulla hormone, and adrenal cortical dysfunction. These topics were lectured twice a week for two weeks (1.5 hrs /class, 6 h) in the fall quarter. The TFF lectures were performed in 2017; the ARS lectures were carried out in 2018; and the FC plus ARS lectures were performed in 2019. Scores of external exams and prerequisite grade point averages (GPAs) in the freshman year were collected to obtain academic background information regarding sophomore students in 2017, 2018, and 2019. Prerequisite subjects included chemistry, biology, physics, mathematics, introductory chemical thermodynamics, basic science exercises, basic pharmaceutical training, biochemistry, functional morphology, organic chemistry, basic analytical chemistry, and physical chemistry.

Learning outcomes
The pharmacology course placed an emphasis on being knowledge-based, so the expected outcomes were that sophomore students fully understood the concept of endocrine systems.

Audience response system (ARS)
The ARS lectures were carried out using Clica TM (http://clica.jp/LP/). This active learning tool collected the students' understanding, reactions, and opinions in real time during a lecture; subsequently, all of the posts can be displayed on the students' smartphones as well as on a screen in the class. Students can view items and collaborate with each other; this activity can also happen between the lecturer and the students. This is so they can understand the lecture from various perspectives while comparing it with their own opinions. The lecture explained each topic in the class, and multiple-choice questions were administered to the students through Clica TM during the lecture. They responded as quickly as possible and the lecturer and the students viewed their responses on the screen and the lecturer added extra explanations depending on the students' incorrect responses using handouts of the topics.

Flipped classroom (FC) plus ARS
Six-hour lectures/four classes were ipped. Prerequisite videos were uploaded on YouTube beforehand, and the URLs of each video were sent to students' emails at least three days before the lecture. Video times for the rst, second, third, and fourth video were 10 min, 16 s; 10 min, 50 s; 13 min, 16 s; and 10 min, 18 s, respectively. Data regarding the video views and watch times were retrieved from YouTube Analytics. During the class, multiple-choice questions were administered to students through Clica TM at the beginning of each lecture. The lecturer and the students viewed their responses on the screen, and the lecturer added extra explanations depending on the students' incorrect responses using handouts of the topics.

Primary outcome
A primary outcome of the study was to compare the correct rates of exams for endocrine systems in the pharmacology course between three ways of teaching such as the TFF lectures, the ARS lectures, and the FC plus ARS lectures. An exam consisted of 30 multiple-choice questions. Identical questions were administered to each of the three classes' cohort but multiple choices were completely reshu ed every year. A subgroup analysis was performed in terms of gender difference.

Anonymous questionnaire survey
Anonymous questionnaire surveys were conducted at the last lecture on the endocrine systems in the pharmacology course in 2018 and 2019. This was done in order to examine the students' preference for the ARS and the FC plus ARS.

Ethical considerations
This study was approved by the ethics committee of Ohu University (No. 220). The lecturer issued a letter to the students to inform them of the purpose of this study at the end of the last lecture. Written informed consent was obtained from those students who were willing to participate in this study. All methods were performed in accordance with the relevant guidelines and regulations.

Statistics
The sample size for the study was calculated using G*Power software; subsequently, the total sample size was calculated as 159. This gure was arrived at by assuming the use a one-way analysis of variance (ANOVA) for statistical analysis with an effect size of 0.25, alpha error probability of 0.05, power of 80%, and the number of groups being set a three.
The Kruskal-Wallis test was carried out for nonparametric distribution data; duly, a one-way ANOVA was carried out for parametric distribution data. Categorical data were analyzed using the Chi square test and the Fisher's exact test. The ANCOVA was performed if background information between the three cohorts was signi cantly different. P values less than 0.05 were considered statistically signi cant. All statistical analyses were performed with EZR ("Easy R") (Saitama Medical Center, Jichi Medical University, Saitama, Japan), which is a graphical user interface for the programming language, R (The R Foundation for Statistical Computing). More precisely, it is a modi ed version of R commander designed to add statistical functions that are frequently used in biostatistics (16).

Results
Background information regarding the sophomore students in 2017, 2018, and 2019 are shown in Table  1. Notably, the total number of included students exceeded 159 and gender differences in each year were not signi cantly different. External exam scores in the fall of 2019 were not signi cantly different. Additionally, the prerequisite GPA for freshman year in each year was not signi cantly different, except for 2018.
Exam scores for pharmacology are compared in Figure 1 Figure 2. There was a signi cant difference between the three ways of teaching methods (p < 0.001).
Furthermore, there was no signi cant difference between the ARS and the FC plus ARS groups (p = 0.487).
Since a prerequisite GPA in terms of gender difference was not a covariance, a subgroup analysis for the correct rate/GPA between the three ways of teaching in terms of gender difference was performed, as shown in Figure 3. Regarding the female students, the ARS was the most effective method in the three ways of teaching (median: TFF 26, ARS 39, FC plus ARS 33, p < 0.001). Conversely, regarding the male students, the ARS and the FC plus ARS were signi cantly different than the TFF, but not signi cantly different between the ARS and the FC plus ARS (median: TFF 25, ARS 35, FC plus ARS 31, p < 0.001). A Bartlett's test of homogeneity of variances revealed that the variances by the ARS between female and male students were signi cantly different (p = 0.020), whereas those by the FC plus ARS were not (p = 0.532).
The students' response rates for both the ARS lectures and the FC plus ARS signi cantly decreased over time [(1 st : 84.7%, 3 rd : 76.5%, and 4 th : 68.1%; p < 0.001 for one-way ANOVA for the ARS lectures) (1 st : 87.8%, 2 nd : 80.0%, 3 rd : 75.0%, and 4 th : 66.5%; p < 0.001 for one-way ANOVA for the FC plus ARS)]. The 2 nd response rates for the ARS lectures were not obtained because of technical issues.
The numbers of video views and watch times regarding the pre-lecture and post-lecture for the FC plus the ARS are shown in Table 2. Video views for pre-lectures gradually decreased; however, the length of the watch time depended on the topic. Some student pharmacists viewed videos again for review after the lectures.
The results of the anonymous questionnaire survey regarding the ARS and the FC plus ARS are shown in Figure 4. Students' readiness, enjoyment, ful llment, and feeling to talk to other people regarding the FC plus ARS lectures were signi cantly better evaluations than the ARS (p < 0.001, p = 0.019, p = 0.006, and p = 0.005, respectively).

Discussion
This study examined which teaching method is the most effective one for student pharmacists. Accordingly, the methods that were considered were the TFF, ARS, and the FC plus ARS. As a result, the ARS and FC plus ARS, which are called "active learning methods," were effective teaching methods in comparison with the TFF. Subsequently, our research strongly indicates there are no comparative studies regarding teaching methods between the ARS and the FC plus ARS. This study presented the conclusion that there was no signi cant difference between the ARS and the FC plus ARS. Since GPA was not a confounding factor for correct rates in terms of gender difference, correct rate/GPA was evaluated ( Figure  3), suggesting that at least the ARS is an effective teaching method for endocrine systems in the pharmacology course at Ohu University.
Some articles positively reported that the ARS is an effective educational method in classrooms (17)(18)(19)(20)(21)(22)(23)(24)(25). Regarding the retention of learned topics, Rubio et al. reported that long-term retention was maintained until three months later (18). However, Doucet et al. mentioned that one-year post exam scores were signi cantly decreased (19). A systematic review reported by Hussain et al. concluded that the ARS improved student recall immediately following an educational activity, but the effects were not lasting (21). Accordingly, the ARS educational effect is for short-term understanding.
In addition, there have been positive articles regarding FC (3,14,26). Although this study demonstrated that the FC plus ARS was signi cantly more positive than the TFF, it was not more signi cant than the ARS. Additionally, a gender difference was found between the ARS and the FC plus ARS and so there are several reasons to consider this result. As Table 2 shows, the number of video views gradually decreased.
Unfortunately, since YouTube analytics was not able to distinguish viewers from others, the faculty members were not able to identify who in fact watched videos or not. Some articles pointed out gender differences in active learning methods (27); nevertheless, they do not speculate our results. To be sure, variances of correct rate/GPA by the ARS for male students were signi cantly different in comparison with the female students because of the signi cantly lower GPAs of the male students in relation to the female students. This could be because there was no signi cant difference between the ARS and the FC plus ARS for the male students.
Response rates to the ARS in the classes were signi cantly decreased as time went on both the ARS lectures and the FC plus ARS lectures. Probably, most students are familiar with using the ARS in the classes and they would have feel bored. Therefore, some ingenuity to maintain high response rates with the ARS would be necessary for students' academic performance. Therefore, a Jeopardy-style review game would be useful to overcome this problem (28).
Both the ARS and the FC plus ARS were new teaching methods for the student pharmacists at Ohu University. Both methods effectively worked to increase the students' motivation to learn (Figure 4).
Readiness for the FC plus ARS group was signi cantly higher than that for the ARS group. This is reasonable because student pharmacists were required to watch the topical videos before attending lectures. In addition to this, some student pharmacists watched videos from the post-lectures to the date of their exams ( Table 2), suggesting that the FC videos would effectively review topics concerning independent study at home by watching the videos again. Moreover, the FC plus ARS ones were enjoyable and provided a sense of ful llment for the student pharmacists. The most important point to support the results would be that the short videos were approximately 10 to 15 minutes in length. Some papers with positive results mentioned that the students' preference was that the length of the videos were long and the subsequent increase to their workloads from watching them led to unsatis ed feelings. (1,5) One interesting article by Fryer reported that incentives offered for educational outputs-such as better grades-are less effective than those for educational inputs which can include attendance, good behavior, or wearing uniforms (29). Of the positive result articles regarding the FC, Giuliano's article was successful in light of this concept (4). In a course grade composition, the pre-quiz weight was increased from 7.5% to 15% to emphasize the importance of preparation prior to class. The current study did not offer any incentives to student pharmacists for educational inputs and outputs. Therefore, some incentives presented to student pharmacists for educational inputs would work much better in understanding topics in pharmacology course.
This study had some limitations, with the rst one being the costs for preparing the FC videos were not evaluated. Although creating video les with PowerPoint software and using YouTube and Clica TM were free, the workloads for preparing the videos such as recording explanations and inserting them into a PowerPoint le took too much time. Furthermore, this study was performed by only two lecturers so that expanding this teaching method to all faculty members would not be guaranteed because of a lack of an evaluation of the time consumption. A way of thinking about using teaching assistants would be a good option to solve this limitation (30). On the other hand, it was reported that faculty time to implement a FC required 127% more time than in the previous year's traditional lecture format. Yet, that time was expected to decrease once the course was established (26). Then, although the current study elucidated short-term effects to keep fundamental knowledge-based contents by the active learning methods, the long-term effects were uncertain. Therefore, examinations of their long-term effects would be necessary for future academic strategies.

Conclusion
This study examined which teaching method is the most effective one for student pharmacists. The methods considered in this regard were the TFF, the ARS, and the FC plus ARS. In conclusion, the ARS and the FC plus ARS, which are called active learning methods, were the most effective teaching methods for endocrine systems in pharmacology course.

List Of Abbreviations
The

Consent for publication
Not applicable.

Availability of data and materials
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.   (1) 4th Adrenal Cortical Dysfunction (2)