Initial experience with pediatrics online learning for nonclinical medical students during the COVID-19 pandemic

Background: To minimize the risk of infection during the COVID-19 pandemic, the learning mode of universities in China has been adjusted, and the online learning of clinical medicine is facing great challenges. This study preliminarily discusses the experience of express team-based learning (eTBL) combined with a flipped classroom (FC) and case-based learning (CBL) online for nonclinical medical students and addresses the distribution of online learning resources used in pediatrics. This study helps to document additional experience in online learning during the global trend of digital learning. Methods: When online learning was fully launched at Sichuan University in the spring of 2020, 236 penultimate-year students of nonclinical medicine majors were selected as the research objects. The penultimate-year students of the same majors in the spring of 2019 were taken as the reference objects. The research objects successively used the methods of eTBL combined with FC and CBL methods to conduct online learning in pediatrics, and students were encouraged to search and share online learning resources. The reference objects used the method of eTBL combined with CBL for offline face-to-face learning, and the test results of the two learning environments were compared. At the end of the pediatrics course in the spring of 2020, the research objects were invited to participate anonymously in an online questionnaire survey involving 12 items on a 5-point Likert scale (1 = strongly disagree, 5 = strongly agree) to evaluate the participation, satisfaction, and distribution of online learning resources used in pediatrics. Results: 1. Student participation and test scores: i. A total of 75.8% (179/236) of the respondents completed the questionnaire effectively, and 7 items on the Likert scale indicated that online the main learning resources for nonclinical medical students. iv. Both online combined learning methods were helpful for students to share online learning resources. eTBL + FC was more helpful in retrieving online learning resources, and the workload was also larger, while eTBL + CBL was more helpful for students to understand course content. qualitative count data, and Fisher’s exact test was used for data of less than 5 in the table. The overall fractional normal distribution was assessed using the skewness and kurtosis Mann-Whitney tests. The self-rating scores of online information retrieval application ability and the average scores of each theme were compared by a two-tailed paired t-test. P < 0.05 indicated that the difference was statistically significant. A bar chart was created to show the distribution of students' participation, understanding and application, and medical information retrieval.


Introduction
Under the current situation of the global COVID-19 outbreak, offline learning was forced to be abruptly interrupted, and China took the lead in "unceasing study" with an online learning strategy. The network platform replaced campus as a new teaching environment, and learning resources were extended to online learning resources, such as literature retrieval, online teaching videos and interactive materials [1]. Different from traditional theoretical learning, clinical medicine pays more attention to the integration of theory and practice, and there is more unpredictability in comprehensive online learning. Its effectiveness may be hindered by factors such as communication network support, social interaction, learners' motivation and preferences [2] [3]. The comprehensive online learning of clinical medicine faces great challenges [1] [4].
Relevant literature published after the outbreak of the epidemic proposed that comprehensive online learning should consider students' satisfaction, interactivity and applicability [2] [4] [5]. It was necessary to adjust the online learning scheme and verify its practical effect on the basis of existing online and offline learning experiences.
In 2019, based on the systematic evaluation and meta-analysis of the effectiveness of online learning and offline learning for medical undergraduates, Leisi Pei et al. [6] proposed that online learning was a potential learning method, and interactive online learning could help improve the online learning effect [7]. Team-based learning (TBL) is an interactive learning method with students as the center and teachers as the guides [8]. Online TBL during the epidemic period has helped strengthen students' communication [8] [9]. A meta-analysis on the effectiveness of TBL in medical education in China suggested [10] that compared with traditional lecture-based learning (LBL), TBL could effectively improve academic performance, attitudes and skills in medical theoretical education. Chaya Gopalan et al. [11] proposed that flipped classrooms (FCs) under the premise of TBL combined with literature retrieval learning were the most suitable learning method for teams and were conducive to improving the cultivation of the application ability of theoretical knowledge [11]. Case-based learning (CBL) addresses the integration of medical theory and practice and is one of the deep learning methods in medical education, along with TBL and FC.
CBL is a good beginning for students to transition from traditional theoretical knowledge learning to practice and is suitable for nonclinical medical students who graduate in the penultimate year [12].
Taking the clinical pediatrics course of the Pediatric Teaching and Research Office in the West China School of Medicine of Sichuan University as an example, the Pediatric Teaching and Research Office has rich experience in TBL clinical teaching. The teaching method of appointing experienced clinical pediatricians, such as attending physicians and professors, is responsible for teaching that has been favored by most students. To maintain students' enthusiasm for clinical pediatrics, avoid the decrease in students' communication during isolation, and achieve the quality of online learning equivalent to offline learning, the Pediatric Teaching and Research Office adjusted the online learning mode of the pediatrics course in the spring of 2020 on the basis of the offline Express TBL (eTBL) learning mode in the spring of 2019. This paper aims to explore and evaluate students' participation, satisfaction, test scores and the use of online learning resources under the online learning environment by designing the online learning mode of pediatrics courses and questionnaires.

Research objects
The objects of this study were penultimate-year undergraduate students of the nonclinical medicine college of Sichuan University who took pediatrics as a selective course in the spring of 2020, involving the School of Public Health, School of Basic Law and School of Pharmacy, with 207 students in grade 2017 from Prevention, Basic Medicine and Forensic Medicine (all four-year students) and 29 students of grade 2016 from Clinical Pharmacy (five-year students). A total of 246 penultimate-year students with the same majors in the spring of 2019 were taken as the reference objects to compare their after-class test scores. Nonclinical medical students who were in other grades or taking pediatrics courses but not participating in eTBL pediatrics were excluded.

This study was approved by the Medical Ethics Committee of West China Second University
Hospital of Sichuan University.

Teaching design
Referring to the offline eTBL learning scheme in the spring of 2019, the online eTBL learning scheme of the same specialty in the spring of 2020 ( Fig. 1) was designed. The research objects used a small private online course (SPOC) for theoretical knowledge learning, and on the basis of eTBL, combined FC and CBL methods successively for integrating theory and practical application. The reference objects adopted offline LBL for theoretical knowledge and the eTBL + CBL method for integrating theory and practical application.
Note：LBL: Lecture-based Learning， eTBL：express Team-based learning， CBL: Case-based learning， SPOC：Small Private Online Course, FC: Flipped Classroom. 3 Implementation method of online learning (Table 1 for each stage) Online learning grouping: All students were divided into 24 groups according to the order of the name list, with a fixed number of 9-10 students in each group. We adopted the principle of voluntary basis for the students, designated the teacher and set a leader in each group. According to the learning schedule and grouping order, the 24 groups were divided into 4 online modified TBL pediatrics classes, each consisting of 6 groups of students.

3.3Course application
For the purpose of reducing the number of students' learning tasks, the order and method of 4 "S" principles were adjusted (see Table 2). Each group previewed the relevant questions/tasks of each theme in advance, integrated the preview materials, and unified the lecture notes/answers. To urge the students to master the key points of each theme comprehensively, the students were randomly selected to explain the basis of the answers or comment and discuss with each other to promote the students' healthy competition in learning. After learning each theme, the teacher would make a brief summary. To encourage discussion among students, the number of effective discussions between groups (the content was not completely repeated each time) was regarded as the evaluation standard of reward, which was included in the class performance score together with attendance. At the same time, to avoid the subjective bias caused by the mutual evaluation between teachers and students, the subjective scoring of teachers and students was canceled, and after-class test questions were used as one of the evaluation contents of students' learning effect. The after-class test questions were from the pediatric question bank on the website of the Sichuan University Course Center. Five single-choice questions were the key and difficult points were discussed among students in the course application stage. Bonus points would be awarded according to the number of effective discussions in class, and all of the students in a group would get the same points. Personal tests (including before and after class) and class performance scores were included in the usual score, accounting for 36% of the final score (personal test 24%+ class performance 12%). to evaluate the students' participation, satisfaction and online learning resource distribution.5 Statistical analysis SPSS22 was used to analyze the data. The data collection database (Jinshuju) automatically generated the proportion of students in the questionnaire. The chi-square test was used for qualitative count data, and Fisher's exact test was used for data of less than 5 in the table. The overall fractional normal distribution was assessed using the skewness and kurtosis Mann-Whitney tests. The self-rating scores of online information retrieval application ability and the average scores of each theme were compared by a two-tailed paired t-test. P < 0.05 indicated that the difference was statistically significant. A bar chart was created to show the distribution of students' participation, understanding and application, and medical information retrieval.

General information
The The questionnaire asked about the learning environment: "Comparing the online environment with the offline environment, which environment can retrieve more online learning resources?" More than half of the students (63.7%, n = 114) chose an online learning environment, less than 5% (4.5%, n = 8) chose an offline learning environment, and nearly 1/3 students chose "the same".
In the questionnaire, questions were asked about the online learning resources between the Average score of each item online and offline learning environments (Fig. 3), "For comparison of eTBL + FC and eTBL + CBL learning methods (the first vs. the second), ① Which learning method has more workload?
② Which learning method shares more online learning resources? ③ Which learning method searches more online learning resources? ④ Which learning method is more helpful to the understanding of pediatrics?" Answer options: "the first is obviously more, the first is more; the two are the same; the second is more, and the second is obviously more." The chi-square test (Fisher's exact test) of the remaining four options (excluding the option "the two are the same") showed that more students believed that the online learning method of eTBL+ CBL was more  The top three were: pre-class preparation (4.83), class discussion (4.28) and preparation of personal confirmation test (3.79) (see Table 3).2.3 Time distribution of medical information retrieval by students. In the questionnaire, the options of time period were divided into four levels ("most or all of the time", "often", "occasionally", "rarely or never"). If any expected frequency was greater than 5, the chi-square test was used,  2 =341.183, P <0.001, suggesting that the number of students choosing different time periods was different. A total of 72.9% (n = 129) of the students "most or all of the time" retrieved information in the pre-class preparation stage; 35-40% of the students "often" retrieved information in the pre-class test stage (37.7%, n = 66) and the course application stage (35.6%, n = 63). Approximately 40% of the students "occasionally" and "rarely or never" retrieved information, and the time period was mainly in the after-class test stage (39.2%, n = 69, 41.5%, n = 73) (Fig. 4).   Students were encouraged to search online learning resources through the objectives of the case study/flipped classroom lecture task in online eTBL combined learning. Students' information retrieval ability was self-rated (1 = very low, 5 = very high). The Wilcoxon signed-rank test showed that students' information retrieval ability after the modified TBL combined teaching method was improved compared with beforehand (median 5 vs 4, U = 591.0, P = 0.007) (Fig. 5).  In the online learning of 2020, the learning method of "the first four themes" was eTBL + FC combined, while the learning method of the "last four themes" was eTBL + CBL combined. Δ, comparison of the after-class test results of the first four themes in 2019 and 2020, ▽, comparison of the after-class test results of the last four themes in 2019 and 2020.

Test results
The results of the paired T-test showed that (Table 6) the total average scores of the last four themes were higher than those of the first four themes. The total average score of eTBL combined with CBL was higher than that of eTBL combined with FC in the spring of 2020, which was consistent with the difference in the same themes before and after the offline learning method in the spring of 2019, but the same offline learning method was used for all themes in the spring of 2019. Therefore, the comparison of the total average scores of the different themes could not be used as a basis for the difference in scores caused by different online learning methods. Whether this was due to the difficulty of the course content or students' gradual adaptation to pediatric learning deserves further study. In the online learning of 2020, the learning method of "the first four themes" was eTBL + FC combined, while the learning method of the "last four themes" was eTBL + CBL combined. Δ, The results of the independent t-test (Table 6) showed that the average score of online learning in the spring of 2020 was higher than that of offline learning in the spring of 2019 (the first four themes: online vs offline, 90.4 vs 83.9, t = 9.187, P < 0.001; the last four themes: online vs offline, 93.2 vs 87.1, t = 6.278, P < 0.001). In summary, the test result of online learning was not worse than that of offline learning.

Discussion
Online education prevails during the epidemic of COVID-19, requiring internet infrastructure support. Jessica C et al. [2] pointed out that during the epidemic period, some students' participation was interrupted due to poor Internet connections and poor broadband in Comprehensive online learning needs not only network support but also appropriate learning methods to guide student learning. FC on the basis of TBL combined with literature retrieval is the most suitable teaching method for the team [11]. FC advocates student responsibility and emphasizes students' pre-class preparation. The questionnaire in this paper shows that online eTBL+ FC has a larger workload than the eTBL+ CBL online learning method, which is more beneficial to online learning resource retrieval. The main reason is that FC requires more time and energy from the students [13] and encourages students to use online learning resources [13] [14]. Online learning has the advantages of time flexibility, acquiring learning resources on demand and strengthening the cultivation of autonomous learning ability [15], which is consistent with the advantages of typical FC. It can effectively improve students' learning enthusiasm, experience and academic performance [13] [14], which is helpful for medical students to strengthen the cultivation of self-directed learning skills and benefit them for life [16].
In medical education, exposure to clinical cases is the best deep-learning method [17].
Undergraduate students in the penultimate year are in the stage of transition from theoretical knowledge to practical application, and CBL is a good choice for students in this stage [12]. Dylan Thibaut et al. [18] suggested that students use the CBL method within a team during the epidemic period in 2020, which would be helpful for students to exchange and share information. More students in our questionnaire thought that online eTBL + CBL was more conducive to students' understanding of pediatrics, and the workload was smaller, which was consistent with the advantages of the typical CBL method with a small workload [17] and flexibility in case selection [17]. It was helpful for students to understand theoretical knowledge and apply it in practice [17], improving their clinical reasoning ability, critical thinking and problem-solving, and was an effective strategy for clinical medical students to learn basic course content [19].
Both online learning methods are carried out in teams to help students share online learning resources, and their test scores are as good as those for offline tests. The questionnaire showed that students had high participation and satisfaction in both online combined learning methods of pediatrics, and the online learning environment encouraged students to retrieve online learning resources. The main motivation for students to retrieve online learning resources was pre-class preparation, class discussion and pre-class testing, which was in accord with the retrieval time distribution of the students. It might be related to learning tasks or objectives that need further study.
A study conducted by Judd T et al. [20] on the selection and use of online resources for  [22]. Combined with the Medline search interface, it helps balance accuracy and sensitivity [23]. In the questionnaire, undergraduates as beginners can correctly choose global medical literature information retrieval tools.
Online medical literature information retrieval skills provide information resources for medical students to understand and master new medical knowledge and theories. Online learning resources are easy to access, and flexible on-demand learning can be used as a learning aid. In the long run, effective use of online learning resources supports lifelong learning and provides potential tools for clinical practice or medical employment.
In this paper, the research objects' cognition of the offline learning environment originates from their learning in other courses. The same group and the same theme cannot use both online combined learning methods simultaneously. The evaluation of students' ability to identify online resources originates from the students themselves, which lacks authoritative evaluation and cannot accurately describe students' actual ability. There are tests before and after online live class. The reference subjects only have two tests on the same themes. The comparison results of average test scores can only be used as a reference.
Our article did not further analyze a small number of students who may not like team learning but prefer to complete cases/tasks online by themselves [24]. There may be conflicts between limited attention and intensive online learning tasks in students' online learning of multiple courses, which have higher demands on students' time and energy [15]. Whether there is mutual influence within these multiple online learning courses remains to be further explored.
When the epidemic situation in some countries or regions improves, medical education will gradually transition to synchronized online and offline learning or even resume the traditional offline teaching mode. The changing epidemic situation and different learning equipment and software in different regions still require the individualized design of teaching modes.

Conclusion
During the COVID-19 epidemic, online eTBL shortens the learning time of typical TBL. eTBL combined with FC or CBL online, in combination with online learning resource retrieval, receives the majority of students' satisfaction, helps students communicate, promotes the cultivation of the application ability of theoretical knowledge, and test results are as good as in the offline environment learning. Students can correctly choose the retrieval tools of online learning resources, and the motivation mainly comes from the requirements of learning tasks. Team-based FC and CBL learning methods emphasize student responsibility, which is conducive to students' sharing of online learning resources. Online eTBL+ FC is more beneficial for retrieving online learning resources, but it also requires more time and energy. Online eTBL+ CBL is more conducive to students' understanding and application and is more suitable for the transition stage between theory and practice. In the long run, choosing an appropriate teaching mode is beneficial to students' study on campus, clinical practice and clinical work.

Availability of data and materials
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.