A meta-analysis of effects of blended learning on performance, attitude, achievement, and engagement in different countries

doi:10.21203/rs.3.rs-536691/v1

Download PDF

Systematic Review

A meta-analysis of effects of blended learning on performance, attitude, achievement, and engagement in different countries

https://doi.org/10.21203/rs.3.rs-536691/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Although the new century has been witnessing increasing popularity of blended learning especially during this special pandemic time, few studies have summarized the effectiveness of blended learning in different countries. This meta-analysis summarizes previous studies on blended learning effectiveness in different countries in terms of students’ performance, students’ attitudes towards blended learning, learning achievement, and student engagement in different countries. Through the meta-analysis via Stata/MP 14.0, it is concluded that blended learning could improve performance, attitude, and achievement in most countries. However, in both China and the USA, blended learning could not significantly improve student engagement in academic activities. No significant differences were revealed in student performance in the USA between blended and non-blended learning. Future research could extend the research into blended learning to more countries and areas across the world.

Educational Philosophy and Theory

Distance education and online learning

Improving classroom teaching

Learning communities

Mobile learning

Pedagogical issues

This special pandemic time has been witnessing the popularity of blended learning approaches. However, very few studies have summarized blended learning effectiveness in different countries. It is thus meaningful and necessary to examine the effectiveness of blended learning across the world especially during this special time.

1.1 Definitions of blended learning

Blended learning, a combination of virtual and physical learning conditions (Al-Qatawneh, Eltahir, & Alsalhi, 2020), is defined as a learning strategy integrating two different educational models, e.g. distance and traditional learning (Bonk & Graham, 2006). There are three most popular definitions of blended learning (Curtis et al., 2005), blending instructional modalities (Bersin & Associates, 2003; Singh & Reed, 2001; Thomson, 2002), instructional methods (Rossett, 2002), and online learning with face-to-face instructional approaches (Ward & LaBranche, 2003; Young, 2002).

1.2 Performance

Most studies positively reported blended learning performances. Blended learning, outperforming full online learning in the aspects of motivation, attitudes, and satisfaction, could improve nurses’ clinical knowledge compared with the traditional learning approach in the UK (McCutcheon, O’Halloran, & Lohan, 2018). Compared with traditional face-to-face learning, blended learning could optimize the learning flexibility in terms of time and space, leading to stable learning performance of undergraduates in The Zurich University in Germany (Mueller, Mildenberger, & Lübcke, 2020). It was revealed that both classroom and online learning could enhance American students’ learning performance, but the blended learning brought about the largest gain in performance in the USA (Hill, Chidambaram, & Summers, 2017). Blended learning could lead to significantly higher learning performance than e-learning, while the flipped classroom could improve intrinsic motivation and self-efficacy in Can Tho University in Vietnam (Thai & Valcke, 2017).

Numerous studies reported that blended learning was beneficial to language proficiency improvements. Blended learning could greatly improve the reading abilities of children in a kindergarten in the USA (Macaruso, Wilkes, & Prescott, 2020). Blended instruction could greatly improve students’ English writing abilities, e.g. content relevance, content sufficiency, organization structure, and language expression in Ankang College, Shanxi China (Zhou, 2018). Blended learning could improve students’ English listening and speaking and critical thinking skills, e.g. analysis, inference, evaluation, induction, and deduction in China (Yang, Chuang, Li, & Tseng, 2013). Blended learning could enable Chinese college students to extensively practice with flexible time and space, greatly improving their English reading skills (Yang, 2012).

Blended learning could also enhance high-order abilities such as communication, problem-solving, and reasoning skills. Blending a class video blog into face-to-face instruction could improve language oral proficiency but failed to greatly improve the voluntariness to communication using the target language in China (Liu, 2016). Blended learning could effectively facilitate communication skills and improve learning outcomes of nursing tertiary students in Singapore (Shorey, Kowitlawakul, Kamala Devi, Chen, Soong, & Ang, 2018). In the blended learning, Chinese students could discuss with peers, propose meaningful ideas, mutually learn and share, improve group work skills, enhance self-perception, and facilitate reasoning skills (Monteiro & Morrison, 2014). Blended learning could enhance acute stroke patients’ competences, e.g. recognition and management in the USA (Lee Gordon, Issenberg, Gordon, Lacombe, Mcgaghie, & Petrusa, 2005).

1.3 Attitude

Most of the blended learning participants held positive attitudes towards blended learning effectiveness. Blended learning, conducive to students’ positive attitude and satisfaction, could improve English listening skills and enhance vocabulary acquisition among junior middle school students in China (Jia, Chen, Ding, & Ruan, 2012). Chinese 11th graders held significantly more positive attitudes towards blended learning than traditional learning (Chang, Shu, Liang, Tseng, & Hsu, 2014). Singaporean nursing college students had greatly positive attitude towards blended learning, as well as communication skills in the blended context (Shorey, Kowitlawakul, Kamala Devi, Chen, Soong, & Ang, 2018). The blended model in active learning classrooms obtained positive evaluation and students held improved attitudes towards physics courses in North Carolina State University in the USA (Beichner, Saul, Abbott, Morse, Deardorff, & Allain et al., 2007). Blended learning could improve nursing students’ motivation, satisfaction, and attitude in clinical supervision skills compared with online-only learning in China (Chang, Shu, Liang, Tseng, & Hsu, 2014).

1.4 Achievement

Many studies reported that blended learning could contribute to higher learning achievements than traditional approaches. Blended learning could lead to significantly higher academic achievements than traditional face-to-face learning in Canada (Bazelais & Doleck, 2018). Online learning activities could improve students’ academic achievements among undergraduate students in University of Granada in Spain, where influencing factors included attendance rate and students’ backgrounds rather than the time they spent on learning (López-Pérez, Pérez-López, Rodríguez-Ariza, & Argente-Linares, 2013). Blended learning via information and communication technologies could significantly improve learning achievements of mechanical couplings in engineering in Spain (Cortizo, Rodriguez, Vijande, Sierra, & Noriega, 2010). A blended and flipped pedagogical approach could improve learning achievements and learning environment and raise the efficiency of space use in the USA (Baepler, Walker, & Driessen, 2014).

1.5 Engagement

Most previous studies reported that blended learning could improve learning engagement. Blended learning, encouraging students to engage in learning even after class, could lead to a significantly higher frequency and level of engagement than the traditional learning in Spain (Pérez-Marín & Pascual-Nieto, 2011). In the technology-oriented blended learning, Chinese freshmen used to spend more time on in-class discussion and writing tasks than the efficiency-oriented group. The interaction was considered an important factor influencing blended learning effectiveness among Chinese freshmen (Yen & Lee, 2011). Undergraduates at Point Loma Nazarene University in the USA spent significantly more time learning in a blended instruction model than in the traditional instruction model (Botts, Carter, & Crockett, 2018). Blended learning could improve Chinese students’ engagement by increasing their learning efficiency and effectiveness (Monteiro & Morrison, 2014).

1.6 Contradictory findings

However, there were contradictory findings about the effect of blended learning in different countries in terms of achievement. It was reported that blended learning could not significantly improve students’ achievements in China although they themselves believed so (Chang, Shu, Liang, Tseng, & Hsu, 2014). No significant differences in Fashion learning achievements were revealed in Hong Kong China between blended learning and traditional face-to-face learning (Yick, Yip, Au, Lai, & Yu, 2019). Blended learning did not indicate any effect on learning outcomes of economics courses in an American university (Olitsky & Cosgrove, 2014). While blended learning could significantly improve self-assessment of students’ knowledge gains, it could not greatly improve their actual learning achievements among Chinese Vocational High School Students (Chang, Shu, Liang, Tseng, & Hsu, 2014).

Contradictory findings were also found regarding the effect of blended learning on attitude, performance, and engagement. There are no significant differences in attitudes towards blended or traditional learning, which might be subject to academic evaluation, teaching experience, and computer skills in the United Arab Emirates (Al-Qatawneh, Eltahir, & Alsalhi, 2020). Different models of blended learning could lead to various performance levels. Efficiency-oriented blended learning could significantly improve problem-solving performance of Chinese freshmen than hybrid-oriented and the technology-oriented groups (Yen & Lee, 2011). No significant difference in learning outcomes was found between blended and traditional instructions in the upper-division quantitative literacy course at Point Loma Nazarene University in the USA (Botts, Carter, & Crockett, 2018). In blended learning, students were less engaged in problem-solving issues and students in Point Loma Nazarene University in the USA spent less time in course learning, followed by reduced seat time (Botts, Carter, & Crockett, 2018). No significant differences in self-efficacy and knowledge score were revealed between blended and non-blended methods although students positively perceived blended learning at an undergraduate university in Alberta, Canada (Berga, Vadnais, Nelson, Johnston, & Olaiya, 2021).

1.7 Research questions

Considering the inconsistent findings regarding the effect of blended learning on performance, student attitude, achievements, and engagement in different countries, we propose four research questions, i.e. (1) Could blended learning positively influence student performance in different countries? (2) Could blended learning positively influence student attitude in different countries? (3) Could blended learning positively influence learning achievement in different countries? (4) Could blended learning positively influence student engagement in different countries?

This meta-analysis was implemented according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) (Moher, Liberati, Tetzlaff, Altman, & The PRISMA Group, 2009). The registration of the meta-analytical protocol was waived by the author’s academic review board due to the characteristics of the study.

2.2 Eligibility criteria

The studies will be considered eligible and included if they (1) focus on the effect of blended learning on performance, student attitude, achievements, and engagement in different countries; (2) are highly evaluated using University of West England Framework for Critically Appraising Research Articles (Appendix A) (Moule et al., 2003); (3) can provide enough data for a meta-analysis; (4) divide the participants into both control and experimental groups for a comparative analysis between blended learning and non-blended learning; and (5) are written in the standard English language.

The studies will be considered ineligible and excluded if they (1) focus on blended learning technologies themselves rather than blended learning effect; (2) cannot provide enough data for a meta-analysis even after we correspond with the authors; (3) are not written in English; or (4) they are poorly evaluated using University of West England Framework for Critically Appraising Research Articles (Appendix A) (Moule et al., 2003).

2.3 Data sources and search strategy

Based on the PRISMA flow (Fig. 1), we conducted the inclusion and exclusion process. To maximize the number of data included, we searched the databases from their inception until February 26, 2021 without time limitation. We entered keywords and index terms, e.g. blended learning, performance, attitude, achievements, and engagement, different countries, into different databases according to their specific syntactical rules. We obtained 12098 results by searching four online databases, i.e. Elsevier ScienceDirect, Taylor & Francis Group, EBSCOhost, and Springer. Then we entered the results into ENDNOTE X8 (Thomson Reuters, New York, USA) to remove those duplicated. Then we invited two researchers to double-check whether or not the results are related to the study by screening the titles and abstracts. After this, they conducted the evaluation of eligibility of the results.

Finally, two researchers met to decide on the included studies for the meta-analysis. They discussed different selected studies and negotiated to address the disputes. Those selected by both of them were directly included in the meta-analysis. A third reviewer will be invited to finally determine the finally selected studies in case two researchers cannot reach an agreement on the inclusion of any study.

2.4 Evaluation of included studies

We evaluated the full texts via University of West England Framework for Critically Appraising Research Articles (Appendix A) (Moule et al., 2003). This framework evaluates the research articles based on five sections, i.e. the introduction, the methods, ethics, the results/findings, and the conclusions. Each section has detailed criteria for evaluation. For example, for the section of introduction, we examine whether there is a clear statement about the topic being investigated and whether there is a clear rationale for the research. For the conclusion section, we examine whether the implications for further research are acknowledged, whether areas for further research are identified, and whether further recommendations are made for practice that come from the results. For the method section, we use different criteria for different methods, e.g. qualitative or quantitative research. We also use specific criteria to evaluate data collection and analysis. We finally included 26 results for the meta-analysis (Table 1). The inter-rater consistency reaches a satisfactory level (Cohen's kappa coefficient = 0.83). This indicates that two researchers mostly selected the same studies or generally reached an agreement on most of the selected studies.

2.5 Data extraction

As shown in Table 1, two researchers extracted specific information such as author, publication year, and the source of the literature. We also collected enough data for the meta-analysis such as means, standard deviations, and numbers of participants for both control and experimental groups. For convenience of analysis, we classified the findings into performance, attitude, achievements, and engagement, followed by the countries where the studies were conducted. The selected were implemented in various countries across the world such as China, the United Arab Emirates, Canada, the USA, Spain, Germany, Singapore, and Vietnam. We will compare different effects of blended learning in these countries. Similarly, both researchers would meet up to discuss different results of data extraction and a third reviewer would be invited to decide the final data if any disagreement occurred between two researchers. The inter-rater consistency also reaches a satisfactory level (Cohen's kappa coefficient = 0.81).

Table 1

The included studies for the meta-analysis
N	Author/year	Source	Subgroup	Country
1	Al-Qatawneh et al., 2020	Springer	Achievement and attitude	The United Arab Emirates
2	Baepler et al., 2014	Elsevier ScienceDirect	Achievement, attitude, engagement, and performance	USA
3	Bazelais & Doleck, 2018	Springer	Achievement	Canada
4	Botts et al., 2018	Taylor & Francis Group	Engagement	USA
5	Chang et al., 2014	EBSCOhost	Achievement, attitude, and performance	China
6	Cortizo et al., 2010	Elsevier ScienceDirect	Achievement	Spain
7	Gordon et al., 2005	EBSCOhost	Achievement	USA
8	Hill et al., 2017	EBSCOhost	Achievement	USA
9	Jia et al., 2012	Elsevier ScienceDirect	Achievement	China
10	Liu, 2016	Elsevier ScienceDirect	Achievement and performance	China
11	López-Pérez et al., 2013	Springer	Achievement	Spain
12	Macaruso et al., 2020	Springer	Achievement	USA
13	McCarthy et al., 2020	Taylor & Francis Group	Achievement	USA
14	McCutcheon et al., 2018	Elsevier ScienceDirect	Achievement and attitude	UK
15	Monteiro & Morrison, 2014	Taylor & Francis Group	Engagement	China
16	Mueller et al., 2020	Taylor & Francis Group	Achievement	Germany
17	Olitsky, & Cosgrove, 2014	Elsevier ScienceDirect	Achievement and performance	USA
18	Pérez-Marín, & Pascual-Nieto, 2011	Springer	Achievement	Spain
19	Shorey et al., 2018	Elsevier ScienceDirect	Attitude	Singapore
20	Thai et al., 2017	Elsevier ScienceDirect	Achievement, attitude, and performance	Vietnam
21	Yang et al., 2013	Elsevier ScienceDirect	Performance	China
22	Yang, 2012	Taylor & Francis Group	Performance	China
23	Yeh et al., 2011	Elsevier ScienceDirect	Performance	China
24	Yen & Lee, 2011	Elsevier ScienceDirect	Achievement	China
25	Yick et al., 2019	Taylor & Francis Group	Achievement	China
26	Zhou, 2018	EBSCOhost	Achievement	China

2.6 Statistical analysis

We meta-analytically examined the data using Stata/MP 14.0. After entering data such as numbers of participants, means, and standard deviations for both groups into Stata/MP 14.0, forest plots will be drawn. We calculated standardized mean difference (SMD or Cohen d) (Cohen, 1988) indicating the effect sizes, weight indicating the degree of the influence on pooled results, and 95% confidence interval indicating the study reliability. Cohen d is produced through dividing the mean difference between both groups by the pooled standard deviation of both groups (Sedgwick & Marston, 2013). The formula is: Cohen d = (M2-M1)/Pooled SD, where M1 indicates the mean of the control group) and M2 indicates the mean of the experimental group. The effect size will be deemed very small if d approximates 0.1, small if d approximates 0.2, medium if d approximates 0.5, large if d approximates 0.8, very large if d approximates 1.2, huge if d approximates 2.0 (Sawilowsky, 2009).

To determine whether a random-effect or a fixed-effect model could be adopted, we also tested the heterogeneity of the effect sizes using I² and p values. The formula to calculate I² is: I² = [(Q-df)/Q] × 100%, where Q indicates the Chi-squared statistics and df means the degree of freedom (Higgins & Thompson, 2002; Higgins, Thompson, Decks, & Altman, 2003). This indicates the degree of percentage of the variability in effect sizes caused by heterogeneity or random errors. According to Higgins & Green (2021), the heterogeneity will be considered not important in case I² ranges from 0–40%, moderate in case I² ranges from 30–60%, substantial in case I² ranges from 50–90%, and considerable in case I² ranges from 75–100%. Generally, if I² is larger than 50% (p < .05), we will adopt a random-effect model to conduct the meta-analysis, and if I² is smaller than 50% (p > .05), we will use a fixed-effect model to run the meta-analysis. The influence analysis program will be used to run the sensitivity analysis. Both Begg’s (Begg & Mazumdar, 1994) and Egger’s (Egger, Smith, Schneider, & Minder, 1997) tests will be used to test the publication bias.

3.1 Tests of publication bias

To test the publication bias, we firstly entered data, e.g. means, standard deviation, and numbers of participants across both groups, into Stata/MP 14.0 to run the meta-analysis. Then, we obtained effect sizes (ES) and standard errors of effect sizes (seES) for the test of publication bias. We tested the publication bias by entering “ES, seES” into Stata/MP 14.0, leading to a funnel plot (Fig. 2) and related data. A dot indicates an individual study, and the middle line is the no-effect line. If the dots are symmetrically distributed along both sides of the no-effect line, there will be an absence of publication bias. On the contrary, the asymmetrical distribution indicates the presence of publication bias. As shown in Fig. 2, it is hard to conclude that the dots are symmetrically distributed, indicating the presence of publication bias. Both Begg’s (Q = 1016, S.D. = 381.89, z = 2.66, p = 0.008) and Egger’s tests (Coefficient = 1.55, S.E. = .48, t = 3.25, p = 0.002, 95%CI = .60 ~ 2.48) also indicate the presence of publication bias.

3.2 A sensitivity analysis

We conducted a sensitivity analysis to test the reliability and stability of the obtained effect sizes using the program “metan-based influence analysis”. To retrieve the result, we entered the data such as means, standard deviations, and numbers of both groups into Stata/MP 14.0. We adopted a random-effect model to conduct the sensitivity analysis due to the high degree of percentage of variability caused by heterogeneity (Q = 1053.01, I² = 89.7%, z = 8.88, p < .01).

Unstable ES estimates often lead to skewed distribution and are frequently located beyond the lower and upper bounds of 95% confidence intervals (Borenstein, Hedges, Higgins & Rothstein, 2009). It is thus a must to identify whether there is any estimate located beyond the scope of 95% confidence intervals (Borenstein, Hedges, Higgins & Rothstein, 2009). As shown in Fig. 3, a dot indicates an estimated effect size of an individual study. All the effect sizes are located within the low and upper bounds of 95% confidence intervals. This indicates that there are no unstable ES estimates. We, therefore, conclude that the meta-analysis results are stable.

3.3 Could blended learning positively influence student performance in different countries?

To determine student performance in blended and non-blended learning modes in different countries, we retrieved 27 effect sizes from different countries, where 18 effect sizes sourced from China, 8 from the USA, and 1 from Vietnam. We failed to obtain an effect size from a study (Yang, Chuang, Li, & Tseng, 2013) because one of the standard deviation values is zero. We obtained meta-analytical data and a forest plot (Fig. 4) after entering means, standard deviations and, numbers across both groups into Stata/MP 14.0 to run the meta-analysis by the variable country.

As shown in Fig. 4, the diamonds at the bottom indicate the pooled results. In the left-most column are displayed the author names and publication years, followed by a middle line with numerous boxes. The middle line is referred to as a no-effect line because if a diamond crosses it, the result will be considered insignificant. A box, integrated with a horizontal line and a dot, indicates an individual study. The length of the horizontal line is negatively related to the reliability of the study. The dot indicates the SMD. On the right are displayed the statistics of SMDs (Cohen d) and 95% confidence intervals after them. The right-most column shows the weights indicating the influence of effect sizes on the pooled result.

We adopted a random-effect model to run the meta-analysis of the data sourcing from China (I² = 91.2%, p < .01), the USA (I² = 90.4%, p < .01) and Vietnam (a single study) due to a generally high degree of percentage of variability caused by heterogeneity (I² = 92.9%, p < .01).

As for the meta-analysis of data sourcing from China and Vietnam, the diamonds are located to the right of the no-effect line. This indicates that student performances in the blended learning context in China (d = 0.77, 95%CI = 0.44 ~ 1.10, z = 4.59, p < .01) and Vietnam (d = 0.66, 95%CI = 0.06 ~ 1.27, z = 2.14, p = 0.032) are significantly higher than the non-blended. However, the diamond retrieved from the data sourcing from the USA crossed the no-effect line, indicating that student performance in the blended learning context in the USA (d = -0.02, 95%CI = -0.27 ~ 0.23, z = 0.19, p = 0.853) is not significantly higher than the non-blended. The overall results indicate that the blended learning can lead to significantly (d = 0.50, 95%CI = 0.27 ~ 0.74, z = 4.24, p < .01) higher student performance than the non-blended since the diamond is located to the right of the no-effect line. In general, we believe that blended learning could positively influence student performance in different countries.

3.4 Could blended learning positively influence student attitude in different countries?

To determine the differences in student attitudes between blended and non-blended learning in different countries, we obtained totally 11 effect sizes from the studies sourcing from the United Arab Emirates, China, Singapore, Vietnam, the UK, and the USA. We adopted a random-effect model to conduct the meta-analysis due to the high degree of percentage of variability of the effects sizes sourcing from different countries caused by heterogeneity (I² = 76.9%, p < .01).

As shown in Fig. 5, students present significantly more positive attitudes in the blended context than in the non-blended in the United Arab Emirates (d = 1.43, 95%CI = 0.98 ~ 1.88, z = 6.17, p < .01), China (d = 0.48, 95%CI = 0.05 ~ 0.91, z = 2.20, p = 0.027), Singapore (d = 0.32, 95%CI = 0.18 ~ 0.47, z = 4.37, p < .01), Vietnam (d = 0.43, 95%CI = 0.01 ~ 0.85, z = 1.98, p = 0.047), the UK (d = 0.63, 95%CI = 0.37 ~ 0.90, z = 4.73, p < .01), and the USA (d = 0.98, 95%CI = 0.78 ~ 1.18, z = 9.41, p < .01) since their diamonds are all located to the right of the no-effect line without crossing it. The overall result also indicates that blended learning could lead to significantly more positive student attitude towards blended learning ( d = 0.59, 95%CI = 0.37 ~ 0.80, z = 5.28, p < .01).

3.5 Could blended learning positively influence learning achievement in different countries?

To identify students’ achievements of blended learning in different countries, we extracted 57 effect sizes, where 2 of them sourced from Canada, 13 from China, 22 from Germany, 3 from Spain, 1 from the United Arab Emirates, 1 from the UK, 1 from Vietnam, 14 from the USA. We adopted a random-effect model to implement the meta-analysis due to the high degree of percentage of variability caused by heterogeneity (I² = 87.4%, p < .01). We entered means, standard deviations, and numbers of participants across both groups into Stata/MP 14.0, then we obtained a forest plot after running the meta-analytical program by the variable country (Fig. 6).

The pooled diamond at the bottom is located to the right of the no-effect line without crossing it. We thus conclude that the students’ overall achievement in the blended learning context is significantly larger than that in the non-blended learning context (d = 0.30, 95%CI = 0.21 ~ 0.40, z = 6.24, p < .01). No diamonds, the pooled results, for different countries cross the no-effect middle line and all of them are located to the right of it. Consequently, students’ blended learning achievements also significantly surpass the non-blended in Canada (d = 0.53, 95%CI = 0.19 ~ 0.86, z = 3.07, p = 0.002), China (d = 0.27, 95%CI = 0.15 ~ 0.40, z = 4.25, p < .01), Germany (d = 0.15, 95%CI = 0.06 ~ 0.25, z = 3.13, p = 0.002), Spain (d = 0.47, 95%CI = 0.09 ~ 0.84, z = 2.44, p = 0.015), the United Arab Emirates (d = 1.99, 95%CI = 1.50 ~ 2.49, z = 7.87, p < .01), the UK (d = 0.47, 95%CI = 0.10 ~ 0.84, z = 2.46, p = 0.014), Vietnam (d = 0.65, 95%CI = 0.05 ~ 1.26, z = 2.11, p = 0.035), and the USA(d = 0.31, 95%CI = 0.14 ~ 0.47, z = 3.62, p < .01). We, therefore, believe that blended learning could positively influence learning achievement in different countries.

3.6 Could blended learning positively influence student engagement in different countries?

To identify whether the blended approach could improve student engagement in learning, we extracted 14 effect sizes, where 3 of them sourced from the USA, and 11 from China. We adopted a random-effect model to conduct the meta-analysis due to a high degree of percentage of variability of effect sizes caused by heterogeneity (I² = 89.5%, p < .01). After entering means, standard deviations, and numbers of participants of both groups into Stata/MP 14.0, we obtain a forest plot (Fig. 7) from the meta-analysis by the variable country.

Figure 7. A forest plot of students’ engagement in different countries

As shown in Fig. 7, the diamonds obtained from the meta-analysis of data sourcing from both China and the USA cross the no-effect middle line. The diamond for the overall result also crosses the no-effect middle line. We thus conclude that there are significant differences in student engagement between blended and non-blended learning in both China (d = 0.14, 95%CI = -0.06 ~ 0.34, z = 1.38, p = 0.169), the USA (d = 0.51, 95%CI = -0.35 ~ 1.38, z = 1.16, p = 0.245), and the overall results (d = 0.23, 95%CI = -0.09 ~ 0.55, z = 1.42, p = 0.156). Therefore, we believe that blended learning could not positively influence student engagement in different countries.

This meta-analysis finds that blended learning could positively influence student performance, student attitude, and learning achievement rather than student engagement in different countries. Most of previous studies found positive influence of blended learning on performance, attitude, and achievement (e.g. Yen & Lee, 2011; Chang, Shu, Liang, Tseng, & Hsu, 2014), while some revealed negative influence of blended learning on the engagement (e.g. Botts, Carter, & Crockett, 2018). Numerous factors account for the findings.

Student performance tends to be enhanced in the blended learning context. With the blended learning method, students learn both in the face-to-face physical classroom and online with information technologies. In the classroom, they can interact with peers and teachers directly for academic issues. They are also under the supervision of the teacher, encouraging them to be involved in learning activities. They are often required to focus on the learning contents because the teacher tends to ask them to answer questions. They have frequent eye contacts with the teacher, which urges them to follow the teaching progress. If they find anything hard to comprehend, they can ask the teacher by raising their hands. After class, they can also keep pace with learning and teaching progress conveniently using online communication technologies. They can log into the Internet and have access to a learning platform to acquire knowledge and share abundant learning resources. They neither need to carry heavy bags and books nor travel a long distance to the classroom. They can learn through the platform whenever and wherever they feel comfortable (Yu & Yi, 2020). This convenient blended mode undoubtedly improves their performance.

Students also tend to hold positive attitudes towards blended learning. The blended learning method brings great convenience to them. They can carry a light portable smart phone everywhere to acquire knowledge and learn with them in their hands or even on a bus. The small device can carry a sea of learning materials. Otherwise, they have to carry tons of books to learn page by page in a physical classroom. They can also interact through the online platform with their peers to address difficult problems and cultivate a favorable learning environment (Yu, Zhu, Yang, & Chen, 2019). Blended learning makes full use of the power of the Internet, combining the online learning with classroom learning. In this way, there evolves an autonomous learning approach, consisting of real-time and non real-time, synchronous and asynchronous learning, opinion sharing, collaborative learning, and group learning based on the concept of cooperation. The formal instruction and informal learning will be seamlessly linked, definitely improving students’ attitudes toward blended learning.

Students tend to achieve great academic success in blended learning. Blended leaning requires the instructors to put the learning materials on the Internet or on a learning platform, and the learners can browse these materials at any time according to their needs. Contact information of experts and instructors is openly accessible on the Internet. If learners encounter problems or want to further explore the knowledge, they can contact relevant experts or instructors at any time. Face-to-face communication can also be used in the physical classroom. Online courses play a very important role in the transmission of learning materials and contents, which facilitates learners or trainees to communicate with each other (Yu, & Wang, 2016). This is conducive to strengthening their in-depth understanding of the learning contents. Through the knowledge constructed by learners themselves, blended learning can creatively form their own thoughts and share with others through the Internet.

Nevertheless, in the blended learning context, students’ engagement may be weakened due to various factors. Some online learning platforms have numerous problems, such as unstable running system environment, slow computer operation speed, easy to jam in online processing, rigid interface design, and lack of humanization. The unreasonable design of menus may cause an abomination of learners. If the platform is not compatible with some software, the system may be unstable or even suspend the learning procedure. If the Internet is too slow, students will not be able to open multiple windows at the same time, which may frustrate their learning enthusiasm, weaken their computer self-efficacy (Sun & Rueda, 2012), and they may finally abandon the blended learning approach.

Various backgrounds of learners may cause different feedback to blended learning. As learners come from all over the world, they, subject to geographical, family, school, and other influencing factors, may have various responses to online learning technologies. It is thus necessary for an educational institute to carry out a training program to improve students’ skills of online technology use (Bernard, Borokhovski, Schmid, Tamim, & Abrami, 2014). They can also train students using a recorded video where the operation instruction can be detailed. An educational department can regularly implement training programs of blended learning to bridge the gap of technology skills among students. In this way, students with different backgrounds can adapt themselves to the blended learning environment.

To improve the engagement in blended learning, it is important to design appropriate curriculum contents (Vaughan, 2007). Teachers or instructors need to integrate the advantages of online learning into offline classroom teaching, design the curriculum based on the needs of learners, provide visual and aural learning stimuli, enhance students’ self-efficacy, trigger their learning interest, enhance their learning engagement, and finally raise their level of knowledge. With strong self-efficacy or satisfaction, students may feel voluntary to receive blended instruction and keep regular engagement in blended learning programs. To improve students’ engagement, MOOCs could be used as a form to blend face-to-face courses (de Moura, de Souza, & Viana, 2021).

5.1 Major findings

This meta-analysis mainly examined the differences in performance, attitude, achievement, and performance between blended and non-blended learning approaches in different countries. It is revealed that blended learning could improve performance, attitude, and achievement in most countries. However, in both China and the USA, blended learning could not significantly improve student engagement in academic activities. No significant differences were revealed in student performance in the USA between blended and non-blended learning.

5.2 Limitations

There are several limitations to this study. Firstly, the meta-analysis cannot include all the publications and non-published works due to the limitation of library sources. Secondly, Both Begg’s and Egger’s tests indicate the presence of publication bias. Thirdly, we could not completely retrieve the identified research due to various reasons.

5.3 Future research directions

Whether in blended or non-blended learning contexts, teachers need to specifically instruct students to collaborate and practice, where students can learn the benefits and challenges of blended or non-blended learning and improve their learning effectiveness (Monteiro & Morrison, 2014). Although the new century has been witnessing the increasing popularity of blended learning especially in this special pandemic time, few studies have been committed to the effectiveness of blended learning in different countries. Future research could extend the research into blended learning to more countries and areas across the world.

Competing interests: The author declares no competing interests.

Al-Qatawneh, S., & Eltahir, M. E., & Alsalhi, N.R. (2020). The effect of blended learning on the achievement of HDE students in the methods of teaching Arabic language course and their attitudes towards its use at Ajman University: A case study. Education and Information Technologies, 25, 2101-2127. https://doi.org/10.1007/s10639-019-10046-w

Arrosagaray, M., Gonzalez-Peiteado, M., Pino-Juste, M., & Rodriguez-Lopez, B. (2019). A comparative study of Spanish adult students' attitudes to ICT in classroom, blended and distance language learning modes. Computers & education, 134, 31-40. http://doi.org/10.1016/j.compedu.2019.01.016

Baepler, P., Walker, J.D., & Driessen, M. (2014). It's not about seat time: blending, flipping, and efficiency in active learning classrooms. Computers & education, 78(9), 227-236. http://doi.org/10.1016/j.compedu.2014.06.006

Bazelais, P., & Doleck, T. (2018). Blended learning and traditional learning: a comparative study of college mechanics courses. Education and Information Technologies, 23, 2889-2900. https://doi.org/10.1007/s10639-018-9748-9

Begg, C.B., & Mazumdar, M. (1994). Operating characteristics of a rank correlation test for publication bias. Biometrics, 50, 1088–1101.

Beichner, R., Saul, J., Abbott, D., Morse, J., Deardorff, D., & Allain, R., et al. (2007). Student-Centered Activities for Large Enrollment Undergraduate Programs (SCALE-UP) project. In E. Redish, & P. Cooney (Eds.), Research-based reform of university physics (pp. 1-42). College Park, MD: American Association of Physics Teachers.

Berga, K. A., Vadnais, E., Nelson, J., Johnston, S., & Olaiya, B. (2021). Blended learning versus face-to-face learning in an undergraduate nursing health assessment course: a quasi-experimental study. Nurse Education Today, 96, 104622. http://doi.org/10.1016/j.nedt.2020.104622

Bernard, R. M., Borokhovski, E., Schmid, R. F., Tamim, R. M., & Abrami, P. C. (2014). A meta-analysis of blended learning and technology use in higher education: from the general to the applied. Journal of Computing in Higher Education, 26(1), 87-122. http://doi.org/10.1007/s12528-013-9077-3

Bersin & Associates. (2003). Blended learning: What works? An industry study of the strategy, implementation, and impact of blended learning. Oakland, CA: Bersin & Associates.

Bonk, C. J., & Graham, C. R. (Eds.) (2006). Handbook of Blended Learning: Global Perspectives, Local Designs (pp. 8-10). San Francisco, CA: Pfeiffer Publishing.

Borenstein, M., Hedges, L., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to Meta-Analysis. John Wiley and Sons, West Sussex. https://doi.org/10.1002/9780470743386

Botts, R.T., Carter, L., & Crockett, C. (2018). Using the Blended Learning Approach in a Quantitative Literacy Course. PRIMUS, 28(3), 236-265, http://doi.org/10.1080/10511970.2017.1371264

Chang, C. C., Shu, K. M., Liang, C., Tseng, J. S., & Hsu, Y. S. (2014). Is blended e-learning as measured by an achievement test and self-assessment better than traditional classroom learning for vocational high school students? International Review of Research in Open & Distance Learning, 15(2), 213-231. http://doi.org/10.19173/irrodl.v15i2.1708

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: L. Erlbaum Associates.

Cortizo, J. L., Rodriguez, E., Vijande, R., Sierra, J. M., & Noriega, A. (2010). Blended learning applied to the study of mechanical couplings in engineering. Computers & education, 54(4), 1006-1019. http://doi.org/10.1016/j.compedu.2009.10.006

Curtis, J. Bonk., Graham, Charles. R., Cross, Jay., & Moore, Michael. G. (2005). The handbook of blended learning: Global perspectives, local designs. Leonard Pfeiffer & Company. NW Suite 800, Washington.

de Moura, V.F., de Souza, C.A., & Viana, A.B.N. (2021). The use of Massive Open Online Courses (MOOCs) in blended learning courses and the functional value perceived by students. Computers & education, 161, 104077. http://doi.org/10.1016/j.compedu.2020.104077

Egger, M., Smith, G.D., Schneider, M., & Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. BMJ, 315, 629–634.

Higgins, J. P. T., Thompson, S. G., Decks, J. J., & Altman, D. G. (2003). Measuring inconsistency in meta-analyses. British Medical Journal, 327(7414), 557-560.

Higgins, J.P.T, & Thompson, S.G. (2002). Quantifying heterogeneity in a meta-analysis. Statistics in Medicine, 21, 1539-1558. http://doi.org/10.1002/sim.1186

Higgins, J.P.T., & Green, S. (2021). Cochrane handbook for systematic reviews of interventions version 5.1.0 [updated March 2011]. The Cochrane Collaboration 2011. 2021 Available from https://handbook-5-1.cochrane.org/

Hill, T., Chidambaram, L., & Summers, J. D. (2017). Playing 'catch up' with blended learning: performance impacts of augmenting classroom instruction with online learning. Behaviour & Information Technology, 36(1), 54-62. http://doi.org/10.1080/0144929X.2016.1189964

Jia, J., Chen, Y., Ding, Z., & Ruan, M. (2012). Effects of a vocabulary acquisition and assessment system on students' performance in a blended learning class for English subject. Computers & education, 58(1), 63-76. http://doi.org/10.1016/j.compedu.2011.08.002

Lee Gordon, D., Issenberg, S. B., Gordon, M. S., Lacombe, D., Mcgaghie, W. C., & Petrusa, E. R. (2005). Stroke training of prehospital providers: an example of simulation-enhanced blended learning and evaluation. Medical Teacher, 27(2), 114-21. http://doi.org/10.1080/01421590400029756

Liu, M.H. (2016). Blending a class video blog to optimize student learning outcomes in higher education. Internet and Higher Education, 30, 44-53. http://dx.doi.org/10.1016/j.iheduc.2016.03.001

López-Pérez, M.V., Pérez-López, M.C., Rodríguez-Ariza, L., & Argente-Linares, E. (2013). The influence of the use of technology on student outcomes in a blended learning context. Educational Technology Research & Development, 61(4), 625-638. http://doi.org/10.1007/s11423-013-9303-8

Macaruso, P., Wilkes, S., & Prescott, J. E. (2020). An investigation of blended learning to support reading instruction in elementary schools. Educational Technology Research and Development. http://doi.org/10.1007/s11423-020-09785-2

McCutcheon, K., O’Halloran, P., & Lohan, M. (2018). Online learning versus blended learning of clinical supervisee skills with pre-registration nursing students: a randomised controlled trial. International Journal of Nursing Studies, 82, 30-39. http://doi.org/10.1016/j.ijnurstu.2018.02.005

Moher, D., Liberati, A., Tetzlaff, J., Altman, D.G., & The PRISMA Group (2009) Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med, 6(7): e1000097. https://doi.org/10.1371/journal.pmed.1000097

Monteiro, E., & Morrison, K. (2014). Challenges for collaborative blended learning in undergraduate students. Educational Research and Evaluation, 20(7-8), 564-591, http://doi.org/10.1080/13803611.2014.997126

Moule, P. Pontin, D. Gilchrist. M. Ingram, R. (2003). Critical appraisal framework. Retrieved Feb. 8, 2021, http://learntech.uwe.ac.uk/da/Default.aspx?pageid=1445

Mueller, C., Mildenberger, T., & Lübcke, M. (2020). Do we always need a difference? Testing equivalence in a blended learning setting. International Journal of Research & Method in Education, 43(3), 283-295, http://doi.org/10.1080/1743727X.2019.1680621

Olitsky, N. H., & Cosgrove, S. B. (2014). The effect of blended courses on student learning: evidence from introductory economics courses. International Review of Economics Education, 15, 17-31. http://doi.org/10.1016/j.iree.2013.10.009

Pérez-Marín, D., & Pascual-Nieto, I. (2011). A case study on the use of blended learning to encourage computer science students to study. Journal of Science Education and Technology, 21(1), 74-82. http://doi.org/10.1007/s10956-011-9283-6

Rossett, A. (2002). The ASTD e-learning handbook. New York: McGraw-Hill.

Sawilowsky, S.S. (2009). New effect size rules of thumb. Journal of modern applied statistical methods, 8(2), 597-599. https://doi.org/10.22237/jmasm/1257035100

Sedgwick, P., & Marston, L. (2013). Meta-analyses: standardised mean differences. BMJ, 347, f7257. https://doi.org/10.1136/bmj.f7257

Shorey, S., Kowitlawakul, Y., Kamala Devi, M., Chen, H. C., Soong, S. K. A. & Ang, E. (2018). Blended learning pedagogy designed for communication module among undergraduate nursing students: a quasi-experimental study. Nurse Education Today, 120-126. https://doi.org/10.1016/j.nedt.2017.11.011

Singh, H., & Reed, C. (2001). A white paper: Achieving success with blended learning. Los Angeles: Centra Software.

Sun, J., & Rueda, R. (2012). Situational interest, computer self-efficacy and self-regulation: Their impact on student engagement in distance education. British Journal of Educational Technology, 43(2), 191-204.

Thai, N.T.T., Wever, B.D., & Valcke, M. (2017). The impact of a flipped classroom design on learning performance in higher education: Looking for the best “blend” of lectures and guiding questions with feedback. Computers & education, 107, 113-126. http://dx.doi.org/10.1016/j.compedu.2017.01.003

Thomson, I. (2020). Thomson job impact study: The next generation of corporate learning.<http://www.delmarlearning.com/resources/job_impact_study_whitepaper.pdf> Retrieved 25.09.2020.

Vaughan, N. (2007). Perspectives on blended learning in higher education. International Journal on E-Learning, 6(1), 81-94.

Ward, J., & LaBranche, G. A. (2003). Blended learning: The convergence of e-learning and meetings. Franchising World, 35(4), 22–23.

Yang, Y. F. (2012). Blended learning for college students with English reading difficulties. Computer Assisted Language Learning, 25(5), 393-410. http://doi.org/10.1080/09588221.2011.597767

Yang, Y. T. C., Chuang, Y. C., Li, L. Y., & Tseng, S. S. (2013). A blended learning environment for individualized English listening and speaking integrating critical thinking. Computers & education, 63(4), 285-305. http://doi.org/10.1016/j.compedu.2012.12.012

Yen, J. C., & Lee, C. Y. (2011). Exploring problem-solving patterns and their impact on learning achievement in a blended learning environment. Computers & education, 56(1), 138-145. http://doi.org/10.1016/j.compedu.2010.08.012

Yick, K.L., Yip, J., Au, S.C., Lai, Y.Y., & Yu, A. (2019). Effectiveness of blended learning in the first year of fashion education. International Journal of Fashion Design, Technology and Education, 12(2), 178-188. http://doi.org/10.1080/17543266.2018.1546910

Young, J. R. (2002). “Hybrid” teaching seeks to end the divide between traditional and online instruction (p. A33). Chronicle of Higher Education.

Yu, Z., & Wang, G. (2016). Academic achievements and satisfaction of the clicker-aided flipped business English writing class. Journal of educational technology & society, 19(2), 298-312.

Yu, Z., & Yi, H. (2020). Acceptance and effectiveness of Rain Classroom in linguistics classes. International Journal of Mobile and Blended Learning, 12(2), 77-90. http://doi.org/10.4018/IJMBL.2020040105.

Yu, Z., Zhu, Y., Yang, Z., & Chen, W. (2019). Student satisfaction, learning outcomes, and cognitive loads with a mobile learning platform. Computer Assisted Language Learning, 32(4), 323-341. http://doi.org/10.1080/09588221.2018.1517093.

Zhou, C. (2018). Empirical study on the effectiveness of teaching model of college English writing within blended learning mode. Educational sciences: theory & practice, 18(5), 1060-1076. http://doi.org/10.12738/estp.2018.5.009

Download PDF

Version 1

posted

You are reading this latest preprint version

A meta-analysis of effects of blended learning on performance, attitude, achievement, and engagement in different countries

Status:

Version 1

Abstract

Figures

1. Introduction

1.1 Definitions of blended learning

1.2 Performance

1.3 Attitude

1.4 Achievement

1.5 Engagement

1.6 Contradictory findings

1.7 Research questions

2. Methods

2.2 Eligibility criteria

2.3 Data sources and search strategy

2.4 Evaluation of included studies

2.5 Data extraction

2.6 Statistical analysis

3. Results

3.1 Tests of publication bias

3.2 A sensitivity analysis

3.3 Could blended learning positively influence student performance in different countries?

3.4 Could blended learning positively influence student attitude in different countries?

3.5 Could blended learning positively influence learning achievement in different countries?

3.6 Could blended learning positively influence student engagement in different countries?

4. Discussion

5. Conclusion

5.1 Major findings

5.2 Limitations

5.3 Future research directions

Declaration

References

Status:

Version 1