3.1 Results
In this meta-analysis study, 16 articles that met the inclusion criteria were selected and used. All data from each article was extracted and based on calculations using CMA v3 software, of the 16 articles, there was 1 article with a “weak” effect (0–0,20), 1 article with a “small” effect (0,21–0,50), 8 articles with a “medium” effect (0,51–1,00), and 6 articles with a “strong” effect (more than 1,00). Visually, the results of the effect size analysis can be seen in the Forest Plot below.
Based on the Forest Plot in Fig. 2 above, it is known that the effect sizes of the 16 studies or research articles are not same. Through the calculation results obtained, the effect size of each study is in four categories, namely “weak” by 6,25% (\(\:n=1\)), “small” by 6,25% (\(\:n=1\)). “medium” was 50% (\(\:n=8\)), and “strong” was 37.5% 37,5% (\(\:n=6\)).
Based on Fig. 2, also it shows that the effect size of each study is in different intervals. One study that has an effect size categorized as “strong” (Prihatiningtyas & Buyung, 2023) is at 95% [3,208; 5,383]. In addition, based on these confidence intervals, we can also detect that one of the studies analyzed has an effect size that is inconsistent with other studies, namely the study conducted by Simamora et al. (2022) is at 95% [-0,445; 0,643], where statistically, the study is indicated to have an insignificant effect size because it contains a point 0 in its confidence interval (Retnawati et al., 2018). In the Forest Plot above, it can also be seen that in the random effects model, the lower limit of the 95% confidence interval is 0,719 and the upper limit is 1,607 with the overall effect size of this meta-analysis study is 1,163 or can be expressed as: \(\:(\text{1,163};\:CI\:95\%\:\left[\text{0,719};\:\text{1,607}\right];\:p\:<\:\text{0,001})\). When associated with the classification of effect size according to Cohen et al. (2007), it can be stated that the effect size is in the “strong” category, which means that ethnomathematics-based learning has a positive effect on students' mathematical literacy skills.
The Prediction Interval value is [-0.733; 3,060] reflects the range over which we can expect effect values from new individual studies in the future. This interval is wider than the Confidence Interval because it takes into account variations between different studies. Thus, we can predict that the results from individual studies in the future may range from − 0.733 (negative effect) to 3.060 (positive effect), indicating the possibility of significant variation in the results of future studies.
Furthermore, the result of the \(\:Z\) test calculation to determine statistical significance, based on Fig. 2, the \(\:z\) score was found to be5,14. This result can be said to be statistically significant at the \(\:p<\text{0,000}\) level. Then, based on Fig. 2, the \(\:Q\) value obtained is 93,57 and the value of \(\:p=\text{0,000}<\text{0,05}\). Thus, the distribution of effect sizes was found to be heterogeneous at \(\:p<\text{0,05}\), i.e. the actual effect sizes varied from study to study. Similarly, the degree of effect size variation between studies was reflected by the \(\:{I}^{2}\) value of 84%, which means that 84% of the observed effect size variance reflects the percentage of variability caused by true heterogeneity (not due to sampling error). Thus, this study had “high heterogeneity” as the \(\:{I}^{2}\) value of 84% exceeded 75% (Higgins et al., 2003; Card, 2012). Since the homogeneity test result was rejected, the estimation model is a random effects model. Since the \(\:p\)-value is \(\:<\text{0,05}\), it can be said that overall the application of ethnomathematics-based learning has a significant effect on students' mathematical literacy when compared to the application of conventional learning.
Furthermore, of the 16 studies observed in this meta-analysis study, it was confirmed that in the heterogeneity test, there were significant differences in the variance of the effect sizes (\(\:Q=\text{93,57};\:p\:<\:\text{0,000}\)). As stated by (Borenstein et al., 2009), if the \(\:Q\) value is significant (\(\:Q>{\chi\:}^{2},\:p<\text{0,05}\)), then part of the variance of the effect size is explained by the continuous variable, in other words, the variable is related to the effect size and the model is significant. Based on this statement, there is potential to conduct further analysis of moderator variables to state the source of variance in each effect size and also Meta Regression test which aims to test whether the independent variable but continuous (predictor) is significant to changes in the effect size or whether this variable has a relationship with the effect size. It should also be noted that this step was carried out at the same time to answer the second research question, where the researcher needs to analyze the level of research characteristics by testing the effect of moderator variables on the 16 effect sizes of 16 primary studies. The following meta-analysis study results for each study characteristic are presented in Table 1 to Table 5 separately along with their analysis below.
Table 1
Meta-analysis Results on Education Level Characteristics
Moderator Variable | Kelompok | \(\:\varvec{n}\) | Combined Effect Size (Hedges g) | Heterogeneity |
Between Classes Effect (\(\:\varvec{Q}\)) | Df | \(\:\varvec{p}\) |
Education Level | Elementary School | 10 | 1,084 | 3,743 | 3 | 0,291 |
Junior High School | 4 | 1,610 |
Senior High School | 1 | 0,671 |
University | 1 | 0,718 |
Overall | 16 | 0,902 |
Based on Table 1 above, it can be seen that for the moderator variable of education level, the highest study effect size is 1,610 (“strong” category) in the study conducted at Junior High School School and the lowest study effect size is 0,671 (“medium” category) in the study conducted at Senior High School. While the largest effect size was conducted in Elementary School, obtaining an effect size of 1,084, the same as Junior High School, the “strong” category. The heterogeneity test results showed that the average effect sizes between education levels were different (\(\:Q=\text{3,743}\) and \(\:p>\text{0,05}\)). Because the \(\:p\)-value \(\:>\text{0,05}\), the distribution of the effect size for the categories on the characteristics of the education level is homogeneous. Thus, it can be concluded that the application of ethnomathematics-based learning to improve students’ mathematical literacy is not influenced by education level.
Furthermore, on the moderator variable of subject matter, the material taught to students when providing learning interventions/experiments is presented in the following table.
Table 2
Meta-analysis Results on Subject Matter Characteristics
Moderator Variable | Group | \(\:\varvec{n}\) | Combined Effect Size (Hedges g) | Heterogeneity |
Between Classes Effect (\(\:\varvec{Q}\)) | Df | \(\:\varvec{p}\) |
Subject Matter | #N/A | 4 | 1,065 | 67,754 | 6 | 0,000 |
2D & 3D Geometry | 1 | 2,461 |
2D Geometry | 7 | 0,654 |
3D Geometry | 1 | 1,105 |
Numbers & Geometry | 1 | 0,718 |
Probability & Sets | 1 | 4,296 |
Transformation Geometry | 1 | 1,814 |
Overall | 16 | 0,993 |
Based on Table 2 above, for the moderator variable subject matter, information was obtained that the lowest study effect size was 0,654 (“medium” category) on 2D Geometry material, followed by Numbers & Geometry material with an effect size of 0,718 (“medium” category). Meanwhile, the other materials are all categorized as “strong” with effect sizes above 1,0, including #N/A where #N/A means it is not explained what material is in the observed study. The results of the heterogeneity test showed that the average effect sizes between subject matter were different (\(\:Q=\text{67,754}\) and \(\:p<\text{0,05}\)). Because the \(\:p\)-value \(\:<\text{0,05}\), the distribution of effect sizes for the categories on the characteristics of the subject matter taught is heterogeneous. Thus, it can be concluded that the application of ethnomathematics-based learning to improve students’ mathematical literacy is influenced by the subject matter.
Furthermore, on the demographic variable of the region where the learning intervention was carried out, it was generally carried out on 3 different islands located in Indonesia, namely Java Island, Kalimantan Island, and Sumatra Island. Actually, in detail, the provinces are also different, of course, for example on the Java Island, it is carried out in West Java Province, there is also in Central Java Province, and also in East Java Province, and so on in other islands. In this case, the researcher unified into 3 different islands as presented in the following table.
Table 3
Meta-analysis Results on Demographic Characteristics of the Intervention Area
Moderator Variable | Group | \(\:\varvec{n}\) | Combined Effect Size (Hedges g) | Heterogeneity |
Between Classes Effect (\(\:\varvec{Q}\)) | Df | \(\:\varvec{p}\) |
Demographic | Java Island | 10 | 1,038 | 33,867 | 2 | 0,000 |
Kalimantan Island | 1 | 4,296 |
Sumatra Island | 5 | 0,836 |
Overall | 16 | 1,190 |
For the moderator variable of students’ demographics, information was obtained that the highest effect size was 4,296 (“strong” category) on Kalimantan Island, followed by Java Island with an effect size of 1,038 (“strong” category), and finally Sumatra Island with an effect size of 0,836 (“medium” category). The results of the heterogeneity test showed that the average effect sizes between student area demographics were different (\(\:Q=\text{33,867}\) and \(\:p<\text{0,05}\)). Because the \(\:p\)-value \(\:<\text{0,05}\), the distribution of the category effect size on the demographic characteristics of the area where the learning was conducted is heterogeneous. Thus, it can be concluded that the application of ethnomathematics-based learning to improve students’ mathematical literacy is influenced by students’ regional demographics.
Furthermore, on the moderator variable of the independent variable, from 16 observed studies, there were 9 models, methods, or learning resources of the independent variable that accompanied ethnomathematics-based learning. We present this in the table below.
Table 4
Meta-analysis Results on Independent Variable Characteristics
Moderator Variable | Group | \(\:\varvec{n}\) | Combined Effect Size (Hedges g) | Heterogeneity |
Between Classes Effect (\(\:\varvec{Q}\)) | Df | \(\:\varvec{p}\) |
Independent Variable | Contextual Teaching Learning (CTL) | 1 | 0,986 | 24,412 | 8 | 0,002 |
Ethnomathematics Based Learning | 2 | 0,886 |
Ethnomathematics Pop Up Book | 1 | 2,461 |
Ethno-Module with RME | 1 | 1,105 |
Flipped Classroom | 1 | 0,718 |
Probing Prompting | 1 | 0,769 |
Problem Based Learning | 5 | 1,128 |
Realistic Mathematics Education (RME) | 3 | 1,154 |
Visual Thinking | 1 | 1,814 |
Overall | 16 | 1,066 |
For the moderator variable of the independent variable, it was found that the study with the independent variable of learning media Ethnomathematics Pop Up Book had the largest effect size of 2,461 (“strong” category), and followed by other learning models. Meanwhile, the Flipped Classroom learning model has the lowest effect size, which is 0,718 (“medium” category). None of them obtained “small” or “weak” effect sizes. This means that these learning models are suitable for use in ethnomathematics-based learning. The results of the heterogeneity test showed that the average effect sizes between independent variables were different (\(\:Q=\text{24,412}\) and \(\:p<\text{0,05}\)). Because the \(\:p\)-value \(\:<\text{0,05}\), the distribution of effect sizes for categories on the characteristics of independent variables is heterogeneous. Thus, it can be concluded that the application of ethnomathematics-based learning to improve students’ mathematical literacy skills is influenced by the independent variables.
Next, on the moderator variable of publication type, the effect size was also calculated. The results of these calculations are presented in the table below
Table 5
Meta Analysis Results on Publication Type Characteristics
Moderator Variable | Group | \(\:\varvec{n}\) | Combined Effect Size (Hedges g) | Heterogeneity |
Between Classes Effect (\(\:\varvec{Q}\)) | Df | \(\:\varvec{p}\) |
Publication Type | Undergraduate Thesis | 2 | 0,677 | 4,381 | 2 | 0,112 |
Proceeding | 1 | 0,797 |
Article | 13 | 1,250 |
Overall | 16 | 0,882 |
For the moderator variable of publication type, it was found that the magnitude of the effect on research published in Undergraduate Thesis was 0,677 (“medium” category), in Proceedings was 0,797 (“medium” category), and in Article was 1,250 (“strong” category). The heterogeneity test results show that the average effect size between publication types is different (\(\:Q=\text{4,381}\) and \(\:p>\text{0,05}\)). Because the \(\:p\)-value \(\:>\text{0,05}\), the distribution of the effect size for the categories on the characteristics of the type of publication is homogeneous. Thus, it can be concluded that the application of ethnomathematics-based learning to improve students’ mathematical literacy skills is not influenced by the type or source of publication.
For the moderator variable of meeting duration, the heterogeneity test results show that the average effect sizes between meeting durations are different (\(\:Q=\text{1,97}\) and \(\:p>\text{0,05}\)). Because the \(\:p\)-value \(\:>\text{0,05}\), the distribution of the effect sizes for the categories on the characteristics of the learning meeting duration is homogeneous. So, from this it can be seen that there is no minimum limit to the number of meetings for ethnomathematics-based learning. Thus, it can be concluded that the application of ethnomathematics-based learning to improve students’ mathematical literacy skills is not influenced by the duration of the learning intervention meeting.
Furthermore, the moderator variable sample size is also taken into account. That is, we can assume whether the condition of too many students also affects the effectiveness of ethnomathematics-based learning on their mathematical literacy? Or is it the small number of students that is ideal for ethnomathematics-based learning? For the moderator variable of sample size, the heterogeneity test results show that the average effect sizes between sample sizes are different (\(\:Q=\text{3,62}\) and \(\:p>\text{0,05}\)). Because the \(\:p\)-value \(\:>\text{0,05}\), the distribution of effect sizes for the categories on the sample size characteristics is homogeneous. Thus, it can be concluded that the application of ethnomathematics-based learning to improve students’ mathematical literacy is not influenced by the sample size in the class.
The next stage, which is the last stage, is for researchers to examine publication bias. The following is a Funnel Plot of this meta-analysis research in Fig. 3.
Based on the Funnel Plot above, it can be understood that the distribution of effect sizes is quite symmetrical around the vertical line. This means that it can be assumed that this meta-analysis study does not have publication bias. However, it needs to be further tested with other tests such as the Trim and Fill method (Duval & Tweedie, 2000), Egger’s Intercept Test (Egger. et al., 1997), and Rosenthal’s Fail-Safe N (Rosenthal, 1979).
Through the results of the CMA v3 calculation, in the random effects model, the observed and adjusted effect sizes are the same, namely 0,92233 (0,78894; 1,05572). Since the two effect sizes are relatively close (or even the same), it can be assumed that there is no publication bias.
Another test also conducted by researchers is Egger's Intercept Test. If the intercept value obtained from this test is not significantly different from 0 (\(\:p>\text{0,05}\)), it can be concluded that there is no publication bias. Conversely, if it is different, (\(\:p<\text{0,05}\)), it is concluded that publication bias occurs. Based on the calculation results using CMA v3, it is obtained that the intercept value is 9,86712 and the \(\:p\)-value (2-tailed) is 0,00002. Because the significance level is less than 0,05, it can be said that there is publication bias in this meta-analysis study.
Furthermore, in the Rosenthal Fail-Safe N test, based on calculations using CMA v3, an N value of 920 was obtained. The result of the calculation was expressed as \(\:5\times\:16+10=90\). Since the value of 920 is higher than 90, it can be concluded that there is no publication bias in this meta-analysis study. Thus, no studies were missing or needed to be added to the analysis due to publication bias.
So, if we compare all the test results, where some results show that there is publication bias and some others do not show this, it can be concluded that the publication bias in this meta-analysis study is not strong enough against publication bias.
3.2 Discussion
Research on the effectiveness of ethnomathematics-based learning on students’ mathematical literacy has been examined through many research studies, and of course is still ongoing today in other parts of the world. In this meta-analysis study, based on effect size inference using a sample of 16 primary studies on the influence/impact/effectiveness of ethnomathematics-based mathematics learning, the overall effect size was 1,163 or can be expressed as: \(\:(\text{1,163};\:CI\:95\%\:\left[\text{0,719};\:\text{1,607}\right];\:p\:<\:\text{0,001})\). These results indicate that ethnomathematics-based learning has a strong positive effect on students’ mathematical literacy when compared to conventional learning.
However, further investigation was also conducted based on several moderator variables that could potentially or possibly influence the effect of the intervention. These variables include: education level, subject matter, student area demographics, duration of learning meetings, independent variables, sample size, and type of publication. Based on the results of the analysis of the seven moderator variables, when referring to their respective \(\:p\)-values, it can be concluded that the application of ethnomathematics-based learning to improve students’ mathematical literacy skills is influenced by the demographics of the student's region, the independent variable, and the subject matter provided. However, the application of ethnomathematics-based learning to improve students’ mathematical literacy skills is not influenced by the duration of the meeting, education level, sample size, and type of publication.
If we compared the findings of this meta-analysis study with previous studies, several previous meta-analysis studies also showed more or less the same thing, although the dependent variable was different. For example, the results of a meta-analysis study from Sulistyowati & Mawardi (2023) showed that the effectiveness of the application of ethnomathematics-based learning on the mathematical abilities of elementary school students differed significantly based on the year of publication, but there were no significant differences in grade level and sample size. Similarly, the results of a meta-analysis study from Apriatni et al. (2022) showed that ethnomathematics-based learning on students’ problem solving ability was influenced by the level of education and the independent variable of the learning model, but not by the year of publication and sample size. Furthermore, a study conducted by Susanto et al. (2023) showed that the dependent variable and education level significantly influenced the effectiveness of ethnomathematics learning on students’ critical thinking skills.
In summary, of the three previous meta-analysis studies on the topic of ethnomathematics, the findings of the researchers’ meta-analysis studies are quite in line with the existing results although some contradictions also exist. For example, the application of ethnomathematics-based learning is influenced by the independent variable (Apriatni et al., 2022), but no study results were found regarding the demographic variables of the student area and the subject matter provided. This means that the findings of these two moderator variables are new findings that can be useful for further research. Furthermore, the application of ethnomathematics-based learning is not influenced by education level (Sulistyowati & Mawardi, 2023) and sample size (Sulistyowati & Mawardi, 2023; Apriatni et al., 2022), but no study results were found regarding the variables of meeting duration and publication type. This means that the findings of these two moderator variables are also new findings that can be useful for further research. The results of the meta-analysis study conducted by researchers contradict the previous meta-analysis study, namely in the meta-analysis study of Apriatni et al. (2022) and Susanto et al. (2023) which states that ethnomathematics-based learning is influenced by the level of education,
Ethnomathematics allows students to construct mathematical concepts as part of their mathematical literacy based on their knowledge of the socio-cultural field. Furthermore, ethnomathematics is able to create good motivation and provide pleasure in the field of education, so that students are expected to pay great attention to explore mathematics education so that it can affect their mathematical abilities (Surat, 2018). This is of course in accordance with the views of Lev Vygotsky, which in his theory states that socio-cultural activities cannot be separated in the development of students. From Vygotsky's theory, researchers agree that social activities play a crucial role in knowledge construction. This means that through ethnomathematics learning, students can build their own knowledge or collaborate with others, and that knowledge is constructed in a cultural context which is something that is already inherent in students.
Overall, the results of this meta-analysis show that ethnomathematics-based learning has a significant positive effect on students’ mathematical literacy. Thus, the results of this meta-analysis study support the use of ethnomathematics-based learning in mathematics education. Ethnomathematics not only makes mathematics more relevant and interesting for students, but it is also effective in improving their mathematical literacy. Therefore, mathematics education should continue to look for ways to integrate culture or local knowledge into the curriculum, in an effort to improve mathematical literacy, NCTM's (2000) mathematics learning process standards, students’ cognitive, affective and psychomotor abilities, or students’ learning outcomes in general.
However, this meta-analysis study also has limitations, including in the literature search, the primary studies obtained by researchers only come from Indonesia. It is not yet known whether other countries also have the same topic regarding the influence/impact/effectiveness of ethnomathematics-based learning on students' mathematical literacy, but as far as the search is concerned, researchers only found Indonesian studies. Thus, if there are primary studies from other parts of the world, it is hoped that the results of this study can be generalized more broadly. Furthermore, the moderator variables observed in this meta-analysis study were limited to only 7 variables. Researchers still feel insufficient, and it is hoped that further research can complement it with other moderator variables so that it can support the research findings, such as the year of publication, experimental design, data analysis techniques, dependent variables, and so on. In addition, because the meta-analysis study conducted by this researcher is the latest study related to students' mathematical literacy, therefore the researcher also recommends that a meta-analysis study be conducted with other dependent variables, namely the 5 standards of the mathematics learning process recommended by NCTM (2000), including: problem solving, reasoning, connection, communication, and mathematical representation.