The Effectiveness of Dynamic Geometry Software Applications in Learning Mathematics: A Meta-Analysis Study

Indonesia Abstract— Hundreds of studies on Dynamic Geometry Software (DGS) influence on mathematics learning with mixed results have been found in the literature. Correspondingly, this meta-analysis study was conducted to assess the overall impact of DGS and analyze the characteristics of the identified studies to help educators decide under what conditions the use of DGS would achieve a higher level of effectiveness. This meta-analysis study investigated 57 effect sizes drawn from 50 journal articles and international proceedings between 2010 and 2020, using the Comprehensive Meta-Analysis (CMA) program as a calculation tool. Meanwhile, the Hedges coefficient is used to calculate the effect size at the 95% confidence level. Based on a random-effect model with a standard error of 0.09, the analysis found an overall effect size of 1.07. This means that learning using DGS has a relatively high positive effect on math skills. Analysis of the study’s characteristics revealed that the DGS used was more effective by considering sample size, student-computer ratio, and education level. These facts can help educators use DGS in the future. Finally, the study’s implications and limitations are discussed, providing crucial information for further meta-analysis studies on DGS’s Paper —The Effectiveness of Dynamic Geometry Software Applications in Learning Mathematics: ...


Introduction
The advancement in computer technology has led to the application of software development in learning [1]- [3]. This further motivates teachers to take steps to integrate computers into the educational environment [4], [5], in order to improve the effectiveness and quality of the education system [6]- [8]. This development also provides additional training and opportunities for students to explore their problemsolving attributes by seeking alternative solutions due to the unlimited computer use capabilities in the learning [9]- [11].
The use of computers has quickly attracted the interest of teachers and researchers in teaching mathematics [12]- [16] and one of the most widely applied software is Dynamic Geometry Software (DGS). This software allows users to make geometric figures and measure several variables from them to determine their properties, drag numbers through the screen, produce geometric constructions, hypotheses, and test them to make generalizations [17], [18]. Several studies have, however, been conducted to determine the effectiveness of DGS in mathematics classes [19]- [26].
Some studies have been conducted to examine DGS's effectiveness on students' mathematical abilities at various levels of education in Indonesia with several inconsistent results. For example, [10]- [31] showed the use of DGS was more effective in improving students' mathematical abilities than conventional methods of learning while [32]- [35] found that it was not better. Currently, there has not been a comprehensive evaluation of the usefulness of DGS on students' mathematical abilities have not been explored much. Meanwhile, educators and stakeholders need accurate information to determine the appropriate conditions for using DGS to achieve higher effectiveness levels.
It is possible to fill this gap by undertaking a study that combines several quantitative findings to provide useful information for practice or policy [36], [37]. This method has been analyzed to consider its implications [38], and up to 2020, no research was found to have combined the results obtained in other studies to draw an objective conclusion. However, reviewing the literature in several studies allows for variation in results but also provides subjective conclusions [39], [40].
Meta-analysis is an objective method for reviewing the literature because of the use of its effect size as the unit of analysis [41]- [43]. Moreover, the effect sizes from each study would be combined to obtain an overall effect size [43], [44]. This technique eliminates the subjective interpretation of several empirical reports on the same topic [45], [46]. Finally, statistical procedures were used to determine the differences in DGS effectiveness based on study characteristics.
Our previous meta-analysis research evaluated the effectiveness of constructivismbased mathematics learning [44] and mathematics software-based mathematics learning [37]. We found that study characteristics such as study class and sample size influenced the effect sizes of the two models on students' mathematical abilities. However, study characteristics such as the ratio between students and computers used have not been investigated. Even though several meta-analyses including [47]- [54] has been focused on computer effectiveness in general, only one study [55] focused exclusively on the effectiveness of DGS on students' mathematical achievement. How-ever, it has not included the ratio between students and computers used as characteristics of the analyzed studies.
This study intends to complement previous research with the aim of (1) assessing the impact of using DGS on students' overall mathematical abilities and (2) determining differences in effectiveness based on research characteristics. These are necessary to help educators decide the most suitable conditions to use the software in improving students' mathematical abilities. These goals were achieved by analyzing primary studies conducted on DGS effectiveness on students' mathematical abilities using meta-analysis as a research tool with the following questions being the center of focus: • Does the use of DGS in mathematics learning produce a large effect size on mathematical abilities than conventional approaches?

Research design
This study was conducted to statistically evaluate and interpret the findings of primary studies that have been conducted on the effect of DGS in mathematics learning on students' mathematical abilities in Indonesia. This involves using a meta-analysis method with statistical analysis of quantitative data from individual studies [41], [42]. The overall effect size is determined to measure the effect of using DGS on students' mathematical abilities. Effect sizes have many advantages over using only tests of statistical significance. [56]. The steps involved in the meta-analysis include identifying the problem, searching for related literature, coding the studies according to certain criteria, conducting statistical analysis, and interpretations [38], [57]- [60]. In this research, this stage was also conducted.

Study search
Empirical data is traced from an electronic database that includes the Educational Resource Information Center (ERIC), Springer Publishing, SAGE publishing, and google scholar. The keywords used include "dynamic geometry software, Cabri, Ge-oGebra, Geo sketchpad, and mathematical abilities." Furthermore, Google Scholar and Portal Garuda were used to assess national journal articles using "dynamic geometry software, Cabri, GeoGebra, Geo sketchpad, and kemampuan matematis" as keywords. The results showed 129 studies had examined the effectiveness of DGS in learning mathematics in Indonesia between 2010 and 2020.

2.3
Coding process and inclusion criteria The information derived from each study was coded using a coding sheet, an instrument used in meta-analysis to collect statistical information needed to transform effect sizes and obtain study characteristics from each identified primary study. The coding form was developed to maintain the analyzed studies' reliability, and this required filling of the coding form separately by two coders who were doctoral students with prior special training in meta-analysis. This coding considers the sample's eligibility requirements under study, i.e., the literature studied is selected from experimental studies on the impact of DGS in Indonesia over the last decade (2010-2020). Moreover, samples with no means, standard deviation, and sample size statistics were excluded. This stage provides a total of 50 research articles that meet the eligibility for analysis, and due to the use of more than one experimental or control group in several studies, 57 effect sizes are analyzed. The formula for Cohen's Cappa, denoted by Cohen κ (7), is a powerful statistic to test inter-coder reliability [61]. The formula is: Where Pr (a) represents an actually observed agreement, and Pr (e) represents a coincidence agreement. A deal level of 0.85 or greater is pre-determined to be considered high. An agreement rate of 0.92 was obtained for this study. Thus, this metaanalysis is reliable. The studies included in the analysis process are presented in Appendix 1.

Statistical analysis
The statistical analysis steps in this study are based on the explanation of Borenstein et al. (2009), namely: a) Calculating the effect size of each primary study b) Conducting heterogeneity tests and selecting the estimation model c) Examining publication bias d) Calculate the p-value to test the research hypothesis Further analysis, namely answering the second to sixth questions, has been carried out because the effect size estimation uses a random-effect model [61]. Software that helps data analysis is the Comprehensive Meta-Analysis (CMA) series 3. Cohen's equation is a good estimate for the population but is biased towards studies that contain small samples [62]. Therefore, the Hedges equation g is applied to avoid this bias while the effect size is interpreted using the classification developed by [63], which is presented in Table 1 below: The CMA program provides an effect size for each study and combined effect size for each group of study characteristics as well as homogeneity between groups known as the Qb value (Qbetwen), resulting in statistically heterogeneous effect sizes Qb> χ2.95; p <0.05, the effect size homogeneity hypothesis is rejected [64]. This means that the study characteristic groups' effect sizes do not measure the same population parameters [38] or that there is a statistically significant difference in the average effect size for each study characteristics group [47].

Research results regarding the first question
The first objective of this analysis was to assess the effect of the use of DGS on students' overall mathematical abilities. Therefore, on the basis of CMA-assisted calculations, the effect size and confidence interval limits were obtained and provided in Table 2.  Table 3 shows the overall effect sizes ranged between 0.10 and 5.50, with a 95% confidence limit, while Figure 1 shows the level of the effect size of the entire study based on [63] classification.  Figure 1 shows the different effect sizes obtained from the studies conducted on using DGS in mathematics learning, while Table 3 illustrates the results of the descriptive meta-analysis according to the estimation method. As shown in Table 3, the mean effect size was calculated to be 0.92, and this was classified as a high level according to [63]. The estimation method is determined through the effect size homogeneity test, which shows the Q value to be 394.93, and it is found to be more than 74.56 (df = 56; p = 0.05) in table tabel χ2. This means that the effect sizes between studies differ. Therefore, the estimation model to determine the impact of using DGS on students' mathematical abilities as a whole could be evaluated using the random-effects model.
The random-effects model in Table 3 showed the lower limit was 0.89 while the upper limit was 1.26 while the average was 1.07 at a 95% confidence interval. This was classified as a very high level, according to [63]. Moreover, the significance test results gave z value as 11.34 and p = 0.00, indicating that the use of DGS in mathematics learning resulted in a larger effect size than conventional approaches.
The challenge in meta-analysis research is to avoid publication bias, namely the fact that statistically significant articles have a higher chance of being published and that researchers also rarely (6%) try to publish insignificant research [65]. This tendency leads to an over-representation of significant studies, similar to the loss of studies that actually exist [38], [66], [67]. Therefore, a study funnel plot was included in determining the existence of publication bias in this study, and the result is presented in Figure 2.  Figure 2 shows that the scattered effect's size is not entirely symmetrical in the middle of the funnel plot. Therefore, Trim and Fill tests were carried out to evaluate the extent of the effects associated with publication bias in the effect sizes obtained from the meta-analysis conducted according to the random effects of the model, and the results are presented in Table 4.  Table 4 shows there was no difference between the size of the observed and virtual effects created according to the random effect model conducted to correct the impacts of publication bias. Thus, there is no publication bias in this study

Research results regarding the second question
Descriptive statistics about the second question are illustrated in Table 5.  Table 5 shows the effect sizes between study groups based on the study year varied. The statistical value of Q obtained from the homogeneity test was 8.10, and since this is greater than 7.81 (df = 3; p = 0.05) in table χ2, the effect size distribution has a heterogeneous structure. This, therefore, means the effect size of mathematical ability using DGS in mathematics learning between study groups differs based on the year of the study.

Research results regarding the third question
Descriptive statistics about the third question are presented in Table 6 Table 6 shows the effect sizes between study groups based on education levels also varied. The Q statistical value obtained from the homogeneity test was 7.07, and since it is greater than 5.99 (df = 2; p = 0.05) in table χ2, the effect size distribution has a heterogeneous structure. This, therefore, means the effect size of mathematical ability using DGS in mathematics learning between study groups differs based on the level of research education.

Research results regarding the fourth question
Descriptive statistics about the fourth question are presented in Table 7 Table 7 shows there was a variation in the effect sizes between study groups based on sample size. This was associated with the Q statistical value obtained from the homogeneity test to be 4.81, and since this was greater than 3.81 with 1 degree of freedom and p = 0.05 in table χ2, the effect size distribution has a heterogeneous structure. This means the effect size of the mathematical ability recorded using DGS in mathematics learning between study groups differs based on the size of the research sample.

Research results regarding the fifth question
Descriptive statistics about the fifth question are presented in Table 8.  Table 8 indicates there was a variation in the effect sizes between study groups based on DGS type. This was associated with the 6.19 obtained as the statistical value of Q from the homogeneity test, and due to the fact that this is smaller than 7.81 (df = 3; p = 0.05) in table χ2, the effect size distribution has a homogeneous structure. Therefore, there was no difference in the effect size of mathematical ability using DGS in mathematics learning between study groups based on the type used.

Research results regarding the sixth question
Descriptive statistics about the sixth question are presented in Table 9.  Table 9 shows a variation in the effect sizes between study groups based on student and computer ratios used in learning. The statistical value of Q obtained from the homogeneity test was 12.83, and since this is greater than 3.84 (df = 1; p = 0.05) in table χ2, the effect size distribution has a heterogeneous structure. This, therefore, means there is a difference between the effect size of mathematical ability using DGS in mathematics learning for study groups differs based on the ratio of students and computers used in the process.

Discussion
This study analyzed 57 effect sizes, and according to the random-effects model, the combined effect size was found to be 1.07. This shows that the use of DGS in mathematics learning has a high positive impact compared to conventional methods. It shows the average student treated using DGS exceeded the 84% mathematical ability of those in conventional classes, which were initially equivalent. Based on the inter-pretation table from [68], the average students ranked 16th in the experimental group were equivalent to those ranked 6th in the control group.
This finding is in line with previous meta-analysis studies conducted by [52] to compare the effectiveness of DGS-based through the analysis of 587 primary studies and found a combined effect size of 1.02. Even though the number of studies included in this present research was ten times smaller than the sample size, very similar results were obtained, reflecting the overall trend. Another previous meta-analysis study compared the effectiveness of math software used on students' math abilities by analysing 51 primary studies, and an effect size of 1.102 was found [37]. Some other related studies showed the use of computers in learning influences students' mathematical abilities [47]- [54], [64]. Therefore, the results of this study and other related research shows using DGS in mathematics learning can improve and a very high effect on students' mathematical abilities.
The results also indicate that more study characteristics are related to effect size. The strongest relationship was found for the following variables: year of study, level of education, sample size, and student to computer ratio. The effect size does not differ based on the type of DGS used in learning.
Significant differences were observed between the study groups based on the research year as observed in the effect size value of 0.82 considered as high level found for the latest year study group, which is smaller than for the oldest year, which was 1.07 and considered very high level. This is very surprising and contradicts previous predictions that the effect size of using DGS on the mathematical abilities of latest year students is greater due to the continuous update of the software and improvement in teachers' quality. However, it is supported by previous studies that the effect size of studies in older study groups was greater than those in newer study groups [47], [49].
The higher values obtained for, the older study groups are associated with the Hawthorne effect, which occurs when students are stimulated to make greater efforts simply due to the novelty of the treatment [47]. As of 2010, the use of DGS in mathematics learning was new in Indonesia.
The results showed a strong relationship between the DGS effectiveness and education level. The effect size of the software's application in tertiary institutions and SHS was found to be greater than on JHS. This was in line with the findings of [55], which showed significant differences in effect size for different school levels but not the same with the results of [37] and [49] that there was no significant difference.
The analysis also showed a strong relationship between DGS effectiveness and sample size, as observed in the effect size variations between small (0-30), which was 1.09, and large sample sizes (31 and over) with 0.89. This means a small sample size should be considered more in teaching settings. This is in line with the findings of [37] and [53] that the effect size of the study group in small samples was greater than those in large samples. However, it is different from the results of [49], [52], [55] that study groups in small samples had smaller effect sizes. This variation, therefore, needs to be further investigated.
This meta-analysis also showed there was no significant difference in effect size based on the type of DGS used. This means every type of DGS is effective in mathematics learning, with the largest combined effect size of 0.98 was GeoGebra from 31 studies. However, this characteristic of this study has not been extensively investigated in the previous meta-analysis.
A significant relationship was also found between the DGS effectiveness and students' ratio to the computers used. This was associated with the extraordinary difference in effect sizes for individuals (one student uses one computer), which was 1.10, and the 0.83 recorded for group implementation (one computer used by more than one student). This, therefore, means individual versions should be considered more in teaching settings. These results are in line with [47] that learning using computers individually has a larger effect size than a collective use.

Conclusion
This study was conducted by integrating the findings of the effects of using DGS on students' mathematical abilities, both as a whole and in several key study characteristics. The results of the analysis reveal that the use of DGS has a high positive impact on students' math abilities. The assessment of the DGS effectiveness based on study characteristics showed it is more effective under certain conditions. First, it was found to be very effective in sample conditions less than or equal to 30. Second, it provides classrooms with a sufficient number of computers, allowing students to use them individually, which is recommended for a higher level of effectiveness. Third, DGS's use was recorded to be more effective in high schools and colleges than in junior high schools. Some differences in effect sizes were observed in terms of the year the studies were conducted. Most recent study groups were found to have smaller values compared to older studies. This shows the consideration of the Hawthorne effect in the mathematics teaching process. Meanwhile, different types of DGS can be used without any exception.
Even though the use of DGS was found to have a very high effect on students' mathematical abilities, the results were only based on studies with certain criteria, with some similar studies not analysed due to inadequate required statistical information. For this study's purpose, only five research characteristics were examined and they include the year the study was conducted, level of education, sample size, type of DGS, and the ratio of students to computers. Meanwhile, some others, such as the study's location, treatment duration, teacher's role as a tutor or instructor, and computer's role as an addition or a substitute for the teacher. Consequently, these conclusions do not reflect the overall effectiveness of using DGS in mathematics learning. Therefore, for further research, a detailed investigation is required to determine DGS's effectiveness using some of the characteristics that have not been investigated. The influence of the problem based learning model assisted by geogebra software on the ability to solve mathematical problems