GBL in Math Problem Solving: Is it Effective?

The present study investigates the difference effect of problem solving model with or without game-based learning (GBL) in learning mathematics. Ninety-one students of grade eight (M = 13.9; SD = 0,5 year) from Indonesia were randomly selected to participated in the study. Data were analyzed by using qualitative and quantitative methods. Three types of tests were used in this study, one for assessing prior knowladge, post-testing, and the other for paper questionnaire. Posttes was composed of two measuring both numeric (procedural and arithmatical accuracy) and reasoning (applied problems in geometry). Results indicated that students who were exposed to the game-based learning within problem solving (PS+GBL) significantly outperformed their counterparts who were exposed on the basis of textbook within problem solving (PS+TB). The positive effects of PS+GBL were observed on both tests of numeracy and applied problems in geometry. In addition, the findings show the positive effects of PS+GBL method on both lower and higher achievers. Keywords—game-based learning, problem solving


Introduction
With the advancement of technologies recently, education methods have been changed greatly compared to their original ways (e.g., [1]- [3]). This situation has led to shift toward studying in how education and training are delivered in the 21 th century. The shift include the initially paradigm of learning is ready to use become equip ability to find out, digital literacy, and critical-creative thinking for the educators and learners. Finding pedagogical approaches, in a technologically driven culture, that cultivate critical-creative thinking is a challenge for mathematics educators [4]. Additionally, The National Council of Teachers of Mathematics mandates that advocating integration of emerging technology within a focus on mathematics learning goals [5]. This mandate encourages for mathematics educators and learning designers to create instructional media of emerging technology which attracts interest as well as comprehension topic of mathematics.
Many research studies have been conducted in the field of integrating technology into education (see [6], [7]), especially in mathematics education (see [8], [9]). One that has attracted many educational practitioners and researchers around the world is gamebased learning (GBL), which is the integration of educational games into the teaching and learning process. [10] GBL briefly defined as "learning through the game". Furthermore, GBL could be broadly define as the actual game used in the classroom to enhance both learning and teaching [11]. The integration of game content aims to create a fun learning climate without overriding the subject that students must learn. This is the kind of motivation that educators and learning designers wish for in their students because it has a positive effect on learning process.
With respect to teaching game-based learning, several studies ( [12], [13]) recomended teacher use mobile devices wich provide learners increasing independence to structure their personal learning process. Despite several years there is controversial among teachers regarding the use of a smartphone as a mobile learning tool in school. On one hand, some teachers disagreed because it would interfere the learning process in the classroom, while on the other hand, some teachers saw a lot of potential that needs to be used in learning ( [14], [15]). However, todays educators and learning designers realize that the mobile device has provided new opportunities for the availability of new and different forms of learning, including the relationshoip between learners, teachers, and learning objects ( [16], [17]). Considering the rapid development of mobile technology recently, mobile learning could be one of the alternative method as a complement of classroom learning. Both games technologies and mobile technologies are increasingly seen as potential aspect for the development of resources to support learning ( [18], [19]). Furthermore, several study [20] have reported that the use mobile learning as the foundation of game-based learning are proven to encourage student motivation and to support their own learning activities.
Despite the advantage of technologies as a teaching and learning device, one important aspect that need to be consider is the goal of the mathematics education. One of the main goals of mathematics education is applying mathematical concepts in daily life ( [21], [22]). The effectiveness of students in applying mathematics in daily life is not only dependent on performing mathematical operations, but it also related to how deep the students' comprehension of understanding the topics. Another main important thing was that the students were given the materials about contextual problems. Indeed, it has been recommended that mathematics should be learned by problem-solving. Problem-solving identically use the daily life contexts in teaching and learning process [23].
Considering the importance of problem solving in learning mathematics, there are both theoretical and empirical grounds for suggesting that students, learn mathematics problem-solving assisted by technology [24]. It can be hypothesised that game-based learning can be well integrated with problem-solving approach in mathematics class. Previous studies have been focused on the effect of game-based learning that is not affiliated to certain teaching method on students' learning process (e.g., [25], [26]). This study, therefore, try to develop game that is based on problem-solving method and investigate its effectiveness in regards to students' learning outcome. The findings of this study might help researchers and educators gain insight into game-based learning particularly using problem-solving method in mathematics class.

Participants
The participants were 113 eight graders (51 males and 61 females) who studied in four heterogeneous classrooms selected from two Indonesian junior high school. They had recently learned the geometry plane as the prior knowladge to teach solid geometry in this study. Classes were similar in terms of size, student's mean age (M = 13.9; SD = 0.5), and levels of mathematics acievement assessed prior to the beginning of the study.

Treatment
In this study, students studied mathematics under on of two conditions. These two conditions include problem solving that was developed on game-based learning (PS+GBL) and basis of the textbook (PS+TB). The two conditions were identical in terms of the subject matter to be learned that is geometry, mathematical task the students solved, lesson structure, and number of hours mathematics was taught.
Under both conditions each period included problem solving steps: (a) present the problem in a general form; (b) restate the problem in an operational representation; (c) formulate alternative hypoteses and produceres for for solving the problem; (d) the hipotheses and carry out procedures to obtain a solution or sets potential solutions; and (e) analyze and evaluate the solution. Each step is practiced through small heterogeneous group activities. Small heterogeneous were composed of four students: one high achiever, one middle achiever, and two lower achievers. Students were assigned to groups by the teacher according to her ordinary evaluation. The differences between the treatments to offer students opportunity-to-learn with game-based learning and textbook that they regularly us.
The PS+GBL condition, student played a game called "GeoGame Adventure". This game was created by the authors using the application Construct 2. This game has been through validation process, expert judgment and field. According to field test and teachers' judgement, this game is considered practical to be used in the classroom. Finally, this game consists of several levels, where each level followed procedure were developed in accordance with problem-solving method: identify the problem; devising a plan about how to solve the problem; and apply the plan gained in solving problems. Some interface of the game are discussed in the following section.
The PS+TB condition, students followed ateaching program that was developed on the basis of the textbook that they regularly use. Nevertheless, teacher followed lesson structure with problem-solving method. Each student in his or her turn read the problem presented in the book. Each group turn devising a plan, tried to solve it and explain his or her reasoning. When none of the group members knew how to solve the problem, they asked for teacher help.

Testing and Measurements
Three test were used in this study, one for assessing prior knowladge, post-testing, and the other for paper questionnaire. Assesing prior knowladge by giving questions on prerequisite material Plane geometry for many questions are the same as posttest. Posttes was composed of two measuring both numeric (procedural and arithmatical accuracy) and reasoning (applied problems in geometry). For the sake of simplicity, all scores on the numeric and reasoning test have been presented in terms of percent correct answers. Each score analyzed using effect-size according to Cohen's formula.
Numeric ability was indicated by the number of correct surface area and volume calculated. There were 15 questions given and students had a maximum time of 30 minutes to complete them. For each item, students received a score of either 0 (no response or incorrect answer) or 1 (correct answer), and a total score ranging from 0 to 15. Kuder Richardson reliability coefficient was α = 0.86.
Reasoning ability was indicated by the extent to which apply the learned material to problems different from those in the treatment and which were more complex. The problems had unusual figures, required more three steps to obtain the solution. It included 10 questions given and students had a maximum time of 95 minutes to complete them. For each item, students received scores ranged from 0 (no response or incorrect answer) to 7 (full correct answer), and a total score ranging from 0 to 70. The Scoring was carried out by the first author. The interrater reliability of the scoring was checked through by mathematics teacher who was not part of thist study. Kuder Richardson reliability coefficient was α = 0.79.
Paper questionnaire used to gain information about students' preferences for the different learning condition. The questions is (translated from Indonesian): "During learning, do you prefer to have game-based learning provided or textbooks provided in learning mathematics?". This questions only for experimental (PS+GBL) condition, and teacher informed the students that they must choose an optional and write reasons for their choice.

Result and Discussion
This study constrasted the influence of a problem-solving approach in either gamebased learning or textbook oriented. Tabel 1 present the mean scores and standard deviations the measuring by treatment and prior knowladge. Four classes were formed varying in learning approach (PS+TB or PS+GBL) during acquisition of Plane geometry. All students were then tested individually both numerical and reasoning skills were assessed after treatment (pretest).
A two-way Analysis of Variance (ANOVA) was performed on pretest. One factor was the treatment (PS+TB; PS+GBL) and the other prior knowledge. According to Tabel 1 prior to the beginning of the study, there was no significant differences between the two treatments on prior knowledge (F(1,109) = 2.23, p > 0.05; M = 56.88 and M = 56.45; SD = 24.39 and SD = 26.36, for the PS+TB and PS+GBL groups respectively). These results showed that the initial conditions of both groups for prior knowledge are the same. Regarding the posttest, a two-way (treatment × prior knowledge) Manova was conducted on both the numeric and reasoning score. There was significant main effect for the treatment F(2,108) = 4.45, p < .05, and for prior knowledge F(2,108) = 4.77, p < .05. However, there was no significant effect the interaction between the treatment and prior knowledge F(2,108) = 1.27, p > .05. Results indicates that there is a differences in effectiveness between learning with PS+TB and PS+GBL. As can be seen from Tabel 1, the interaction was due to a large difference mean score between PS+TB and PS+GBL under the numeric test (3.70) and reasoning test (7.35).
Both lower an higher achievers benefited from the GBL on numeric (procedural and arithmatical accuracy) and reasoning (applied problems in geometry), it analized with Effect-Size were calculated according to Cohen's formula as the difference between the mean scores of the control (PS+TB) and the experimental (PS+GBL) condition, devided by the standard deviation of the control condition. Numeric score was found for lower achieving studens ( .53) and for higher achieving students (.91). That result showed that effect-sizes of lower and higher achievers were significant on numerical ability. In addition, at the mean of posttest, showed that lower achievers in PS+GBL condition perform significantly better on numerical that higher achievers in PS+TB (Effect-Size = .09). It is possible as pointed out by [27], have reported intervention game-based can be effective in reducing disparities in achievement, particcularly in number skills. Hence, students in lower achiever can more easily do numerical test. Whereas in reasoning score was found for lower achieving studens ( .51) and for higher achieving students ( .57). That result showed that effect-sizes of lower and higher achievers were significant on reasoning ability. The finding regarding the effects of GBL on lower and higher achieving students extends previous findings (e.g., [28], [29]).
The experiment provides further evidence that the use of game-based learning is benefical for students both lower and higher achievers. Furthermore, students who were exposed to the game-based learning within problem-solving (PS+GBL) significantly outperformed their counterparts who were exposed on the basis of textbook within problem solving (PS+TB). This result complement the previous findings (e.g., [25], [30]), which show that game-based learning integrated with problem solving can be use effectively in mathematics class. It becomes interesting to see information about students' preferences between game-based learning (which they have been through) with their usual learning (basis of the textbook). Frequency data for the students' preferences are shown in Tabel 2. According to Tabel 2 for both learning conditions, students indicated a strong preference for game-based learning. The reasons why students preferred game-based learning were exmined, and similar statements were grouped together. Two category stated: first, students reported that game-based learning can visualize geometry material so that helped them understand the material more easily (87.3%), and second, gamebased learning helped them in knowing how to solve the context problems accurately (12.7%). Problem solving approach in this game make connect to life experiences, so encourage students to make connections between their knowledge and application in their lives. Tabel 3 is some description related to the interface of the GeoGame Adventure used in this study.
The game-based learning effect has been shown to be very robust over 18 years of research. Most of the studies have been conducted with weel-defined problems in areas such as mathematics and science [25], [31]- [33]. However, more recent research has found similar effects in more ill-defined problem domains such as architecture [34]; cultural heritage [35]; engineering practice [13]; career guidance topics [36]; sectors of e-health and e-commerce [37]; teaching Structured Query Language [38]; music instruction [39]. In the present study, similar to previous studies, students opportunityto-learn with game-based learning were shown to be superior in learning outcomes.
The hypothesis in this study that game-based learning can be benefical for problemsolving in mathematics class was confirmed. That is, students who were exposed to the game-based learning within problem solving (PS+GBL) significantly outperformed their counterparts who were exposed on the basis of textbook within problem solving (PS+TB). Support for this hypothesis comes from Figure 1 that the mean score of the PS+GBL students on posttest was haigher that that of PS+TB students (M = 71.9 and M = 56,54; SD = 18.56 and SD = 26.05 for PS+GBL and PS+TB students respectively). These findings are cinsistent with several studies [40] that student who use instructional media of emerging technology in class group activities have better rather than students who use textbook for the same activities. Despite in some courses the textbook is central to class discourse, the study ( [41], [42]) have reported that the advancement of the technologies making decreased students' motivation to learn using textbook. Therefore, there is reason may be the cause of success in this study. Table 3. Description of GeoGame Adventure (Just Sample One of The Level)

Figure Information
Start form identify the problem presented game. Character touch check-points for presents a problem to learners that must be solved in a game mission.
Devising a plan about how to solve the problem. If the Materi button is on tap, than material review will present summary concepts needed to solve the problem Apply the plan gained in solving problems. Character touch problem-points for presents various problemsolving questions.
In solving questions, there are two possibilites that will accor. If the student is coreect (BENAR) in the answer, then will get points of problems. Otherwise if they wrong (SALAH), then it will reduce the life of the character. For information, in this game must collect 3 point of problems to get continue next level.
for choosing correct figure some monsters Overall from this study show that application of game based learning with problemsolving approach (i.e. GeoGame Adventure) yielded good learning effects that were better than those generated from general traditional instruction (i.e. Textbook oriented). Nevertheless, we do not reject the use of textbook as a learning tool in learning mathematics or any other learning domain. As reported research [43] has demonstrated that using of textbook still considered necessary in a lesson, such as an aid by the student in task-solving

Conclusions and Future Work
This study adds to the already of evidence that the use of appropriately designed game-based learning is effective as a learning instructional media. In addition, this study shows that game-based learning are practical and workable in modifying the learning processes that take place within heterogeneous classrooms.
The findings of this study appoint several suggestion for further research. First, just one subject were conducted in this study (i.e., geometry). Thus, the other subject under the similar condition is not known. It would be particularly interesting if comprising deifferent subjects lesson and covering an entire academic year. Second, this study investigated the differential effects of the treatments on lower and higher achievers. However, the effect on middle achieving students is not known at study. Thus, it would be interesting to investigate the differential effect on lower, middle, and higher achievers. Finally, these study has successfully tested game-based learning in problemsolving mathematics class. These findings also call for the design of game-based learning on different condition. The extent to which game-based learning used in the present study are appropriate for different condition is not known at present and may be investigated in future research.

Authors
Loviga Denny Pratama has expertise on educational technology and is a scholar at postgraduate of Yogyakarta State University (loviga.denny2016@student.uny.com).
Wahyu Setyaningrum is a lecturer at Faculty of Mathematics Education in Yogyakarta State University and also served as secretary to deputy director for international partnership. She is currently PhD graduate at University of Dundee, UK (wahyu_setyaningrum@uny.ac.id).