Gamification and Education: Achievements, Cognitive Loads, and Views of Students

—In this mixed method research study, the effect of gamification strategies on students’ cognitive load levels and achievements was examined along with student opinions about gamification. The topic of spreadsheets was covered for six weeks in a sixth grade information technologies and software course. The sample consisted of a control group of 48 students who were trained via traditional procedures and an experimental group of 46 students who were trained using gamification strategies. Independent sample t-test, Mann-Whitney U test and descriptive analysis were applied to the data. A significant difference was found between the two groups that indicated higher achievement in the experimental group. When comparing cognitive load levels, the experimental group also scored higher than the control group. Interviews indicated that the students had positive views about gamification strategies.


INTRODUCTION
Gamification is a new trend where game elements are used in unlikely contexts, such as education, health or social areas, and marketing ( Koivisto & Hamari, 2014). Gamification is becoming more popular day by day, and it is used especially widely in the business world. Any application, task, process, or context may be gamified. One of its most popular current applications is the social networking website Foursquare, where users share their locations and win rewards. For instance, the title of mayor is given to users with the greatest number of checkins at a given location (Glover, 2013). The base of gamification is formed by game elements such as reward and competition, as demonstrated by Foursquare. Accordingly, Gartner, an information technology research company known world-wide, has stated that 50% of companies will be using gamification strategies in 2015 (Gartner, 2011). These developments have attracted the attention of the education world, raising the question, "Is gamification useful in education?" The New Media Consortium's recent Horizon Report (2014 Higher Education Edition) indicated that gamification is becoming widespread among educators, as it increases the creativity and productivity of students. Using games in education has been common for generations, but gamification is quite a new concept. Badges, rewards, and cumulative, competitive scores provide visible incentives for student behaviors, but gamification offers much more than these simple ideas. The most valuable reasons to encourage gamification in education are its stimulating elements, such as immediate feedback, feeling of achievement, challenge and defeating (Kapp, 2012).
Based on the literature, gamification may be used to provide incentives for expected behaviors in education and to ensure that these behaviors help students to reach intended learning outcomes (Lee & Hammer, 2011;Simões, Redondo, & Vilas, 2013). Moreover, gamification has many cognitive, emotional, and social benefits (Domínguez et al., 2013;Lee & Hammer, 2011). Individuals have the chance to improve problem solving skills as they spend countless hours applying them in games (Gee, 2003). Individuals also become ready to face learning failures, since games can evoke feelings of curiosity and disappointment (Lazzaro, 2004). The literature has further demonstrated that the use of gamification in education increases motivation towards and participation in a course (Hakulinen, Auvinen & Korhonen, 2015;Lee & Hammer, 2011;Muntean, 2011). In a study that investigated the effect of gamification on learning for university students, an online learning application was developed using game strategies; gamification was shown to increase motivation and participation but had no effect in terms of achievement (Domínguez et al., 2013). Hanus and Fox (2015) investigated gamification's effects on students' motivation, social comparison, effort, satisfaction, empowerment, and academic performance over 16 weeks. They found that gamified course students had lower motivation, satisfaction, empowerment, and achievement scores than traditional course students.
In another study, social networking websites and learning management systems used in gamification strategies were compared; while the gamification method increased student participation, students trained by traditional methods were more successful (Marcos, Domínguez, Saenz-de-Navarrete, & Pagés, 2014). The researchers attributed the success of traditional approaches to the competitive nature of the gamification applications and the students' overall familiarity with traditional ways. Emotional responses such as anxiety can emerge as a result of the competition to reach resources and rewards in games (Hwang, Hong, Cheng, Peng, & Wu, 2013). This situation may affect stress levels and result in distraction from the main aim (Jones, 1995). Anxiety is closely related to performance and consumes cognitive resources. In this respect, the cognitive load levels of students may be negatively affected by gamification. Moreover, Kirschner, Sweller, and Clark (2006) suggested that constructivist learning environments can exceed students' working memory capacity. Cognitive load theory suggests that overloading working memory with high cognitive loads negatively affects students (Sweller, Van Merrienboer, & Paas, 1998). Accordingly, instructional designs should be developed that help learners to use their memory capacities effectively (Paas & Van Merrienboer, 1993;Sweller et al., 1998). PAPER GAMIFICATION AND EDUCATION: ACHIEVEMENTS, COGNITIVE LOADS, AND VIEWS OF STUDENTS Sweller, van Merrienboer, and Paas (1998) introduced three different cognitive loads: intrinsic, extraneous, and germane. Intrinsic load, depending on the content that must be learned, is formed in the active working memory. Due to the nature of man, this intrinsic load will be higher for difficult issues or complex information (Clark, Nguyen, & Sweller, 2006). The extraneous load is the overloaded working memory, which accumulates as a result of poorly designed instructional materials, causing adverse affects to learning . The germane load refers to the formation of mental structures and processes that enable regulation (Sweller et al., 1998). Excessive knowledge, applications, and life experiences that do not provide benefit to learning cause redundant cognitive load in the learning environment, which also adversely affects learning (Paas & Van Merrienboer, 1993). Therefore, decreasing factors that boost cognitive loads in the learning environment is suggested (Jeung, Chandler, & Sweller, 1997). During a course period where gamification is used, learners should consider all aspects of the activity, including rewards and learning objectives. The students do group work with their friends and compete at the same time, trying to win rewards and points while completing tasks in gamification settings. Competition can affect learning both positively and negatively (Reeve & Deci, 1996). Investigating the effect of this situation on memory capacity and cognitive load during learning may offer important insight into gamification strategies for education.
Various studies do appear in the literature, but the number is insufficient, especially with regard to experimental studies about gamification (Domínguez et al., 2013;Hanus & Fox, 2015). According to Landers and Callan (2011), studies in this area are still in their infancy. In addition, Simões, Redondo and Vilas (2013) stated in their study that gamification increases motivation but indicated that more studies should be conducted on different platforms. Few studies have been done in this field, with almost none addressing the effect of gamification on cognitive load. This study will make a major contribution to the literature and provides a valuable sample and resource for teachers and schools that want to implement the gamification method.
Three questions were investigated for the current study: • Does gamification have an effect on the achievement of students in education? • Does gamification have an effect on the cognitive load levels of students in education? • What are the views of students on using gamification in education?

II. METHODS
This study applied the mixed research method: collecting, analyzing, and interpreting both qualitative and quantitative data about the main facts in a single study or a series of studies (McMillan & Schumacher, 2010). Gamification in education was evaluated, and the effect of this method on the achievements and cognitive load levels of students were examined. Quasi-experimental design was used to investigate the effect of gamification on cognitive load and achievement, which comprised the quantitative dimension of the study. Quasi-experimental studies differentiate from experimental research in terms of random assignment of the groups (Cresswell, 2014). Since the groups were not assigned randomly, the study design was quasi-experimental. For the qualitative dimension, data was collected from the experiment group by using semistructured interviews to evaluate gamification.

A. Sampling and Implementation Process
The sample of the study was composed of 94 sixth grade students from two schools in Turkey. A unit on spreadsheets was taught to the 46 students in the experimental group using gamification strategies and to the 48 students in the control group using traditional methods. The implementation stage of the study took place over six weeks for two course hours (120 minutes) each week.
The same content was taught in both groups during the same weeks with the same activities. Gamification strategies were added to the activities of the experimental group. Elements of gamification such as competition, rewards, collecting points, and group work were implemented. In most applications, rewards and badges were displayed on the leader board. In this study, the badges were collected in an electronic environment, ClassDojo. A username was registered for each student, and an online class was formed (see Figure 1).
After completing activities, the students received rewards from this platform (see Figure 2).
In addition, using the online game-based question/answer platform Kahoot, the students reviewed the week's topic after each lesson (see Figure 3). The students in the control group completed the same activities using traditional methods. The teacher taught the topic, and the students did the given activities. Kahoot and ClassDojo were used in every lesson for the experimental group. As seen in Table 1, every week had a different gamification activity.

B. Data Collection Tools
In the study, a cognitive load scale, achievement test, and semi-structured interview form were used as data collection tools. The achievement test was developed by the researchers and included 10 multiple choice questions about the spreadsheets unit. It was used to evaluate the effects of gamification on achievement. The test was reviewed by two information technologies teachers, and revisions were made according to their feedback before determining the final version. Each question was worth 10 points, with a maximum score of 100. Cronbach's alpha was tested and found to be 0.72.
The cognitive load scale was developed by Paas and Van Merrienboer (1993) and translated to Turkish by Kılıç and Karadeniz (2004), and it was used to evaluate the effect of gamification on cognitive load (Appendix A). The cognitive load scale is a subjective rating of one question from 1 to 9. The validity and reliability of this scale was tested by Kılıç and Karadeniz (2004). Cronbach's alpha was found to be 0.78. The Spearman Brown test result was found to be 0.79. The students responded to the cognitive load scale at the end of each week, six times in total. The average of these six measurements represented the cognitive load score for each student.
The semi-structured interview form (Appendix B) was developed by the researchers and used to determine the views of students on using gamification in education. Two experts were consulted to ensure the reliability and validity of the interview form, and the final form was revised per their feedback.

C. Data Analysis
The quantitative data in this study was analyzed by SPSS software. Before data analysis, some tests regarding homogeneity and normality were applied to ensure the internal consistency of the research. Although the achievement variable data set did not have normal distribution, the cognitive load variable data set did. In order to analyze the distribution of data, the Kolmogorov Smirnov test was conducted, and the achievement variable test result was found to be p = .000. The cognitive load variable test result was found to be p = .200. Therefore, parametric and non-parametric tests were used to analyze the data, such as the independent sample t-test and Mann-Whitney U test, applying inferential statistics to determine whether a difference occurred between achievement and cognitive load levels of students. The significance level in inferential analysis was considered to be 0.05. Additionally, descriptive analysis was applied to the qualitative data.

A. Gamification and Student Achievement
Since the academic achievement variable did not have normal distribution, a condition of parametric tests, the Mann-Whitney U test, a non-parametric test, was used. When the pre-tests applied to the control and experimental groups were analyzed, no significant difference was seen: U = 993, p = .389.
A significant difference was observed between the experimental and the control groups in terms of the academic achievement variable when the scores of the post-tests were analyzed: U = 578.5, z = -4,067, p = .000. As seen in Table II, the academic achievement rate of the experimental group (Md= 58.92) was higher than that of the control group (Md = 36.55).

B. Gamification and Cognitive Load
The data obtained by considering the research question, "Does gamification have an effect on the cognitive load levels of students in education?", was analyzed using an independent sample t-test. As shown in Table III, a significant difference was found between cognitive load levels: t(92) = 2.89, p =.005. The mean cognitive load score of PAPER GAMIFICATION AND EDUCATION: ACHIEVEMENTS, COGNITIVE LOADS, AND VIEWS OF STUDENTS the experimental group ( X = 3.98) was higher than that of the control group ( X = 3.25).

C. Students Views on Gamification
Qualitative data was collected from the 28 students in the experimental group via semi-structured interviews. The aim of these interviews was to obtain common views on the gamification method. Findings indicate that most of the students had positive responses. Most students liked the method (n = 25) and wanted to use it in other courses (n = 24). In addition, the students reported that this method made lessons fun (n = 17) and learning easier (n = 16). One student explained, "As the lesson became more enjoyable, I wanted to learn more and compete. At first we did not want to complete the tasks and activities given during the lessons. In this way we always want to have some activities" (Student 10). Another student expressed, "I think I learn better with the gamification method. I think this method is good for learning" (Student 19).
Even though most of the students had positive ideas about the gamification method, some students had negative opinions. A few students (n = 3) did not like the gamification method because it caused an unnecessarily competitive environment that resulted in uneasiness. Some students (n = 2) found the method unnecessary, felt it had no benefit to learning (n = 3), and did not want to use it in other courses (n = 3). One student stated, "If I am to summarize this method with a single word, I would say redundant. It caused jealousy between students" (Student 1).

IV. CONCLUSION AND DISCUSSION
In this mixed method study, a gamification-based strategy was compared with traditional methods. The purpose of the study was to investigate the effect of gamification on students' achievements, cognitive load levels, and perceptions. The experimental group students, who were taught using the gamification method, earned better achievement scores than the control group students. Similarly, Simões et al. (2013) observed in their study held with students aged 6 to 12 that gamification made school more interesting and motivating, which accordingly increased achievement. Cheong, Cheong, and Filippou (2013) investigated the effect of gamification on learning, engagement and enjoyment , also found that gamification increased student learning level in their study. In contrast, Domínguez et al. (2013) found that gamification had no effect on student achievement in their study with university students. Hanus and Fox (2015) found that university students taught with gamification strategies had lower achievement scores than peers taught by traditional methods. This difference in terms of achievement might be caused by the design and content of the gamification process, as well as by the high average age in the study samples. In this direction, Koivisto and Hamari (2014) identified a negative correlation between age and ease of using gamification applications in their study on demographic differences in terms of perceived benefits of gamification. Even though a higher success rate was obtained with the experimental group, the average cognitive load scores of the experimental group were quite higher than those of the control group. One possible reason for this difference stems from dealing with the necessities of gamification at the same time as completing tasks. This situation may be According to the qualitative data obtained in the study, students showed positive attitudes towards gamification strategies and wanted other lessons to be taught via this method. Cheong et al. (2013) also found that students liked this method, which increased engagement. The competitive class environment and the chance to win rewards and cups affected the attention students paid to lessons and learning in a positive way. In the same way, in a study on fifth grade students, cups and rewards had a positive PAPER GAMIFICATION AND EDUCATION: ACHIEVEMENTS, COGNITIVE LOADS, AND VIEWS OF STUDENTS impact on cognitive understanding and success, and it was at least partially proven that external rewards such as cups and badges do not lead to negative results (Filsecker & Hickey, 2014). Despite its accompanying high cognitive load, the interest in gamification may be related to the novelty effect for the experimental group (Koivisto & Hamari, 2014).
In conclusion, gamification can increase both cognitive load and achievement levels, and students generally have positive thoughts regarding gamification strategies. When gamification is applied, the cognitive load factor must be considered and certain precautions must be taken to maximize effectiveness.

A. Limitations and Future Directions
The sample groups in the study were chosen from two schools per terms of time and attainability. The pre-test scores showed no meaningful difference between the groups. However, the selection of groups from two schools may be a restriction of the study because of the probability of socioeconomic differences. Another limitation for this study is that different teachers taught the experimental and control groups.
Suggestions for future studies • The effect of gamification on success and cognitive load could be further investigated using a longer time interval and larger sample with stronger integration. • The effect of gamification on different kinds of courses could be assessed. • The effect of gamification on retention of learning could be researched.

Suggestions for managing cognitive load in gamification
• Activities could be designed with no goals; thus, students use their limited working memory capacity for learning the subject. • Competition elements in class activities should be calculated for effectiveness in terms of anxiety. • Complex gamification activities should not take place in the classroom.