The Effect Of Gamified Student Response System on Students’ Perception and Achievement

Rapid advances in technology affect many areas of our lives, one of which is education. In the local Indonesian context, teachers and schools sometimes still see technology as a threat to the teaching and learning process. To get the benefits of information technology advances in teaching and learning process at schools; we need to find a technology that is easy for the teacher to use and fun for the students to learn it. Kahoot! is a form of technology in the Student Response System (SRS) with a gamification approach that can increase student involvement in the classroom. This study aims to look at the students' perceptions of Kahoot! And see the effect on non-cognitive aspects, namely Likeness and Interesting, and cognitive aspects, namely Achievement. As many as 153 students were involved in this study consisting of 127 students in the experimental class and 26 students in the control class. We implemented Kahoot! in chemistry lessons for three months in the innovative class then surveyed to see their perceptions. We also compare the academic score of a chemistry lesson from both courses before and after the implementation. More than 90% of the students perceived Kahoot! as easy, fun and engaging, and 83.75% of respondents felt more positive towards the topic. The results show that Kahoot! has a positive effect on student perceptions and non-cognitive aspects, but the application of Kahoot! has no direct impact on the cognitive elements observed, namely Achievement. Keywords—Student Response System, Gamification, Kahoot!, Chemistry lesson.


Introduction
Our present world is changing rapidly due to technological advancements. It affects many areas of our lives, one of which is education. Today's students are digital natives. Their ability to use various applications of sophisticated technology development often exceeds the knowledge of their teachers [1]. This situation has impacted on teaching and learning process in the classroom. For decades, the teacher gives materials in the school in a conventional way. The teacher gave lectures in front of the school, while students listened and took notes. This learning model provides little interaction between teacher and students because students tend to engage passively with the lesson materials. This conventional way where the teacher is seen as the primary source of information has been deemed unattractive [2], therefore need to be changed.
One technology that can be used for teaching purposes is the Classroom Response System (CRS), which is also known as the Audience Response System (ARS) or Student Response System (SRS). SRS is a technology designed to facilitate classroom management while providing various tools that function to facilitate the teachinglearning process [3]. The use of SRS has been known for increasing student involvement in the classroom [4] [5] [6] [7], improving students' attitudes towards the subject and their perception about their learning environment [4] [8], increasing the level of student attendance in the class [7] [9], and increasing the ability of students to remember subject matter [10]. However, some studies also emphasise the importance of teacher or instructor creativity when using SRS [7] [10] [11]. Therefore, teachers or instructors need to consider using the gamification approach when using SRS.
Gamification is an informal term for the usage of elements of video game in a nongame context to enhance user experience and user engagement [12]. Gamification approach to SRS is expected to be able to improve students' attitudes and achievements.
The SRS application used in this study is Kahoot!. Unlike other forms of SRS such as Clicker, which requires specific hardware devices, Kahoot! utilises students' own devices, such as smartphones, tablets, or laptops, so that schools do not need to buy special equipment. It merely uses the school's wifi network, classroom's projector and teacher's laptop

Context
In the local context of Indonesia, teachers and schools sometimes still see technology as a threat to the teaching and learning process. Many schools forbid their students from carrying or using their devices, such as smartphones, tablets or laptops, at school. Some of the more modern schools, such as private XYZ High School in Central Java, Indonesia, try to embrace technology by allowing students to bring or use their devices at school. The school even provides free wifi at the school. However, while infrastructure and regulations support the use of technology at schools, it does not mean that teachers are ready to adapt and use technology in their classrooms. Based on preliminary interviews with teachers and the headmaster of XYZ High School, we found that teachers had to struggle to get students' attention from their devices. To get the benefits of information technology advancements in teaching and learning at schools, we need to find a technology that is easy for the teacher to use and fun for the students to learn it. In this study, we used Kahoot!, an SRS technology to fill the need. Kahoot! was implemented in the chemistry lesson. The chemistry lesson was chosen because this subject was considered as one of the difficult topics. The average score of the end of the semester in the 2017/2018 school year was 5 (out of 10), and the number of students who choose chemistry as the elective exam in the majors/specialisation of science in 2017/2018 national exam was 2 (out of 150 students).

1.2
The objective of the paper This study aims to look at students' perceptions of the use of Kahoot! in the chemistry lesson and whether the implementation of Kahoot! can help improve student academic achievement.

2
Theoretical Background

SRS, gamification and kahoot!
Students Response System (SRS) is a technology designed to facilitate classroom management while providing various tools that serve to facilitate the teachinglearning process [3]. SRS which is also known as Classroom Response System (CRS) or Audience Response System (ARS) has been successfully applied in university classrooms [3] [7], helps increase students' focus during class [16], improve students' attitudes towards the subject and their perceptions about their learning environment and [4] [8], increase attendance in the classroom [7] [9], and make it easier for students to remember material [10]. SRS looks more advantageous for classes with large spaces where teacher-student interaction is more limited [4] [11] [17].
Some studies also note an increase in achievement of students [8] [10] [11] [16], but some studies argue if the use of the Student Response System can directly improve student grades [4] [7] because there are many factors to consider. Improved grades are an indirect result of students' attitudes towards subjects [4] [7]. These studies also emphasise the importance of teacher or instructor creativity when using SRS [7] [10] [11].
The first form of SRS is clickers which require a set of tools that are shaped like a remote control where students can answer the teacher's question by clicking the button on the remote control [10]. This type of SRS has several disadvantages, such as high costs, limited mobility, and other difficulties related to technical setup and installation [2]. With the advancement of internet technology, new forms of SRS began to emerge. One of the popular SRS applications available on the internet is Kahoot!.
Gamification is an informal term for the usage of elements of video game in a nongame context to enhance user experience and user engagement [12]. Gamification is used as a means to support user involvement and improve service use, such as social interaction, user activities, or quality and productivity [18]. The gamification approach in the Student Response System is expected to be able to improve students' attitudes and academic achievement in the classroom. The game approach has to have the following elements: a sufficient level of challenge, abstraction or fantasy, and arousing curiosity [19]. Mader et al. added other essential game elements: teamwork and immediate feedback [9]. They also suggested that cooperation within competition [9].
Kahoot! is a game-based SRS developed from a research project called the Lecture Quiz [20]. The researchers wanted to create a platform where teachers and students in the classroom could interact through games of knowledge using the available infrastructure. Kahoot! makes the class a quiz program where the teacher is the host and students are the participants. Participants get points when correctly answering questions given around the subject. At the end of the quiz, the host announces the winner. Kahoot!'s main advantage is that there is no requirement of a particular device. The teacher can use their laptop and class' digital projector, while the student can use their smartphones or laptops. See Figure 1 for illustration.

Educational learning outcomes categorization
Hunsu et al. categorised aspects of learning outcomes observed in studies of the usage of Student Response Systems [7]. As shown in Table 1, these aspects are divided into two main categories: cognitive and non-cognitive. The non-cognitive aspects are divided into two groups, namely Behavioural and Attitudinal. This study will look at one cognitive issue, that is Achievement, which is based on academic grades, and three aspects of non-cognitive attitudinal, namely Perception and Attitudes, Interesting and Likeness. Measured if the student thought they enjoyed the class

Method
The research was conducted using an experimental method carried out at XYZ high school in Central Java, Indonesia. The subjects chosen are chemistry. Kahoot! was implemented for three months from after midterm exam until the final semester of 2017/2018. During that time, the students played Kahoot! once every 2 weeks for quizzes, brainstorming or weekly tests. After completing the experiment period, we took the student's final exam scores and compared them with their midterm scores. Students from experimental classes are given a questionnaire to see the effect of Kahoot! in non-cognitive aspects.
There were six classes to be observed that are five classes were used as an experimental group that received treatment, namely Kahoot! implementation in chemistry subjects and one class was used as a control group that did not receive treatment.

Sample and questionnaire
Students who involved in this study were the 12th grader of XYZ High School (age 17-18 y.o). All of them studied chemistry with the same teacher. The number of students involved is 153, as shown in Table 2. We used 12th-grade science class 1, 2, 3, 5 and 6 as experimental classes. We assigned 12th-grade science class 4 as control class because specifically in this class, students are prohibited from carrying devices to school. Thus, there were 127 students in the experimental class and 26 students in the control class. We experimented by integrating the use of Kahoot! into chemistry lessons for three months. Kahoot! was played once every two weeks during the period. To see the effect Kahoot! on non-cognitive aspects, we asked the students from experimental classes to fill out a questionnaire at the end of the semester (the end of the research period). Meanwhile, to see the effect of Kahoot! on cognitive aspects, we took midterm exam scores (before the research is conducted) and compared them with final exam scores (after research has been conducted). During the study, we also made field notes to document other things that happened during the research process.  Table 3 shows the questions students must answer in the questionnaire. The questionnaire consists of 3 parts:   Table 4 shows the learning outcome aspects observed in the questionnaire. We also used this questionnaire to see how the students perceived the elements of gamification in Kahoot!. We analysed most of the data from the first part of the survey. The second and third sections of the questionnaire will be used as a supplement to strengthen the results of this first section. Specifically for cognitive aspects, we used mid and final semester exam scores.

Result, Analysis and Discussion
The data obtained in this study is analyzed into two categories of educational learning outcome: the non-cognitive and the cognitive aspects [7].

Non-cognitive
Of the 127 students enrolled in the experimental classes, four students were absent during the questionnaire filling session. Thus the analysis was carried out to 123 students who filled out the questionnaire.  Table 5 shows that statement 1.1, 1.2, 1.3 and 1.4 get "Agree" answers from more than 90% of respondents. Comments 1.5, 1.6 and 1.8 get "Agree" answers that agree more than 80% of respondents. Only statement 1.7 (I want Kahoot! was played more often in chemistry subject) has the least accepted answer among the others, which is 79.84%. It shows that students have a positive perception of the implementation of Kahoot! in the classroom.  Table 6 shows that out of 123 students, only one student had ever played Kahoot! before it was implemented in the chemistry lesson. So in general, Kahoot! was seen as something new for students. The result of the second question shows that most students play Kahoot! by using a friend's smartphone, not their smartphone. It means that Team mode was used more often than the classic way. Team Mode allows students who do not own a smartphone or who think that their smartphone is slow to be still able to participate in the Kahoot! quiz by joining as a team with other students whose smartphone is better. While from the result of the third question, we can see that more students prefer the team mode to the classic way. It highlights the students' preference for cooperation within the competition to individual competition. Cooperation within competition leads to higher task enjoyment. The team mode also encourages discussion that has a positive influence on students' learning [9].
In the third part, students are asked to answer questions in the form of an open question. Students can use the third part of the questionnaire to clarify their opinions from questions in sections one and two or use it to tell the things they have experienced or the things they have observed that have not been revealed through the items in parts one and two. For this reason, the results of each question will be displayed in separate tables. Table 7 shows the answers students wrote in their own words. Some students write a lot of things, and some students write a little. For example, below is a response from one student: "Kahoot! is fun, easy to play and testing our knowledge. We can to work together and compete with other students. Kahoot! brings a different atmosphere. I was not sleepy nor lazy. I was excited. Kahoot! Swag! I like this. " This answer contains her perception of Kahoot!, being easy to play (Perception and Attitude), testing the students' knowledge, enabling students to work together and compete against one another (Gamification). Kahoot! helped her not getting sleepy nor lazy, and it was exciting (Interesting).
Each answer that appears is shown in Table 7 with the number of students who mentioned it. From the 13 variations of solutions that emerged, nine of them were related to gamification, namely competition, cooperation, challenges and feedback. Although Likeness and Interesting are the two non-cognitive aspects observed in this research, and Table 7 shows that those are not the only non-cognitive aspects affected by Kahoot! Experience and Effective elements also emerged from the answers, like the one written by a student.
"It was fun because it was the first time I used the internet to learn chemistry. Usually, I use the internet to play online game and social media only."  Table 8 shows that technical factors such as an unstable internet network made the session less fun. Some students reported having difficulty logging in. Some students logged out suddenly during the quiz and had to log in again. This technical factor is related to school infrastructure. Improvements in technical factors require involvement from the school. The problem with WiFi or internet connection is common during the study of Kahoot! implementation [16].
Another factor that is considered making the session less fun was the quiz design. It is where the creativity of the teacher is challenged [10]. Kahoot! naturally is suitable for a trivia quiz, which requires a short time to answer. Therefore for mathematical questions, the teacher or instructor must adjust several settings such as Question Time (maximum time allowed to solve a problem). Monotonous items make students bored. Students also complained about the times when the teacher did not provide an explanation or discussion after specific questions. It is part of the teacher's facilitation skill. Facilitation skill and quiz design are factors that can be improved by the teacher. Another factor that also mentioned in Table 8 is the element of gamification, such as an unpleasant feeling of losing. Losing and winning are the competitive aspect of gamification. Questions that are considered too difficult (or too easy) mean they do not give an adequate level of challenge [19]. However, it can be improved by the teacher through quiz design skill.  Table 9 shows the recommendations the students gave to improve the implementation of Kahoot! in future sessions. Improvements in technical factors related to WiFi networks turned out to be the most widely recommended. The next recommendation is related to quiz design and class facilitation during study sessions using Kahoot!. This recommendation can be made by increasing the teacher's capacity in adjusting settings in Kahoot!. Two students mentioned a real prize, means they expect to get something from the teacher when they won the game. It shows the negative effect of the reward system. It motivates the students extrinsically, not intrinsically [9]. The next recommendation is related to the Likeness aspect. The students wanted Kahoot! to be played more often and longer during each session. They also recommended it to be used in other subjects, even used in other schools.

Cognitive
The following analysis is on cognitive aspects, which is about the students' academic achievement. In this aspect, the data is taken from the experimental classes and the control class. The data used is the score of students on the midterm and final exam.  Table 10 shows that the average score dropped at the end of the semester. This decline occurred in all experimental classes. The overall average score in the experimental classes dropped from 33.92 to 23.90. The data shows a decrease of 10.02 points or 29.5%. In the control class, the average score also dropped from 38.5 during the midterm exam to 22.77 during the final semester exam. There was a decrease of 15.73 points or 40.86%. This data shows that there is a factor that strongly influences the impairment of all classes. According to the teacher, the difficulty level of the final exam is usually higher than the midterm. It is because of the number of materials tested in the final exam is more than those in the midterm. The result is consistent with the research of Baumann that there are many factors that influence student achievement. The use of Kahoot! alone is not enough to encourage direct achievement improvement [4]. Kahoot! helps increase students' motivation, which in turn can increase student academic grade. If we look at the average value after normalisation, it turns out that both in the experimental classes and the control class, the score rises. The increasing average score after normalisation demonstrates that the understanding of the material becomes more evenly distributed amongst the students.
Other data obtained after the final semester exam is that there are five students decided to take chemistry subjects in the national examination as elective exams. Compared to the previous year where only two students chose chemistry as the elective exam, it means there is a 150% increase. This increase means that they are more confident in taking the chemistry exam. In the non-cognitive category, this is the selfefficacy aspect [7].

Discussion
Ninety-nine per cent of students uses Kahoot! for the first time in this study. Therefore, they are still very enthusiastic about playing it [20]. It is understandable because Kahoot! is still considered as something new for students. Kahoot! was used in only one subject, namely chemistry. The results might be different if Kahoot! has been used in most items and is used for several semesters [20].
Although this study aims to see the effect of Kahoot! on non-cognitive aspects, namely Likeness and Interesting, during the survey, we see Kahoot! also affect Perception And Attitude, Experience, Effective and Self-Efficacy. The result shows Kahoot! has a positive influence on non-cognitive aspects which is consistent with Hunsu et al.'s finding [7].
In the cognitive aspect, all classes experience the same trend of decreasing score, (both in the experimental classes and in the control class). It shows that strongly affect all types. The implementation of Kahoot! alone could not have a direct effect on students' achievement. It is consistent with Bauman and Hunsu Study [4] [7].
It was also obtained from the open questions that the factor that students liked most about Kahoot! is competition and cooperation. Other gamification elements evident in this study are challenges and feedback.
To make the session better, students recommended improvement related to technical matters, such as the strength of the school's internet network. Other recommendations are related to quiz design and facilitation factors which can be improved by the teacher [7] [9][ 10] [11].

Conclusion
With more than 90% of the students perceived Kahoot! as easy, fun and engaging, and 83.75% of respondents felt more positive towards the topic, students of XYZ High School perceived that Kahoot implementation made them enjoy chemistry lessons more. Kahoot! has a significant influence on the non-cognitive aspects of the results of the teaching-learning process, especially the attitudinal aspects, namely Interesting, Likeness, Perception and Attitude, Experience, Effective and Self-Efficacy. These aspects are essential in the teaching and learning process because they are related to students' perception of the lesson.
Implementation of Kahoot! in short time (three months) does not have a direct effect on cognitive aspects, namely achievement because this aspect is influenced by many other factors and is an indirect effect of students' attitude toward the subject.
Although Kahoot! has integrated the concept of gamification in their applications, teachers or instructors are still required to be creative in designing the quizzes and facilitating the class