The Applicability of Augmented Reality Technologies for Evaluating Learning Activities

a new trend in the development of immersive technologies has become augmented reality (AR), which is in demand due to its property to implement visual objects to enrich the learning content. The paper is devoted to the study of the applicability of AR technologies for evaluating learning activi-ties since there is a problem of inconsistency of teaching approaches with tools that lead to biased results. This led to the development of the “AR Quiz” application that contains interaction types such as touch-based, voice, input field, gaze and gesture that stimulate activities. In combination with 10 other forms of assessment materials, its application field has expanded and the tasks for students have diversified. The present study provides the calculation of validity and reliability coefficients of the assessment materials contained in the “AR Quiz” application that reflects the suitability of indicators for the purpose, accuracy and stability of measurements. The paper reveals positive attitudes of expert teachers and students towards the use of AR when evaluating learning activities. Along with integration map of compliance of AR interaction types with assessment materials, the paper provides recommendations for teachers on evaluating learning activities based on AR.


Introduction
In the modern world, education is rapidly developing due to the introduction of innovative technologies that are driven by the digital transformation and the need to form new skills required in the 21st century. Nowadays, the formation of skills is the main and more important task of learning than the acquisition of theoretical knowledge as it was in the past. The P21 organization (The Partnership for 21st Century Skills) in partnership with teachers, education experts and world-class business leaders has identified a list of skills that will be in demand in the 21st century. They were grouped into three main areas: Learning and innovation skills; Information, media and technology skills; Life and Career skills [1]. The relevance of formation of these skills has become very noticeable among students in the context of an ever-changing economy and technological progress which is the main factor that leads them to achieve success at school, at work and in life.
The variety of innovative educational technologies allows one to make optimal choice for solving issues of skill formation. However, being a fundamental basis, evaluating learning activities requires improvement and remains to be the main problem of the present research since the tools used are not always applicable. Moreover, when assessing learning activity, the knowledge model often prevails thus the learning activity model, which is characterized by the functional actions of students, is not always taken into account [2], [3]. Consequently, this leads to a mismatch of instruments for the purpose of the assessment. The main reason for this is the lack of tools that stimulate processes and phenomena as objects of study.
According to the scheme ( Figure 1) that illustrates the process of competence formation, learning activities form skills, and they, in turn, together with knowledge and skills, form competencies. Based on this, by evaluating the com-ponent part as a learning activity, we will be able to evaluate the learner's skills more deeply. Having studied the possibilities of the rapidly developing augmented reality (AR) technology, which can form the skills of the 21st century [4], it can be assumed that its use as a tool for evaluating learning activities will be more effective. In this regard, there is a need for research that aims to find out the applicability of augmented reality technology for evaluating learning activities.
AR is a combination of the physical world with virtual objects in real time [5]. AR can contain virtual objects as computer graphics, text, sound, feedback mechanisms and etc., which makes it possible to expand the boundaries of under-standing our reality.
Research on the use of AR technologies in education has shown that it has a positive effect on student performance in STEM learning, enhancing the assimilation, understanding of educational material and creating a mediated learning environment [6], [7]. Practice shows that AR also helps to cope with difficulties in studying stereometry in mathematics, developing the imagination and spatial thinking of students [8]. AR increases the proportional, probabilistic, combinatorial, and correlational reasoning indicators of abstract thinking skills in physics teaching [9]. Physical and perceptual involvement of students in the study of Computing Education [10], the synchronous combination of the virtual world with reality and the possibility of using it as a very interesting media for history learning [11] shows the breadth of AR technologies application. The freedom to create various functional scenarios as well as the enrichment of audio-visual accompaniment gives favorable results in Immediate-Action Commander Training [12]. AR technology has shown its applicability in sports education and training [13], Mechanical Engineering Education [14] and Civil engineering education [15].
The main advantages of using AR in the educational process: ─ The use of AR increases the indicators of attention, relevance, confidence and satisfaction according to Keller's ARCS motivation model which determines the reason for studying the material and the duration of working hard. The ability to control and jointly create new knowledge in an AR environment helps to achieve a high level of concentration when performing tasks, to describe and analyze the content of training in more detail, and to increase involvement in learning activities with less cognitive effort [16], [17]. ─ Approaches used in the organization of learning activities in the AR environment complement the mechanisms of interaction with objects in a similar way to the elements of gamification [18]. The implementation of gamification elements which contributes to an increase in quantitative and qualitative data necessary for the analysis of the learning process and improves student engagement and learning-related behavior [19], [20]. ─ Forms the conceptual understanding of students [21]. The properties of the absence of restrictions in space and time arouses interest in the nature of the origin of things, forming a conceptual understanding among students. ─ AR environment influencing students through factors of uncertainty, excitement and novelty increases the desire to learn and integrate through developing visual thinking and achievement [22]. Students, based on their existing knowledge, skills and abilities add new ones deepening their visual thinking. ─ Develops moral imagination which is a critical ability that helps to find alternative solutions to various dilemmas, taking into account all possible consequences [23]. ─ Helps to form spatial thinking. It involves understanding and moving various objects mentally or in physical space [8]. ─ Providing information in a multimedia context, in a virtual form, arises a huge interest of students. Multimedia tools combine 3D models, animations, videos, sounds, texts, and graphics, and AR includes all the features and principles of multimedia tools [24].
Nevertheless, the huge potential of AR technology has not yet been fully realized and the methodological aspects of evaluating learning activities on its basis have not been sufficiently studied.
Analyzing the theoretical and methodological aspects of learning, it can be assured that mastering any skill is done only through practical experience. The reflection of practical experience in the educational process is learning activity. According to the European Union, learning activity is any activity organized with the aim of improving knowledge, skills, and competence of a student [25]. At the same time, learning activity is not merely a "learning task from the teacher", but it has a stronger connection with students' academic performance [26]. The implementation of learning activities in a virtual environment includes opportunities for the development of various skills, observation of achievements and communication with reality [27].
Evaluation of learning activities using AR plays an important role in the stage of skill formation, providing an opportunity to verify the correctness or inaccuracy of the chosen learning strategy. Moreover, AR has a positive effect on the learning outcomes since it synchronizes the speed of information delivery with timely perception of the student., which.
AR-based assessment materials developed for the evaluation of learning activities help to determine the most important criteria for the selected learning activity: correctness, compliance with the content, influence on the student's motivation and use its formative, supporting and developing functions. Moreover, they derive objective indicators for evaluating learning activities, diagnose various problems of the learning process, determine the quality of the current program, show the dynamics of mastering learning activities, and develop self-assessment.
Practice has shown that it is necessary to take into account the auxiliary function of assessment using AR, which often remains in the shadows, but has a strong impact on the final achievements of the student. Since the learning outcomes of each course are defined as skills, students should be evaluated in accordance with the level of proficiency in these skills expressed in learning activities.
The rest of paper is organized as follows. The methods section describes the stages of conducting a study, which consists of developing and using an AR-based mobile application for evaluating learning activities; identifying the validity and reliability of assessment materials; student survey; conducting advanced training courses for teachers of IT disciplines and survey them. The following sections present the results for each stage and recommendations for future research.

Methods
The first stage of the research was the development of a mobile application based on AR technology called "AR Quiz", which evaluates the learning activities of students by enhancing the properties of assessment materials.
"AR Quiz" application ( Figure 2) contains 20 assessment materials on the discipline "Information and Communication Technologies" in 10 different forms. The "AR Quiz" application was developed on the Unity3D platform using 3D models in formats .fbx, .3dmax, .blend, which are built to work in mobile devices. Smartphones and tablets, which almost every student has, have high portable properties in order to be able to carry them around. They have full-color dis-plays, built-in cameras, powerful processors, sensors and most importantly are individual. Due to these characteristics, mobile devices have become the most suitable devices for working with AR applications [28]- [30].

Fig. 2. "AR Quiz" Application Interface
The main goal of developing an "AR Quiz" application is to adhere to the didactic principles of AR [31] and to implement interactivity, which evaluates learning activities and provides feedback to students. If we recall the main three characteristics of AR that Azuma defined: combines real and virtual, is interactive in real time, is registered in three dimensions [5], then it becomes clear that interactivity largely determines the functionality of AR. In our case, this is the evaluation function.
The following types of interactions with AR objects were implemented in the "AR Quiz" application: 1. Touch-based interaction. It includes all the uses of on-screen sensors that are performed by the fingertips to manipulate 2D and 3D objects. Interaction through Buttons is used for pre-defined C sharp events, animating AR objects and controlling the application interface. The imported Lean Touch package from the Assets Store allowed to implement interactions such as Select, Scale, Rotate and repositioning with the Touch Pad ( Figure 3). Drag and drop objects ( Figure 4, Figure 5) to the desired area and passing the position on the x, y, z axes was implemented using the Vector3 function.   2. Voice Interaction. Voice interaction is divided into three types: voice command, dictation and grammar recognition [32]. In the "AR Quiz" application Voice command was implemented, namely the system recognizes the specified keywords and triggers events. For the purpose of implementation in mobile devices, The Speech To Text service provided by IBM Watson was used. The conversion of the human voice into the written word happens due to deep-learning AI (Artificial intelligence), which provides a highly accurate mechanism for speech analysis. The API (application programming interface) ( Figure 6) of the service allowed merging with Unity3D.

Fig. 6. API of Speech To Text from IBM Watson
The task of one of the assessment materials in the "AR Quiz" application was to correctly install the AR CPU (central processing unit) into the motherboard using voice control (Figure 7).

Fig. 7. CPU in Motherboard
The OnRecognize recognition function contains keywords such as "go", "back", "up", "down", "left", "right", etc. for controlling the AR object of the CPU. Below is a snippet of the C sharp script: 3. Input Field. Entering a text or numeric value into a field. It was used to enter the answer to a given question and feedback from students ( Figure 8).  5. Gaze interaction. Interaction occurs due to the direction track of the gaze as a ray, which seems to come from the camera and is determined by a point in the center of the smartphone screen ( Figure 10). When a point collides with an AR object, the program calls an event. This is the reproduction of relevant information, position change or selecting an object. Gaze interaction fits perfectly in tasks such as hotspot.

Fig. 10. Gaze interaction
The parameters like gravity, kinematics, drag of the Rigidbody component ( Figure  11) and setting the physical form using the Collider component helped to add the physical properties for AR objects, which brought them as close as possible to our reality.

Fig. 11. Component Rigidbody
All the assessment materials contained in the "AR Quiz" application were based on the following well-known forms: ─ TRUE OR FALSE is a type of question that requires the student to choose one correct choice from two options. ─ MULTIPLE CHOICE is a type of question that requires the student to choose one correct answer from several suggested options ─ MULTIPLE RESPONSE is a type of question that requires the student to select all of the correct choices from suggested options.
─ FILL IN THE BLANK is a type of question that requires the student to enter the answer in a blank field. ─ WORD BANK is a type of question that requires the student to drag and drop the correct choice into the empty box. ─ MATCHING DRAG AND DROP is a type of question that requires the student to drag and drop items in the second column to match items in the first column. ─ MATCHING DROP-DOWN is a type of question that requires the student to select items from drop-down menus to match items in the first column. ─ SEQUENCE DRAG-AND-DROP is a type of question that requires the student to drag and drop items to arrange in sequence. ─ NUMERIC is a type of question that requires the student to enter the correct numeric value. ─ HOTSPOT is a type of question that requires the student to click on the correct area within the image.
The second stage of the study was the Pre-test and Post-test for 30 students of the experimental group, aimed at evaluating learning activities on the subject of "Information and Communication Technologies". Both tests contained 20 assessment materials in 10 different forms, which were given above. For each correct answer, the student received 1 point, the highest score is 20.
The developed "AR Quiz" application, which integrates forms of assessment materials with interaction types was used only in post-test, so that students could compare the capabilities of AR technology and give feedback to researchers.
The third stage included receiving feedback from students. At the end of the testing, they were offered a number of statements to express their agreement or disagreement on the Likert scale.
In the fourth stage of the study, the following were calculated using the Pearson Product Moment Correlation Coefficient formula: validity -an indicator of objectivity confirming the suitability of the test for the purpose, as well as reliability -reflecting the accuracy and stability of measurements of AR-based assessment materials used. In order to identify the reliability indicator, after 2 weeks, the students repassed this test under identical conditions [33].
The last stage was arranging advanced training courses for 40 IT teachers on the development of assessment materials based on AR technologies for evaluating students' learning activities. After completing the course, a questionnaire was conducted to reveal teachers' expert view.

Results and discussions
The results of the study on the use of AR technologies for evaluating students' learning activities showed a positive trend which are presented as follows:

Results of student feedback
Students were offered statements on the completed test and were required to indicate the degree of their agreement on the Likert scale. In order to get the most accurate results, 5 gradations of answer options were used, where 1-completely disagree, 2-disagree, 3-difficult to answer (in the middle), 4-agree, 5-completely agree. Figure 12 shows the average values of 30 students for each statement. We can see that in most cases, the average values are greater than 3. This al-lows us to conclude that passing the test using AR technologies was an interesting experience for students, as they expressed their involvement and great interest in the implementation of learning activities based on AR. This statement received the maximum average value, about 4.87. The average value of the statement that testing contributes to learning something new was 3.93. It is the smallest of the results obtained, which still exceeds the value of 3.

The validity coefficient of "AR Quiz" application
Calculating the validity coefficient of the test application "AR Quiz" clarifies the compliance of the assessment materials contained in it for the purpose of testing. In our case, this is an assessment of learning activities. The correlation with the independent experts' estimates helped to establish the numerical validity coefficient empirically.
The results of the calculations are interpreted as follows: a) if takes a value in the range from 0.6 to 1, the validity is considered high; b) if takes a value in the range from 0.3 до 0.6, the validity is considered middle; c) if takes a value less than 0.3, the validity is considered low. According to the results, the validity coefficient of "AR Quiz" Application was ≈ 0,85 which is high and corresponds to the purpose of testing.

3.3
The reliability coefficient of "AR Quiz" application The reliability coefficient was measured by repeated testing on the same sample of subjects, two weeks after the first test. Reliability reflects the accuracy of the test measurements, as well as the stability of the test results to the action of random factors. The Pearson's formula (1) was applied, where -correlation coefficient of the reliability, -number of students, -test results of first test, -results of second test. The results of the reliability coefficient calculations are interpreted as follows: takes a value in the range from 0.9 to 1, the reliability is considered extremely high; b) if takes a value in the range from 0.8 to 0.9, the reliability is considered high; c) if takes a value in the range from 0.7 to 0.8, the reliability is considered middle; d) if takes a value less than 0.7, the reliability is considered low.
According to the results, the reliability coefficient of "AR Quiz" Application was ≈ 0,83, which is a high level and shows its stability. It should be noted that in most cases, students scored more points in the second test than in the first one. This shows that the use of AR technologies for evaluating learning activities helps to improve the results of residual indicators.

3.4
Survey results of teachers 40 teachers of IT disciplines took advanced training courses. During the course, they created AR-related projects and implemented the main types of interaction with AR objects to use in the educational process and during the development of assessment materials.
According to the survey results, the use of AR technology for evaluating students' learning activities was supported by 19 teachers for 10 points, 4 teachers for 9 points, 7 teachers for 8 points, 5 teachers for 7 points, 3 teachers for 6 points and 2 teachers for 5 points out of the maximum possible 10 points, where 1-completely do not support, 10-fully support ( Figure 13).

Fig. 13.
Teachers' evaluation supporting the use of AR technology to assess students' learning activities To the question "At what stage of learning is it better to use augmented reality technology?", where choosing several answers were available at the same time (Table 1), the option -evaluation of knowledge, skills and abilities (educational activity) was chosen 32 times. According to the teachers' expert view, AR technology is quite applicable for evaluating learning activities and does not violate the methodological aspects of teaching. In addition, an integrated map of AR interaction types with forms of assessment materials was proposed ( Figure 14).  According to the integrated map, touch-based interaction is the most widely applicable, since it can be used to implement the largest number of forms of assessment materials. This type of interaction maximizes the activity of the student and allows them to be evaluated more.
The study results allowed to formulate a number of recommendations for evaluating learning activities based on AR technology: ─ To design a detailed evaluation scenario in the AR environment; ─ To follow the principle of continuity and logical sequence of students' activities; ─ To include tasks that promote students' actions and stimulate their motivation; ─ To take into account the physiological features of students' information perception; ─ To make it flexible for the purpose to correct your mistakes; ─ To increase interactivity in combination with various forms of assessment materials; ─ To calculate the validity and reliability of assessment materials for evaluation; ─ To use the formative, supporting, developing and auxiliary functions of evaluating learning activities that lead to the formation of skills.
The results of the study and recommendations help teachers to reduce the level of routine actions and help to achieve justified feedback from students when evaluating learning activities. By integrating the types of AR interactions with the forms of assessment materials, teachers will be able to develop their assessment materials in other subject areas, thus influencing the development of pedagogical measurement in general.

Conclusion
According to the conducted research, it can be concluded that AR technology is quite applicable for evaluating learning activities, because it meets the methodological requirements and the purpose of evaluation. Moreover, AR enhances the characteristics of assessment materials by adding touch-based, voice, input field, gaze and gesture interactions.
The developed "AR Quiz" application helped to confirm the effectiveness of the implementation of AR in the process of evaluating educational activities and to obtain quantitative research indicators reflecting the degree of validity and reliability, the opinions of students and teachers. The phased research allowed to consider the main aspects that affect the final result.
In the future, it is planned to deepen research on the evaluation of learning activities using other devices in different subject areas, which will further expand the boundaries of the development of augmented reality in education.

6 Authors
Talgat Sembayev is a PhD student at the Department of Computer Science, Information Technologies Faculty, L.N. Gumilyov Eurasian National University, 010000, Nur-Sultan, Kazakhstan.