Paper—An Evaluation of the “PicsAR” Research Project: An Augmented Reality in Physics Learning An Evaluation of the “PicsAR” Research Project: An Augmented Reality in Physics Learning

This paper is one of the outcomes of the PicsAR (Physics Augmented Reality) research project that is focusing on the evaluation of students’ abstract thinking skills while utilizing augmented reality (AR) in the atomic model. The sequence of the research has emerged into three steps: developing, validating, and the evaluation of the PicsAR. The research utilized an ADDIE model: “Analysis-Design-Development-Implementation-Evaluation”. In Spring semester 2019, the authors conducted these steps and resulted in a pocket of PicsAR booklet and AR application based on Android. Then, the trial of application was conducted to 33 students in private high school in Surabaya, Indonesia. Parameters of evaluation included the quality of PicsAR, impact on students abstract thinking skills, and the research outcomes. The results indicated (1) the process of developing AR in atomic model fulfil the criteria of product quality: validity, practicality, and effectiveness, (2) performing of students’ abstract thinking skills reached at least 66.67% in the combination of good and very good categories of all reasoning categories, (3) through PicsAR research project resulted in two prior publications and one property right. The atomic model is one of abstract physics concept representative in presenting the use of augmented reality in physics learning; therefore, the recommendation of this research is another abstract physics concepts should address the use of AR as a media for learning. Keywords—Abstract thinking, augmented reality, physics, atomic model


Introduction
Since industrial revolution 4.0 was issued, there were the displacements of learning from traditional to digitalization. For example, traditional teaching or tutoring began to shift to online learning. The use of technology is spread out in learning process; we recognize the use of online learning via Massive Open Online Courses (MOOCs), social media (Youtube, Facebook, WhatsApp, Line, Edmodo, etc.), virtual learning, and e-learning [1]. The trend of literacy was also shifted from traditional literacy to digital literacy, such as from paper book to e-book [2]. Consequently, the use of applications and software is necessary to meet the demands of the industrial revolution.  [14][15] Milgram and Kishino defined Milgram's Reality-Virtuality Continuum ( Figure 1) [14][15]. They describe a continuum that stretches from the real environment to virtual environments. They concluded that AR is closer to the real situation, while Augmented Virtuality (AV) is closer to the virtual environment [14]. In other words, AR itself is an effort to incorporate the virtual world into a real environment directly, while AV is an effort to organize a real situation into a virtual world.
In the context of education, AR can be used to visualize objects spatially and temporally. Make objects that are very small or far within reach visible. This can be done by increasing the microscopic scale, decreasing the macroscopic scale, or by making the interior exterior of everything made invisible to be visible In this study, augmented reality was developed using several applications, namely 3D blender which is useful for making 3-dimensional animation, vuforia which is useful for making markers to be used, and unity which is useful for uniting 3dimensional animation with markers that have been made. This form of augmented reality output itself is an (apk.) format that can be installed directly into an Android smartphone.
For the operation of AR media is quite easy when opening the application directly directed to the smartphone camera, the marker available on the hand-out will be detected by the camera so that it will bring up an animated 3-dimensional object on the smartphone screen, or can be seen in Figure 2. The mechanism of action of AR [16] This augmented reality media was developed to provide facilities for students to carry out the learning process, especially with an abstract concept. Learning content in media in the form of 3-dimensional animation is useful to help students understand an abstract concept. In this application can be used in several kinds of images that have been programmed and will produce 3-dimensional animation that is different in each image.
The main research questions (RQs) of this evaluation are threefold: 1) is the PicsAR of atomic model fulfil the criteria of product quality: validity, practicality, and effectiveness? (2) To what extent do the students' abstract thinking skills after using AR technology? (3) To what extent do the PicsAR research project resulting in the research outcomes? All the three RQs is an elucidation of the evaluation parameters referred to this study.

Method
The research adopted Research and Development (R & D) paradigm. R & D is " an industry-based development model in which the findings of the research are used to design new products and procedures, which then are systematically field-tested, evaluated, and refined until they meet specified criteria of effectiveness, quality, or similar standards" [17]. The sequence of the research has emerged into three steps: developing, validating, and the evaluation of PicsAR. The developing and validating utilized an ADDIE model: Analysis-Design-Development-Implementation-Evaluation. Parameters of evaluation included the quality of PicsAR, impact on students abstract thinking skills, and publication and property right resulted. In Spring semester 2019, the authors conducted these steps and produced a pocket of PicsAR booklet and PicsAR with an application based on Android (see Figure 3). Then, the trial of application was conducted to 33 students in private high school in Surabaya, Indonesia. The process of developing PicsAR in atomic model Based on the explanation in method, the research on the development of learning media based on augmented reality to practice abstract thinking skills has been carried out using the ADDIE research design (Analysis, Design, Development, Implementation, and Evaluation). The following is an explanation of the stages above.
Analysis: At this stage an analysis of the gap between expectations and actual facts is carried out. The hope is that there is a hand-out book or booklet that is able to display 3-dimensional objects so students will understand more, and develop learning media based on augmented reality. But in reality, there is currently no hand-out that has not been able to display 3-dimensional objects directly, and the use of augmented reality-based learning media is still rarely used. In addition, the pre-research questionnaire was also completed for 33 students of grade eleven of a private high school in Surabaya, Indonesia. Based on the results of the preliminary study, it was found that as many as 91% of students had difficulty with the concept of atomic models. They argued that atomic model is one of the physical concepts that is difficult to observe directly in physical phenomena. And also, the use of instructional media in classrooms still tends to use blackboards and textbooks.
Design: Before making augmented reality application, first the authors make a design and concept of media based on the results of the analysis that has been done. Next, we start making augmented media reality that was approved at seminar process. In addition, at this stage, a learning tool design with Basic Competence (BC 3.10 and BC 4.10) of grade eleven of Indonesian curriculum, includes syllabus, lesson plans, worksheets, and hand-outs which will be validated at a later stage and make research instruments to be used.
Development: At this stage the augmented reality media and the learning tools that have been completed still have to go through the validation stage, which later there will still be input or suggestions from two lecturers and one validator teacher before the augmented reality media and learning tools are used during the learning process. The results of the validation of augmented reality media will be discussed in the subchapter of the research results, while the results of the validation of the learning tools will be attached to the attachment page.
Implementation: During the implementation phase, trials of augmented reality media and validated learning tools were tested. This trial was conducted on students by using a one shot-case study trial design, where students were treated by providing learning using augmented reality media on atomic model's material. When learning takes place, the learning steps are observed by three observers from a school teacher and two pre-service physics teachers from a public teaching university in Surabaya by using a learning implementation sheet, which later on the results of these observations will be used to determine the practicality of the augmented reality media. After being given treatment then given a test in the form of abstract thinking skills of students to find out whether the media is augmented reality. In addition, students were asked to fill in the questionnaire responses to the use of augmented reality media. The results of students' abstract thinking skills tests and the results of student responses will be used to find out which augmented reality media is effective or not to be used. The results of the implementation of learning, abstract thinking skills tests, and student responses will be discussed further in the sub-chapter of the practicality and effectiveness discussion.
Evaluation: The results of the evaluation phase are to assess the feasibility of augmented reality media in terms of "validity, practicality, and effectiveness" [3,23]. According to Riduwan [18], if the results of validity, practicality, and practicality of the media get a percentage of ≥ 61% with a good category, then it can be said that augmented reality media is feasible to use [18].

The performance of PicsAR in atomic models
In fact, the researchers developed PicsAR of atomic models included Dalton, Thomson, Rutherford, Bohr, and quantum mechanics models. In this description, the authors sample to the two atomic models: Bohr model and quantum mechanics model due to its accuracy in performing atomic models.
Atomic model of Bohr with PicsAR: Bohr supposed that electrons in atoms move in circular orbits around the nucleus, but this assumption poses a problem. According to classical physics, charged particles (such as electrons) that move in a circle continuously will lose energy. Because electrons lose energy, they will move into the positively charged nucleus. But the atom remains stable. Bohr solved this problem in a way that was almost the same as that of Planck on the problem of the nature of radiation emitted by hot objects. Bohr assumed that the applicable physical laws were inadequate to describe all aspects of the atom.
Subsequently, Bohr adopted Planck's idea that energy is quantized. Based on his model, Bohr postulates three essential points:

a)
Only a certain set of orbits is allowed for one electron in a hydrogen atom. This orbit is known as the stationary (fixed) state of the electron and is a circular path around the nucleus. b) As long as the electrons are in a stationary path, the electron energy remains so that no energy in the form of radiation is emitted or absorbed. c) Electrons can only move from one stationary path to another stationary path. At this transition, a certain amount of energy is involved, the magnitude of which is in accordance with the Planck equation [19].  According to the uncertainty principle, if we know the momentum of the electron with high accuracy, it will be difficult to determine the position of the electron. In the quantum mechanical model, the location of electrons cannot be explained so simply, in contrast to the Bohr atomic model where electrons move in orbit around the nucleus.
If talking about the probability that the electron will be in a certain position. As it turns out, the square of the wave function (ψ2) at a certain point in space represents the probability that electrons will be found at that location. For this reason (ψ2) is called "probability density" or "electron density". One way to represent the probability of finding electrons around the atom is to find where the density of points that represents the probability of finding electrons. Areas with high point densities correspond to relatively large values for (ψ2) and therefore regions where there is a high probability of finding electrons. Based on this representation, the atomic model of quantum mechanics is often depicted with an atom consisting of a nucleus and surrounded by a cloud of electrons [19]. The summary of atomic model of quantum mechanics and its' AR performance is illustrated in Figure 6.

The performance of the students' abstract thinking skills
The main target of the use of PicsAR in physics learning is increasing students' abstract thinking skills. In simple criteria, if the percentage of abstract thinking skills reaches percentage more than 61%, then the PicsAR application can be identified as an effective learning media [3,8]. The indicators abstract thinking skills were assessed including "proportional, probabilistic, combinatorial, and correlational reasoning" [3]. The detail information of students' performance of their abstract thinking skills is illustrated on Table 1.
Based on Table 1, the students' abstract thinking skills indicated six essential points: 1. The students' proportional thinking skill achieved 100% of good and very good category. 2. The students' probabilistic thinking skill achieved 66.67% of good and very good category.
3. The students' combinatorial thinking skill achieved 75.75% of good and very good category. 4. The students' correlational thinking skill achieved 66.67% of good and very good category. 5. The order of students' abstract thinking skill from highest to lowest is proportional, combinatorial, probabilistic, and correlational. The comparison among them is illustrated in Figure 7 and Figure 8. 6. The overall students' abstract thinking skills reached 77.28% of good and very good category and fulfill the criteria 61% [3,8].

Discussion and Conclusion
Parameters of evaluation included the quality of PicsAR, impact on students abstract thinking skills, and publication and property right resulted. The results indicated (1) the process of developing AR in atomic model fulfil the criteria of product quality: validity, practicality, and effectiveness [23], (2) performing of students' abstract thinking skills (proportional, probabilistic, combinatorial, and correlation reasoning) reached at least 66.67 out of 100 point of all reasoning categories, (3) through PicsAR research project resulted two prior publications and one property right [25][26]. The summary of the evaluation results is illustrated in Table 2. The PicsAR fulfil the criteria of validity, practicality, and effectiveness. 2

Students' abstract thinking skills
If the percentage of abstract thinking skills reaches percentage of ≥ 61%, then the PicsAR can be said to be effective for use [22,25,26] Students' abstract thinking skills (proportional, probabilistic, combinatorial, and correlation reasoning) reached at least 66.67% in the combination of good and very good categories. 3 Outcome of research project A number of publication and property right The project resulted 2 journal articles and one property right [25][26].
All types of abstract thinking skills could be trained by the implementation of augmented reality in the atomic model. This study confirms the success of the previous research such as [4,8,9,10,11,13,14,16,21,24], suggesting the use of AR in the learning process, especially of abstract concepts. Proportional thinking skill is the ability to transfer proportional reasoning from two dimensions to three dimensions. Meanwhile, probabilistic thinking skill is the ability to use the information to decide whether a conclusion might be right or likely not correct. On the other hand, correlational thinking skill is the ability to use data to determine the strength of the reciprocal relationship or inverse relationship between the variables reviewed with other variables. Then, combinatorial thinking skill is the ability to use a combination of factor that might be related to the problem.
Based on the three parameters above, the conclusions of the research are: 1. The PicsAR research project of the atomic model was developed fulfil the criteria of product quality: validity, practicality, and effectiveness. 2. The students' abstract thinking skills indicated at least 66.67% of students performing the combination of good and very good categories of all reasoning categories, 3. The PicsAR research project resulted in two kinds of research outcomes: two prior publications and one property right.
The atomic model is one of abstract physics concept representative in presenting the use of augmented reality in physics learning, therefore, the recommendation of this research is another abstract physics concepts should address the use of AR as a media for learning.