The Enhancement of OSH Training with an Augmented Reality-Based App

kaalif@unimas.my Abstract — Recent advances in training approaches and technology have identified the need to improve training in the field of occupational safety and health (OSH). While many researchers have examined the use of augmented reality (AR) and technology-based classrooms to enhance conventional training practices, quantitative evidence for their effectiveness and motivational impact remains scarce. To contribute to filling this research gap, an AR-based application was developed to enhance OSH training. The enhanced training session was conducted in a closed, safe environment with ten participants with no background in OSH. Pre-test, post-test, and instructional material motivation surveys (IMMS) using a five-point Likert scale were deployed to measure the effectiveness and motivational impact in accordance with the respective ARCS factors. The empir-ical results obtained show that participants performed significantly better in the post-tests than in the pre-tests (mean = 14.60, s = 1.71, t(9) = – 12.11 with p-value < 0.001). The IMMS results show that the perceived motivation arising from the AR-based application resulted in the highest mean score for the satisfaction factor (4.80) followed by the relevance (4.65), confidence (4.60), and attention (4.55) factors. The results of the evaluation suggest that the AR-based application boosted participants’ active learning behavior, engagement, and interest during the enhanced training session and produced a better learning outcome and experience.


Introduction
Occupational accidents are unpredictable and unintentional events that may cause harm to the equipment, and injury or death to employees. Training is one method used to minimize the number of occupational accidents [1]. Safety training is highly important as part of creating a climate of safety. For example, in Malaysia, organizations require workers to attend safety training, especially those in high-risk and hazardous occupations [2]. With the outbreak of the global pandemic, occupational safety issues have been further challenged [3]. This further magnifies the significance of an efficient, timely, and affordable occupational safety and health (OSH) training program. Thus, the central problem to be highlighted is the inefficiency of current OSH training practices, though the subject matter is highly relevant to all members of an institution [2]. Although the maintenance of safety and health is the utmost priority, employees and operatives in the actual working environment are still not well-prepared by the traditional training delivery method [4]. Hence there is a pressing need to try a different approach to seeking improvements to the current training system and methodology to establish a safer and healthier workplace environment [5]. Based on this need, the problem can be further elaborated into two sub-problems. Firstly, the conventional training delivery method used in current practices is ineffective [6]. It has been reported that the number of OSH training sessions conducted is insufficient [7]. Given the situation, the lack of training and its ineffectiveness will have future negative impacts. Intervention and support for the improvement of the training methodology are imperative to overcome this issue. Secondly, the lack of interest among institutional members toward OSH contributes to the motivational issues they face when required to attend related training [8]. As OSH training is regarded as dull and stagnant despite its importance [9], there is a need for this training to be better implemented [10], which would increase workers' and employees' motivation to learn. Hence, under these circumstances, the following research questions were established: 1. How can OSH training be improved with technology? 2. What are the effects of enhanced OSH training on trainees' performances? 3. What are the impacts of enhanced OSH training on trainees' motivation?
Thus, this paper reports the research work based on the hypothesis that the enhancement of OSH training using AR technology has significant effects and impacts on trainees' academic performances and motivation. Thus, the following research objectives were established: 1. Enhance OSH training using an AR-based application. 2. Investigate the effects of the enhanced OSH training program on trainees' performances. 3. Measure the impacts of the enhanced OSH training program on trainees' motivation.

2
Background study

OSH training in organizations
Training intervention is considered a method of risk control designed to improve workers' knowledge, skills, and attitude concerning workplace hazards and safe, healthy working practices. Under the hierarchy of risk control, training intervention is categorized as a form of 'administrative control' used to supplement other more reliable risk control methods, listed as elimination, substitution, and engineering control [11]. Training intervention is commonly deemed less effective because it relies on the individual's capability and willingness to retain, maintain, and practice what they have learned in the classroom. Its vulnerability is associated with the human natural tendency to be error-prone (i.e., be subject to slips, lapses, mistakes and violations) that affects everyone in the training process (the training program developer, the trainers and the learners). Training intervention should be used as a risk control measure in addition to other more reliable methods, not as the sole form of intervention [12], to protect workers from workplace hazards.
Nevertheless, OSH training intervention is increasingly popular in the work environment. In Malaysia alone, RM700 million was spent on training fees in 2019, an increase of 8% from the previous year, as reported by the Human Resource Development Fund [13]. Meanwhile, the National Institute of Occupational Safety and Health (NIOSH), the biggest provider of occupational safety-and health-related training in Malaysia, trained on average 126,000 workers annually between 2013 and 2016 [14]. This statistic revealed that tremendous effort has been made by stakeholders to put the workforce on the training roster while demonstrating the importance of training to the country's economy and the development of the nation's most important assets -its people.
OSH training in an organization is traditionally conducted in the classroom. Generally, the two main categories of OSH training are competency training and non-competency training (generic). The former is largely based on the requirements set by national legislation while the latter is based on specific workplace learning needs. The law only recognizes an individual as a 'competent person' when they have undergone certified competency training and passed the necessary examinations [15]. This arrangement is highly regulated by countries to ensure that a 'competent person' is of the highest quality and provides the best services. In high-risk industries, such as the oil and gas and petrochemical sectors, competency training requirements are not set by the country's legislation but rather by the industry itself. Industrial associations collectively set the standards of training in terms of duration, content, delivery, assessment, physical resources and other requirements [16]. In both cases, training is usually conducted at the training provider's premises to facilitate the monitoring of compliance to standards.
Non-competency OSH training, on the other hand, is rather loose in terms of its specification and depends on the training provider to meet the demands of the open market. It can be conducted either at the training provider's or the client's premises. The duration, content, and delivery of this form of training are either fixed (generic) or customized to the client's needs. In an organization with highly skilled workers, training is conducted internally with no dependence on third parties whatsoever. Instructor-led training is the most popular approach, and the quality of delivery is dependent on the trainer's level of competency and years of experience [17].
OSH training centres on enabling workers to recognize known hazards and apply suitable methods for protection. However, worker education prepares an individual to deal with potential hazards or unforeseen problems, with guidance provided to become better informed and take actions that focus on eliminating the hazard [18]. It is imperative to enhance the knowledge, skills, attitudes, and habits of people involved in order to prevent accidents. Several researchers have outlined the effectiveness of OSH training using the quantitative method, qualitative method and meta-analysis. Findings tend to conclude that OSH training has significantly improved employees' knowledge, skills and attitudes in terms of OSH implementation [2]. It is important to note that better safety training has stronger associations with awareness of active methods, such as workshop and external training courses; such courses seem more effective than the passive method [19]. For instance, farmers easily understood the practical element of a training program that involved finding solutions for risk prevention [20].

OSH training with technology
Nowadays, a growing amount of workplace safety and health training is conducted online. Trainers and instructors have moved away from live seminars and toward selfpaced videos to deliver content. The teacher or instructor pre-records the recordings. Aryal et al. [21] asserted that one strength of their training program, Promoting U through Safety and Health (PUSH), is the online interactive content. This is an important method of disseminating knowledge to the younger generations, who are the target audience for their training. Additionally, online education can be used to facilitate sensemaking and social learning [22]. It seems certain that online training is the only alternative to face-to-face training, especially since the outbreak of COVID-19.
Generally, the idea of using Industrial Revolution 4.0 (or IR4.0) technology has been recommended for some time. Lee [23] reported that the future looked bright for augmented reality as a tool for information delivery and knowledge transfer in OSH training. The interest generated in industries and research publications in the education and training fields highlight the positive development of the use of this technology. An investigation was conducted on migrant workers' levels of satisfaction with e-learningbased OSH [24]. The results obtained from this study showed positive feedbacks in terms of the motivation to learn, learners-instructors interaction, the learning environment and previous experience related to e-learning. This signifies that technology in education may enhance the OSH training experience.
AR technology has been widely used in a variety of areas. They include training in the automobile [25], aerospace [26], and medical fields [27], to name just a few. Hence, the use of augmented reality for workplace safety training has become widespread. However, as AR has been used by many organizations, they are not without any challenges and deficiencies remain [28]. Thus, appropriate AR-based training content has become a key prerequisite. It would help organizations to improve the effectiveness of their staff training. However, according to [29], the creation of serious augmented reality games through the implementation of AR, such as those for fire safety, enables the development of dynamically enhanced visualizations of workplace scenarios for the purpose of acquiring risk-prevention awareness in safety training. Hasanzadeh et al. [30] also demonstrated that a mixed-reality environment is an appropriate medium for inducing behavioral changes under a variety of experimental conditions. For instance, workers' risk-taking actions can be assessed in a risk-free environment.
In the OSH field, researchers proposed a framework for adapting mobile-based virtual reality (VR) and augmented reality to enhance the traditional form of construction safety education via experiential learning [31]. A prototype was developed and the evaluation of the results showed that the proposed system was effective in improving access to safety information and transferring safety knowledge. The authors were convinced that the integration of VR and AR can alleviate the limitations of current forms of OSH training and enhance workers' levels of safety competency. Pereira et al. [32] reported that applying panoramic AR as a highly engaging OSH training environment enhanced the hazard-recognition skills in real construction projects. Immersive visualizations of hazardous situations in a real environment context allowed trainees to evaluate the possibility of injuries before starting an activity. In addition, creating a learner-centric exploration of the construction environment, promoting high-level interaction with the job site, and providing a channel for feedback on hazard-recognition progress enhanced the hazard identification skills of the trainees. The current module included some question samples about the plant which participants could answer while the plant environment was simulated. Future studies might include certain real-life situations in a virtual environment, which would help participants learn how to react [33] in a given situation.
Meanwhile, a gamification-based approach adapts and integrates game mechanics and ludic methodology into a more serious training setting. According to a review of the literature, gamification is increasingly being used to educate OSH training participants about different forms of workplace safety and risk. For example, research by Vigoroso et al. [34] demonstrated that gamification, as a method of delivering training, has been successfully implemented in safety, health, and rehabilitation. Shirokov [35] argued that a high volume of occupational safety and health training material can increase the cognitive load on training participants. Thus, gamification-based training is a way to make 'heavy' learning material 'lighter'. In another study, Cerezo-Narváez et al. [36] used the Lego® Serious Play® (LSP) technique to develop a more enjoyable gamification-based training approach. They contended that LSP could be used to elicit more complex ideas from participants, thus enhancing training competencies in industrial risk prevention. Indeed, embedding gamification elements in serious OSH training has become a current trend. However, in terms of the creation of serious augmented reality games, such as those for fire safety, Somerkoski et al. [37] stated that the authentic learning experience determines whether an augmented reality game is suitable for serious training or purely recreational purposes.
The background study above highlights the limitless potential of adopting and implementing technology-based classrooms for the benefit of OSH training. Among the listed measures, AR-based applications are highly desirable and recommended in the OSH field.

Methodology
In this study, the design-based research (DBR) methodology was implemented to achieve the research objectives.

Analysis
For the purpose of identifying enhancements to educational or training practices, Zabala-Vargas et al. [38] recommended DBR, as this methodology is systematic and flexible. The main output for this research is the learners' (i.e., the participants') academic performance, whereas the secondary output is their motivation. The needs analysis was performed with OSH experts to identify the most suitable content to be developed and applied in an enhanced training session to test the research hypothesis.

Design
The participants in this project were ten people from a non-OSH background (male = 4, female = 6, mean age = 23.5). In addition, the participants were selected from among university organizational members. Their non-OSH background is important as OSH is an essential form of knowledge for all workers in all industries. With no prior knowledge of OSH, the enhanced training session would be able to highlight the improvised methodology in terms of its effectiveness and motivational impact. Furthermore, the enhanced training session was conducted in a safe, face-to-face environment, along with the use of an AR-based application. This application was used as the main learning tool for the participants in the process of identifying and understanding hazards.

Development
An AR-based app is designed as a tool to help participants learn hazard identification; particularly utilized is marker-based AR. With this app, participants were given the opportunity to install the app on their smartphones. Their devices were used throughout the entire training session to firstly identify, and then understand, the concept of chemical and mechanical hazards. Stickers with related hazard symbols were used as the markers for the app. The app features and requirements were provided and reviewed by OSH experts. Vuforia software was used to technically develop the app.

Implementation
The enhanced training was conducted with an initial briefing and introduction to the topic, hazard identification. Then, a pre-test was held to assess the participants' prior knowledge of hazard identification at the workplace. Next, the participants were introduced to the AR-based application, the set-up procedures and the app usage. The training period focused more on how the participants would utilize the application to scan the markers provided and understand the hazard descriptions. The participants were shown the industrial equipment and laboratory chemicals represented as the hazard symbols in the markers. At the end of the training session, the participants underwent a post-test and completed a survey.

Testing
A pre-test and a post-test were conducted to measure the effectiveness of the gamified OSH training. There were 18 questions in the tests, addressing the important aspects of hazard identification. For the motivational impact measurement, a survey consisting of a set of questions was designed based on the Instructional Materials Motivation Survey (IMMS). The IMMS is deployed to assess the motivational characteristics of instructional materials or courses using the attention, relevance, confidence, and satisfaction (ARCS) model of motivation [39]. This model evaluates the strength of feeling following the completion of the learning task and training. In this study, ten statements and a five-point Likert scale were prepared, addressing the four ARCS components that were adopted and modified from the original IMMS.

Results
The results section is divided into two sections, which are the effectiveness test and the motivational impact. The aim of this study was to enhance the effectiveness of hazard identification training with the usage of an AR-based application, and the training enhancement was evaluated in terms of its effectiveness and motivational impact. The effectiveness test results were obtained from a paired t-test for the difference of means between the pre-test and post-test marks achieved by the participants. Meanwhile, the motivational impact was obtained from the mean and standard deviation of the ARCS component, evaluated from the statement scores in the IMMS.

4.1
Effectiveness test Table 1 shows the results of the pre-test and post-test for the ten participants in the training. Table 1 shows the results (in the form of marks obtained) of the pre-test and post-test of the ten students selected. It is observable that the marks range for the participants in the pre-test was 8 to 13 out of 18 questions, with a mean and standard deviation of 10.90 and 1.52, respectively. The participants' results show an improvement in the post-test, where the marks range is 12 to 17 out of 18 questions. The mean and standard deviation for the post-test were 14.6 and 1.712, respectively.
The paired t-test for the difference of means between the pre-test and post-test marks was conducted (see Table 2), with the null hypothesis being there is no significant increase in mean marks between the pre-test and post-test, and the alternative hypothesis being there is a significant increase in mean marks between the pre-test and post-test. The test was conducted using MS Excel to obtain the values, as shown in Table 2. Based on the results from the sample obtained (n = 10), there was an increase in mean marks from mean = 10.90 (s = 1.17) to mean = 14.60 (s = 1.71), and the increase is significant with t(9) = -12.11 with p-value < 0.001. This indicates that the application of AR technology was able to enhance the OSH training and improve the teaching-learning efficiency.   Table 3 shows the scores of each statement, mean, and standard deviation for the IMMS. In the IMMS deployed, statements 8 and 9 measured the attention factor. Statements 1, 4, and 6 measured the relevance factor; statements 2, 7, and 10 measured the confidence factor; and finally, the satisfaction was measured by statements 3 and 5. The attention factor refers to the learner's personal interest in the training session and the AR-based application used. The relevance factor shows the usefulness of the training and the application in relating the content to authentic situations and experiences. The confidence factor focuses on the development of individual self-belief in being able to meet the expectations of the training content. The satisfaction factor reveals the level of contentment felt after completing the training and achieving a good performance in the post-test. The mean and standard deviation for each ARCS component is tabulated in Table 4. Based on the results, the mean scores for all the components in the ARCS model (attention: mean = 4.55, s = 0.51; relevance: mean = 4.65, s = 0.59; confidence: mean = 4.60, s = 0.52; satisfaction: mean = 4.80, s = 0.41) for the participants' motivation was above the average and positively high. These results indicate that the enhanced OSH training contributed positively to the participants' interest in obtaining the knowledge and achieving the learning outcomes.

Effectiveness of the enhanced training
Based on the results of the pre-test and post-test, AR was shown to enhance the effectiveness of training, as AR has been one of the nine pillars of industry in Industrial Revolution 4.0 (IR4.0) that is getting more attention and gaining in popularity in the educational [40] and training sectors [41]. AR has been proven to have the capacity to appeal to its audiences, which will lead to an increase in learner attentiveness [42]. Moreover, AR provided better learning experiences as the participants' learning engagement levels are thoroughly enhanced [43]. AR can impose digital information in addition to real, physical surroundings, thus deepening the process of active learning [44]. According to Koenig et al. [45], one open research challenge in designing mixed-reality technology-based training is how to replicate the training environment as closely as possible to the real world.
Furthermore, AR also enhances the learning experience in the training by allowing participants to develop spontaneous learning behavior such as peer discussion and preemptive studying, as observed by the trainer during the training session [46]. In addition, AR has continually been reported to exert a more positive influence on learning than conventional methods [47]. Another observation made was that AR helps to facilitate an interactive, learner-centric learning environment among the learners, with minimal trainer intervention [48]. Hasanzadeh et al. [30] also demonstrated that a mixedreality environment is an appropriate medium for inducing behavioral changes under a variety of experimental conditions, for instance, in assessing workers' risk-taking actions in a risk-free environment. Thus, the findings in this study are in line with the observations obtained by Corvino [49] in highlighting the strength and benefits of AR in the OSH training field.

Motivational impact on participants
In terms of motivation, the high scores in all four components of the ARCS model obtained via the IMMS indicate that the enhanced OSH training using the AR-based app could hold trainees' attention, establish the relevance of materials, boost trainee confidence, and make their learning achievements more satisfactory [50]. This was because all factors were highly fostered with the usage of AR technology in the enhanced training [51]. Furthermore, the results obtained from the IMMS suggest that the participants felt that AR increased the experience of engagement. It made them feel totally immersed and they required higher amounts of concentration, leading to a more motivating training session [52] [53]. Moreover, AR components lead to the specific topic delivered, and the enhanced training more generally, being more fun and attractive in terms of its appeal [54], especially when diverse content is used [55].

Limitation of the AR-based application
Nonetheless, some limitations remain in implementing the enhanced training that utilizes augmented reality. One limitation is the poor development of the visual content due to the lack of actual footage of the exact machinery, which meant a lack of sophistication and complexity. Certain parts of the augmented reality model that were created for the experiment in this study were less realistic in terms of their elaboration, compared to the authentic pictures or videos of the actual accident scenes and preventative actions.

Conclusion
In summary, this paper has provided evidence that the enhancement of occupational and safety training using augmented reality technology has significant effects and impacts on trainees' academic performances and motivation. In particular, the enhanced training stimulates a method of active training that allows more successful knowledge retention and lifelong learning. This is especially relevant in the context of the COVID-19 global pandemic, in which occupational safety issues are further challenged by the inefficiency of training practices, resulting in the lack of employee readiness to operate in a real working environment. Overall, the enhanced training helped improve training efficiency and boost participant interest levels. This work also suggests that future related research might examine any correlation between the training methods and the actual accident prevention actions undertaken by the participants involved.