Road Safety Education Courseware: A Study of Satisfaction and Learning Performance among Primary School Students in Malaysia

There is a significant increase in road accident statistics in Malaysia and this reflects the culture formed among users related to road safety. In developing a positive road safety culture, road safety education should be introduced from an early age. The effort by the Ministry of Education Malaysia in implementing road safety education in primary schools since 2007 is a wise move towards the formation of that culture. As the road system becomes busier and more complex, children need to be educated with sufficient knowledge and skills to cope with increasingly challenging road situations. This paper introduced the ROSE courseware which incorporates multimedia, AR and VR technologies for the purpose of assisting teachers and students to understand and acquire skills related to road safety. A study was conducted among 30 primary school students in using the ROSE courseware for road safety education. The study provides an insight into the relationship between perceived ease of use, perceived usefulness, and enjoyment and students’ satisfaction in using the ROSE courseware utilising Pearson correlation and regression analyses. Regression analysis resultsindicate the significant relationships between perceived usefulness and enjoyment and students’ satisfaction in using the ROSE courseware. However, the relationship between perceived ease of use and satisfaction is notsignificant.Using paired sample ttest, this study also looked at the students' learning performance due to the intervention of the ROSE courseware for RSE. The outcomes showed that the mean scoredifference between the pre-test and post-test is significant. By using the ROSE courseware, thestudents gained significantly higher level of road safety knowledge which indicates that the learning performance among the participants has been enhanced through the implementation of the ROSE courseware. Keywords—Road Safety Education, ROSE courseware, Multimedia, Augmented Reality, Virtual Reality 48 http://www.i-jim.org Paper—Road Safety Education Courseware: A Study of Satisfaction and Learning Performance...


Introduction
In Malaysia, the statistics of road accidents show that the number of road accidents hasincreasedfrom 373,071 in 2005 to 533,875 in 2017 [27]. From 1974 to 2010, road accidents had been increasing at an average rate of 9.01% per annum in Malaysia [20]. Malaysia is estimated to have more than 20 road accidents deaths per 100,000 people in 2020 [20]. Human error is a major cause of a large number of these accidents which can be attributed to irresponsible and aggressive behaviour among road users in Malaysia [26]. Casualties can be reduced if road users have a welldeveloped road safety culture while they are young. The Malaysian government takes these accident statistics seriously and to address the road safety issue holistically, the government has taken several measures which include education, whereby the road safety education (RSE) program has been introduced in 2007 [60] and implemented in stages in the primary schools starting with year one [1]. RSE not only allows students to understand the road safety and regulations, but also to enable the students to experience the road environment in order to make RSE more effective. Besides, children need RSE-related knowledge as well as advanced cognitive skills to interact in road traffic situations.
At present, RSE in primary schools across Malaysia is still adopting the conventional teaching and learning method without the use of computer or mobilebased application as a supplement. Among the first computer based RSE application that has been developed for Malaysian schools is ViSTREET [13]. It is a VR-based RSE learning application for school children. Version 2.0 of Virtual Reality Modelling Language (VRML) was used to generate VR-based scenarios and every recognized skill or problem is addressed by a distinct module. Users need to find a solution to the given problems by utilizing the problem manipulation area where all the scenarios created in ViSTREET were problem-based.
This paper introduces the Road Safety Education (ROSE) courseware, an application that combines multimedia, virtual reality and augmented reality technologies in supporting teachers and students to understand and experience RSE. This courseware was developed based on the mixed learning environment whereby it includeslearning materialssuch as images, text, videos, animations, virtual environment and augmented reality scenes. The content of the ROSE courseware incorporates all the contents of the RSE skills prepared by the Ministry of Education of Malaysia.
Multimedia blends graphics, text, audio, video and animation and it is a technology that provides opportunities to escape from conventional teaching. Multimedia has been proven to be able to increase the effectiveness of learning [18], create studentcentred learning [36], improve the level of understanding [65], create fun learning atmosphere [2], enable users to observe behaviour and engage in the learning process [48] and enhance learning motivation [69]. Fig. 1 shows some snapshots of multimedia related learning materials available in the ROSE courseware. Virtual reality enables users to experience the sensation of presence in different physical places [6]. In teaching and learning in schools, VR is able to provide a new way of reaching more students [40][56] [5], facilitates themin experiencing learning [63] [38] [61] [7] and allows them to practice risk-free procedures through simulations [57] [54] [33] [32]. For ROSE courseware, VR has been used to simulate zebra and pedestrian bridge crossings as shown in Fig. 2. Finally, augmented reality (AR) allows digital objects to be overlayed onto the real environment allowing users to interact with them [37]. In learning, AR provides many benefits which include; enhancing fun and entertainment, [21] [44], in situ interactive visualization [4] [50] [29], and improved learning performance and motivation [21] [9] [62] [10] [8] [55]. For ROSE courseware, marker-based AR has been implemented and it has been used for the purpose of teaching students about road signs. Fig. 3 shows samples of the road sign markers and a video related to the stop sign. User and learning performance evaluations have been conducted among 30 students of a primary school. User evaluation was conducted in determining the students' perception towards using the ROSE course in terms of perceived ease of use, perceived usefulness, enjoyment and satisfaction and also to determine the relationships between perceived ease of use, perceived usefulness, and enjoyment (independent variables) and satisfaction (dependent variable). Meanwhile, learning performance evaluation was conducted in determining the achievement of the students with and without the intervention of the ROSE courseware.

Research model
The ROSE courseware is intended in providing information about RSE interactively utilizing multimedia, AR and VR to primary school students. Thus, it is important to understand the relations between the variables whenusing the ROSE coursewarefor RSE. The model for this study is shown in Fig.4.

Research hypotheses
The following hypotheses have been formulated for this study. Hypothesis1: There is a relationship between Perceived Ease of Use and Satisfaction.
Hypothesis2: There is a relationship between Perceived Usefulness and Satisfaction.
Hypothesis3: There is a relationship between Enjoyment and Satisfaction.
Hypothesis4: There is a significant difference between the pre-test and post-test mean scores.

Research design
Since this study involves user and learning performance evaluations, it has been designed to utilise hypotheses testing, correlational and one group pre-test and posttest. Quantitative data has been utilised in this study through the use of questionnaires for the user evaluation and RSE-related questions for the learning performance evaluation. The user evaluation determined the participants' perceptions towards using the ROSE courseware and also the relationships between the independent variables and the dependant variable among a sample of 30 primary school students. The learning performance evaluation was carried outin determining the students' learning performance with and without the intervention of the ROSE courseware.

Participants
Both evaluations have been conducted among a sample of 30 students of a primary school in Sintok, Kedah, Malaysia. The sample has been selected utilizing purposive sampling, a techniqueof taking sample based on criteria set by the researchers [59]. The number of samples was 30% out of 100 which was the total number of year four students in this school and this fulfills the minimum number of samplesas suggested by [14].

Measurements
The measurements that were used in the user evaluation include perceived ease of use, perceived usefulness, enjoyment and satisfaction. Perceived ease of use is the degree in which one thinks that using technology will be effortless [16]. According to the Technology Acceptance Model, one of the determinants of one's intention to use an applicationis perceived ease of use [17]. Since the ROSE courseware incorporates multimedia, AR and VR technologies, perceived ease of useis vital to the primary school students so that they are able to use the ROSE courseware easily. Perceived usefulness is one's level of confidence by using certain technology will improve one's performance [66]. [16] concurs with [66] by defining perceived usefulness as the degree where one believes that utilising a technology will improve his/her performance. The ROSE courseware can only be accepted by the primary school students when they perceive that its use is going to enhance their performance in learning about road safety. Enjoyment is defined as the extent to which an activity is considered as giving pleasure and joy in its rights, apart from the consequences of performance [67]. Enjoyment is a good feeling that can reduce tension and increase learning motivation [66]. Satisfaction is defined as the users' response after using a system which makes the users feel positive [47]. Satisfaction in learning is achieved when the students' needs and desires are met in the learning process [31]. Based on previous studies, multimedia, AR and VR are able to provide satisfaction [

Instrument
There are two types of instruments used in this study namely; user evaluation instrument and learning performance instrument. The user evaluation instrument is a set of questionnaires which comprises of measurements that include; perceived ease of use, perceived usefulness, enjoyment and satisfaction. The items for perceived ease of use were cited from [25], perceived usefulness were cited from [16], enjoyment were cited from [66] and lastly satisfaction were cited from [12]. The user evaluation instrument comprises of demographic data of participants and participant's perceptions towards using the ROSE courseware. It uses a five-point Likert scale ranging from 1 to 5 (strongly disagree -strongly agree). Meanwhile, the learning performance instrument consists of a set of 15 multiple choice questions related to the RSE contents.

Procedure
All the 30 participants were gathered in a computer lab. Firstly, the learning performance instruments were distributed among the participants. A predetermined identification has been assigned to each participant's learning performance instrument for the purpose of data analysis and also to ensure that all the pre-test data was collected anonymously. The participants were given 30 minutes in answering the questions. Then, the participants were given 10 minutes to rest before proceeding with a short briefing on the function and interfaces of the ROSE courseware. Next, they were given an hour to be familiar with using the ROSE courseware. Later, the participants were handed with a set of questionnaires and they have to complete the user evaluation within 30 minutes. Once finished, they were given an hour to rest. For the final task, they were asked to return to the lab to complete the learning performance evaluation. The participants were again handed the learning performance instrument and they were again allocated 30 minutes to answer the questions. All the post-test data was again collected anonymously. Table 1 shows the participants' demographic data. The participantscomprised of 13 males and 17 females. They were 10 years old. The participants comprised of 25 Malays, 3 Chinese, 2 Indians, and 1 Siamese.

Validity and reliability
Since all the measurements and items were adapted from previous studies, they have undergone the proper validation process and are deemed valid. Reliability analysis was performed in ensuring the items' stability or consistency of an instrument [58]. SPSS version 22.0 was used to calculate the Cronbach alpha for all the measurements. Based on Table 2, the Cronbach alphas forperceived ease of use, perceived usefulness, enjoyment and satisfaction are 0.789, 0.719, 0.747, and 0.835 respectively. They are considered reliable sincethe Cronbach alphas for all measurementsare greater than 0.7 [46] [23].  Table 3 shows the mean for all the measurements and items. A5-point Likert scale was utilised in the instrument for this study to determine the mean. Since Likert scale is an ordinal scale and it should be converted to numerical scale before interpreting the mean as recommended by [52]. In converting from Likert scale to numerical scale, Sugiyono equation [64] was applied. The numerical scale has to be reclassified into strongly disagree, disagree, agree and strongly agree. The Sugiyono equation is as follows: RS = (m -n) / b = (5-1)/4 = 1 Note: RS -Score range m -highestpoint on Likert scale n -lowestpoint on Likert scale b -classification number The result from the equation was then used to produce a numerical scale consisting of strongly disagree with mean ranges from 1 -1.99, disagree with mean ranges from 2 -2.99, agree with mean ranges from 3 -3.99, and strongly agree with mean ranges from 4 -5. The results of the descriptive statistics analysis based on the numerical scale showed that perceived ease of use has mean of 4.21 (strongly agree), perceived usefulness has mean of 4.24 (strongly agree), enjoyment has mean of 4.15 (strongly agree), and satisfaction has mean of 4.22 (strongly agree).

Correlation
The correlation analysis is used in determiningthe relationship between the independent and dependant variables. Table 4 shows the outcome of the Pearson correlation analysis which shows that thereare positive correlations between all the independent variables and the dependent variable. The correlation valuesfor perceived ease of use, perceived usefulness, and enjoyment are .954, .949, and .928 respectively. Since the values are greater than 0.70, the correlation among the variables is very strong [49].

Regression
The purpose of the regression analysis is to estimate the relationships between the independent and the dependent variablesin order to verify the hypotheses and the results of the regression analysis are shown in Table 5. The R 2 value is 0.976 which reflects the students' satisfaction towards the use of the ROSE courseware. The independent variables are the predictors in this analysis. The R 2 value for this multiple regression is 0.976 which means that 98% of satisfaction variation is related to variation in perceived ease of use, perceived usefulness and enjoyment; leaving 2% unexplained. According to [15], the acceptable p-value should not more than 0.05 and the t-value should be greater than 1.645.

Learning performance evaluation
A one group pre-test and post-test was implemented to observe the participants' learning performance due to the intervention of the ROSE Courseware for RSE. The questions for this evaluation were based on the RSE contents. Fifteen multiple choice questions were distributed among 30 students and a predetermined identification has been assigned to each respondent's questionnaire booklet for the purpose of data analysis. The students were allocated 30 minutes to answer the questions. All the data was collected anonymously. In assessing the learning performance of the respondents, paired-sample t-test was applied. The pre-test and post-test mean scores which represent the participants' learning performances are shown in Table 6. The mean scores increased from 17.97 to 23.20. This indicates that the students were able to improve their learning performance and perform better compared to the previous test. The questionsfor the pre and posttestswerethe same; however, the questions' order has been changed.  Table 7 presentsthe paired-sample t-test results. The pre-test mean score represents the participants' background knowledge pertaining to RSE which has been gained from various sources including newspapers, social media, documentaries, YouTube, TV, magazines, internet, and many others. The post-test mean score represents the participants' learning performance through their background knowledge coupled with knowledge gained through the use of the ROSE software. Since the significant value is 0.000 which is significant at 0.01, the difference between the participants' pre-test and post-test mean scores is significant.By comparing the outcomes of pre-test and post-test, participants who used the ROSE courseware gained significantly higher level of knowledge. Generally, the learning performance for the RSE among the participants could be enhanced through the implementation of the ROSE courseware.

Hypothesis testing
Hypothesis testing has been conducted to examine the relationships between the independent variables and satisfaction. For that purpose, the following hypotheses have been formulated.
Hypothesis1: There is a relationship between Perceived Ease of Use and Satisfaction.
Hypothesis2: There is a relationship between Perceived Usefulness and Satisfaction.
Hypothesis3: There is a relationship between Enjoyment and Satisfaction.
Hypothesis4: There is a significant difference between the pre-test and post-test mean scores.
As hypothesized in H1, there is a relationship betweenperceived ease of use and satisfaction. As shown in Table 5, perceived ease of use has nosignificant relationship with satisfaction with p = 0.06. Since the p-value is higher than 0.05, the hypothesis is rejected. As hypothesized in H2, there is a relationship between perceived usefulness and satisfaction. Table 5 shows that perceived usefulness has significant relationship with satisfaction with p = 0.04. Since the p-value is 0.04 which is smaller than 0.05, the hypothesis is supported. As hypothesized in H3, thereis a relationship between enjoyment and satisfaction in using the ROSE courseware. Table 5 shows that enjoyment has significant relationship with satisfaction with p = 0.03. Since the pvalue is 0.03 which is smaller than 0.05, the hypothesis is supported. As hypothesized in H4, there is a significant difference between the pre-test and post-test mean scores. Based on Table 7, thep-value is 0.000 which indicates that this hypothesis is supported.

Conclusion
Road accident statistics continue to show an increase in terms of deaths and injuries among road users in Malaysia. Therefore, road users need to be educated from an early age in relation to road safety rules and regulations. Nowadays, children need to be inculcated with sufficient knowledge and skills in coping with the complex road situations so that they can develop a road safety culture within themselves in preparation for today's increasingly challenging road traffic spectrum. This paper introduced the ROSE courseware which incorporates multimedia, AR and VR technologies for the purpose of assisting teachers and students to understand and acquire skills related to RSE. This paper also discusses the participants' perceptions whereby they strongly agreed on the perceived ease of use, perceived usefulness, enjoyment and satisfaction of using the ROSE courseware. It also gives an overview of the relationship between the independent variables and satisfaction in using the ROSE courseware among the students. The results support empirically the statistically significant relationships between perceived usefulness and enjoyment and satisfaction in using the ROSE courseware. Previous empirical studies have shown that perceived usefulness is one of the key determinants of the use of a particular technology [22] [45]. Nevertheless, the outcome of this study indicates that the relationship between perceived ease of use and satisfaction is not significant. The results of thisstudy are similar to the studies by [51] and [22], where AR and VR are not acknowledged by new users as easy to use since they need to be familiar and comfortable with the new learning environment. In order to observe the performance of students in learning with and without using the ROSE courseware, a one group pre-test and post-test was conducted. The results of the paired sample t-test showed that the difference in the participants' mean scores between the pre-test and post-test is significant. The participants who have used the ROSE courseware gained significantly higher level of RSE knowledge which indicates that the learning performance among the participants has been enhanced through the implementation of the ROSE courseware. The ROSE courseware is not intended to replace the current approach of teaching RSE in schools. Instead, it is just an additional learning tool that can be used by teachers and students inside or outside the classroom to increase RSE-related understanding. It not only provides information that can improve knowledge but also can improve students' skills through the utilisation of AR and VR technologies.

7 Authors
Abdul Nasir Zulkifli is an Associate Professor at the School of Creative Industry Management & Performing Arts, Universiti Utara Malaysia. He has been teaching in various fields including Mechanical Engineering, Manufacturing Engineering, CAD, CAM, Management of Technology, Multimedia, Virtual Reality and 3D Animation for the last 34 years. His research areas include Virtual Reality, Augmented Reality, and Mobile applications in training and education. Email: abdulzulir@yandex.com Nur Fadziana Faisal Mohamed is currently a senior lecturer at the School of Multimedia Technology and Communication, Universiti Utara Malaysia. She has won gold medals for innovative products in numerous exhibitions at national and international levels. Her research interests include health behaviour change support systems, virtual / augmented reality, interaction design development and health informatics. Email: fadziana@uum.edu.my Mustafa Moosa Qasim is a lecturer in the Computer Science Department at the University of Basrah, Iraq. He has a bachelor degree in Computer Science, a master's degree in Information Technology and Ph.D. in Information Technology. His research focus is in the field of Software Engineering, specializing in software architecture. He