Paper—Prototyping and Usability Evaluation of Road Safety Education Courseware for Primary Schools... Prototyping and Usability Evaluation of Road Safety Education Courseware for Primary Schools in Malaysia

The increasing number of road accidents and deaths among children in Malaysia is a concern. One of the main causes stems from lack of knowledge and skills in dealing with the complexity and condition of the road system. Recognizing the significance of road safety among students, road safety education has been introduced to primary schoolsby the Ministry of Education in 2008. Initially it was blended into the Malay language subject with the goal to equip the children withthe knowledge and skills about road safety. However, road safety teaching and learning still uses the conventional methods without the use of specialized tools. This paper introduces ROSE courseware, a specialized learning tool developed to meet the needs of the Malaysian primary school curriculum for road safety education. Its learning environment is a blend of virtual reality, augmented reality and interactive multimedia, which enables children to understand and acquire skills related to road safety through interactive, real-time and immersive learning tool. In order to receive the children’s feedbacks pertaining to their experience in interacting with the ROSE courseware, a usability evaluation was conducted. The results show that the children strongly agreed on the usefulness, information quality and interface quality of the ROSE courseware. Keywords—Road Safety Education, Virtual Environment, Augmented Reality, Multimedia, Prototyping, Usability


Introduction
Children are exposed to various accidents while on the road since they have to interact with other road users who use the road for a variety of purposes [9][21] [43][44][45]. The risk of children being involved in road accidents is higher due to the road conditions [3], complexity of the transport systems, interactions between children and other road users as well as children's interactions with the physical and social environment [53]. In addition, physical and cognitive limitations also put children at high risk of road accidents [9]. Researchers such as [72] [68] and [61] highlightthat childrenhave physical limitation in their ability to see or be seen by other road users.
In Malaysia, statistics of fatal accidents involving children are alarming because of higher rates of deaths among children. Between 2007 and 2009, 12% of children between the ages of 1 and 18 suffered fatal road accidents in Malaysia [43]. This number exceeds the 5-10% deaths among the high-income countries [74]. Human error is a major cause of a large number of these accidents which can be attributed to irresponsible and aggressive behavior among road users in Malaysia [27]. Casualties can be reduced if road users have a well-developed road safety culture while they are young. Children need RSE-related knowledge as well as advanced cognitive skills to interact in road traffic situations.
In Malaysia, the RSE program has been introduced in 2007 [64] and in 2008 and implemented in stages in the primary schools starting with year one [2]. Initially, the RSE isblended into the Malay language subject and the lessons areconducted according to the teacher's preparationwhich includes using exercise books, short videos and posters and are carried out without practical training. [2] RSE not only allows students to understand the road safety and regulations, but also to enable the students to experience the road environment in order to make RSE more effective. At present, RSE in primary schools across Malaysia is still adopting the conventional teaching and learning method without the use of computer or mobile-based application as a supplement. Therefore, this paper introduces Road Safety Education (ROSE) courseware, an application that combines multimedia, virtual reality and augmented reality technologies in supporting teachers and students to understand and experience RSE.
Multimedia is defined as more than one medium consisting of a combination of graphics, text, audio, video and animation [39]. Multimedia is a tool that provides a wide range of learning opportunities, thus escaping from the conventional and confining form of teaching [76]. Multimedia is able to provide variety of resources in the control of teachers and students [1]. For teaching and learning, multimedia has been proven to be able to improve the level of understanding [70], increase the effectiveness of learning [20], create student-centered learning [35], enhance learning motivation [73], and create fun learning atmosphere [5].
Meanwhile, virtual reality is a 3D environment that is computer simulated or imagined which allows users to experience the sensation of presence in different physical places [11]. It is acomputer-generated environment in which pictures, textures, sounds and other realistic sensations areused to simulate the user's physical presence in a virtual environment [40]. VR provides many distinctive benefits when it is used in teaching and learning in schools. VR offers new tools for educators and provides new ways to reach more students [41][60] [10]. VR can stimulate, motivate and enhance the students for specificpurposesandat the same time facilitating students to experience learning [69] [38][65] [12]. VR allows students to practice risk-free procedures through simulations [57][33] [62]. Finally, augmented reality is defined as a technology where computer-generated objects are added to the real-world environment to create a new layer to the environment where users can interact with [36]. Virtual reality differs from augmented realityin the sense that its environment is made up of synthetic objects that replace the real objects while for augmented reality; its environment is a combination of real objects superimposed with virtual objects in order to enhance the environment.

2
Prototyping of the Rose Courseware The prototyping of the ROSE courseware allows developers to discover and use the guidelines and steps as discussed in this paper in producing a supplementary active learning material for Road Safety learning. Prototyping is crucial since it involves the process of developing a new product. Prototyping of the ROSE courseware consists of three stages namely; virtual environment (VE) authoring, augmented reality (AR) authoring and multimedia (MM) authoring as shown in Fig. 1. VE Authoring: VE authoring consists of four phases which include; scene definition, object modeling , optional database conversion and VE development. The scene definition is to ensure that the courseware specifications are properly set. In this phase, all contents related to the RSE were gathered. The scenes basically introduced the road environment to the students. Other materials such as graphics, 3D visualization scenes, information, images, and maps were gathered from various sources. In this courseware, a zebra and a pedestrian bridge have been selected. A zebra crossing consists of parallelwhite stripes uniformly painted on road's surfaces [4] while a pedestrian bridge is a separate crossing that is built over the road [30]. The use of zebra crossing and pedestrian bridge is sometimes dangerous for children if they do not have the skills and knowledge to use it for crossing. Thus, the ROSE courseware should allow the children to practice and consequently learn how to cross the road using the zebra crossing and pedestrian bridge.
The object modeling phase involvesgeometric modeling where polygons were shaped to suit the need of the scene. Next, they were textured to make them look more realistic. The objects were then animated so that they move according to the needs in the scene. Autodesk 3DS Max was used for both objects geometric modeling and animation. The scene involves road, zebra crossing, pedestrian bridge, school buildings, vehicles and pedestrians. Fig.2 shows some of the 3D models that were used for the RSE learning environment.

Fig. 2. 3D models used in the ROSE courseware
When modeling for virtual environment, it is important to consider the computer capabilities in terms of graphic and random access memory (RAM). These determine the smoothness of the navigation whenever the user navigates the VE of the ROSE courseware. All the 3D models have to be developed with minimum polygon count in order to minimize the storage space and consequently avoiding lag in the VE [75].
Converting the files format phase involved conversion of file format prior to VE delopment phase. Since Quest3D comes with a number of directly usable 3D objects, in this study the models were converted to the Microsoft DirectX (.x) format. The final phase is the VE development where Quest3D was used.
Meanwhile, Fig.3 and 4 show the real scene and virtual scene of the zebra crossing and pedestrian bridge together with step-by-step instructions. Augmented Reality Authoring: AR authoring involves three phases namely; define marker, add virtual content, and AR environment. BuildAR which is a marker-based AR was used in developing the AR application. Sensors such as camera or webcam detects the marker and displays the virtual content. In define markerphase, road signs were used as markers. Figure 5 shows some of the markers that were used for the ROSE courseware. In add virtual content phase, videos were added as virtual contents for the AR application. Researchers such as [23] [19] have encouraged the use of video since it is the most suitable form of multimedia element to convey information to low-literacy populations such as children. Video allows children to observe and imitate the behavior tasks or actions [49]. Fig.6 shows a video explaining about the purpose of a stop sign. Once all the markers have been defined and the virtual contents have been added and attached to the specific markers, the final phase is the AR environment. Meanwhile, all the markers consisting of road signs were produced in the form of cards so that the users can use them to view the virtual contents attached to the specific markers. Multimedia Authoring: Multimedia authoringconsists of five phases namely; acquire media, digitized media, edit media, import media and add interactivity. In acquire media phase, all the contentsin the form of text, picture, sound and video related to RSE were gathered from teachers, supplementary sources and internet. Next, the contents were digitized for easy storage in digital formats. Pictures were saved in JPG format, audios in mp3 format and videos in mp4 format. In edit the media phase, the digitized media were enhanced using Adobe Photoshop for graphics, Audacity for sound and Adobe Premier for videos. Next in the import media phase, all the digitized media were imported into Adobe Flash to add special effects to improve the look and behavior. Figure 7 shows a scene in the ROSE courseware that utilizes video. The final prominent phase is add interactivity which permits the users to interact with the courseware. It consists of incorporating interactive features such as buttons and links to the courseware. Fig. 8 shows the Main page of the ROSE courseware that consists of interactive buttons and links. Package Application: Package Application involves integrating VE, AR, and MM into Adobe Flash to produce the ROSE courseware. Adobe Flash is a multi-platform application for advanced interactive content [29]. Adobe Flash has seamlessly integrated the VE application, AR application and MM application to form the ROSE courseware. Fig.9 shows the pages that provide the links to the VE and AR applications. Testing and Debugging: Any new courseware needs to go through testing and debugging process whether a commercial product or an enterprise or personal application program. Testing involves identifying errors, which when removed will improve the quality and reliability of the courseware. Meanwhile, debugging involves locating and fixing faults (or bugs) in the courseware. For ROSE courseware, testing all the VE, AR and MM applications is important to ensure that the courseware is functioning well without any error. The following section discusses on the usability evaluation that has been conducted among a sample of target users.

Usability Evaluation
Usability evaluation is considered part of the software development process [52]. Usability evaluation is a systematic process of data collection to better understand the users and how the product is being used in performing certain task under stated conditions [55]. Usability evaluation refers to the level of user's experience and satisfaction when interacting with this courseware. It is about the usefulness of the courseware, the quality of the information provided and the quality of interfaces. It provides the result for the overall satisfaction of the user. This will provide answers regarding whether the courseware was designed and developed according to the users' requirements towards the learning process. Also, it would ensure that the courseware credibility is previously established and to validate it as a reliable application for RSE based on the curriculum. Usability evaluation data on a particular learning application is essential in ensuring and proving that any introduced solution has reached its objective or not.
Sample: For this study, a sample of 30 year four students was chosen using the purposive sampling. It is a technique of taking sample based on criteria set by researchers [63]. The number of samples was 30% from the totalyear four students in this school which is 103. The number of samples fulfills the minimum number of samples of 30 as suggested by [16]. The usability evaluation was conducted in a primary school in Sintok, Kedah, Malaysia.
Instrument: For the evaluation, the instrument used was Post-Study System Usability (PSSUQ) questionnaire. PSSUQ covers three major dimensions in determining the usability aspects of a system namely; system usefulness, information quality, and interface quality [34] as shown in Fig. 10. System usefulness covers the attributes of how the user manages his/her tasks well and addresses the troubles or pleasures related to the use [47] and uses ease of use, learnability and satisfaction attributes. Ease of use is the level in which a personaccepts that using a specific system will be free from physical and mental effort [18]. Learnability is when new users initiate effective interactions and then reach maximum performance levels [51]. Lastly, satisfaction describes a user's subjective response when using the product. Meanwhile, information quality refers to the substance of the messages and the syntactic relationships between them [34]. It uses attributes that include; functionality, outcome/future use and errors/system reliability. Functionality is the user's belief concerning the adequacy of the features of a product from the opinion of an imagined demanding user [31]. Outcome/future use is the level in which a personaccepts that his/her work will be improved [18]. Errors/system reliability is the system's ability to recover from errors, or the existence of serious errors [59]. Lastly, interface quality measures the surface level presentation and physical input/output components such as the readability of characters and graphics on the screen as well as the properties of the buttons such as size, location, and feedback [31]. The instrument uses a 5-point Likert scale labelled with 1-strongly disagree, 2-disagree, 3not sure, 4-agree and 5-strongly agree.

Fig. 10. Usability Measurements and Attributes Relationship
Procedure: Prior to the evaluation, the ROSE courseware has been installed on 31 PCs equipped with webcam located in a computer lab. All the participants were briefed on the use of the courseware and also the purpose of the evaluation. Special attention was given on the use of the VE and AR applications of the courseware. The participants were given approximately 30 minutes to be familiar with the interfaces and functionality of the courseware. Next, another 30 minutes were given to them in answering the questionnaire.

Results
Demographic Characteristic: Table 1 shows the participant's demographic data whereby 13 male and 17 female participants participated in the evaluation. The participants comprised of 25 Malays, 3 Chinese, 2 Indians, and 1 from other races. Reliability: Reliability is the consistency of a questionnaire in achieving the same results [42]. Cronbach alpha (α) was calculated using SPSS 22.0 software in establishing the reliability between the data item which evaluates the level of internal consistency between various estimations of a measurement. According to [26] and [48], the minimum value of α should be 0.7. The Cronbach alpha scores for all measurements are shown in Table 2 which indicated that α for all measurements are greater than 0.7 and they are reliable.

Descriptive Statistics
The purpose of descriptive statistics is to summarize the data so that they can be clearly presented [50] [25]. Descriptive statistics reflect the participants' perceptions towards the use of the ROSE courseware through the mean scores for all measurements. Table 3 shows the results for System Usefulness, Information Quality, and Interface Quality. Conversion from Ordinal to Numerical Scale: The Likert scale used in the questionnaire is based on ordinal scale [28] and it is not suitable to assess the degree of participants' agreement or disagreement with a statement [63]. This means that Likert scale can only rate or rank the responsesand the range between responses is not quantifiable. Meanwhile, numerical scale takes into account therange between responses and categorized them as strongly disagree, disagree, agree and strongly agree with no neutral position. Hence, numerical scale has been used to determine the level of participants' agreement or disagreement toward a statement as recommended by [56]. In convertingthe scale, the equation RS = (m -n) / b was used, where; RS is the score range, m is the highest score on scale, n is the lowest score on scale, and b is the number of group. The score range is 1 and the numerical scale based on the categories is shown in Table 4. The descriptive statistics results of Table 3 show that the mean score for System Usefulnessis 4.26, Information Qualityis 4.19, and Interface Qualityis 4.24. Since all the measurements have mean scores of 4 -5, all of them are categorized as strongly agree according to Table 4. This proves that the children strongly agreed on System Usefulness, Information Quality, and Interface Quality of the ROSE courseware. System Usefulness has thehighest mean score of 4.26 and Information Qualityhas the lowest mean score of 4.19.

Conclusion
Road accident statistics continue to show high mortality and injury among children in Malaysia. Children need to acquire the necessary skills to deal with complex traffic situations so that they can form the right road safety culture in their life. Recognizing the significance of road safety among students, road safety education has been introduced to primary schools by the Ministry of Education in 2008 by incorporating into the Malay Language subject. Several previous studies have shown that one of the most effective teaching methods for road safety is to simulate almost real traffic conditions to the children. This paper discusses the prototyping as well as the usability evaluation of the ROSE courseware. The ROSE courseware has been developed to assist teachers in conducting RSE in the classroom so that the students are able to understand and acquire skills related to RSE. The courseware has been developed by combining interactive multimedia, augmented reality and virtual environment. The multimedia component provides a powerful learning tool which breaks the conventional chalk and talk form of teaching and provides the teachers with an impressive range of teaching resources. Meanwhile the virtual reality component helps users feeling the sensation of presencein a physical place with realistic images and sounds. Lastly, the augmented reality component features interactive real-time display which overlays information on users' real-world view. A usability evaluation was conducted among a sample of thirty primary school students. It was conducted for the purpose of having a better understanding of the level of user's experience and satisfaction when interacting with this courseware in terms of usefulness of the courseware, the quality of the information provided and the quality of interfaces. The results revealed that the students strongly agreed on all the three measurements. Findings from this study show a very encouraging response among children on the use of the ROSE courseware for RSE. Further studies are needed to determine the effectiveness of the ROSE software in improving children's learning performance for RSE.

6
the last 34 years. His research areas include Virtual Reality, Augmented Reality, and Mobile applications in training and education. Email: zulkifliaanasir@gmail.com Nur Fadziana Faisal Mohamed is currently a senior lecturer at the School of Multimedia Technology and Communication, Universiti Utara Malaysia. She has won gold medals for innovative products in numerous exhibitions at national and international levels. Her research interests include health behaviour change support systems, virtual/augmented reality, interaction design development and health informatics.Email: fadziana@uum.edu.my Mustafa Moosa Qasim is a lecturer in the Computer Science Department at the University of Basrah, Iraq. He has a bachelor degree in Computer Science, a master's degree in Information Technology and Ph.D. in Information Technology. His research focus is in the field of Software Engineering, specializing in software architecture. He finds himself consistently tend to the theories that investigate the behaviour, persuasion, motivation, and education, within the construct of mobile architecture and mobile application design. He has won several awards and medals through research grants. He is also considered as Ambassador in Journal Recent Patents on Computer Science from 2019 to present. His recent publications could be reached at SCOPUS ID, ORCID, Google Scholar, Researchgate, Academia, Publons. Email: mustafa.mq87@uobasrah.edu.iq Nur Afiqah Abu Bakar is an Administrative Officer at the Universiti Utara Malaysia Kuala Lumpur Campus. She has a bachelor degree and master's degree in Information Technology specializing in Multimedia. Her research areas include Multimedia, Virtual Reality and Augmented Reality.Email: nurafiqah@uum.edu.my