Components of Adaptive Augmented Reality Model for Heritage Mobile Application

— Adaptive augmented reality (AAR) is the latest concept of the augmented reality that responds and adapts to a real-time context and the characteristics of the users. This concept is said to provide adaptation of 3D augmented reality, better engagement and more emotional experience to the users. In AAR, the information is provided to the users through interaction tracking and sensor on the device. However, this technology is still in its infancy. Therefore, a study to explore positive ways of providing relevant information that can enhance user experience is urgently required. The objective of this paper is to identify the components of an AAR model for heritage mobile application. It is hypothesised that such model would enhance user experience and provide better engagement. Consequently, models from past studies on Augmented Reality, UX Design and storytelling are reviewed. Findings indicate that personalisation and emotion are among the important components. Inclusion of these components seems to allow users to be more engaged and inspired to learn about the cultural heritage, hence enhances user experience.


Introduction
Augmented Reality (AR) has lately been used in a variety of industries and contexts [1], [2]. It enables users to engage with computer-generated information while viewing the actual environment [3], [4] using smartphone, Head Mounded Display unit, or see-through glass devices. Nowadays, advancements in mobile phone technology allow AR technology to be better integrated into mobile phone. AR application also can digitally preserve the intangible and tangible heritage and cultural items [5]. Signage is typically used to provide information and a tale in a museum or historic site. However, AR is now feasible to provide more effective ways to experience the stories by "living through" them digitally [6]. For example, Liestøl [7] applied storytelling in Augmented Reality to recreate the historical event of the DDay assault on Omaha Beach. The AR application may be used in a variety of settings. AR applications must be able to adapt to our continuously changing real-world environment. AR should develop into an intelligent device capable of detecting the user's preferences and surroundings [8]. Future AR should feature seamless registration, rich content, and an adaptive user interface [9]. The adaptive process in a mobile adaptive interface can occur in the context of the user, the environment, and the device itself [10], [11].
Since this technology is still in its infancy, research to investigate positive methods of giving relevant information that might increase user experiences is urgently needed [1], [12]. The AAR concept is to provide the user with a more adaptive and engaging augmented reality system. It could assist museum visitors in becoming more engaged in the exhibition or artefact of their choice [13]. Personalization and emotion are among crucial elements of digital storytelling. The purpose of using the element of personal and emotion engagement in a digital story is to immerse the listeners into the story [14], [15]. AR technology combined with digital storytelling elements is perceived as a beneficial combination in presenting cultural heritage content to improve user experience.

2
Literature review

Augmented reality
Augmented Reality (AR) is overlaying virtual content (text, audio, video, 3D object) over a real-world view (through a monitor, cell phone, or head-mounted display) without replacing the real environment [16], [17]. Azuma et al. [16] identified these three features in the AR system: 1. real and virtual objects merged in a real environment, 2. Interactive and real-time, 3. Alignment of the virtual and real object. By agreeing with Azuma, [18] describe an AR system as "one that combines real and computer-generated information in a real environment, interactively and in real-time, and that aligns virtual objects with physical ones." AR is "an interactive, real-time direct or indirect view of a physical environment that has been enhanced by the superimposing of computer-generated sensory information, such as images, sounds, videos, and haptics." [19]. These definitions have linked AR to terms like interactive, real-time, and merging with virtual contents to enrich the information of the physical world. The size of AR gadgets, such as mobile phones and Google Glass, have shrunk as technology has advanced. AR was only commonly used in the military and aircraft industry before being made available to the general public. When adaptive been featured in AR, it provides additional benefits when applied in a variety of fields.

Adaptive augmented reality
Adaptive Augmented Reality (AAR) is an augmented reality application that provides valuable and effective real-time information based on the user's unique characteristics, interests, and context [13], [20]. [10] study AAR in the conception of plasticity in HCI. They are trying to produce an AR application that able to "foster functional ability, ease of use and portability". The AAR concept here is applied in "size of augmentations, the illumination level of the scene and ambient noise". The application able to adapt the size of the augmented object is adjusting accordingly base on the distance, tune the illumination on the scene to suit on the lighting of the environment and leveling the sound depend on the noise around the area. The adaptability in AR is similarly discussed by [9], addressing that in the future AR should be able to adjust the 3D object's contrast accordingly to the environment lighting level. Mobile phones now have built-in sensors that help improve adaptability in mobile AR. Mobile phone sensors can be used to acquire data about the user, the environment, and the device. The sensor input can happen explicitly or implicitly. According to [11], explicit modalities are methods by which the system receives input from users through gesture, speech, touch, or other means. Implicit modalities are information of environment changes (such as temperature, noise, light) and changes of the device (such as battery, orientation, time). For example, a user's current location could trigger POI based on the GSP sensor. As for the environment in low-light conditions, the screen illuminance will be increased to provide a clearer image on the screen. The AR display screen can be adjusted based on the mobile device's orientation (either landscape or portrait) in terms of the device. [21] conducted research to improve the information offered to museum visitors based on their psychological state during their visit. Sensors are used to monitor to be tailored to the visitor's level of engagement and interest. Their work has been a game-changer for AAR in the field of heritage.
AAR for culture heritage. The ARtSENSE project is one research conducted on AAR in the context of heritage [22]. This research consists of five academic researchers (from various fields), two from industries, and three representatives of museum from France, Spain, and United Kingdom. This research they are developed an AR guide to assist visitors in art museums. The main objective of this research is to prove an AR guide that can enhance the visiting experience by providing augmented content based on users' interests. The prototype was developed to provide visitors with multimedia information based on the interpretation of three sensors: visual, audio, and psychophysiological. In this way, the information presented to the visitor is custom to each individual. The system suggested the suitable contents to the user accordingly based on the data collected captured from the sensors. More related multimedia information is presented when visitors show signs of appeal; else, an alternative will be prompted based on the preference. This project claims not just to enhance the visiting experience of the museum visitor. However, it also a tool to collect and provide data on cognitive and learning in informal learning environments is experienced [13].

UX design process model
Hammady et al. [23] presented a User Experience (UX) model of marker-less augmented reality application implemented at the museum in England and Egypt. The model is a framework for developing augmented reality (AR) apps for museums to offer the best possible user experience while visiting the museum. As shown in Figure 1, this model comprises three key phases: requirements, design, and evaluation. The requirements phase is the initial stage of that model. This phase is critical for gathering essential information such as the application's objective, identifying the room size, getting relevant and appropriate content, developing and designing skills, and, last but not least, defining UX aspects.
The following step is the design phase. This stage focuses on attaining the desired impact of the user experience based on the requirements specified in the previous stage. First, understand the primary function and methods for achieving it. The platform, UI elements, scenario, and events included in the application are then identified. The third phase is to brainstorming for the elements mentioned in the previous stage. In phase four, designing, the ideation is put into action. The user interface (UI), buttons, interactions, and theme creative are designed with the user experience (UX) in mind. The prototype was then tested for flaws and errors at the end of this step. The evaluation step is the model's last stage. Experts were involved in evaluating and commenting on the prototype, which used qualitative and quantitative methods. Finally, in this study, the UX design process model can be applied to construct the prototype. The UX aspects will be crucial components in compelling the user to use the AR application in the historical site.

Digital storytelling core elements
The digital storytelling conceptual model comprises core elements as a guideline for producing digital storytelling (DST), which can be interactive or non-interactive [24]. As depicted in Figure 2, the core elements are divided into four distinct clusters (construction, appeal, origin, and interaction), and each element plays a significant part in both types of storytelling. Considering that storytelling in AR also can be interactive or non-interactive, the conceptual model is considered in this study. There are eight elements in the Construction cluster: minimal, perspective, dramatic question, significant content, story map, soundtrack, tempo, and articulation. The elements in this cluster are related to the content creation of the digital story.
Essentially, minimal in DST recommend that a story not be too long by composing the crucial content of the story and dismissing the irrelevant. Perspective is about the attempt of DST to convey a certain message from the standpoint, whether in first person or third person viewpoint. Dramatic question is an element about establishing a question that creates suspense in the digital storytelling to keep the audience's attention from the beginning until the story finishes. Significant content in DST refers to the story as the primary focus, with multimedia being used to highlight the story's essence and substance. The story map indicates that a DST should have the classic arc of storytelling: beginning, middle, and closure and the elements of tension, conflict, and resolution. The soundtrack element suggests a sensible and cautious use of music as a supportive element in digital storytelling since music can evoke audiences' emotions.
The element tempo talks about the rhythm and beat of the story, which might impact the audience's interest and retention of the digital story. Lastly, articulation is the concern that using the digital storyteller's voice in telling the story would have a more significant influence on the DST. Appeal was designed to impact viewers' experience toward the digital story, and there are three elements: personal, expression, and engagement. The element personal in DST suggested that the story should be a personal tale of the storyteller in order for people to feel affection for his life. Next, expression is mainly concerned with transmitting messages or information in particular DST situations without narration. Alternatively, symbolism and metaphor can be employed to be interpreted by the audience. Then, engagement is the element of engaging and capacity of a story to captivate an audience through emotion or content, to the point that the story feels real to them. Element of emotion is a factor that encourages users to engage with the application, resulting in increased trust and enjoyment from using it [25].
The next cluster is Origin that focuses on elements that establish the story. Two elements in the cluster are intention and the element 'user contribution' belonging to the Interaction cluster. The element intention means a digital story begins with a purpose by the digital storyteller, which will be disclosed at the finale. In contrast, the element user contribution allows the audience to participate in the DST content development process.
The other element in Interaction cluster is collaboration. Both elements in this cluster imply that the audience interacts with digital storytelling by giving input that helps shape the story. Comparably, collaboration is more about interacting with other users to create and experience digital storytelling.

AAR core components
Shiratuddin and Tenh [26] examined several previous works on adaptive augmented reality to establish the adaptive concept's core knowledge elements. Components of such a concept were gathered and offered a formal definition using model analysis, covering four models.These four models are A2R Museum Augmentation Model by Damala and Stojanovic [13], Conceptual framework for Adaptive Multimodal Interfaces in Mobile Augmented Reality by Abidin, Arshad, and Shukri [11], Conceptual framework for Adaptive Augmented Reality by Tenemaza et al. [20] and last but not least, i-ARA Adaptive Augmented Reality by Hervás et al. [27].
As shown in Table 1, user profiles, media, interaction, environment data, and device components are all included in each AR model. All of the previously stated models include the User Profile and Environment elements as adaptive augmented reality components. User information (such as age, gender, and height) and physiological condition are acquired to identify the users' characteristics or interests. The augmented content is typically based on the characteristics of the multimedia contents (images, videos, animations, text, audio comments, sounds, and 3D objects). Depending on the device used to interact with the content, interaction can take the form of gesture, word, touch, or sight. To enable this, the majority of models make use of the device's sensors. In the case of a mobile phone, sensor data (camera, GPS, compass, and other) was utilised to adapt relevant information about an intriguing object, location, or even a person. The core components of adaptive apps are the environment (EV) and user personalization (UP).
EV refers to changes in the visual, audio, and interactivity elements of the content being displayed. The user model focuses on UP, which involves sensing users' data to react to EV and pre-planned scenarios. Adaptivity can be applied with users' data such as age, gender, and height. Other components (OC) are defined by the service or device layers. Result of analyzing these four models, the researchers proposed the core elements of Adaptive Augmented reality as: & context in EV refers to how creative the story telling is applied

Method
In this study, models from past studies on Augmented Reality, UX Design and Storytelling are scrutinised in order to identify the components that can enhance user experience and provide better engagement. Through model analysis, three past models were considered: AAR core components, DST core elements, and UX Design. Components from these models were aligned to be composed as one.
Summary. As depicted in Figure 3, the EV component for AAR is closely linked to the elements of DST in Interaction cluster, Appeal cluster, and Construction cluster. The EV components are dealing with the visual, audio, and interactivity elements of the content being displayed; while the three elements in cluster of appeal is to give impact the experience of viewers toward digital story, and the elements in the cluster of construction is related to the content creation of the digital story. Furthermore, in the context of DST, interactive elements meant the audience could interact and provide input to shape the story. User-content creation is one of the interactions that users can perform in mobile AR [28]. The user can view or interact with the AR content and takes on the role of creator to create virtual information. The user can be a creator of augmented reality material at the location and then post his or her information to the public using a mobile phone. As a result, users can collaborate with other users to build up details about their present location. On the other hand, the User Experience design model proposed by Hammady et al. [23] is adapted for designing the prototype or the actual application.

Conclusion
AAR is a relatively new concept that has evolved from prior AR technologies. The emergent of AAR offer the opportunity to create more powerful augmented reality application where by the appropriate content will be display based on characteristic of users and real-time context. The objective of this paper is to identify the components of an AAR model that could enhance the user experience and engagement by including the elements of digital storytelling and adapting the UX design model for augmented reality. As a result, models from previous research on Augmented Reality, UX Design, and DST are examined. Hopefully by including these components could formulate a model that help heighten the engagement and user experience. On the other hand, this model is an initial model that not yet been tested empirically which can be done to testify proposed hypothesis. articles, modules, book chapters and monographs. She has served at least 50 times as the External Examiners to PhD and MSc students. She has also successfully supervised 100 postgraduate and undergraduate students, and taught Research Methodology, Usability Engineering, Internet Applications and Media Design. She has served in more than 50 committees and journal editorial boards. She can be contacted at shuhada@ uum.edu.my