Tobias in the Zoo – A Serious Game for Children with Autism Spectrum Disorders

Nowadays ubiquitous technology can be a suitable way to motivate and engage children in interactive learning activities in order to promote their cognitive and social skills. Technologies, like augmented reality (AR), have the ability to catch the children’s imagination and to promote their attention, as they can experiment artificial, safe and fascinating environments. Children with autism spectrum disorders (ASD) usually have difficulty to recognize facial expressions and to understand associated emotions. We propose to design and develop an innovative GameBook to assist children with ASD to recognize and acquire emotions by engaging their attention and motivation, increasing their competence on this handicap. The GameBook will contain a story that can be read by text or listen by audio. The story will describe some scenarios and real world situations which will conduct the children to become involved on fictional contents associated with emotions. The child will have to interact with these scenarios, by playing with one 3D AR avatar with different facial expressions and choosing the correct one to the right situation and environment described in any page of the GameBook. This GameBook will promote the interaction between the child/storyteller and his/her imagination as well as will help the child to identify the correct emotional face to the situation. The GameBook can be played on any mobile device, such as a tablet, a smartphone or a laptop, with either an external web camera or an inbuilt camera. In order to test it, an exploratory study in a classroom context with ASD children will be performed. We also intend to observe the impact of the game on children interaction, as well as to quantify and evaluate their performance, assess the usability of the technology, and evaluate how it affects the child emotion reactions and the benefits it offers.

Abstract-Nowadays ubiquitous technology can be a suitable way to motivate and engage children in interactive learning activities in order to promote their cognitive and social skills.Technologies, like augmented reality (AR), have the ability to catch the children's imagination and to promote their attention, as they can experiment artificial, safe and fascinating environments.Children with autism spectrum disorders (ASD) usually have difficulty to recognize facial expressions and to understand associated emotions.We propose to design and develop an innovative GameBook to assist children with ASD to recognize and acquire emotions by engaging their attention and motivation, increasing their competence on this handicap.The GameBook will contain a story that can be read by text or listen by audio.The story will describe some scenarios and real world situations which will conduct the children to become involved on fictional contents associated with emotions.The child will have to interact with these scenarios, by playing with one 3D AR avatar with different facial expressions and choosing the correct one to the right situation and environment described in any page of the GameBook.This GameBook will promote the interaction between the child/storyteller and his/her imagination as well as will help the child to identify the correct emotional face to the situation.The GameBook can be played on any mobile device, such as a tablet, a smartphone or a laptop, with either an external web camera or an inbuilt camera.In order to test it, an exploratory study in a classroom context with ASD children will be performed.We also intend to observe the impact of the game on children interaction, as well as to quantify and evaluate their performance, assess the usability of the technology, and evaluate how it affects the child emotion reactions and the benefits it offers.

I. INTRODUCTION
Autism Spectrum Disorder (ASD) is a lifelong group of neurodevelopmental disabilities characterized by abnormalities in social interaction, communication and restrictive and repetitive behaviors [1].There are three main types of ASD, Autistic Disorder, Asperger's Disorder and Atypical Autism.Asperger's Disorder and Atypical Autism are milder forms of Autistic Disorder and produce similar features in people affected [2].The United States Centre for Disease Control (CDC) estimates that 1 in every 68 children in the U.S.A. has an ASD [3], and the number of children diagnosed with ASD is growing worldwide [4].This disability affects their development capabilities especially in areas like communication and interaction with others [5] and causes language problems such as delayed responses or very little to no speech, perseveration or echolalia.Children increasingly isolate themselves and get more comfortable doing repetitive activities or tasks alone [6].Autistic people often require on-going care and support throughout their lives.ASD children struggle with significant relationships and behavioural challenges and in most cases have serious implications for inclusion social adulthood [7].America's Autism Society has estimated that the cost for caring of an autistic child their whole life ranges between $3.5 million to $5 million [8].Within the ASD, individuals are commonly referred to as high-functioning or low-functioning depending on the degree of language delay and verbal intelligence quotient (IQ) [9].One of the most promising applications in the intervention process of ASD children came from developing tools for promoting children's social and communication skills [10].Several studies show that the majority of people with ASD exhibit a natural affinity with technology and a positive attitude towards computerbased training [11].This is primarily due to the fact that software programs offer a predictable and structured environment that can accommodate their need for organisational support and their preference for routine and repetitive behaviours [12].
There are numerous practical advantages on using technology on ASD [13].The heterogeneity of symptom severity in people with ASD can pose practical difficulties for care providers [14].Technology has the potential to provide individually tailored interventions that are suitable for a wide variety of abilities [15].Technological interventions allow its use at different speeds and locations, and never lose patience with the frequent repetition that many people with ASD desire [16].This offers the opportunity to reduce some of the crippling personnel costs associated with autism care [17].Nowadays technology is ubiquitous and can be a safe and a good way to motivate and engage children in interactive learning activities in order to promote their cognitive and social skills [18].
Technology is being used increasingly on a variety of pedagogical contexts, both as assistive technologies and as tools for helping us to understand user's motivation [19].Recent research on therapeutic technology is aimed to improve eye contact, determine facial expressions and other behaviours that impact social interaction.In fact children with ASD usually have difficulty to recognize facial expressions and to understand associated emotions, to imitate or use emotional expressions, to understand and control their own emotions, or to interpret emotions or empathy with others [20].Technologies, like augmented reality (AR), have the ability to catch the children's imag-PAPER TOBIAS IN THE ZOO -A SERIOUS GAME FOR CHILDREN WITH AUTISM SPECTRUM DISORDERS ination and to promote their attention, because they can experiment artificial, safe and fascinating environments.In 1994, [21] showed a spectrum where extremes were the known real environment and at the opposite end, the virtual environment, complemented with varying degrees of involvement of real and virtual elements.The AR is at the forefront of modern technological society development and AR applications emerge from everywhere, with applicability in various fields.
AR is a variation of Virtual Environments (VE), or virtual reality as it is more commonly called.VE technologies completely immerse the user inside a synthetic environment.While immersed the user cannot see the real world around him.In contrast, AR allows the user to see the real world with virtual objects superimposed upon or composited with the real world [22].
Other authors defined the AR systems as combining "real information and the one created by a computer (computer-generated), complementary, interactive and real-time" [23].As stated by [24], the AR has grown exponentially because it allows improving the users' understanding, knowledge and interaction with the real world.According to [25], the research and development necessary for the implementation of AR took place in the last four decades.However, only from the 90s, there was a growth and significant progress in the field of AR increasing the number of researchers in this field, especially in recent years [26].
If we look back and explore how AR began, the name of Ivan Sutherland appears with the first AR system ever created [27].The retinal image of the real objects that people see is only two-dimensional.The fundamental idea behind those system consisted on a three-dimensional display to present the user with a perspective image which changes as he/she moves.
The AR is at the forefront of modern technological society development and AR applications emerge from everywhere with applicability in various fields.According to [22] at least six classes of potential AR applications have been explored: medical visualization, maintenance and repair, annotation, robot path planning, entertainment, and military aircraft navigation and targeting.
AR is an optimal interface technology and a helpful tool to support autistic children capabilities.With AR it is possible to create more attractive and interactive interfaces that can be manipulated by hand, without using conventional peripherals such as the keyboard and the mouse [28].This characteristic of AR promotes the interaction between the child and the object, raising their interest and their curiosity in the task/activity [29].
The goal of the work presented in this paper is to design and develop an innovative GameBook to assist children with ASD in order to recognize and acquire emotions by engaging their attention and motivation, increasing their competence on those handicaps.
This paper is structured as follows: Section II presents AR games for children with ASD, Section III shows the developed GameBook and finally, Section IV presents the conclusions and future work perspectives.

II. AR GAMES FOR CHILDREN WITH ASD
This section presents some examples of available games with AR as well as the evolution tendencies.

A. Existing games with AR
Besides the proliferation of technologies that allows the development of mobile applications capable to use the AR as mean of interaction within the game, there are scarce applications that support socialization and social skills practice for children with ASD in the emotional context.However, these interactive technologies as facilitators of the act of socialization are able to provide support to children with ASD.
As an example, they could provide feedback to the children pronunciation, as happens in visiBabble [30], a system that processes infant vocalizations in real-time.The system replies to the infant's syllable-like productions with vibrantly colored animations and records the acoustic-phonetic analysis, reinforcing the production of syllabic utterances that are associated with later language and cognitive development.The VocSyl [31] is a software package that utilizes a suite of audio visualizations representing a myriad of audio features in abstract representations.These visualizations aim to provide children with language impairments a new persistent modality to experience and practice speech-language skills.Also, they could assist children with ASD on managing their schedules, like in [32] where it is presented a sociotechnical architecture composed by three components.A personal travel assistant that uses real-time Global Positioning Systems (GPS) data from the bus fleet to release just-in-time prompts; a mobile prompting client and a prompting script configuration tool for caregivers; and a monitoring system that collects real-time task status from the mobile client and alerts the support community of potential problems.
These interactive technologies can provide visual support more appealing to children with ASD, making the utilization of the application more enjoyable.In [33] are presented three prototype systems that refer the design challenges of the use of large group displays, mobile personal devices, and personal recording technologies; it explains the interventions related with these prototypes along with the results from two focus group discussions around the interventions.
More focused in the development and practicing of the children with ASD social skills have emerged other projects.Thus, multitouch displays were developed [34], where it was conducted a study with three children with autism to enhance their social skills.SIDES: Shared Interfaces to Develop Effective Social Skills [35] is a tool designed to help adolescents practice effective group work skills using a four-player cooperative computer game that runs on tabletop technology.Both, with the objective of engage children with autism in social activities and provide them the possibility to learn new forms of social expression.
Another method used to engage and encourage socialization is the storytelling in a way to promote the practice of suitable social skills, such as the StoryTable [36].
In [37] the authors present an interactive system that explores the potential of AR technology to visually conceptualize the representation of pretense within an openended play environment.Results from an empirical study involving children with ASD aged 4 to 7 demonstrated a significant improvement of pretend play in terms of frequency, duration and relevance using the AR system, in comparison to a non-computer assisted scenario.

PAPER TOBIAS IN THE ZOO -A SERIOUS GAME FOR CHILDREN WITH AUTISM SPECTRUM DISORDERS
Several mobile applications were developed for inviting and supporting socialization, such as [38], a mobile outdoor game for small groups of children aged 7-10, that was designed with the aim to encourage social interaction between the players and to promote physical activity.Another project [39] considers a mobile technology that empowers children to collaboratively read and create stories.But, none of these solutions have been used to support the socialization of children with autism.Regarding that, [40] is an Android application wireless connected to a server, designed with both individual and group features in mind, enlarging each child with autism social skills, needs, and capabilities, and empowering synchronized group interactions and shared experience.It uses the smartphone's camera to augment a real-life social situation with visual support -mimicking the Social Compass curriculum that augments social stories with paper-based visual supports.
Recently, Samsung has created "The Look at Me Project", a pilot program developed with the collaboration of a multidisciplinary team of clinical psychologists, cognitive psychologists, and psychiatrists to help ASD children.The app aims to improve individual's ability to make eye contact and keeps children motivated and highly concentrated by using the camera function of digital devices that often appeal to children's interests.It also features a point system, themed missions, various rewards, and visual or sound effects to keep children engaged.Levels can be customized based on the children's achievements.Each mission in the app requires interaction between parents and their children to encourage positive relationships and connections.The recommended time to use the app is 15-20 minutes each day.The app also gives feedback to parents on their children's daily performance through a systematic parental dashboard.The app is currently under clinical testing to verify its effectiveness [41].

B. Evolution Tendences
The exponential growth of the use and availability of smartphones and tablets, combined with the decrease in acquisition costs of these devices, as well as the increase in their processing capabilities, storage, additional sensing capabilities (i.e., accelerometers, magnetometers, and gyroscopes) and interaction with the user, makes them a suitable tool for families, caregivers and children.Simultaneously, the era of "wearable computing" is a true fact with the introduction of new equipment's for our daily use, like Google glasses or the Apple watch.
These multi-purpose devices allow its use in educational support and could combine this feature with the playful side and socialization.
At the same time, these technological platforms are more accepted, prevalent and discreet which makes them more socially acceptable.
Thus, the trend considers the features of technological platforms to create new tools, more appealing and enjoyable, to work in the development of the socialization skills in children with autism.

III. GAMEBOOK-TOOL PRESENTATION
This section presents the developed GameBook, including the methodology, the storyboard, the characters, the scenarios, the sounds, the available mini games and how it was implemented.

A. Methodology
Given the potential benefits of AR we aim to design a tool based on AR in order to improve social capabilities of children with ASD.
For the purpose of designing therapeutic and affordable games to children with ASD, a GameBook based on AR is proposed in this paper.Our goal is to create an innovative serious game based on a book style with interaction inputs to assist children with ASD with social interaction and communication skills.We aim to promote children with ASD recognizing and acquiring emotions by engaging their attention and motivation, increasing their competence on this handicap.

B. About the GameBook
The GameBook presents the story of Tobias's adventure during a visit to a zoo park (Figure 1).The player can read the story by text or listen it by audio.The story will describe five scenarios and interactions with animals and real world situations which will conduct the children to become involved on fictional contents associated with emotions.The child will have to interact on these chapters, by playing with Tobias and learning his five different facial expressions, choosing the appropriate one to each situation and environment described at any page of the GameBook.
We believe that GameBook will increase the ability of the child to interact with the story and promote his/her imagination and engaging.This game tries to help him/her to make the proper choice by selecting the correct facial emotion for a certain scenario and situation.
The GameBook can be played on any mobile technology, such as a tablet, smartphone or laptop, with either an external web camera or an inbuilt camera and is available on an online server, which can be accessed by the url: www.tobiasadventures.in/thezoo.

C. GameBook Characters
The character is the most important element that gives "life" to the game.The character Tobias was designed to captivate the children attention and to empathize with them.In the design of this avatar we try to promote a face with characteristics that allows the players to feel some attraction with him.These expressions are designed by special detail in the avatar essentially with his lips and eyes.Usually they are accompanied by body gestures consistent with emotion showed through the face in order to reinforce the intended message.
Besides, it was given a cartoonish look to Tobias (Figure 2), which was deemed by the group members as potentially more familiar and more fun to the children than a photo-realistic alternative, as was done by [10].This is also in line with recent studies reporting that individuals with ASD and their caregivers recommend that assistive software should be designed with fun in mind [42].We tried to line the Tobias' head and eyes bigger than the rest of its body in order to arouse the child's attention.The character expresses five different basic human emotions during the game, as cited on [43] repertoire that served as the basis and foundation to countless works of scientific research, namely, joy, sadness, fear, anger and disgust.

D. GameBook Scenarios
This serious game includes a total of five different scenarios, designed to involve the child on real life situations (Figure 3).All activities of the game took place on the zoo.These scenarios will promote the child's contact with different environments and will create some emotional reaction on Tobias.The child will have to recognize properly the emotion that Tobias is feeling on that context with the purpose of advance to the next scenario.

E. Text of the Story
The narration of the game story is associated to the scenarios and Tobias' trip.The fictitious text of the story will report linked events and situations of the adventure of Tobias in the Zoo.
This story follows a structured sequence that begins on: Chapter 1 -Starts with Tobias' anniversary day and the party his mother prepared to him.He was very happy because he received a gift card to visit the zoo.In this chapter we reinforce "Happy" felling's.
Chapter 2 -It is the zoo day visit.During the trip Tobias started to get angry as the trip was taking too long, because it had a long line of cars to the zoo and many traffic lights.In this chapter we reinforce "Angry" felling's.
Chapter 3 -It is about Tobias' emotions watching the zoo animals in their cages.By going in the elephant cage he was very sad because he could not touch the baby elephant trunk and he wanted to play with them.In this chapter we reinforce "Sad" felling's.
Chapter 4 -During the visit, Tobias crossed the lions' cage and got scared by the big size of lions and their big teeth.In this chapter we reinforce "Fear" felling's.
Chapter 5 -When Tobias was eating a snack of ice cream, a bird flew over the sky very close to him.Tobias dropped the ice cream and it smeared on the floor.Tobias got disgust of ice cream and did not want to eat it anymore.In this chapter we reinforce "Disgust" felling's.

F. Sounds
The game has an audio story telling feature that allows the child to listen the story by audio sound.This audio narration has a voice of a child with eight years old, Tobias' same age.The audio is synchronized with the game and starts always when the player begins a new chapter of the story.Moreover the game has some sounds of animals' voices and background music.
We believe that those features will improve the interaction of the child player at the game and will turn the game more easily to understand helping him/her to be involved in the game story.

G. Mini Games
At the end of each chapter the player has a simple quiz before proceeding to the next chapter (Figure 4).On this quiz the child is questioned about which emotion Tobias felt on the situation described before.Here he/she has to choose the proper facial expression.In order to help him/her, the game allows the child to observe Tobias avatar in 3D representation by AR.We believe that this feature will increase the attention of the child player and reinforce the memory skills of recognize facial expressions.
At the end of the book the player has a simple and funny Memory Challenge game (Figure 5).In this mini game, the child has to select the proper pair of all 5 emotions characters to finish the game.We believe that this technique is useful to children that have difficulties to remember faces, to encourage the child to observe and analyze different faces and to examine his/her facial expressions.Certainly, it will improve his/her capability to recognize some facial human emotions.This mini game is also very useful because it will help the child to remember and make associations of previous situations described during the game book narration.The child can play as many as possible (without overdoing it!), in order to obtain all correct combinations.With this consecutive process of observe Tobias' facial expressions several times, we believe that would help and improve their memory search process.

H. Implementation of the GameBook
To create this serious game, the following technologies were used: Wordpress® to develop and to construct the "home" of the game; the 3D flip book uses the HTML5/CSS3 and JavaScript/J query web languages.The game was fully responsive for best performance to adapt to any screen size device, computers, tablet´s or mobile phones.
To the development of the AR feature it was selected the ArUco® (Augmented Reality library from the University of Cordoba).This option was held on the easiness of use, reliability and cross-platform compatibility because it relies on OpenCv (Open Source Computer Vision Library), an open source computer vision and machine learning software library.Also, ArUco is open source and above all it is compatible with the Android platform.
This is an advantage because we aim to have the GameBook compatible with several devices, thus enabling the use of the GameBook by a higher number of people.Other features that were contemplated on choosing the ArUco for combining the AR in the GameBook were the easy integration with OpenGL (Open Graphics Library), the detection of markers with simple C++ code, the large number of possible different markers (up to 1024) and the detection of several markers at the same time.
This concern came from the need to make the Game-Book design and construction the easiest possible, in order to facilitate the production of a new GameBook to the development of others skills in children with autism.

I. Storyboard of the GameBook
The game has an intuitive interface for any kind of player and will simulate a real book.The child will read the narration story, listen the audio sounds, observe the images of the story and will browse to the next page.On each page the player will have to select on the quiz game the correct choice of emotion.
The game has five levels and when the player finishes the game with success he/she will have to play a final challenge, one memory game with all Tobias facial expressions presented in the game.In the end the child will receive two special bonus (Figure 6): • PRINT: The player will have the possibility of print the complete book story with cover and chapters scenarios descriptions and illustrations.• SHARE RESULTS: The player will have the possibility to share the results by several ways, for example, e-mail or social networks.

IV. CONCLUSION AND FUTURE WORK
The development of this Gamebook arises as a response to a need that was presented by the parents, caregivers and professors of ASD children.According to them and after research, the tools available to practice the child emotion reactions that use new technologies were scarce.The few existing were unattractive, with gaps or were not in the native language.
Regarding the importance of the quality of the tool (Gamebook) and its application in the educational and everyday context of the ASD children, we pretend to perform a case study.This methodology is "an empirical inquiry that investigates a contemporary phenomenon and in depth and in its real life context, especially when the boundaries between phenomenon and context are not clearly evident" [44].
The main objective of the case study is to observe the impact of the game on children interaction, as well as to quantify and evaluate their performance, assess the usability of the technology, and evaluate how it affects the child's emotion reactions and the benefits it offers.We believe that the Gamebook will be used on various contexts and institutions with children with ASD, as well as in schools with classes of students with special learning needs that integrate children with ASD and by the parents and families of ASD children.
So, for the study, we will setup a convenience nonprobabilistic sampling, where will be gathered the children with ASD from the institutions, schools and families.The timing of the study will be defined by the parents, caregivers and educators, in order to integrate the Gamebook utilization in their daily routines, with the aim of making the study the least intrusive to ASD children.However, the study was intended to be held for a minimum period of three months and a maximum of six months.As research methodology it will be used the mixed method: it will be quantitative on the data collection procedure, its representation will be arranged by graphs and tables; it will be qualitative regarding the data analysis method to be performed assessments of the results obtained comparing them with the results obtained in other studies [44].
A survey will be applied to the parents, caregivers and educators, in different moments along the time of the study, in order to collect the data obtained of applying the Gamebook in the educational context of children with ASD.In parallel, it will be held a direct observation in the natural environment that occurs the case study; this will provide useful information to add additional information on the subject or topic that will be studied [44].
The data collected will be analyzed by the statistical analysis software IBM SPSS Statistics®, which will allow conducting a quantitative and qualitative analysis on the impact in the evolution of children considering the interpretation and reaction to different emotions, through the adoption of Gamebook in their activities in educational and family contexts.
The design and development of GameBook serious game was presented.The aim of this project was to make available a cost effective tool implemented in a platform usually pleasant for children with ASD.
This is an innovative game that has some functionality as graphical interface, characters interactions, audio and text narrations and the AR technology which makes this game functional and exciting, captivating the children attention and increasing their motivation while developing social skills.
In the near future, a system that will allow monitoring the child's results and these outcomes will be available on the website of the game.This system will be synchronized with the game and it will have a login and member area to teachers and caregivers.These features will allow following the child´s evolution.We also intend to disclosure information about the project and the game download, instructions, updates and categories, on this online portal in an accessible and intuitive way.
Furthermore, new stories as for example, Tobias adventures in the Sea or in the Space, with new characters, scenarios and situations can be easily included.
It is the authors' belief that the GameBook will increase children with ASD engagement in the learning activities promoting their memory, cognitive and social skills and allowing them to reinforce facial expression recognition.

Figure 1 .
Figure 1.Icon of the Tobias Adventures Game in the Zoo (Mobile Version).

Figure 3 .
Figure 3. Interface of the GameBook, showing an example of the initial page.
PAPER TOBIAS IN THE ZOO -A SERIOUS GAME FOR CHILDREN WITH AUTISM SPECTRUM DISORDERS

Figure 4 .
Figure 4. Example of Interface of the Quiz Game.

Figure 5 .
Figure 5. Interface of the Memory Game