Design and Implementation of Interactive Mobile Application for Autistic Children in Physical Education Class

Each autistic child has different characteristics and severity of symptoms ranging from not being able to do daily routine tasks such as exercising by themselves to not being able to care for themselves and needing caretakers to look after them for most of the time. In the Autistic Research Centre, Khon Kaen University Demonstration School, Thailand, every morning, one of their routine tasks is exercise, especially on the treadmill, which can reduce their weight and improve their focus on school during the day. The aim of this research is to find video interaction in the form that encourage autistic children’s exercise on a treadmill suitable for individual child. Six interaction techniques were designed and tested. The techniques included: Face Detection (FD), Upper Body Detection (UBD), Color Detection (CD), QR Code Detection (QCD), Motion Detection (MD), and Adaptive Video Interaction (AVI). All interactions were developed on the iOS platform and tested on eight autistic children. Each child was tested at least 12 sessions on the treadmill, and performance of the six techniques were evaluated from the accuracy of application detecting the child’s presence on the treadmill compared to the actual number of times that each child disappeared. The results showed that the AVI was the most suitable technique for autistic children with an average accuracy rate of 92.31%. The study shows that the application is able to reduce the caretaker’s burden and motivate autistic children to focus while exercising. Keywords—Autistic, Video interaction, Mobile devices, Physical education class.


Introduction
Autistic Spectrum Disorder (ASD) is a group of disorder that occurs from brain abnormality causing difficulties in various developments such as language, social skills, and in having interests and behavior that are limited and repetitive. The symptoms of the disorder appear before the age of three with varying severity depending on language development, Intelligence Quotient (IQ), and other coexisting abnormalities. These difficulties are lifelong, especially for social skill development. Only 1 -2% of autistic children can live normally like others. ASD cannot be entirely cured but can part, we will discuss two topics including: autistic therapy and mobile interaction for autistic children.

Autistic therapy
Therapy helps relieve the severity of symptoms in autistic children, stimulate positive behavior, and increase the ability or develop skills in areas such as speaking, daily living, careering, etc. At present, there are many types of therapy. Apart from those according to the main therapy guidelines, one of the most popular therapies are called Alternative Therapies. The Alternative Therapies can be chosen to suit the symptoms and responses received from each child and used in conjunction with the main therapy. Examples of Alternative Therapy include art therapy, music therapy, animal therapy, and acupuncture. Such therapies use either equipment or various creatures as a medium. Art therapy uses drawing equipment, music therapy uses musical instruments, animal therapy uses dogs, cats, fish, etc., and acupuncture uses needles. Electronic devices are also used in therapy, including interactive toys [3], [4], robots [5], [6], and smartphones [7]- [12]

Mobile interactions for autistic children
Interaction with autistic children on mobile devices is retaliation or communication between children with autism and mobile devices. When compared to normal people, autistic children may only respond to some interactions. Relevant research findings reveal that three interaction patterns were applied to autistic children.
Interactive toys interaction: Interactive toys can interact with players through attractive patterns such as light, sound, and texture. In [3], interactive cubes were used as the medium for interaction between the games and the autistic children to inspire, increase curiosity, and enhance social skills. The result of the study found that applying interactive cubes can stimulate children to participate in the game longer, help to reduce repetitive behavior, and increase curiosity. The [4] aim of the game was to improve social interaction behavior of autistic children. Two games were created for comparison. The first game had no goal and did not use additional components. The second game had a goal and used smart toys. The result was that more social interaction was present in the second game than in the first.
Interactive robot interaction: A robot is a type of machine which has different structures according to its purpose. In general, robots are created to work with difficulties, dangerous situations, or when high accuracy is needed. For autistic children, robots are used to develop social and communication skills [5]. The robots will have a child-friendly appearance, such as having cartoon characters or animals that are liked by the children. This will help prevent children from being afraid and will induce them to interact more. It was also found that using robots as a substitute for teaching led to better learning in children. Nowadays, in addition to the movement of the head, arms or legs, robots can also interact with users by talking.
Camera interaction / Movement interaction: Currently, smartphones are widely used and has become an indispensable part of human daily life. Smartphones are used not just for communication but also for entertainment and learning. Various research has also used smartphones for medical use to assist in treating patients. In [10], Facial Expression Recognition and Automatic Voice Recognition techniques were applied to help children with autism learn and increase academic skills. The World of Kids system was designed to help detect user behavior on mobile devices with the purpose of identifying the emotions of mobile device users via face detection, to acquire games that are suitable for the user's mood, [11] to use the digital activity schedule manager on smartphones to help increase management skills, and to schedule activities to make autistic children able to live without having to rely on others, which leads them to have a better quality of life. And [12] use AR on mobile devices to improve the literacy in academic and learning skills for children with autism using the phonics learning method.
Camera interaction or Movement interaction uses a camera to interact with people. Kinect is mostly used in the research for autistic children, in [15]- [19], Kinect was used for interaction and was called motion-based, touchless, interaction for children with autism to practice movement skills as well as develop intellectual skills [15]. Games that focus on movement without using touch, has the potential to be used in therapy for children with autism [18]. In [19], proposing the hand-eye coordination technique in playing games to treat children with autism. In addition, Eye-Tracker goggles for detecting eye movements were used in [21] to study the behavior of children with autism.

Interaction design
The design goal is to detect whether or not the child is still on the treadmill. The presence of the child on the treadmill indicates that s/he is still running. On the other hand, his/her absence on the treadmill indicates that s/he has left. Moving equipment with multiple interactions are utilized.
Interaction design for autistic children is developed based on the children's most common behaviours which include eye contact, tactile defensiveness, low motivation, communication problems [22], and abnormalities in their behaviours such as difficulties in controlling their bodily movements, communication, emotional difficulties, and behavioural expressions. From these characteristics and behaviours, the most suitable interaction was to use external appearances and actions of the children combined with some stimuli to urge them to interact as needed. Such stimuli included food, snacks, cartoons, or anything else that the child likes [4].
This study gathered and designed six interaction techniques to interact with autistic children while exercising on a treadmill: 1) Face Detection, (FD) 2) Upper Body Detection (UBD), 3) Colour Detection (CD), 4) QR Code Detection (QCD), 5) Motion Detection (MD), and 6) Adaptive Video Interaction (AVI). Fig. 1 shows the working process of the first five techniques which were similar in process but different in how presence detection was made.
Face Detection (FD): FD used a video camera with real-time image processing to detect facial features. If the eyes and mouth were detected, it meant that the child was on the treadmill.
Upper Body Detection (UBD): UBD detected bodily parts from head to chest. This method is similar to FD but with a broader target and easier detection. When upper bodily parts were detected, it meant that the child was on the treadmill.
Colour Detection (CD): CD detected colour from the video image by retrieving still images frame by frame and getting the colour value from each pixel to compare. RGB or HSV colour code was used for comparing. If colour from the video image matched with pre-determined colour such as the colour of the child's clothes, it meant that the child was on the treadmill.
QR Code Detection (QCD): QCD detected the QR Code generated from wanted message. If the algorithm detected the QR Code, data was read and compared to the prepared data. If prepared data matched with data from the QR Code on the child's clothes, it meant that the child was still on the treadmill.
Motion Detection (MD): MD detected motion or movement by defining initial frame as a reference frame for comparing to the current frame. If the difference was more than the defined threshold, it meant there was motion and the child was still on the treadmill.
Adaptive Video Interaction (AVI): AVI used a mobile camera as the interaction between autistic children and applications or games on mobile devices. There were three sub-techniques combined in this type of interaction: 1) Face Detection

System architecture design
Architecture design of exercise application for autistic children on treadmill consists of three parts -treadmill, iPhone/iPad and speaker. The iPhone/iPad was placed in front of the treadmill to detect the child's face and body, and the speaker is used to make the sound from iPhone/iPad louder so that the child can hear better as show in Fig. 3.   Fig. 3. The Architecture Design The system involves a user, a caretaker and an expert. The user is an autistic child, who would run on a treadmill with an interaction with the application. The care taker can see the report of the child's exercise, and the expert, assisting the care taker to improve the child's behavior, can monitor the exercise, give comments or advises arising during the session, and see the report.
The experimental place needed to be sealed, quiet, without any distractions, secured, and sufficiently bright. According to Fig. 3, iPhone/iPad could only be placed in front of the autistic child, so that the FD or UBD could be employed. The speakers should be placed near the child so that s/he could hear the counting clearly, which would encourage him/her to run for a distance indicated by an expert.

Graphic user interface design
Graphic User Interface (GUI) was universally designed so it can be used on both iPhone and iPad. GUI displayed two parts. The first one was the time or the counting number, indicating the goal s/he had to achieve. The speaker was used to let the child know how long s/he had to run. The second part was the video showing the child's face or any other pictures or objects that s/he likes to encourage him/her to run. This is shown in Fig. 4.

Application development
The test application was developed on the iOS 10.3 platform using the Open CV library for camera management and used still images or videos to stimulate the child's interest. The developed application can gather each individual child's information, which can be adjusted to give the most suitable monitor for each child. Moreover, the application can also carry out the personal report which the caretaker and an expert can use to revise the activity plan. The application was tested on an iPhone 6 and iPad mini 4 as shown in Fig. 5.

Application testing
The application was tested with eight autistic children (all male) from Special Education Division (Autistic Research Centre), Khon Kaen University Demonstration School as shown in Fig. 6.

Fig. 6. Testing ASD's EX Application
The samples consisted of eight children aged between 7 to 12. The experts and care takers designed the study plan for the children. The duration of the tests was one semester and was part of the physical education class. Each autistic child must be tested in six techniques for 30 minutes per session and not less than 12 sessions (an average of two sessions for each technique). The accuracy of all six techniques was evaluated by the professional who took care of the autistic children. The test results are shown in Table 1. From Table 1, FD and UBD were used to detect facial and bodily parts, so the additional equipment was not required. Regarding CD technique, the color to be detected had to differ from the surrounding so that it would function the most accurately. In the experiment, the children were asked to wear a shirt of a specific color to be detected. QCD used QR Code attached on the children's shirt where it could be clearly detected by a camera. MD made use of movement detection technique, which required reference frame where there was no child. After setting the frame, the child was asked to get on the treadmill and started running. As for AVI, no additional equipment was required. Instead, FB, UBD, and MD were mutually utilized to detect whether the child was still on the treadmill. Table 1 shows that AVI gains the highest accuracy rate (92.31%), with MD at 85.71%, FD at 54.55%, and QCD at 10%, respectively. Fig. 7 illustrates that, in most techniques, there are more number of undetectable (by application) than the number of actual disappearing (observed by expert). However, MD is the only techniques with less number of undetectable (by application). This is because MD detects the movement, which possibly allows errors to occur. For instance, when a child leaves the treadmill and the MD cannot detect his/her absence due to the insufficient lighting or unsuitable environment.

Fig. 7. Six interaction techniques results
The result from Table 1 can summarize whether each of the six techniques can be used with autistic children or not. As shown in Table 2, MD and AVI were found to be suitable for autistic children while FD, UBD, CD and QCD were not.  Table 2, QCD technique is least suitable for children with autism, for QR Code is difficult to be detected from a distance. CD slightly outperforms QCD because it detects the shirt's color. However, insufficient lighting in the test environment results in noise and color error, making detection ineffective. QCD and CD face the same problem. That is, some children refused to put QR Code on their shirt or did not want to wear the shirt provided. FD and UBD are suitable only for some children who look at the camera and do not turn to their left or right (stay focus). MD works fairly well, but it could not actually detect children's presence. Finally, AVI is the most suitable technique for children with autism. It combines advantages from FD, UBD, and MD, so it performs better detection and accurately detect the presence of the children on the treadmill.

Expert Opinions
The experts studied these six techniques during the experiment sessions. After they observed and analyzed all autistic children, all of them prefer the AVI technique since it is the most accurate way to interact with autistic children. The experts said they could rely on the AVI technique and allowed caretaker to handle multiple children at the same time. In addition, the great value from the results in the system's reports could help the experts adapt the study plan individually for each child. The reports revealed some significant information such as time recording of presence and absence that leaded to study plan adjustment. In short, the experts strongly believed that system could help monitoring, logging, analyzing, and planning the physical activity for autism children.

Discussion
Using the camera to interact with autistic children is an appropriate technique since the study of autistic children's behaviours' shows that some interaction regularly used by normal people are not effective with autistic children as they cannot understand what they are instructed to perform. For instance, some children can use touch interaction but cannot understand the meaning, making them randomly touch the device. Therefore, effective patterns of interaction for these children must be based on their bio-information (heart rate, blood pressure or body temperature), actions and movements.
According to the results of the experiment, camera-based interaction and proper techniques work well with autistic children. However, the key factor in this research is the behaviours of children with autism, and the aim is to deal with unwanted behaviours. According to the experiments, while face detection, upper body detection, colour detection and QR code detection techniques worked well with normal people, the results are opposite with autistic children because some of them were hyperactive and kept turning left and right or back and forth, which prevented the detection to function properly. In addition, in the experiments where colour detection and QR code detection techniques were used, it was required to stick extra accessories on the autistic children (coloured paper, QR code), but some of the children did not like it and thus removed it from their bodies. It is discovered that interaction using external physical features was the most attractive option for autistic children.
In fact, MD technique perfectly works by using reference frame comparing to current frame. The movement is detectable if their difference is more than the defined threshold. Since the problem of MD technique is that human presence at defined position cannot be detectable, this study presents how to improve reliability of MD technique by using FD and UBD techniques. Therefore, in this paper, we have proposed Adaptive Video Interaction (AVI) method. In designing this interaction, we aimed to adjust the interaction patterns for autistic children, to increase their concentration and focus on running as much as possible so as to reduce the burden on caretakers. While AVI uses cameras to interact like other techniques tested in this research, it combines Face detection, Upper body detection and Motion detection techniques to work together and additionally uses stimulus techniques to attract attention of autistic children to be on the treadmill as long as possible, which gives better results than using Face detection or Motion detection techniques alone. The outcome is that most of the autistic children had more concentration and could stay on the treadmill longer. In addition, AVI also uses counting support techniques to help children with autism count the numbers they were instructed to run. In the normal running process without using the application, there must be a caretaker to help the children count until they reach the target numbers. The counting support techniques can help the children to count numbers without the supervision of caretakers. This research will be greatly useful for children with autism, teachers and caretakers both in the classroom and at home. In addition, access to the application is easy and free.

Conclusion
In this study, six mobile camera interaction techniques were used to support autistic children in physical education class. They include: 1) Face Detection, 2) Upper Body Detection, 3) Color Detection, 4) QR Code Detection, 5) Motion Detection, and 6) Adaptive Video Interaction (AVI). These techniques were developed for the ASD's EX iOS application and installed on an iPhone 6 and iPad mini 4. The application was tested on eight autistic children from Special Education Division (Autistic Research Centre), Khon Kaen University Demonstration School.
The results show that the AVI technique is the most suitable interaction. The results from this study confirms and extends the findings of prior research [16], [18], providing additional empirical evidence that AVI does have a strong potential to improve attention in autistic children and reduce the tasks of caretakers.

Future Work
For further development, more motion detection devices such as smart watches or activity tracking devices should be added in order to improve the accuracy of children's activity detection as well as to collect bio-information such as heart rate for further analysis.
The experts will analyze the data and create a lesson plan to suit each child to encourage them to stay more concentrate.