KinFit : A Factual Aerobic Sport Game with Stimulation Support

Overweight and obesity is a situation where a person has stacked too much fat that might affect negatively his/her health. Many people skip doing exercises due to several facts related to the encouragement, health-awareness, and time arrangement. Diverse aerobic video games have been proposed to help users in doing exercises. However, we observe some limitations in existing games. For instance, they don’t give correct scores while wearing Arabic traditional suits, they don’t consider showing immersive realistic scenes, and they don’t stimulate users to do exercises and keeping them encouraged to play more. We propose in this paper an aerobic video game that displays real scenes of aerobic coaches and keeps the user notified about doing exercises. It is a kind of serious games that allows users to learn aerobic movements and practice with aerobic coaches. It contains several exercises in which each can be played on normal screen or in fully immersive virtual reality (VR). While the user is playing, he/she can see the playing score with the estimated amount of burned calories. It stores the time when the user plays to remind him/her about doing exercises again. The profound user studies demonstrated the usability and effectiveness of the proposed game. Keywords—Aerobic Game, Multimedia Systems, Interactive Trainer, HumanComputer Interaction, Virtual Reality, Metabolic Energy, Kinect.


Introduction
Sport and physical activities are important for all individuals.A systematic time of doing physical exercises assist an individual to stay fit and healthy.In fact, it is not easy to maintain fixed regiment and it is even more challenging for older adults [1].The World Health Organization (WHO) induces at least 150 minutes of aerobic physical activities every week, or at least 75 minutes of sturdy physical activities that are done in 10 minutes slots [2].In addition, the WHO highlights on the affirmative consequences of doing these exercises such as decreasing the cases of strokes, fewer cases of coronary heart disease, fewer rate of mortality, and a synoptic health improvement [2].Besides these advantages, it is predominately not easy for individuals to preserve the routine of doing physical activities due to several reasons.Some of these reasons include motivation, time management, boredom and others [3].Dishman discusses the practical outcomes that doctors and patients have on ameliorating individuals' motivation to do physical exercises by having social interaction [4].In this context, a social sport mobile based application was proposed by Higgins [5].It enumerates the number of walking steps and allows sharing milestones on social media.The developed mobile application proved it enhanced the motivation of individuals to exercise more.Another aspect of improving the motivation for individuals is to allow them to play video games.In fact, video games have exceeded television watching as form of entertainment and have been used as preferred and attractive sedentary activity option [6].Consequently over the years, several multimedia sport games have been proposed [7][8][9].In this article, we attempt to solve the following limitations: 1. Correct scoring while wearing Arabic traditional suits: Existing video games; like [10], [1], and others; do not consider hidden parts of the person who is wearing traditional suits, resulting incorrect scoring results.2. Absence of realism: Majority of existing aerobic games; such as [11], [12], and others; use 3D avatars which often decrease the sense of realism especially for adults and older people.3. Stimulation support: To the best of our knowledge, existing aerobic video games do not consider reminding the user to do exercises regularly.They also do not encourage the user to play more.
In this paper, we propose an aerobic video game which we call "KinFit".It allows individuals to watch real scenes of aerobic coaches and keeps individuals aware of doing exercises.The proposed game uses Microsoft Kinect sensor [11] to assess the movement accuracy of the user.It has variety of aerobic exercises that were designed by aerobic coaches in the state of Qatar.The user can see his/her playing score including the approximate amount of burned calories while playing the game.To keep the user informed about doing exercises and to keep him/her motivated, it stores the playing time to notify him/her about the possibility of doing exercises again in that time.The conducted experiments demonstrated the usability and the efficiency of the proposed aerobic game.
The rest of the paper is organized as following: Section 2 presents the literature of existing computerized systems designed for improving rehabilitation and aerobic physical activities; Section 3 provides details about the proposed serious game; Section 4 presents and discusses the experiments and the conducted user studies; and finally Section 5 concludes and summarizes the paper.

Literature Review
Several aerobic video games have been proposed to help people in playing sport at their home [8,9,13].Chatzitofis et al. [14] proposed a system for e-coaching and monitoring exercise performance.The objective of the system is to raise physical activities of users and their fitness level, enhance the efficiency of activity-based rehabilitation and coaching, and thereafter motivate them to become more energetic.The system uses Microsoft Kinect sensor [11] to track the motion of the user and uses 3D avatars while presenting the motions for him/her.The proposed system demonstrated that it enhanced the training and rehabilitation of users.It motivated people to become more energetic to do more physical exercises.Thus, it enhanced the quality of their lives.Wang et al. [10] proposed an interactive coaching computerized system designed for elderly users.The system is designed to track remotely their motions when they perform health coaching and physical therapy at home.It is also used to motivate them to become more active.Microsoft Kinect sensor [11] is used in order to track the motion of users.The preliminary studies and feedback results of the proposed remote health coaching system were positive.Ofli et al. [1] proposed an interactive exercise coaching computerized system designed for older adults.The system provides guidance for users by giving a collection of video exercises, tracks and measure their motions, records their achievements, and provides real-time feedback.It also aims to reduce the risk of falling and enhance the motivation to make physical exercises.The recorded video exercises do not include aerobic activities.Microsoft Kinect sensor camera [11] is used in order to track the motion of the users.Authors are planning to select wider selection of exercises with a flexible user interface.
Pedraza-Hueso et al. [15] developed a VR based serious game called ReaKinG.It tracks the motion of users while doing physical activities and cognitive rehabilitation therapies.The activities and therapies include aerobic sport exercises.The game uses Microsoft Kinect sensor [11] in order to assess the motion of the user while playing.It provides audio-visual feedback during exercises and enables therapist to configure therapies remotely.It uses 3D avatars in exercises.The game was designed to provide enjoyable experience that increases the motivation of a user including his/her rehabilitation.Jofré et al. [16] proposed a virtual training computerized system that can be customized according to the required difficulty of physical activities.It provides visual guidance and performance feedback in real time while performing the activities.The system employs natural user interface implemented in immersive Cave Automatic Virtual Reality Environment (CAVE) with conversational communication feature [17,18].The feature uses Automatic Speech Recognition (ASR) mechanism to respond to the user.The system has collection of aerobic exercises and several physical activities.Exercises are designed with 3D animated avatars.It uses Microsoft Kinect sensor [11] to follow up the motion of the player.Nathan et al. [19] proposed a video game that tracks the motion of the user and employs several predictive algorithmic models to estimate the Metabolic Energy (ME) spent.The video game uses two Microsoft Kinect cameras to track the motions.In order to estimate the ME, the collected data from the two devices were classified using machine learning approaches such as Gaussian, k-Near Neighbour (kNN) and linear progression.The proposed solution demonstrated that Kinect device can be used to estimate the energy spent while playing the video game.Li et al. [12] also examined the employment of virtual trainer in interactive video games.They proposed bodily motion detection using Microsoft Kinect sensor and the user have to mimic the motions displayed on the screen by 3D avatar trainer.If the user performed the correct motion he/she will gain points.The proposed video game indicated its effectiveness in motivating users.
In summary, several researches have been conducted about rehabilitation and interactive coaching exercises [12,[19][20][21].However, existing games do not consider long realistic aerobic activities.This is unfortunate because capturing long time aerobic exercises requires persistent movement and continuous measurements.The continuous measurements include positions of user, joints angles, and speed detection.In addition, they do not consider encouraging notifications to keep the user motivated and active.Thus, the main focus of our work considers the design and development of computerized interactive exercises that can assess the performance of users while doing long aerobic exercises and can keep them motivated to play by reminding them with email notifications.

3
The Proposed Game The proposed serious-game can help users to play different aerobic exercises that are designed by aerobic trainers in the state of Qatar.Using a friendly multilingual User Interface (UI), the user can select his/her intended exercise in order to play it.The exercise shows some required sport information including the required time to finish the exercise, estimated burning calories, and some advices mentioned by the aerobic trainer.In order to record the information of the user on the server, the user has to create an account and use it in order to access the game.The user can play exercises in nonimmersive mode on normal 2D screen or in fully-immersive mode using headset mounted display (HMD) using Oculus device [22].While playing the exercise, the user can see in real-time the score and the estimated burned calories.After playing an exercise, the game stores the playing information including the achieved scores and the time spent while playing.These information are used in order to keep the user motivated and to encourage him/her to play again.

Applied Methodology
During the design process of our game, we met nutritionists who work at Hamad Medical Corporation1 (HMC).We asked them about the appropriate sport exercises that are suitable for all ages and can be done without moving in large areas or can be done while standing.Therefore, we arranged with a photographer to record high quality videos.We took also the permission from aerobic trainers to record required video clips with the required motion data.Figure 1 shows a trainer and his skeleton captured by Kinect sensor.During the development process, we used the IEEE 1074-2006 standard criterion to develop the game.It is a standard for evolving software project life cycle procedure [23].We defined modules, classes, and designed the required user interfaces of the game.We managed our game to load automatically new aerobic videos without setting them manually in the game.This is to enable adding new aerobic video clips at any time.Eventually, we tested our game and conducted a usability study to assess it.

KinFit Game Architecture
The architecture of the game is client-server architecture as shown in Figure 2. The server is responsible about sending email notifications to the user to remind him/her about playing again and to keep him/her motivated.The game is developed with Unity 3D [24].It is responsible about performing the core functionalities.These functionalities include: (a) attempting to connect the server; (b) attempting to connect Kinect sensor; (c) attempting to connect the HMD device (i.e., Oculus [22]); (d) loading and storing user information; (e) sending information to the online server; (f) loading video exercises; and (g) calculating the scores while playing exercises.

Matching Score
To compute the matching score between the user and the game player, we compare the joints' angles of the user with the joints' angles of game player [25].Since Microsoft Kinect device may not identify all joints properly when the user is wearing Arabic traditional suit, we consider also the distance between the set of user's joints in the 3D space and set of joints of the game player are computed.The set of joints include all joints of the left arm, right arm, left leg, right leg, spine, and the head joint.Thus, the distance between a single user joint  " # and a game player joint  " $ is computed as [26]: where, x, y and z are the points of the joint in the 3D space, and i is the index of the joint in the set of joints.We used the spine shoulder joint, spine mid joint, and spine base joint to normalize other joints in the 3D space and their default distance between each joint.Consequently, in order to avoid the latency while the user is playing, we considered to split each video exercise into several intervals to compute the score.The minimum distance  3 in each interval  5 will be selected as the best distance and used for the computation of the score as follows: where r is the index of the interval in the finite set of video intervals V, and k is the index of the distance of every interval.Thus, if the difficulty of the exercise is higher means the interval I will have a lower value.Therefore, we compute the sum s of all selected distances between all joints in the collection n as follows: where l is the index of the selected minimum distance of every frame.Thus, if s is less than or equal to threshold  ( ≤ ) then we note that the user performed the correct action (motion) shown by the game player, and the matching score will be incremented by 1. Otherwise, if s is greater than , then we mark that the user did not perform the correct action, and therefore, the score will not be incremented.The conditions of the score can be summarized as follows: Eventually, in order to show the matching percentage (%) in real time on the screen, we divide the completed intervals  ] as follows: × 100 (5)

Estimation of Burned Calories
In order to estimate the amount of burned calories, we used the method of Harris-Benedict equation (HBE).The Basal Metabolic Rate (BMR) equation in kilocalorie per day (kcal/day) for a male with an age a (in year), weight w in kilogram (kg) and height h in centimeter (cm) is computed as follows [27]:  = 66.5 + 13.8 ×  + 5.0 × ℎ − 6.8 ×  (5) and the BMR equation for a female is computed as follows [27]: We used the HBE equation because it is commonly used by several online healthcare websites [28][29][30].In order to lose 1 kg of weight, the user will require an energy of 7500 kcals.The Metabolic Energy of Task (MET) is used in order to estimate the burned calories between the different sport activities.For instance, the MET of dancing aerobic in general is equal to 6.5, the MET of light weight lifting is equal to 3.0, and so [31]. 1 MET is equal to a metabolic rate that consumes 1 kcal / kg of the total body weight per one hour.The total amount of burned calories C of an exercise with MET for a period of time (in minutes) is equal to [27,32]:

Exercising Reminder
The game stores the start time and the finish time of played exercises.We marked these data useful information in order to remind the user to play again on that time.The server attempts to send email notifications to the user to inform him/her about playing again.In the first week of using the game, we configured the server to remind the user every day before half-hour of the recorded starting time of the exercise.After the first week, the game uses only the days in which the user used to play exercises.We considered also to show the player who burned calories at most every week.We created a Gmail2 account in order to send email notifications.We configured the server use the created account to send maximum two reminders per day.Figure 4 shows a sample email message sent to a user.

Technologies and Specifications
The KinFit game uses Microsoft Kinect V2 sensor in order to track the movement of user's joints.Microsoft Kinect Software Development Kit (SDK) has a module that enables recording the movement of these joints.The recommended software and hardware technologies are [33]

Game Operation
Whenever the game is launched, it loads its settings, attempts to connect to the server, checks if the Kinect device is accessible, and requests the user to login or to create an account.Figure 5 shows the registration page.Upon successful login, the application loads available videos and displays them for the user as shown in Figure 6.The green Kinect icon on the top right indicates that Kinect sensor was successfully loaded.However, the red HMD icon indicates that the headset was not detected.Once the user selects an exercise the user will be requested to stand in front the Kinect device and to start the exercise.When the exercise starts, the user must perform the same actions performed by the trainer.Otherwise, the user will lose some scoring points.The game shows the user his matching score, the estimated burned calories, and the used time while playing as shown in Figure 7.After the completion of the exercise, the game stores scores and attempts to send them to the server.The server stores the scores online and reminds the user to play again by sending an email to him.

Game Evaluation
In order to quantitatively assess our game technology, we considered different evaluation studies.These studies include: (a) Usability and User Satisfaction studies; (b) Dressing Analysis; and (c) Encouragement Analysis.We obtained ethics approval from Qatar University Institute Review Board before the beginning of the analysis.All users gave their written approval before the start of the study.We asked 25 participants to try the game.We asked them to stop at any time during the session if they feel tired or unsatisfied.Moreover, we asked participants to fill a survey questionnaire after they used the game.The questionnaire includes usability and satisfaction analysis.We discuss these experimental studies in details in the following sections.

Usability and User Satisfaction
We designed a survey questionnaire in order to assess the usability and the user satisfaction for the game.We considered the different perceptions in [35] during the process of designing the survey.We requested the users to fill the survey after they used the game.They were asked to rate the usability and satisfaction statements, on a scale of 1 to 5, where 1 means 'Strongly Disagree' and 5 means 'Strongly Agree'.Table 1 summarizes the results of the filled surveys.In brief, most users were satisfied with the proposed game and conveyed that it increased their motivation to play for longer time.

Usability and Satisfaction statements Average
I was satisfied in using the aerobic game 4.07 The game is easy to learn and easy to use 4.40 The aerobic game helped me to do more exercises 4.13 The aerobic game raised my motivation 4.20 The email notifications reminded me to play the game in regular timing 4.33 The email notifications encouraged me to become a winner 4.47

Dressing Analysis
The game was used by male users and female ones.The dresses of these users vary from athletic wearing to ghutra cultural wearing [36], an Arabic traditional suit such as Abaya or Thoub.We tested the different wearing of users in two lighting conditions: (a) Light condition: The game is tested where lights of the room are turn on; and (b) Dark condition: No lights are turned on in the room while playing the game.Users who were wearing athletic dresses were properly captured from the Kinect sensor.However, users who were wearing ghutra were not detected properly from the Kinect sensor.In fact, the joints of the lower part of their body were not identified correctly.These joints include hip, ankle, and foot joints.The playing score was always less than 40% event if the users are doing the correct motions.However, after we employed our scoring algorithm, the playing score was not decreased less than 80%.

Email Notifications Analysis
We studied the effect of using email notifications to users to remind them to play the game.We put an assumption that receiving email notifications will encourage users to play the game more.For a period of one month, we considered the data stored on the server to know how many times each user is accessing the game.We compared the usage of the game before and after enabling the email notifications service.Figure 8 shows the obtained results prior and after using the email notifications.In fact, the email notifications service encouraged the users to play more the game.It reminded him/her to remain healthy by doing exercises regularly.Thus, showing them the winner who achieved the highest score made users more competitive to become winners and thus to play the game more.
In addition, we considered to study how the email notifications affect the total number of minutes while using the game.We computed the total number of minutes for a sample of 7 users played in one month.In fact, the email notifications encouraged users to play the game for more times.Thus, it affects significantly the total number of minutes while playing the game.Figure 9 shows the total number of minutes while playing the game before and after using the email notification service.Consequently, the email notifications affected slightly the time of playing the game in every session.Thus, a user who was playing for approximately 30 minutes without reading email notifications used to play for slightly longer time after reading emails.The average number of minutes before using the game without notifications was 35±14 minutes, and after using the game with notifications, it became 39±6 minutes.Figure 10 shows the average number of minutes while playing the game before and after using the email notification service.

Discussion
Our proposed game solves the limitation of computing the appropriate score while wearing traditional suits, such as Thoub or Abaya.When the user is wearing traditional suit, Microsoft Kinect Sensor cannot identify all the joints especially the lower part of his/her body.Therefore, missing joints will often result incorrect scores.For this reason, we proposed a solution that continuously checks if the lower part of the body is being detected to include it in the scoring without depending only on using angles.Our proposed game is limited to exclude joints that were not identified in order to fix the computation of the score.The proposed game meets the demands of users who wear traditional suits and prefer to stay wearing in public places.For this reason, we configured our game to play recorded video clips of real coaches to give young and older people the sense of realism.Video clips can operate in normal 2D mode and in fully immersive mode with Oculus device.Moreover, old people who are not able to continuously go to the gym found the game advantageous.They had the feeling that we brought the gym to their home.Consequently, we noted that the exercising reminder kept them aware from doing exercises.To assess the usability of the game, we conducted survey questionnaire with participants.Participants were satisfied in using the game and they became more motivated when they received email notifications.Some users were looking for a solution that can assess the motion of all their body without exclusion.However, due to the limitation of the employed sensor, we were not able to provide full solution to track their joints completely.As future work, we plan to study additional solutions to solve this limitation.

Conclusion
In this article, we presented an aerobic video entertainment game that shows real videos of aerobic coaches.The game is designed in collaboration with aerobic coaches in the state of Qatar.The game is a kind of serious games that keeps users notified about doing exercises and motivated by sending them email messages.It helps them to learn aerobic movements and practice with aerobic coaches.It uses Microsoft Kinect sensor in order to track the motion of a user and to assess his/her performance.Several exercises were prepared in which each exercise was recorded in non-immersive mode for the use on normal display screen, and in fully-immersive mode for the use of HMD.The user is able to see his/her score and the estimated burning calories while playing the game.The game stores when the user played the game in order to notify him/her to play again.The conducted studies showed it motivated the users to do more exercises.Future research work will consider the employment of machine learning techniques and bigger number of samples to obtain more accurate assessment results.We plan to make the game publicly available and assess it with different cultural backgrounds.We plan to improve the matching analysis while a user is wearing traditional suits.We plan to add required components, such as custom sensors and features, to enable people with visual impairments to use the game.In addition, the design and implementation of an augmented reality (AR) based game is planned.

Fig. 1 .
Fig. 1.The skeleton of the coach on the left and his picture on the right.

Fig. 2 .
Fig. 2. The proposed architecture of the game.

Fig. 4 .
Fig. 4.An email notification received by a user.

Fig. 7 .
Fig. 7.A user playing an exercise: The trainer is in the middle of the screen; The scores are on the top of the screen; The user is in the window on the bottom left.

Fig. 8 .
Fig. 8. Average use per one user per week.

Fig. 9 .
Fig. 9.Total number of minutes in the different notification modes.

Fig. 10 .
Fig. 10.Average number of minutes while using the game in the different notification modes.