Special Focus Paper—An Adaptive Virtual Reality-Based Serious Game for Therapeutic Rehabilitation An Adaptive Virtual Reality-Based Serious Game for Therapeutic Rehabilitation

Conventional motor rehabilitation therapies followed by stroke survivors during their recovery process are typically intense and involve numerous repetitions of task-specific recommended exercises. However, due to the lack of short-term or immediate results, patients tend to lose motivation on the rehabilitation process, and on performing the therapist recommended exercises at home between sessions. The work described in this article proposes to engage the patients with the therapeutic exercises by transforming the latter into specially designed serious games. After an initial evaluation by therapists, a pilot study was conducted on a rehabilitation clinic that showed that this system is adequate for about 75% of the clinical cases treated. The proposed game was globally wellaccepted by patients, showing to be motivational and engaging. These promising results lead us to believe that immersive-based games can indeed be a valuable aid for motor rehabilitation therapies. Keywords—Motor rehabilitation, engagement, stroke, sensory stimulation, virtual reality, serious games


Introduction
The human brain controls everything that the body does. When a traumatic incident occurs, e.g. a stroke, survivors may have to deal with different kinds of impairments, depending on the location and size of the affected area. Hemiplegia or hemiparesis is one of the most common motor impairments, which results in total or partial paralysis on the contralateral hemisphere of the body, concerning the affected brain area [1]. Naturally, individuals will experience a vast set of heterogeneous difficulties during the execution of their daily living activities. To overcome these difficulties, they commonly tend to overuse the healthy limb and forget the affected one. Conversely, to address any possibility of recovering the functionality of the affected zones, post-stroke rehabilitation protocols should start as soon as possible, once the individual's condition is considered stable.
Motor rehabilitation and occupational therapies are crucial during the recovery process of patients, as they to try to lighten these difficulties and lead them to recover the lost strength in their limbsmostly arms and shoulders. However, they are known to comprise task-specific oriented exercises through numerous repetitions [1,2]. As one-third of stroke survivors leave the hospital in a depressive state [3], along with the lack of immediate or short-term self-noticeable results, patients frequently lose interest in the recovery process. Moreover, as their motivation is not enough for them to keep performing the recommended exercises at home between sessions, it results in lower and slower recovery rates [2]. Literature reveals that enjoyment during training plays a major role when asking patients to perform the same exercises for long periods and/or demand them to perform a high number of repetitions [3,4]. To this end, one can consider transforming therapeutic practices into playful ones by making use of gaming approaches during these sessions.
Serious Games (SG) have attracted the interest of researchers in many areas as they have proved to be bring an interesting added value for many different human skillimprovement applications. Although making use challenging principles that make regular games attractive, as they are have the purpose to take the player/user to learn something new, or (re)acquire a certain skill [5]. When these are applied to motor rehabilitation, if combined with the use of a head mounted displays (HMDs), patients will not be able to see the affected body parts, therefore creating the perfect opportunity to map their possible movements into the required game controls.
One clear advantage of using Virtual Reality (VR) in these contexts, is that the individual will focus on the game itself, instead of his/her limitation. Along with it, other documented advantages of VR-based rehabilitation include better patient engagement, enhanced motor (re)learning by visual stimuli, monitoring patients' performance and tracking recovery over time, and even allowing tele-rehabilitation [6,7,8].
In this work, we explore the development of a virtual reality-based serious game for the upper limb rehabilitation that can be played by patients in different recovery stages. To this end, we have carefully designed it to follow the rehabilitation principles, while motivating the patients to keep performing the exercises beyond what is usual with conventional techniques.

Target patient condition and therapeutic phase
As stated before, one of the most common consequences of stroke is hemiplegia or hemiparesis. The recovery process is typically assessed by monitoring the evolution of patient's muscle tone and dexterity recovery [9].
In early stages, patients are unable, or have large difficulties, in both performing any kind of movement and keeping their affected limbs in stable positions because of hypotonia. This low muscle tone state is depicted as a decrease in resistance to passive movements and overtime is replaced by spastic hypertonia, which corresponds to an abnormally increase in muscle tone that leads to an extremely rigid state that complicates movement execution [1,10].
Other impairments may also result from a stroke affecting speech, vision, swallowing, and cognition. The most common type of visual issue is hemianopia, which results in the loss of the left or right visual field of both eyes. Individuals are unable to see a section of their vision area, which leads to blind zones and processing issues.
Patients undergo rigorous and complex motor rehabilitation therapies to regain normal muscle functioning, starting with exercises that strengthen the muscles of the arms and shoulders, and later for hands and fingers. The rehabilitation plans are typically defined based on an initial evaluation of the patient condition. The condition evolution is periodically assessed based in functional dexterity improvement and muscle tone in terms of spasticity, rigidity, and flaccidity [10,11]. Spasticity refers to an increased resistance to passive movement that is velocity-dependent, while rigidity is velocityindependent. Flaccidity is characterized as a great decreased resistance to passive movement, meaning that the patient is to not able to start or perform movements.
Visual scanning training refers to a common therapeutic procedure that seeks to lead with these issues by forcing them to look in a systematic way around the whole visual field [1]. Therefore, the work presented in this article not only addresses dexterity recovery, but also visual rehabilitation by means of this technique.

Sensory stimulation on post-stroke motor rehabilitation
Motor rehabilitation is commonly associated with the idea that large amounts of exercises are used to promote muscle strengthening and articulation functionality. However, in case of strokes, the therapeutic process must above all stimulate the construction of alternative neural pathways that enable the necessary stimuli to reach the muscles and the consequent sensory information to return to the brain. In fact, after a traumatic incident, the brain attempts to maintain control of everything, adapting its functioning to a whole new situation by employing neuroplasticity mechanisms [3]. Neural reorganization and adaption rely on remapping the brain's sensorimotor interactions, either by transferring functions from the damaged areas to the adjacent healthy ones or by establishing new neural pathways [12,13].
Most human learning comes from the observation of someone executing object-oriented actions, which triggers our visuomotor cells (mirror neurons). These visuomotor cells can be linked to the motor (re)learning and allow brain to consolidate their previous sensorimotor representations [14]. Given that immersive technologies use made-up information to trick brain inputs and mislead perception during its usage, the individual could perceive completing a certain action through a visual representation and auditory feedback, without actually performing it alone. Thus, these alternative stimuli (visual, sound, or haptics) may contribute to enable patients to (re)learn how to perform and enjoy the demanded tasks that contribute for their recovery process.

Developed Serious Game for Therapeutic Rehabilitation
Games for therapeutic purposes must be simple, but challenging enough to keep patients engaged without developing any type of frustration or boredom. After analyzing different games with therapists, a Tetris-like game was considered adequate for the intended purpose. This type of game does not only allow the execution of movements that strengthen patients' muscles of the arms and shoulders, but also produces the necessary stimulation of their cognitive activity and sensory inputs.
As for any video game, the main objective of Tetris is to score as many points as possible, in this case by forming complete horizontal lines inside the playing area, with the falling shapes (tetrominoes). During the fall of each piece, the player must adjust its orientation and horizontal location so that it will fill in the best position as possible given the existing holes left by the previous ones.
The rotations may be either performed by pressing a button on a controller or by performing a pronation/supination movement with the hand, eventually acquired with the same controller. As they complete one or more horizontal lines without any empty spaces, the lines disappear from the scene, adding a certain amount of points to their score. However, with the increasing score comes increased difficulty, which causes tetrominoes to fall faster and pushing the game to become progressively more challenging. Nevertheless, this difficulty level is dynamically reduced if the patient is unable to score any points after a given time period, as an approach to avoid frustration feelings that would result in early abandoning or lack of interest in repeating the game.
Since different motor-impaired patients have inherently different difficulties, a calibration process is required before playing. This process allows the game to dynamically adapt himself to the player's motor capabilities. Following the same purpose, two playing modes with the same game elements were designed and carefully developed, allowing us to expand the spectrum of patients that may play the game. The Regular Mode mimics the known version of Tetris. However, players must guide the falling tetrominoes using a metallic pad that is placed on the bottom of the playing area, as shown in Figure 1. This pad acts like a "magnet", as it attracts the shapes to the same position that it is on the playing board.
The Timed Mode was attentively designed to ease the required tasks of players. They have a certain amount of time to decide the most advantageous position for the tetromino while moving and rotating it on top of the playing board, as shown in Figure 1. Note that during this time, the player directly controls the shape by moving the metallic pad. Once the timer runs out, the tetromino falls immediately. An evident advantage arises from this playing mode, as such the individual has more time to decide what he/she is going to do. On the other side, it also fastens the learning curve of the inexperienced users, allowing easy adaption to the game.
The game was developed with the purpose of supporting a play-station that can be both taken to the clinic and home environment. It only requires a desktop/laptop with the used graphics motor (OpenAR), an Oculus Rift, the Razer Hydra Motion Controllers and a regular pair of headphones/speakers. The OpenAR is a graphics and animation engine developed in the Institute of Systems and Robotics. It provides basic support for manipulation and rendering of 3D objects and scenes, interfacing with Oculus Rift and Bullet physics engine. The choice of Razer Hydra Motion Controllers was based on its adequacy for these type of patients and given that it provides the information about the position and orientation of the user's hand while playing. This setup can be easily moved around the clinical environment given its simple configuration and no particular space requirement, other than a small table.

Visual and cognitive stimulation
Following the visual scanning technique referred in Section 1, special attention was paid to the placement of the objects in the virtual scene, promoting visual exploration. To this purpose, changing the distance between the playing area and the user's point of view leads to an interesting variant to the game. Bringing the playing board (virtually) closer to the user, forces him/her to perform larger head and gaze rotation amplitudes to be able to track and guide them to the appropriate position. Nonetheless, if the player has a severe handicap, this distance is augmented and fewer head movements are required. Applying this visual stimulating variant depends both on the player's stage of recovery and on his/her therapists advises.
As it can be played with either a head mounted display or a regular TV/computer screen, it may also stimulate the 2D/3D visuospatial skills retraining. Whenever an individual is trying to find the most advantageous position for the tetromino, he/she is stimulating this ability. On the other side, although without being a truly cognitive demanding games, it still provides an adequate challenge, as for the interaction it requests the establishment of connections between perception, cognition and decision making.

Pilot Study
The serious game described above was presented to a few therapists of a local rehabilitation center, Caritas Diocesana de Coimbra. They tested and evaluated it as perfectly adequate for its intended purpose, allowing the conduction of a pilot study in order to understand its usability in real cases. The 15 participants that took part on this pilot study were 60% male and 40% female, with ages between 30 and 90 years old. Most of them belong to the age group between 50 and 69 years old and have no prior experience with technology. Before playing by following their therapist's advises, and after their informed consent agreement, they were introduced to what they are about to experience and take part of, as well as the technology that is going to be used.

Analysis of the results
After using the system, the participants answered to a User Experience Questionnaire (UEQ) that is a commonly used, fast and reliable questionnaire that measures the user experience of interactive products [15]. It makes use of 26 semantic differential items that evaluate 6 distinct aspects: Attractiveness, Perspicuity, Efficiency, Dependability, Stimulation, and Novelty.
There were 4 participants that couldn't use the head-mounted display due to their health condition, but still played the game using a regular screen. As a result, we separately analyzed the experiences, both screen and virtual reality-based, using the Data Analysis Tool provided along the UEQ. However, one of the questionnaires for the screen-based experience revealed major inconsistencies for the answers, which means that it would mislead the results. Therefore, this one was not considered, reducing the overall sample to three questionnaires. Having such a small sample, one cannot infer major conclusions, such as the ones for the experience presented below.
For this experience, the overall sample counts with eleven participants, which naturally leads to a wide amplitude of answers for the same items. Figure 2 presents the overall classification for the six scales evaluated by the questionnaires, based on their mean values. The scale of Attractiveness shows the higher value, which means that our developed serious game can indeed be motivation and engaging. However, as most of the users have no prior knowledge of technology and never experienced virtual reality before, the items regarding Perspicuity scored the lowest values. The overall results still show that the experience was pleasing for the participants, suggesting that the system can indeed be used in clinical environments,

Fig. 2.
Overall classification of the six scales means for the virtual reality experience. Those six scales are presented in the X axis, while the mean values are presented in the Y axis.

Results of patients
The results obtained by having patients playing our developed serious game at the rehabilitation clinic also form a fundamental part of this work. During the conduction of the pilot test, their enthusiasm was incredible and revealed that how much they appreciated it. Some parameters that can be helpful for a subjective evaluation of therapists were collected during the gameplay and are presented below:  Fig. 3. A patient playing our serious game in virtual reality.
The total distance covered by the affected upper limbs of all the participants was approximately 385 meters. Considering that the perimeter of a football field is 340 meters, one can infer that they performed a successful lap around it. Also, they played together for about 431 minutes, which is enough for completing almost five football matches.

Conclusion
The work described in this article aimed at the development of virtual reality-based serious games for upper limb rehabilitation. A motivational game was created and evaluated as important to aid patients in the development of skills on attention/concentration, coordination, and movement control. The 15 participants of the conducted pilot study represent 75% of the clinical cases that pass through our local rehabilitation center. Although they are all in different states of their recovery, they have all managed to play the proposed game several times. One observed advantage of the HMD over the regular screen is that it leads patients to pay more attention to the task they are performing, which is important in clinical spaces that have a large flow of people. Our promising results lead us to believe that immersive technologies may indeed be a valuable aid for conventional motor rehabilitation therapies, requiring further work to explore different therapeutic exercises.

Authors
Bruno Ferreira is a Ph.D. student in Electrical and Computer Engineering at the Faculty of Sciences and Technologies of the University of Coimbra. He is also a researcher engineer at the Institute of Systems and Robotics, which main areas of interests are immersive technologies for health applications and machine learning applied to human pose estimation and action recognition.
Paulo Menezes is with the Department of Electrical and Computer Engineering of the Faculty of Sciences and Technology at the University of Coimbra. He is a senior researcher of the Institute of Systems and Robotics where he has recently created the Immersive Systems and Sensory Stimulation Laboratory. His main research interests are focused on human behavior analysis, affective interaction, and immersive systems, with application to health, wellbeing and industrial environments. Email: PauloMenezes@isr.uc.pt