The Application of KINECT Motion Sensing Technology in Game-Oriented Study

—The learning environment based on the KINECT Motion Sensing technology is able to fully mobilize the learners' multi-sensory organs, closely combine study with sports and enhance human-computer interactions, which can be conducive to the learners' health, greatly increase the relishes of learning and promote effective learning in the game, and finally compensate for the shortage of human-computer interactions in the traditional mouse and key-board mode. The article elaborates on the KINECT Motion Sensing Technology and its educational applications status by analyzing its effective supports for game-oriented studying environment, based on which the article establishes a game-oriented learning environment. Eventually the article reveals an applicable case of game-oriented teaching and learning as a reference for related researches.


INTRODUCTION
It has been a long time since games and education related to each other. The popularity of online games makes people aware of the potential influence of games among young people, meanwhile the development of games greatly inspires people to research the diversities and possibilities of its applications in education. [1] Gameoriented learning is an intersection of the serious gaming and e-learning, which adopts teaching forms like roleplaying, etc. [2][3] It makes learners obtain knowledge and skills within a highly immersive, individualized, interactive, interesting and entertaining learning experience, and it breaks away from the traditional one-way preaching mode, highlighting interactions and focusing on communication in order to achieve better communication and learning effects, so as to achieve the purpose of learning through play. [4][5] [6] Game-based learning environment not only emphasizes designing a game-based learning system but also focuses on the human-computer interactions in this environment, of which the flexible interactive mode of simplicity, nature, and humanization is more suitable for the needs of the human-computer interaction. [7] American psychologist William James believes that the mood begins in the perception of the physical changes. Your body has already responded even before you feel the current mood. The declaration of KINECT is that "you are the controller!" and to dialogue with the machine using the most natural language. [8]KINECT Motion Sensing enables learners to control and interact with the learning environment in a simple and natural way by directly using languages or gestures, which provide learners with a real sense of expe-rience, and can greatly increase the learning fun and achieve effective game-based learning. [9][10] [11].

II. KINECT MOTION SENSING
Image Recognition and Motion Sensing have been the research focus in the field of computer graphics over the years. In the commercial areas, Japan's Nintendo launched WII hand-held motion sensing controller home consoles in November 2006, which achieved human-computer interactions by the hand-held controllers and was highly welcomed by the teenagers. [12] In 2010, Sony Corporation of Japan introduced the PS MOVE, the new generation of motion sensing equipment with a PSEYE camera. A lightemitting ball through the top handle of the camera can determine the location and space depth of its users in the 3D space thus realize the interaction of motion sensor and make players experience the game in a way of reality and ease. [13] In the research areas, some institutions like CAD & CG State Key Lab of Zhejiang University employed the USB camera that is combined with a special algorithm to recognize simple human gestures. Tsinghua University used the front cameras of mobile phones or USB cameras to identity human faces and tracks the mobile location, eventually to develop new mobile and computer games. Pranav Mistry of MIT developed a device named " sixth sense", it combines cameras, projections, gesture recognition and cloud computing .It uses ordinary paper as screens , which is a usable gestural interface device comprising a neck worn pendant that contains both a data projector and camera to complete human-computer interactions. [14] In addition, Google has done the similar study of motion sensing and launched Gmail Motion to identify the human postures through an ordinary camera and algorithms, so as to browse, read, and reply emails. [15] These technologies meet people's needs for interactions to some extent, however, they still have shortcomings in terms of natural extent of interactions, cost, development of application software and the levels of precision identification, etc.
KINECT is a motion sensing input device by Microsoft for the Xbox 360 video game console on June 14th, 2010 and is a relatively mature commercial human motionsensing device at present. KINECT was first announced by Prime Sense, but purchased and commercialized by Microsoft which started the code named "Project Natal" afterwards. KINECT is a 3D Motion-sensing camera that tracks and interprets multiple movement and gestures; meanwhile it functions in image recognition, microphone input, voice recognition, community interaction, and human-face recognition.

SHORT PAPER THE APPLICATION OF KINECT MOTION SENSING TECHNOLOGY IN GAME-ORIENTED STUDY
The KINECT tracking recognition function mainly includes four parts: infrared projector, infrared camera, RGB camera and multi-array microphone. [16] The infrared projector projects the infrared spectra to irradiate human for random reflection spots, which are read by infrared camera to create a visible depth map of human and objects, and then to identify human action. Moreover RGB camera can shoot a video image in its viewing range, make auxiliary calibration to the body movements, while the microphone array from four microphones captures sound and filters the background noises to locate the sound source and recognize the speech.

III. RESEARCH STATUS
KINECT motion sensing technology, launched by Microsoft, was first used in the Xbox360 video game console and then has become a most popular entertainment interactive technology. KINECT exerts its advantages greatly in Physical Educations, such as KINECT Sports developed by Rare includes football, volleyball, bowling, track and field, table tennis and boxing so that students can use their body movements to do indoor simulation trainings conveniently as well as advance virtual confrontations with their partners; Frontier Developments developed KINECT imals in which players can interact with virtual animals and compete intensely.
[17] Learners can communicate with 40 different kinds of cats and develop close and friendly relations with them. Animals are able to obey the instructions from the learners, such as jumping, circling, etc, but also can respond to learners' touch, showing they are cute and tamed. It is beneficial for learners to access knowledge about animals and to cultivate their awareness of protecting animals.
In the application area, there are the Fitting System based KINECT in the Low-Carbon Science and Technology Museum in Hangzhou, KINECT Pioneer from "Microsoft Campus Elite Plan", the Digital Museum Navigation System from Zhejiang University, KINECT-based Intelligent Learning Space Projects from Beijing Jiao tong University, research on real-time synthesis of highdimensional human body animation based on KINECT from University of Technology, etc. A Chinese online research finds out that researchers and research institutions about the application development of KINECT motion sensing interactions have shown a rapid upward trend, such as building the KINECT development forums or QQ groups for learners and researchers. Regrettably, most studies tend to serve for the commercial games and entertainments, yet only a few studies focus on game-based learning. It has been one of the research hotspots to integrate the excellent human-computer interactive methods with educational theories for the purpose of obtaining effective learning through playing. Our research of application in KINECT motion sensor technology is required to be strengthened especially for the application in gameoriented learning environment by educational researchers in terms of successful application in foreign countries and advantage analysis of domestic initial-starting.

GAME-BASED LEARNING ENVIRONMENT
The game-based learning is distinguished by rules, competition, educational goals, learning objectives, learning outcomes, feedbacks, conflicts, interactions and typical stories. [18] Theory of role-playing supports most elec-tronic games theoretically. Experience and interactions is particularly important in the game-based learning. Educational games and game-based learning environment have made significant achievements in education, but there are still some shortcomings. [19][20] [21]For example, it is not conducive to the health of the learners by using the mouse, keyboard and touch screen to interact; Highly immersive virtual environment is still unable to achieve the actual experience of the natural man-machine interactions; Voice can not be controlled in the operation of games; In terms of interactions, man can not get rid of the auxiliary handheld operating devices for computer and television; It is unable to achieve a flexible health mode of operation; Multiplayer's can not interact in real space at the same time. By building a game-based learning environment, KINECT motion sensing technology can effectively solve the problems as above, whose support is reflected in the following aspects: A. Movements and learning are closely combined.
KINECT combines sports with study, establishes interactions between sports and the learning system, hence greatly improves learners' physical quality while they are acquiring knowledge.

B. Health and safety interoperability.
In the learning environment based on the KINECT interactive technology, learners operate the system with body languages to obtain game experience, without touching any auxiliary equipments like mice, keyboards, handles or touch screens, which is helpful to the learners' health, especially when the KINECT focuses more on the requirements of sterile medical environment.

C. Multi-sensory interactions enhances the sense of actual experience in game-based learning.
KINECT voice recognition and body language recognition are used in game-based learning environment to enhance the sense of actual experience. Learners communicate with the characters in the game with voice or body language, such as shaking hands, saying hello, which can make up and expand the traditional games interactions.
D. Flexible motion interactive experience promotes the understanding and mastery of knowledge and skills. In the game-based learning environment, KINECT motion sensing offers an incredibly immersive gaming experience. Learners move and interact in the same mode as the reality using the various parts of their own bodies, which enable them to quickly grasp the abstract knowledge and skills. Take the Tennis motion Game for example, learners play tennis as usual with the hand movements to interact with the computer so that they can quickly grasp the tennis skills.

E. Multiplayer can experience the interaction at the same time.
KINECT can acquire more than one players' motion to realize multi-learner's competition and cooperation in the game-based learning. When facing complex problems, multi-learners accomplish the job by collaboration; meanwhile, multi-learners compete with each other in the game to get their evaluation, learn from each other and improve together.

A. The objectives and structures of system design
Based on the combination of KINECT motion sensing and virtual reality technology, this paper designs the game "Mulan Joins the Army "(from a Chinese historical legend) for primary school students to role-play. During this game, learners can dialogue with ancient characters through voice recognition to experience the heroine's hardships hard-working and filial virtues, and her unwillingness to be left behind by the men in a motion interactive way. Gradually, players easily accomplish their knowledge construction in an environment filled with entertainment, games and fun. The design is divided into two parts.
(1) Hardware and software tools In terms of hardware equipments, such as Nintendo Wii, Sony PS equipments, Idoing, CyWee, Eedoo, iSecMove and so on. The study shows that Microsoft's KINECT motion sensing is way better than the other environments referring to the operation and control of the entire game learning environment, which has superiority in the function of motion sensing identification, sensitivity and the development of open SDK; Currently there are many development tools for game-based learning environment, for example VRP, Converse3d, WebMax, and Adobe Flash, Flex, Virtools, Quest3D, Unity3d, etc. The paper develops the game adopting Unity3D virtual reality engine that is considered to develop fast and of high performance-price ratio, and is based on Windows7 system. (2) Design Strategies The motion game based interactive learning environment uses 3D virtual reality technology to reproduce the scene of the story, to increase its immersions; KINECT device captures participants' bone trajectory, and defines their sports body language with fuzzy calculation strategy so that participants can interact with virtual roles in the system with voice recognition and gestures. Taking students' motion sensor interactive as an introduction, this study puts the traditional story of " Mulan Joins the Army " as the main line to make the learners melt into the game-based learning environment and expand their deep understanding of the history and culture after their own experience and feelings. Students learn in game-based environment by experiencing their senses of vision, hearing, and touching, which is beneficial for their development of studying initiative, self-learning ability, and creativity.

B. The working process of the system
The operation of the system is a process during which users experience and response to the interactive learning environment based on the KINECT motion sensing equipments and Unity3d procedures. As shown in Figure  1, infrared projector emits safety infrared laser to project on the interactive objects in order to get their 3D depth and different light spots. Then according to the spots, the infrared camera can distinguish objects' predetermined motions, and capture their voice by KINECT's microphone array. Meanwhile the RGB camera get objects' moving images and process these images with Microsoft's KINECT program to send the specific control or interactive commands to the Unity3d-based learning environment; A custom action library is founded to match objects' action in the game learning environment. The operation correction procedures can correct the objects' movements, and prompt them act and speak correctly for the ideal interactions and communications with the characters, which can ensure the objects highly immerse into the learning environment; Finally objects complete the construction of their own knowledge through interactive game-based learning environment which is full of entertainment and fun.

VI. APPLICATION AND ANALYSIS OF CASE
In this study, the system mainly develops along the story of "Mulan Joins the Army", and is divided into six sections including "Farming Men and Weaving Women ", "Join the Army for father" Market Shopping", "Readiness Training", "Bloody Battle ", " Homecoming Gloriously". In accordance with the order of the story, pupils act as the leading role and interact in the KINECT system with their motions and voice. During this progress, they can experience the peaceful life, the brutality of war, the joy of homecoming, the elegance of being better than the men, most importantly they can also acquire knowledge and skills.
At the beginning, Farming Men and Weaving Women describe the pupils a quiet picture of life in the Northern Wei Dynasty where men tilled on the farm and women wove at home. In this section, pupils can learn the people's hardships in the feudal periods and taste their sweet and quiet life. Once a pupil makes action of weeding, his corresponding virtual character in the game environment also weeds with a hoe in his hand in the fields, whose action frequency is synchronous with the pupil's. As shown in Figure 3, a girl is experiencing weaving like Mulan. When she raises her hands on his chest to make weaving action, SHORT PAPER THE APPLICATION OF KINECT MOTION SENSING TECHNOLOGY IN GAME-ORIENTED STUDY the screen shows Mulan sitting before weaving loom scenario at the time.
"Join the Army for father" is mainly carried out in dialogues. Pupils are supposed to be Mulan and have dialogues with her father and brother to get the feeling of traditional Chinese filial duty.
"Market shopping" is some kind of a logic game. Pupils use hands instead of the mouse to interact with the game to set the location of the market according to the up North, down South, left West, and right East direction. Students are asked to choose the correct items from various markets in accordance with the contents of the story. And only by doing this, can the whole experience continues to be carried out.
In "Readiness Training", students practice the archery and sprint skills in the training field. If a student makes an action of pulling the bow, Mulan will do the same action and shoot the scarecrow in the training field in the screen. As shown in the figure 4, when the pupil spurts sprint, Mulan will do the same to sprint the scarecrow. As long as achieving the certain number of physical sprints, Mulan's skill value will increase to continue the game. After this section, pupils will know the importance of practicing the necessary skills in the battle.
"Bloody battle" is carried out under simulated conditions of the war. Once the pupil makes an action of attacking, Mulan and other soldiers will ride horses with their sprints at hand, rushing to the enemy in the scene. This teaches the students that only bravely fighting the enemy can people win the victory. Pupils can also experience the brutality of war and learn that people's desire for peace.
"Homecoming gloriously" is also carried out with dialogues. Students can experience the excitement of returning home after 10-year's triumph, and the homesickness, the proud of being better than men, the beauty of peaceful life, so that they will love their country more.
The whole interactive game-based learning "Mulan Join the Army" makes students go through Mulan's story, learn about things like the ancient people's hardships, the brutality of war and so on, unconsciously complete their own knowledge construction in a funny and interesting learning environment, which reflects the "edutainment" learning effect.

VII. CONCLUSION
KINECT motion sensing game-based learning environment offers a new sensory experience for the students to fully mobilize all senses to get involved in learning and interactions. It is a leap forward and expansion for the traditional game-based learning environment. It draw near the distance between the students and the knowledge, because of which sports and learning are closely integrated to enable students to participate directly in to the entertainment-oriented learning and take advantage of the educational potential of the games to promote the students to construct knowledge. Motion sensor features in the requirements of learners' sportive holistic health interactive, which provides a new way for the expression of knowledge and information and will have a profound impact on the design of the game-based learning environment.