A New Physical Education Teaching System and Training Framework Based on Human-Computer Interaction and Auxiliary Interaction

—There is a major problem with off-class learning of physical education (PE): the lack of guidance on the students. This paper attempts to solve the problem, better arouse the interest of students in PE, further improve the quality of PE teaching, and promote the all-round development of students. After summarizing the traditional PE teaching and training framework, the authors constructed a universal PE teaching assistant training framework, based on motion capture and virtual reality (VR). The proposed framework is applicable to the PE teaching system embedded with human-computer interaction (HCI) and auxiliary interaction. Then, the PE course of martial arts was taken as the research object. Based on the learning requirements of martial arts learners and the function requirements of martial arts teaching system, the authors designed and implemented a PE teaching system based on HCI and auxiliary interaction. The results show that, through HCI, the proposed system can provide feedbacks to the real-time performance of learners, and enable the learners to find their errors in martial arts movements; all the design goals were realized by the system, namely, natural HCI, and assisted training and scoring of martial arts movements; the system clearly improves the learning experience and learning efficiency, and the research provides a good reference for other PE teaching systems.


Introduction
The motion capture technology is a means of digitally recording the positions and postures of moving objects using sensors based on the principles of computer graphics technology [1]. Decades ago, scholars have started the research on motion capture and now this technology has been widely used in various fields. Kinect, a somatosensory device, has realized somatosensory control without controller, making the application of motion capture technology easier, at the same time, it has also changed the status quo of HCI [2]. As the motion capture technology is developing and becoming more mature, now more and more scholars are engaged in the research of assistant training systems based on motion capture.
The VR technology can realize the natural interaction between humans and computers and the transmission of various information via various auxiliary equipment [3]. With the advancement and ripening of VR technology, combining VR with PE could provide learners with richer extracurricular learning resources and better learning environment. The interactions between the learners and the virtual environment can stimulate learning enthusiasm and initiative, and they facilitate learners to intuitively understand the shortcomings of their movements and the differences between their own movements and the standard movements; in this way, exercises and the selfevaluation of learners have been well combined, which can drive the learners to quickly master the learning content and their learning efficiency has been improved greatly [4].
To this end, this study built a universal PE teaching assistant training framework based on motion capture and virtual reality. With Kinect as the system development platform and Unity 3D as the 3D modeling platform, this paper designed and implemented a PE teaching system based on HCI and auxiliary interaction after carefully analyzing the learning requirements of martial arts learners and the function requirements of martial arts teaching system.

2
The Universal PE Teaching Assistant Training Framework

The traditional PE training framework
In traditional PE training exercises, PE teachers formulate the teaching goals and plans according to the syllabus and the required teaching content, then they teach sports knowledge and skills to students in the class, and make them do the corresponding exercises. In this process, when there're errors in the real-time performance of the students, PE teachers would give targeted guidance and evaluation until the students can master the learning content and meet the training standards, and then move on to the practice of the next movement. The specific training framework is shown in Figure 1 [5].

The universal PE teaching assistant training framework
Based on the framework of traditional PE training, combining with motion capture and VR, this paper established a universal PE teaching assistant training framework that is applicable to PE teaching systems embedded with HCI and auxiliary interaction, as shown in Figure 2 below.

Fig. 2. Framework of universal PE teaching assistant training
The universal PE teaching assistant training framework has four modules: controller, domain knowledge, learner, and sports evaluation [6], as shown in Figure 3.
Controller module: The main function of the controller module is to provide corresponding feedback instructions for the movement completion level of the learners to guide them to better complete the learning of the course content. After the learning of current content is finished, this module provides new learning content or related learning content, and updates the skill level of the learner after evaluation. Domain knowledge module: Since different sports programs have different domain knowledges, the content of this module should be developed after consulting experts in various fields. The domain knowledge module includes three parts: reference library, practice library, and instruction library [7]. The reference library should contain the scores of various skills of higher-level athletes or expert coaches of certain sports programs, the skills and the learners' exercise contents should be coupled to facilitate subsequent comparison and scoring of the learners' exercise situations.
The practice library should be divided according to the difficulty level, and it should cover all the exercises involved in the sports programs. The interactive exercise contents of different difficulty levels should be separated from the standard exercise contents, that is, a learner's standard exercise scores will not affect his/her scores in the real-time performance of the exercise contents.
The instruction library stores all the feedback instructions that the system may give for the learner's real-time performance when the learner interacts with the system. Of course, since the instructions contained in the instruction library may not cover all the errors that may occur during the learner's practice, the construction of the instruction library is a process that needs to be gradually improved.
Learner model: The learner model mainly refers to that, when learners use the system for interactive learning, the system gives feedback instructions for the realtime performance of the learners, and when learners use the system under the practice-only mode, the system makes comprehensive evaluation and gives feedback instructions to the learners' skill levels.

Sports evaluation module:
Sports evaluation is the basis for learners to obtain feedback instructions and skill-level updates during practice. Since the evaluation standards of different sports programs are different, they need to be determined after consulting professionals. Sports evaluations can be divided into the following three levels [8].
Motion capture: use motion capture technology to capture the movements of athletes or expert coaches or senior learners, and obtain relevant data of the sports programs such as the skeleton posture and movement speed of the learner, and the movement trajectory of sports equipment (e.g., ping pong ball, ping pong racket) during exercise.
Feature extraction: dynamic and kinematic features and time correlation are the main contents of feature extraction of sports programs. According to the features of different sports programs, after analyzing the movement data obtained in motion capture, the real-time performance of the sports programs could be obtained.
Sports evaluation: compare the movement features of learners during practice with those of professional athletes or coaches in the reference library, analyze, and give scores and feedback instructions to the learners' real-time performance.

Design and Implementation of the PE Teaching System Based on HCI and Auxiliary Interaction
Based on the universal PE teaching assistant training framework proposed in previous chapter, this paper took the sports program martial arts as the research object to carry out research on the design and implementation of the PE teaching system based on HCI and auxiliary interaction.

Requirement analysis of the martial arts teaching system
As a learning tool, the proposed system should be able to meet the requirements of the teaching content, teaching process, and learner needs of the martial arts, therefore, it's necessary to analyze the requirements of the martial arts teaching system.
Learning requirements of martial arts learners: This study selected 132 students who had taken the martial arts elective course in a university as the research objects, and conducted a questionnaire survey on their learning requirements for martial arts. A total of 124 valid questionnaires were collected, with an effective rate of 93.9%. Figure 4 shows the main problems existing in the classroom teaching of martial arts course, according to the figure, 88% of the students think that due to the limited class time, it is difficult for them to remember the martial arts movements taught by the teacher; 81% of the students think that teachers cannot give targeted guidance to their movements, so they don't know whether their movements are right or not; 75% of the students think they can't keep up with the teacher's speed and they haven't mastered some of the movements; and 62% of the students feel that teachers mainly rely on their own experience in evaluation, and their judgements are quite subjective.  Figure 5 shows the survey results on factors that have restricted off-class selflearning of martial arts, according to the figure, 85% of the students have the intention to practice martial arts by themselves after class, however, they couldn't practice because there's no one to instruct them; 82% of the students wish to practice martial arts in training class, but it couldn't be realized because of the high training cost; moreover, students who are not interested in martial arts and who do not have time to practice martial arts accounted for 52% and 53%, respectively. Function requirements of martial arts teaching system: According to the analysis of the learning requirements of martial arts learners, at first, the proposed system should meet the requirements of PE teaching, provide learners with the necessary teaching resources of martial arts, mainly the standard martial arts movement demonstrations; besides, the system should also have other teaching resources such as martial arts movement decomposition teaching videos, images, tips, and introduction of the history and culture of martial arts, moreover, functions such video playback, slow play, pause, fast play should also be provided to facilitate students' learning [9]. Secondly, the proposed system should be able to assist the evaluation, score the learner's movements, and correct their errors, in this way, the learners can understand the differences between their movements and the standard movements, figure out their shortcomings, do self-correction exercises, and achieve the learning goals; at last, the proposed system should be able to display the martial arts movements of learners in 3D animation, then the 3D animation of the learners could be compared with that of the standard movements, this not only adds fun to the martial arts learning, but also allows learners to observe their own movements from all directions and find out their errors.

3.2
Design of the martial arts teaching system System development goals: Based on above learning requirements and function requirements, this study adopted the motion capture technology to collect martial arts movements through the depth sensor Kinect, a few martial arts practitioners and experts were invited to assist the construction of the standard database and reference database [10]. The proposed system has many functions; besides the teaching of regular martial arts movements, it also has contents such as the introduction of the history and culture of martial arts, the movement decomposition and tips, and the images of the movements, etc. Through HCI, the system gives feedback instructions to the realtime performance of the learners, and the feedback instructions include the scores of the completed movements, the text prompts for wrong movements, and the demonstration of correct movements performed by virtual figures generated by the 3D rendering technology, in this way, the system can realize the goals of natural HCI, and improving learners' learning experience and learning efficiency.
Function modules of the system: Based on system requirement analysis and system development goals, this paper designed four function modules for the proposed system, as shown in Figure 6.
Teaching module: The teaching module provides various teaching resources related to martial arts, including video guidance of standard martial arts movements, images of decomposed martial arts movements, and various videos, texts, and images of martial arts. Students can make use of these resources for their martial arts learning.
Movement collection module: The movement collection module has two parts, one part collects the movement data of senior martial arts practitioners and experts, and builds the standard database after data processing; the other part collects movement data of martial arts learners, compares them with the standard database after data processing, and waits for scoring [11].

Fig. 6. Function modules of the martial arts teaching system
Testing & scoring module: This module compares the learner's movement data with the movement data in the standard database, gives analysis, and give scores according to the setting in the system; at the same time, it proposes improvement suggestions for errors existing in the learner's movements, and gives demonstrations of the correct movements to help learners do correction exercises.
3D reconstruction module: The 3D reconstruction module uses collected learner movement data to generate virtual figure model, and restores the learner's martial arts movements in the form 3D animation, learners can watch the animation of their own martial arts movements and compare with the standard martial arts movements in the teaching videos, this form improves learning efficiency and increases the fun of martial arts learning [12].

Detailed design of the system
With some system functions as examples, here we give a part of the detailed design contents and processes of the proposed system.
Human body posture feature extraction: Joint angle feature extraction. The extraction of human body posture features is an important link to realize the HCI and auxiliary interaction of the martial arts teaching system. Figure 7 shows a schematic diagram of the joints of human skeleton. The 3D coordinates of the 20 joint points can be obtained using Kinect [13]. However, not every joint point is meaningful for the description of the movements, therefore, according to the characteristics of the martial arts movements, this paper chose 8 joint points for the joint angle feature extraction and analysis, including knee joints, hip joints, elbow joints, and shoulder joints [14].
where, d(,) is the Euclidean distance of the two vectors; SJoint,i is the instantaneous speed of the Joint (any joint point) in the i-th frame; PJoint,I is the space coordinate of this joint point in the i-th frame.
System scoring design: Based on the features of martial arts movements, this study selected the differences between the learners' movement sequence, movement sequence speed, movement duration and the standard movement sequence, movement sequence speed, movement duration as the scoring criteria, after comprehensive evaluation, the system calculates and gives the scores. Figure 8 shows the flow of the scoring module, and Figure 9 lists the rules of the scoring algorithm [16].  Calculation of human body posture angle difference: Assume A and B are respectively a test sequence and a standard sequence, A=(A1, A2, …, Am), B=(B1, B2, …, Bn); their lengths are m frames and n frames, respectively; and there are k corresponding frames between the two, then there is [17]: where, DAngle is the Manhattan distance between sequence A and sequence B; Bi is any frame in sequence B, θi is the angle eigenvalue corresponding to Bi; B'i is the frame in sequence A corresponding to Bi; θ'i is the angle eigenvalue corresponding to B'i; DiAngle is the Manhattan distance between θi and θ'i, which is used to describe the distance between angle eigenvalues.
Calculation of human body posture speed difference: The calculation principle of human posture speed difference is similar to that of the human posture angle difference, and the calculation formula is [18]: Calculation of the scores: According to the mapping formula (4), the distance value between two movement sequences obtained above is mapped into the score. The smaller the distance value, the more similar the two sequences, and the higher the score [19].
where, α and β are mapping parameters, which should be reasonably determined through experiments, and S is the score of angle or speed.

3.4
Realization of the proposed system System implementation flow: According to the overall architecture design and detailed design of the proposed system introduced above, this study adopted Kinect to capture the learners' martial arts movements in real time, Figure 10 gives the flow of system implementation.

Fig. 10. System implementation flow
Learners can use the learning module to view texts, images, and videos of the teaching resources and learn by themselves, or they can practice the movements according to the learning content, the depth sensor Kinect in the human movement sequence collection module will collect the data of their skeleton joint movement sequences, then the system will analyze and compare the data with the data of martial arts professionals, and calculates and scores the performance of the learners. The higher the score, the more standard the learner's movement are (full score is 10 points), and meanwhile the system will also give feedback about the errors in the learner's movements during practice for learners to correct them. In addition, the 3D reconstruction module will drive the model in Unity 3D according to the learner's skeleton movement sequence data to reproduce the learners' martial arts movements in 3D animation, then, the learners can compare their movements with the standard movements and find out their errors, in this way, their learning efficiency can be improved and the fun of learning has been increased.
Collection of movement data: After the course is over, learners can go to the martial arts practice classroom and let the movement collection module collect the data of their movements. They only need to complete the martial arts movements to be practiced within the recognizable range of Kinect, the system will mark and capture the movements of the main joint points, and collect and record the data. Figure 11 shows a photo of martial arts movement data collection. Fig. 11. The system is collecting data of the martial arts movements of a learner Fig. 12. Skeleton structure of the learner's movements Figure 12 shows the skeleton structure generated by the system based on the collected data of the learner's movements.
3D animation: After the learner skeleton data collected by Kinect was processed, it will be transmitted to Unity 3D and drive the model in Unity 3D to reproduce the learner's martial arts movements in 3D animation, as shown in Figure 13. Fig. 13. Reproduce the learner's martial arts movements in 3D animation

Realization of system scoring function:
After the learner has completed the martial arts exercise, the system will extract the eigenvalues of the movement data, compare them with the eigenvalues in the standard database, and calculate the scores. At the same time, the system will give corresponding feedback instructions to the errors existing in the learner's martial arts movements during the exercise, learners can click to view details and get information such as the difference between their movements and the standard movements, and the images and videos of the standard movements. Figure 14 shows a screenshot of the evaluation results of a learner's martial arts exercise.

Conclusion
This paper summarized the traditional PE training framework, and researched a new PE teaching system and training framework based on HCI and auxiliary interaction to trigger students' interest in sports, and improve learning efficiency and teaching quality. The specific conclusions are as follows: 1. Based on the traditional PE training framework, this study adopted motion capture technology and VR to construct a universal PE teaching assistant training framework for the PE teaching system based on HCI and auxiliary interaction. 2. With the sports program martial arts as the research object, this study conducted a questionnaire survey to figure out the learning requirements of martial arts learners and the function requirements of the martial arts teaching system, then, it's determined that the proposed system should have functions such as providing learners with teaching resources of martial arts, assisting evaluation, and reproducing the martial arts movements in 3D animation 3. This study designed and implemented the proposed PE teaching system based on HCI and auxiliary interaction. The research results showed that the proposed system achieved the design goals of natural HCI, assisting training, and giving scores, and it can improve the learning experience and learning efficiency of learners.