Study of the Hand Anatomy Using Real Hand and Augmented Reality

—Anatomy is considered one of the foundation studies for all of the health science students especially medical and nursing students. Anatomy of the hand is complicated. It composes of bones, nerves, blood veins, muscles, and tendon. Memorising all the details about all those parts is tedious work and need much imagination. With the advances in computer graphics and human-computer interaction techniques, understanding how those body parts move is easy to understand in a visual presentation. Augmented Reality (AR) is the technique that allowed the computer-generated objects to overlay on top of the real world. In this study, we concentrate on studying the bones only. We have selected the Leap Motion, which is the device that can detect the hands and fingers, like a tracking device, and marker-based AR technique for displaying the computer generated bones on top of the real hand. Since the Leap Motion detects the hands and shows the bone in real time, so when a user moves the hands such as waving, all the 3D virtual bones move to the new position just like the real hand. Besides using this tool as the educational tool to help the students have better learning about anatomy, it can also be used as an assessment tool for anatomy class as well. Results from testing this tool with volunteer students indicate that it helps them to understand the hand anatomy better and faster than traditional ways.


Introduction
Human anatomy is considered one of the foundation studies for all of the health science students especially nursing and medical students. In the past, cadaver dissection was used in teaching and learning anatomy in every medical school since it helps students learn about human body ranging from skin layer, muscles, and bones that constructed into the human body. Even though learning from cadaver dissections is useful and give a good experience to the students, but there is no clear evidence on the effectiveness of this kind of learning. Besides that, there is some inadequacy to cadaver specimens, so the students will have some limitation to access the specimens.
To solve the problems, some schools start using animal specimens as part of the learning experiences in the laboratory, but with the shape and size, it cannot replace the cadavers. Study of the anatomy can also be done using the figures from the textbook. Usually, trying to figure out the 3D image from the 2D image is not difficult if the object is not so complex such as symmetric-shaped or geometric shape object. In the case of the complex object such as human organs, imagination may not fulfill this purpose. During the past decade, there were tools available to help students understand human anatomy easily such as video or even computer-generated 3D models of the human organs, but the interaction with those media is still limited. With the advances in technologies especially in computer hardware and computer graphics, new techniques for displaying 3D models is possible such as hologram [1], virtual reality and augmented reality. Holography system is the technology that captured the light scattered from the object and present them as a 3D object. This technique requires complex setup and unique viewing device, so it may not be practical to work within the near future. On the contrary, virtual reality and augmented reality are technologies that are available today, and they can apply to many areas of usage ranging from, advertising, marketing, architecture, construction, entertainment, military, travel, medicine and education [2]. By definition, Virtual Reality (VR) is the computergenerated environment that simulates real-life situation. Augmented Reality (AR), on the other hand, is the situation where the computer-generated object is overlaid atop of the real world. Even though AR is considered a form of VR, but they are quite different. In VR, users immersed in the virtual environment but in AR, the user can experience the virtual objects or situations while still in the real world.
As mentioned earlier, AR and VR are different, so applications for AR and VR will cover different aspects. For VR, all applications must be in virtual space, so the users needed the devices that put them in the virtual world. This condition can be done by making the environment around the user to look virtual such as by projecting computer-generated image on the wall around the user as in the CAVE (Cave Automatic Virtual Environment) or by covering the user's eyes to let them see only the virtual world as in the case of using the head-mounted display system. The different between the two is that, in CAVE, the user can look around and see the different view, but in the head-mounted display, the tracker is needed to control the view of the environment when the user turns their heads. For Augmented Reality, the device must display the virtual world and real world in the same scene. Examples of VR applications can be seen in Figure 1 (a) where human anatomy can be displayed, and the user can interact with them in CAVE. Figure 1 (b) shows the AR application in which user with the head mounted device in real space can see and interact with the model of human anatomy that appeared in the virtual space.

Literature review
Even though, many research papers indicated that the use of cadaveric dissection (dissection of a dead human body) is mandatory as a fundamental part of medical training, but due to the limitation of the cadaveric specimen, the expensive in operation, time consuming and potentially hazardous, many medical school in some countries such as Australia and the UK, have stopped using cadavers altogether and move to computer-simulated anatomy instead [3]. Some medical schools also try to balance the use of digital technologies with cadaver specimens in anatomy classrooms. Research on using Computer-Assisted Learning and computer animation in supporting anatomy learning are available for more than 25 years [4]. With the advance in computer graphic technology and the powerful processor, the simulation of human anatomy can replace the cadaveric dissection in many aspects.
Besides computer simulations and computer-assisted learning, the new technology such as AR and VR can be used for anatomy teaching and learn as well. In Virtual Reality (VR), students can study the anatomy of the human body by peeling off the components layers by layers starting from the skin to bones, which make them understand clearly about the human body. Kažoka. D. et al [5] had investigated the usefulness and user satisfaction of using the Anatomage Table (Virtual Dissection Table) in teaching and learning of Human Anatomy course at Riga Stradis University, Riga, Latvia, and found that virtual dissection technology seems to have a promising role in future anatomy training. Fairén et al. [6] had set up the VR system that allowed the nursing students to see and interact with the 3D model of human organs. After the session, the evaluation of the performance along with the student satisfaction of the system was investigated. The results confirm that VR is a powerful tool that helps the students to understand the anatomy of human body. For AR, Chien et al. [7] used an interactive AR technology to help medical students to understand and memorize the 3D anatomy structure. Results from the experiment indicated that the system could help medical students to learn the complex anatomy structure better and faster than the traditional way. Research on AR in medical education [8], has also mentioned the work on visualizing human anatomical structure with AR and concluded that the technology is promising because of its strong visualization. Moro et al. [9] found that both AR and VR, not only give immersive experiences to the students but also increased learner immersion and engagement on the anatomy topics as well.
In the last few years, there has been a lot of research related to using AR in teaching and learning anatomy of the human body but almost all of them related to the models that are static, just like in cadaver dissections laboratory. In some experiments, the student may be able to rotate, enlarge or shrink the model but they cannot experience with the moving part of the body or dynamic model and see how the bones or muscle behaved when they are moving. For the real dynamic movement of the model, Kugelmann et al. [10] used Augmented Reality Magic Mirror to check with 880 first-year medical students to see whether AR can be an additional resource for anatomy course beside cadaver lab. For magic mirror concept, the augment 3D organs or skeletons can be projected on the body of the user at the precise location, so when the body moves, the 3D virtual organs or skeleton will move as well. The evaluation from the students indicated that the system is a beneficial addition to the anatomy class except for the limitation on the visual resolution.
In this research, we are attempting to find the alternative way for the students to learn hand anatomy effectively using AR and their own hands.

3
Conceptual framework To set up the AR experiment on hand anatomy learning that can replace or at least reduce some process in training or learning with the cadaver, three criteria need to be considered. The first criteria are how to capture the image of user's arm and hand. The second criteria are how to map the virtual image of the bones, muscle, etc. in place of the user's arm which is in the real world and the third criteria are how to track and monitor the augmented system when the arm is moving. The three criteria are related to each other as can be seen as a flowchart in Figure 2.

Experimental setup
The experimental setup for studying hand anatomy requires both hardware and software to fulfil the requirement. For the hardware, besides the computer which required high processing power with extra RAM, the setup required webcam for detecting the hand of the user. The webcam is used as a video camera that streams the captured image in real time and sent them to a computer. The specification of the webcam can be varied depending on the requirement of the system, i.e., how far the webcam is apart from the hand, etc. In our experiment, the distance between the camera and the hand is around 1 meter, so we select Logitech c920, which is perfect for our purpose. The other hardware device required for this setup is the "Leap Motion." The Leap Motion is a USB peripheral sensor device that supports hand and finger motions as input but requires no contact. The device consists of two cameras and three infrared LEDs which track infrared light outside the visible light spectrum. The detecting range of the "Leap Motion" is from 0 to around 1 meter from the object. Detail appearance of the Leap Motion is displaying in Figure 3, and the hardware setup for the study of the hand anatomy is showing in Figure 4.
For the software, besides the application program for creating the 3D image of the human body and organs which in this case is Maya, we also need Unity 3D and Qualcomm's Vulforia. Unity 3D is a cross-platform game engine, used for developing 2D and 3D video games and simulations for computers and Vuforia is an Augmented Reality Software Development Kit (SDK) for mobile devices that enables the creation of Augmented Reality applications. This same technique can also be used for the study of the arm and hand muscle and blood vessel as well, as can be seen in Figure 6. Figure 7 shows the ability to interact with the system and get the results in the form of data which can be used for other teaching and learning. For example in student training, information about the hand anatomy can be displayed if the condition in displaying is fulfilled such as when touching part of the bone. Using this condition, we can create a self-tutorial session for the students, or we can generate the quiz for learning assessment, checking whether the students can remember the name of each bone or not.

Results and discussion
In order to verify that this new setup is useful and can help medical students to learn about hand anatomy, we have arranged the usability evaluation test to check how well users can learn and use a product to achieve their goals. Since the usability is a combination of many factors such as ease of learning, ease of understanding, efficiency in use and etc., so we conduct this usability based on these factors. We have selected the health sciences students, i.e., medical students, nursing students and environmental health students as our target group since they have experience in anatomy learning. Eighteen volunteers participate in the activity which covered, listen to the brief explanation of the purpose, participate in using the system and then finally answering the questionnaires. The Likert scale was used for the closed end questions. A Likert Scale is a type of rating scale used to measure attitudes or opinions of the respondents on a level of agreement. The highest score is five, and the lowest score is one. The number of students in each group that participate in the evaluation is displaying in Table 1. Results from the questionnaires for usability and users satisfaction are showing in Table 2. From the results, it is quite clear that the students participate in the experiment are satisfied with the system (4.63/5). In term of usage, students seem to enjoy how easy to learn the anatomy of their hands (4.61/5) plus they can understand the anatomy of hand in action at every angle in any levels from bones to muscle to skin with the blood vessel and nervous system in between (4.44/5). They also prefer using the system for reviewing the parts that are not clear to them at any time and as many times as they wanted (4.56/5). Comparing the system with the textbook or other media, the participants indicated that the system could be used as the substitute for those media (4.39/5). For the efficiency of the system, the result indicates that it is suitable for learning hand anatomy (4.56/5).

Conclusion
From the results, we can conclude that the system for studying hand anatomy using AR seemed to be the best choice for cadaver laboratory replacement or partial replacement which can save both time and budget for anatomy learning. Even though the system is useful for studying the anatomy of arm and hand, but anatomy learning must cover the whole body, so this limitation needs to be solved. Since this limitation is due to the ability of Leap Motion in sensing, so a new sensor is required. The best candidate of the sensor that can cover the whole body is Microsoft's Kinect. Kinect already has skeleton tracking of the whole body, so using them for studying anatomy in action is possible. The only drawback may be the resolution which may not satisfy the users. With the advance in technology both in hardware and software, learning anatomy with AR may soon replace cadaver completely.