A Virtual Reality System for Training Operators

This paper presents an immersive system for virtual reality-based training of operators. Recent studies show that virtual reality simulations may be helpful in different fields, namely, industry and education. In this particular case we present a simulator for the training of tower cranes operators reducing their costs and risks. This immersive system integrates real-time 3D computer graphics with other modalities, such as, user pose tracking, stereoscopic headmounted display, pendant remote controller, physics behaviour, and spatial sound, in order to create a more realistic and immersive experience to the user. The results demonstrate two different simulator setups, one fixed and one portable, where the fixed offer a larger traceable area and the portable can be moved wherever is needed. The simulator is robust, reliable and provides an experience very similar to reality. Furthermore, it is possible to extend the simulation to other types of cranes and different types of scenarios.


I. INTRODUCTION
Many recent technological advances like powerful portable computing devices, wearable sensors, wireless communications, haptic devices, or lightweight HMDs are opening the frontiers of the development and use of Virtual reality (VR) systems [1]. This enables VR to provide more realistic immersion feelings in more naturalistic environments. These environments can now be enriched with more ecologically relevant stimuli, embedded in meaningful and familiar contexts..
A VR system typically delivers left and right eyes' images, which are generated through the graphics pipeline of a computer system, and updated in real-time. A computer maintains a database that describes each particular scen, through a description consisting in geometric, radiant, acoustic, behavioural and physical information about the set of objects that compose that scene. As a result, the participants see renderings of the database that consist in perspective projections of the 3D models onto the 2D displays that vary according the viewpoint as he/she moves along the scene. This point of view is determined by the head position and orientation of the participant, which in turn must be estimated in real-time by some tracking system [2].
The immersion level that these systems provide depends on what the virtual environment can deliver. For example, the extent of field of view, the number of sensory systems it simulates, the quality of rendering, the extent of tracking, the realism of the displayed images, the frame-rate, latency, and so on [3].
Several applications of interactive visual simulation are currently in their production stages, and these include ve-hicle simulation, entertainment, vehicle design, architecture design and spatial arrangement, medical training, and psychological treatments [4,5,6]. Depending upon the applications, the interactive visual simulators provide different degrees of fidelity for the simulated scenarios. Among these systems we can say that the flight simulators are one of the most mature and representative applications.

II. VIRTUAL REALITY IN EDUCATION AND TRAINING
There are various motivations for the use of VR-based systems in many different areas for the education or training of people. On one hand, it enables the exploration of environments, places or systems, which are not available for the common student or trainee. On the other hand, it allows the experimentation and development of skills without any risk of injury for the trainee or other person, or damaging of sensitive material or goods. Similar to the process of training flight pilots, the training of tower crane operators is time-intensive, expensive, and potentially dangerous. The time element results from a trainee needing considerable practice to develop the coordination required for safe and efficient 3D manoeuvring of massive objects. The trainee must learn to transport objects with tolerable sway and to place them with minimal superfluous motion of the crane. The costs are related to the need to effectively disable a productive crane so that a trainee can use it to practice moving loads. The danger results from an unskilled trainee manoeuvring large pieces of equipment in a confined space in which other equipment is located and individuals may enter. While improved training exercises will likely decrease the training time, a productive crane will still need to be taken out of service and trainees will still need to experience difficult situations using expensive machinery. Thus, the development of an immersive system using VR for training proposes may be a useful tool in industry areas.
This study aims to develop an immersive virtual reality simulator for the training of tower crane operators. This system may help trainees to improve their skills, selfefficacy and confidence while operating a tower crane.

III. DEVELOPING AN IMMERSIVE VR SYSTEM
The proposed system aims to induce on the user the complete sense of immersion, to make the training process as similar as possible as the one achieved using a real tower crane. For this, the user will receive both visual and auditory stimuli through a Head-Mounted Display (HMD). As stated by Slater [2], visual motion and audible cues are more important for the immersion of a user than the level of detail of the rendered scene. Furthermore, these devices are commonly equipped with Inertial Measurement Units (IMU) that are used to detect the user movements and make possible the estimation of user pose SHORT PAPER A VIRTUAL REALITY SYSTEM FOR TRAINING OPERATORS from the integration of these signals. This is an important requirement for providing a realistic spatial sensation. It is well known that these types of sensors present drift-related problems that need to be compensated by some other method. Here we propose a computer vision based estimation of the user pose that is used to establish the viewpoint.
In this study we present two distinct setups, one fixed and one portable. In the fixed setup the position of the user is estimated from a network of 5 cameras. The user's head pose is obtained by estimating the pose of a cube, composed by 5 markers, which is attached on the top of the HMD. This is illustrated in figure 1 that shows the fixed setup designed for the simulator that comprises, the headmounted display (1), the markers cube (2), the joystick (3), and the set of cameras (C1, C2, C3, C4, and C5). To free the user from the limited operation area normally imposed by the cables that connect the HMD and the computational system, we opted by a solution that uses a portable computer, which is solely responsible for rendering the scene, whereas the user pose estimation is computed by a desktop computer that sends the estimated values via a wifi link.
In the portable setup the user position is estimated through a single camera placed in the HMD. The user head position and orientation results from the transformation between this camera and a set of marker posters.

A. User Pose Tracking
For the fixed setup, to track the user pose we compute the transformation between the set of five cameras [7] (Figure 1: C1, C2, C3, C4, C5) and user's head. For this homography estimation we used a set of simple markers forming a cube (five in total: front, back, left, right, and top) fixed to the HMD. These markers not only enable fast and robust identification of its points, but also contain a binary code that can be used to distinguish each face of the cube [8].
In the portable setup, the pose is estimated by the transformation between the front camera attached to the HMD and the marker posters in front and sides of the user.
To sum up, both setups return the user head pose the difference is the method to calculate it.

B. Head-Mounted Display
Virtual environments have the ability to provide a user with a sense of being in the real environment that the virtual environment is replicating. A typical virtual environment allows a user to have a 360-degree view via a headmounted display associated with head tracking. The 40degree field of view offered by the Z800 3DVisor is close to natural field of view. The Z800 (Figure 1: 1) also includes stereo ear buds and a microphone, being the latter not used in this setup [9].

C. Pendant/Joystick Controllers
Nowadays operators command crane motion in much the same manner as an operator decades ago through the use of a control pendant or actuating joystick. The commands used are very similar to the real and replicate a realistic experience (Figure 1: 3).

D. Physics Behaviour
When a user pushes a button, the simulator must recalculate the new position of the object according to its current position, velocity, acceleration, altitude, wind speed, and gravity. For this purpose, a physics engine was needed and the Bullet Physics Library was used. This is integrated in OpenAR [10], which was the graphics framework used for the development of the simulator. The OpenAR has been developed in the laboratory and it is based on OpenGL.

E. Stereo Sound
Localizing objects from stereo sound is a plus to the sense of presence and immersion. Because of that most of the objects in the scenario have their own sound providing a better spatial experience.

F. Operator Mobility
One of the common problems of HMDs is the wire length that constrains the area and movements that the operator can perform. To solve this issue, we have opted for the use of a backpack that carries a laptop computer responsible for the rendering of the scene on the HMD and generation of ambient sounds. This laptop is connected via Wi-Fi to a second computer, which receives the cameras' streams and computes the user pose in real-time.

IV. SIMULATION
The simulator consists essentially in two scenarios. In the first one, the crane can be operated from the ground, where the trainee can move freely around the virtual scenario by moving himself in the real room and exploring different points of view by turning his head around naturally. In the second mode, the trainee controls the crane inside of the cabin where his movements are limited to the cabin size but may move his head to control the point of view in the simulator.
Moreover, this two operating modes are compatible either to fixed or portable simulator setups. Although, concerning real environment space issues, the ground control operation is optimized to the fixed simulator setup, where the trainee has more space to move freely, and the cabin control operation to the portable simulator setup, where the trainee is sited inside of the cabin and can only look around.

A. Interaction and Visualisation
The crane can be operated from the ground using a control pendant. This pendent have all the commands needed to manipulate the crane. Furthermore, the trainee can move freely around the real environment while being tracked by the tracking system. In the virtual scenario all his movements are replicated, including position and point of view (Figure 2: left). In the case of cabin control, the trainee manipulates the crane using two joysticks, similar to real cabin interface. Additionally, the trainee has a limited space to move but he can turn his head around to control the point of view during the simulation (Figure 2: right).

V. CONCLUSION AND FUTURE WORK
This study presents an immersive system for VR training simulations using two setups: fixed and portable; and two training scenarios: ground crane control and cabin crane control. The two setups can be used with both simulation scenarios with their strengths and weaknesses. The fixed setup provides a larger traceable area than portable setup, in other hand, the portable setup advantage is the mobility, i.e., all system can be placed anywhere.
In conclusion, the simulator is robust, reliable and provides an experience very similar to reality. It is possible to extend the simulation to other types of cranes, different types of scenarios and improve the graphical and haptic realism. Thus, this immersive VR simulator represents an important contribution for the use of the VR in the real contexts, such as industry.