HoloLens, Augmented Reality and Teamwork: Merging Virtual and Real Workplaces

— Two projects are introduced: a cooperative game aimed at fostering behavioral, cognitive and meta-cognitive skills within organizations (Holo-Transportation), and a technical procedure aimed at saving lives (Holo-Defibrillation). e-REAL were the educational setting: an environment where physical and digital objects co-exist and interact in real time. Both the two projects were self-instruction learning simulation programs developed to maximize learning results, reducing instructor intervention and cost. Cascading and growing technical know-how about defibrillation were the main target of Holo-Defibrillation, while Holo-Transportation were aimed at empowering teamwork by fostering cognitive flexibility, emotional intelligence and self-awareness, as well as at honing the epistemic acumen related to reviewing ongoing processes and procedures, errors and alternative options.


1
Logosnet: e-REAL Immersive Simulation Labs, Houston, USA, Lugano, Switzerland, Torino, Italy 2 Simulation Center in Medicine and Health Science SIMNOVA, Università del Piemonte Orientale, Novara Italy 3 Department of Control and Computer Engineering DAUIN, Politecnico di Torino, Italy

Abstract-Two projects are introduced: a cooperative game aimed at fostering behavioral, cognitive and metacognitive skills within organizations (Holo-Transportation), and a technical procedure aimed at saving lives (Holo-Defibrillation). e-REAL were the educational setting: an environment where physical and digital objects co-exist and interact in real time. Both the two projects were self-instruction learning simulation programs developed to maximize learning results, reducing instructor intervention and cost. Cascading and growing technical know-how about defibrillation were the main target of Holo-Defibrillation, while Holo-Transportation were aimed at empowering teamwork by fostering cognitive flexibility, emotional intelligence and self-awareness, as well as at honing the epistemic acumen related to reviewing ongoing processes and procedures, errors and alternative options.
Index Terms-Holo-Transportation, Holo-Defibrillation, Mixed Reality, Simulation, e-REAL.

I. PROJECTS' OBJECTIVES
Two projects are introduced here: Holo-Trasportation, a cooperative game aimed at fostering behavioral, cognitive and meta-cognitive skills within organizations, and Holo-Defibrillation, a technical procedure aimed at saving lives within healthcare.
Both the two projects were self-instruction learning simulation programs developed to maximize learning results, reducing instructor intervention and cost.
Microsoft HoloLens technology and other new AR caps were used, enabling users to interact with high definition holograms.Learners used natural gestures, body movements and spoken commands to perform their tasks, with virtual 3D objects anchored to the learning environment.
The projects' environment was e-REAL, that is the merging of real and virtual worlds: an environment where physical and digital objects co-exist and interact in real time.e-REAL enables a number of activities using gestures and spoken commands: it submerges learners in an immersive reality-context related-where the challenge at hand is created by sophisticated, interactive computer animation in 3D and holographic projections.In such a way, not only systemic understanding and situational awareness but also relevant cognitive aids are provided.
e-REAL is an environment where learners have to face challenging situations that require, in order to be appropriately managed and solved, an effective teamwork aimed at facilitating complex tasks by generating repeated loops of observation, decision, action, and re-evaluation.
The success factors within e-REAL are the ability to communicate, negotiate and cooperate in order to translate the knowledge of what needs to be done into effective team activity 1 .
II. HOLO-DEFIBRILLATION Providing an effective cognitive aid to both lay people and healthcare providers in Basic Life Support and Defibrillation (BLSD) procedures is a relevant educational goal in resuscitation. 2 A self-instruction learning simulation program was developed to maximize learning results, helping to enhance motor skills and the retention of cognitive knowledge, reducing instructor intervention and cost.Microsoft HoloLens technology was used, enabling users to interact with high definition holograms.The solution was developed as a mixed reality self-instruction training environment allowing assessment, using a standard low-cost CPR manikin to deliver tactile information.The manikin was "augmented" by interactive virtual environment reproducing realistic scenarios.Learners used natural gestures, body movements and spoken commands to perform their tasks, with virtual 3D objects anchored to the manikin and to the environment.We believe such a project is the first in the domain of BLSD.
Holo-Defibrillation is an interactive mixed-reality learning environment, providing learners with real-time feedback.In training mode, a self-instruction program is provided.After a short induction phase, users are guided through the sequence of actions required.Non-player characters (NPC) with different roles (by-standers, patient relatives, medical team members) deliver information or assist the trainee during the procedure.In evaluation mode, no guidelines are given, as the system only records and analyzes actions performed by the assessed trainee.MR techniques provide more interactivity and increase realism, by enriching the real world with digital information registered into the user's perception of the environment. 3,4,5,6n order to assess the prototypal system, we performed a qualitative evaluation with a user panel.At the end, participants were given a questionnaire asking them to rate a number of statements using a 5-point Likert scale (1: strongly disagree, 5: strongly agree).
Our test is ongoing: 23 users (4 doctors and 19 residents) participated in the early experiments by testing the application once in training mode and filling the questionnaire.All of them had previous BLSD training and never (or rarely) used an AR application.Results showed a general appreciation of the system (average rate 4.7), which was found pleasant (4.1) and easy to use (4).The cognitive load required to operate the HoloLens was minimal (1.8): users could focus on the BLSD procedure and on carrying out actions requested.Voice (4.4) and gesture interaction (3.8) were evaluated positively, although some hand recognition errors reduced the feeling of having the right level of control over the system (3.4).Virtual contents were found realistic (3.8) and the reduced field of view of the HoloLens was not perceived as a severe limitation (1.3).Participants judged the system as capable of providing a real benefit as a training tool (4.2) and to help them to be more effective (4.2).
Our findings show that an interactive mixed reality learning environment could be an effective training tool for teaching BLSD procedures to both lay persons and healthcare providers.Holo-BLSD is a self-instruction learning program that requires minimal instructor intervention and, thus, can help to maximize learning results reducing the training cost.As major features, the system offers realistic haptic feedback through a CPR manikin and a virtual scenario that can be easily reconfigured to generate many different situations, including extreme and dangerous ones.All the data generated and the events occurring during the simulation (actions performed, NPC data, victim vital signs) are logged allowing instructors to immediately identify errors and difficulties of the trainees.Logging can also support an effective debriefing and, eventually, the creation of a library of training events.Future work will include a thorough assessment of both the training and evaluation tools proposed.

III. HOLO-TRANSPORTATION
Holo-Transportation is a cooperative game aimed at fostering behavioral, cognitive and meta-cognitive skills.A self-instruction learning simulation program was developed: deep spatial navigation is the simulated environment where 2 teams, comprised of minimum 2 people each, have to cooperate sharing information and knowledge relevant to navigate in the right way.The teams are in different places and can't directly look at each other: team wearing the HoloLens has to provide the most appropriate info to the other team, that has to manage the space navigation by the commands available on a big surface they are facing (part of an e-REAL interactive wall).Only in a few special situations the "drivers" are shown with images and recordings from the other team working with the HoloLens, sharing in such a way the same visual outputs.Learners are asked to make critical decisions, recognize a situation requiring rapid intervention, communicate and cooperate to share relevant information and knowledge, manage unforeseen events, deciding between parallel processing or performing one task at time in a sequence-taking into consideration critical contextual factors such as a lack of time, scarcity of resources and tools, previous impacting factors.
In order to assess the prototypal system, we performed a qualitative evaluation with a user panel.At the end, participants were given a asking them to rate a number of statements using a 5-point Likert scale (1: strongly disagree, 5: strongly agree).
Our work is ongoing: 88 users (across industry: healthcare, energy, banking, etc.) participated in the early experiments by testing the application and filling the questionnaire.All of them never (or rarely) used an AR application.Results showed a general appreciation of the system (average rate 4.4), which was found pleasant (4) and easy to use (3.7).The cognitive load required to operate the HoloLens was minimal (1.8).Voice (4.8) and gesture interaction (4.8) were evaluated positively, although some hand recognition errors reduced the feeling of having the right level of control over the system (3).Virtual contents were found engaging (4) and the reduced field of view of the HoloLens was not perceived as a severe limitation (1.3).Participants judged the system as capable of providing a real benefit as a training tool (4.8) and to help them to be more effective (4.8).
Our findings show that an interactive mixed reality learning environment could be an effective training tool for enhancing team performance across industries.Holo-Transportation is a self-instruction learning program that requires minimal instructor intervention and, thus, can help to maximize learning results reducing the training cost.As major features, the system offers a virtual scenario that can be easily reconfigured to generate many different situations.All the data generated and the events occurring during the simulation are logged allowing instructors to immediately identify errors and difficulties of the trainees.Logging can also support an effective debriefing and, eventually, the creation of a library of training events.
Teamwork is a key target for soft-skills educators: soft and behavioral skills to enhance teamwork involve aspects of cognitive and social psychology, sociology and anthropology that have to be addressed in a proper way.Teamwork and cognitive collaboration from within an organization are probably two of the biggest factors that contribute to whether a firm succeeds or not.Formation of and communication between teams can often be beset with various issues that can stop a team's effectiveness dead in its tracks.Even more problematic is that the traditional formation of teams along the lines of functionality tends to create an unbalanced team that only has experience tackling the issues from one perspective, and is therefore unlikely to come up with 'outside of the box', innovative solutions.
Fostering cognitive collaboration both within and between teams means first understanding the people who make up those teams, knowing what their thinking and behavioral preferences are, and organizing them together in a way in which complements these different mental style.
From this approach to teamwork and team building derives a major benefit: a team that is balanced in different mental attributes has members who look at situations and communicate their ideas in different ways, leading to more varied solutions, which have been challenged and tested by those who view the situation differently.Individuals with differing behavioral preferences allow for a team to find an optimal way through cognitive collaboration to actually go about the challenge of solving issues and to communicate their solutions.
e-REAL is an effective setting to perform a number of tasks focusing on teamwork: learners are completely immersed in a 3D and holographic scenario where they can interact by natural gestures.The scenario integrates a wide range of challenging situations developed by visual storytelling techniques and delivered in an immersive (and interactive) way with tools tailored to provide feedback: so behavioral, cognitive and meta-cognitive aspects of performance are addressed and reviewed.
What today we know about learning from cognitive psychology is that people learn by practicing, with feedback to tell them what they're doing right and wrong and how to get better.The project's focus is about 8 specific key performance indicators: knowledge circulation and use of available information, effective communication, setting priorities and time keeping, workload distribution, stress management within the team (mainly when a fixation's error is occurring and detected), teamwork (mainly dealing with leadership and fellowship, power distance and cooperation).In such a way, by a teamwork activity it could be achieved a very relevant target: fostering cognitive flexibility, emotional intelligence and self-awareness, as well as honing the epistemic acumen related to reviewing ongoing processes and procedures, errors and alternative options.
Future work will include a thorough assessment of both the training and evaluation tools proposed.
Such a self-instruction learning simulation program was developed to maximize learning results, helping to enhance communication and knowledge sharing, as well as cognitive flexibility, uncertainty tolerance and the retention of cognitive information, reducing instructor intervention and cost.Microsoft HoloLens technology was used again, enabling users to interact with high definition holograms.The solution was developed as a Mixed Reality selfinstruction training environment, allowing also self and team assessment.A self-instruction program is provided.After a short induction phase, users are guided through the sequence of actions required in order to perform deep spatial navigation.Mixed Reality techniques jointly with the e-REAL training environment provide more interactivity.