An Interactive Mixed Reality Ray Tracing Rendering Mobile Application of Medical Data in Minimally Invasive Surgeries

Abstract

Visualization of patient’s anatomy is the most important pre-operation process in surgeries, minimally invasive surgeries are among these types of medical operations that counts totally on medical visualization before operating on a patient. However, medicine has a problem in visualizing patients’ through looking through multiple slices of scans, trying to understand the three-dimensional (3D) anatomical structure of patients. With Mixed Reality (MR) the developments in medicine visualization will become much easier and creates a better environment for surgeries. This will help reduce the excessive effort and time spent by surgeons to locate where the problem lies with patients without looking through multiple of two-dimensional (2D) slices, but to see patients’ bodies in 3D in front of them augmented in their reality, and to interact with it whatever pleases them. Moreover, this will reduce the number of scans that doctors will ask their patient’s for, which will result in less harmful x-ray dosages for both the patient and the radiologist. Biomedical development in medical visualization is an active research topic as it provides the physicians with required devices for clinically feasible way for diagnosis, follow-up and take decisions in different disease life line. Current clinical imaging facility can provide a 3D imaging that can be used to guide different interventional procedures. The main challenge is how to map the information presented in the digital image with the real object. This is commonly implemented by mental processing that requires skills from the medical doctor. This paper contributes to this problem by providing a mixed reality system to merge the digital image of the patient anatomy with the patient visual image. Anatomical image obtained from Computed Tomography (CT) or Magnetic Resonance Imaging (MRI) is mapped over the patient body using virtual reality (VR) head-mounted device (HMD).