How VR, AR and MR Can Improve Surgical Outcomes
If you’ve ever faced the possibility of going under the knife, you must have experienced at least some mild anxiety, knowing that surgery outcomes are not guaranteed and even the best surgeons can have a bad day. In fact, you’re usually asked to sign a paper called a consent form, a legal document that states that you have been informed about the risks of the treatment and you allow your doctor to go ahead with the surgery, by taking full responsibility for the risks involved.
Surgical procedures vary widely in the level of risk they pose. For example, it’s very rare for a healthy individual to die during a minor dental procedure. Or during carpal tunnel surgery which is performed on a patient’s hand and wrist, often in an outpatient surgery centre. But during some open heart surgeries, the heart is actually stopped for almost an hour before being restarted, which poses a much higher risk. Also, surgeries related to trauma, such as a serious car accident, have a higher risk level than a planned and scheduled procedure.
And as far as the higher-risk surgical procedures are concerned, virtual reality technology (VR) has proved a real blessing. VR is an immersive experience combining computer technologies with reality headsets to generate the realistic sounds, images and other sensations that replicate a real environment but is an entirely virtual world. VR allows surgeons to advance the field and make surgery safer for patients.
The use of VR in a surgical setting allows for a more realistic imitation of the human body which helps to visualise complex anatomic structures, which is very useful in neurosurgery. Talking about his work, Dr Gary Steinberg, Chair of Neurosurgery at Stanford Moyamoya Center, said: “One of the advantages that VR provides is the ability to see and appreciate a 3D view of the patient’s brain, anatomy and pathology…We can simulate or rehearse the entire surgical procedure beforehand, so when we get to surgery, it’s not a surprise, it’s as if we performed the surgery before”.
Surgeons and patients alike benefit from VR-based surgery which first and foremost provides a safe environment for both doctors and patients. In preparing for neurosurgery for instance, the 3D VR image is taken from a radiologic image, which comes either from a CT scan, an MR scan, or a digital subtraction angiography (a radiography of blood or lymph vessels). Data is taken from all these different modalities and fused into one single, unique image. This 3D image allows surgeons to look around the entire brain and to thoroughly examine the pathology they are dealing with, be it a venous malformation, an aneurysm or a tumour, by looking at its relationship to the blood vessels and brain substance itself.
VR can also give real time feedback regarding surgical progress, highlighting errors and successful procedures. Aside from VR, there is also AugmentedReality (AR) which adds or augments specific elements of a physical, real-world environment using computer-generated sensory input such as sound, video, graphics or GPS data creating a new layer that exists on top of our own world.
For surgeons, the advantages of using of VR/AR simulators and assistance during surgery improves the learning curve and reduces the errors during the learning stage of future surgeons (who would want to be a surgeon’s first?). It increases the speed and accuracy of surgical procedures – trained residents are found to be almost 30% faster in surgical dissections (1). It provides medical assistance in remote locations. There is also the issue of time and cost reduction in the training of future doctors and surgeons.
For patients, the use of VR/AR simulators and assistance during surgery reduces risk and trauma, since VR-trained surgeons were found to perform only 1/6 of the errors of their non-simulator trained counterparts (1). VR also offers a more comprehensive diagnosis for patients with time reduction in rehabilitation and physical therapy.
Several surgery domains and clinical applications are currently using VR/AR surgical planning or surgical training tools, including ocular surgery, liver surgery, neurosurgery, spine surgery, maxillofacial surgery, breast surgery, plastic surgery, orthopedic surgery and laparoscopy — a low-risk, minimally invasive surgical diagnostic procedure used to examine the organs inside the abdomen that requires only small incisions.
The number of VR-related research publications in PubMed underscores that this technology is not new in healthcare and we are already at the stage where we can compare different types of VR/AR devices. A study that just came out in the Journal of Minimally Invasive Therapy & Allied Technologies evaluated three types of hysteroscopic simulators: Hyst Sim VR, Virtual Reality Uterine Resectoscopic Simulator and Essure Sim TM. These VR simulators were found to match reality very closely, increasing the diagnostic and surgical skills of gynaecologists and all surgeons attained significant improvements between their pre-test and post-test phases, independent from their starting level of expertise (2).
While the future of surgery looks all bright and shiny with the adoption of the new technology, some caution must be taken when implementing it. According to a very recent study that compares cognitive load and performance in immersive VR versus conventional VR simulation training, the increased complexity of the learning situation in immersive VR could potentially induce high cognitive load thereby inhibiting performance and learning, which results in a poorer performance than conventional VR-simulation training (3). The study agrees that immersive VR offers some potential advantages over conventional VR such as more real-life conditions but it recommends introducing immersive VR in surgical skills training after initial training in conventional VR.
A step forward from VR/AR is mixed reality (MR), also known as hybrid reality. If VR is immersing people into a completely virtual environment and AR is creating an overlay of virtual content that doesn’t interact with the environment, MR is the merging of real and virtual worlds to produce new environments and visualisations where physical and digital objects co-exist and interact in real time.
And MR technology is already here. Christened the Microsoft HoloLense, it blurs the boundaries between AR & VR. Microsoft describes HoloLense as “the first self-contained, holographic computer, enabling you to engage with your digital content and interact with holograms in the world around you.” HoloLense has been adopted for 3D visualisation during spinal surgery, one of the most complicated al surgical procedures (4).
Aside from surgery, virtual reality is used for a variety of medical procedures, including cancer treatment (5), by creating interactive maps of tumours; and physical therapy, by having patients play games that encourage movement.
In surgery however, VR has perhaps the greatest potential and this is a rapidly developing field in digital health.Compared with a classical surgical procedure, the use of VR technology or an MR headset provides an unprecedented gain in accuracy and safety in the surgical procedure which can tremendously improve outcomes.
- Seymour, Neal et al. (2002) Virtual reality training improves operating room performance: Results of a randomized, double-blinded study. Annals of surgery. 236. 458-63; discussion 463.
- Vitale SG, Caruso S et al. (2019) The value of virtual reality simulators in hysteroscopy and training capacity: a systematic review. Minimimum Invasive Therapy Allied Technologies., Jun 6:1-9.
- Frederiksen JG et al. (2019) Cognitive load and performance in immersive virtual reality versus conventional virtual reality simulation training of laparoscopic surgery: a randomized trial. Surg Endosc. 2019 Jun 6
- Gregory TM, Gregory J, Sledge J, Allard R, Mir O. Surgery guided by mixed reality: presentation of a proof of concept. Acta Orthop. 2018;89(5):480–483.
- Al-Halabi, Wadee & Aseeri, Sahar & Ahmad Almeleak, Rasha & Taha Al-Hashmie, Hend. (2019). Effect of Virtual Reality on breast cancer patients. 110-115.