Novel approach proves mixed reality is a feasible option for future intraoperative visualisation

A preliminary study has demonstrated the feasibility of using a novel visualisation approach as a valuable adjunct tool for minimally invasive percutaneous procedures. This is the conclusion of Gerard Deib (Division of Interventional Neuroradiology, The Johns Hopkins Hospital, Baltimore, USA) and colleagues following a feasibility study, recently published in The British Medical Journal (BMJ). mixed

The investigators used optical see-through head mounted displays to offer a mixed reality experience with unhindered procedural site visualisation during three commonly performed minimally invasive procedures using high resolution radiographic imaging. The three selected procedures were: vertebroplasty, kyphoplasty, and discectomy.

Deib et al write: “Adequate image guidance is necessary to reliably visualise anatomic landmarks and successfully deliver medical devices during percutaneous spine procedures. Radiography monitors displaying the fluoroscopic images used for guidance purposes are typically not aligned with the procedural axis, rendering an indirect visualisation shown to hinder hand-eye coordination. Optical see-through head mounted displays with high resolution and high contrast capabilities offer real time mixed reality visualisation of radiographic images that can be projected over the interventional site without hampering direct visual control of procedural manipulations.”

Percutaneous vertebroplasty, kyphoplasty and discectomy were successfully performed using the optical see-through head mounted displays image guidance on a lumbar spine phantom using commercially available devices. The percutaneous vertebroplasty and the kyphoplasty were “technically adequate”, with the study authors reporting PMMA (polymethylmethacrylate) filling 60–70% of the L2 and L3 vertebrae, endplate to endplate deposition, and an equal amount of cement on either side of the midline.

For the discectomy, the Dekompressor device was successfully placed into the mid-portion of the L3–4 intervertebral disc.

Dosimetry and procedural time for the three percutaneous operations under study compared favourably with typical procedural times. Conventional and mixed reality visualisation modes were equally effective in providing image guidance, with key anatomic landmarks and materials reliably visualised.

The authors conclude: “In this single user preliminary study, optical see-through head mounted display visualization was not found to be inferior to traditional monitors in terms of procedural duration, dosimetry, or ability to visualise key anatomic structures, devices, or material components. There was no perceived impairment in the operator’s ability to visualise real world structures while wearing the head mounted display and using the virtual monitor display, as demonstrated by the completion of all procedural steps without removing the head mounted display, turning off the virtual monitor, or resorting to a traditional monitor.”

Mixed reality visualisation

The operator, a neurointerventional fellow with one year’s experience of vertebroplasty and kyphoplasty, obtained a clear view of the pedicle outline and the articulating processes of the facet joints during the initial set-up and planning.

Mixed reality visualisation—the merging of real and virtual worlds to produce new environments and visualisations where physical and digital objects co-exist and interact in real-time—was achieved using the Microsoft HoloLens system. The system used for real time visualisation of intraoperative medical images in the mixed reality environment could utilise any of a head, world, or body anchored display, with an average delay of 214 (±30) ms, and a refresh rate of 41.4 (±32.0) Hz for typical image sizes. Each display mode provided equally effective image guidance.

When head anchored, the mixed reality display remains at a constant position within the operator’s field of view, and depth position remains unchanged throughout the procedure. This display was deemed especially useful at the start of the operations, when the surgeon needed to look at multiple things at once, for example, the phantom patient, the angiography C arm, and the angiography table control panel.

In contrast, the world anchored relies on the display being locked to a specific position in the environment by a hand gesture from the operator. This proved particularly helpful for visualising a single space, such as when advancing a needle or device under fluoroscopic guidance, though as the display is in a stationary position, it will occasionally be out of the operator’s field of vision.

Finally, the body anchored mode, though it combined the most convenient features of the world and anchored displays, required constant repositioning of the virtual monitor within the operator’s field of view, and therefore a period of acclimation to the mixed reality environment is necessary. This display type consistently remains in a specific position within the operator’s field of view, and only moves when the surgeon’s head movement would result in the display leaving their field of vision.