3D Holographic Visualization for Pre-procedural Planning of Spatially Complex Brain Aneurysm Treatment

The purpose of this study was to develop and demonstrate a novel imaging platform that post-processes volumetric imaging data (e.g. Computed Tomography (CT) or Magnetic Resonance angiography) to provide holographic visualization for pre-procedural treatment planning of a morphologically complex brain aneurysm. Digital CT images were segmented, using the Materialise Mimics software, into three-dimensional digital models that were imported into a prototype application for the Microsoft HoloLens. Feedback from testing the prototype indicated potential for augmented reality to assist an interventionalist in spatial understanding and depth perception of spatially complex anatomy and could increase condence in pre-procedural planning. Future studies will be conducted with additional cases to further validate the utility of the platform in surgical planning decisions and to expand the platform for patient/resident education, telemedicine, and intra-operative use.


Background
Brain aneurysms (otherwise known as cerebral or intracranial aneurysms) occur when the wall of a cerebral artery is weakened, resulting in excessive bulging of the blood vessel. Upon rupture, a patient will be exposed to subarachnoid hemorrhaging (SAH), in which a stroke can occur. According to Johns Hopkins Medicine, the fatality rate for patients who experience SAH is around 46%, but can rise up to 80% if left untreated. 3 Currently, the major treatment options for patients include endovascular therapies (including coiling and ow diversion devices) and surgical clipping. Endovascular coiling is a less-invasive treatment that involves a surgeon guiding a catheter from the patient's groin to the location of the aneurysm, where they deploy detachable devices that block or redirect blood ow (as shown in Fig. 1a). 4 Surgical clipping is a more invasive surgical treatment in which the surgeon gains access to the aneurysm site and blocks (i.e., occludes) blood ow by attaching a metal clip to the neck of the aneurysm (see Fig. 1b). 5 Although these treatments are the current standard of care for hospitals across the United States, there are many problems that have been unaddressed. Current methodologies for endovascular therapy involve imaging with cerebral angiography, a two-dimensional (2D) imaging modality for an inherently three-dimensional (3D) problem. If an interventional neuroradiologist (INR) cannot get good visualization utilizing biplane angiography, there is risk of complications. Additionally, angiographic imaging (which is based on uoroscopy) produces X-rays that pass (i.e., transmit) through the patient, exposing both the patient and the caregiver to radiation. 6 To produce a high-quality image, INRs have to align the C-arm of the uoroscopy machine at the correct angle co-axially to the tumor, which can prolong procedure time (especially for less experienced INRs). Furthermore, according to Pierot et. al, one of the main di culties that was evident through endovascular coiling was complex aneurysm shape and location in proximity to the main cerebral artery. 4 One major drawback to the surgical clipping method is that it is more invasive. When this procedure is performed, a part of the patient's skull is removed for the surgeon to gain access to the aneurysm. With surgical clipping, exposure and visualization of the cerebral aneurysm, the parent artery, and important branches is also important. If care is not taken to understand the 3D anatomy, complications can also result. Overall, many of the issues that arise from current standard-ofcare procedures are a result of spatially complex aneurysm anatomy and location, as well as determining what angle to image the patient from.
Augmented reality (AR) offers a solution to current standard-of-care issues by providing surgeons with 3D visualization of a patient's intracranial vasculature. The proposed platform system utilizes processed patient-speci c virtual models from Digital Imaging & Communications in Medicine (DICOM) imaging data that are segmented with Materialise Mimics software (Leuven, Belgium) and projected three-dimensionally in space for pre-operative planning. The use of an untethered headmounted display (HMD), such as the Microsoft HoloLens (1st generation; Redmond, WA), 7 has the potential to assist an INR in 3D spatial understanding of complex aneurysm shapes and the location of the aneurysm in relation to critical structures of the brain. The use of the HoloLens could also provide in-depth pre-procedural insight as to the morphology of the vasculature and the most e cient camera placements for angiography. The Microsoft HoloLens is a completely free-standing, untethered HMD for viewing augmented/mixed reality (AR/MR). It utilizes six degree-of-freedom positional head tracking and spatial mapping to enable the user to view and interact with 3D virtual images overlaid on the physical environment in the user's eld of view (F.O.V.). 7 The HoloLens uses diffractive planar waveguides-which are ber optic devices that transmit visual data to the user's eyes-to allow for visualization of the 3D images. 8,9 In turn, this technology would have the potential to assist in eye-hand coordination as a result of enhanced 3D visualization, as well as the potential to bene t education for patients and less experienced surgeons. Furthermore, the use of AR could potentially decrease overall procedure time and radiation exposure to clinicians and patients, while simultaneously con rming pre-procedural insight (which will be determined in a future study).
The use of the Microsoft HoloLens in combination with registration techniques, such as Vuforia SDK markers (for tracking and alignment of virtual models with the physical world), has been studied and proven to be accurate, with a mean hologram drift of 1.41 mm in a study by Frantz et al. 10 In a previous study at the Cleveland Clinic, similar AR-based technology exhibited potential to be able to assist interventional radiologists in the performance of abdominal tumor ablation procedures in both a clinical and pre-procedural setting. 11 The objective of this project is to evaluate the use of a novel prototype imaging system for assisting an INR in visualization and performance of pre-procedural planning for treatment of a morphologically complex brain aneurysm.

Methods
Sets of computed tomography (CT) data in DICOM imaging format (Digital Imaging and Communications in Medicine) were collected from a patient who was diagnosed with a cerebral aneurysm between the right internal carotid and the right-sided middle cerebral artery off of the Circle of Willis. The 2D image slices of the CT were segmented into 3D digital models using Materialise Mimics software for the skull and a section of the cerebral artery that encompassed the aneurysm (as shown in Figure 2). The skull was segmented through thresholding grayscale values that optimized reconstruction of the bone. As shown in Figure 3, a smooth and anatomically accurate model of the skull was created, including its thin structures. The walls of the model were slightly thickened to improve durability of a planned 3D-printed model of the patient's skull and vasculature, without sacri cing any anatomical accuracy. A separate mask was added using arterial phase data. The cerebral arteries and aneurysm were segmented in a similar manner. Individual mask les were exported in stereolithographic (STL) le format and imported into MeshLab for decimation. The different meshes were decimated to 120k faces (for optimized hologram stability) and exported as 3D object (OBJ) les, as shown in Figure 3. From here, OBJs were imported into Unity Engine as assets, where an app for the Microsoft HoloLens was built with the ARToolKit. The app was built in Visual Studio 2019 and deployed to the Microsoft HoloLens.
The projected holograms are registered to a speci c location in space using the Vuforia SDK, which is a proprietary imageprocessing software integration into Unity Engine (San Francisco, CA) that allows the projected holograms to be tracked in real-time to a physical image target. 12 The Vuforia engine utilizes the camera on the HoloLens to detect contrast-enhanced features of an image target within a user's F.O.V. and compares them to the same image in a digital database (stored in the Unity app). Figure 4 exhibits the image target pattern used, with and without the feature detection. The difference between the physical and virtual allows the software to calculate the target's position in HoloLens coordinates, and will project the virtual objects aligned to the image target. This results in hologram stability with little drift, 13 even when the caregiver walks around the projection and views the holographic scene from various angles. A 3D-printed Vuforia image target was used to track the position of the head and to align the virtual model to the physical marker. An image target using a test cube can be seen in the Unity scene shown in Figure 5.
After initial prototyping, feedback was collected from an INR at the Cleveland Clinic to determine further features that would bene t both the clinicians and the patients pre-and intra-procedurally.

Results
An initial prototype app was successfully built and deployed to the HoloLens. The prototype app consisted of the anatomical segmented digital models of the skull and a region of interest surrounding the aneurysm vasculature from Mimics that could be registered and tracked with a Vuforia marker. One feature enabled in this prototype was a clipping plane, so that as the user moves their head closer to the holograms, the layers of the digital models are pealed back slice by slice -revealing an in-depth look at the inside of the skull.
The prototype app sets an origin based on the user's position in the room when the app is started. It then registers the anatomical holograms to the physical Vuforia marker. The registration occurs when the user gazes at the image target on a at surface from above. Once the holograms are registered to the marker, the user can move freely about the room within a ve-meter radius of the app's origin 13 and view the holograms from different angles to understand the 3D nature of the aneurysm in relation to the skull and cerebral vasculature. One observation was that the registration was sometimes slow at the start, and occasional hologram "drift" would occur. Measurement of this accuracy and drift was outside the scope of this study, but there are many ways in which this can be improved (see DISCUSSION). Figure 6 shows screen captures from the HoloLens running the prototype app.
Upon evaluation of this prototype to an INR at the Cleveland Clinic, useful feedback was collected for both how it may assist doctors in the operating room and angiography suite, as well as potential future improvements that could improve the patient-caregiver experience. One comment on the prototype regarded what areas that this AR technology would help, speci cally for 3D visualization with eye-hand coordination and spatial understanding, as well as for patient and resident education.
Feedback from a Cleveland Clinic caregiver regarding future improvements of the prototype involve the addition of a scale-up function, in which a voice command/user-interface (UI) could raise the holograms anteriorly and scale them by a factor of two. In addition, a zoom and rotate function with hand gestures could potentially help doctors see the anatomy up close to assist with pre-operative planning. Voice commands could also be used to toggle holograms on/off, which could further ease-of-use for the end-user.
For patient and resident education, it was suggested that possibly two HoloLenses could be utilized cooperatively so that patients could better understand the procedure and disease, and so that residents-in-training or younger surgeons would be able to walk through a complex procedure with an experienced surgeon -even remotely from different institutions.

Discussion
In this project, an AR application for the Microsoft HoloLens was built to enable pre-procedural planning and spatial understanding of a cerebral aneurysm in relation to other critical structures of the brain. Anatomical digital models were built using segmented DICOM data in Materialise Mimics that were processed and incorporated into the Unity app as 3D digital models. While the prototype build was successful, there were many observations and feedback of features that could be improved or required future investigation.
One observation was that the registration of the anatomical holograms to the Vuforia image target was sometimes delayed, and the user experienced occasional hologram drift after the anatomical holograms were registered. This may have resulted because another user registered the anatomical holograms before the INR donned the HoloLens. When the HoloLens was passed to the INR, the inertial measurement unit (IMU) within the HMD may have needed recalibration/reregistration. This drift could potentially be avoided by holding a training session for those using the app with simple voice commands that could easily register the holograms to the desired location.
The question of accuracy (i.e. the accuracy of the registration of the virtual images to the patient's real anatomy via the Vuforia tracking system) is important-especially in a clinical setting. The accuracy of the Vuforia SDK integration with the Microsoft HoloLens has been previously studied for a neuronavigational app, and is reported to have a mean hologram drift of 1.41 mm. 10 While accuracy was outside the scope of this current study, these results could be veri ed through methods described by Frantz et al in a future study. 10 Future accuracy measures could verify patient safety and pre-procedural plan con dence for INRs.
In a future study, the prototype app could be integrated with the second generation of the Microsoft HoloLens (HoloLens 2).
The HoloLens 2 has many upgraded features, such as an increased eld-of-view, a more ergonomic and comfortable design, and real-time eye, hand, and head tracking (as opposed to the rst generation HoloLens, head-tracking only). 7 Furthermore, this early feasibility project lends to plenty of future work in which the prototype could be built out to examine feasibility for patient/surgeon education, remote collaboration between caregivers, or even intra-operative use with the anatomical holograms projected directly to the patient. While the current feasibility study worked directly with one INR at the Cleveland Clinic, a standard system usability survey with a Likert-style questionnaire (1 = strongly disagree, 5 = strongly agree) could collect data in the future from many clinicians to better shape the needs of the end-user and functionality of the prototype application. 14

Conclusions
Overall, this evaluation provides a rst step towards development of an augmented reality imaging platform for neuronavigational procedures, such as cerebral aneurysms of complex morphologies. The development of a prototype app for pre-procedural planning and spatial understanding of complex anatomy showcases the potential for AR technology to bene t clinicians in many ways. Some bene ts of this technology include 3D visualization of the anatomy to visualize locations of an aneurysm in relation to critical structures of the cerebral vasculature and skull base, as well as providing insight for a clinical plan during the pre-operative phase.
This project opens the door to various future studies that would build out the platform to be a multi-functional application that could be used not only pre-operatively, but also possibly intra-operatively to guide surgical decisions, alignment of the Carm for imaging, and to act as an adjunct to standard-of-care methodologies. Future work to further validate the utility of the platform in surgical planning decisions can provide clinicians with the con dence to implement this novel technology into their daily procedures.

Availability of data and materials
Data sharing is not applicable to this article as no datasets were generated or analyzed during the current study.     Vuforia image target pattern utilized in the app, with and without feature detection Unity scene capture of a test cube object's position being tracked by a Vuforia marker pattern. where the cerebral aneurysm is outlined in yellow.