3D-printed patient model and initial frameless fixation
The 3D-printed patient model was produced in the following three stages: 1) creation of a stereolithography (STL) file for 3D printing; 2) printing physibles using a 3D printer; and 3) post-processing performed via manual editing. The process was approved by the institutional review board (Seoul National University Hospital IRB No. 1811-040-96 and Chungbuk National University Hospital, IRB No. 2019-06-015) and used to validate set-up errors, fiducial marker coordinates, and HDMM values. The 3D-printed patient model had a frameless adapter with head cushion and was fixed with a thermoplastic nano mask (Elekta instrument AB, Stockholm, Sweden). In the fixed-AR setting, we measured the initial fiducial marker coordinates after planning CBCT and HDMM, which keep the motion value for 10 min, as shown in Figure 2.
3D scanning and planning CBCT
The 3D-printed patient model with frameless fixation was taken for TrueDepth scanning using Heges application with iPadPro 13 cameras (Apple Inc., One Apple Park Way Cupertino, CA 95014, USA). The .stl scan file of the scan was imported to Meshmixer version 3.5 (https://www.meshmixer.com) to trim the background scanning, and smoothing processes were performed to improve the pixilation an obtain more uniform triangles.
The 3D-printed patient model with frameless fixation was also taken for planning CBCT scanning using Leksell GammaPlan Version 11.1 (Elekta instrument AB, Stockholm, Sweden), and exported to CBCT dicom files. The dicom files were imported to MEDIP software (MedicalIP, Seoul, Republic of Korea) for 3D-rendering, trimming the background, and smoothing processing. This procedure is shown in Figure 3.
Inside-out AR navigation and running fixed-AR
The inside-out AR navigation was developed by AR framework using ARkit®. The inside-out AR navigation was processed in three steps. First, device recognition is visualized, followed by QR marker recognition, and AR implementation and registration within the running environment. The QR marker is attached to the right mask fixation button adjacent to the matching target that has minimal light reflection and is unlikely to be easily obscured by other objects. The ARkit is based on Visual Inertial Odometry (VIO), which measures the device location from inertial measurement unit (IMU)-based data that has a fast collection and calibrates using camera images. Moreover, ARkit supports close-loop processing that corrects the trajectory by matching the trajectory with the starting point when moving the device and returning to the starting point during calculation with the VIO algorithms. This is collectively referred to as visual inertial simultaneous localization and mapping. The loop-closure process can be omitted due to the nature of this AR system wherein the device operates in a fixed state.
The QR marker images are used for estimating the position recognized by the camera in the feature detection algorithm, which commonly considers the corner and intersection of lines or the part with clear color contrasts (Black and White) as a feature point. After the device position is determined, the QR marker, recognized by the scale of objects in virtual space, is determined by calculating the distance between the device and the QR marker from the size of the recognized marker in the preceding step. After all the requisite data are calculated, the 3D scanning or CBCT-based virtual models are displayed at the designated positions from the marker for confirmation by the user. In case of errors in the automatically processed pre-registration, the user can correct the registration error by adjusting the position of the 3D virtual model using the fine-tuning function and then fix the position of the models to complete the registration.
Validating the fixed-AR navigation registration
The registration accuracy was measured using the following three methods: intuitive validation through visualization; set-up errors; and quantitative validation. The 3D-printed patient model was created and matched with 3D virtual models. Set-up errors were assessed by comparing the planning and pretreatment CBCTs3, which navigated to re-fixation using fixed-AR system in the virtual models. Set-up errors were investigated by translational (mm) and rotation (°) methods. The registration accuracy of initial fiducial marker was validated by comparing planning CBCT to pretreatment CBCT, which navigated re-fixation in fixed-AR through the virtual models. The fiducial markers were attached to both hemispheres of the 3D-printed patient model in eight points.
We defined the error of coordinates as follows:11
Δx=x coordinate in planning CBCT + pretreatment CBCT or virtual models
Δy=y coordinate in planning CBCT + pretreatment CBCT or virtual models
Δz=z coordinate in planning CBCT + pretreatment CBCT or virtual models
Furthermore, the 3D error (Δr) was defined as a localization error by the following formula:
The differences of HDMM values were investigated to initial fixation and re-fixation navigated fixed-AR in the virtual models. The mean accuracy was evaluated by measuring the X, Y, and Z coordinates for the three measurements.
The function of fixed-AR
For cases when automatic registration using the QR marker could be misaligned, the processing is discussed in detail. A fine-tuning function of adjusting the position of the virtual 3D model of AR navigation was developed. The virtual space is seen on the device screen through the following local coordinates, which defined the model orientation. Opacity-adjustment function can be adjusted by using opacity-adjustment function. These functions allow AR implementation by further co-registration in the brain.