Over the past three decades, laparoscopic hepatectomy has gradually gained increasing recognition because of its unique advantages; however, laparoscopic surgery still has many shortcomings, such as a lack of tactile sensation, limited vision, and the inability to see through the complex internal structure of the liver [3]. Alleviating these shortcomings has become one of the most concerning topics for hepatobiliary surgeons. With the rapid development of medical imaging technology, 3D reconstruction, preoperative simulated resection, fluorescence laparoscopy, and AR technology have all been gradually applied in the management of liver tumors in the perioperative period, which has achieved encouraging results.
In the assessment of preoperative liver function, the traditional Child‒Pugh classification still significantly differs among patients with Child A liver function [4], and the MELD score is usually used for patients who need liver transplantation [5]. Preoperative ICG testing can be used to assess liver reserve function safely and accurately, and this method is now widely used [6]. In addition, hepatic resection must take into account the residual liver volume after hepatectomy resection; normal livers can tolerate hepatectomies with residual liver volume/total liver volume ratios greater than 20%-25% [7], whereas hepatectomies with cirrhosis must retain sufficient residual liver volume. Combining the ICG-15 values with preoperative 3D simulated resection techniques to calculate the appropriate simulated postresection residual liver volume from the ICG-15 values may reduce the proportion of postoperative hepatic failure that occurs. Our experience is that through the three-dimensional reconstruction technique, the tumor location and the relationship between the tumor and the liver blood vessels are displayed in three dimensions, the extent of tumor infiltration is judged, and the ratio of the residual liver volume accounted for is calculated after the preresection line of the liver is selected. On the basis of these results, the hepatic resection plane is repeatedly adjusted to develop an optimal surgical plan. The advantage of this scheme is that, on the one hand, it can maximize the preservation of normal liver tissue and reduce the incidence of liver failure, and, on the other hand, it can avoid the main hepatic blood vessels and reduce excessive dissection of blood vessels to reduce intraoperative bleeding.
Since Ishizawa et al. [8] first used ICG molecular fluorescence imaging to navigate hepatic resection in 2009, this technique has been widely used in the diagnostic hand and surgery of hepatobiliary and pancreatic tumors. Laparoscopic fluorescence imaging plays an important role in detecting residual tumors and microscopic lesions at the margins of liver sections, defining tumor boundaries intraoperatively, and improving postoperative tumor-free survival [9, 10]. ICG fluorescence is able to successfully identify superficial lesions as small as 1–2 mm that were previously unidentifiable via preoperative or direct visualization; thus, ICG fluorescence improves the sensitivity of hepatic tumor detection [11]. Uchiyama et al. [12] reported that the sensitivity of ICG fluorescence for tumor detection was 98.1%, whereas that of conventional imaging methods was 88.5%. In addition, ICG fluorescence detection remains highly reliable for tumors within 8–10 mm below the hepatic envelope [13]. We applied fluorescence laparoscopy to laparoscopic hepatic tumor resection and achieved good results. First, this method can rapidly locate the position of the tumor and visualize and expose the boundary of the tumor during the resection process, which is convenient for allowing the operator to timely predict and adjust the resection line and improve the ratio of R0 resection of the tumor. Although fluorescence laparoscopy has been widely used in laparoscopic liver resection, for the resection process of complex livers, the tumor periphery is often adjacent to important blood vessels and the biliary system, which makes smooth implementation of laparoscopic surgery difficult. At the same time, the deep location of some liver tumors cannot be accurately localized laparoscopically, which further increases the difficulty of surgery. Owing to the reliance on fluorescent laparoscopic imaging of the tumor alone, it is impossible to predict and judge the location of important blood vessels in advance, which leads to an increase in intraoperative bleeding and the rate of electrocautery.
In recent years, with the development of AR technology, successful cases of the application of this technology in laparoscopic hepatectomy have been reported [14, 15]. AR technology is an intraoperative application based on a 3D visualization system that projects a preoperative 3D model of the liver into the created 3D space and fuses the captured real-time laparoscopic images into a 3D spatial background to create an interactive environment. The main difficulties of this technology include three aspects: image alignment, continuous image tracking, and selective image fusion. The 3D visualization-based mixed reality navigation system and the 3D visualization-based mixed reality laparoscopic projection navigation system, developed by our center in cooperation with computer engineers, have been granted national patents in China and have been applied for in clinics with good results.
First, in the process of image alignment, the general alignment can be completed according to the contour of the liver and the location of the gallbladder. Second, when verifying or predicting the alignment of tumors, blood vessels and bile ducts are necessary. The secondary image alignment is performed according to the location of the dissected important ducts or tumors. The alignment often requires manual cooperation to address the process, and, in the present study, the alignment error for the 10 patients was 6.3 ± 0.6 mm. The registration error is a measure of accuracy that represents the error in the position of the alignment picture farther away from the alignment target [16]. The registration error is affected by many factors, including respiratory movement, liver position changes, frame shifts and pneumoperitoneum pressure. Within a certain range of registration errors, the surgeon's main concern is whether the technical solution can verify and predict the vessels intraoperatively. In terms of verifying blood vessels, when the operator is unsure of the origin of blood vessels during the processing of ducts in liver sections, previous experience involves roughly determining the origin of the blood vessel from the memory of preoperative film readings, which involves large errors and may result in damage to important blood vessels. In contrast, currently, for complex hepatic resections, the type of pipe encountered can be verified by AR, the need for preservation or resection can be quickly determined intraoperatively, and the average number of vessels verified in this study was 5.6 ± 0.6. In addition, in terms of predicting the vasculature, the ability of the MHV to predict the presence of tumors resected in the middle lobe of the liver cannot be ignored. In previous studies, intraoperative ultrasound was used to localize the MHV on the surface of the liver, and the difficulty lies in the fact that the distal branches of the MHV are encountered during resection from the pedunculated side to the cephalad side. It is often impossible to predict whether the MHV is to the left or to the right of the resection line. AR can predict the location of the MHV according to the branch vessel's shape and position. In addition, AR can predict the ductal structure of important parts in real time and provide early warning. It can be used to predict vessels on average 4 times in 10 patients in this study, which reduces possible intraoperative bleeding.
There are many challenges that need to be solved in augmented reality navigation technology. How to continuously and rapidly track the images of interest and how to quickly and selectively fuse the structures of interest are the focuses of subsequent research. Most of the reported solutions are in the experimental stage [16, 17] and have not been applied in clinical practice. In laparoscopic liver resection, image tracking, rendering and fusion, enhancement of stereoscopic visualization of the anatomical structures in the liver, and real "perspective" effects and early warning require the cooperation of surgeons and computer engineers.
In this study, real-time navigation hepatectomy using AR combined with laparoscopic fluorescence images during complex hepatectomy provides the operator with more information to guide the operation, thus compensating to a certain extent for the lack of tactile sensation and limited field of view in laparoscopic surgery, avoiding damage to important ductal structures and improving the precision and safety of the operation, which has broad development prospects; however, at present, the technology is still in the primary stage, and the current research samples are relatively small, and continuous research is needed.