Body Navigation-loaded Ultrasound Acquisition Technology: a Pilot Comparison With Conventional Ultrasound

Background: To investigate the usefulness of body navigation-loaded ultrasound including a real time transducer location and the inspection site compared with conventional ultrasound images. Methods: Under the approval of institutional review board, we prospectively enrolled total 29 healthy adult volunteers. One gastrointestinal radiologist performed abdominal ultrasound simultaneously using Ultrasound Navigation Image Convergence System developed by researchers. Subsequently, an equivalent conventional ultrasound image set was prepared. Three radiologists independently evaluated the two ultrasound image sets regarding the recognition of the target organ (2-points), the transducer location (2-points), and the transducer orientation (1-point). At intervals of two-weeks, conventional ultrasound images were analyzed rst, and body navigation-loaded images were later analyzed. The score differences between the rst and second evaluations were compared using the Wilcoxon signed rank test. Inter-rater agreement of three reviewers was obtained by the Fleiss’ Kappa test. Results: A total of 1402 navigation-loaded ultrasound images were obtained. Ultrasound operator carefully selected a total of 203 images for analysis. In all three reviewers, the interpretation score of each evaluation was signicantly increased in the second analysis using the body navigation-loaded ultrasound image (in reviewer A, from 4.07±1.56 to 4.79±0.69 points; in reviewer B, from 3.83±1.59 to 4.49±0.88 points; in reviewer C, from 3.43±1.60 to 4.19±1.01 points; P<.0001). The inter-rater agreement of each evaluation also increased signicantly in the second analysis using the body navigation-loaded ultrasound image (P<.0001). Conclusion: The body navigation-loaded ultrasound imaging system allows other medical staffs to easily and accurately interpret ultrasound images.


Background
Ultrasound is an economical, non-invasive, and effective imaging modality used in almost all clinical practice. In conventional ultrasound, the ultrasound image is not automatically labeled with information regarding the location or orientation of the transducer. There is also no information about the part of the body being examined. Although variedly implemented in different countries, ultrasound images without body marks may be interpreted from the image itself, the operator's memory, and so on, and can therefore lose their value as an objective image. In clinical practice, knowledge of the transducer location and orientation may not be necessary because standardized views are widely obtained using ultrasound [1]; these have been optimized for their utility in differentiating normal from abnormal ndings. However, accurate information about the transducer location or orientation could be important in ultrasound scans where the standardized view is less established or information regarding the transducer and patient's posture change may be required.
To overcome these problems, annotation using a body mark or text is essential. However, there is a disadvantage in that the operator must expend additional time and effort using various control buttons.
Moreover, the ultrasound examination site intended by the operator and the body mark may not exactly match, and there may be errors during the manipulation process. To complement the existing body mark system of ultrasound, several methods have been developed to install equipment that enables threedimensional (3D) position calculation, on the ceiling of the ultrasound room or the probe itself, and a tracking system that can monitor the position of a device within a subject and superimpose a graphic symbol on the diagnostic image of the subject [2][3][4]. However, these methods also require the transducer to have additional hardware, such as a gyro sensor and an acceleration sensor, and these methods seem to be di cult to apply in clinical practice.
We devised a novel technology that minimizes the necessity for complex hardware and intuitively includes ultrasound location information in a conventional ultrasound image. This technology was designed to be applicable to various organs and to be capable of recording the transducer position with respect to the body. It used a 3D depth camera capable of photographing the inspection site with the transducer and a software specially designed to merge the acquired image with the conventional ultrasound image. The technology was experimentally applied to the most frequently performed abdominal ultrasound, and this pilot study aimed to investigate the accuracy of image interpretation and the effectiveness of body navigation-loaded ultrasound, which included a real-time transducer location and an inspection site, compared with conventional ultrasound.

Study population
With the approval of the institutional review board, we prospectively enrolled 29 healthy adult participants through in-hospital recruitment. Informed consent was obtained from all participants. In addition, consent was obtained to publish the body navigation-loaded ultrasound images obtained in this study. From December 2018 to January 2019, one gastrointestinal radiologist performed abdominal ultrasound of various organs in these participants, with or without NPO (nothing by mouth) for the participants' convenience. In two participants, with additional informed consent, ultrasound was applied to areas other than the abdomen to assess the applicability of this technology in various organs.

Concept of body navigation-loaded ultrasound acquisition technology
This technology uses a commercial 3D depth camera positioned to capture the ultrasound scene within the vicinity of the patient at the same time when the operator obtains a conventional ultrasound image.
The images taken with the inspection site and transducer location are combined in real time in the form of thumbnails and inserted into conventional ultrasound images using a commercial computer and gateway system [5] containing a 3D mesh lter [6], which are then automatically transmitted to the picture archiving and communication system (PACS) server (Fig. 1).
Ultrasound protocol and acquisition of navigation-loaded images Ultrasound was performed by a radiologist with 8 years of experience in gastrointestinal radiology using an iU22 ultrasound machine (Philips, Seattle, WA, USA) with a 5 MHz convex transducer. In the ultrasound image, additional information, such as graphic body marks or text, was not attached. Instead, the transducer and inspection site were captured using a Kinect v2 camera (Microsoft Corporation, Redmond, WA, USA), which can provide 3D depth information and object recognition using a software development kit. The camera was connected to the arm behind the ultrasound monitor using an overhead camera stand and xed such that the inspection site could be viewed from above. In addition, we used a specialized software called MeshGateway, developed by our researchers, which included the Ultrasound Navigation Image Convergence System using 3D Depth Measurement Technique and the ability to transmit navigation-loaded images to the PACS server (Fig. 2). The process of installing the camera and preparing the software took approximately 1 to 2 minutes. If the inspection site was the same and there was no need to manipulate the camera and software, no additional time would be required thereafter.

Preparation for image analysis
A total of 1,402 body navigation-loaded images were acquired from 29 participants. The ultrasound operator carefully selected seven images per patient. Inappropriate images, such as those captured when the transducer was moving or those that were out of focus were excluded. As a result, 203 images were selected for analysis, which formed the body navigation-loaded ultrasound image set. Subsequently, an equivalent conventional ultrasound image set was prepared. Two image sets of identical organs were created as portable document format les for image analysis.

Ultrasound image analysis and scoring
Two board-certi ed gastrointestinal radiologists (Reviewer A with 20 years of experience and Reviewer B with 10 years of experience, respectively) and one trainee radiologist (Reviewer C with 2 years of experience) independently evaluated the two image sets. The conventional ultrasound image set was evaluated rst, followed by the body navigation-loaded ultrasound image set, two weeks later.
The reviewers were instructed to assess ultrasound images based on the following: 1) recognition of the target organs (2-points) such as speci c liver segments, the extrahepatic bile duct, gallbladder, pancreas, spleen, kidneys, stomach, small and large bowels, urinary bladder, abdominal muscles (e.g., psoas muscle), aorta, or the uterus; 2) estimation of transducer location according to the nine regions of the abdomen (2-points) [7]; and 3) estimation of transducer orientation (1-point): transverse, longitudinal, or oblique orientations.
The target organ and transducer location and orientation intended by the ultrasound operator were considered gold standards, and each ultrasound image was scored from a maximum of 5 points to a minimum of 0 (Fig. 3).
For recognition of the target organ, when the target organ was correctly recognized and the directions (right or left) and exact location were accurately speci ed, a score of two points was assigned. If the direction was incorrect or the exact anatomical structure was not speci ed, one point was assigned. If it was perceived as a different organ, a score of 0 was assigned.
For the scoring of the transducer location, if it matched the ultrasound operator's answer, a score of two points was assigned, and if the location was evaluated as immediately adjacent in nine regions [7], one point was assigned.
The transducer orientation scoring was assigned one point if the orientation matched the ultrasound operator's answer. Because the transducer angle could be changed freely during the examination, an angle variation of the transducer within approximately 10° was de ned as a transverse or longitudinal orientation.

Statistical analysis
The mean difference between the evaluations was obtained by the Wilcoxon signed-rank test using IBM SPSS Statistics software, version 20.0, for Windows (IBM Corp., Armonk, NY, USA). Inter-rater agreement of three reviewers was obtained by the Fleiss' Kappa test using SAS (version 9.4; SAS Institute Inc., Cary, NC, USA). Statistical signi cance was set at P < 0.05.

Results
Twenty-two men and seven women were recruited for this study (mean age: 45.1 years, range: 27-59 years). Images of various organs of the abdomen were taken, and standard images were taken for the liver, pancreas, kidneys, and urinary bladder (Fig. 4).
For reviewer A, the interpretation scores were 4.10 ± 1.50 points for the rst analysis using the conventional ultrasound image set and 4.76 ± 0.63 points for the second analysis using the body navigation-loaded ultrasound image set. For reviewer B, the rst and second analysis scores were 3.82 ± 1.53 and 4.40 ± 0.90 points, respectively. For reviewer C, the rst and second analysis scores were 3.43 and 4.19 points, respectively. There was a statistically signi cant increase in the interpretation scores of all reviewers (P < 0.001) ( Table 1). Data are expressed as mean ± standard deviation.
Among the three reviewers, the mean score difference in each analysis session was statistically different in both the conventional ultrasound image set and the body navigation-loaded ultrasound image set ( Table 2). Data are expressed as mean ± standard deviation.
In the rst analysis, the inter-rater agreement of target organ recognition showed moderate agreement (Fleiss' Kappa value of 0.610), and the estimation of the transducer location and transducer orientation showed weak agreement (Fleiss' Kappa values of 0.425 and 0.571, respectively). In the second analysis, the kappa value in all evaluation categories increased and showed a moderate agreement (Fleiss' Kappa value from 0.619 to 0.792). In both the rst and second evaluations, the agreement for the transducer location was the lowest (Table 3).

Discussion
In this pilot study, the authors acquired body navigation-loaded ultrasound images that included information regarding the inspection site and transducer location. In the ultrasound image analysis, the mean score of interpretations of body navigation-loaded ultrasound images increased signi cantly for all raters compared with conventional ultrasound images without body marker. In addition, it was con rmed that the inter-rater agreement was improved in the second analysis that used the navigation-loaded ultrasound image set. These results were obtained not by physicians but by experienced gastrointestinal radiologists who are familiar with ultrasound imaging. Thus, body navigation-loaded ultrasound imaging is expected allow radiologists to interpret ultrasound images more accurately and objectively. The authors also believe that it will certainly be helpful to physicians who may be unfamiliar with the ultrasound image. However, based on our results, the presence or absence of navigation-loaded images did not signi cantly affect the target organ recognition of easily recognizable organs, such as the left lateral section of the liver, portal vein bifurcation, spleen, or urinary bladder. These organs were recognized nearly 100% of the time in all the primary and secondary analyses.
In the present study, there was a statistically signi cant difference in the interpretation scores of the reviewers ( Table 2). Reviewer A (senior radiologist) had a higher interpretation score. Apart from the low scores of the less experienced resident, the difference in the scores of the secondary evaluation in which the two experienced radiologists interpreted the navigation-loaded ultrasound image was caused by the transducer location interpretation. When evaluating the boundaries of nine regions of the abdomen, the senior radiologist's interpretation of the transducer location was more consistent with the operator's intention. Because subjective interpretation of the boundary area was possible, the inter-rater agreement of the transducer location also showed lower agreement than that of the other evaluation categories (Table 3). If the four abdominal quadrants with a clear anatomical landmark of the umbilicus were used as the standard for transducer location interpretation, it was expected that all raters would show a higher interpretation score and a high degree of agreement.
The most important advantage of navigation-loaded ultrasound is that it increases the operator's convenience and is expected to allow the operator to focus solely on ultrasound examination. In some cases, the operator's increased effort may be required while adding appropriate body marks or text available on the ultrasound equipment to ultrasound images. This process is cumbersome and timeconsuming because radiologists or clinicians must make a diagnosis while taking ultrasound images and add a body mark to the images using various control buttons. Recently, using commercially available products like touch screen labels and scan assistant make the process of labeling very stream-lined and easy [8]. However, the authors' navigation-loaded image can minimize this process because the exact information regarding the transducer and inspection site is automatically integrated with the ultrasound image in real time. Thus, this navigation-loaded ultrasound image is expected to assist in interpreting ultrasound images where the marking of speci c locations is required, or where the distinction between the right and left sides is important. The disadvantage of the authors' semi-automated technology compared to the existing ultrasound body mark system is that it may take a short time to x the camera and set the software just before starting the ultrasound examination.
The body navigation-loaded ultrasound technology developed by us did not require highly advanced skills or equipment, but there were some issues that had to be addressed during its development. The rst issue was to protect patients' privacy, such as faces or breasts. This study aimed to minimize the exposure of sensitive areas of the body by applying a 3D mesh lter while maintaining the shape, size, and ratio of the body taken by the 3D depth camera. A 3D depth camera allows utilize an object recognition function that cannot be implemented with a general camera, thereby the body can be animated. Additionally, the location of the transducer could appear as it was, without modi cation. The result was a more intuitive understanding of the navigation-loaded ultrasound image. The second issue was to determine how to simultaneously acquire ultrasound images and 3D depth images to increase operator convenience. Because it is not e cient to use the ultrasound image acquisition button and the camera capture button separately, this problem was solved by setting the camera to simultaneously capture an image when the ultrasound image acquisition button was pressed, using the MeshGateway software on a computer connected to the ultrasound system. The third issue was to determine how much inspection site and transducer information should be included in the ultrasound image in photos taken with a 3D depth camera. With the camera xed in a position above the patient's head, body parts other than the inspection site were cropped, and adjustments were required before starting the ultrasound examination so that the inspection site and location of the transducer did not deviate from the cropped area. After this issue was resolved, it was necessary to decide where to place a navigation-loaded image on the conventional ultrasound image. Care was taken not to overlap the information regarding the ultrasound parameters or scanned ultrasound images. Depending on the ultrasound vendor and speci c model, it was expected to be variable, but we considered it better to insert the thumbnail in the upper right part of the ultrasound image. While using convex transducers, navigation-loaded image could be properly inserted into the empty portion of the ultrasound image that appeared to be radial.
There are several limitations to this pilot study using this experimental technology. First, the diagnostic usefulness of this technology compared to the conventional ultrasound body mark system has not been evaluated. Comparing different methods of labeling images may have resulted in better outcome. Some physicians may feel that the technology has no signi cant advantage over the current graphic body mark system. Although this pilot study was conducted only on abdominal ultrasound, where the standard view protocol is widely used, it is expected to be useful in other organs such as the musculoskeletal joints, peripheral vascular systems, and breast, where the inspection site and transducer information is more important than in the abdomen. Figure 5 demonstrates that this technology appears to be useful for identifying ultrasound images of various organs. Second, protecting patients' privacy and increasing the accuracy of the inspection site and transducer information are trade-offs for each other. In this study, navigation-loaded images were acquired with a low-resolution thumbnail, focusing on the protection of the participants' privacy. In addition, we are developing a program using arti cial intelligence that automatically crops sensitive areas [9,10], removes patients' faces [9][10][11], or completely replaces them with a simple body mark according to the inspection site and maintains the appropriate eld of view. Nevertheless, due to patients' privacy issues, it is expected to be of little or limited use in gynecologic transvaginal or translabial ultrasound. The solution to this is to turn off the camera function in the software and then proceed with the conventional ultrasound examination. This method can be easily applied even if the patient refuses a new body marking system. Third, because camera installation and special software settings are required before starting the ultrasound scan, this process would be quite cumbersome. To overcome these shortcomings, the authors plan to develop an embedded system that applies the aforementioned arti cial intelligence technology. Nevertheless, the authors' method is still applicable only to a permanent ultrasound room, and it will be di cult to apply to a portable ultrasound system. Fourth, our system is expected to be inaccurate for cineloop images because it only provides information about the static ultrasound transducer. However, we will be able to express the movement of the probe if necessary through software improvements in the future. Fifth, although the inter-rater agreement in the body navigation-loaded image increased, the kappa value did not increase almost perfectly. Thus, it cannot completely replace existing annotation methods such as text or arrows in a speci c condition.

Conclusions
According to this pilot study, body navigation-loaded ultrasound technology is expected to assist in interpreting ultrasound images of suitable organs easily and objectively, and it is expected to improve the operator's convenience by reducing the use of body marks or text mark systems. In addition, the authors are expected to be able to apply this technology to various body organs through further improvements in the technology.    Ultrasound image interpretation and scoring. (a) This case was evaluated as an oblique scan of the left hypochondriac region by identifying the ultrasound image to be of a spleen in the rst interpretation. However, in the second interpretation, it was con rmed that the transducer location was an epigastric region, and thus, reviewer B could accurately identify that the liver dome was captured by ultrasound. (b) In the rst interpretation session of reviewer B, the transverse scan of the kidney can be perceived; however, the left and right kidneys cannot be distinguished. Therefore, the rst interpretation scored 2 points. However, the second interpretation using body-navigation-loaded ultrasound images shows that it is a transverse scan of the right kidney.

Figure 4
Application of body navigation-loaded ultrasound in the abdomen of one of the participants. The pancreatic body, several hepatic regions, gallbladder, extrahepatic bile duct, right kidney, left kidney, and urinary bladder are shown in order. The last ultrasound in the middle row is an image of the extrahepatic bile duct (asterisk), indicating that it has been taken in the left lateral decubitus position.