Methods
- Hardware configuration
The evaluation system consists of (1) the da Vinci research kit (dVRK), (2) a simple four DOF ECS, and (3) the stereo viewer and VR HMD for 3D stereoscopy. This system is divided into two subsystems (Fig. 2). As shown in Fig. 2a, the Ubuntu PC sends control commands from the MTM to the PSM. A windows PC displays 3D stereoscopic images captured from the ECS to the display devices, as shown in Fig. 2b.
1.1 The da Vinci research kit
The dVRK is a surgical robot system for research that is based on the da Vinci surgical robot system. It comprises a pair of MTMs and PSMs, control boxes, a foot pedal, and the stereo viewer (Fig. 3). The stereo viewer provides 3D stereo images to the users, the PSMs mimic the movement of the MTMs and the control boxes transfers the commands between the PSMs and MTMs. The foot pedal is used for additional functions.
1.2 A Simple Four DOF ECS
Since the ECS is not a basic component of the dVRK, a previously developed four DOF ECS is used to obtain 3D stereoscopic images (Fig. 2b) [34]. The ECS used in this research has a couple of complementary metal-oxide semiconductor (CMOS) cameras placed horizontally at the end. The images captured from the CMOS cameras are sent to the display devices as a stereoscopic image through a stereo calibration process.
1.3 Stereo viewer and VR HMD for 3D stereoscopy
The stereo viewer transmits different images from each of the left and right displays to obtain 3D stereoscopic images through the binocular parallax. It is designed to use a posture as shown in Fig. 4a. The user sits on a stool or chair and leans on an armrest looking down into the lenses. The images captured from the cameras go through a stereo calibration that uses an OpenCV library (C++) on Microsoft Visual Studio 2019 (Microsoft, Redmond, WA, USA).
The VR HMD used in this study is the VIVE Pro Eye (HTC, New Taipei, Taiwan). It uses high-definition displays with a combined resolution of 2,880 × 1,660 pixels and 615 pixels per inch with 110° wide FOV; it is worn as shown in Fig. 4b. Unlike the stereo viewer, the user operates the dVRK in a comfortable position while not looking down. When the participants wear the VR HMD, it is emphasized that the VR HMD is securely fixed in a correct position. However, if the VR HMD is loose or not positioned correctly, the images will be out of focus and blurred, which makes it impossible to properly see the objects on the display.
The VR HMD is used with sensors that are mounted diagonally in the corner of the room that is above a person’s height, and it slightly looks down to detect the position and movement of the VR HMD. However, in this study, the stereo viewer limited the detection area between the VR HMD and the sensors. If the connections are lost, the VR HMD considers that the VR HMD is removed from the user and it stops streaming the images. Therefore, the sensors are installed in optimal positions that are set empirically to maintain the connection between the VR HMD and the sensors regardless of the head position during the evaluation (Fig. 5). In addition, a function changing the source of the cameras between the front of the VR HMD and the ECS was applied to reduce the disconnection.
- Evaluation process
The evaluation aims to verify the replacement of the stereo viewer with the VR HMD by comparing the performance and ergonomic aspects of a 3D stereoscopic display device while using the quantitative and qualitative results of each system with the stereo viewer or the VR HMD. The overall evaluation process is detailed in this section.
Three surgeons and seven novices participated in the study. The surgeons are experts in the field of robotic surgery, with specializations in thyroid surgery, gastrointestinal surgery, and urology. In contrast, the novices are graduate students in engineering. The objectives, processes, precautions, and the explanations on how to use the system were provided before the peg transfer tasks. After the introduction, the warm-up peg transfer task was performed for 7 min by using the stereo viewer and the VR HMD. As a quantitative evaluation, two stages of peg transfer tasks were then performed for 32 min in total, and each stage was performed in the order of the stereo viewer and the VR HMD for 8 min per device. The process was completed by performing a qualitative evaluation, which includes a questionnaire that is designed for this research and a modified NASA-TLX, which is a subjective and multidimensional workload assessment tool.
2.1 Peg transfer task – Quantitative evaluation
A peg transfer task is a widely used evaluation method for MIS, laparoscopy, and robot surgery. As shown in Fig. 6, the participants are instructed to grasp one ring with the left PSM (PSM_L) from the left peg and to hand it over to the right PSM (PSM_R) (Fig. 6a, b). They were then instructed to place the ring on the right pegs. It is referred to as “one execution is performed” by finishing the transfer of one ring from one peg to another. When all rings are transferred to the right side, the aforementioned process is performed in the reverse order (Fig. 6c). The participants continued the “execution” for 8 min. During these tasks, the execution time and the number of executions were acquired. Subsequently, to exclude the unintended effect of mistakes or coincidences, only the values within a 99.9% confidence interval were reflected in the statistics to enhance the objectivity of the results.
For the evaluation of the performance of the display devices, these results were categorized into two indicators: “average execution time” and “total number of executions.” The “average execution time” is the average of the execution time records of each participant and the total number of executions is the total number of executions that are performed by the participants during each stage. Meanwhile, the goal of this study is to compare the ergonomic characteristics between the two imaging devices with their performance as imaging devices. Therefore, the controls of the ECS were not considered and the ECS was fixed at a proper position within the sight of all six pegs.
2.2 Questionnaire and modified NASA-TLX – Qualitative evaluation
After the peg transfer task, the participants were asked to fill in the questionnaire and “modified NASA-TLX” [see Additional file 1]. The questionnaire consists of three sections: (1) visual and ergonomic performance, (2) comparison between the devices, and (3) comments and feedback.
First, the “visual and ergonomic performance” section consisted of eight questions that were rated on a 5-point Likert scale. This section comprehended the user experience that is related to the visual aspect, physical restriction, and the benefits from the replacement of the stereo viewer to the VR HMD. The options of each question were converted into scores that ranged from 1 to 5 with a reference score of 3; therefore, the closer the score is to 5, the more positive answers were obtained for the VR HMD; the closer the score is to 1, the more negative answers were obtained for the VR HMD.
Second, the “comparison between the devices” section consisted of five multiple-choice questions. In this section, the participants compared the stereo viewer to the VR HMD for five criteria and they were rated on seven levels with a reference level of four. The closer the grade is to seven, the better the VR HMD device is than the stereo viewer for that criteria; the closer the grade is to one, the better the stereo viewer is than the VR HMD for that criteria.
Third, the “comments and feedback” section consisted of three subjective questions. These questions were designed to confirm the unexpected discomforts and the various opinions of the surgical robot system. The “modified NASA-TLX” was based on the original NASA-TLX, which is a common workload assessment tool that rates a task in six categories (physical, mental, temporal, effort, performance, and frustration). The “modified NASA-TLX” determined how difficult the task is for each category by comparing the stereo viewer with the VR HMD. Raw scores with a range of 0 to 100 implied the workload of the VR HMD were converted into workload scores with a range of 1 to 6. The workload of the stereo viewer was set to 50 for the raw score, and it was converted to the reference value, which was 3.5 points for a concise comparison between the two devices.
Availability of data and materials
The data used and analyzed during the current study are available from the corresponding author on reasonable request.