Development & Evaluation of Virtual Reality (VR) based Novel Vision System for the da Vinci Surgical Robot

Background: Robot surgery has become prevalent because of its various advantages as a progressive method based on empirical researches of conventional open surgery and minimally invasive surgery. However, the da Vinci surgical robot system, the most widely used and researched surgical robot, still requires an ergonomic improvement because of the uncomfortable posture in which it has to be operated. The stereo viewer—the current vision system of the da Vinci surgical robot—requires a user to maintain a posture wherein the user is looking down, which causes discomfort and results in musculoskeletal disorders. To overcome this limitation, a virtual reality (VR) head-mounted display (HMD) is proposed by previous researchers as an appropriate option to replace the stereo viewer, as it enables surgeons to move freely during surgery instead of having to look down on the stereo viewer. Presently, there is no direct comparison between the stereo viewer and a VR HMD by surgeons. Comparative evaluations were performed using peg transfer tasks, a questionnaire, and a NASA-Task Load Index (TLX). These were planned and performed by surgeons and novices to determine if the stereo viewer can be replaced by the VR HMD and to investigate whether the VR HMD has ergonomic. Results: Based on the results of peg transfer tasks, completion times when using VR HMD were shorter than those when using the stereo viewer. In these tasks, the participants performed more executions using the VR HMD compared to the stereo viewer. Based on the questionnaire, the participants favored the VR HMD compared to the stereo viewer, with respect to its visual and ergonomic performance. The modied NASA-TLX showed positive perceptions for the VR HMD. Conclusions: This comparative evaluation conrmed that the VR HMD can be employed as a potential alternative for the stereo viewer in a surgical robot system to achieve ergonomic improvements. The VR HMD improved the task performance of the surgical robot system, and it provided an ergonomic operation environment.


Background
Until the early 20th century, most surgeries were performed as open surgeries with large incisions at the surgical site and the capability of the surgeon was judged based on the size of the incision they made during the operation. Open surgery leads to considerable amounts of blood loss, complications after surgery, surgical-site infections, and it forces the patient to stay at the hospital for a long duration [1][2][3][4].
Minimally invasive surgery (MIS) reduces post-operative pain, blood loss, recovery time, and the size of the scars, which allows patients to recover faster than open surgery [5,6]. Today, MIS operations are prevalent in several hospital departments, and they are employed in all departments in the medical eld.
Further, a MIS is also called an endoscopic surgery because the surgeon relies on an endoscope to view the inside of the patient's body.
However, endoscopic surgery is limited by ve degrees of freedom (DOF) and restrictions caused by hand tremors [7] can be induced because of fatigue and the surgeon's loss of concentration, which hinders the surgery. To overcome these problems, a master-slave surgical robot system is introduced, wherein the controller side is spatially separated from the actuator side. The surgical robot system has a high DOF that tolerates a larger range of motion and it eliminates hand tremors that allows improved precision when using surgical tools. Further, robotic surgery shows better post-operative results than conventional endoscopic surgery [8].
Stereoscopy is an indispensable aspect to consider in MIS. Owing to the lack of haptic feedback in MIS, visual feedback is an essential part of surgical operation. One study highlighted the signi cance of visual feedback for the delicate manipulation of surgical robots [9]. Thus, most MIS operations, including robotic surgery, require high de nition stereoscopic images. Stereoscopic images can be created in a variety of ways such as using an anaglyph, a shutter, and polarization; however, these methods use a single display and have the disadvantages of a low resolution, icker, ghosting, and discomfort that is caused by posture limitation [10][11][12]. However, a dual-display system can overcome these drawbacks since it provides high-de nition images with realistic depth perception. The da Vinci surgical robot system (Intuitive Surgical, Sunnyvale, CA, USA), which is widely used globally, provides 3D stereoscopic images to the surgeon through a dual-display stereo viewer that helps the surgeon minimize operation time and errors during surgery by providing improved spatial senses.
Although the da Vinci surgical system offers 3D stereoscopic images with high depth perception, it requires ergonomic improvements because the stereo viewer forces the surgeon to have an uncomfortable posture; thus, causing pain to the neck, back, and shoulder. Ergonomic research studies on the da Vinci surgical system indicate that the surgeon needs to maintain an ergonomically bad posturelooking down vertically-when operating the system because of the structural limitations of the da Vinci surgical system console [13][14][15][16]. Moreover, the most widely used ergonomic evaluation tool, the rapid upper limb assessment (RULA), indicates that the best posture is to keep the neck straight, and it is harmful to look down or perform overextension [17]. Looking downward strains the neck and shoulder muscles and this deforms the cervical vertebrae, which causes musculoskeletal disorders, such as "military neck" or "turtle neck" [18]. It is recommended to maintain a neutrally vertical position of the neck [13][14][15][16][17][18]. Therefore, we suggest solving these issues by replacing the stereo viewer with a head-mounted display (HMD) as a comfortable and compact device to mitigate the burden of the surgeon's posture. The HMD is a display that is mounted on the head that provides immersive images with a wide eld-of-view (FOV). The surgeon can move the upper body freely using the HMD as the device is xed on the head and the screen is always in front of the surgeon regardless of the posture. Thus, the surgeon is not forced to maintain an uncomfortable posture, which will help minimize the fatigue on the surgeon's upper body.
Owing to the advantages of the HMD, various attempts have been made to introduce it to the medical eld over the past few years [19][20][21]. However, HMD models before the Oculus DK1 (Oculus, Menlo Park, CA, USA) showed inferior results compared to the existing devices because of their low performance and high cost [22]. After performing research on the Oculus, new research on virtual reality (VR) HMDs in a variety of elds emerged after the VR HMDs with high-resolution displays and their low fabrication costs were released [23][24][25][26]. For a surgical robot system, Zinchenko et al. proposed using a VR-based endoscope control system (ECS) for MIS; however, Zinchenko et al. did not consider speed control and collision avoidance [27]. Jo et al. suggested an improved VR-based ECS that uses a collision avoidance strategy [28]. From their research, the movement speed of the ECS varies based on the degree of inclination or rotation at each axis. Further, the collision avoidance algorithm was evaluated to demonstrate that the algorithm can prevent collisions between the surgical tools and the endoscope when the intention of the surgeon is blended with a compensation signal in a proper ratio. Dardona et al. performed a comparison of the control interfaces for the ECS between the VR HMD and a conventional clutched method [29]. The participants indicated that the VR HMD is easier to learn and it is comfortable since it does not cause dizziness or physical inconvenience. Several studies have shown that using the VR HMD is e cient in a medical environment, even in the operating room [30][31][32][33]. Although previous studies have proved the advantages of the VR HMD, these studies lack clinical opinions and analyses on surgeons that are skilled with surgical robot systems. Without clinical opinions, it is uncertain whether the VR HMD is suitable for the surgical robot. This is because surgical robots are used by skilled surgeons that specialize in areas such as thyroid surgery, gastrointestinal surgery, and urology. In these elds, surgery is very delicate; accidents can result in serious threats, and possibly death, to the patient. Thus, their opinions are deemed to be the most important in determining the validity of adapting the VR HMD on a surgical robot, which includes the advantages and the effectiveness of the VR HMD from a clinical point-of-view. Without their professional opinion, it is unclear whether the suggested system is valid in an operating room.
In this study, a series of comparative evaluations performed by surgeons and novices were conducted to verify that the stereo viewer can be replaced with a VR HMD for the da Vinci surgical robot system and to determine if the VR HMD has ergonomic advantages. The performance of the VR HMD was demonstrated through the quantitative and qualitative analyses of peg transfer tasks and questionnaires that includes modi ed NASA-TLX results. The analyses indicate that the VR HMD outperforms the stereo viewer in terms of quantitative results for surgeons and novices and it also received positive satisfaction reviews from a qualitative perspective. Based on these results by the surgeons and novices, the VR HMD is an appropriate replacement for the stereo viewer. In addition, the VR HMD will improve the task e ciency and provide convenient surroundings to the surgeon with a more comfortable posture and low task load compared with the stereo viewer. Finally, from the standpoint of a skilled surgeon, the VR HMD can be applied to surgical robots.

Surgeons
The results of the peg transfer task performed by three surgeons showed that they achieved better performance when using the VR HMD compared to using the stereo viewer ( Table 1). The average execution times decreased from stage 1 to stage 2. Besides, in stage 1 for surgeon 2, the average execution time was reduced, and the total number of executions increased when using the VR HMD. When comparing both devices, the difference between stages 1 and 2 for the VR HMD was larger than that for the stereo viewer in terms of the average execution time (

Novices
The results of the peg transfer task for the seven novices showed that when the VR HMD was used, it achieved a better performance compared to using the stereo viewer in general ( Table 2). Except for stage 1 for novice 2, the average execution time in most novices was reduced, and the total number of executions increased for the VR HMD. When comparing both devices, the variation between stages 1 and 2 of the VR HMD was larger than the stereo viewer in terms of the average execution time (Stereo viewer: −0.4 s, VR HMD: −3.6 s), and the total number of executions (Stereo viewer: +2.9 rep., VR HMD: +4.1 rep.).

Overall results
The VR HMD showed a better performance than the stereo viewer as summarized in Tables 3 and 4. The results of surgeon 2 and novice 2 were excluded because of the intervention during the process. As demonstrated in the overall average, the execution time was decreased while the total number of executions was increased when using the VR HMD (Tables 3, 4). The variation in the results for novices 4 and 7 showed opposite signs; however, the differences in the data are negligible.  The results for the three surgeons are summarized in Tables 5 and 6. The participants provided scores between 3.0 and 4.7 with respect to the "visual and ergonomic performance" section of the VR HMD. The highest score (4.7) was obtained for a question that asked if there was a time-lag between the patientside manipulator (PSM) control and the display of the VR HMD. However, they obtained relatively low scores of 3.3 and 3.0 for questions with respect to the weight and pressure, respectively. The "comparison between the devices" section scored 5.0 or higher from all ve perspectives. In particular, the "immersiveness" received the highest rating of 6.0.

Novice
The results of the seven novices are listed in Tables 5 and 6. In the "visual and ergonomic performance" section, the participants provided scores that ranged from 3.4 to 4.4. They provided the highest score (4.4) for the Master tool manipulator (MTM) workspace expandability. The lowest scores (3.4) were assigned to the weight and pressure, which is similar to the responses of the surgeons. The "comparison between the devices" section scored 5.0 or higher for all ve perspectives. The "immersiveness" was rated as high as the results for the surgeons.

Overall results
The overall results of all ten participants are listed in Tables 5 and 6. The "visual and ergonomic performance" section of replacing the stereo viewer with the VR HMD received positive responses that were above the average value (3.0) for all questions. However, it displayed slightly low scores of 3.4 and 3.3 for the weight and pressure, respectively. Meanwhile, the overall scores in the "comparison between the devices" section were above 5.0, and the "immersiveness" achieved the highest score. Table 5 Average scores of the "visual and ergonomic performance" by the groups. The results of the three surgeons are shown in Fig. 1a. Besides the "performance," the median values were 3.0 and they showed that the workload was lower for the VR HMD than the stereo viewer in general. The workload score of the "performance" was rated in the range of 3 to 5 points.
The results of the four novices are shown in Fig. 1b. The VR HMD's workload score was generally lower than the stereo viewer. In particular, the workload scores of the "performance" and "frustration" were relatively low in all six categories with 2 and 1.5 points, respectively.
Overall, the results of the seven participants showed that the workload score of the VR HMD is lower than the stereo viewer in general (Fig. 1c).

Discussion
With the proposed system, a series of comparative evaluations were performed between the stereo viewer and the VR HMD. This evaluation was performed to validate the suitability of the VR HMD for a surgical robot system instead of the stereo viewer from the view point of the surgeon and the novice.
According to most results of the peg transfer tasks, the VR HMD displayed improved results compared to the stereo viewer, as summarized in Tables 3 and 4. In particular, there was reduction in the average execution times and an increase in the total number of executions that resembles the results of Jo et al. and Dardona et al. [28,29]. In the VR HMD, the average execution time decreased by up to 21.18%, and the total number of executions increased by up to 21.02%, which makes the difference larger as the stage progresses (Tables 3 and 4). It seems as if the participants became used to the sense of distance of the VR HMD, which resulted in a better performance. And there were no signi cant differences whether the participants are wearing eyeglasses or not when comparing the results of all participants. The average execution time was shorter in the novice group than the surgeon group when using the VR HMD because the surgeons are more familiar with the stereo viewer, which resulted in them spending more time to get acquainted with the VR HMD. However, the surgeons were pro cient in robotic systems, which shows more consistent results than the novices. In the novice group, the difference in the average execution time was greater than the surgeons. This seems to come from the individual differences such as the VR HMD experience and dexterity. Thus, the difference in the quantitative results between the novices and surgeons using the VR HMD system relies on their empirical or innate familiarity with the VR HMD.
Meanwhile, surgeon 2 and novice 2 showed different results from the other participants in stage 1 of the peg transfer tasks. There were some interventions during the peg transfer tasks with surgeon 2 and novice 2. In stage 1 for surgeon 2, the surgeon had no experience with a VR HMD and he did not fully tighten the hairband when mounting the VR HMD on his head. In a VR HMD system, it is important to tighten the hairband; by loosely wearing the VR HMD, this can blur the scene. Stage 1 showed a difference because the blurred view that is affected by the performance of surgeon 2; however, in stage 2, the results were similar to those of the other participants as an additional check that was performed to ensure that the VR HMD was tightly fastened on the head (Table 1).
In stage 1 for novice 2, the participant showed a high astigmatism that was caused by a delayed adaption to the VR HMD. The participant reported that one object in sight was observed by double vision and that they experienced dizziness. Thus, additional stereo calibration was required for novice 2 to relieve double vision and dizziness. Although dizziness persisted even after calibration, stage 1 was conducted to maintain the same environment as the other participants and to follow protocol. Persistent dizziness and astigmatism in novice 2 may have led to the difference from other participants in the stage 1 result. However, after stage 1, novice 2 showed better results compared to stage 1, which is similar to surgeon 2 ( Table 2). For both participants, after troubleshooting the problems, the results were improved in stage 2 and they are similar to the results of the other participants (Tables 1 and 2).
As mentioned earlier, the familiarity of the VR HMD affects the quantitative results; however, the results can also be in uenced by how the device is mounted or set up. According to Tables 3 and 4, the familiarity or settings of the VR HMDs affects the quantitative results; however, the tendency of the performance improvement using the VR HMD system is not in uenced by the presence of intervening participants. Thus, the VR HMD based surgical robot system is better than the system with the stereo viewer or is at least alike in the quantitative results.
Based on the questionnaire results, the VR HMD showed some advantages in terms of ergonomics that are similar to other studies [13,17]. The overall scores of the questionnaire achieved at least 75% of the maximum score. This means that the VR HMD performs better than the stereo viewer for all criteria that are designed to compare the stereo viewer versus the VR HMD (Table 5, 6). In particular, the "blocking view" and "immersiveness" showed the most positive responses, which indicates that the view was not hampered by the VR HMD and the participants could focus better on the task using the VR HMD.
Meanwhile, the participants reported relatively low scores for the "sense of weight and pressure" (Table 5) in the questionnaire. It is presumed that the structural limitations of the VR HMD itself and the familiarity of the new device in uenced the scores. In the section of "workspace expandability," the surgeons who used the surgical robot system frequently reported low scores since they were accustomed to the discomfort that is caused by the limited workspace of the MTM because of the stereo viewer (Table 5).
Considering the volume of the devices, the volume of the stereo viewer is ~ 220,000 cm 3 , and the volume of the VR HMD is ~ 11,000 cm 3 , which is only about 5% of the stereo viewer. Thus, replacing the stereo viewer with the VR HMD can signi cantly reduce the size of the surgical robot system and augment the space e ciency during the operation. As aforementioned, the overall responses to the questionnaire were positive, and the structural improvements of the VR HMD and the continuous experiences of the system can enhance the user satisfaction.
Based on the results of the modi ed NASA-TLX, the VR HMD was rated slightly easier in six criteria for the surgeons and novices, except for the "performance" section from the surgeon group (Fig. 1a). Although all of the data were concealed until the evaluation was nished, the surgeons indicated that it was di cult to use the VR HMD in the "performance" section. Since they are experts with surgical robot systems, they expected a higher accomplishment and were not satis ed with the results of the peg transfer task. Even so, the overall rating indicated that the VR HMD is easier than the stereo viewer (Fig. 1c). Thus, the VR HMD is a suitable alternative to the stereo viewer in a surgical robot system according to the overall qualitative results.
At the end of each evaluation, the participants provided comments and feedback regarding the inconveniences that are caused by the surgical robot system and the usage of the VR HMD. They pointed out the inconveniences of the existing surgical system for two different aspects. First, they need to maintain a discomfort posture using the surgical robot. Second, they felt inconvenience and fatigue from the limitation of the MTM's DOF and workspace. Further, the VR HMD is narrowly used as a display device in this study. Therefore, based on these limitations and inconveniences, it will be possible to enhance the surgical robot system by implementing attractive functions that are adapted on the VR HMD in future investigation. For example, the use of a position-adjustable MTM suitable for VR HMDs or substituting existing MTM for a wireless controller results in a restriction-less and fatigue-less convenience control. In addition, consolidating existing interfaces from reality to virtual circumstances, using hand gestures for control methods, or overlaying 3D model anatomy on the patient's body would greatly improve the usage of VR HMD for the surgical robot system.
Consequently, for the quantitative and qualitative results for the novices and surgeons, the VR HMD system exhibited similar or improved usability or suitability when compared to the conventional stereo viewer in a surgical robotic system. Moreover, it is expected that the VR HMD system will not only solve problems such as a military neck or turtle neck by providing ergonomic posture convenience to a surgeon performing robotic surgery, but it will also maximize the e ciency of the robotic surgery and the ability of the surgeon. Therefore, the VR HMD has su cient advantages to replace the conventional stereo viewer.

Conclusions
From the evaluations, the VR HMD displayed similar or better visual and ergonomic performances in comparison to the stereo viewer. The participants were rapidly acquainted to the VR HMD with little compliance issues regardless of whether they were wearing eyeglasses or not. As shown in the quantitative results, the participants performed peg transfer tasks faster when using the VR HMD.
Although there are some individual differences, the results improved when the participants used the VR HMD in general. For the qualitative results, the participants reported that the VR HMD was better than the stereo viewer from a visual and ergonomic aspect.
Based on the results, if the stereo viewer is replaced with the VR HMD, the overall performance of the surgical robot system is expected to increase in several aspects. First is the high task e ciency from the concentration retention with a more immersive view and a wide eld of view (FOV). Besides, ergonomic improvements have been made and users were no longer needed to have an uncomfortable posture during the operation, which can reduce fatigue. Further, it is expected that the space-e cient utilization of the operating room will be possible because of the volume reduction of the surgical robot that is accompanied by structural modi cations.
Therefore, the substitution of the stereo viewer with the VR HMD is reasonable and bene cial for a variety of features that is based on professional and general perspectives of the surgeons and novices.
Accordingly, it is expected to improve the convenience of medical personnel and it allows safer operations to the patients. These enhancements provide a new paradigm to the surgical robot system, and they improve the surgical environment. In addition to the bene ts of the VR HMD that have been veri ed in this research, it is expected that the application of VR technologies on a surgical robot will aid surgeons and medical personnel. This includes transferring the physical interfaces to a virtual space, overlaying 3D models of internal organs on a screen, and etc. Thus, the VR HMD can be a great alternative to the stereo viewer in a surgical robot system.

Hardware con guration
The evaluation system consists of (1) the da Vinci research kit (dVRK), (2) a simple four DOF ECS, and (3) the stereo viewer and VR HMD for 3D stereoscopy. This system is divided into two subsystems (Fig. 2).
As shown in Fig. 2a, the Ubuntu PC sends control commands from the MTM to the PSM. A windows PC displays 3D stereoscopic images captured from the ECS to the display devices, as shown in Fig. 2b.

The da Vinci research kit
The dVRK is a surgical robot system for research that is based on the da Vinci surgical robot system. It comprises a pair of MTMs and PSMs, control boxes, a foot pedal, and the stereo viewer (Fig. 3). The stereo viewer provides 3D stereo images to the users, the PSMs mimic the movement of the MTMs and the control boxes transfers the commands between the PSMs and MTMs. The foot pedal is used for additional functions.

A Simple Four DOF ECS
Since the ECS is not a basic component of the dVRK, a previously developed four DOF ECS is used to obtain 3D stereoscopic images (Fig. 2b) [34]. The ECS used in this research has a couple of complementary metal-oxide semiconductor (CMOS) cameras placed horizontally at the end. The images captured from the CMOS cameras are sent to the display devices as a stereoscopic image through a stereo calibration process.

Stereo viewer and VR HMD for 3D stereoscopy
The stereo viewer transmits different images from each of the left and right displays to obtain 3D stereoscopic images through the binocular parallax. It is designed to use a posture as shown in Fig. 4a.
The user sits on a stool or chair and leans on an armrest looking down into the lenses. The images captured from the cameras go through a stereo calibration that uses an OpenCV library (C++) on Microsoft Visual Studio 2019 (Microsoft, Redmond, WA, USA).
The VR HMD used in this study is the VIVE Pro Eye (HTC, New Taipei, Taiwan). It uses high-de nition displays with a combined resolution of 2,880 × 1,660 pixels and 615 pixels per inch with 110° wide FOV; it is worn as shown in Fig. 4b. Unlike the stereo viewer, the user operates the dVRK in a comfortable position while not looking down. When the participants wear the VR HMD, it is emphasized that the VR HMD is securely xed in a correct position. However, if the VR HMD is loose or not positioned correctly, the images will be out of focus and blurred, which makes it impossible to properly see the objects on the display.
The VR HMD is used with sensors that are mounted diagonally in the corner of the room that is above a person's height, and it slightly looks down to detect the position and movement of the VR HMD. However, in this study, the stereo viewer limited the detection area between the VR HMD and the sensors. If the connections are lost, the VR HMD considers that the VR HMD is removed from the user and it stops streaming the images. Therefore, the sensors are installed in optimal positions that are set empirically to maintain the connection between the VR HMD and the sensors regardless of the head position during the evaluation (Fig. 5). In addition, a function changing the source of the cameras between the front of the VR HMD and the ECS was applied to reduce the disconnection.

Evaluation process
The evaluation aims to verify the replacement of the stereo viewer with the VR HMD by comparing the performance and ergonomic aspects of a 3D stereoscopic display device while using the quantitative and qualitative results of each system with the stereo viewer or the VR HMD. The overall evaluation process is detailed in this section.
Three surgeons and seven novices participated in the study. The surgeons are experts in the eld of robotic surgery, with specializations in thyroid surgery, gastrointestinal surgery, and urology. In contrast, the novices are graduate students in engineering. The objectives, processes, precautions, and the explanations on how to use the system were provided before the peg transfer tasks. After the introduction, the warm-up peg transfer task was performed for 7 min by using the stereo viewer and the VR HMD. As a quantitative evaluation, two stages of peg transfer tasks were then performed for 32 min in total, and each stage was performed in the order of the stereo viewer and the VR HMD for 8 min per device. The process was completed by performing a qualitative evaluation, which includes a questionnaire that is designed for this research and a modi ed NASA-TLX, which is a subjective and multidimensional workload assessment tool.

Peg transfer task -Quantitative evaluation
A peg transfer task is a widely used evaluation method for MIS, laparoscopy, and robot surgery. As shown in Fig. 6, the participants are instructed to grasp one ring with the left PSM (PSM_L) from the left peg and to hand it over to the right PSM (PSM_R) (Fig. 6a, b). They were then instructed to place the ring on the right pegs. It is referred to as "one execution is performed" by nishing the transfer of one ring from one peg to another. When all rings are transferred to the right side, the aforementioned process is performed in the reverse order (Fig. 6c). The participants continued the "execution" for 8 min. During these tasks, the execution time and the number of executions were acquired. Subsequently, to exclude the unintended effect of mistakes or coincidences, only the values within a 99.9% con dence interval were re ected in the statistics to enhance the objectivity of the results.
For the evaluation of the performance of the display devices, these results were categorized into two indicators: "average execution time" and "total number of executions." The "average execution time" is the average of the execution time records of each participant and the total number of executions is the total number of executions that are performed by the participants during each stage. Meanwhile, the goal of this study is to compare the ergonomic characteristics between the two imaging devices with their performance as imaging devices. Therefore, the controls of the ECS were not considered and the ECS was xed at a proper position within the sight of all six pegs.

Questionnaire and modi ed NASA-TLX -Qualitative evaluation
After the peg transfer task, the participants were asked to ll in the questionnaire and "modi ed NASA-TLX" [see Additional le 1]. The questionnaire consists of three sections: (1) visual and ergonomic performance, (2) comparison between the devices, and (3) comments and feedback.
First, the "visual and ergonomic performance" section consisted of eight questions that were rated on a 5point Likert scale. This section comprehended the user experience that is related to the visual aspect, physical restriction, and the bene ts from the replacement of the stereo viewer to the VR HMD. The options of each question were converted into scores that ranged from 1 to 5 with a reference score of 3; therefore, the closer the score is to 5, the more positive answers were obtained for the VR HMD; the closer the score is to 1, the more negative answers were obtained for the VR HMD.
Second, the "comparison between the devices" section consisted of ve multiple-choice questions. In this section, the participants compared the stereo viewer to the VR HMD for ve criteria and they were rated on seven levels with a reference level of four. The closer the grade is to seven, the better the VR HMD device is than the stereo viewer for that criteria; the closer the grade is to one, the better the stereo viewer is than the VR HMD for that criteria.
Third, the "comments and feedback" section consisted of three subjective questions. These questions were designed to con rm the unexpected discomforts and the various opinions of the surgical robot system. The "modi ed NASA-TLX" was based on the original NASA-TLX, which is a common workload assessment tool that rates a task in six categories (physical, mental, temporal, effort, performance, and frustration). The "modi ed NASA-TLX" determined how di cult the task is for each category by comparing the stereo viewer with the VR HMD. Raw scores with a range of 0 to 100 implied the workload of the VR HMD were converted into workload scores with a range of 1 to 6. The workload of the stereo viewer was set to 50 for the raw score, and it was converted to the reference value, which was 3.5 points for a concise comparison between the two devices.

Availability of data and materials
The data used and analyzed during the current study are available from the corresponding author on reasonable request.