Construction and Evaluation of a Virtual Reality Laparoscopic Surgery Collaborative Training Platform: Taking Laparoscopic Cholecystectomy as an Example

Backgroud: Virtual reality (VR) technology represents the future of medical education due to its unique advantages, especially with the Covid-19 pandemic lasting. We developed a laparoscopic VR surgery collaborative training platform hoping to shed light on future medical education in China. Methods: We constructed a VR surgery training platform and designed surgery curriculum on laparoscopic cholecystectomy (LC). 36 rst-year postgraduate students in China standardized training program for resident doctor (C-STRD) from the Third Xiangya Hospital of Central South University were enrolled for validation trials. In the Phase I trial, 12 students performed LC in the exploration mode. After training in the surgery learning mode, they performed LC again. The LC scores before and after training were compared. In the Phase II trial, another 12 students were randomly assigned to either the collaborative group or the control group. The former trained with a senior surgeon collaboratively in the surgery learning mode and then performed LC alone in the exploration mode. The latter trained in the surgery learning mode by themselves and performed LC in the exploration mode. The LC scores between groups were compared. The user experience (intention to use, skills improvement, usability, degree of enjoyment) were analyzed through questionnaires from the above 24 students. Interest in surgery learning of Phase I students was compared with 12 students who didn’t experience the VR platform. Results: In Phase I trial, the mean LC scores of the students were elevated from 56.83 to 61.17 (p=0.042) after learning in surgery learning mode. In Phase II trial, collaborative group students had higher scores than their rivals (67.17 vs 61.33, p=0.014). Most students have a positive users’ experience regarding the intention to use and skills improvement. Collaborative group students had higher evaluation regarding usability. Students who experienced the VR platform were signicantly more interested in future surgery learning (3.60 vs 2.58, p <0.05). Conclusion: Our study constructed a VR platform for collaborative surgery training, which showed an excellent training effect. Medical students rated the platform highly, and their interest in learning increased.

practice for the purpose of training a future physician. Surgery training is the most empirical and practical discipline in C-STRD and as in resident programs of other countries.
Traditional hands-on learning opportunities in the operation room are vital but scarce for postgraduate students enrolled in the surgical residents training programs around the world, which relies on the hospital size and patient number [3]. A survey study of general surgery residents showed that 78% participants had managed robotic-assisted cases, but over 50% of them had not received any formal training on the robotic platforms. [4] As for Chinese medical students in C-STRD, hands-on surgical opportunities are also very precious and limited. There are reasons why their hands-on skills cannot always be obtained through repetition-based practice in the operation room, the primary one being the patient safety concerns. Besides, the mandatory social distancing caused by the Covid-19 pandemic has also brought challenges to traditional Halstead's apprenticeship mode classes for surgical students, [5] forcing them to shift to simulator courses.
In uenced by other high-stakes elds like the military, in which simulation is a necessity before the reallife experiments, surgical training has incorporated simulators to optimize the surgical learning curve [6].
These simulations are extra crucial for disciplines with a steep learning curve like laparoscopic surgical techniques. As a representative of minimally invasive surgery, laparoscopic surgery has become a mandatory technique for C-STRD students in China.
Various simulations have been utilized to facilitate the acquisition of laparoscopic skills outside the highstakes operating room. Although simulations like animal, dry-lab models and cadavers can avoid the risks in actual surgery, they have drawbacks such as high costs, ethical issues, and limited teaching resources. Therefore, these clinical simulations need an upgrade meet to the needs of medical education around the globe.
Virtual Reality (VR) technology in medical education has attracted much attention. VR produce a digital environment that is very similar to the real world regarding visual, haptic and auditory sensations and give users an immersive feeling and experience with the aid of interactive devices. VR based simulators are widely used in teaching laparoscopy and endoscopy procedures. [7] The advantages of VR for surgical education are, rst of all, by designing a scienti c curriculum, it can effectively alleviate the shortage of teaching resources and provide a way to quantitatively assess students' performance. Secondly, students can conduct low-cost and repeatable surgical skills training with no time or space restrictions, which helps optimize their learning curve. As Rizzetto et al. put it, the integration of VR and medical education will reshape learning resources, expand the dimension of surgery education, and enable new teaching methods and personnel training modes.
[8] Long before being used as a validation tool for surgeons' skills, [9] the rst and most widely used scenario of VR has always been surgery education. [8] In China, VR technology has made great strides throughout the years in many areas. To meet the speci c demand for surgical education, future VR surgical research should embed educational theory into their VR platforms to eliminate the existing defects: 1). Lack of speci c manifestations of Skinner's Reinforcement Theory in VR design, [10] like real-time feedback, an intelligent quantitative evaluation and guidance; 2). Lack of collaborative training mode weakens the ontological nature of multi-person collaboration in laparoscopic surgery. The training platform should enable students to work as a team with collaborative guidance from their mentors to accelerate knowledge construction, consistent with Flander's Interaction Analysis System [11]; 3). Lack of a high-delity and high-freedom exploration mode fails to fully use immersive virtual space and Vroom's Expectation Theory. [12] To address the above issues, we constructed the prototype of an intelligent VR laparoscopic collaborative training platform combining VR technology with these educational theories. Then we assessed its effectiveness in surgical education for Chinese postgraduate students who were enrolled in the C-STRD program.

Construction of VR platform
Firstly, we entered the operating room to document the surgical instruments, measure the room dimensions and record laparoscopic cholecystectomy (LC) procedures in the Third Xiangya Hospital of Central South University. Then, our developers and doctors discussed and determined the operating room structure, surgical instruments placement, and the steps of the surgical procedure together.
Secondly, we constructed a 3D model of the virtual operating room, which was a one-to-one replica of reality. 3D Models within the operating room, including surgical equipment and human organs, are produced by 3D max software (Autodesk, Inc, USA). Blender software (Autodesk, Inc, USA) was used to process exemplary models and special effects. Diffuse re ection and normal mapping techniques were used to improve the concavo-convex and realism of 3D models. Then the model was transferred to Unity software (unity technologies, Inc, USA) for model adjustment and rendering. Pictures of representative 3D models were shown in Figure1A-1C.
To allow user to interact with the 3D models in the virtual environment, we equipped the platform with intelligent hardware HTC VIVE kit (HTC Corporation, China), including helmet, laser locator and handles ( Figure 2A). When the user puts on the helmet, the laser locator captures the movement and direction of the head; in the meantime, 3D dynamic image is displayed in the helmet. Besides, the user can interact with virtual models through a pair of handles supported by the SteamVR plugin (Valve Corporation, USA). The whole interaction program was written in C# (Microsoft Corporation, USA). The User Interface (UI) design was developed using UGUI (Unity technologies, Inc, USA), which supports adding images, text and buttons, etc.
We achieved multi-person collaboration in a local network based on a low-level real-time transport layer, with the aid of the Multi-player HLAPI plug-in in Unity software (Unity technologies, Inc, USA). All training modes of the platform support multi-person with different devices operating the same procedure simultaneously, as shown in Figure 2A. The collaborative training follows Flander's Interaction Analysis System [11], including collaborative training of multi-person on the same surgical task or students operate collaboratively with mentors offering guidance, as shown in Figure 2B, where the teacher demonstrates the surgical training operation, and the student learns while performing the same surgical task.
As for training modes, we designed exploration mode and surgery learning mode. In the exploration mode, students could freely practice the LC operation alone or collaboratively with their fellow students or mentors. Based on the completeness and correctness of the pre-de ned LC operation steps, we constructed the scoring system with the GameManager component in Unity software (Unity technologies, Inc, USA) to offer immediate quantitative feedback after training. Notably, a crisis mode will be automatically activated if emergencies like bleeding, bile duct rupture occurs, which was also included in the scoring system. Different from the exploration mode, the surgery learning mode contains stepwise guidance in the UI so that students could practice alone smoothly, as shown in Figure 3A and Figure 3B. Besides, the UI displayed information like instruments names to enhance students' memory.

Validation of training e cacy and users experience
The two-phase validation trials of VR training e cacy were prospectively designed and carried out. After that, students' users experience of the platform was gathered through a questionnaire and analyzed. Thirty-six rst-year postgraduate students who were enrolled in the C-STRD program from the Third Xiangya Hospital with zero experience in LC operation were invited with their written informed consent, and they were randomly assigned to Phase I trial, Phase II trial or just the survey.
In Phase I validation trial of platform training e cacy, these 12 students assigned to Phase I rstly performed LC operation in the exploration mode, with a score given by the platform. Then, they entered the surgery learning mode for one training session to learn LC operation with UI guidance. Finally, they entered the exploration mode to perform LC operation again with the platform recording their score. The scores before and after the training session of surgery learning mode were compared.
In Phase II of the validation trial, the collaborative training e cacy was assessed. The 12 students assigned to Phase II were divided into the collaborative training group or the control group, randomly. The collaborative training group consisted of 6 students and 1 senior surgeon, who coached the students collaboratively in the surgery learning mode ( Figure 3B). Then the students performed LC operation in the exploration mode independently, with score recorded. The 6 students in the control group trained once in the surgery learning mode and then entered the exploration mode to perform LC operation with their scores recorded. The scores of these two groups of students in the exploration mode were compared.
We further investigated the students' users experience of our VR training platform by questionnaire. Firstly, questionnaires were given to students of both Phases regarding the intention to use the platform, skills improvement, usability, degree of enjoyment. Secondly, the 12 students in Phase I validation and 12 who did not train on the platform were selected to survey to verify whether our platform can increase students' interest in surgery learning. Likert ve-point scale method was used in designing all the questionnaires in this study.

Statistical Analysis
All statistical analysis were performed using the SPSS Statistics 26.0 (IBM, Armonk, New York). A homogeneity of variance test was performed using Levene's test. The student's t-test was used for comparison between two groups. The reliability of the questionnaire was assessed according to Cronbach's alpha coe cient.

Platform training enhanced the operation level of students
Levene's test was used before applying the independent samples for Student's t-test. F=2.764 (p=0.111) indicated equal variance between the two sets of scores (before and after the training session of surgery learning mode). Table 1 showed the result of comparing nal and initial Phase I LC operation performances in exploration mode. The mean score was 56.83 on the rst try and 61.17 on the second try (p=0.042), telling a signi cant improvement in performance after receiving the training session in surgery learning mode. The effect size (Cohen's d) value was 0.88, indicating that the training e cacy of our platform was evident. Collaborative training was more effective than individual training Levene's test was used to examine the homogeneity of variance test between the collaborative group and control group before applying the independent samples for Student's t-test. F=0.343 (p = 0.593) indicated that the variance between the two groups was equal. As shown in Table 2, the mean score of the control group was 61.33, and that of the collaborative group was 67.17 (p=0.014), indicating that there was a signi cant difference in the performance between the two groups. The effect size value was 1.71, showing that collaborative training led to a substantial improvement in students' performances. Participants have a positive user experience with the platform The questionnaire contained a total of 4 questions, including the intention to use this platform, skills improvement, usability, degree of enjoyment. The Cronbach's alpha coe cient of the questionnaire was 0.781, indicating that the questionnaire had an acceptable consistency. A total of 24 students' (Phase I and Phase II) questionnaires were obtained. Our statistics on the students' intention to use the platform, skills improvement, the ease of use of the platform and the degree of enjoyment were shown in Figure 4A-4C. The VR platform was evaluated as effective ("strongly agree" and "agree") for improving their practical skills in 9/12 of Phase I students, 6/6 of Phase II collaborative students, 4/6 of Phase II control students. The platform was evaluated as effective ("strongly agree" and "agree") at 'intention to use the platform' in 9/12 of Phase I students, 6/6 of Phase II students in collaborative group, and 5/6 of Phase II students in control group. Half of the Phase I students felt that they enjoyed the platform. Despite that 7/12 Phase I students and 4/6 of Phase II students in control group found the usability of the platform dissatisfactory ("disagree" and "strongly agree") or average ("neutral"), 3/6 of Phase II students in collaborative group praised the usability of this platform. 4/6 Phase II students in collaborative group enjoyed themselves ("strongly agree" and "agree") more than Phase II students in control group (3/6). Overall, the students had a positive user experience on using the platform system, with some negative user experience regarding the platform's usability.

Platform training increased students' interest in surgery learning
As shown in Table 3, there is a signi cant difference between the 12 Phase I students and 12 students who hadn't experienced the platform. Phase I students were more interested in surgery learning than those who hadn't experienced the VR platform.

Discussion
In 1993, China began implementing standardized residency training, and by 2013, the C-STRD program was widely available nationwide [13]. The C-STRD program is a key stage in improving the clinical competence of Chinese residents. According to the classic Halstedian paradigm (see one do one, teach one), [14] young surgeons were trained by experienced surgeons to acquire surgical skills. At present, even in very large residency training institutions, the opportunity of hands-on experience in the operating room is still limited and precious, and the Halstedian paradigm is not perfect due to potential patient's safety issues, training costs and scarce opportunities. [15] Mandatory yet complex techniques such as minimally invasive surgery are known for their steep learning curve, urging the surgical students to train harder and more often. Based on the above, efforts have been made to study surgical simulators to assist the traditional surgical teaching mode in educating the C-STRD students.
In surgical simulation, VR application plays a vital role by increasing simulator delity, immersion stats and user experience.
[16] A signi cant advantage of VR in surgery education is to allow patients to stay safe without being exposed to risks in students' training process. [17] In the current pandemic, VR technologies are necessary for students to gain appropriate expertise without face-face classes, which could bridge the gap between theory and practice.
[18] Other advantages of VR platforms are global connectivity, real-time exchanges, lower cost and better convenience of repeat training. The Chinese education system, including medical education, places great emphasis on incorporating educational theories from China and abroad into the educational practice so that the power of VR can be maximized.
To fully use of immersive VR space, we designed an active training schedule, [19] which we call the exploration mode. Applying Vroom's Expectancy Theory to surgical education, it is crucial student needs to know that each practice counts and will add to their future career success. [12] Indeed, the skills gained through the surgical simulator can be translated into real-world surgical level improvement. [20] In our case, the C-STRD students' performance in the exploration mode did improve after the training. The beauty of the exploration mode lies in the opportunity to fail at any given time without consequences, which develops their independent learning skills.
According to the Skinner's Reinforcement Theory, repetition and feedback are critical to learning skills. [10] Objective quanti cation of the students' performance is a must. [21] Traditionally, the assessment of surgical skills was carried out by observation in the operating room, which was time-consuming and lacked objectivity and reliability. [22] The simulator compensates for this limitation by calculating taskbased metrics so that the possibility of quantitative measurement of student performance increases. [23] De nition of metric was a variable sampled in the execution of the task, which in our case was taskdependent (completion of the surgical procedure and correctness of the operation), similar to the work of N. Enayati et al.. [24] The monitoring of metrics associated to the students' interaction with the platform can offer reliable objective quantitative measurements on the student's learning. It is worth noting that compared to the already mature residency evaluation systems in Europe and the United States, China's residency evaluation system is still evolving [25]. It is believed that the powerful educational tools represented by VR can contribute to the scienti c evaluation system in C-STRD program.
Previous VR simulator platforms have rarely reported on collaborative training. Considering the collaborative nature of laparoscopic surgery, we believe it is imperative to include the concept of multiperson collaboration in the design. Collaborative training design is also inconsistent with Flander's Interaction Analysis System, [11] which offers a way for assessing the process of teacher-student interaction. In our setting, after training with senior surgeons, the students' performances were better than those who trained alone. Mariani et al. suggested that future VR simulator platforms should evaluate the platform's impact on training situations under subjective expert guidance, [14] and we are approaching this goal through collaborative design.
Another merit of collaborative design is for students to work on the same procedure as a team, which is broadly accepted as the most effective method to train Nontechnical skills (NTS).
[26] NTS were de ned as the cognitive (decision-making, situational awareness), social (teamwork, communication, leadership), and personal resources (resilience) that are important in the surgical setting, which is highly valued in the Chinese educational system. [27] In our case, Phase II collaborative group students had better comments than those who trained alone regarding the degree of enjoyment, usability and intention to use the platform. It is possible these positive users' experiences could turn into NTS. Since surgical incidents may more often be induced by de cits of NTS than de cient technical skills,[28] including situational awareness, decision-making and communication, [16] we should address the importance of collaborative training in surgical education.
VR replaces the physical world with an entire virtual world. It has long been used in many aspects of medicine, like pre-operation evaluation of the patient-speci c anatomy in high resolution, [29,30] postoperative pain management, perioperative stress reduction and relaxation, [31] cancer-related symptoms relieving during chemotherapy. [32] Unlike similar technologies Augmented Reality (AR) and Mixed Reality (MR), which have been used in surgical planning and intraoperative guidance, [33,34] VR with its immersive character, is more suited for medical training. In a VR medical education study, all the participants believed the VR course was a valuable learning tool for them to gain a more comprehensive understanding of anatomy and surgery, [35] which is inconsistent with our nding that most students thought the VR platform improved their surgical skills and worth learning.
Currently, VR in general surgery is used in two main areas of surgical teaching: robotics and laparoscopy surgery.
[36] Most studies were carried out using commercial VR platforms like dV-Trainer Mimic (Seattle, WA, United States) and LapSim VR (Surgical Science, Gothenburg, Sweden). They simulate robotic and laparoscopic surgery, respectively. [37] Several RCTs have demonstrated the ability of LapSim to translate the skills learned from the platform to actual clinical procedures.
[38] We developed the VR platform for laparoscopic training because we believe laparoscopic surgery is most suited for VR surgical training. After all, the way of looking at a monitor resembles the actual scene in the operation room where the students look at a monitor while operating the surgical instrument. A similar method of learning is through tablet-simulators, which is also effective in transferring knowledge to real-world tasks. [39] One of the key technologies to achieve VR immersion is visual rendering. The rendering engine can make the creation and deployment of 3D scenes easier and have gained more attention in the build process of surgical simulators, especially Unity. [40] We use the most reliable "Unity" for the visual rendering of 3D models, which already include all the features necessary to create 3D scenes. After rendering, we use a head-mounted display (HMD), the HTC Vive (HTC Corporation China), to allow students to completely immerse themselves in the virtual world. [41] The advantage of using such HMD lies in the degree of immersion that can be realized. The success of our VR laparoscopy training platform is evidenced by students' improved scores after training.
Haptic rendering is another critical element of the next generation of VR surgery simulators. Haptic devices guarantee that the students can operate and interact as if they were in the real world.
[16] The importance of haptic feedback is mentioned in almost all studies. But just like the Da Vinci surgical robot, most VR platforms have not yet developed haptic feedback. Hence, the tele-operative approach compensates for the absence of haptic feedback with "visual feedback". [42] In our case, we implemented the real-time tissue deformation program with C# to achieve such visual feedback.
This study has the following limitations. Firstly, the small sample size from single-center may hamper the statistical signi cance and translational medicine value. To go beyond this pilot study, we need to test the platform with C-STRD students of different levels in the future, as well as study whether the training effects have a direct correlation with clinical outcomes such as reduction of operating time or blood loss. Secondly, the main reason for the students' negative evaluation of the platform's usability was that the HTC VIVE control devices were different from real clinical instruments, which takes time to adapt. Currently, there are no such commercially available haptic feedback devices specially made for surgical education. It is necessary to develop surgery-speci c haptic feedback devices which resemble surgical instruments. Thirdly, the current version as a prototype could not enable remote collaborative surgery. In the future, we plan to achieve this goal by operating on a reliable web server.

Conclusion
Our study integrated VR technology with educational theory related to surgical education to build a collaborative laparoscopic surgery training platform, which to our knowledge is the rst of its kind in China. The platform improves C-STRD students' laparoscopic surgical skills by allowing them to train in different modes, which compensate for the limited education resources and scarce hands-on practice opportunities. With the merits of low cost, repeatability, and patient-safe, the VR platform is well accepted by the training students and improves their interest in surgery learning. The VR platform may shed light on future medical education in China.