Implementation of a structured robotic colorectal curriculum for general surgery residents

There is increasing demand for colorectal robotic training for general surgery residents. We implemented a robotic colorectal surgery curriculum expecting that it would increase resident exposure to the robotic platform and would increase the number of graduating general surgery residents obtaining a robotic equivalency certificate. The aim of this study is to describe the components of the curriculum and characterize the immediate impact of the implementation or residents. Our curriculum started in 2019 and consists of didactics, simulation, and clinical performance. Objectives are specified for both junior residents (post-graduate years [PGY]1-2) and senior residents (PGY3-5). The robotic colorectal surgical experience was characterized by comparing robotic to non-robotic operations, differences in robotic operations across post-graduate year, and percentage of graduates achieving an equivalency certificate. Robotic operations are tracked using case log annotation. From 2017 to 2021, 25 residents logged 681 major operations on the colorectal service (PGY1 mean = 7.6 ± 4.6, PGY4 mean = 29.7 ± 14.4, PGY5 mean = 29.8 ± 14.8). Robotic colorectal operations made up 24% of PGY1 (49% laparoscopic, 27% open), 35% of PGY4 (35% laparoscopic, 29% open), and 41% of PGY5 (44% laparoscopic, 15% open) major colorectal operations. Robotic bedside experience is primarily during PGY1 (PGY1 mean 2.0 ± 2.0 bedside operations vs 1.4 ± 1.6 and 0.2 ± 0.4 for PGY4 and 5, respectively). Most PGY4 and 5 robotic experience is on the console (PGY4 mean 9.1 ± 7.7 console operations, PGY5 mean 12.0 ± 4.8 console operations). Rates of robotic certification for graduating chief residents increased from 0% for E-2013 to 100% for E-2018. Our robotic colorectal curriculum for general surgery residents has facilitated earlier and increased robotic exposure for residents and increased robotic certification for our graduates.


Introduction
The use of robotic technology in colorectal surgery has increased dramatically over the past decade [1][2][3][4]. This trend is consistent among high-and low-volume colorectal centers with a robotic platform, especially for operations in the pelvis. In response to this shift in surgical approach, demand for colorectal robotic training for general surgery residents has increased. A recent survey of general surgery residency program directors showed that 78.6% of programs believe that training at the robotic console should be a requirement for general surgery training [5]. This sentiment has been echoed by general surgery residents who believe that earlier exposure to robotic surgery is a necessary component of training [6]. To meet this demand, many general surgery residencies have started to train their residents on the robotic platform, but many of these programs lack a formal robotics curriculum to guide this exposure [7]. The ideal structure or implementation of such a curriculum is undefined, including how to best to help residents gain robotic console experience, not just observation of robotic operations [8][9][10][11][12].
To address this gap in general surgery training, we describe our robotic surgery curriculum for general surgery residents, with a focus on the residents' colorectal rotation. We hypothesized that implementing this curriculum would increase resident exposure to the robotic platform-both at the bedside and console-and would increase the number of graduating general surgery residents obtaining an Intuitive Surgical® Equivalency Certificate in robotic surgery.

Robotic curriculum structure
In 2019, we introduced our robotic general surgery resident curriculum, which combines didactics, virtual simulations, and clinical performance across four surgical services utilizing the robotic platform: bariatric, colorectal, general, and thoracic surgery. In this study, we focus on the colorectal portion of this curriculum, which is available in Appendix 1. Our curriculum is divided into pre-clinical (Phase 0), junior resident (Phase 1), senior resident (Phase 2), and post-residency training (Phase 3) phases, each with specific focuses on didactics, simulation, and clinical practice (Fig. 1). Our hospital uses the DaVinci Xi © robotic surgical platform and our curriculum described here focuses solely on this platform.

Pre-clinical didactics
The first phase of our curriculum (Phase 0) encompasses the pre-clinical portion of robotics training, extending prior to and through the first week of residency (intern orientation). Incoming residents complete online didactic modules provided by Intuitive Surgical® focusing on the vocabulary, components, and processes used in robotic surgery and complete built-in assessment tools to reinforce this knowledge (https:// learn ing. intui tive. com, [Accessed 2 February 2023]). During intern orientation, didactics continue with hands-on in-service training with Intuitive Surgical® representatives using the DaVinci Xi © robotic platform. This training teaches incoming residents how to dock the robotic platform, orient the platform limbs for different operative areas, and insert and exchange instruments. Each aspect of this in-service training emphasizes safety and includes instructions for troubleshooting various scenarios when working with the robotic platform. Phase 0 concludes with an overview of the SimNow® robotic console simulator. This phase of the curriculum is intended to provide residents with the foundational knowledge needed to begin bedside assisting and training on the robotic console simulator prior to any clinical rotations.

Simulation
Upon completion of pre-clinical didactics, residents transition to "Phase 1" of our curriculum. This phase focuses on progressively developing technical skills through simulation using the SimNow® platform in preparation for operating and improving efficiency on the console. Of the 70 modules included on the SimNow® platform, 16 are mapped specifically to the colorectal rotation (Supplemental Table S1). The colorectal simulation objectives are focused around preparing for a robotic segmental colectomy, as residents most frequently participate in sigmoid and right colectomies. Accordingly, simulation exercises focus on the components of the case the resident might perform during those cases. For instance, early simulation goals for junior residents (loosely correlating with post-graduate years [PGY]1-2) focus on developing motor memory for rudimentary instrument manipulation (e.g., EndoWrist ® manipulation, and camera driving) by completing the Clutch, Competitive Needle PoseMatch 90, Energy Pedals 2, Wrist Articulation 2, Running Suture, and Right Hemicolectomy modules with a score of 90% or greater. Senior residents (PGY3-5) progress through the curriculum during "Phase 2" by completing simulations that focus on improving proficiency in the use of the third arm, suturing, and using energy devices. This is accomplished by completing the simulation modules 30-degree Scope Swap, ATW Needle Driving, 3-Arm Relay, Vessel Energy Dissection, and Common Enterotomy Closure with a score of 90% or greater. The choice of these simulation exercises is set by the attending surgeons of the service.

Clinical experience
In addition to three fellowship-trained colorectal surgeons, our colorectal service is comprised of a PGY 1, PGY 4, and PGY 5 resident with physician assistant (PA) support for bedside assisting during most robotic operations. This service is intermittently staffed by a PGY 2 or PGY 3 resident as needed.
While developing rudimentary technical skills via simulation in Phase 1, PGY 1 residents begin their robotic clinical experience by bedside assisting. As part of this experience, junior residents learn port placement, troubleshooting of the robotic arms, and insertion and removal of instruments, all in an effort to develop the surgical assistant skillset. This early exposure also allows junior residents to progress toward their bedside assisting requirement of 10 operations (across all rotations) for their robotic equivalency certificate. After gaining bedside experience, residents progress to "Phase 2" of our curriculum and begin operating at the console using a dual console platform. The dual consoles allow the resident the autonomy needed to fully operate using the robotic platform, while also allowing the attending surgeon to take immediate control of the robot if needed. Our junior residents usually begin their console experience by performing small portions of operations-such as suturing-at the discretion of the attending surgeon. Of note, we set the expectation that all residents complete the simulation modules assigned with a score of at least 90% before participating in clinical experience with the robotic console. Tracking of simulation performance is through the SimNow platform.
Senior resident console experience begins with mobilization of the colonic attachments and basic suturing. Operative experience then progresses with performing more complex portions of a robotic segmental colectomy, culminating in completing full mobilization of the colon, most of the mesenteric division, part of the total mesenteric excision, and intracorporeal anastomosis, if indicated and appropriate. As residents demonstrate competency in these operations, they are allowed further operative autonomy on various robotic colorectal operations at the attending surgeon's discretion. Throughout their operative experience, residents of all levels are expected to continue refining their technical skills by revisiting relevant simulation modules during their colorectal rotation.
The proportion of each robotic operation that a given attending allows their residents perform on the console is dependent upon the given attending, where the attending is on their robotic learning curve, the difficulty of the specific case, the previous experience of the resident, and whether a PA is present to bedside assist. Feedback on operative performance is provided to residents in a variety of forms. Residents receive informal verbal feedback from the attending surgeon after the operation is complete, as well as feedback via review of video recordings of the operation with the attending surgeon, the SIMPL evaluation app (simpl.org/ simpl, [accessed 9 February 2023]) [13], the Global Evaluative Assessment of Robotic Skills (GEARS) feedback form [14], or a combination of these methods.

Equivalency certificate
There is currently no robotic requirement for general surgery residents put forth by the American College of Graduate Medical Education (ACGME) or the American Board of Surgery (ABS) [15]. In expectation of such a requirement being implemented in the near future, there is ongoing development of a standardized robotic curriculum and evaluation in the form of the Fundamentals of Robotic Surgery [16]. Due to this lack of formal robotics educational requirements, our program requires all graduating residents to obtain an Intuitive Surgical da Vinci ® robotic equivalency certificate (https:// www. intui tive. com/ en-us/-/ media/ ISI/ Intui tive/ Pdf/ resid ent-train ing-certi ficate-letter-210352. pdf). Achievement of this certificate includes online trainings, case-logging, and verification of robotic experience by a residents' program director. Though attainment of this certification is becoming more common, this Equivalency Certificate has yet to become a nationally accepted standard for general surgery residents [17].
To acquire their robotic equivalency certificate, graduating residents submit their robotic case-logs demonstrating at least 10 robotic bedside operations and 20 robotic operations in which they completed at least 50% of the operation on the robotic console (this 50% robotic console participation requirement is for the robotic equivalency certificate, not for ACGME case-logging). This case log is combined with a copy of the resident's online assessment completion certificates and a letter from our chief of robotic surgery certifying that the resident has completed the requirements for an equivalency certificate.

Cohort creation and statistical analysis
The Institutional Review Board of Virginia Mason Medical Center exempted this study from IRB review.
Robotic operations on the colorectal rotation with resident involvement from 2017 to 2022 were tracked using ACGME case logs (Supplemental Table S2), with annotation by residents noting whether a given laparoscopic operation was robotic, and if they were at the bedside or console for that operation. Based on these case logs, we created a longitudinal description of robotic colorectal experience for each resident. If a laparoscopic operation was missing a "robotic" descriptor, it was included as a laparoscopic operation. If an operation was logged as a robotic operation but was missing descriptors for "bedside" or "console", it was excluded from further analysis, except if it was logged during the PGY 1 year as those were assumed to be bedside cases. Summaries of robotic experience are stratified by year, PGY, and resident cohort by entry year (i.e., E-2017). To quantify the impact of increasing robotics on other surgical approaches, we also report total major colorectal case numbers for residents, divided into open, laparoscopic, and robotic approaches. Equivalency certificate obtainment was tracked by surveying graduating residents before their graduation.
Summary statistics are reported as mean ± standard deviations for continuous data, and proportions (%) for categorical data. Comparisons across groups (PGY and graduation cohorts) were analyzed using a one-way analysis of variance (ANOVA) test. All analytical procedures were conducted using Microsoft Excel 365 (Microsoft Corporation, Seattle, WA), with statistical significance considered at a two-sided alpha value of 0.05.

Discussion and conclusion
Robotic colorectal surgery is becoming more common in the United States and there is a corresponding need for formalized robotic training of general surgery residents to prepare them for this evolving surgical landscape. As such, we implemented a structured robotic colorectal surgery curriculum in 2019 that combines online and in-person didactics, simulation, and clinical practice. Since this curriculum has been implemented, robotic experience for our residents has increased at the bedside and console. Our data suggest that senior residents have had substantial console experience since the implementation of our curriculum, without compromising exposure to either open or laparoscopic colorectal operations. This finding is consistent with previous literature showing that adding a robotic curriculum improves general surgery residents' surgical education without diminishing other aspects of their training [18]. The robotic colorectal experience contributes to the overall robotic training of our general surgery residents, such that all residents are currently graduating with a robotic equivalency certificate. While this curriculum has laid an important foundation for the residents' colorectal robotic training, there are several remaining challenges ahead. First, standardizing case-logging for colorectal robotic cases for residents is in its infancy. We chose to use ACGME case logs to reflect the resident experience, which, until recently, did not specifically distinguish robotic operations from laparoscopy. We instruct residents to use case log annotation to report an operation as robotic and whether they performed the bedside or console role. However, we found variability in how residents chose to annotate their case logs, leading to missing and misclassified robotic operations. To help minimize these inconsistencies, the ACGME added a "robotic" field to their case logging in 2022; however, this does not solve the problem of delineating robotic operations as either "bedside" or "console". Similarly, our data suffer from a lack of granularity regarding what portions of the operation residents are performing. Specifically, our residents are not currently recording if they performed more than 50% of a given operation on the console, which is a requirement for an Intuitive Surgical ® Robotic Equivalency Certificate but not for ACGME case logging. Second, case logging and bedside/console experience are imperfect measures of resident competency in a particular operation. This has been a long-standing criticism of case logs or case minimums [19], but is the norm for measuring resident experience. Our curriculum tries to address this issue in three ways. First, we note to residents there is a "Phase 3" (Fig. 1) in their robotic training. This is the growth, experience, and autonomy that develops in fellowship and practice beyond the foundational skill acquired during their residency training. Second, there are 'key steps' for the robotic colorectal operations (e.g., mobilization of colon, mesenteric division, or intracorporeal anastomosis) that residents can complete and personally track as discrete measures of their progress. Third, we offer flexibility for each attending surgeon to choose their preferred modality for feedback. Resident simulation practice scores are available through SimNow ® , which residents can review with attendings (or attendings can check prior to resident involvement in a case). We also offer a variety of postoperative feedback modalities, including review of video recordings of an operation/steps with the attending surgeon, the SIMPL evaluation application [13], the GEARS feedback form [14], or a combination of these methods. Not all operations or steps lend themselves to each of these feedback methods, and offering this flexibility may improve the quality of feedback without placing undue administrative burden on attending surgeons. Anecdotally, junior residents seem to prefer the SIMPL feedback, while senior residents prefer the video review. To investigate the role of each modality and quality of feedback, we plan to review which feedback modalities are most commonly used and preferred by residents as our experience progresses. Similarly, we acknowledge that "equivalency certification" based on robotic case number does not necessarily impart competency on the robotic console. In addition, the use of the Intuitive Robotic Equivalency Certificate as a sign of competency for our curriculum has also come into question. This is due to the lack of data surrounding the establishment of the bedside and console case numbers needed to achieve the certification, along with a lack of competency evaluation as part of this certification. There is a current movement to establish a "Fundamental Skills of Robotic Surgery (FSRS)" curriculum and competency testing similar to the current Fundamentals in Laparoscopic Surgery (FLS) curriculum and testing that is required of general surgery residents to complete before their graduation, but this has yet to be implemented as a graduation requirement by the ABS [20]. In the absence of another, widely accepted benchmark for robotic surgery in residency, the importance of the Intuitive Robotic Equivalency Certificate cannot be understated. Several of our graduating residents have used both their robotic experience and this certification to help attain both fellowship placement in specialties utilizing the robotic platform and employment/credentialling in practices where a robotic skillset is an essential component of their careers.
There are several important limitations that need to be considered. First, as the study is retrospective, it is unclear whether the increasing resident experience with robotic colorectal operations is truly due to the structured curriculum or what portion would have occurred without it. Another possible confounder is our institution's change in practice over time. In 2020, we experienced a 17% decrease in robotic colorectal operations due to the pandemic. Despite the variable use of robotics during the years of the study, one vital role of this curriculum was safety. We believe that resident familiarity with the robotic platform before ever participating in an operation creates a safer environment for residents to learn the controls of the robotic arms, both at bedside and at the console. Similarly, requiring residents to meet specific learning and technical objectives prior to using the console and performing more difficult portions of robotic operations helps create motor memory and safe habits (i.e., keeping instruments in view and avoiding collisions, which both are flagged as part of simulation exercises). Furthermore, while the simulation exercises we selected for the colorectal rotation focus on operative steps residents perform during segmental colectomy, they are not exclusive to colorectal operations, and are reinforced by repeated practice across other rotations.
A second limitation is this study's focus on only the colorectal robotic experience, such that the case numbers are driven by the PGY 1, 4, and 5 residents. In addition, because residency at our program spans 5 clinical years, only our E-2017 cohort had a full 5 years of case-logs captured during the study period. This limits our comparisons of the progression of robotics experience between cohorts, as each cohort has not experienced the full breadth of our robotic curriculum. We therefore avoided between-group comparisons. Regardless, we can still characterize individual PGY years. We plan to continue our examination of resident robotic colorectal experience, with a specific focus on feedback and actual time spent on the console, which is available through the MyIntuitive ® App via logging into the robotic console for each operation.
A third limitation is the possible confounding of resident robotic experience by PA bedside assisting, which occurred inconsistently throughout our study period. This could have reduced the number of bedside assisting opportunities for junior residents while potentially increasing console opportunities for senior residents. Finally, an important rationale behind this curriculum was designed was patient safety, as objective and simulation-based curricula have been previously described to improve outcomes [21]. However, this initial descriptive study did not link specific patient safety outcomes to the pre-and post-curriculum period, in part because of the very low rate of safety events, but also because retrospectively it is difficult to tell whether an event was due to resident involvement or not.
Despite these limitations, the implementation of our structured robotic curriculum parallels an increase in the amount of early robotic colorectal exposure for junior residents and console experience for our senior residents. Implementation of this robotic curriculum has been dependent upon structured progression with consistent resident feedback, though standardizing the modality of this feedback is still in progress. In addition, standardizing resident case logging of robotic operations is paramount to ensure accurate and consistent data collection. With this curriculum in place, we expect our residents' robotic experience to continue to grow as we continue to develop the best methods for feedback and competency assessment. As more cohorts complete the curriculum and faculty becomes more knowledgeable and comfortable with it, we expect our general surgery residents to be better prepared for an increasingly robotic colorectal landscape.

Author contributions Kenley Unruh and Vlad
Simianu constructed the study, analyzed data, wrote the main manuscript text, and prepared all tables and figures. All other authors reviewed the manuscript and contributed to revising and editing.
Funding This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.