A retrospective study of using a three-dimensional printed kidney model to evaluate R.E.N.A.L. nephrometry score: an educational tool to improve the diagnostic accuracy of urology residents

The R.E.N.A.L. nephrometry scoring system is used to evaluate the complexity of renal tumors; however, inconsistency among evaluators, especially junior physicians, is an issue. The objective of the study was to evaluate whether it is useful for junior physicians to use a three-dimensional (3D) kidney model when evaluating the R.E.N.A.L. nephrometry score. and retrospectively evaluated the R.E.N.A.L. nephrometry scores of 64 renal tumors (62 which underwent partial nephrectomy at The expert evaluated 64 R.E.N.A.L. nephrometry scores with tomography (CT) scan while four residents evaluated 32 cases using only CT and the other 32 cases using CT and kidney model. Consistency between the and residents was assessed by Cohen’s kappa Patient-specic 3D kidney models were created in a gird style using a 3D based on the CT or magnetic resonance imaging of


Results
For all four residents, the accuracy of the overall R.E.N.A.L. nephrometry score was signi cantly higher by 3D model and CT than CT only (p < 0.001). Regarding individual components of the R.E.N.A.L. nephrometry score, the accuracy of "E," "N," "A," and "L" scores was higher by 3D model and CT than by CT only (p = 0.020-0.089).

Conclusion
Patient-speci c 3-D printed kidney model would improve the resident's understanding of renal tumor complexity and could be an important educational tool for residents.

Background
The R.E.N.A.L. nephrometry score was developed in 2009 as an index of renal tumor complexity [1]. Several studies have shown that the R.E.N.A.L. nephrometry score re ects the degree of surgical di culty of partial nephrectomy and perioperative/postoperative complications [2,3]. Thus, measuring the R.E.N.A.L. nephrometry score before a partial nephrectomy is important for urologists.
However, the R.E.N.A.L. nephrometry scores vary between individuals performing the assessment. The consistency rate of the R.E.N.A.L. nephrometry score was reported to be low with the urologists assessment compared with the radiologists assessment or collaborative assessment by the tumor board including urologists, medical oncologists, radiation oncologists, pathologists, and radiologists [4].
Owing to recent advances in three-dimensional (3D) printing technologies that allow easy preparation at low costs, models have been widely used to improve the level of understanding of anatomy, pre-operative planning, and education [5][6][7]. However, as far as we know, using 3D kidney models to determine the R.E.N.A.L. nephrometry score by urologists has not been reported previously. Therefore, urology residents from our institution determined the R.E.N.A.L. nephrometry score to assess whether the use of 3D kidney models led to more accurate scores.

Survey design
Using a 3D kidney model, we retrospectively investigated the R.E.N.A.L. nephrometry scores of patients who underwent robot-assisted partial nephrectomy (RAPN) at our institution between March 2016 and September 2019. This study received the approval of the Institutional Review Board from our institution with an opt-out system (A-18-210).
The study included an analysis of 64 tumors from 62 patients. Tumors were divided into groups A and B in a time series (n = 32 each). In other words, the rst 32 tumors were included in group A and the remaining in group B. The R.E.N.A.L. nephrometry score was determined by one expert and four urology residents. The expert had 25 years of experience in urology and had conducted approximately 60 RAPN procedures while being part of the Department of Urology. Among the four residents, two had 1 year of experience in urology, one had 2 years of experience in urology, and one had 3 years of experience in urology. Four residents had previously scored 5-15 cases of renal tumors using the R.E.N.A.L nephromtery score prior to the study.
The expert determined the R.E.N.A.L. nephrometry score by using contrast-enhanced computed tomography (CT) images in all 64 cases. The four residents were divided into two groups. First, residents 1 and 2 assessed the cases in group A by using only contrast-enhanced CT images. Second, they assessed the cases in group B by using both CT and 3D kidney models. On the other hand, residents 3 and 4 assessed the cases in group A by rstly using contrast-enhanced CT and 3D kidney models.
Second, they assessed the cases in group B by using only contrast-enhanced CT. Groups were formed to ensure the grades were not biased.
Contrast-enhanced CT was performed at a slice thickness of 0.75-1.0 mm. Axial/coronal/sagittal images were assessed. For the preparation of the 3D kidney models, we used a contrast-enhanced CT in 63 cases and plain magnetic resonance imaging (MRI) in one case. Residents received orientation using a study by Kutikov et al. [1], and they evaluated 64 tumors. They were allowed to use all CT views as the expert surgeon.

Development Of The 3D Model
We used a UP plus2 (Beijing Tiertime Technology Co., Ltd., Beijing, China) 3D printer, which is a Fused Deposition Modeling type of 3D printer that utilizes acrylonitrile butadiene styrene resin to make the 3D kidney models as described in previous studies [8,9]. The Standard Triangulated Language (STL) data from contrast-enhanced CT scans or plain MRI were prepared by a radiology technologist. The 3D models were made from STL data with the 3D printer. The 3D models were life-size kidneys based on measurements taken from the patient using the square-block type model (Fig. 1).

Statistical Analyses
Each component (R.E.N.A.L.) and the overall nephrometry score were evaluated by the expert and the residents. The consistency rate between the assessments of the expert and the residents was assessed using Cohen's kappa score. Pearson's chi-square test was used to evaluate case variability in groups A and B.
Differences in the rate of consistency of R.E.N.A.L. nephrometry score between those evaluated using only CT and those evaluated using a 3D model and CT were assessed using a paired t-test. All statistical analyses were conducted using JMP v.14.0 (SAS Institute Inc., Cary, NC, USA).

Results
Among the 62 analyzed patients (64 tumors), the mean (standard deviation [SD]) age was 59 (12.5) years. Of the 62 patients analyzed, 37 were men and 25 were women. In 33 cases, renal tumors were located in the right kidney and 31 cases were located in the left kidney. The mean tumor size was 24 (7.7) mm. RAPN was performed in 49 cases. Histopathological evaluation showed that the tumor was clear cell carcinoma in 49 cases, papillary cell carcinoma in three cases, AML in seven cases, and other pathology types in ve cases.
The R.E.N.A.L. nephrometry scores for groups A and B evaluated by the expert are shown in Table 1. In groups A and B, there were no differences in score variation in each category.  Regarding individual "E," "N," "A," and "L" categories, a higher kappa score was observed in the 3D model series (p-value =, 0.02 to 0.08; Table 3). The kappa score in the "R" category was not available because almost all cases were cT1a tumors.

Discussion
To the best of our knowledge, no studies have investigated whether 3D kidney models allow for a more accurate assessment of the R.E.N.A.L. nephrometry score among urology residents. Our study had two main ndings. First, the rate of overall R.E.N.A.L. nephrometry score consistency between residents and the expert improved with the use of the 3D kidney models. Second, the "E" score signi cantly improved in each category.
This study showed that using the 3D kidney models the R.E.N.A.L. nephrometry scores determined by residents were similar to those determined by the expert. Although the rate of R.E.N.A.L. nephrometry score consistency was reportedly low in physicians with lesser clinical experience [10], we observed that the use of the 3D kidney models resulted in an increase in the rate of consistency among residents. The effectiveness of 3D model for residents has been reported in various departments [11,12]; however, this study found that it is also useful for renal cell carcinoma.
A previous study reported that when students used 3D kidney models, the rate of R.E.N.A.L. nephrometry score consistency between students and physicians improved [13]. Those cases were rst assessed by CT, followed by a re-assessment using 3D kidney models; thus, assessing the same cases twice was a drawback. Our study differed from the aforementioned study because assessments of different cases were separately performed using CT only and CT and 3D models. As frequent assessments improve the accuracy of the R.E.N.A.L. nephrometry score for residents, we divided the cases into one group where the R.E.N.A.L. nephrometry score was rst determined by CT only followed by CT and 3D kidney models, and another group where the R.E.N.A.L. nephrometry score was rst determined by CT and 3D kidney models followed by CT only. This resulted in a study design in which a learning curve was taken into account; thus, we assumed that this would be helpful in reducing bias.
Based on the study results, we concluded that in professionals with few years of experience, the level of understanding of anatomy and ability to read CT images could be lower as compared with senior physicians; thus, 3D kidney models can be useful in improving the level of understanding of anatomy and can play important roles in education for residents.
Furthermore, our study showed that in the R.E.N.A.L. nephrometry score determined by residents, only the "E" score of each item signi cantly improved by using 3D kidney models. A higher rate of consistency in the C index than the R.E.N.A.L. nephrometry score and PADUA score between individuals was reported previously [10]. Measuring only the direct distance is easier and may reduce errors. In contrast, "E" requires space perception; thus, a 3D model, which is easy to understand in terms of spatial recognition, would be useful, particularly in less-experienced doctors.
One of the reasons may be variation in the cases. In a study evaluating the rate of consistency between radiologists, though the rate of consistency was high (0.88), more than 50% of the cases were of high complexity [14].
In a study showing low rates of consistency, there were a few tumor cases with high complexity, and most cases had low or moderate complexity [4]. In our cases, most tumors were cT1a (95%) and only a few cases (6%). had high complexity. Accordingly, the di culty in achieving consistency would vary depending on the case. The 3D kidney models were considered useful, at least for junior physicians, to determine the R.E.N.A.L. nephrometry score, particularly in cases in which the evaluation was di cult. Whether a similar result would be produced with 3D virtual reality images rather than 3D models is a question that remains unanswered. Because 3D models were created using STL data of 3DCT, a 3DVR could be similarly effective as the 3D model. An assessment of the R.E.N.A.L. nephrometry score conducted using 3DVR was shown to predict postoperative complications e ciently [18]; thus, assessment using 3DVR and 3D models are likely to become relevant in the near future.
In our study, 3D models were considered to improve junior resident's anatomy understanding and improve their diagnostic ability. 3D kidney models have been reported to be effective for student education and patient education [13,19]; however, it was also found to be effective for educating junior physicians. Due to the widespread use of robotic surgery, junior physicians have performed robot-assisted radical prostatectomy [20]. RAPN is also likely to be performed by junior physicians in the near future, and it is important to improve the ability to read CT images.
This study had some limitations. First, as most cases were cT1a high-risk cases, larger tumors could not be assessed. Further research is needed to verify in which cases the model is useful by increasing the number of renal tumors with high complexity.
Second, although it was better that there was CT and 3D model of the consistency rate of all the residents, because the evaluation of the expert was not validated by other experts, it is unknown whether the evaluation of the expert was accurate. The possibility remains that the expert could have made an incorrect assessment.

Conclusions
The 3D kidney models could be useful in educating residents. Whether the use of 3D models by experts will have a positive effect and will affect outcome of RAPN is a subject of future studies. Ethics approval and consent to participate:

Abbreviations
The study protocol was reviewed and approved by the Institutional Review Board at Jichi Medical University in accordance with Helsinki Declaration and Act on the Protection of Personal Information in Japan (A-18-210) and obtaining additional consent from patients was not required by the ethics committee for this retrospective study with an opt-out system.

Consent for publication:
Not applicable Availability of data and materials: The datasets used and analyzed during the current study are available from the corresponding author on reasonable request.
Competing interests: 20. Davis JW, Kamat A, Munsell M, Pettaway C, Pisters L, Matin S. Initial experience of teaching robotassisted radical prostatectomy to surgeons-in-training: can training be evaluated and standardized? BJU Int. 2010;105:1148-54. Figure 1 Sample of our three-dimensional model (right renal tumor). Green, brown, red, blue, and white areas indicate the renal tumor, kidney, artery, vein, and correcting system, respectively.