Feasibility of Stone Recurrence Risk Stratification Using the Recurrence of Kidney Stone (ROKS) Nomogram

doi:10.21203/rs.3.rs-2566248/v1

Download PDF

Research Article

Feasibility of Stone Recurrence Risk Stratification Using the Recurrence of Kidney Stone (ROKS) Nomogram

https://doi.org/10.21203/rs.3.rs-2566248/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 17 Apr, 2023

Read the published version in Urolithiasis →

You are reading this latest preprint version

Purpose: We sought to assess the performance of the Recurrence of Kidney Stones nomogram for risk stratification of recurrence in a retrospective study.

Materials and Methods: We performed a case-control study of 200 patients (100 with and 100 without subsequent recurrence) who underwent kidney stone surgery between 2013-2015, with at least 5 years of follow-up. We analyzed the performance of the 2018 ROKS nomogram via area under the receiver operating curve (ROC-AUC) for predicting 2- and 5-year stone recurrence. We evaluated the nomogram’s ability to stratify patients based on low or high risk of recurrence at: a) an optimized cutoff threshold (i.e. optimized for both sensitivity and specificity), and b) a sensitive cutoff threshold (i.e. high sensitivity (0.80) and low specificity). Time-to-recurrence data were estimated using the Kaplan Meier method.

Results: The ROKS nomogram demonstrated fair ability to predict recurrence at 2 and 5 years (ROC-AUC of 0.67 and 0.63, respectively). At the optimized cutoff threshold, recurrence rates for the low and high-risk groups were 20 % and 45% at two years, and 50% and 70% at five years, respectively. At the sensitive cutoff threshold, the corresponding recurrence rates for the low and high-risk groups was of 16% and 38% at two years, and 42% and 66% at five years, respectively. Kaplan-Meier analysis revealed a significant recurrence-free advantage between the groups for both cutoff thresholds (p<0.01, Fig. 2).

Conclusions: The ROKS nomogram could serve as a tool for recurrence risk stratification into lower and higher risk groups and facilitate adherence to risk-based follow-up protocols.

kidney stone

kidney stone recurrence

risk stratification

The clinical management of kidney stone disease relies on the individual risk of future stone events. Such risk stratification, based on clinical characteristics, is recommended by clinical guidelines to appropriately tailor the intensity and burden of preventative interventions, often prescribed over the long term¹. However, what constitutes high versus low-risk criteria is not clearly established, and as a result there is poor agreement among clinicians when predicting individual recurrence risk².

Within this context, the development of the recurrence of kidney stone (ROKS) nomogram sought to provide individual-level probability of future symptomatic stone events. The nomogram uses 13 features from the clinical history, including stone characteristics on imaging^3,4. However, the nomogram has limited adoption in clinical practice primarily due to its limited predictive performance, including when evaluated in external cohorts^2,5,6.

Whether ROKS can be used to discriminate high and low risk populations among kidney stone patients, rather than individual-level risk prediction, is unknown. Therefore, our goal was to assess the performance of the ROKS nomogram to risk-stratify kidney stone patients based on future risk of stone events, including both 2 and 5-year recurrence. To do this, we performed an external evaluation of the nomogram, identified the optimal risk thresholds for high-low risk stratification, and report their respective predictive potential.

Patient Cohort

After local institutional review board approval, a retrospective review was performed of all patients who underwent kidney stone surgery from 1997–2015 at our institution. Patients were identified using the an institutionally maintained and deidentified database of the electronic health record⁷. Patients were identified via CPT codes (see Appendix 1) for having received percutaneous nephrolithotomy (PCNL), ureteroscopy (URS), or shockwave lithotripsy (SWL). Patient demographics and comorbidities based on ICD codes were extracted from the Synthetic Derivative (see Appendix 2). We selected a contemporary cohort of 200 patients (100 without recurrence and 100 with recurrence) with at least annual follow-up for 5 years for their stone disease.

Stone Recurrence

Stone recurrence episodes after index surgeries were identified by the first occurrence of 1) kidney stone surgeries (CPT codes, appendix 1), due to stone growth new stone on imaging or asymptomatic hydronephrosis, or 2) emergency room visits for symptomatic kidney stones (ICD codes, appendix 2). Then, manual chart review was performed to adjudicate all recurrence episodes. Patients who had secondary planned surgeries or recurrence events within three months of index surgery were excluded to ensure that recurrence episodes were not immediately attributable to a staged procedure or a complication from the index surgery.

Statistical methods

We identified the performance of each predictor in the 2018 ROKS nomogram for our patient population via Cox proportional hazard regression analysis⁴. Estimates of recurrence at 2- and 5-years follow-up were calculated for each patient. These predictions were used to assess the discriminative performance of the nomogram for our patient population using the area under the receiver operating curve (AUC-ROC).

Then, we evaluated the ability of the ROKS nomogram to stratify patients based on low or high risk of recurrence at two thresholds along the ROC obtained from: a) an optimized cutoff point (i.e. a point optimized for both sensitivity and specificity along the ROC), and b) a sensitive cutoff point (i.e. a point with high sensitivity (0.80) and low specificity along the ROC). The points were chosen in line with current statistical guidelines in urology⁸. This sensitive cutoff point was selected to determine the clinical utility of the ROKS nomogram as a tool for identifying high risk of stone recurrence. Time to recurrence was compared between the risk groups via survival analysis with Kaplan-Meier estimation and log rank testing. All analysis was performed with p < 0.05 as significant and conducted in ‘R’ (R Foundation for Statistical Computing, Vienna, Austria)⁹.

Patients followed up over a mean ± SD time of 96 ± 38 months. The mean ± SD time to recurrence was 29 ± 32 months. Table 1 summarizes patient demographics and clinical characteristics.

Table 1. Patient Characteristics

Demographics	Overall (N = 200)	Recurrence (N = 100)	No recurrence (N = 100)	P value
Age (mean ± SD)	49 ± 15	47+-15	52+-16	0.03
Body mass index(mean ± SD)	30.4 ± 8.0	30+-7	31+-9	0.47
Gender, male (N,%)	101 (50)	45	56	0.15
Race, Caucasian (N,%)	174 (87)	88	86	0.67
Race, African American (N,%)	12 (6)	6	6	0.99
Ethnicity, Hispanic (N,%)	6 (3)	2	4	0.41
Past Medical History
Family history of nephrolithiasis (N,%)	86 (43)	50	36	0.05
Hypertension	78 (39)	36	42	0.38
Diabetes mellitus	38 (19)	17	21	0.47
Bowel disease	38 (19)	25	13	0.03
Gout	7 (3.5)	4	3	0.70
Hyperparathyroidism	2 (1)	1	1	0.99
Cystinuria	2 (1)	1	1	0.99
Index surgery
Ureteroscopy and laser lithotripsy	160(80)	76	84	0.16
Percutaneous Nephrolithotomy	38(19)	23	15	0.15
Shock-wave lithotripsy	2 (1)	1	1	0.99

ROKS nomogram demonstrated fair discriminative ability in predicting risk of recurrence with an AUC-ROCs of 0.67 and 0.63 at 2 and 5 years, respectively (Fig. 1a,b). At 2 years recurrence prediction, sensitivity and specificity were 0.60 and 0.68 at the optimized cutoff point, and 0.80 and 0.45 at the sensitive cutoff point, respectively. At 5 years recurrence risk prediction, sensitivity and specificity were 0.54 and 0.72 at the optimized cutoff point, and 0.80 and 0.26 at the sensitive cutoff point, respectively (Fig. 1c).

For the two-year recurrence prediction, the AUC-ROC values of 0.623 for the optimized cutoff point and 0.505 for the sensitive cutoff point were examined. At the optimized cutoff threshold, 120 patients were classified as low, while 80 were classified as high risk. Recurrence rates for the low and high-risk groups were 20% and 45% at two years, and 50% and 70% at five years, respectively. Then, for the sensitive cutoff threshold, 75 patients were classified as low risk and 125 patients were classified as high risk. The corresponding recurrence rates for the low and high-risk groups was of 16% and 38% at two years, and 42% and 66% at five years, respectively. For both thresholds, KM analysis revealed a significant recurrence-free probability between the groups (p < 0.01, Fig. 2a,b).

Figure 2: KM curves evaluating time to recurrence between low and high risk groups for using a ROC-cutoff point threshold of (a) 0.623 by optimizing sensitivity and specificity, and b) 0.505 for high sensitivity, low specificity. The shaded areas represent the 95% confidence interval of the respective curves

Of the 13 predictors used by the ROKS nomogram, BMI, family history of stone disease, any calcium oxalate monohydrate stone, having a stone in the renal pelvis or lower pole, and having a stone > 6mm in diameter all associated with risk of recurrence in the cohort (Table 2, HRs: 1.03, 1.8, 0.3,0.61, 0.26, respectively).

Table 2

Performance of individual predictors from the ROKS nomogram in the cohort
	No Recurrence (N = 100)	Recurrence (N = 100)	HR (95%CI)	p
Age at last stone episode (years, mean, SD)	52+-15	47+-15	1.01 (0.99–1.02)	0.5
BMI at last stone episode	31+-9	30+-7	1.03 (0.99–1.1)	0.06
Sex (male, N)	56	45	0.83 (0.49–1.44)	0.52
Family History of Stone disease, N	37	49	1.8 (1.07–3.10)	0.03
Any stone to be found uric acid or brushite or struvite, N	20	22	0.38 (0.15–0.97)	0.04
Any stone to be found calcium oxalate monohydrate, N	80	78	0.30 (0.13–0.67)	< 0.01
Ureterovesical junction stone, N	8	28	.0.87 (0.48–1.57)	0.64
Renal pelvis/lower pole stone, N	31	53	0.61 (0.34–1.1)	< 0.01
Prior incidental stone seen on imaging, N	28	41	0.85 (0.49–1.48)	0.57
Prior suspected event without actual stone seen, N	81	85	1.78 (0.84–3.7)	0.13
Number of stones in both kidneys, N
1	36	27	Ref
> 2	64	73	1.2 (0.66–2.04)	0.6
Diameter of largest stone, N
< 3mm	6	4	ref
3-6mm	18	20	0.67(0.21–2.1)	0.51
> 6mm	76	76	0.26 (0.84 − 0.77)	0.02

Our study suggests that the ROKS nomogram modestly predicts recurrent stone events when including a broad definition of recurrence. This further emphasizes the limitations of utilizing the nomogram for counseling kidney stone patients. However, we found that the ROKS nomogram successfully stratified patients by risk-level (i.e., low vs. high) for stone recurrence based on different cutoff points. For example, using a sensitive cutoff point for stratification predicted the risk of kidney stone recurrence for the low and high-risk groups to be 16% and 38% at two years, and 42% and 66% at five years, respectively. This finding could facilitate the development of surveillance protocols for kidney stone recurrence by risk group.

Currently, the American Urologic Association (AUA) recommends postoperative imaging after stone surgery to assess for residual stone and silent hydronephrosis with periodic imaging thereafter to assess for new stone formation or stone growth^1,10. However, there is limited evidence supporting specific protocols for stone surveillance. Unnecessary imaging can be costly and lead to excess radiation exposure^11,12. The lack of data supporting long term imaging use has led to a wide variation follow-up imaging utilization after stone surgery. For example, in an evaluation of MarketScan data, 29% and 15% of patient who underwent percutaneous nephrolithotomy do not undergo postoperative imaging by 3 and 12 months respectively¹³. Comparatively, 55% and 39% of patients who underwent ureteroscopy, and 23% and 16% of patients who underwent shock wave lithotripsy had no postoperative imaging at 3 and 12 months, respectively¹⁴. Both studies, moreover, reported significant variations in postoperative imaging modalities as well, emphasizing a lack of consensus for kidney stone surveillance. Thus, a rational approach for follow-up of kidney stone patients is to tailor the frequency of surveillance imaging to specific patient risk factors.

As recurrence risk varies across patients, it is necessary to personalize stone care to optimize stone surveillance strategies. However, clinicians may not be able to distinguish patients with low or high risk of recurrence, particularly compared to the ROKS nomogram². Additionally, prior attempts at external validation of the ROKS nomogram has been limited. Previously, Iremashvili et al demonstrated only moderate discrimination of the nomogram (0.655 at 2 years and 0.605 at 5 years) for prediction of recurrence risk¹⁵. We found a similar, modest prediction of risk with a broader definition of recurrence (i.e. including both symptomatic and radiographic recurrence). However, despite limitations of the nomogram as an external prediction tool, our findings suggest it could be used for stone risk stratification. By stratifying patients as low or high-risk for recurrence, the nomogram could help guide follow-up management and improve care for kidney stone patients.

There are several limitations to this study. The retrospective design cannot account for confounders and bias due to omitted variables. All patients were identified after being referred to a tertiary medical center and may have a higher baseline risk for stone recurrence. Furthermore, the nomogram performance may differ for other definitions of kidney stone recurrence⁶. Additionally, as this study was done at a single institution, we cannot assess for recurrence events that occurred at other institutions or at home. It is possible further analysis with a larger dataset could refine our models. Despite these limitations, this study demonstrates the feasibility of kidney stone risk stratification using the ROKS nomogram.

We found that the ROKS nomogram has potential to serve as a tool for recurrence risk stratification into lower and higher risk groups, despite only modest prediction of stone events when including a broad definition of stone recurrence, including symptomatic and radiographic recurrence. This could facilitate adherence to risk-based surveillance protocols.

Pearle MS, Goldfarb DS, Assimos DG, et al. Medical management of kidney stones: AUA guideline. J Urol. 2014;192(2):316–324. doi:10.1016/j.juro.2014.05.006
Forbes CM, McCoy AB, Hsi RS. Clinician Versus Nomogram Predicted Estimates of Kidney Stone Recurrence Risk. J Endourol. 2021;35(6):847–852. doi:10.1089/end.2020.0978
Rule AD, Lieske JC, Li X, Melton LJ 3rd, Krambeck AE, Bergstralh EJ. The ROKS nomogram for predicting a second symptomatic stone episode. J Am Soc Nephrol JASN. 2014;25(12):2878–2886. doi:10.1681/ASN.2013091011
Vaughan LE, Enders FT, Lieske JC, et al. Predictors of Symptomatic Kidney Stone Recurrence After the First and Subsequent Episodes. Mayo Clin Proc. 2019;94(2):202–210. doi:10.1016/j.mayocp.2018.09.016
Iremashvili V, Li S, Penniston KL, Best SL, Hedican SP, Nakada SY. External Validation of the Recurrence of Kidney Stone Nomogram in a Surgical Cohort. J Endourol. 2019;33(6):475–479. doi:10.1089/end.2018.0893
D’Costa MR, Pais VM, Rule AD. Leave no stone unturned: defining recurrence in kidney stone formers. Curr Opin Nephrol Hypertens. 2019;28(2):148–153. doi:10.1097/MNH.0000000000000478
Roden DM, Pulley JM, Basford MA, et al. Development of a large-scale de-identified DNA biobank to enable personalized medicine. Clin Pharmacol Ther. 2008;84(3):362–369. doi:10.1038/clpt.2008.89
Assel M, Sjoberg D, Elders A, et al. Guidelines for Reporting of Statistics for Clinical Research in Urology. J Urol. 2019;201(3):595–604. doi:10.1097/JU.0000000000000001
R Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing; 2021. https://www.R-project.org/
Fulgham PF, Assimos DG, Pearle MS, Preminger GM. Clinical effectiveness protocols for imaging in the management of ureteral calculous disease: AUA technology assessment. J Urol. 2013;189(4):1203–1213. doi:10.1016/j.juro.2012.10.031
Rodger F, Roditi G, Aboumarzouk OM. Diagnostic Accuracy of Low and Ultra-Low Dose CT for Identification of Urinary Tract Stones: A Systematic Review. Urol Int. 2018;100(4):375–385. doi:10.1159/000488062
Brisbane W, Bailey MR, Sorensen MD. An overview of kidney stone imaging techniques. Nat Rev Urol. 2016;13(11):654–662. doi:10.1038/nrurol.2016.154
Dai JC, Ahn JS, Holt SK, May PC, Sorensen MD, Harper JD. National Imaging Trends after Percutaneous Nephrolithotomy. J Urol. 2018;200(1):147–153. doi:10.1016/j.juro.2018.01.078
Ahn JS, Holt SK, May PC, Harper JD. National Imaging Trends after Ureteroscopic or Shock Wave Lithotripsy for Nephrolithiasis. J Urol. 2018;199(2):500–507. doi:10.1016/j.juro.2017.09.079
Iremashvili V, Li S, Penniston KL, Best SL, Hedican SP, Nakada SY. Role of Residual Fragments on the Risk of Repeat Surgery after Flexible Ureteroscopy and Laser Lithotripsy: Single Center Study. J Urol. 2019;201(2):358–363. doi:10.1016/j.juro.2018.09.053

No competing interests reported.

Download PDF

Journal Publication

published 17 Apr, 2023

Read the published version in Urolithiasis →

Editorial decision: Major revision
02 Mar, 2023
Reviews received at journal
25 Feb, 2023
Reviewers agreed at journal
20 Feb, 2023
Reviewers invited by journal
14 Feb, 2023
Editor assigned by journal
13 Feb, 2023
Submission checks completed at journal
13 Feb, 2023
First submitted to journal
08 Feb, 2023

You are reading this latest preprint version

Feasibility of Stone Recurrence Risk Stratification Using the Recurrence of Kidney Stone (ROKS) Nomogram

Status:

Journal Publication

Version 1

Abstract

Figures

Introduction

Materials And Methods

Patient Cohort

Stone Recurrence

Statistical methods

Results

Discussion

Conclusion

References

Additional Declarations

Status:

Journal Publication

Version 1