To predict each patient’s probability of obtaining a colonoscopy, we developed a risk prediction model using data from patients with an abnormal fecal test at the 26 STOP CRC clinics. We followed guidelines set forth by the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnoses statement (18, 19). This model was designed to be put into practice at community clinics using data available in the EHR. Our objective was to predict patients who may benefit from interventions to complete the recommended follow-up.
Setting and Participants
This retrospective analysis used data from the STOP CRC project and included eligible patients who have returned a FIT with an abnormal result during the study period. OCHIN, formerly the Oregon Community Health Information Network, is a nonprofit health care controlled network that provides a centrally hosted EHR for primary care clinics. The STOP CRC project included 26 clinics in Oregon and California that served as the setting for this cohort. This project was approved by the Kaiser Permanente Northwest IRB (Protocol #4364). Clinics operated in diverse settings were diverse in size and were part of 8 health centers.
To be eligible in the STOP CRC study, patients had to have been 50-75 years old and not up to date with CRC screening including fecal testing in the past 11 months or colonoscopy in the past 9 years. Patients were excluded from STOP CRC if they had co-morbid conditions that would make screening inappropriate, such as a history of CRC, colectomy, or dialysis. Our complete inclusion criteria are described elsewhere (20). We then assembled a complete retrospective cohort of STOP CRC patients who subsequently completed CRC screening by FIT test and obtained an “abnormal” result. All patients with at least one abnormal FIT result from February 4, 2014, through February 28, 2016, were identified (n=1723). For patients with more than one abnormal test result, the date of the most recent result was time zero, the start of follow-up.
Outcome and duration of follow-up
The outcome measure of interest was whether a patient received a colonoscopy within 6 months of receiving their abnormal FIT test result. For the Cox model, the outcome was determined if a colonoscopy was completed within 180 days following the abnormal FIT test. Patients were not censored for loss to follow-up, as community clinics do not track membership. Completed colonoscopies were determined through procedure codes in the EHR.
We selected variables for our risk prediction model based on previous studies that identified predictors of failure to complete CRC screening or colonoscopy, but limited variables to those that would be available in the EHR in these community clinics (Table 1). Predictor characteristics were measured during the year before time zero unless otherwise specified. Predictors included clinic systems, patient demographics, community level characteristics, self-reported behavior (e.g., smoking history), clinical findings (e.g., body mass index, and the number of missed appointments), medications (e.g., antihypertensive medications), and diagnoses (e.g., history of cardiovascular disease). All coding and measurement of variables are described in the Appendix. Community data variables were collected at the Census tract level for all variables except for emergency department (ED) visits per 1,000 enrollees; this was collected at the county level. Community-level variables were obtained from the ADVANCE Clinical Data Research Network, which is a data-source integrated into the OCHIN data (21).
We evaluated the characteristics predicting follow-up colonoscopy using a Cox proportional hazards model and a logistic model in SAS® System Software. We fit a full model of patients with complete data and used a step-down process to manually remove the weakest characteristics one covariate at a time to simplify the model so that the final model retained at least 90% of the variation explained of the full model.
For the final model, we calculated the mean observed risk of completing the colonoscopy and plotted mean observed and predicted risks in quintiles using risk predictiveness curves that showed the distribution of predicted risks of completing the colonoscopy (22). Discrimination was measured by a bootstrap corrected C-statistic. Variation explained was measured with an R2 statistic. The Cox regression coefficients were then translated into a simplified point-based risk scoring system to improve use in the clinical setting. A higher number of points mean a higher likelihood of completing a follow-up colonoscopy (23). This allows clinicians to translate the model into practice without calculating the regression equation exactly. Table 3 shows the expected and observed probability of completing a colonoscopy within 6 months of an abnormal FIT test by points. The points assignment reflects the variations in hazard ratios across patient characteristics. The clinician could add up the points to determine likelihood of completing the follow-up colonoscopy.