Prognostic models for short-term annual risk of severe acute complications and mortality in patients living with type 2 diabetes using a national medical claim database

doi:10.21203/rs.3.rs-1604014/v1

Download PDF

Research Article

Prognostic models for short-term annual risk of severe acute complications and mortality in patients living with type 2 diabetes using a national medical claim database

https://doi.org/10.21203/rs.3.rs-1604014/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this older preprint version

Read the latest preprint version →

Objective

Prognostic models in patients living with diabetes allow physicians to estimate individual risk based on medical records and biological results. Clinical risk factors are not always all available to evaluate these models so that they may be complemented with models from claims databases. The objective of this study was to develop, validate and compare models predicting the annual risk of acute severe complications and mortality in patients living with type 2 diabetes (T2D) from a national claims data.

Research design and methods

Adult patients with T2D were identified in a national medical claims database through their history of treatments or hospitalizations. Prognostic models were developed using logistic regression (LR), random forest (RF) and neural network (NN) to predict annual risk of outcome: (1) severe acute cardiovascular (CV) complications, (2) other severe acute T2D-related complications, and (3) all-cause mortality. Risk factors included demographics, comorbidities, the adjusted Diabetes Severity and Comorbidity Index (aDSCI) and diabetes medications. Model performance was assessed using discrimination (C-statistics), balanced accuracy, sensibility and specificity.

Results

A total of 22,708 patients with T2D were identified, with mean age of 68 years and average duration of T2D of 9.7 years. Age, aDSCI, disease duration, diabetes medications and chronic cardiovascular disease were the most important predictors for all outcomes. Discrimination with C-statistic ranged from 0.715 to 0.786 for severe acute CV complications, from 0.670 to 0.847 for other severe complications and from 0.814 to 0.860 for all-cause mortality, with RF having consistently the highest discrimination.

Conclusion

The proposed models reliably predict acute severe complications and mortality in patients with T2D, without requiring medical records or biological measures. These predictions could be used by payers to alert primary care providers and high-risk patients living with T2D.

Type 2 Diabetes

Cardiovascular disease

Risk prediction

Health insurance claims

Type 2 diabetes (T2D) is one of the leading causes of morbidity and mortality worldwide, and the growing diabetes epidemic represents a major challenge to public health (1). Informing both primary care physicians and patients at the individual level of the risk could allow better targeting of active prevention of complications.

Patients with T2D have a two- to threefold higher risk of suffering from a cardiovascular disease (CVD), including a higher risk of myocardial infarction (MI), stroke, unstable angina, and congestive heart failure and a higher rate of CVD-related death than the population not living with diabetes (2–4). CV complications in diabetic patients are among the most prevalent complications (5), but other acute complications, such as metabolic disorder (ketoacidocetosis, severe hypoglycaemia etc.), acute renal failure, obliterating atherosclerosis of lower limbs or sepsis frequently occur, demonstrating a certain degree of severity of the disease (6).

Because of the chronicity and the progressive nature of the disease, long-term damage and organ failure may develop simultaneously or consecutively (7). Predicting the onset of acute complications or mortality in the short term (annual risk) by physicians is more complicated than for long-term damage (≥ 5 years) but remains critical to ensure that appropriate preventive actions are taken.

Preventive medicine increasingly relies on modelling techniques to estimate the individual’s ‘absolute risk’ of a future event in order to inform therapeutic decisions. Risk models have been developed for patients living with diabetes, mostly using risk factors available from electronic medical records (e.g., body mass index, smoking status, biological and lipids markers, blood pressure) (8, 9).

However, while these models are useful for clinical level risk prediction, they require information that is not always readily available without generalized electronic medical records and do not perform optimally when applied to populations other than the ones from which they were derived (9, 10).

Medical claims databases have emerged as an efficient source of data for disease monitoring surveillance (11). Such databases collect automatically demographics and hospital diagnoses, procedures, medications generated through the provision of health services. With the growing use of patients' personal insured accounts, payers have the ability to directly interact with primary care physicians and patients to communicate individual risks, so that general practitioners or specialists could identify high-risk profiles and recommend specific preventive actions. Reliable prediction of the short-term risk of acute events among patients with T2D based upon claims data would be an additional tool in the prevention of severe diabetes-related complications.

The objective of this study was to demonstrate the feasibility of developing prognostic models for short-term annual acute diabetes-related complications and mortality based on medical claims database and to compare the performance of different modelling approaches, including machine learning models.

Data source

A representative sample of 1/97 of all insured individuals from the French National Health Data Information System (SNDS) which contains all medical claims for the entire French population was used. This real-world database has been previously fully described elsewhere (12), and gathers information on all reimbursed healthcare resources. In France, a compulsory public insurance scheme is applied to all individuals to cover the majority of costs. Information on private insurance schemes that patients subscribe to cover the complementary part was not available in the database. It includes information on demographics, medical history, diagnoses and procedures related to in-hospital admissions, prescriptions, laboratory assays, and date of death.

Study design

A retrospective observational study design was used to model the risk of acute complications and mortality in patients with T2D (Additional file 1). The index date was fixed at the 1st of January 2014 corresponding to the inclusion date for all patients living with T2D to this date. Patients living with T2D were identified through a 2-year historical window before index date and were followed during a total of 4 years, to the 31st of December 2017 and censored after the first occurrence of an acute complication for the CV and other acute analysis, and death for the all-cause mortality analysis.

Population

Patients living with T2D were included in the study and were identified in a 2-step process. Claims database may presents some uncertainty for a reliable diagnostic based on ICD-10 between type 1 and type 2 diabetes, so that patients living with diabetes mellitus were first identified, and patients identified with type 1 diabetes criteria were excluded in a second step.

Adult patients living with diabetes mellitus were identified through a validated algorithm (13), based on 3 inclusion criteria evaluated during the 2-year historical window (2012 and 2013). Criteria were mutually exclusive and were applied in hierarchical order: 1/ individuals with ongoing long-term disease (LTD) with a diagnosis of diabetes mellitus (patients are eligible for full reimbursement of healthcare costs capped at NHI tariff and the diagnosis is recorded in the claims database), 2/ individuals without LTD for diabetes but hospitalized with a primary or secondary diagnosis of diabetes mellitus, and 3/ individuals having 3 deliveries at different dates of oral antidiabetics or insulin, without LTD or diabetes-related hospitalization. Diagnoses codes were based on the ICD-10 classification and included sections E10.x, E11.x, E12.x, E13.x, E14.x.

Patients identified with type I diabetes were excluded from the study if they filled one of the following criteria: individuals under 45 years of age when insulin was first delivered, or individuals under 45 years of age at the start of LTD for diabetes.

Study outcomes

Study outcomes included severe acute CV complications, other severe acute complications, and all-cause mortality. Complications were considered severe because they were identified through diagnoses from recorded hospitalizations in the claims database. Complications managed solely in outpatient care could not be traced. Severe acute CV complications included admissions with primary diagnosis of heart failure (HF), peripheral arterial disease, myocardial infarction (MI), stroke, unstable angina (UA), transient ischemic attack (TIA) and CV-related death (Additional file 2). Cardiovascular death was defined as death occurring at hospital during admission with a primary or secondary diagnosis of CV complications, or sudden cardiac arrest, cardiogenic shock, and other form of heart disorder. Other severe acute complications included admissions for metabolic disorder such as ketoacidosis coma, ketoacidosis, acidosis and hypoglycemia, acute renal insufficiency, amputation and sepsis of any type. All-cause mortality included in-hospital and out-hospital death occurring during the 4-year follow-up, regardless of the cause of death.

An additional analysis was conducted to assess ability of models to predict multiple acute complication within the same year (≥ 2 any acute complications separated by at least 30 days).

Risk factors

For outcomes’ prediction, potential risk factors were derived from a 5-year historical window before index date. Risk factors included demographics (age, gender), diabetes-related information (long-term disease coverage for diabetes, duration of long-term disease, insulin and antidiabetic treatments), pre-existing chronic conditions (CVD, cardio-neurovascular disease, chronic respiratory disease, history of cancer or ongoing cancer, and inflammatory or rare disease), Charlson Comorbidity Index (CCI) (14), and adapted diabetes complications severity index (aDCSI) (15). The aDCSI is a validated score used specifically with claims database and includes 7 categories of diabetes complications (ophthalmic, renal, neurologic, cerebrovascular, cardiovascular, peripheral vascular, and metabolic). Each category is scored with either 0 (no complication), 1 (non-severe complication), or 2 (severe complication), except for neurologic complications, which score a maximum of 1.

Diabetes medications (ATC class A01B) including sulfonylureas, meglitinides, metformin, thiazolidinediones, a-glucosidase inhibitors, and insulin, were stratified and introduced into the models as follow: insulin only, insulin and antidiabetic treatment, antidiabetic monotherapy, bitherapy or tritherapy. Treatments for cardiovascular risk, such as b-blockers, calcium channel blockers, antihypertensive combinations were also included as covariates.

Pre-existing chronic conditions were identified through different medical algorithms combining LTD and history of chronic and acute diagnoses (Additional file 2). Overall, the presence of a condition was identified when individuals had ongoing LTD coverage for the condition, or individuals were hospitalized with a primary or secondary diagnosis related to the condition during the 5-year historical window before index date. The presence of LTD coverage and its duration were used as predictors.

Chronic cardiovascular disease, which was included in the CCI, was defined by chronic conditions diagnostic code (LTD) and recent acute complications including: coronary disease, stroke, chronic and acute heart failure, peripheral arterial disease, cardiac rhythm disorders and valvular disease occurring before the year preceding the index date (Additional file 2).

Prognostic models

Pooling of repeated observations (PRO) was used to predict annual onset of each outcome. For each patient, the at-risk period was stratified into annual periods during which outcomes were observed. The PRO method pools observations over disjoint time intervals of equal length into a single sample to predict short-term risk of events (16). Since risk factors vary over time, the method accounts for time dependent covariates, and risk factors were revaluated each year.

Distinct frameworks of prognostic models were developed, including standard approach with a logistic regression (LR) model, and machine learning approach with a random forest (RF) model and a neural network (NN) model. For LR model, the risk factors included in the final prognostic models were checked for multicollinearity and chosen using a stepwise variable selection approach based on Akaike’s Information Criterion.

The RF method was developed by Brieman (17) and relies on binary prediction trees. Variable selection is mechanically carried out because each split uses only a single covariate, and approximates variable interaction through the hierarchical structure of the node splits. RF method corresponds to a collection of bootstrap samples that aim to reduce the variance of the prediction. For this study, RF used the SAS procedure HPFOREST, parametrized with a number of 50 maximum trees and a number of 20 maximum nodes.

The NN model had a multilayer perceptron architecture, as described by Bishop (18), composed of several successive sets of neurons (layers). Parameters and weights were estimated by minimizing the loss function by using the SAS procedure HPNEURAL. The number of layers and neurons in each layer were fixed manually and step-by-step, by minimizing the average absolute error. An architecture of 4 layers was chosen, with k*4 neurons (being the number of risk factor, different in each set) for the first layer, k*2 for the second layer, k for the third layer and k/4 for the last layer.

Measurement of performance

A cross-validated approach was used as an internal validation: the population was split into a training set (75%) and a validation set (25%). The training set was used to develop the prognostic models, and the validation set was used to assess the performance of the models. The performance of the final prognostic models was evaluated based on discrimination, accuracy, sensitivity and specificity in the validation set. The discrimination was assessed with the C-statistic that measures how well a prognostic model differentiates between patients with and without the outcome. It varies between 0.5 (no better than chance) and 1.0 (perfect discrimination) and corresponds to the proportion of subjects whose observed and predicted outcomes are concordant. A naïve bootstrap confidence interval was estimated by resampling 100 data sets, with replacement, dividing into training and validation sets. Balanced accuracy was computed as a measure of overall predictive performance and the average of two proportions: the proportion correctly predicted for those who experienced the outcome (sensibility) and the proportion correctly predicted for those who did not (sensitivity). A score of 1 indicates a perfect model, and 0.5 indicates that the model is no better than chance. Balanced accuracy addresses the well-known phenomenon that binary classifiers tend to be biased toward the more frequent class, yielding an overly optimistic estimate of accuracy (19). Sensitivity and specificity were determined using the threshold that maximized the sum of both.

Calibration of models were also presented as a goodness-of-fit. Calibration refers to the agreement between predicted and observed outcomes. Observed risk versus median predicted risk were compared for several sub-groups of patients. Calibration was also presented graphically with regression slopes of observed versus predicted risk for the different outcomes, and a 95% confidence interval calculated to evaluate the goodness of fit.

Statistical analysis

These analyses were conducted in accordance with the TRIPOD (Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis) requirements (20). Results are presented overall and for population subgroups (patients with and without chronic cardiovascular disease (CVD) and patients under and over 65 years of age). Sensitivity analyses were conducted by excluding HF in acute CV complications and by predicting multiple acute complications within the same year (≥ 2 any acute complications separated by at least 30 days). Additional analyses were conducted to assess robustness of models, by predicting the 1-year risk of outcomes during the 1st year only (without the PRO method), and by predicting the 4-year risk based on risk factors assessed at baseline only (without the PRO method).

Population

A total of 25,549 patients were identified with diabetes mellitus, mainly identified through ongoing long-term disease coverage for diabetes mellitus (n=17,371, 68.8%), antidiabetic or insulin therapy (n=5,328, 21.1%), or a recorded diagnosis of DM (n=2,550, 10.1%) during the previous 2 years before the index date (Additional file 3). Among them, 2,541 patients (9.9%) identified with type 1 diabetes were excluded, and 22,708 patients with T2D were included in the study for analysis. Mean age was 68 years (11.4), 53% patients were male, and average duration of T2D was 9.7 years (7.5) (Additional file 4). 45% of patients had a CCI of 2 or more, in which T2D participated for 1 point. Predominant comorbidities were CVD (28%), cancer (12%) and chronic obstructive pulmonary disease (12%). For the majority of patients, no previous severe acute diabetes-related complication was observed within the 5 previous years, as defined by the aDSCI, with 83% of patients having aDSCI evaluated to 0. A high proportion of patients with CVD was included, but most of them did not experience a recent CV complication (only 15% with aDSCI ≥ 1).

Description of outcomes

The proportion of patients with any acute complication was 14,9% (n=3,386) during the at-risk period (4 years), corresponding to an incidence rate of 41.4 severe acute complications per 1000 patients-year (Table 1). Acute CV complications were predominant (77%), including HF (27%), peripheral arterial disease (13%), MI (7.5%), stroke (10%), UA (7%), TIA (7%) and CV-related death (5%). Other acute complications included metabolic disorder (7%), sepsis (7%), renal disorder (7%), and amputation (2%). Median time to events was 15 months (7-24) and no difference was observed between occurrence of acute CV complications and other acute complications. Similar proportions of complications were included between training (n=2,874, 11.8%) and validation (n=1,112, 13.4%) sets. The 4-year mortality rates were 10.7% (n=2,608) and 12.9% (n=1,046) in the training and validation sets respectively.

Comparison of patients with and without severe acute complication

Patients with any acute complication during the complete at-risk period were older (median age: 74 vs 67 years old), had a higher CCI (index > 1: 68% vs 41%) and a higher aDCSI (score > 0: 37% vs 14%) at baseline. Without age adjustment, they were more likely to have chronic cardiovascular disease (53% vs 23%), cancer (15% vs 12%), COPD (17% vs 11%), neurologic disease (6% vs 4%), and chronic kidney failure (2% vs 0.5%).

Models’ performance

Overall, models performed well in predicting severe acute diabetes-related complications and all-cause mortality. with C-statistic being superior to 0.70 for all models in the validation dataset and RF outperforming other models for accuracy and discrimination (Table 2).

Balanced accuracy ranged from 0.665 to 0.717 for acute CV complications, from 0.645 to 0.801 for all complications and from 0.745 to 0.794 for all-cause mortality, suggesting models are more accurate at predicting severe acute complications and mortality than chance alone, with RF having systematically the highest accuracy. Discrimination with C-statistics ranged from 0.715 to 0.786 for acute CV complications, from 0.670 to 0.847 for all complications and from 0.789 to 0.837 for all-cause mortality, with NN having consistently the highest discrimination.

Subgroup analyses and sensitivity analyses are presented in Table 3. Discrimination was systematically superior in patients without CVD, and in patients under 65 years of age. Additional sensitivity analyses suggested similar performances when predicting outcomes during the first year of follow-up only, or when predicting outcomes within the 4-year period from baseline risk factors (without PRO method) (Table 3). Results were shown to be robust in predicting 2 or more events within one year, with slightly better performance than predicting a single event.

Including the type of resource used during the last year (number of visits at general practitioner, ophthalmologist, cardiologist, endocrinologist, dental and other specialist, and number of biological assays for HbA1c, glycaemia, creatinine, albuminuria and lipids, and ECG, number of admissions, number of days, outpatient costs) (not shown) was assessed but did not improve model performance.

Risk factors

The same set of predictors were introduced in all models across outcomes. Age was the risk factor with the largest impact on the risk of having a severe acute complication, whether with LR or RF (Additional file 5 and Additional file 6), followed by the aDSCI. Insulin and antidiabetic medications were associated to an increased and protective effect respectively, reflecting a selection bias for which more severe patients are prescribed insulin. Treatments for cardiovascular risk was not retained by the LR model, but was in the top 5 of risk factors that contributed the most for the RF model. Other risk factors, such as CVD, psychiatric disorders and chronic end-stage renal disease were consistently associated with a significant risk of event across outcomes. Additionally, for all-cause mortality, COPD, liver or pancreas disease, cancer and neurodegenerative disease were associated to an increased risk of death, whether with LR or RF.

Calibration and predicted risks

Calibrations of models were presented for several subgroup populations (Additional file 7).

Overall, all models underestimated risk of outcomes, particularly for patients with severe profiles and for all-cause mortality.

Patients with observed severe acute complications were at 2 to 3-fold increased predicted risk of any acute complication, and patient with observed death were at 5 to 8-fold increased risk of all-cause mortality. Patients with pre-existing CVD at baseline were at 3 to 4-fold increased risk of any acute complication and of all-cause mortality. Patient over 75 years of age at baseline were at 2 to 3-fold increased risk of CV acute complication and 3 to 6-fold increased risk of all-cause mortality. Similarly, patients without treatment or with insulin therapy, alone or in association, were at 2 to 4-fold increased risk of any complications or all-cause mortality.

One-year risks stratified by aDSCI and CVD were also presented for the RF model (Figure 1). In patients with CVD, risks increased with aDSCI score, reaching 12% of death rate for patient with an aDSCI of 5. In patients without CVD, risks of acute CV complication or all-cause mortality were similar between patients with aDSCI between 0 and 4, while the risk of any acute complication was increasing with aDSCI.

We developed prognostic models for the annual risk of acute severe complications in patient living with T2D, including acute CV complications, other acute severe complications and all-cause mortality. The originality of this study lies in the fact that we used exclusively data from a national claims database and it compared different modelling approach, including logistic regression model and machine learning approach with random forest and neural network. We demonstrated the feasibility of applying these approaches to medical claims data to predict the annual risk of acute complication in this population.

The main finding of this study was that RF model performed well in predicting severe acute complications and all-cause mortality with simple and few risk factors derived from medical claims database, such as age, LTD, antidiabetic medications, aDSCI, history of CVD and presence other comorbidities. This framework makes possible a routine evaluation of the different individual risks by the national health insurance communicated for patients and for primary care providers to identify high-risk patients and to personalize prevention.

In a recent meta-analysis on CVD prediction in patients with T2D, authors reported that C-statistics ranged from 0.64 to 0.80 for prognostic models developed in patient living with diabetes (9). In their meta-analysis, authors estimated a pooled C-statistics of 0.67 based on validation studies of the following prognostic models: UKPDS calculator (21), the ADVANCE model (22), the DCS model (23), the Fremantle model (24) and the NDR model (25). Our study highlights results from LR, RF and NN models were comparable with the existing findings for CV prognosis (26).

The crucial difference between models developed in this study and models included in this meta-analysis lies in the risk factors, which were derived from biological results and medical records in the latter. In contrast, we used only information that is available from claims database. Our study demonstrated the feasibility of prognostic models based on claims information and the possibility to derive relevant risk factors such as diabetes duration (available with LTD duration), history of diabetes-related complications (aDSCI) and antidiabetic treatments which are not systematically included in all clinical models described above. However, some risk factors identified in the UKPDS, NDR, or Advance models remain difficult to identify, and thus, limit the predictive accuracy. For example, smoking or obesity do exist, but are typically underreported in medical claims.

Prognostic models did take into account diabetes therapeutic strategies during the study period. Adjustment was made for patients without treatment, or patients receiving insulin and/or antidiabetic medication This analysis conducted with LR showed that patients receiving antidiabetic treatments were at lower risk of outcome (acute CV or other complications and all-cause mortality) compared to patients receiving no treatment.

Concerning acute CV outcomes, the comparison with the literature may be limited as authors mostly predict composite of strong endpoints (MI, stroke, CVD-related death, and CHD). In our study, we enlarged CV outcomes to other severe acute DM-related complications (UA, TIA and peripheral arterial disease) because they were also associated to major morbidity in DM patients. Hospitalization for CHF represented an important part of acute CV complications (~ 25%) in our study. Excluding CHF in acute CV complications was associated to a deterioration in discrimination, with C-statistics decreasing from 2 to 4 points.

To our knowledge, this study is also the first study to include a wide range of acute complications requiring hospitalization such as metabolic disorder with ketoacidosis coma, ketoacidosis, acidosis, hypoglycaemia, sepsis, acute renal insufficiency and amputations. Model performances for predicting these other acute complications were overall similar to those for acute CV complications, and better performance for RF model.

Machine learning models performed better than logistic regression to predict mortality and had better performance than the existing literature. In a recent study predicting the 5-year mortality risk in older adults with T2D (27), the authors presented balance accuracy of 0.77 and C-statistics of 0.74 with model including key risk factors, such as biological markers, BMI, and smoking. Our study showed that without the latest risk factors, but with other key risk factors (disease duration, aDSCI and diabetes medications), similar results can be achieved.

A second finding was that RF had the best accuracy and discrimination among models. Compared to a similar claims database study that predicted the 3-year risk of adverse outcomes with machine learning, discrimination performance of RF (C-statistics 0.77–0.86) model in our study was relatively close to the performance with complex machine learning models (C-statistics = 0.77), such as gradient boosting decision tree, recurrent neural networks, multilayer perceptron and transformers (28). On a broader perspective, machine learning models have been shown to be replicable and transferrable to local healthcare systems when built on a national database, while a model constructed on a local level cohort was difficult to transfer to a national level (29). On top, factors that usually contribute to risk of bias, including small study size, poor handling of missing data, and failure to deal with overfitting were not present in this study (30).

Survival models based on hazard ratio estimation were not included in our framework for two reasons: first, they were considered to use a different approach with instantaneous risk estimation which is different than binary outcome prediction, and second, dates were only available on a monthly basis, for anonymous purposes, which would have certainly underestimated performance of these models.

Main predictors identified with LR and RF were age and history of diabetes related complications described with aDSCI, diabetic medications, and diabetes duration. Additionally, chronic CV disease, psychiatric disorder and chronic end-stage renal disease were consistently associated with a significant risk of event across outcomes. Finding age and comorbidities as the most important risk factors for the three outcomes was consistent with the literature (8, 31). This study also confirmed that the aDSCI, initially developed for the prediction of mortality and risk of hospitalization (32), was an important predictor for acute complications and all-cause mortality from this claims database.

The first limitation was the identification of study outcomes, based on probabilistic algorithms using health insurance claims data that have not been fully validated, which could lead to the misclassification of outcomes and comorbidities. Second, patients may have experienced complications or major events prior to the start of data availability, such as CVD, and comorbidities may have been underreported or misclassified. Third, a recorded diagnostic code on a medical claim may be inaccurate. Taking that into account, authors applied additional inclusion criteria to differentiate type 1 and type 2 diabetes by using a minimum onset age of 45 years, conjointly applied with insulin delivery or LTD, before which patients were assumed to have type 1 diabetes.

Finally, censoring patients at 1st occurrence of event prevented the prediction of multiple and recurrent complications. However, our sensitivity analyses showed robustness in predicting 2 or more events within one year, with slightly better performance than predicting a single event.

Risk evaluation is essential to individualize therapy, and is encouraged by clinical practice guidelines for the management of risk factors. However, in practice, biological risk factors are not all systematically available and health professionals other than diabetologist may not be familiar with clinical models, which can limit their use in some cases. The use of routine prediction models from real-world claims database is to provide a transparent platform to communicate this risk to the patient and help all health professionals to make a quick and accurate assessment of their patients' risk and optimize their management care.

Medical claims databases are a valuable resource to develop prognostic models that have a strong potential to identify patients at high risk of acute complications within a certain time window (33, 34). Risks could be routinely assessed by the national health insurance, owner of the data, and communicate thereafter to patients, and primary care providers to personalize prevention.

Models based on national medical claims data could reliably predict severe acute complications and mortality in patients with T2D, without requiring medical records or biological measures. These models could be relevant for all health professionals to identify high-risk patients and optimize their monitoring, and more generally for payers to implement preventive measures.

aDSCIAdapted Diabetes Severity and Comorbidity index

ATC Anatomical Therapeutic Chemical

BMI Body Mass Index

CCI Comorbidity Charlson Index

CHF Chronic Heart Failure

COPD Chronic Obstructive Pulmonary Disease

CV Cardiovascular

CVD Cardiovascular Diseasese

DM Diabtes Mellitus

LR Logistic Regression

LTD Long-Term Disease

MI Myocardial Infarction

NDR Swedish National Diabetes Register

NHI National Health Insurance

NN Neural Network

PRO Pooling of repeated observations

RF Random Forest

T2D Type 2 diabetes

TIA Transient Ischemic Attack

UA Unstable Angina

Ethics approval and consent to participate

The database is de-identified and fully compliant with the patient confidentiality requirements of the Health Insurance Portability and Accountability Act (HIPAA), approved and authorized by the French regulatory institution (Commission Nationale de l'Informatique et des Libertés, Cnil) under the registration number 918343.

Consent for publication

Not applicable

Availability of data and materials

The datasets analyzed during the current study are not publicly available due to the mandatory approval and authorization needed from the French regulatory institution. Aggregated data are available from the corresponding author on reasonable request.

Competing interests

The authors declare that they have no competing interests.

Funding

The study was fully funded by Public Health Expertise.

Authors' contributions

All authors conceived of the article, wrote and edited the manuscript. All authors contributed to interpretation of data. All authors contributed to critical revision of the manuscript All other authors were involved in the writing and editing of the manuscript. A.V and I.D.Z. are guarantors of this work and, as such, had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Acknowledgements

Not applicable

Cho NH, Shaw JE, Karuranga S, Huang Y, da Rocha Fernandes JD, Ohlrogge AW, et al. IDF Diabetes Atlas: Global estimates of diabetes prevalence for 2017 and projections for 2045. Diabetes Res Clin Pract. 2018;138:271-81.
Almdal T, Scharling H, Jensen JS, Vestergaard H. The independent effect of type 2 diabetes mellitus on ischemic heart disease, stroke, and death: a population-based study of 13,000 men and women with 20 years of follow-up. Arch Intern Med. 2004;164(13):1422-6.
Emerging Risk Factors C, Sarwar N, Gao P, Seshasai SR, Gobin R, Kaptoge S, et al. Diabetes mellitus, fasting blood glucose concentration, and risk of vascular disease: a collaborative meta-analysis of 102 prospective studies. Lancet. 2010;375(9733):2215-22.
Laakso M. Cardiovascular disease in type 2 diabetes from population to man to mechanisms: the Kelly West Award Lecture 2008. Diabetes Care. 2010;33(2):442-9.
Deshpande AD, Harris-Hayes M, Schootman M. Epidemiology of diabetes and diabetes-related complications. Phys Ther. 2008;88(11):1254-64.
Faselis C, Katsimardou A, Imprialos K, Deligkaris P, Kallistratos M, Dimitriadis K. Microvascular Complications of Type 2 Diabetes Mellitus. Curr Vasc Pharmacol. 2020;18(2):117-24.
Orasanu G, Plutzky J. The pathologic continuum of diabetic vascular disease. J Am Coll Cardiol. 2009;53(5 Suppl):S35-42.
Chamnan P, Simmons RK, Sharp SJ, Griffin SJ, Wareham NJ. Cardiovascular risk assessment scores for people with diabetes: a systematic review. Diabetologia. 2009;52(10):2001-14.
Chowdhury MZI, Yeasmin F, Rabi DM, Ronksley PE, Turin TC. Prognostic tools for cardiovascular disease in patients with type 2 diabetes: A systematic review and meta-analysis of C-statistics. J Diabetes Complications. 2019;33(1):98-111.
Crawford F, Chappell FM, Lewsey J, Riley R, Hawkins N, Nicolson D, et al. Risk assessments and structured care interventions for prevention of foot ulceration in diabetes: development and validation of a prognostic model. Health Technol Assess. 2020;24(62):1-198.
Jung J, Im JH, Ko YJ, Huh K, Yoon CG, Rhee C, et al. Complementing conventional infectious disease surveillance with national health insurance claims data in the Republic of Korea. Sci Rep. 2019;9(1):8750.
Tuppin P, Rudant J, Constantinou P, Gastaldi-Menager C, Rachas A, de Roquefeuil L, et al. Value of a national administrative database to guide public decisions: From the systeme national d'information interregimes de l'Assurance Maladie (SNIIRAM) to the systeme national des donnees de sante (SNDS) in France. Rev Epidemiol Sante Publique. 2017;65 Suppl 4:S149-S67.
Fuentes S, Cosson E, Mandereau-Bruno L, Fagot-Campagna A, Bernillon P, Goldberg M, et al.Identifying diabetes cases in health administrative databases: a validation study based on a large French cohort. Int J Public Health. 2019;64(3):441-50.
Huang YQ, Gou R, Diao YS, Yin QH, Fan WX, Liang YP, et al. Charlson comorbidity index helps predict the risk of mortality for patients with type 2 diabetic nephropathy. J Zhejiang Univ Sci B. 2014;15(1):58-66.
Chang HY, Weiner JP, Richards TM, Bleich SN, Segal JB. Validating the adapted Diabetes Complications Severity Index in claims data. Am J Manag Care. 2012;18(11):721-6.
Cupples LA, D'Agostino RB, Anderson K, Kannel WB. Comparison of baseline and repeated measure covariate techniques in the Framingham Heart Study. Stat Med. 1988;7(1-2):205-22.
Brieman L. Setting up, using, and understanding random forests 2003.
Bishop CM. Neural Networks for Pattern Recognition Press OU, editor1995.
Brodersen KH OC, Stephan KE, Buhmann JM. . The balanced accuracy and its posterior distribution. 20th International Conference on Pattern Recognition2010. p. p. 3125–8.
Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD Statement. BMC Med. 2015;13:1.
Stevens RJ, Kothari V, Adler AI, Stratton IM, United Kingdom Prospective Diabetes Study G. The UKPDS risk engine: a model for the risk of coronary heart disease in Type II diabetes (UKPDS 56). Clin Sci (Lond). 2001;101(6):671-9.
Kengne AP, Patel A, Marre M, Travert F, Lievre M, Zoungas S, et al. Contemporary model for cardiovascular risk prediction in people with type 2 diabetes. Eur J Cardiovasc Prev Rehabil. 2011;18(3):393-8.
Elley CR, Robinson E, Kenealy T, Bramley D, Drury PL. Derivation and validation of a new cardiovascular risk score for people with type 2 diabetes: the new zealand diabetes cohort study. Diabetes Care. 2010;33(6):1347-52.
Davis WA, Davis TM. Cardiovascular risk prediction in adults with type 1 diabetes: the Fremantle Diabetes Study. Diabetes Res Clin Pract. 2010;90(3):e75-8.
Cederholm J, Eeg-Olofsson K, Eliasson B, Zethelius B, Nilsson PM, Gudbjornsdottir S, et al. Risk prediction of cardiovascular disease in type 2 diabetes: a risk equation from the Swedish National Diabetes Register. Diabetes Care. 2008;31(10):2038-43.
Young JB, Gauthier-Loiselle M, Bailey RA, Manceur AM, Lefebvre P, Greenberg M, et al. Development of predictive risk models for major adverse cardiovascular events among patients with type 2 diabetes mellitus using health insurance claims data. Cardiovasc Diabetol. 2018;17(1):118.
Griffith KN, Prentice JC, Mohr DC, Conlin PR. Predicting 5- and 10-Year Mortality Risk in Older Adults With Diabetes. Diabetes Care. 2020;43(8):1724-31.
Ravaut M, Sadeghi, H., Leung, K.K. et al.Predicting adverse outcomes due to diabetes complications with machine learning using administrative health data. Digit Med 2021;4(24).
Kim E, Caraballo PJ, Castro MR, Pieczkiewicz DS, Simon GJ. Towards more Accessible Precision Medicine: Building a more Transferable Machine Learning Model to Support Prognostic Decisions for Micro- and Macrovascular Complications of Type 2 Diabetes Mellitus. J Med Syst. 2019;43(7):185.
Andaur Navarro CL, Damen JAA, Takada T, Nijman SWJ, Dhiman P, Ma J, et al. Risk of bias in studies on prediction models developed using supervised machine learning techniques: systematic review. BMJ. 2021;375:n2281.
Mukherjee T, Robbins T, Lim Choi Keung SN, Sankar S, Randeva H, Arvanitis TN. A systematic review considering risk factors for mortality of patients discharged from hospital with a diagnosis of diabetes. J Diabetes Complications. 2020;34(11):107705.
Young BA, Lin E, Von Korff M, Simon G, Ciechanowski P, Ludman EJ, et al. Diabetes complications severity index and risk of mortality, hospitalization, and healthcare utilization. Am J Manag Care. 2008;14(1):15-23.
Gruss SM, Nhim K, Gregg E, Bell M, Luman E, Albright A. Public Health Approaches to Type 2 Diabetes Prevention: the US National Diabetes Prevention Program and Beyond. Curr Diab Rep. 2019;19(9):78.
Manuel DG, Rosella LC. Commentary: assessing population (baseline) risk is a cornerstone of population health planning--looking forward to address new challenges. Int J Epidemiol. 2010;39(2):380-2.

Table 1

Incidence of acute complications during the 4-year risk period

Type of complication	Number (N, %)	Incidence per 1,000 pts-yr
Acute CV complications	2,606 (77%)	31.8
Heart failure	924 (27.3%)	11.3
Peripheral arterial disease	429 (12.7%)	5.2
MI	253 (7.5%)	3.1
Stroke	349 (10.3%)	4.3
Unstable Angina	239 (7.1%)	2.9
TIA	226 (6.7%)	2.8
CV death	186 (5.5%)	2.3
Other acute complications	780 (23%)	9.5
Metabolic disorder	252 (7.4%)	3.1
Sepsis	234 (6.9%)	2.9
Renal disorder	241 (7.1%)	2.9
Amputation	53 (1.6%)	0.6
All acute complications	3,386 (100%)	41.3
Note: CV, Cardiovascular; MI, Myocardial Infarction; pts-yr: Patient-year; TIA Transient Ischemic Attack

Table 2

Comparative performance for severe acute complications risk prediction models

	Training set C-statistic		Validation set
	Training set C-statistic	C-statistic	95%CI	Sensitivity	Specificity	Balance accuracy
Acute CV complications
LR	0.738	0.715	0.658-0.772	73%	60%	0.665
RF	0.944	0.786	0.747-0.825	71%	72%	0.717
NN	0.739	0.738	0.686-0.790	70%	67%	0.685
Other acute complications
LR	0.738	0.706	0.653-0.759	59%	70%	0.645
RF	0.981	0.847	0.800-0.894	80%	81%	0.801
NN	0.732	0.670	0.626-0.714	74%	52%	0.630
All-cause mortality
LR	0.823	0.814	0.769-0.859	76%	73%	0.745
RF	0.941	0.860	0.821-0.899	80%	79%	0.794
NN	0.845	0.841	0.791-0.891	82%	71%	0.765

Note: CV, Cardiovascular; LR, Logistic Regression; NN, Neural Network; RF, Random Forest.

Table 3

Comparative performance for subgroup and sensitivity analyses (C-statistic)

C-statistic		Subgroup-analyses				Sensitivity analyses
C-statistic	Reference	Without CVD	With CVD	<65 years	≥65 year	Excluding HF	Multiple event	1^st year only	4-year risk
Acute CV complications
LR	0.715	0.679	0.642	0.811	0.669	0.753	0.752	0.701	0.688
RF	0.786	0.739	0.722	0.848	0.788	0.796	0.839	0.793	0.769
NN	0.738	0.693	0.647	0.84	0.699	0.785	0.78	0.716	0.724
Other acute complications
LR	0.706	0.664	0.676	0.776	0.668	0.737	0.731	0.705	0.691
RF	0.847	0.809	0.789	0.949	0.797	0.869	0.885	0.857	0.837
NN	0.670	0.619	0.609	0.748	0.649	0.727	0.684	0.654	0.661
All-cause mortality
LR	0.814	0.793	0.761	0.877	0.762	-	-	0.814	0.793
RF	0.860	0.844	0.795	0.931	0.823	-	-	0.860	0.855
NN	0.841	0.833	0.756	0.928	0.787	-	-	0.841	0.835
Note: CV, Cardiovascular; HF, Heart Failure; LR, Logistic Regression; NN, Neural Network; RF, Random Forest.

No competing interests reported.

ArticleDiabetesCarDiabSupplementary.docx

Download PDF

Version 1

posted

You are reading this older preprint version

Read the latest preprint version →

Prognostic models for short-term annual risk of severe acute complications and mortality in patients living with type 2 diabetes using a national medical claim database

Status:

Version 1

Abstract

Figures

Background

Method

Data source

Study design

Population

Study outcomes

Risk factors

Prognostic models

Measurement of performance

Statistical analysis

Results

Population

Description of outcomes

Comparison of patients with and without severe acute complication

Models’ performance

Risk factors

Calibration and predicted risks

Discussion

Conclusion

Abbreviations

Declarations

References

Tables

Additional Declarations

Supplementary Files

Status:

Version 1