INTRODUCTION: There are currently more than 450 primary immune deficiency (PID) diseases, and about 7,000 rare diseases that together afflict around 1 in every 17 humans. Computational aids based on data mining and machine learning might facilitate the diagnostic task by extracting rules from large datasets and making predictions when faced with new problem cases.
OBJECTIVE: In a proof-of-concept data mining study, we aimed to predict PID diagnoses with a supervised machine learning algorithm based on classification tree boosting.
METHODS: Through a data query at the USIDNET registry we obtained a database of 2,396 patients with common diagnoses of PID, including their clinical and laboratory features. We kept 12 diagnoses and 286 features that were included in the model. We used the XGBoost package with parallel tree boosting for the supervised classification model, and SHAP for variable importance interpretation, on Python v3.7. The patient database was split into training and testing subsets, and after boosting through gradient descent, the predictive model provides measures of diagnostic prediction accuracy and individual feature importance. To correct for imbalanced classification, after a baseline performance test, we used the Class Weighting Hyperparameter, or scale_pos_weight.
RESULTS: The twelve PID diagnoses were CVID (1,098 patients), DiGeorge syndrome, Chronic granulomatous disease, Congenital agammaglobulinemia, ID not otherwise classified, Specific antibody deficiency, Complement deficiency, Hyper-IgM, Leukocyte adhesion deficiency, ectodermal dysplasia with immune deficiency, Severe combined immune deficiency, and Wiskott-Aldrich syndrome. For CVID, the model found an accuracy on the train sample of 0.80, with an area under the ROC curve (AUC) of 0.80, and a Gini coefficient of 0.60. In the test subset, accuracy was 0.76, AUC 0.75, and Gini 0.51. The positive feature value to predict CVID was highest for upper respiratory infections, asthma, autoimmunity and hypogammaglobulinemia. Features with the highest negative predictive value were high IgE, growth delay, abscess, lymphopenia, and congenital heart disease. For the rest of the diagnoses, accuracy stayed between 0.75 and 0.99, AUC 0.46-0.87, Gini 0.07-0.75, and LogLoss 0.09-8.55. See tables and figures.
DISCUSSION: Clinicians should remember to consider the negative predictive features together with the positives. We are calling this a proof-of-concept study to continue with our explorations. A good performance is encouraging, and the feature importance might aid feature selection for future endeavors. In the meantime, we can learn from the rules derived by the model and build a user-friendly decision tree to generate differential diagnoses.