Are we there yet? AI on traditional blood tests efficiently detects common and rare diseases

doi:10.21203/rs.3.rs-4354480/v1

Download PDF

Article

Are we there yet? AI on traditional blood tests efficiently detects common and rare diseases

https://doi.org/10.21203/rs.3.rs-4354480/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Chronic workforce shortages, unequal distribution, and rising labor costs are crucial challenges for most healthcare systems. The past years have seen a rapid technological transition to counter these pressures. We developed an AI-assisted software with ensemble learning on a retrospective data set of over one million patients that only uses routine and broadly available blood tests to predict the possible presence of major chronic and acute diseases as well as rare disorders. We evaluated the software performance with three main approaches that are 1) statistics of the ensemble learning focusing on ROC-AUC (weighted average: 0.9293) and DOR (weighted average: 63.96), 2) simulated recall by the model-generated risk scores in order to estimate screening effectiveness and 3) performance on early detection (30–270 days before established clinical diagnosis) via creating historical anamnestic patient timelines. We found that the software can significantly improve three important aspects of everyday medical practice. The software can recognize patterns associated with both common and rare diseases, including malignancies, with outstanding performance. It can also predict the later diagnosis of selected disease groups 1–9 months before the establishment of clinical diagnosis and thus could play a key role in early diagnostic efforts. Lastly, we found that the tool is highly robust and performs well on data from various independent laboratories and hospitals on widely available routine blood tests. Compared to decision systems based on medical imaging, our system relies purely on widely available and inexpensive diagnostic tests.

Health sciences/Medical research/Translational research

Health sciences/Biomarkers/Diagnostic markers

artificial intelligence

ensemble methods

clinical chemistry

clinical pathology

blood test

diagnostics

The broad adoption of digital databases, modern testing, screening, and monitoring devices, and the expansion of digital communication channels created the opportunity to distribute tasks involving skilled medical professionals and form professional networks to discuss challenging cases. This transformation of the available tools of healthcare systems and the pandemic-induced restrictions resulted in an accelerated transition and acceptability of virtual medicine. Laboratory follow-ups are a perfect setting for virtual encounters. Testing can be carried out in a controlled remote environment convenient for the patient, and the rest of the communication between the medical expert and the patient can be done via digital means.

The healthcare provider (HPC) makes the clinical diagnosis based on their clinical experiences, findings, and the results of the diagnostic tests they ordered. Clinical pathology/laboratory medicine physicians may perform reflex testing, i.e., further testing if necessary, based on initial test results. For that, several semi-automated check mechanisms were developed (e.g., Mentzer index, that is, MCV divided by the RBC count, which is used to differentiate iron deficiency anemia from beta-thalassemia; the discrepancy between hemoglobin and hematocrit). Clinical pathologists may notify the healthcare providers of the results; however, direct clinical pathologist communication is usually restricted to conditions that require immediate intervention. To support the diagnostic process of the HCPs in non-critical situations, a more comprehensive and in-depth analysis of the laboratory test results could be helpful. Such in-depth evaluation is not feasible at the clinical pathologist level due to the vast number of tests performed and the lack of patient data (e.g., previous test results and clinical findings). Therefore, other means of laboratory result evaluation are sorely needed.

Diagnostic error and delay plague 15–20% of all patient-doctor encounters[1, 2], which often manifests as late or delayed diagnosis and is the sixth leading cause of death in the US[3]. Contrary to that, laboratory error rates are low, 0.33%, ranging from 0.1–3.0%[4]; thus, laboratory test results can serve as reliable and high-quality data for decision-support algorithms and artificial intelligence. Diagnostic errors in internal medicine are of a complex origin and are mostly based on no-fault, system-related, and cognitive errors. Cognitive errors are primarily due to the false processing of available information and often account for the most severe misdiagnoses[2]. Rare diseases present an even bigger issue for delay in recognition and misdiagnosis. A study of late-onset Pompe disease showed that 48% of patients were incorrectly diagnosed in the past, with an increased mean time to diagnosis of 10.5 vs. 2.5 years (from symptom onset)[5].

Artificial intelligence (AI) in healthcare is a rapidly developing field that started expanding around 2012 with the improvement in deep learning. The major application and area of development has been image-based diagnosis, while other fields (genome interpretation, biomarker discovery, patient monitoring) have also recently gained more importance [6]. Clinical chemistry and blood test interpretation have so far been less in the focus of medical AI despite the challenges posed to HCPs by the broad set of increasingly specialized laboratory tests and guidelines. Although the vast amount of data may aid diagnostic accuracy, the human element leads to errors, and borderline results are easily misinterpreted by medical professionals[7].

In our current work, we developed an AI-powered tool to diagnose common and rare diseases based on routine clinical laboratory results and provide evidence for the potential of this and similar tools to transform laboratory medicine by improving diagnostic accuracy and enabling earlier diagnosis. Our main contributions are that

We propose an AI tool for diagnosing rare and common chronic diseases with high performance based merely on widely available, low-cost blood tests and show that depending on the nature of the disease the method can signal the diseases in an earlier stage.
The AI models were trained on an over million-sized cleaned dataset dipped from a population-representative large patient set including all healthcare levels from secondary to quaternary care.
On the national level, significant savings in the healthcare system may come from using AI-aided inexpensive and non-specific blood test evaluation for early diagnosis.

The software development leveraged 15 years and 1.3 million of complete, uncleaned, but anonymous patient medical record data (2000–2015, directly from the HIS - Hospital Information system) in cooperation between the University of Debrecen Clinical Center (UDCC), County Hospitals of Szabolcs-Szatmár-Bereg (CHSSB) and UDCC’s contractual software developer partner (Aesculab Medical Solutions - Black Horse Group Ltd.). The sites involved in the research are the following: hospitals of the clinical campus in Debrecen (UDCC), the András Jósa University Hospital, and additional smaller hospitals in neighboring cities, five hospital units, and their outpatient centers. Table 1 shows the diseases and number of patients involved in the study with descriptive statistics of each group.

Table 1

Descriptive statistics of the ensemble training set. Patients with multiple diseases (comorbidities) are included in each disease category.
	Patient # (N)	Patient rate (%)	Patients’ age (years)	Male to female ratio	Patients 18 + yo.
All patients incl. all healthy individuals	1,163,723	100.00%	38.12 ± 23.37	44.64%	77.37%
Thyroid diseases	73,143	6.29%	49.32 ± 18.37	16.70%	94.85%
Liver diseases	42,404	3.64%	54.23 ± 15.39	60.56%	98.24%
Kidney diseases	54,214	4.66%	61.08 ± 20.62	43.59%	96.17%
Inflammatory bowel diseases	41,954	3.61%	30.91 ± 26.96	45.36%	59.08%
Lipid metabolism disorders	79,589	6.84%	59.16 ± 13.24	45.05%	99.53%
Nutritional anemias and disorders	16,669	1.43%	44.90 ± 26.69	33.87%	79.20%
Anemias other than nutritional	76,790	6.60%	48.11 ± 29.27	44.70%	77.47%
Diabetes mellitus (type 1 and 2)	75,028	6.45%	61.33 ± 15.07	47.14%	97.62%
Systemic autoimmune disorders	18,070	1.55%	53.20 ± 17.13	21.05%	96.22%
Gallbladder and pancreatic disorders	49,357	4.24%	57.50 ± 17.85	36.82%	98.46%
Disorders involving immune mechanism	10,757	0.92%	22.20 ± 23.06	50.20%	44.89%
Cardiovascular diseases	238,062	20.46%	60.58 ± 15.44	45.52%	99.34%
Leukemias and lymphomas	7,167	0.62%	59.14 ± 18.34	49.15%	94.83%
Malignancies of the digestive system	18,007	1.55%	65.90 ± 12.30	55.79%	99.89%

Due to the cardinality of the patients and the variability of the contextual factors regarding age, gender, or occupation and availability of the healthcare system, we developed a framework to preprocess the data. As a result, we managed to identify disease groups where we meet all necessary conditions to apply machine learning while achieving a reliable, reproducible, and scalable system. The dataset we used contained data from multiple hospitals and laboratories. One of our biggest challenges was to make the data comparable and to find robust models that are as insensitive to data from different sources as possible.

Creating the set of disease cases

Finding valid disease labels before applying machine learning methods is essential for adequately functioning disease classification software. This validation procedure must be mainly machine-driven as tens of thousands of medical cases are impossible for medical doctors to review. We used a multilevel structure of mainly machine lead and partly manual validation procedures. The first layer of disease categorization was the ICD-10 coding, which is mandatory in the Health Information System (HIS) database. Every case has at least one code applied in the HIS. We labeled each case with our disease group, disease, or sub-disease labels based on the ICD-10 code(s) of the case in the HIS (data of the early 2000s recorded initially with ICD-9 framework, but the transition to ICD-10 already happened on institutional levels before accessing the datasets thus we did not need to address any transitional issues). Both the primary and additional diagnoses were labeled. As a subsequent layer, we tried to leverage available textual anamnestic data to 1) remove non-similar miscategorized cases and 2) to find additional disease cases that miss proper ICD-10 diagnosis. We used the classic data mining library of regular expressions (regex), the statistical descriptors term frequency-inverse document frequency (TF-IDF) and bag-of-words[8], and mathematical methods like cosine similarity matrices and a neural network, the bidirectional encoder representations from transformers (BERT) model[9]. This is a crucial step, as this procedure can alter the number of labeled disease cases by 10%-50%. Manual case validation is the last step of the labeling process, which usually counts as verification (for diseases of massive case occurrence), but is also a significant step with rare diseases (along with similarity calculations due to their heavy under and misdiagnosis) (Fig. 1). This procedure can result in multiple conditions per patient (each patient case has one binary condition for each disease group - yes or no - but a single patient can have multiple “yes” calls for the different diseases) (Fig. 2).

Train and test protocol

Data shown in the article is based on a random 50–50 split between train and test for software development. The sampling procedure was always repeated 30 times. Data shows the average of the different models and train-test data. The whole raw data was split for training purposes; the split was made to account for temporal and regional differences in the dataset. For time-based separation, we usually used a five- to eight-year split depending on the participating hospital, partially due to HIS needs (system changes) and to achieve more homogenous data. The most common time separation was the following: 2000–2005, 2000–2008, 2007–2015, 2006–2010, and 2011–2015. Regional separation was used to separate data from the two leading county hospitals (UDCC and CHSSB), smaller hospitals (CHSSB data), and clinical departments from each other. The training was carried out on the intersections of the separated data, and the resulting performance metrics are the averages of the best-performing models with a restriction that the final ensembles always contain test-train on different regional separated data (trained on one hospital/department, tested on another) with different laboratories involved. The lab tests were assigned to the medical (disease) cases in two different ways. 1) By direct linkage during the diagnostic procedure either automatically (within the HIS) or manually (with rare diseases almost entirely) by an MD during the creation of disease cases (i.e., when a specific blood test result at the beginning of diagnostics resulted in a specific diagnosis). This accounted for 50–60% of all connections. 2) Every lab test less than two weeks prior and not more than four weeks after the received diagnosis was directly linked to the medical case. In 40–50% of the cases, the HIS did not provide a tool for an automatic association, and the manual assignment would have been time-consuming; those tests were not linked. After pairing lab test results with diagnostics, we filtered them to the most commonly used blood tests based on the department’s statistics. We only kept blood tests that are routinely done for almost all incoming patients. Data is available at least once for not less than 25% of the hospital population in certain spatiotemporal data sets used for training (see Table 2 for more details).

Table 2

List of blood tests used for the AI. Abbreviations are as follows. ALB: albumin, ALP: alkaline phosphatase, ALT: alanine transaminase, AMY: amylase, AST: aspartate aminotransferase, BASO: basophil count, Ca: calcium level, CHOL: cholesterin level, CK: creatine kinase, Cl: chloride level, CREA: creatinine level, CRP: C-reactive protein, dBIL: direct bilirubin concentration, eGFR: estimated glomerular filtration rate, EO: eosinophil count, ESR: erythrocyte sedimentation rate, Fe: iron level, FER: ferritin level, FRUC: fructosamine level, GGT: gamma-glutamyl transferase level, GLUC: glucose level, HCT: hematocrite, HDL-C: high-density lipoprotein concentration, HGB: hemoglobin concentration, K: potassium level, LDH: lactate dehydrogenase level, LDL-C: low-density lipoprotein concentration, LIP: lipase level, LYM: lymphocyte count, MCH: mean cellular hemoglobin, MCHC: mean corpuscular hemoglobin concentration, MCV: mean corpuscular volume, Mg: magnezium level, MONO: monocyte count, MPXI: mean peroxidase index, MPV: mean platelet volume, Na: sodium level, NEU: neutrophil count, P: phosphate level, PDW: Platelet distribution width, PLT: platelet count, RBC: red blood cells count,RDW: red cell distribution wid,th, tBIL: total bilirubin concentration, TG: triglyceride level, TP: total protein level, TRF: transferrin level, TSH: thyroid stimulating hormone level, UA: uric acid level, UREA: urea nitrogen level, WBC: whole blood count.
Blood count	WBC, RBC, HGB, HCT, MCV, MCH, MCHC, PLT, RDW, MPV, PDW, NEU, LYM, MONO, EO, BASO
CH metabolism	GLUC, FRUC
Kidney function	UREA, CREA, eGFR
Metabolic breakdown products	UA, tBIL, dBIL
Ions	Na, K, Cl, Ca, Mg, P
Protein parameters	TP, ALB
Inflammatory markers	hsCRP, ESR, MPXI
Iron panel	Fe, TRF, FER
Lipid panel	CHOL, TG, HDL-C, LDL-C
Enzymes	AST, ALT, GGT, ALP, LDH, CK, LIP, AMY
Hormones	sTSH

In the development process we also analyzed the possibility of providing risk assessment for rare diseases. Although each rare disease affects less than one in 2000 per definition, as there are over 7000 rare diseases altogether, every 17th person in the population is affected by a rare disease. This is already a significant number; rare diseases often present a unique diagnostic challenge that results in underestimation and underdiagnosis. The involved rare disease groups, including 15 rare diseases (conditions), following the Orphanet classification: rare dyslipidemia (familial hypercholesterolemia, familial chylomicronemia syndrome), rare inborn errors of metabolism (Pompe disease, Wilson’s disease, hemochromatosis, Gaucher disease, Gilbert’s syndrome, and similar rare conditions of bilirubin metabolism and excretion), rare genetic hepatic diseases (Caroli disease, primary biliary cholangitis), rare hematologic diseases (cryoglobulinemia, hemophilia), rare genetic renal diseases (Bartter syndrome and similar conditions), rare systemic and rheumatologic diseases (adult-onset Still’s diseases and similar rare autoimmune arthritis) and rare endocrine diseases (Cushing disease, Addison's disease). We included diseases based on the prevalence and expertise within the Clinical Center, where the development and consultation with rare disease specialists occurred.

Furthermore, as patient data in the hospital information system also included diagnostic codes for malignant diseases if present, we could train our software on this information and see if we can predict the occurrence of malignant diseases in advance. Due to the topic’s sensitive nature, this evaluation was made visible only to the healthcare provider, not the patients.

Software structure

The software package consists of three major parts: a) an ensemble machine learning core, b) an extensive database of laboratory test results, and c) a clinical report generator. The disease representation learned by the mathematical models is used to classify the patients into one or multiple disease groups. For more details on the software structure see Appendix A and Fig. 3. For an extended description of the learned representation and the machine learning models of disease classification see Appendix B. To demonstrate the quality of the learned representation, we show that the patients with different diagnoses align in the space using the UMAP[11] framework (Fig. 4a and Fig. 4b).

Evaluation measures and model selection

Our evaluation procedure includes various performance metrics: Receiver Operating Characteristic (ROC) Area Under Curve (AUC)[8], diagnostic odds ratio (DOR)[12], accuracy, and sensitivity[8], for more details, see Appendix C.

After identifying the best hyperparameters per method, we combined the output of the best methods for each classifier with an ensemble layer based on Adaptive Boosting. The final model for a single disease is a linear combination of the six base machine learning methods of all the disease models.

We evaluated the software performance with three main approaches. Details on all approaches can be found in Table 3. The generated performance statistics aimed to answer the question of how effective the used models are in making aggregated predictive decisions tested on retrospective non-training data. We mapped the continuous zero-one logistic score to a one-five scale risk score. Recall (sensitivity) shows a monotonous increase by this risk score in the probability of the presence of the disease. This confirms that our model estimates the risk and is coherent with the real risk. Lastly, early detection scores assess the possibility of evaluating the risk of the analyzed disease groups earlier than the clinically validated diagnosis happens.

Table 3

Average model performance statistics of the used ensemble models
	Average ensemble statistics					Avg. tested recall by risk score²					Early detection scores³
	Avg. AUC	Avg. DOR	Avg. acc.	Avg. sens.	Avg. [email protected] TPR	1	2	3	4	5	Avg. LOP (30+)	Avg. LOP (90+)	Avg. LOP (180+)	Avg. LOP (270+)
Thyroid diseases	0.9368	73.54	93.03%	79.27%	16.88%	3.39%	28.82%	58.13%	69.28%	85.42%	-1.15%	-3.35%	-4.70%	-4.66%
Liver diseases	0.9124	48.32	93.54%	62.59%	27.24%	1.37%	8.85%	33.11%	44.68%	74.58%	-3.99%	-5.45%	-6.68%	-10.26%
Kidney diseases	0.9718	112.40	95.01%	80.11%	6.43%	2.53%	29.93%	50.92%	76.87%	91.06%	no loss	no loss	no loss	-2.84%
Lipoprotein metabolism disorders	0.9302	49.86	93.82%	80.79%	22.62%	4.08%	24.28%	39.97%	57.19%	97.91%	-1.63%	-4.13%	-6.52%	-5.19%
Diabetes mellitus	0.9412	62.13	92.12%	77.46%	22.19%	4.51%	24.31%	35.44%	59.01%	94.66%	no loss	-2.10%	-2.17%	-1.93%
Non-infected inflammatory bowel diseases	0.8859	72.04	93.48%	42.72%	19.26%	0.35%	4.79%	11.14%	27.35%	36.86%	-3.33%	-3.30%	-4.97%	-4.89%
Nutritional anemias	0.9264	64.40	94.66%	68.78%	9.15%	2.24%	8.25%	26.05%	41.30%	65.08%	no loss	no loss	no loss	no loss
Other anemias	0.9358	50.13	91.95%	79.38%	12.59%	4.30%	29.22%	46.38%	61.47%	82.20%	no loss	no loss	-3.23%	-5.68%
Systemic autoimmune disorders	0.9317	126.39	94.66%	66.38%	7.68%	2.83%	30.04%	49.89%	72.68%	85.11%	no loss	no loss	no loss	-4.43%
Gallbladder and pancreatic disorders	0.8847	62.29	93.30%	60.02%	32.81%	1.39%	7.36%	14.41%	39.83%	77.60%	-6.04%	-6.10%	-8.72%	-7.61%
Other immune disorders	0.9049	93.89	94.50%	63.65%	4.12%	0.89%	5.29%	8.89%	38.89%	79.31%	no loss	no loss	no loss	-6.96%
Cardiovascular diseases	0.9246	34.60	91.13%	79.40%	30.57%	7.68%	26.56%	40.62%	64.08%	85.38%	-3.17%	-6.57%	-6.52%	-5.96%
Leukemias and lymphomas	0.9739	376.88	96.83%	81.14%	4.82%	1.63%	22.39%	54.90%	70.57%	87.65%	no loss	no loss	no loss	-7.36%
Adenocarcinomas of the digestive sys.	0.9514	251.33	95.48%	62.94%	22.42%	1.05%	12.71%	20.13%	40.61%	54.17%	-1.25%	-3.10%	-5.84%	-20.35%

In a diagnostic time scale, the disease appears at an unknown time point, while diagnostic tests lead to an established clinical diagnosis only at a later time point. The software generally considers only the medical cases with a valid diagnosis at the model training stage (only blood tests related to a given diagnosis, either based on HIS data or timely, were used for training). However, we also briefly evaluated the possibility of having more data available on the diagnosed person (i.e., full medical history of blood tests containing both “related” and “unrelated” data to a given diagnosis). We saw that if a broad history of multiple blood tests is available to a person and used for model training, the prediction (ROC-AUC score) performance increases significantly with on average + 0.0272 ± 0.0213.

Table 3 shows that recall grows monotonously and proportionately to the increase of the prediction score (projected to one-five), which reinforces the hypothesis that the binary classification works well (the higher the prediction score, the higher the actual chance of the disease presence considering a single medical case). As the classification uses “relevant” medical cases (set of medical cases which resulted in a specific diagnosis) along the clinical disease progression, this means that by using only routine blood tests, the software can make the categorization along the time of the clinical manifestation of the examined diseases (we also carried out detailed testing outside of this time window for early detection). The classification works well for straightforward diagnoses (e.g., kidney, thyroid, or lipid metabolism disorders), hematology (traditionally mainly done from a complete blood count panel), and complex diseases. We saw a clear presentation pattern for these pathologies during the disease progression, where prediction is possible without disease-specific testing. For many disease groups (primarily for more complex ones, i.e., inflammatory bowel diseases and rare diseases), the classification performance may also vary due to the higher rate of underdiagnosis and misdiagnosis.

Table 4

Percentage of patients with available blood test results in specific days before the establishment of the diagnosis
	30 + days before	90 + days before	180 + days before	270 + days before
Thyroid diseases	9.3%	7.8%	6.5%	5.4%
Liver diseases	18.2%	14.2%	12.1%	10.6%
Kidney diseases	9.9%	8.1%	6.8%	5.7%
Lipoprotein metabolism disorders	16.9%	14.5%	12.8%	11.4%
Diabetes mellitus	9.7%	7.8%	6.8%	6.0%
Noninfective inflammatory bowel diseases	15.8%	12.3%	10.4%	9.2%
Nutritional anemias	21.3%	16.6%	14.0%	12.1%
Other anemias	32.1%	21.7%	15.5%	11.1%
Systemic autoimmune disorders	5.6%	4.4%	3.2%	2.5%
Gallbladder and pancreatic disorders	19.1%	16.6%	14.8%	13.2%
Other immune disorders	15.5%	11.2%	9.1%	7.8%
Cardiovascular diseases	11.9%	9.9%	8.5%	7.5%
Leukemias and lymphomas	2.1%	1.4%	1.0%	0.6%
Adenocarcinomas of the digestive sys.	3.2%	2.4%	2.0%	1.4%

When evaluating the possibility of early detection, we used only “non-relevant” disease cases, which cannot be associated directly with the studied disease manifestation. Around ten percent of patients had “non-relevant” cases 30–360 days before established diagnosis. This number is generally higher with widespread chronic diseases, with complications of other (chronic) diseases, and with diseases with less straightforward diagnostic procedures. This low number can be due to the low participation rate in screening programs and the fact that our data source healthcare institutes mainly provide more advanced care and fewer screening visits. Despite the limitations, because of the massive sample size, even this limited data can provide useful insights in early screening analysis.

From the seven rare disease categories, all received reasonably high ROC-AUC scores. Diseases with direct or semi-direct markers within routine blood work show exceptionally high ones (dyslipidaemias, hepatic and metabolic diseases, hematologic disorders), mainly in line with the findings of common disease groups. Even diseases that require special blood testing (e.g., Wilson disease, Cushing’s syndrome, or Addison's disease) show a clear pathologic pattern on routine blood work that can be used for educated prediction. Table 5 shows the aggregated average performance of the software in each rare disease.

Table 5

Estimated performance for the selected rare diseases
	Number of patients (N)	Average ROC-AUC score
Familial Chylomicronemia Syndrome (FCS)	21	0.9951 ± 0.0194
Familial Hypercholesterolemia (FH)	459	0.9664 ± 0.0103
Primary Biliary Cholangitis (PBC)	477	0.9449 ± 0.0109
Caroli disease	25	0.8853 ± 0.0971
Gaucher disease	162	0.8836 ± 0.0343
Gilbert’s syndrome and similar conditions	2730	0.9455 ± 0.0079
Bartter syndrome and similar conditions	28	0.8604 ± 0.0841
Hemochromatosis	500	0.9527 ± 0.0134
Wilson’s disease	141	0.8627 ± 0.0267
Coagulation defects (acquired or hereditary)	2271	0.9139 ± 0.0065
Cryoglobulinemia	189	0.9333 ± 0.0223
Pompe disease and similar conditions	51	0.8990 ± 0.0578
Still's disease and similar autoimmune conditions	117	0.9237 ± 0.0346
Cushing’s syndrome	334	0.8601 ± 0.0235
Addison's disease	483	0.8893 ± 0.0163

Furthermore, with reverse-engineering logic, we determined the importance of each laboratory parameter in our AI's diagnosis of each disease group. Data suggests that our AI considers those parameters (lab tests) important in confirming or excluding the diagnoses that are part of the clinical diagnostic definitions in medical literature, and human medical professionals also consider them in their clinical reasoning. This indicates that our AI has found similar pathophysiological changes in the diseases that had been identified through decades of medical research by humans.

We calculated the importance of individual blood tests in the AI’s decision-making in a multistep process where models ran multiple times with one or more tests missing from the training. Based on the drop in the average model performances (ROC-AUC), we created an Importance Ranking of all the used blood tests: tests whose absence created a higher loss in performance received a higher Importance Rank. The rank is an average based on the multiple training runs of multiple models. The ranking is described with the score of relative importance where 100 is the value for the most important test (calculated separately for each examined disease group), and the rest of the tests are represented with a value as a percentage compared to the most important one.

The average relative importance, see Table 6, of the individual blood tests was measured for all disease groups. The categories show how important the test is, on average, for deciding the presence or absence of the disease group.

Table 6

The average relative importance of individual blood tests in the diagnosis of each disease group. Very important: The relative importance of the given blood test for the examined disease group is higher than 70. Important: The relative importance of the given blood test for the examined disease group is between 40 and 70. Not important: The relative importance of the given blood test for the examined disease group is lower than 1. All other parameters with an average relative importance between 1 and 40 were considered intermediate and are not shown in the table.
	Very important	Important	Not important
Thyroid diseases	sTSH	Ca, P	PDW, MPXI, LIP, AMY
Liver diseases	AST, ALT, GGT	PLT, GLUC, TG, tBIL, FER	PDW, LDL-C
Kidney diseases	eGFR, Mg, UREA, CREA, hsCRP	CK, sTSH, LDH	FER, TRF, LIP
Lipoprotein metabolism disorders	CHOL, TG	ALP, GLUC, CREA, MPV, LDL-C	Mg, LIP, kBIL
Diabetes mellitus	GLUC, FRUC	TG, eGFR	FER, TRF, kBIL
Noninfective inflammatory bowel diseases	GLUC, FE, AMY, LIP, tBIL	MCH, MCHC, RBC, UREA, ALB, CRP, TRF, GGT, MONO, NEU	Cl, HDL-C
Nutritional anemias	Fe, HGB, MCV	MCH, MCHC, RDW, FER, TRF, CK, RBC, P, tBIL, UREA, Na	-
Other anemias	HGB, HTC, MCH, MCHC	MCV, RDW, Fe, P, NEU	Mg, AMY, LIP, FRUC
Systemic autoimmune disorders	MCHC, RDW, tBIL, CK, MONO, sTSH	MCH, UA, GLUC, HDL-C, HTC, ALP, CHOL, CREA, CRP, MPXI, RBC, K, Mg, GPT, ESR	-
Gallbladder and pancreatic disorders	GGT, LIP, AMY	tBIL, eGFR, kBIL, LDH, ALP, ESR, K, CREA, CK, MONO	PDW, MPXI, Fe, EO
Other immune diseases	MCH, LIM, CREA, GGT, RBC, WBC, MCHC, GLUC, CRP	TP, K, Ca, CHOL, MONO, ALT	-
Cardiovascular diseases	ALP, eGFR, GLUC, CHOL, MPV, RBC, CK	UREA, ALB, tBIL, TP, Ca, RDW, MCHC	-
Leukemias and lymphomas	WBC, LIM, RDW, TP, LDH, ESR	MONO, NEU, UA, PLT, BASO	FRUC
Adenocarcinomas of the digestive sys.	RDW, LIM, CRP, RBC	LDH, ALP, GLUC, GGT, MONO, K	-

The importance of laboratory tests in the medical diagnostic processes is unquestioned; recent surveys show that 60–70% of clinical decisions are affected by laboratory test results[13], and at least 80% of guidelines, which are aimed at establishing a diagnosis or managing a disease, require laboratory testing[14]. The low error rate of laboratory tests makes laboratory test results a valuable base for AI-supported medical diagnostics. Numerous studies have shown that laboratory tests can effectively identify chronic diseases or infections[15–19]. However, there are certain common and rare diseases where traditional diagnostic processes are not always effective, resulting in delayed diagnoses and the eventual loss of early screening opportunities. Inflammatory bowel diseases (IBD) and rare metabolic disorders, for example, similarly share diagnostic challenges despite having different clinical and laboratory characteristics. IBD is notoriously difficult to diagnose based on non-specific blood test results, while rare metabolic diseases have more specific findings but are often overlooked due to their rarity. Nevertheless, most patients undergo routine phlebotomies during their hospital stays, which presents a significant opportunity for early screening and risk assessment based on blood test result patterns.

We previously showed that machine learning is ideal for diagnosing rare dyslipidaemias[20, 21], but analyzing a wider set of rare diseases shows that many rare diseases may be ideal candidates for computer-aided screening efforts, even ones that usually require (multiple) specific diagnostic tests for confirming the diagnosis. Although our initial data is promising due to the lack of a larger specific dataset, further studies are needed with the involvement of multiple sites and a higher number of patients to quantify this possibility properly.

Moreover, we tested the software’s general capability for early screening of supported diseases by evaluating the blood test results of patients who appeared regularly at hospitals for screening or other purposes but had not yet received an official diagnosis. As shown in Table 3 in the Results, we found that AI-aided diagnostics can boost early screening efforts, save time, and may allow earlier therapeutic intervention. The model's performance statistics did not, or only slightly, drop within six months before the medical diagnosis. Our retrospective data analysis determined that with the AI-aided blood test interpretation, most of the analyzed disease groups may be diagnosed at least one-to-six or even one-to-nine months before the current medical diagnosis. A significant drop in the performance of early diagnosis was only visible in more acute disease groups, like liver diseases and some cardiovascular pathologies.

Regarding early diagnosis, the regional and hospital-specific changes and patient behavior (e.g., frequent hospital goers) can impact the time of possible early diagnosis. It is also a question of debate what the earliest point is when a disease can be diagnosed, as most of the time, exact thresholds of parameters are to be met for definitive diagnosis. However, we can achieve an earlier diagnosis with more frequent follow-ups for those patients in a "pre-definitive disease stage". Furthermore, guideline updates may later include these pre-definitive disease stages with early treatment options to improve outcomes. Our data suggest that it is plausible that many common diseases may be diagnosed 30–90 days before the clinical diagnosis using AI-aided evaluation based on routine phlebotomy results (see Table 4). Our results also indicate that diagnosis earlier than 180 days will only be possible in a few cases without using other specific blood markers. It is also important to mention that the lowest amount of early diagnostic data (i.e., people’s participation in screening programs) is available for malignancies, a category where early diagnosis is most critical for effective therapy and where our findings show a possibility for very effective AI-assisted early screening. On the national level, significant savings may come from using AI-aided inexpensive and non-specific blood test evaluation for early diagnosis, thus, involving AI in laboratory result evaluation may facilitate the much-needed paradigm shift in healthcare towards early treatment and prevention.

The list of laboratory tests included in our AI analysis covers screening laboratory tests that are commonly used worldwide: basic and complete metabolic panel (ALP, ALT, AST, bilirubin, BUN, creatinine, sodium, potassium, chloride, albumin, total protein, glucose, and calcium), complete blood count with differential, and screening tests for lipid, iron, hepatic and thyroid metabolisms. These tests are among the most frequently ordered[22], allowing easy translation of our findings to help efficiently screen a wide range of diseases.

The common laboratory tests used as input for our software are part of routine phlebotomy due to their importance in general diagnostics of many acute and chronic diseases. For example, TSH tests are widely recommended for first-line thyroid dysfunction screening[23, 24]. At the same time, AST, ALP, GGT, and bilirubin levels alone may often identify the correct type and etiology of liver disease, allowing for a targeted investigation approach during clinical examination and further diagnostic testing[25]. Although these tests are part of general medical education and common medical knowledge, the clinical interpretations are far from trivial[25, 26]. The proper evaluation of liver tests, besides diagnosing acute liver pathologies, also helps diagnose Wilson’s disease; thus, they can distinguish rare pathologies from common medical conditions[25]. The comparison of certain blood analytics and other factors may also raise issues that are hard to handle and may lead to misinterpretation or misdiagnosis. For example, sTSH test reference ranges and values are highly sensitive to measurement technology, circadian patterns, analytical platforms, and geographic regions[27]. Therefore, following AI-aided analysis, including interpretative comments in laboratory reports, could decrease error rates, thus improving the quality of laboratory information and patient safety[28]. Although the major driver for including interpretative comments is new and complex tests in the laboratory report[29], interpretation of common lab tests is also welcome. In a survey in the UK, 88% of primary care doctors and nurse practitioners found interpretative comments on thyroid, gonadotropin, and glucose tolerance test reports helpful[30]. AI-supported decision-making may also decrease the rate of missed or late diagnoses originating from HCP’s incorrect interpretation of even basic laboratory tests. Self-reported testing practices for anemia show the overuse of screening laboratory tests, the underuse of bidirectional endoscopy to evaluate new-onset iron-deficiency anemia, and the misinterpretation of iron studies also included in our analysis[7].

In conclusion, this study illustrates the effectiveness and potential of AI-aided decision support systems in disease classification using blood tests. Despite the inherent challenges posed by variations in testing methods and equipment, we demonstrated that robust mathematical models and ensemble methods can yield reliable disease classification across diverse laboratory settings. We found that our AI system excels in the early detection of rare diseases, addressing a critical need in healthcare where economic constraints often limit screening for less common conditions. By facilitating pre-screening and early diagnosis, implementing our system can cut screening costs and improve patient selection for personalized treatments. Moreover, the ability of our AI tool to utilize retrospective data enhances its immediate applicability, offering substantial benefits to both patients and healthcare providers. We found that integrating AI into laboratory analyses represents a promising improvement in healthcare, which may promote early diagnosis, quality care, and cost-efficiency without necessitating the introduction of new testing methodologies. This study highlights the expected transformative impact of AI in laboratory medicine.

Ethics approval

The study was carried out as a part of UDCC’s data mining and machine learning initiatives under ethical permissions TUKEB 44549-2/2018/EKU and DE RKEB/IKEB 6697 − 2024.

Consent for publication

All authors have read and agreed to the published version of the manuscript.

Competing interests

All authors declare no financial or non-financial competing interests.

Funding

This research was funded by the National Research, Development, and Innovation Office (NKFIH, grant number: K142273). G. Toth received funding through the Society to Improve Diagnosis in Medicine's fellowship program, which is financially supported by the Gordon and Betty Moore Foundation. B. Daróczy received funding from the Ministry of Innovation and Technology NRDI Office within the framework of the Artificial Intelligence National Laboratory Program and from the C.N.R.S. E.A.I. project ”Stabilité des algorithmes d’apprentissage pour les réseaux de neurones profonds et récurrents en utilisant la géométrie et la théorie du contrôle via la compréhension du rôle de la surparamétrisation”. The funders played no role in study design, data collection, analysis and interpretation of data, or the writing of this manuscript.

Author Contribution

ÁN designed the study while ÁN, BD and ZK defined the methodology and developed the models. MH, PF, BN, GB and ZH validated the results. ÁN and GT wrote the original draft while BD and MH reviewed and edited the manuscript. BD, GP Jr., JK, MH, GP and IÉ supervised the study. All authors read and approved the final manuscript.

Acknowledgement

We are immensely thankful to Synlab AG (Synlab Hungary, Mr. Zsolt Csernák, Chief Medical Officer, and his medical team) for providing help, support, and international cooperation in the validation, testing, and further research works of the developed software.

Data Availability

Both source code and source medical data are proprietary technology and can not be shared publicly. However, for clinical and research use, the developed software is accessible free of charge to non-profit medical organizations (using standard HL7 communication of blood test data and outputting standard textual JSON/HL7 with disease/disease group predictions both qualitatively - 0 and 1 - and quantitatively with estimated risk percentages).

Barwise A, Leppin A, Dong Y, Huang C, Pinevich Y, Herasevich S, Soleimani J, Gajic O, Pickering B, Kumbamu A. What contributes to diagnostic error or delay? A qualitative exploration across diverse acute care settings in the US. J Patient Saf. 2021 Jun 1;17(4):239–48.
Graber ML, Franklin N, Gordon R. Diagnostic error in internal medicine. Arch Intern Med. 2005 Jul 11;165(13):1493–9.
Tudor Car L, Papachristou N, Bull A, Majeed A, Gallagher J, El-Khatib M, Aylin P, Rudan I, Atun R, Car J, Vincent C. Clinician-identified problems and solutions for delayed diagnosis in primary care: a PRIORITIZE study. BMC Fam Pract. 2016 Sep 9;17(1):131.
Carraro P, Plebani M. Errors in a stat laboratory: types and frequencies 10 years later. Clin Chem. 2007 Jul;53(7):1338–42.
Hobson-Webb LD, Kishnani PS. How common is misdiagnosis in late-onset Pompe disease? Muscle Nerve. 2012 Feb;45(2):301–2.
Yu KH, Beam AL, Kohane IS. Artificial intelligence in healthcare. Nat Biomed Eng. 2018 Oct;2(10):719–31.
Read AJ, Waljee AK, Sussman JB, Singh H, Chen GY, Vijan S, Saini SD. Testing Practices, Interpretation, and Diagnostic Evaluation of Iron Deficiency Anemia by US Primary Care Physicians. JAMA Netw Open. 2021 Oct 1;4(10):e2127827.
Baeza-Yates R, Ribeiro-Neto B, Ribeiro B de AN. Modern Information Retrieval. ACM Press; 1999. 548 p.
Devlin J, Chang MW, Lee K, Toutanova K. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In 2019 [cited 2023 Aug 28]. p. 4171–86. Available from: https://aclanthology.org/N19-1423
Tan PN, Steinbach M, Karpatne A, Kumar V. Introduction to Data Mining. Pearson; 2019. 839 p.
McInnes L, Healy J, Saul N, Großberger L. UMAP: Uniform Manifold Approximation and Projection. J Open Source Softw [Internet]. 2018 Sep [cited 2023 Sep 20]; 3(29). Available from: https://par.nsf.gov/biblio/10104557-umap-uniform-manifold-approximation-projection
Glas AS, Lijmer JG, Prins MH, Bonsel GJ, Bossuyt PMM. The diagnostic odds ratio: a single indicator of test performance. J Clin Epidemiol. 2003;56(11):1129–35.
Rohr UP, Binder C, Dieterle T, Giusti F, Messina CGM, Toerien E, Moch H, Schäfer HH. The Value of In Vitro Diagnostic Testing in Medical Practice: A Status Report. PloS One. 2016;11(3):e0149856.
The Lewin Group. The value of diagnostic innovation, adoption and diffusion into health care. Jul 2005. https://www.lewin.com/content/dam/Lewin/Resources/Site_Sections/Publications/ValueofDiagnostics.pdf
Brinati D, Campagner A, Ferrari D, Locatelli M, Banfi G, Cabitza F. Detection of COVID-19 Infection from Routine Blood Exams with Machine Learning: A Feasibility Study. J Med Syst. 2020 Jul 1;44(8):135.
Gunčar G, Kukar M, Notar M, Brvar M, Černelč P, Notar M, Notar M. An application of machine learning to haematological diagnosis. Sci Rep. 2018 Jan 11;8:411.
Alsheref FK, Gomaa WH. Blood Diseases Detection using Classical Machine Learning Algorithms. Int J Adv Comput Sci Appl IJACSA [Internet]. 2019 57/31 [cited 2023 Sep 20];10(7). Available from: https://thesai.org/Publications/ViewPaper?Volume=10&Issue=7&Code=IJACSA&SerialNo=12
AlJame M, Ahmad I, Imtiaz A, Mohammed A. Ensemble learning model for diagnosing COVID-19 from routine blood tests. Inform Med Unlocked. 2020;21:100449.
Aslan MF, Celik Y, Sabanci K, Durdu A. Breast Cancer Diagnosis by Different Machine Learning Methods Using Blood Analysis Data. Int J Intell Syst Appl Eng. 2018 Dec 27;6(4):289–93.
Paragh G, Harangi M, Karányi Z, Daróczy B, Németh Á, Fülöp P. Identifying patients with familial hypercholesterolemia using data mining methods in the Northern Great Plain region of Hungary. Atherosclerosis. 2018 Oct;277:262–6.
Németh Á, Harangi M, Daróczy B, Juhász L, Paragh G, Fülöp P. Identifying Patients with Familial Chylomicronemia Syndrome Using FCS Score-Based Data Mining Methods. J Clin Med. 2022 Jul 25;11(15):4311.
Horton S, Fleming KA, Kuti M, Looi LM, Pai SA, Sayed S, Wilson ML. The Top 25 Laboratory Tests by Volume and Revenue in Five Different Countries. Am J Clin Pathol. 2019 Apr 2;151(5):446–51.
Garber JR, Cobin RH, Gharib H, Hennessey JV, Klein I, Mechanick JI, Pessah-Pollack R, Singer PA, Woeber KA, American Association of Clinical Endocrinologists and American Thyroid Association Taskforce on Hypothyroidism in Adults. Clinical practice guidelines for hypothyroidism in adults: cosponsored by the American Association of Clinical Endocrinologists and the American Thyroid Association. Endocr Pract Off J Am Coll Endocrinol Am Assoc Clin Endocrinol. 2012;18(6):988–1028.
Esfandiari NH, Papaleontiou M. Biochemical Testing in Thyroid Disorders. Endocrinol Metab Clin North Am. 2017 Sep;46(3):631–48.
Agrawal S, Dhiman RK, Limdi JK. Evaluation of abnormal liver function tests. Postgrad Med J. 2016 Apr;92(1086):223–34.
Metra BM, Guglielmo FF, Halegoua-DeMarzio DL, Civan JM, Mitchell DG. Beyond the Liver Function Tests: A Radiologist’s Guide to the Liver Blood Tests. RadioGraphics. 2022 Jan;42(1):125–42.
Soh SB, Aw TC. Laboratory Testing in Thyroid Conditions - Pitfalls and Clinical Utility. Ann Lab Med. 2019 Jan;39(1):3–14.
Sciacovelli L, Aita A, Padoan A, Pelloso M, Antonelli G, Piva E, Chiozza ML, Plebani M. Performance criteria and quality indicators for the post-analytical phase. Clin Chem Lab Med. 2016 Jul 1;54(7):1169–76.
Vasikaran S, Sikaris K, Kilpatrick E, French J, Badrick T, Osypiw J, Plebani M, IFCC WG Harmonization of Quality Assessment of Interpretative Comments. Assuring the quality of interpretative comments in clinical chemistry. Clin Chem Lab Med. 2016 Dec 1;54(12):1901–11.
Barlow IM. Are biochemistry interpretative comments helpful? Results of a general practitioner and nurse practitioner survey. Ann Clin Biochem. 2008 Jan;45(Pt 1):88–90.

No competing interests reported.

supplementary.docx

Download PDF

Version 1

posted

You are reading this latest preprint version

Are we there yet? AI on traditional blood tests efficiently detects common and rare diseases

Status:

Version 1

Abstract

Figures

Introduction

Methods

Creating the set of disease cases

Train and test protocol

Software structure

Evaluation measures and model selection

Results

Discussion

Conclusion

Declarations

Competing interests

Funding

Author Contribution

Acknowledgement

Data Availability

References

Additional Declarations

Supplementary Files

Status:

Version 1