Parkinsonian syndrome (PS) is characterized by bradykinesia, resting tremor, and rigidity, and it represents the phenotype observed in various neurodegenerative disorders. Positron emission tomography (PET) imaging plays an important role in diagnosing PS by detecting the progressive loss of dopaminergic neurons. This study aimed to develop and compare five machine-learning models for classifying [18F]PR04.MZ PET images between patients with PS and subjects without evidence for dopaminergic deficit (SWEDD). A dataset of [18F]PR04.MZ PET images from 204 subjects was analyzed and classified into PS compatible (1) and SWEDDs (0) by three blinded expert readers. The images were preprocessed to generate two and three-dimensional datasets. Five different pattern recognition algorithms, commonly used for image analysis, were trained and validated, comparing their performance to the majority reading of expert diagnosis considered as the standard of truth. Three models outperformed the others, achieving an accuracy greater than 98%. The results demonstrated that our machine-learning models, combined with [18F]PR04.MZ PET images, provide highly accurate and precise tools to support clinicians in PET image analysis. This approach may reduce the time required for interpretation and increase certainty in the diagnostic process.
Research Article
Image-based machine learning model as a tool for classification of [ 18 F]PR04.MZ PET images in patients with parkinsonian syndrome
https://doi.org/10.21203/rs.3.rs-3215495/v1
This work is licensed under a CC BY 4.0 License
Version 1
posted
You are reading this latest preprint version
Parkinson’s Disease
Machine Learning
Positron Emission Tomography
[18F]PR04.MZ PET tracer
Parkinsonian syndrome (PS) is an umbrella term comprising several heterogeneous and complex neurodegenerative disorders involving dopaminergic neurons in the substantia nigra pars compacta (SNpc) [1-3]. Among PS, Parkinson’s disease (PD) is the most common, and its prevalence is dramatically rising in Latin American countries [4-6], due to the industrialization and the improvement in life expectancy [4].
The diagnosis of PS is mainly based on clinical manifestations [7], requiring a high degree of expertise. To assist in this process the International Parkinson and Movement Disorder Society (MDS) published [8] and validated [9-10] a guideline to assure reproducibility in the diagnosis across medical centers and clinicians. This guideline has been updated including information on comorbidities such as diabetes mellitus (type II) [11-13], cognitive deficits [14-16], low plasma urate levels in men [17-20] and lifestyle factors such as physical inactivity [21-28], known to increase risk of PD. This updated MDS criteria also mentioned neuroimaging [29-34] to overcome the challenge of earlier and differential PS diagnosis, as well as improve diagnostic certainty and reinforce clinician’s decisions [35-36]. The reduction of presynaptic biomarkers, such as dopamine transporters (DAT), indicates the loss of nigrostriatal neurons and the subsequent dopaminergic deficit in the striatal regions of the brain. [18F]PR04.MZ is a PET tracer with high affinity, selectivity, and specificity for DAT, suitable for detecting dopaminergic deficit in the striatum and SNpc even before any motor symptoms appear [37-42].
However, image processing, modeling, and statistical analysis, requires specialized software packages, often limited to certain clinical centers. Furthermore, analyzing scan images, particularly those of complex anatomical organs like the brain, presents a challenging task that demands skill and experience to detect anomalies or changes accurately. While the availability of new radiotracers such as [18F]PR04.MZ are promising, their routine clinical use relies on reproducible methods of PET imaging quantification to minimize variability between readers and generate standardized clinical reports. Artificial intelligence (AI) offers an intriguing approach to interpreting PET imaging data, and the rapid advancements in machine and deep learning applied to diagnostic neuroimaging have demonstrated promising outcomes in various clinical scenarios [43-49]. Computer-aided diagnostic (CAD) methods utilizing neuroimaging exhibit increasing levels of sensitivity (recall) and specificity for accurate diagnoses [35-36], while AI-based approaches facilitate statistical prediction estimation through classification methods.
Image-based machine learning models are widely employed as CAD tools for classification tasks. These models apply different techniques such as support vector machines (SVM), which establish a hyperplane for binary classification and optimize the margin between the hyperplane and the classes [50-52]. Also Random forest [53], an ensemble method, which employs decision trees on data subsets, averaging their predictions to enhance accuracy and mitigate overfitting; this approach has proven effective for classifying patients with PD [54]. Logistic regression models also facilitate binary classification by transforming linear regression values into a range between 0 and 1 using a logistic function. And finally, K-nearest neighbor (K-NN) identifies the K closest data points based on Euclidean distance to form a classified set. In contrast, artificial neural networks are more intricate, simulating human brain neural networks and excelling in image processing tasks. Comprising layers that capture pattern complexity, ANN have demonstrated remarkable performance in this domain.
In this article, we aim to identify a suitable machine-learning model to facilitate the diagnostic based on [18F]PR04.MZ PET/CT imaging. We specifically examine and compare the performance of SVM, Random Forest, Logistic Regression, K-NN, and Neural Networks, assessing precision, recall, F1-score, accuracy and AUC as critical parameters to classify between patients with PS from subjects without evidence for dopaminergic deficit (SWEDD).
2.1 Patient population
In this study, 204 subjects (118 men, 86 women, 61.6±12.9 years, range 20-90 years), were referred to the Center for Nuclear Medicine & PET/CT PositronMed, Santiago, Chile, for PET/CT imaging with [18F]PR04.MZ to evaluate nigrostriatal neuronal integrity. The compiled database was split into two groups: 129 PET scans (63%) destined to train the models and 75 (37%) to test them.
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and national research committee and with the principles of the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards. This is a retrospective analysis of previously acquired PET data approved by the regional ethics committee board (CEC SSM Oriente, permit 20140520) and written informed consent has been obtained from all participants.
2.2 PET Imaging acquisition.
All images were acquired on a state-of-the-art WB PET/CT scanner (Biograph mCT Flow, Siemens Healthineers, Erlangen, Germany). Patients were allowed to rest for 55 min after injection of 4.99±0.82 mCi [18F]PR04.MZ (184.0±30.0 MBq, range: 94.7-234.6 MBq) before being placed head-first supine in the PET/CT scanner. A low-dose CT was performed for attenuation correction before the emission scan was performed in LIST-mode from 60-90 min post-injection (p.i.). PET images were corrected for random, scatter, attenuation, and time-of-flight, reconstructed by an ordered subset expectation maximization (OSEM) algorithm (2 iterations and 21 subsets) followed by post-reconstruction smoothing (Gaussian, 4 mm FWHM), corrected for motion if necessary, and averaged, following the manufacturer’s instructions (Siemens). Images consisted of 110 planes of 256x256 voxels of 1.59x1.59x1.5 mm3. PET scans were reframed into 6 static frames of 300 sec each, reviewed, corrected for motion and averaged for the period of 60-90 min post-injections. PMOD software (Version 3.4, Zurich, Switzerland: PMOD Technologies LLC. Available from: https://www.pmod.com/) was used to co-registered images to CT scans and to normalize to Montreal Neurological Institute (MNI) space. Volumes of interest (VOI) were outlined for the left and right anterior, posterior, and total putamen, caudate nucleus, SNpc, and cerebellum. Specific binding ratios (SBR) were calculated from SUVmean values as follows:
2.3 General workflow
PET imaging technique with [18F]PR04.MZ tracer implemented to acquire a PET image dataset. Later, three independent expert readers assessed and labeled two-hundred-four PET images. The images were preprocessed to generate 2D and 3D datasets for the subsequent training and analysis of five different ML models. Final statistical analysis was performed to evaluate accuracy, precision, recall, F1-score and AUC (Figure 1).
2.4 Classification of PET images
All PET images (axial and coronal views) and quantitative signal-to-background-ratio (SBR) values (Figure 2) were assessed and classified by three blinded expert readers. A deficit of >40% (eq. to 2 x SD) from the mean value of the healthy control group in any region was considered pathological.
Each scan was assigned as follows: normal (0) or abnormal (1), showing absence (0) or presence (1) of asymmetric binding, absence (0) or presence (1) of a rostro-caudal gradient, and being compatible (1) or not (0) with PS. The final label was assigned to a scan without evidence for dopaminergic deficit (0) or a scan showing dopamine deficiency compatible with PS, based on the agreement of at least two readers.
2.5 Imaging preprocessing and feature selection: 2D and 3D datasets
The data preprocessing pipeline aims to submit two distinct dataset dimensions, therefore, each input was processed differently (Figure 3). For the 2D dataset, the preprocessing stage focused on selecting the region of interest (ROI), targeting the striatal and midbrain regions within each image. To achieve this, images underwent initial cropping and resizing to homogenize the input data. As a result, reduced pseudo-color images with 80 x 40 resolution were obtained (Figure S1).
The preprocessing of the 3D dataset focused on the volume of interest (VOI), including 10 consecutive 2D images, comprising the striatal and midbrain regions. The unsupervised model applied to achieve this selection is described as follows:
1. From the acquired 3D image, 45% and 30% were removed from the upper and lower slices, respectively. This standardized inner volume was used as a first filter to select the VOI (Figure 4).
2. Density-based spatial clustering of applications with noise (DBSCAN) is a technique of cluster building around an arbitrary initial point. The parameter ‘eps’ marks the neighborhood radius of the point to analyze if it has ‘min samples’, the minimum number of points that the cluster can contain; if not, it’s considered as noise. The sklearn.cluster.DBSCAN function was used to select the slice with the highest density as the central slice of the output-optimized brain box. Therefore, the cluster show the tracer uptake in the target region of interest (ROI), i.e. caudate nucleus, anterior and posterior putamen of both left and right brain hemispheres (Figure 5).
3. Finally, a set of ten slices were selected to wrap the slice with the highest density, resulting in a reduced resolution scan output, as illustrated in Figure 6.
The success of this process allowed the reduction of the input images for the machine learning classification, before splitting them into the training/validation/test set for the supervised learning classification.
2.6 Machine learning model pipeline
After the dataset labeling and image preprocessing, 63% of the images were exclusively used to build a trained model using ML. To achieve our goal we compared five different classifiers (SVM, random forest, logistic Regression, K-NN and neural network) and systematically performed a 10-fold cross-validation method (Figure 7).
All classifiers' performance were evaluated on a set of the remaining images (37%) using the labels assigned by expert readers (Step 6).
2.6.1 2D-dataset
The 2D dataset was divided as described in Table 1. A total of 129 2D images were assigned to the training set, while the remaining 75 were for testing.
Table 1: Group composition and demographic details
|
|
SWEDD Training |
PS Training |
SWEDD Testing |
PS Testing |
|
Amount |
54 |
75 |
39 |
36 |
|
Age |
63.3±13.9 |
59.0±13.9 |
60.9±11.5 |
65.3±9.5 |
During the preprocessing stage (Figure 3), the ROI was selected and, prior to resizing the scans to a lower resolution of 40 x 80, the margins of each pseudo-color image were removed. This step allowed the input homogenization before running the algorithm.
To avoid overfitting, the training process systematically incorporated the 10-fold cross-validation technique, where the training dataset was splitted into 90% to train the model and 10% to validate its performance (Figure 7).
For training of the preprocessed images (step 5 in Figure 1), Python 3.8.2 scikit-learn functions (http://scikit-learn.org) were used to implement the ML models. To determine the optimal model, GridSearchCV was used to find the best parameter grid across all possible parameter value combinations. The selected parameters are shown in Table 2.
Table 2: Best parameter selection for each machine learning model using 2D dataset.
|
Model |
GridSearchCV selection |
|
SVM |
SVC(C=1, kernel='linear', probability=True, random_state=21) |
|
RF |
RandomForestClassifier(criterion='entropy', random_state=42) |
|
LR |
LogisticRegression(C=5, penalty='l1', random_state=21, solver='liblinear') |
|
K-nn |
KNeighborsClassifier(n_neighbors=1) |
TensorFlow keras (https://www.tensorflow.org/ guide/keras/sequential_model) was used to implement the neural network. The neural network was composed of three convolutional layers, each one with a 3x3 kernel layer, a ReLU activation function, followed by one max pooling layer size of 2x2 and one flatten with activation ‘relu’. The classification layer included a dense layer, activation function ‘sigmoid’, that allows full connectivity between neurons in preceding and succeeding layers. Finally, we compiled the model using a ‘binary_crossentropy’ loss, ‘adam’ optimizer and measured its performance with metric ‘accuracy’.
Each of the models were iteratively executed and validated, comparing their performance in terms of precision, recall, F1- score and accuracy (Figure 7).
2.6.3 3D-dataset
The results of the preprocessed 3D dataset were submitted as input for the supervised classifier. This 3D array was flattened into a pixel vector. Similar to the approach used for the 2D dataset, all of the machine learning models applied to the 3D-dataset employed the same hyper-parameter optimization grid search technique (Table 3). Also, a 10-fold cross-validation technique was used to prevent overfitting and create a generalized model, capable of adapting to unseen data. The models were later compared using the previously mentioned indicators: precision, recall, F1- score and accuracy.
Table 3: Best parameter selection for each machine learning model using 3D dataset.
|
Model |
GridSearchCV selection |
|
SVM |
SVC(C=1, kernel='linear', probability=True, random_state=0) |
|
RF |
RandomForestClassifier(n_estimators=45, random_state=0) |
|
LR |
LogisticRegression(C=1, penalty='l1', random_state=0, solver='liblinear') |
|
K-nn |
KNeighborsClassifier(n_neighbors=2, weights='distance') |
To select the best neural network combination of hyperparameters, the Ax Adaptive Experimentation Platform (https://ax.dev), which simplifies the search for an optimal neural network configuration, was used. The metric defined in the network compiler seeks a ‘binary accuracy’ optimization. AX library.get_next_trial() allows to iteratively create a neural network with a combination of parameters for 25 experimental trials. Finally, ax client.get_best parameters select the best set of parameters (Table 4).
Table 4: Summary of the best set of parameters for neural network binary accuracy optimization
|
Hyperparameter |
Best combination |
|
`learning rate' |
0.00013850114679874224 |
|
`dropout rate' |
0.010574775359670174 |
|
`num hidden layers' |
5 |
|
`neurons per layer' |
17 |
|
`batch size' |
8 |
|
`activation' |
'tanh' |
|
`optimizer' |
‘rms’ |
|
`keras cv' |
1.0000164359863488 |
To address overfitting, Ax platform was set to assign a ‘learning_rate’ within the range [0.0001, 0.001], and a ‘dropout_rate’ within [0.01, 0.05]. According to Table 4, 0.013% is the best learning rate to get the minimum loss function, also meaning more epochs and computational resources will be needed, while 1.06% dropout rate is the percentage of neurons dropped between the three hidden layers. Also, ‘EarlyStopping’ (https://keras.io/api/callbacks /early_stopping) with (patience=20) was included in the models’ callback, which prevents it from continuing to train once it has stopped improving.
Accuracy and loss value metrics were used to select the best model. Graphing both indicators allowed the comparison of the model performance with the training and validation data after each iteration. Accuracy facilitates the visualization of how precisely it is to classify the pattern of an image; meanwhile, loss value indicates the error that adds each training epoch. The convergence behavior of both graphs issues the best model for later classification of the test set. After setting the model to train for 500 epochs, and configuring early stopping (patience=20), the algorithm converged as shown in Figure 8, and was ready for final classification.
2.7 Statistical analysis
The inter-observer reliability between the clinicians and the image-based model identification was calculated using Cohen’s exact Kappa (κ) [54]. We considered the level of agreement according to the following value ranges for κ: slight (0.00 to 0.20), fair (0.21 to 0.40), moderate (0.41 to 0.60), substantial (0.61 to 0.80), and almost perfect (0.81 to 1.00) [56]. Calculations were performed using 95% confidence intervals (CI’s), and p<0.05 were considered significant. The statistical analysis was performed using R ”IRR”–CRAN, package 3.5.1 version (http://www.R-project.org).
To analyze the performance of the five different models, accuracy (1), precision (2), recall (3) and F1-score (4) were evaluated using the information for true positive (TP), true negative (TN), false positive (FP), and false negative (FN) and according to the following equations:
Where accuracy measures the percentage of samples the model has correctly classified, precision is the quality of the ML model in classification tasks rated according to the number of TP in a total of PS predictions, recall reports the number of PS subjects the ML model can identify and F-score measuring the performance of precision and recall between models.
In addition to the metrics mentioned above, Receiver Operating Characteristics (ROC) curves were also displayed. By showing the True Positive Rate (TPR) and False Positive Rate (FPR), ROC curves help to evaluate the performance of each classifier. That’s mainly why ROCs are commonly useful in binary classification problems. In an illustration, the “perfect” ROC curve is the one closer to the top-left corner of the image, since the ideal scenario implies having high TPR and no FPR.
The Area Under the curve (AUC), in this case the ROC curve, rates on a scale from 0 to 1 the value of the area covered by the curve. This metric, along with other indicators, contribute to the comparison between models’ performance.
Three independent expert readers blindly analyzed the [18F]PR04.MZ PET/CT images. As an additional measure of control, the study included eighteen healthy volunteers, all of whom were correctly classified by each expert reader. The Kappa statistic showed an almost perfect value (κ = 0.946; p<0.001, 95% CI 0.918–0.966), indicating a 97% agreement between the clinical and neural network diagnosis. Furthermore, only one case was misclassified by our algorithm (Table S1).
3.1 Performance with the 2D dataset
After training, the classifiers were tested with the remaining dataset portion (37%). Table 5 shows the results obtained from the python classifiers. Neural networks scored higher in all indicators compared to the other ML model.
For the 2D datasets, the k-NN presented the lowest performance versus the other model (Table 5).
Table 5: Classification performance average five image-based ML classifiers with testing images from the 2D dataset.
|
Classifier |
Precision |
Recall |
F1-score |
Accuracy |
AUC |
|
SVM |
0.86 |
0.84 |
0.82 |
82.67% |
0.9614 |
|
Random Forest |
0.80 |
0.71 |
0.67 |
69.33% |
0.9375 |
|
LogReg |
0.81 |
0.77 |
0.75 |
76.00% |
0.9621 |
|
K-NN |
0.69 |
0.57 |
0.48 |
54.67% |
0.6636 |
|
NNs |
0.97 |
0.97 |
0.97 |
97.33% |
0.9993 |
Figure 9 presents the ROC curves and summarizes the AUCs. Both of these metrics contribute to the comparison task between the models. Even though the database was small, neural networks achieved 0.9993 of area under the ROC curve, which in addition to the precision, recall, F1-score and accuracy, aim to choose it as the best of the models for the 2D input dataset.
3.2 Performance with 3D datasets
Table 6 shows the results achieved by each classifier for 3D datasets. All classifiers performed very accurately, although they still differ in the predicted probability of class membership (Table S1).
Even though k-NN showed a poor performance for 2D datasets, the results for the 3D dataset were significantly better. In general terms, our results showed that the models performed much better with 3D datasets than with 2D datasets (Table 6, Figure S2).
Table 6: Classification performance average of four ML classifiers and neural networks with testing images from the 3D dataset.
|
Classifier |
Precision |
Recall |
F1-score |
Accuracy |
AUC |
|
SVM |
0.97 |
0.98 |
0.97 |
97.33% |
0.9871 |
|
Random Forest |
0.99 |
0.99 |
0.99 |
98.67% |
0.9925 |
|
LogReg |
0.99 |
0.99 |
0.99 |
98.67% |
0.9684 |
|
K-NN |
0.91 |
0.91 |
0.91 |
90.67% |
0.9867 |
|
NNs |
0.99 |
0.99 |
0.99 |
98.67% |
0.9727 |
As in the 2D dataset, neural networks performed well in identifying PS compatible patients, although in this case surpassed by RF and LogReg. In comparison with the 2D approach, there was an important improvement in the models AUCs, specially in random forest and k-nearest neighbors (Figure 10). Overall, even though all images were processed as a vector of pixels, three-dimensional inputs presented higher performances for accuracy, precision, recall and F1-score metrics in all models (Figure 11).
Figure 11 states that there is additional information that can be obtained from the depth dimension given by 3D images, and therefore, that a single image may place excessive emphasis on a specific region [57-58].
The world population is aging, and we are witnessing an increase in chronic and neurodegenerative disorders [4-6]. PET imaging of DAT is a useful diagnostic method for tracking dopaminergic neurodegeneration in the striatum and ventral area of the SNpc in movement disorders [1-3]. Previous studies demonstrated that PET imaging with [18F]PR04.MZ provides an excellent image quality with high sensitivity for evaluation of patients with movement disorders [37-42].
In this article, we present three main contributions. First, we select and develop five different models that aim to classify a [18F]PR04.MZ PET 2D and 3D scans. Then, we conduct separate comparisons of the models’ performance using various metrics that support the evaluation of the models. And finally, we compare the performance of the models across both 2D and 3D approaches. All models could discriminate between PS and SWEDDs with high accuracy, and precision, similar to the expert clinician's performance. However, the neuronal network model showed the highest performance (across both 2D and 3D approaches), with a 97% agreement with the clinical diagnosis. As a whole, our work provides valuable insights about using machine learning tools to discriminate SWEDDs and PS patients, therefore facilitating less experienced neurologists in evaluating the scans in the clinical routine.
Other studies apply machine learning image-based techniques for the classification of neuroimage data. To facilitate a comparison with the achievements presented in this article, Table S2 compiles the relevant information from these studies, i.e. dataset size, radiotracer used, ML classifier employed and best results obtained. Under the baseline that all of the studies included in Table S2 can be compared by the fact that they all used Digital Imaging and Communications in Medicine (DICOM) as an input to train related ML classifiers mainly to discriminate patients with dopaminergic deficit versus SWEDD, our study presents a competitive proposal for the use of AI for diagnostic purposes.
Given its promising potential as a novel radiotracer, [18F]PR04.MZ PET scans are expected to be extensively distributed in clinical centers. Our results demonstrate that machine learning models can effectively discriminate PET scans with an accuracy exceeding 98%. Therefore, these models will serve as tools to aid in the analysis of neuroimages using this particular tracer.
Regarding the results indicating that all models perform better with 3D images as input, it supports the fact that brain structure is complex and multidimensional data may reflect this more accurately. A study comparing multiple approaches with different dimensions for brain image auto-segmentation of MRIs exposes that 3D approaches achieve higher performance for limited training data compared to 2D and even 2.5D [66], nevertheless, it does use more computational memory.
In order to explore the use of publicly available tools for people without programming expertise, we compared our 2D dataset models’ performance with the trial version of the Google platform algorithm. The 129 labeled 2D images were uploaded and used to train a model on the platform algorithm, while the remaining 75 images were used for testing (Table S3). With an accuracy rate of 93.33% and a recall value of 0.94 (Table S3), the Google Cloud Vision API showed superior sensitivity and specificity averages compared to individual reports of clinicians. Consequently, we came to the conclusion that it is worth considering exploring cost-effective alternatives, such as business tools in the cloud, if you want to use ML for data classification tasks.
In machine learning, interpretable models correlate with the level of trust, given the traceability of its prediction or decision steps. Models with low interpretability but high accuracy are less explainable, known as black-box models. In the higher degree of explainable models, we found logistic regression, while neural networks tend to be more accurate but less explainable (Figure 12).
Our CAD approach performed well on neural networks models in both, two and three-dimensional datasets. However, interpretability is not included in the model. Some studies use techniques to address interpretability in neural networks, such as Local Interpretable Model Agnostic Explanations (LIME) [67], but covering this technique as a NN model’s approach is beyond the scope of this investigation. Nevertheless, models such as logistic regression and random forest also performed with high scores in all the evaluated indicators, becoming better alternatives than neural networks.
4.1 Limitations and future work lines
In this initial approach, the dataset size was small due to the restricted number of patients available for the study, as [18F]PR04.MZ PET scans are an emerging tool. As the dataset increases, future work lines could also incorporate augmentation techniques [68-69] in order to boost that growth.
It is also important to mention that the accuracy and other performance metrics reported are based on the standard of truth established by three specialists and the image specifications. Therefore, all models trained using this information as a reference will perform in accordance with the provided data. Since the CAD system developed in this study aimed to replicate this assessment, it may not be exempt from misdiagnosis. Currently, the definitive diagnosis of Parkinson's disease (PD) can only be confirmed through an autopsy, and it is worth noting that all subjects who participated in this study and exhibited PD-compatible symptoms are alive, implying that their diagnoses are based on PET image and clinical assessments.
As [18F]PR04.MZ PET scans become widely used as a DAT tracer, future work lines with larger datasets have the potential to deliver more robust and generalizable results. Additional data could be incorporated into this approach to train models that may allow differentiation of patients, i.e. movement disorders syndromes often referred to as PS. Studies that apply such multiclass classification techniques are opening novel lines of research and applications involving image and computational methods [70]. Within the context of [18F]PR04.MZ PET/CT images, a dataset containing a follow-up of the advance of symptoms and patient deterioration could also allow stage classification.
Computational advancements such as artificial intelligence in medical decision-making have faced ethical debates and raised important questions about the role of AI in healthcare. One key consideration is the extent to which artificial intelligence will influence medical decisions. Additionally, the question arises whether the increased accuracy achieved by machine learning tools contributes in giving greater trust in their use.
Our study demonstrated 97% agreement between the diagnoses made by consensus expert assessment and ML techniques. Even though our dataset is small, this finding suggests that machine learning has the potential to serve as a valuable diagnostic tool and integrate into routine clinical workflow.
Although neural networks are powerful classification tools, often hard to beat, our research shows that other pattern recognition algorithms, i.e, random forest and logistic regression, can also be interpretable alternatives for image-based medical classification tasks, as long as it is applied to 3D image databases. Thus, the adoption of such models in standardized clinical reports has the potential to substantially reduce the time required for patient analysis and improve diagnostic certainty.
Interested parties may inquire about access to the dataset directly from any of the authors, in accordance with the data sharing policies and procedures established for this study.
Acknowledgements: We would like to thank Phillipe Salles and Paula Saffie Awad (all CETRAM) for their clinical inputs and reviewing the manuscript.
Funding: This research has received financial support from ANID, Chile (grant Fondecyt regular 1220908).
Conflicts of Interest: The authors declare to have no conflict of interest.
- Berg, D., Borghammer, P., Fereshtehnejad, S.M., Heinzel, S., Horsager, J., Schaeffer, E. and Postuma, R.B., 2021. Prodromal Parkinson disease subtypes—key to understanding heterogeneity. Nature Reviews Neurology, 17(6), pp.349-361.
- Oxtoby, N.P., Leyland, L.A., Aksman, L.M., Thomas, G.E., Bunting, E.L., Wijeratne, P.A., Young, A.L., Zarkali, A., Tan, M.M., Bremner, F.D. and Keane, P.A., 2021. Sequence of clinical and neurodegeneration events in Parkinson’s disease progression. Brain, 144(3), pp.975-988.
- Surmeier, D.J., 2018. Determinants of dopaminergic neuron loss in Parkinson's disease. The FEBS journal, 285(19), pp.3657-3668.
- Leiva, A.M., Martínez-Sanguinetti, M.A., Troncoso-Pantoja, C., Nazar, G., Petermann-Rocha, F. and Celis-Morales, C., 2019. Chile lidera el ranking latinoamericano de prevalencia de enfermedad de Parkinson. Revista médica de Chile, 147(4), pp.535-536.
- León, J.B., 2018. Epidemiología de la enfermedad de Parkinson en España y su contextualización mundial. Revista de neurología, 66(4), pp.125-134.
- Dorsey, E.R., Elbaz, A., Nichols, E., Abd-Allah, F., Abdelalim, A., Adsuar, J.C., Ansha, M.G., Brayne, C., Choi, J.Y., Collado-Mateo, D. and Dahodwala, N., 2018. GBD 2016 Parkinson’s Disease Collaborators. Global, regional, and national burden of Parkinson’s disease, 1990–2016: a systematic analysis for the Global Burden of Disease Study 2016. Lancet Neurol, 17(11), pp.939-953.
- Marsili, L., Rizzo, G. and Colosimo, C., 2018. Diagnostic criteria for Parkinson’s disease: from James Parkinson to the concept of prodromal disease. Frontiers in neurology, 9, p.156.
- Postuma, R.B., Berg, D., Stern, M., Poewe, W., Olanow, C.W., Oertel, W., Obeso, J., Marek, K., Litvan, I., Lang, A.E. and Halliday, G., 2015. MDS clinical diagnostic criteria for Parkinson's disease. Movement disorders, 30(12), pp.1591-1601.
- Postuma, R.B., Poewe, W., Litvan, I., Lewis, S., Lang, A.E., Halliday, G., Goetz, C.G., Chan, P., Slow, E., Seppi, K. and Schaffer, E., 2018. Validation of the MDS clinical diagnostic criteria for Parkinson's disease. Movement Disorders, 33(10), pp.1601-1608.
- Heinzel, S., Berg, D., Gasser, T., Chen, H., Yao, C. and Postuma, R.B., 2019. MDS Task Force on the Definition of Parkinson’s Disease. Update of the MDS research criteria for prodromal Parkinson’s disease. Mov Disord, 34(10), pp.1464-1470.
- Yue, X., Li, H., Yan, H., Zhang, P., Chang, L. and Li, T., 2016. Risk of Parkinson disease in diabetes mellitus: an updated meta-analysis of population-based cohort studies. Medicine, 95(18).
- Yang, Y.W., Hsieh, T.F., Li, C.I., Liu, C.S., Lin, W.Y., Chiang, J.H., Li, T.C. and Lin, C.C., 2017. Increased risk of Parkinson disease with diabetes mellitus in a population-based study. Medicine, 96(3).
- De Pablo-Fernandez, E., Goldacre, R., Pakpoor, J., Noyce, A.J. and Warner, T.T., 2018. Association between diabetes and subsequent Parkinson disease: a record-linkage cohort study. Neurology, 91(2), pp.e139-e142.
- Darweesh, S.K., Wolters, F.J., Postuma, R.B., Stricker, B.H., Hofman, A., Koudstaal, P.J., Ikram, M.K. and Ikram, M.A., 2017. Association between poor cognitive functioning and risk of incident parkinsonism: the Rotterdam study. JAMA neurology, 74(12), pp.1431-1438.
- Schrag, A., Anastasiou, Z., Ambler, G., Noyce, A. and Walters, K., 2019. Predicting diagnosis of Parkinson's disease: a risk algorithm based on primary care presentations. Movement Disorders, 34(4), pp.480-486.
- Weintraub, D., Chahine, L.M., Hawkins, K.A., Siderowf, A., Eberly, S., Oakes, D., Seibyl, J., Stern, M.B., Marek, K., Jennings, D. and PARS Investigators, 2017. Cognition and the course of prodromal Parkinson's disease. Movement Disorders, 32(11), pp.1640-1645.
- Weisskopf, M.G., O'reilly, E., Chen, H., Schwarzschild, M.A. and Ascherio, A., 2007. Plasma urate and risk of Parkinson's disease. American journal of epidemiology, 166(5), pp.561-567.
- Gao, X., O'Reilly, É.J., Schwarzschild, M.A. and Ascherio, A., 2016. Prospective study of plasma urate and risk of Parkinson disease in men and women. Neurology, 86(6), pp.520-526.
- Chen, H., Mosley, T.H., Alonso, A. and Huang, X., 2009. Plasma urate and Parkinson's disease in the Atherosclerosis Risk in Communities (ARIC) study. American journal of epidemiology, 169(9), pp.1064-1069.
- Kobylecki, C.J., Nordestgaard, B.G. and Afzal, S., 2018. Plasma urate and risk of Parkinson's disease: a Mendelian randomization study. Annals of Neurology, 84(2), pp.178-190.
- Fang, X., Han, D., Cheng, Q., Zhang, P., Zhao, C., Min, J. and Wang, F., 2018. Association of levels of physical activity with risk of Parkinson disease: a systematic review and meta-analysis. JAMA network open, 1(5), pp.e182421-e182421.
- Yang, F., Trolle Lagerros, Y., Bellocco, R., Adami, H.O., Fang, F., Pedersen, N.L. and Wirdefeldt, K., 2015. Physical activity and risk of Parkinson’s disease in the Swedish National March Cohort. Brain, 138(2), pp.269-275.
- Logroscino, G., Sesso, H.D., Paffenbarger, R.S. and Lee, I.M., 2006. Physical activity and risk of Parkinson’s disease: a prospective cohort study. Journal of Neurology, Neurosurgery & Psychiatry, 77(12), pp.1318-1322.
- Sasco, A.J., Paffenbarger, R.S., Gendre, I. and Wing, A.L., 1992. The role of physical exercise in the occurrence of Parkinson's disease. Archives of neurology, 49(4), pp.360-365.
- Xu, Q., Park, Y., Huang, X., Hollenbeck, A., Blair, A., Schatzkin, A. and Chen, H., 2010. Physical activities and future risk of Parkinson disease. Neurology, 75(4), pp.341-348.
- Chen, H., Zhang, S.M., Schwarzschild, M.A., Hernan, M.A. and Ascherio, A., 2005. Physical activity and the risk of Parkinson disease. Neurology, 64(4), pp.664-669.
- Sääksjärvi, K., Knekt, P., Männistö, S., Lyytinen, J., Jääskeläinen, T., Kanerva, N. and Heliövaara, M., 2014. Reduced risk of Parkinson’s disease associated with lower body mass index and heavy leisure-time physical activity. European journal of epidemiology, 29, pp.285-292.
- Thacker, E.L., Chen, H., Patel, A.V., McCullough, M.L., Calle, E.E., Thun, M.J., Schwarzschild, M.A. and Ascherio, A., 2008. Recreational physical activity and risk of Parkinson's disease. Movement disorders, 23(1), pp.69-74.
- Barber, T.R., Klein, J.C., Mackay, C.E. and Hu, M.T., 2017. Neuroimaging in pre-motor Parkinson's disease. NeuroImage: Clinical, 15, pp.215-227.
- King, A.E., Mintz, J. and Royall, D.R., 2011. Meta‐analysis of 123I‐MIBG cardiac scintigraphy for the diagnosis of Lewy body–related disorders. Movement disorders, 26(7), pp.1218-1224.
- Wen, M.C., Heng, H.S., Hsu, J.L., Xu, Z., Liew, G.M., Au, W.L., Chan, L.L., Tan, L.C. and Tan, E.K., 2017. Structural connectome alterations in prodromal and de novo Parkinson's disease patients. Parkinsonism & Related Disorders, 45, pp.21-27.
- Dayan, E. and Browner, N., Alterations in striato-thalamo-pallidal intrinsic functional connectivity as a prodrome of Parkinson’s disease. Neuroimage Clin. 2017; 16: 313–8.
- Knudsen, K., Fedorova, T.D., Hansen, A.K., Sommerauer, M., Otto, M., Svendsen, K.B., Nahimi, A., Stokholm, M.G., Pavese, N., Beier, C.P. and Brooks, D.J., 2018. In-vivo staging of pathology in REM sleep behaviour disorder: a multimodality imaging case-control study. The Lancet Neurology, 17(7), pp.618-628.
- Saeed, U., Compagnone, J., Aviv, R.I., Strafella, A.P., Black, S.E., Lang, A.E. and Masellis, M., 2017. Imaging biomarkers in Parkinson’s disease and Parkinsonian syndromes: current and emerging concepts. Translational neurodegeneration, 6(1), pp.1-25.
- Ravina, B., Eidelberg, D., Ahlskog, J.E., Albin, R.L., Brooks, D.J., Carbon, M., Dhawan, V., Feigin, A., Fahn, S., Guttman, M. and Gwinn-Hardy, K., 2005. The role of radiotracer imaging in Parkinson disease. Neurology, 64(2), pp.208-215.
- Saeed, U., Lang, A.E. and Masellis, M., 2020. Neuroimaging advances in Parkinson's disease and atypical Parkinsonian syndromes. Frontiers in Neurology, 11, p.572976.
- Riss, P.J., Debus, F., Hummerich, R., Schmidt, U., Schloss, P., Lueddens, H. and Roesch, F., 2009. Ex vivo and in vivo evaluation of [18F] PR04. MZ in rodents: a selective dopamine transporter imaging agent. ChemMedChem: Chemistry Enabling Drug Discovery, 4(9), pp.1480-1487.
- Riss, P.J. and Roesch, F., 2009. Efficient microwave-assisted direct radiosynthesis of [18F] PR04. MZ and [18F] LBT999: selective dopamine transporter ligands for quantitative molecular imaging by means of PET. Bioorganic & medicinal chemistry, 17(22), pp.7630-7634.
- Juri, C., Chana, P., Kramer, V., Pruzzo, R., Amaral, H., Riss, P.J. and Rösch, F., 2015. Imaging nigrostriatal dopaminergic deficit in holmes tremor with 18F-PR04. MZ-PET/CT. Clinical Nuclear Medicine, 40(9), pp.740-741.
- Kramer, V., Juri, C., Riss, P.J., Pruzzo, R., Soza-Ried, C., Flores, J., Hurtado, A., Rösch, F., Chana-Cuevas, P. and Amaral, H., 2020. Pharmacokinetic evaluation of [18 F] PR04. MZ for PET/CT imaging and quantification of dopamine transporters in the human brain. European journal of nuclear medicine and molecular imaging, 47, pp.1927-1937.
- Juri, C., Kramer, V., Riss, P.J., Soza-Ried, C., Haeger, A., Pruzzo, R., Rösch, F., Amaral, H. and Chana-Cuevas, P., 2021. [18F] PR04. MZ PET/CT imaging for evaluation of nigrostriatal neuron integrity in patients with Parkinson disease. Clinical Nuclear Medicine, 46(2), p.119.
- Lehnert, W., Riss, P.J., Hurtado de Mendoza, A., Lopez, S., Fernandez, G., Ilheu, M., Amaral, H. and Kramer, V., 2022. Whole-body biodistribution and radiation dosimetry of [18 F] PR04. MZ: a new PET radiotracer for clinical management of patients with movement disorders. EJNMMI research, 12, pp.1-9.
- Suzuki, K., 2017. Overview of deep learning in medical imaging. Radiological physics and technology, 10(3), pp.257-273.
- Prashanth, R. and Roy, S.D., 2018. Early detection of Parkinson’s disease through patient questionnaire and predictive modelling. International journal of medical informatics, 119, pp.75-87.
- Latif, J., Xiao, C., Imran, A. and Tu, S., 2019, January. Medical imaging using machine learning and deep learning algorithms: a review. In 2019 2nd International conference on computing, mathematics and engineering technologies (iCoMET) (pp. 1-5). IEEE.
- Khachnaoui, H., Mabrouk, R. and Khlifa, N., 2020. Machine learning and deep learning for clinical data and PET/SPECT imaging in Parkinson's disease: a review. IET Image Processing, 14(16), pp.4013-4026.
- Mei, J., Desrosiers, C. and Frasnelli, J., 2021. Machine learning for the diagnosis of Parkinson's disease: a review of literature. Frontiers in aging neuroscience, 13, p.633752.
- Tăuţan, A.M., Ionescu, B. and Santarnecchi, E., 2021. Artificial intelligence in neurodegenerative diseases: A review of available tools with a focus on machine learning techniques. Artificial Intelligence in Medicine, 117, p.102081.
- Muschelli, J., 2019. Recommendations for processing head CT data. Frontiers in neuroinformatics, 13, p.61.
- Vapnik, V., 1999. The nature of statistical learning theory. Springer science & business media.
- Illán, I., Górriz, J., Ramírez, J., Segovia, F., Jiménez‐Hoyuela, J. and Ortega Lozano, S.J., 2012. Automatic assistance to Parkinsonˈs disease diagnosis in DaTSCAN SPECT imaging. Medical physics, 39(10), pp.5971-5980.
- Oliveira, F.P., Faria, D.B., Costa, D.C., Castelo-Branco, M. and Tavares, J.M.R., 2018. Extraction, selection and comparison of features for an effective automated computer-aided diagnosis of Parkinson’s disease based on [123 I] FP-CIT SPECT images. European journal of nuclear medicine and molecular imaging, 45, pp.1052-1062.
- Ho, T.K., 1995, August. Random decision forests. In Proceedings of 3rd international conference on document analysis and recognition (Vol. 1, pp. 278-282). IEEE.
- Patra, A.K., Ray, R., Abdullah, A.A. and Dash, S.R., 2019, November. Prediction of Parkinson’s disease using Ensemble Machine Learning classification from acoustic analysis. In Journal of physics: conference series (Vol. 1372, No. 1, p. 012041). IOP Publishing.
- McHugh, M.L., 2012. Interrater reliability: the kappa statistic. Biochema Medica. 2012; 22 (3): 276–82.
- Parnetti, L., Gaetani, L., Eusebi, P., Paciotti, S., Hansson, O., El-Agnaf, O., Mollenhauer, B., Blennow, K. and Calabresi, P., 2019. CSF and blood biomarkers for Parkinson's disease. The Lancet Neurology, 18(6), pp.573-586.
- Tufail, A.B., Anwar, N., Othman, M.T.B., Ullah, I., Khan, R.A., Ma, Y.K., Adhikari, D., Rehman, A.U., Shafiq, M. and Hamam, H., 2022. Early-stage Alzheimer’s disease categorization using PET neuroimaging modality and convolutional neural networks in the 2D and 3D domains. Sensors, 22(12), p.4609.
- Wen, J., Thibeau-Sutre, E., Diaz-Melo, M., Samper-González, J., Routier, A., Bottani, S., Dormont, D., Durrleman, S., Burgos, N., Colliot, O. and Alzheimer's Disease Neuroimaging Initiative, 2020. Convolutional neural networks for classification of Alzheimer's disease: Overview and reproducible evaluation. Medical image analysis, 63, p.101694.
- Comte, V., Schmutz, H., Chardin, D., Orlhac, F., Darcourt, J. and Humbert, O., 2022. Development and validation of a radiomic model for the diagnosis of dopaminergic denervation on [18F] FDOPA PET/CT. European Journal of Nuclear Medicine and Molecular Imaging, 49(11), pp.3787-3796.
- Sun, X., Ge, J., Li, L., Zhang, Q., Lin, W., Chen, Y., Wu, P., Yang, L., Zuo, C. and Jiang, J., 2022. Use of deep learning-based radiomics to differentiate Parkinson’s disease patients from normal controls: a study based on [18F] FDG PET imaging. European Radiology, pp.1-11.
- Zhou, Y. and Tagare, H.D., 2021, December. Self-normalized Classification of Parkinson’s Disease DaTscan Images. In 2021 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) (pp. 1205-1212). IEEE.
- Ozsahin, I., Sekeroglu, B., Pwavodi, P.C. and Mok, G.S., 2020. High-accuracy Automated Diagnosis of Parkinson's Disease. Current Medical Imaging, 16(6), pp.688-694.
- Chakraborty, S., Aich, S. and Kim, H.C., 2020. Detection of Parkinson’s disease from 3T T1 weighted MRI scans using 3D convolutional neural network. Diagnostics, 10(6), p.402.
- Segovia, F., Górriz, J.M., Ramírez, J., Martínez-Murcia, F.J. and Castillo-Barnes, D., 2019. Assisted diagnosis of Parkinsonism based on the striatal morphology. International Journal of neural systems, 29(09), p.1950011.
- Wenzel, M., Milletari, F., Krüger, J., Lange, C., Schenk, M., Apostolova, I., Klutmann, S., Ehrenburg, M. and Buchert, R., 2019. Automatic classification of dopamine transporter SPECT: deep convolutional neural networks can be trained to be robust with respect to variable image characteristics. European journal of nuclear medicine and molecular imaging, 46, pp.2800-2811.
- Avesta, A., Hossain, S., Lin, M., Aboian, M., Krumholz, H.M. and Aneja, S., 2023. Comparing 3D, 2.5 D, and 2D Approaches to Brain Image Auto-Segmentation. Bioengineering, 10(2), p.181.
- Magesh, P.R., Myloth, R.D. and Tom, R.J., 2020. An explainable machine learning model for early detection of Parkinson's disease using LIME on DaTSCAN imagery. Computers in Biology and Medicine, 126, p.104041.
- Alam, M.S., Wang, D. and Sowmya, A., 2021, November. Image data augmentation for improving performance of deep learning-based model in pathological lung segmentation. In 2021 Digital Image Computing: Techniques and Applications (DICTA) (pp. 1-5). IEEE.
- El Jiani, L. and El Filali, S., 2022, December. Overcome medical image data scarcity by data augmentation techniques: A review. In 2022 International Conference on Microelectronics (ICM) (pp. 21-24). IEEE.
- Martins, R., Oliveira, F., Moreira, F., Moreira, A.P., Abrunhosa, A., Januário, C. and Castelo-Branco, M., 2021. Automatic classification of idiopathic Parkinson’s disease and atypical Parkinsonian syndromes combining [11C] raclopride PET uptake and MRI grey matter morphometry. Journal of neural engineering, 18(4), p.046037.
No competing interests reported.