Comparison of Machine Learning Methods for the Detection of Focal Cortical Dysplasia Lesions: Decision Tree, Support Vector Machine and Artificial Neural Network

doi:10.21203/rs.3.rs-142897/v1

Download PDF

Research article

Comparison of Machine Learning Methods for the Detection of Focal Cortical Dysplasia Lesions: Decision Tree, Support Vector Machine and Artificial Neural Network

https://doi.org/10.21203/rs.3.rs-142897/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

Background: Accurate classification of focal cortical dysplasia (FCD) has been challenging due to the problematic visual detection in magnetic resonance imaging (MRI). Hence, recently, there has been a necessity for employing new techniques to solve the problem.

Methods: MRI data were collected from 58 participants (30 subjects with FCD type II and 28 normal subjects). Morphological and intensity-based characteristics were calculated for each cortical level and then the performance of the three classifiers: decision tree (DT), support vector machine (SVM) and artificial neural network (ANN) was evaluated.

Results: Metrics for evaluating classification methods, sensitivity, specificity and accuracy for the DT were 96.7%, 100% and 98.6%, respectively; It was 95%, 100% and 97.9% for the SVM and 96.7%, 100% and 98.6% for the ANN.

Conclusion: Comparison of the performance of the three classifications used in this study showed that all three have excellent performance in specificity, but in terms of classification sensitivity and accuracy, the artificial neural network method has worked better.

Nuclear Medicine & Medical Imaging

Artificial Intelligence and Machine Learning

Focal Cortical Dysplasia (FCD)

decision tree (DT)

support vector machine (SVM)

artificial neural network (ANN)

machine learning

Computer-Aided Diagnosis (CAD)

An ANN based model for detecting focal cortical dysplasia (FCD) from MRI images is proposed.
FCD detection accuracy of the proposed model is 98.6%.
The proposed method can be a valuable tool to improve the preoperative evaluation of patients with drug- resistant epilepsy.

Focal cortical dysplasia (FCD) is a malformation of cortical development, which was often characterized by cortical thickening and blurring of the white/gray matter junctions on magnetic resonance (MR) imaging [1]. FCD is most observed in patients who have drug-resistant epilepsy and are candidates for surgery [2]. With technological advancements in MRI, FCD diagnosis has been revolutionized, and epilepsy surgery seems to be more successful by the better detection of the lesion locations. However, many cases are still being reported to have normal MRI despite the malformation [3]. Therefore, finding new diagnosis methods with better performance than human interpretations is needed.

Recently, computer-aided diagnosis (CAD) systems have attracted much attention from medical imaging and diagnostic radiology researchers. This technology aims to interpret medical images faster and without dependence on the experience of radiologists in diagnosing brain disorders. The CAD system facilitates the processing of medical images using pattern recognition techniques [4].

In a CAD system, several steps are performed: First, the MRI image is provided as the input for the CAD system and as training samples. Then, preprocessing is performed to remove the samples not related to the diagnosis. In the segmentation step, images are grouped into regions. Feature extraction on images is performed after that. Finally, based on the extracted feature matrix and subject labels, two or more classes are defined in the classification stage.

In recent years, researchers have developed various machine learning systems to study epilepsy. El Azami et al. [5] developed a machine learning system based on SVM classification to detect epileptogenic abnormalities in patients who have epilepsy. They applied the system to 11 patients with 13 FCD lesions (3 MRI-positive and 10 MRI-negative) and 77 healthy subjects. The SVM classifier detected all lesions in patients with positive MRI, while 7 out of 10 patients with negative MRI were correctly diagnosed.

Gill et al. [6] proposed an automated FCD lesions detection algorithm using surface-based morphometric features extracting from T1 images. For each image, morphological, intensity, and gradient features were extracted at both cortical and subcortical surfaces. Using 5-fold cross-validation, the decision tree classification was performed on 41 patients with FCD lesions and 38 healthy subjects. The results showed a sensitivity of 83% and a specificity of 93%.

Besson et al. [7] proposed a method for identifying FCD lesions on T1-weighted MRI images based on surface features including cortical thickness, curvature, inner cortical surface depth, gradient magnitude at the white matter / gray matter interface, and cortical signal intensity. Automatic detection was performed by the neural network. This method was tested on 19 patients with FCD and was detected in 89% of cases.

In a study conducted by Adler et al. [8], surface-based morphometry and neural network methods were used to identify FCD lesions in a group of children. The neural network classification was trained using data from 22 patients with FCD lesions and 28 healthy subjects. FCD was identified in this group of children with a sensitivity of 73%.

Also, Jin et al. [9] employed the methods used in Adler's study for many patients with confirmed FCD lesions, including children and adults. The results showed a sensitivity of 73.7% and a specificity of 90%.

In this study, we aimed to find the best method for classifying images of patients with FCD lesions from normal images. In the first step, we obtained structural images, preprocessed and processed the images using Freesurfer software, and extracted surface-based features from each Desikan-Killiany Atlas region [10] (34 per hemisphere). Then, we classified the images using the common classifier and compared the results.

Subjests

A. Study group

The set of images for evaluating machine learning models includes MRI images of 28 healthy subjects from the ADNI database and 30 patients with FCD confirmed by the Milan Epilepsy Surgery Center. The patients and control demographics are summarized in Table 1.

MRI protocol

We used two different scanners in this study, according to a study by Jin et al. [9] who reported that scanners do not affect classification performance in FCD patients.

For control group, 21 images were acquired on a Siemens 3.0 T scanner(Munich, Germany): a 3D T1-MPRAGE sequence (TR = 2,300 ms, TE = 2.98 ms, slice thickness = 1 mm) and a 3D T2-FLAIR (TR = 4,800 ms, TE = 343 ms, slice thickness = 1 mm), and 7 subjects underwent a Siemens 1.5 T scanner: a 3D T1-MPRAGE sequence (TR = 2,300 ms, TE = 3.05 ms, slice thickness = 1.2 mm) and a 3D T2-FLAIR (TR = 6,000 ms, TE = 418 ms, slice thickness = 0.9 mm).

The patient group underwent MRI scans on a Phillips 1.5 T scanner (Philips Healthcare; Best, The Netherlands) with the following parameters: a 3D-volume FFE T1-weighted sequence (TR = 7.3 ms, TE = 3.3 ms, slice thickness = 0.9 mm) and a 3D T2-FLAIR (TR = 140 ms, TE = 11 ms, slice thickness = 0.9 mm).

Image preparation

A. Images Processing

In the preprocessing stage, the image quality is improved for subsequent processing to make it easier and faster to execute. During this stage, depending on the type of processing, several steps are performed, including removing the skull, removing the noise and bias field correction. In this study, Freesurfer software (version 6.0; Athinoula A. Martinos Center for Biomedical Imaging at Massachusetts General Hospital, Boston USA) [11–13] was used for pre-processing and processing stages. Cortical reconstruction performed by the Recon-all pipeline had several steps as follows: transferring raw image voxels to the isotropic space, normalize the images for bias field correction, skull stripping, automatic subcortical segmentation, white matter segmentation, and determining the WM/GM interface. Then, two experienced neurologists labeled FCD lesions on reconstructed images using ITK-SNAP software (version 3.6.0; Penn Image Computing and Science Laboratory - PICSL -, and Scientific Computing and Imaging Institute - SCI, USA).

B. Feature extraction

At this step, from the images processed by Freesurfer software, 816 morphological and intensity-based features were extracted for all subjects. These features include cortical thickness, Gaussian curvature, mean curvature, intensity contrast, and two new features that have not been explored before, the folding index and the curvature index. Information about the extracted features is shown in Table 2.

A total of 408 morphological and 408 intensity-based features were obtained. All features were normalized using z-score normalization [20].

C. Classification

For accurately automatic detection of patients from healthy subjects, three classifiers were trained separately: decision tree (DT), support vector machine (SVM), and artificial neural networks (ANN) classifications. Subsequently, their results were compared to one another.

1. Decision Tree (DT)

DT is one of the powerful and most common methods of classification and prediction. It is a graph or tree-like model consisting of nodes and branches. DT is used to create a learning model to predict the variable values by learning simple rules from training data results. In this study, we compared different k-fold cross-validations of the DT classification.

2. Support Vector Machine (SVM)

SVM is a supervised learning model for data analysis algorithms. It is used for classification and regression analyses. For classification, the SVM training algorithm divides data into different classes by a hyperplane [21]. The linear, quadratic, cubic, and Gaussian SVM models were employed to classify images in the present study.

3. Artificial Neural Networks(ANN)

ANN is an idea for information processing that is inspired by the neural system and, processes information like the brain. [22]. ANN is a simplification of the complex neural network of neurons. The structure of an ANN algorithm consists of several layers of neurons. Each layer has separate data receiving, processing, and passing to the next layer. In a multilayer ANN, neurons, similar to synaptic connections in the human brain, are transferred to all neurons in the next layer with different weights [23]. In this study, the neural network toolbox of Matlab was used for classification. Input images of each class are divided into two groups: training and testing images. 70% of each class was selected as a training sample, 15% as a test and, 15% as a validation.

The overall procedure is shown in Fig. 1.

In the method section, the process of classifying images was examined using three standard classifiers. To compare these three methods, the results of each classification system are reviewed and reported.

In the DT classification method, 48 images of both FCD and normal groups were used. To evaluate the algorithm in different conditions, ten different k was applied in k-fold cross-validations. The parameters of sensitivity, specificity, and accuracy were also calculated (Fig. 2).

To assess the SVM, this classifier was utilized to classify the 48 MR images with linear, quadratic, cubic, and gaussian kernel functions.In this method, ten different k was applied in k-fold cross-validations for each kernel. Figure 3 illustrates the mean parameters of sensitivity, specificity, and accuracy of these k-fold validations for each kernel functions.

In this classification method, the feed-forward neural network was applied to detect images with FCD lesions. It included the input, output, and hidden layers. After specifying the number of neurons in each layer, the ANN classification was evaluated on the extracted features.

For 48 images and 30 iterations for each, the mean values of sensitivity, specificity, and accuracy of the ANN classifier were obtained 96.7%, 100%, and 98.6%, respectively.

Figure 4 compares the three decision classification methods, SVM and ANN. The results show that the ANN algorithm performs better than the other two methods.

For the final validation of the classification system, ten images with visual undetectable FCD lesion with the same neural network structure used for MRI-positive images were evaluated. The result showed a sensitivity of 91.3%.

Classification of FCD lesions based on cortical surface features helps accurate identification of patients from healthy controls. In this study, three standard classification methods for the automatic detection of FCD lesions are compared.

In Gill et al. study, a two-stage algorithm was designed using a DT classifier. They evaluated the approach using 5-fold cross-validation in FCD patients and healthy controls. We used ten different folds to classify the images using DT. The result of this study indicated that the sensitivity, specificity, and accuracy had the highest values in k = 15. The same values were acquired for the other k > 15.

In El Azami et al. study, a CAD system was developed. It consisted of an OC-SVM classifier allowing the combination of features. They used a Gaussian kernel to ensure data separation. In our study, four kernels were used for SVM classification: linear, quadratic, cubic, and Gaussian. An average of 30 iterations per kernel run. The result of the simulation revealed that the cubic kernel had the best performance for the detection of FCD lesions.

Similar to Besson et al., Adler et al., and Jin et al., surface-based features were evaluated using ANN classification in this study. The average statistical parameters were obtained after 30 iterations. The improved values of the parameters of interest, i.e. sensitivity, specificity, and accuracy have obtained in this study. This significant improvement can be attributed to new features and increased iteration in classification training and testing. Therefore, it seems that the use of surface-based features, especially features related to curvature and folding, will have a significant impact on the identification of lesions.Table 3 shows the performance comparison between our proposed methods and the techniques used in other studies.

The present study aimed to use the quantitative features of the MR images and classification methods to detect FCD lesions. There are several essential innovations in this study: First of all, FLAIR images were used to improve the segmentation of brain images in Freesurfer software. Second, two new features have been used that have not been used before, and third, three methods were compared to find the best classification method. Of the three classification methods investigated in the present study, the ANN showed the best performance. Therefore, this method was used to analyze negative images with excellent accuracy of 91.3%.

In this study, three different classification algorithms, including decision tree, SVM, and ANN, were applied to detect FCD lesions on MR images. Then, their performances were compared to one another. The results showed that the ANN algorithm had higher sensitivity, specificity, and accuracy than the other two methods. Therefore, it is suggested that the ANN method can be used as the optimal classification method in a computer-aided FCD lesion diagnosis system.

FCD: Focal Cortical Dysplasia

DT: Decision Tree

SVM: Support Vector Machine

ANN: Artificial Neural Network

CAD: Computer-Aided Diagnosis

TE: Time to Echo

TR: Repetition Time

Ethics approval and consent to participate

This article is taken from the senior thesis and involving human participants were reviewed and approved by [Research Ethics Committee of School of Medicine- Mashhad University of Medical Sciences]. Since we used pre-prepared data, there is no need for consent statement.

Ethics code: IR.MUMS.fm.REC.1396.506

Consent for publication

Not applicable

Availability of data and materials

The datasets generated and/or analysed during the current study are not publicly available due [Not having permission to sharing from the institution where the data was received] but are available from the corresponding author on reasonable request.

Competing interests

The authors declare no conflict of interest.

Funding

No funding was obtained for this study.

Authors' contributions

HZ Study idea. ZG Data acquisition. ZG Manuscript preparation and figure preparation. ZG and SZ analysis and interpretation of data. HZ manuscript editing. All authors have read and approved the manuscript.

Acknowledgements

The authors would like to thank the Vice-Chancellor for Research Department at Mashhad University of Medical Sciences to support this project.

Antel SB, Collins DL, Bernasconi N, Andermann F, Shinghal R, Kearney RE, Arnold DL, Bernasconi A. Automated detection of focal cortical dysplasia lesions using computational models of their MRI characteristics and texture analysis. Neuroimage. 2003 Aug 1;19(4):1748-59.
Sisodiya SM. Surgery for malformations of cortical development causing epilepsy. Brain. 2000 Jun 1;123(6):1075-91.
Ahmed B, Brodley CE, Blackmon KE, Kuzniecky R, Barash G, Carlson C, Quinn BT, Doyle W, French J, Devinsky O, Thesen T. Cortical feature analysis and machine learning improves detection of “MRI-negative” focal cortical dysplasia. Epilepsy & Behavior. 2015 Jul 1;48:21-8.
Lazli L, Boukadoum M, Mohamed OA. A Survey on Computer-Aided Diagnosis of Brain Disorders through MRI Based on Machine Learning and Data Mining Methodologies with an Emphasis on Alzheimer Disease Diagnosis and the Contribution of the Multimodal Fusion. Applied Sciences. 2020 Jan;10(5):1894.
El Azami M, Hammers A, Jung J, Costes N, Bouet R, Lartizien C. Detection of lesions underlying intractable epilepsy on T1-weighted MRI as an outlier detection problem. PloS one. 2016;11(9).
Gill RS, Hong SJ, Fadaie F, Caldairou B, Bernhardt B, Bernasconi N, Bernasconi A. Automated detection of epileptogenic cortical malformations using multimodal MRI. InDeep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support 2017 Sep 14 (pp. 349-356). Springer, Cham.
Besson P, Colliot O, Evans A, Bernasconi A. Automatic detection of subtle focal cortical dysplasia using surface-based features on MRI. In2008 5th IEEE International Symposium on Biomedical Imaging: From Nano to Macro 2008 May 14 (pp. 1633-1636). IEEE.
Adler S, Wagstyl K, Gunny R, Ronan L, Carmichael D, Cross JH, Fletcher PC, Baldeweg T. Novel surface features for automated detection of focal cortical dysplasias in paediatric epilepsy. NeuroImage: Clinical. 2017 Jan 1;14:18-27.
Jin B, Krishnan B, Adler S, Wagstyl K, Hu W, Jones S, Najm I, Alexopoulos A, Zhang K, Zhang J, Ding M. Automated detection of focal cortical dysplasia type II with surface‐based magnetic resonance imaging postprocessing and machine learning. Epilepsia. 2018 May;59(5):982-92.
Desikan RS, Ségonne F, Fischl B, Quinn BT, Dickerson BC, Blacker D, Buckner RL, Dale AM, Maguire RP, Hyman BT, Albert MS. An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest. Neuroimage. 2006 Jul 1;31(3):968-80.
Dale AM. Optimal experimental design for event‐related fMRI. Human brain mapping. 1999;8(2‐3):109-14.
Dale AM, Fischl B, Sereno MI. Cortical surface-based analysis: I. Segmentation and surface reconstruction. Neuroimage. 1999 Feb 1;9(2):179-94.
Fischl B, Dale AM. Measuring the thickness of the human cerebral cortex from magnetic resonance images. Proceedings of the National Academy of Sciences. 2000 Sep 26;97(20):11050-5.
Dale AM, Fischl B, Sereno MI. Cortical surface-based analysis: I. Segmentation and surface reconstruction. Neuroimage. 1999 Feb 1;9(2):179-94.
Han X, Jovicich J, Salat D, van der Kouwe A, Quinn B, Czanner S, Busa E, Pacheco J, Albert M, Killiany R, Maguire P. Reliability of MRI-derived measurements of human cerebral cortical thickness: the effects of field strength, scanner upgrade and manufacturer. Neuroimage. 2006 Aug 1;32(1):180-94.
Pienaar R, Fischl B, Caviness V, Makris N, Grant PE. A methodology for analyzing curvature in the developing brain from preterm to adult. International journal of imaging systems and technology. 2008;18(1):42-68.
Ronan L, Pienaar R, Williams G, Bullmore ED, Crow TJ, Roberts N, Jones PB, Suckling J, Fletcher PC. Intrinsic curvature: a marker of millimeter-scale tangential cortico-cortical connectivity?. International journal of neural systems. 2011 Oct;21(05):351-66.
Van Essen DC, Drury HA. Structural and functional analyses of human cerebral cortex using a surface-based atlas. Journal of Neuroscience. 1997 Sep 15;17(18):7079-102.
Salat DH, Lee SY, Van der Kouwe AJ, Greve DN, Fischl B, Rosas HD. Age-associated alterations in cortical gray and white matter signal intensity and gray to white matter contrast. Neuroimage. 2009 Oct 15;48(1):21-8.
McLeod, S. A. (2019, May 17). Z-score: definition, calculation and interpretation. Retrieved from https://www.simplypsychology.org/zscore.html
Siuly S, Zhang Y. Medical big data: neurological diseases diagnosis through medical data analysis. Data Science and Engineering. 2016 Jun 1;1(2):54-64.
Hykin, (1994) Neural Networks: A comperhensive Foundation, Macmillan, New York.
Basheer, I. A., & Hajmeer, M. (2000). Artificial neural networks: fundamentals, computing, design, and application. Journal of microbiological methods, 43(1), 3-31.

Due to technical limitations, tables are only available as a download in the Supplemental Files section.

Tables.docx

Download PDF

Review #2 received at journal
14 Mar, 2021
Editorial decision: Major revision
14 Mar, 2021
Reviewer #2 agreed at journal
09 Jan, 2021
Reviewer #1 agreed at journal
07 Jan, 2021
Review #1 received at journal
07 Jan, 2021
Editor assigned by journal
06 Jan, 2021
Reviewers invited by journal
06 Jan, 2021
Submission checks completed at journal
06 Jan, 2021
Editor invited by journal
06 Jan, 2021

You are reading this latest preprint version

Comparison of Machine Learning Methods for the Detection of Focal Cortical Dysplasia Lesions: Decision Tree, Support Vector Machine and Artificial Neural Network

Status:

Version 1

Abstract

Figures

Highlights

Background

Methods

Subjests

A. Study group

MRI protocol

Image preparation

A. Images Processing

B. Feature extraction

C. Classification

1. Decision Tree (DT)

2. Support Vector Machine (SVM)

3. Artificial Neural Networks(ANN)

Results

Discussion

Conclusion

Abbreviations

Declarations

Ethics approval and consent to participate

Consent for publication

Availability of data and materials

Competing interests

Funding

Authors' contributions

Acknowledgements

References

Tables

Supplementary Files

Status:

Version 1