Comparing NeuroQuant and volBrain Software for Automated Brain Analysis

Background and Purpose Automated brain volume analysis from MRI images, is gaining an important role in computer-aided diagnosis. This study compares the volumes of brain segments that measured by two automated brain analysis software: NeuroQuant and volBrain. The aim is to investigate the quantitative and qualitative reliability of automated brain analysis. Methods We compared same brain segment volume measurements, from 45 patients, that calculated with both NeuroQuant and volBrain software. The statistical two-tailed unpaired t-test method used to determine if there is a signicant difference on the same segment measurements performed from each software. Additionally, least square method used, to provide any evidence of linear correlation between them. In a case of extreme difference, manual volume measurement of a segment was performed, with JIM8 software. From the comparison of brain segments volumetric measurements made by NeuroQuant and volBrain, the differences for the Intracranial Cavity, Putamen, Thalamus and Amygdala considered extremely signicant, the Whole Brain, Cerebellum, Hippocampus considered not signicant, White Matter considered quite signicant. In most cases strong linear correlation between the two software measurements found.

1. Introduction 1.1 The role of automated brain volumetry Brain structure volumetry from MRI images, is becoming a new tool in the understanding the nature and evolution of atrophy related diseases [1]. It features the factor of accurate measurement in the world of clinical psychiatry and neurology, where diagnosis sometimes is based on relative self-reports and tests [2]. In the past, manual segmentation and manually calculated volumes of a brain segment produced results that believed to be accurate and reliable. But manual segmentation volume measurement, required investigator expertise in neuroanatomical structures' boundaries and it was so time-consuming, that its practice was limited [1], [3]. The automated brain volume analysis from MRI images does not involve any human expert, it is a user-friendly, fast modern method of computer-aided diagnosis (CAD) for follow-up and treatment of atrophy diseases. It aims to provide e ciency and reliability and will be the major player in the scienti c eld of neuroimaging research. These days there are many software packages available, that provide fast and easy brain segment volume measurements.

How it works
For a general brain volumetry from MRI images, most software packages use 3D T1 sequences of thin slices, without paramagnetic contrast enhancement. Preferably an inversion recovery (IR) pulse sequence (like GE's BRAVO) provides better contrast between white and gray matter. Then the images are compressed and processed either on site by installed software or remotely by the software provider where data send anonymously.
The process of brain analysis follows four main steps: (a) brain extraction from the skull and other bone structures, (b) tissue classi cation as gray matter (GM), white matter (WM) and cerebrospinal uid (CSF), (c) main structures recognition such as left, right hemisphere and cerebellum, (d) and nally minor subcortical structures recognition, seen in Image 1. Each step uses a combination of computer techniques and specially developed algorithms for segmentation and the contrast between tissues for volume measurements [1].
The output of a brain volumetry software process is a report (PDF le) where the results are presented. Volume measurements from the main and minor structures are presented in their absolute values expressed in cubic centimeters, and as a percentage of the intracranial volume (ICV). Some software also includes for each structure, a normal range of volume percentage according to the patient's sex and age. This range has been already calculated from each software manufacture in the period of initial trials with healthy individuals and it is kept as a database for reference [1]. Each software manufacture has several volumetric options available on its electronic platform, according to the areas of brain structures that a user might need.

The Reason For This Research
In our MRI lab (BIOIATRIKI Ampelokipon SA, Athens, Greece), we had access to brain volume analysis software NeuroQuant (NQ) provided by the Hellenic Academy of NeuroImmunology (HANI) for a limited time. NQ is created by CorTechs Labs Inc.
San Diego, CA USA) and has FDA and CE approval for clinical use. After the period was over, we tried to nd an alternative and more affordable brain volumetry software, so we started using volBrain (VB) which is free for a limited amount of analysis per day. VB is created by the cooperation of ITACA at Valencia University (Spain) and Pictura Research Group, University of Bordeaux (France), it developed for research purposes and does not hold any certi cation yet [2].
When a measurement of a material body is performed using different methods and technics, the results are expected to be different because of the procedures each method uses, but a difference over 10% is considered signi cant and is unexpected from a reliable method. Because of the differences in the software algorithms each developer used, we expected only minor differences in the brain segments volumes to occur. For typical reassertion reasons and for a small sample of patients, we performed a comparison from the brain analysis results that NQ and VB produce. To our surprise, signi cant differences were found in some structures' volumes. These differences raised questions about the reliability of automated brain segment analysis as a method and challenged the way the results from such analysis should be interpreted regarding atrophy determination. As we did not nd any other similar study comparing these two software packages, we performed our own investigation.

Methods And Materials
45 patients with known or suspicion for multiple sclerosis (MS) (17Male/28Female with ages from 16 to 70 years (44y average age)) performed MRI brain scans with GE Discovery 3.0T MR system for brain volume analysis licenced by HANI.
All patients were scanned using the same 3D T1 sequence, which was set in accordance with the NQ protocol (3D/GRE/FSPGR, FOV 25.6cm, Slice Thickness=1.2mm, TE/TR=min/5.7ms, TI=600ms, frequency/phase=192/192, Band Width 31.25 kHz). We have used the NQ software for volume measurements of brain structures related to MS and other neurodegenerative conditions; more speci cally we had access to "Multi Structure Report" and "General Report" from the reports' options of NQ intertnet platform. The initial DICOM images zipped and transmitted to NQ. Then the same images were also analysed by the VB software using "Volumetry Report" option from the VB internet platform. The initial DICOM images converted to NIFTI format, using ITK Snap software, before transmitted to VB for processing.
All exams were anonymized during processing, and the system's exam numbers were used as a key for identi cation in the comparison. The Scienti c Council and Bioethics Committee of BIOIATRIKI SA, Healthcare Provider Group, Athens, Greece, approved this research. All patients were informed consent for study participation, for those under 18 informed consent obtained from their parents or legally authorized representatives, and data handling procedures were according to GDPR protocols. All methods were carried out in accordance with relevant guidelines and regulations.
From the reports that the two software packages (NQ and VB) produced, we compare the volumes of some structures that we consider important, according to the pathology we usually meet in our laboratory and the needs of the volumetry processing: Intracranial Volume (ICV), is an important measurement in volumetry as it is used to express the percentage of every other structure in the brain. Hippocampus is associated with memory consolidation and decision making [4].
Cerebellum has key role in motor movement and balance control [5]. Thalamus has many roles from relaying sensory and motor signals to the regulation of consciousness and alertness [6]. Amygdala, named by the Greek word for almond, is related with neuropsychiatric diseases like depression, anger, sleep debt and others [7]. Whole Brain volume is considered the sum of white and gray matter and it is used as a measure of atrophy. Putamen is involved in learning and motor control [8]. Cerebral White Matter constitutes the network of nerve bres [9], its' volume is important factor in MS patients atrophy determination and other in ammatory diseases.
The segments' volume measurements from each software presumed as independent, so the statistical two-tailed unpaired t-test method was used to determine if there is a signi cant difference between the measurements of each software package on the same brain segment. The free online software Graphpad, used for the t-test analysis. The least squares method was used to investigate whether there is any correlation between NQ and VB measurements. Microsoft Excel software was used for linear regression coe cient R square calculation.

Results
Statistical differences were found as following:

The Segments With Signi cant Differences In Volume Measurements
The signi cant differences between the measurements in ICV, Putamen, Amygdala and Thalamus between the two software packages utilized in this, were not expected initially. Such differences raise the question of which software produces valid and reliable results. As there is no gold standard, a search in the bibliography has produces some indications regarding the real volume of different brain segments. In the bibliography studies healthy individuals participated. This investigation performed with MS patients. As brain volume reduces through age in faster rate in MS patients than healthy population, brain segments' volumes from this study are expected to be smaller than the average volume bibliography mentions.

Putamen volume
The average volume of the putamen [10] is 7.80±0.50 cm 3 . The NQ average was 10.41±1.78 cm 3 , which deviate 33.46% over from the bibliographic reference. On the other hand, the VB average was 7.46±1.17 cm 3 near the bibliographic reference.

Thalamus volume
The average size of the thalamus, which was calculated using the Stereology and FreeSurfer methods [11], was found 14.30 cm 3 . The NQ average was 13.58±2.73 cm 3 , close to the aforementioned bibliography, an expected comparison as NeuroQuant and FreeSurfer have shown good to excellent inter-method reliability in brain volumetric measurements [12]. VB average was 9.57±2.02 cm 3 which deviates 33.1% from the reference. Another study [13] based on MRI and volumetric measurements on anatomical sections, calculated the average thalamus volume at 11.22 cm 3 . Compering to the result of thalamus volume from this study, NQ had a deviation of 21.0% higher and VB had a deviation of 14.7% lower.

Amygdala volume
The average size of each amygdala [14] is 1.24±0.14 cm 3 which makes a total (left and right) volume of 2.48 cm 3 . NQ average was 3.25±0.66 cm 3 , a deviation of 30.1%, where VB average was 1.47±0.29 cm 3 which deviates 40.7% from the bibliography reference.
Amygdala is the brain segment were NQ and VB, had the greatest difference between them and they both had the greatest difference from the reference. To have our own opinion on the subject, we randomly selected ve patients and performed manually volume measurements of the right amygdala. Axial and coronal images of 2mm thickness, reformatted from the original 3D T1 sequence. An experienced neuroradiologist traced the anatomic boundaries of amygdala on the images and the volumes measured using JIM8 software. We measured the right amygdala volume from the two planes, and considered the average measurement, to minimize measurement errors. The data are presented at table 2. For these patients, the average measurement for the right amygdala was calculated 1.182 cm 3 . For the same area (right amygdala) NQ gave a volume estimation of 1.716 cm 3 , which deviates 45.2% from our measurement whereas VB gave a volume estimation of 0.786 cm 3 , which deviates 33.5% from our measurement. The deviation of our measurement with that from bibliography was 4.8% lower, an expected result regarding patient's disease. 6. Further Analysis NQ measurements were greater than VB measurements in all segments. This difference expressed as a percentage (of NQ greater than VB) is the following: Intracranial Cavity 8.73%, Whole Brain 1.81%, Cerebellum 2.82%, White Matter 7.10%, Amygdala 121.06%, Hippocampus 3.30%, Putamen 39.02%, and Thalamus 41.94%. This observation is also mentioned and in other studies that compered NQ with FreeSurfer and FIRST software packages [12], [15].
Graph 1 presents the graphical representation of data from the volumes calculated by NQ and VB for each brain segment.
The least-squares regression linear equation and squared correlation coe cient R2 are appear on each graph. This is an equation in the form of VB(NQ)=a(NQ)+b. According to the R 2 value, there is extremely strong correlation between ICV, Cerebellum, Whole brain measurements, strong correlation in Thalamus and White Matter measurements and a good correlation in Putamen, Hippocampus and Amygdala measurements. This means that for brain segments of ICV, Cerebellum, Whole Brain, Thalamus and White Matter it is safe to convert measurements taken from one software to measurements that the other software would give if it were used, just by applying the least-squares regression linear equation. It also means that both software NQ and VB can distinguish the relative smaller or bigger volume of a structure as it differs on each patient's anatomy.

Conclusion
The comparison of NQ and VB presented signi cant differences in some brain segment's measurements. Both software packages also had important deviation in the estimation of some segment's volumes relative to what the bibliography mentioned as expected values. This means that the absolute values of the measurements are not reliable in all cases.
The strong linear correlation in most measurements means that NQ and VB software packages can qualitatively detect the relative size of each brain segment. This detection by the means of different measurements through the time, if the volume of a structure remains the same or gets smaller or bigger, has a key role in the use of such software in terms of computed aided diagnosis and patient monitoring. So, automated brain volumetry software can be a good qualitatively tool in treatment of patients with atrophy related diseases.  Figure 1 If the graph is a straight line it means that each software gives a sense of the size of a segment, even if there are differences in the measurement. It means that there is a mathematical linear equation of the form f(x)= ax+b, where f(x) is VB volume and x is the NQ volume. The linear correlation is stronger when R2 is closer to 1 (above 0.7). If the graph is not a straight line the relation of analogy does not exist, R2 is closer to 0 (below 0.4). The volumes on each axis are expressed in cubic cm. Graphical representations of data per segment (all volumes in cubic cm) The least squares regression equation and squared correlation coe cient R2 appear on each graph.