Background: We introduce a novel speech processing framework, the MIT CBMM Open Voice Brain Model (OVBM), combining implementations of the 4 modules of intelligence: The brain OS chunks and overlaps audio samples and transfers CNN features from the sensory stream and cognitive core creating a multi-modal graph neural network of symbolic compositional models for the target task.
Methods: Our approach consists of pre-training models to extract acoustic features from selected biomarkers and then leverage transfer learning to combine the biomarker feature extractors into a graph neural network to provide an explainable diagnsotic for Alzheimer's Dementia (AD) using speech recordings.
Results: We apply OVBM to the automated diagnostic of Alzheimer's Dementia patients, achieving above state-of-the-art accuracy of 93.8% using only raw audio, while extracting a personalized subject saliency map to track relative disease progression of 16 explainable biomarkers.
Conclusion: By using independent biomarker models, OVBM lets health experts explore biomarker features and whether there are common biomarkers features between AD and other diseases like COVID-19. We present a novel lungs and respiratory tract biomarker created using 200.000+ cough samples to pre-train a model discriminating English from Catalan coughs. Transfer Learning is subsequently used to transfer features from this model with various other biomarker OVBM models. This strategy yielded consistent improvements in AD
detection, no matter the combination used. This cough dataset sets a new benchmark as largest audio health dataset with 30.000+ subjects participating in April 2020, demonstrating for the rst time cough cultural bias.