Neurological disorders represent the anomalies relevant to the human nervous system. They also contain biochemical, anatomical or electrical modifications in the central nervous system, the spinal cord or the brain. These disorders provoke different symptoms. Early diagnosis of such changes is necessary for treatment, with the aim of limiting disease progression. An accurate and CAD system is introduced in this paper to classify brain MRI, which overcomes crucial problems in pattern classification, such as extracting certain features in the training phase. Our contribution is to merge second-generation wavelet (SGW) networks and deep learning architectures, hence suggesting novel supervised feature extraction approaches for pattern classification. Our novel architecture allows classifying the dataset classes by reconstructing a deep stacked second-generation wavelet autoencoder. Combining curvelet pooling (CP) with the Adam gradient calculation method can enhance the accuracy of the autoencoder. We build CP with Adam in this work utilizing both a Haar curvelet (CurvPool-AH) and a Shannon Curvelet (CurvPool-AS). This network can be attained following many SGW autoencoders ending up with one Softmax classifier at the final layer. We also find CurvPool carried out fairly well on all datasets. However, overfitting is a problem with all the approaches we consider. CP, particularly when merged with an adaptive gradient and curvelets chosen specifically for the data, has the potential to surpass the present methods. Our architecture is tested with various image datasets. These latter can be as follows: DS-66, DS-90, DS-160, and DS-255. Based on 5×5 cross-validation, our suggested approach can outperform the state-of-the-art methods as regards the classification accuracy.