Pneumonia is a potentially fatal lung infection caused by various viral infections. Because pneumonia is like other pulmonary diseases, identifying and treating it on chest X-ray images can be difficult. Therefore, a more reliable method for detecting pneumonia in X-ray images is required. So, an optimized Deep Learning approach is used in this study, in which x-rays of the chest are handled in four stages. The first stage involves performing image enhancement, followed by a data augmentation stage, and the third stage involves feeding the results to deep learning algorithms (CNN, VGG16, InceptionResNetV2, Xception, Resnet50, and hybrid model) in which image features are retrieved for further processing. These extracted features are then utilized in the subsequent step, where they are inputted into various machine learning algorithms, including Logistic Regression, Decision Tree, Random Forest, SVM, and AdaBoost. These algorithms are responsible for classifying and diagnosing the images. To evaluate the proposed approach, a comprehensive dataset comprising 8,217 images (5,259 pneumonia and 2,958 normal) from combined chest X-ray and Mendeley sources was employed. The findings of the experiment reveal that the hybrid model, combined with the SVM classifier, demonstrates exceptional performance in terms of both training and testing accuracy, exceeding 97.70%. In conclusion, the proposed framework successfully leverages the extracted features and machine learning algorithms to achieve highly accurate classification and diagnosis of pneumonia cases. The hybrid model, in conjunction with the SVM classifier, exhibits remarkable performance in accurately identifying pneumonia from chest X-ray images.