This work proposes a novel framework of an explainable ensemble of neural networks to classify NSCLC samples into adenocarcinoma and squamous cell carcinoma, using DNA methylation data. The framework utilizes an ensemble of shallow neural networks and soft-voting decision fusion for classification. Subsequently, these neural networks are interpreted via SHapley Additive exPlanations (SHAP) to highlight the most relevant DNA methylation CpG probes. The proposed framework-based model achieves a classification accuracy of 0.989, outperforming other ensemble models such as XGboost, Random Forest, AdaBoost, CatBoost, and GradientBoosting. SHAP analysis reveals 702 relevant CpG probes, that are mapped to a set of 499 signature biomarkers. While 104 of these signature biomarkers are potentially druggable (using DGIdb database), 40 of them overlap with the OncoKB cancer genes list. In the future, the framework could be made robust enough to classify other carcinomas. Moreover, multiomics data-based classification could provide better accuracy and more stable biomarkers.