Design of Meta-Predictor for Cholinergic Pattern. For a predictive model, predictor variables and dependent variables are generally chosen (or selected after manipulation) from variables of raw data. However, there was not in common molecular property information between CWAs and known cholinergic agents, and toxicity index was rarely available.[1, 2, 16, 24] The available data on cholinergic agents were their structures and cholinergic activities (Figure 3 and Table S1). Meanwhile, the only common information about CWAs and harmful agents was molecular structure. Expectedly, linking between CWAs and cholinergic data didn’t produce any common variable. Thus, a practical problem was how to create a unified descriptor (predictor variable) of the chemicals from the limited data. To define a unified descriptor, an important property of hazard and toxic agents is their toxicity profile together with molecular mechanics to lead to rescue from toxicity. Notably, the in-depth mechanism of respective toxicity is not clear for most all agents and is very different from each other. In CWAs, some nerve agents show higher structural congenericity and relatively more distinct mechanisms based on acetylcholinesterase (AChE) rather than other CWAs such as blister agent, asphyxiants, choking (pulmonary damaging) agents, incapacitating agents, lachrymating agents, and vomit agents.[1-2, 25-26] It is well-known that nerve agents and organophosphorus inhibit AChE at cholinergic synapses, thereby inhibiting the degradation of acetylcholine (Figure 3A). Accumulation of the released acetylcholine, causes end-organ overstimulation, which is recognized as cholinergic crisis.[1]
Thus, the limited knowledge motivated us to investigate a hazard and toxic space in terms of their cholinergic effect on the nervous system (of Figure 3). Notably, the aim of this study was not only cholinergic DTI prediction of individual chemicals but also was the detection of CWA from NE chemicals using cholinergic patterns of known chemicals. For this purpose, we designed a meta-predictor to describe the patterns using the structure-activity relationship (SAR) of cholinergic agents (Figure 4). First of all, the biochemical activities of cholinergic agents were embedded together with the molecular descriptors for a machine to learn the SAR. Secondly, the experimental activity data of ChEMBL (a public database) disciplined the machines to judge the relationship between the five cholinergic targets and chemicals, which is called drug target interaction (DTI). The trained DTI models of Figure 4 (200 classifiers of four type machines, ten differently divided data, and five targets) were internally and externally validated to elucidate the binominal cholinergic patterns (active/inactive) of a chemical. Thirdly, the cholinergic pattern of known CWAs and NPSs as harmful agents were predicted by the 200 binary classifiers, and the predicted values were transformed into an array type data as shown in Figure 4. Finally, the predicted array data was used as meta-predictors to build the CWA detection model. Even if real cholinergic patterns of these harmful chemicals are unknown, a chemo-centric approach allowed us to infer the pattern. The chemo-centric approach means if two similar molecules are likely to possess similar properties, they can share biological targets or may show similar pharmacological profiles. [27-32] Notably, this study used only two types real data: chemical structures of all chemicals (ChEMBL, CWAs, and NPSs) and cholinergic activities of ChEMBL chemicals (Figure 3B).
Robust DTI Classification Models for Meta Prediction. To realize the designed meta-predictor, two types of 2D molecular fingerprints (FCFP, ECFP) captured molecular structures of all cholinergic agents.[33] These extended-connectivity and functional-class fingerprint are well-known molecular representations, which precisely describe molecular structure and functional groups (groups of atoms having their own characteristic properties) in a molecule, and show their competent performance in drug design and large-scale prediction.[33] Thus, ECFP and FCFP were used to describe the cholinergic SAR under machine learning (ML) algorithms of random forest (RF), support vector machine (SVM), decision tree (DT), and k-nearest neighbor (KNN).[34-36] The DTI were trained for each cholinergic targets of acetylcholinesterase (AChE), butyrylcholinesterase (BuChE), nicotinic acetylcholinesterase receptor (nAChR), muscarinic acetylcholinesterase receptor (mAChR), and vesicular acetylcholine transporter (VAChT).[37] Firstly, statistical performance for the nAChR classifier was evaluated (Table1 and Table S2). Expectedly, the receiver operating characteristic curve (ROC) plots of nAChR classifiers demonstrated the robust predictability irrespective of data division into training and test (Table S2 and Figure S2). When Area Under ROC (AUC) of test data was compared, RF, SVM, and KNN models (AUC: 0.961 – 0.998) produced AUC higher than DT (AUC: 0.739 – 0.889). Furthermore, we applied other statistical metrics including accuracy, F1 score, and Matthews correlation coefficient (MCC), which informative and truthful score in evaluating binary classifications than accuracy and F1 score. Notably, the MCC values of every model were reliable (Test: MCC ~ 0.438 – 0.978, Train: 0.474 – 0.956), and the MCC values of test sets were at par with those of train sets. Secondly, the learning of the mAChR dataset followed a similar pattern to nAChR models, along with AUC of 0.807-0.998 and MCC of 0.608-0.974 (Table1 and Table S3). The mAChR models produced slightly higher predictive performance than the nAChR models. The overall DT model presented a lower performance than RF, SVM and KNN models. Thirdly, BuChE models also showed reliable prediction performance with AUC of 0.771-1.000 and MCC of 0.420 – 0.986 and slightly lower than the classification models of nAChR and mAChR (Table1 and Table S5). Fourthly, we further analyzed the classification metrics from AChE models. Despite the large data size (n = 3,098), the classification performance revealed at par performance for AUC of 0.774-0.999 (Table1 and Table S4). Finally, VAChT models of the smallest dataset outperform those of nAChR, mAChR, AChE, and BuChE (Table1 and Table S6). To visualize the predictive power of the cholinergic DTI models, the best performing models were described by ensemble-AUC values (Figure5 and Table S7).
Table1: The classification performance of selected best model based on ensemble-AUC for train and test set.
Target
|
ML
|
AUC
|
MCC
|
ACC
|
F1-Score
|
nAChR
|
RF
|
0.994 (0.987)
|
0.918 (0.975)
|
0.959 (0.987)
|
0.959 (0.987)
|
DT
|
0.845 (0.871)
|
0.678 (0.764)
|
0.836 (0.871)
|
0.824 (0.854)
|
SVM
|
0.994 (0.989)
|
0.936 (0.978)
|
0.968 (0.989)
|
0.968 (0.989)
|
KNN
|
0.741 (0.737)
|
0.551 (0.558)
|
0.741 (0.737)
|
0.791 (0.792)
|
mAChR
|
RF
|
0.997 (0.977)
|
0.952 (0.954)
|
0.976 (0.977)
|
0.976 (0.977)
|
DT
|
0.841 (0.820)
|
0.673 (0.642)
|
0.837 (0.820)
|
0.834 (0.813)
|
SVM
|
0.996 (0.981)
|
0.959 (0.962)
|
0.979 (0.981)
|
0.979 (0.981)
|
KNN
|
0.992 (0.958)
|
0.911 (0.917)
|
0.956 (0.958)
|
0.955 (0.958)
|
AChE
|
RF
|
0.997 (0.981)
|
0.942 (0.962)
|
0.971 (0.981)
|
0.971 (0.981)
|
DT
|
0.832 (0.789)
|
0.627 (0.597)
|
0.808 (0.789)
|
0.824 (0.813)
|
SVM
|
0.996 (0.986)
|
0.943 (0.972)
|
0.971 (0.986)
|
0.972 (0.986)
|
KNN
|
0.982 (0.818)
|
0.704 (0.683)
|
0.832 (0.818)
|
0.856 (0.846)
|
BUChE
|
RF
|
0.999 (0.973)
|
0.949 (0.948)
|
0.974 (0.973)
|
0.974 (0.973)
|
DT
|
0.796 (0.773)
|
0.523 (0.566)
|
0.761 (0.773)
|
0.760 (0.799)
|
SVM
|
0.995 (0.973)
|
0.961 (0.947)
|
0.980 (0.973)
|
0.980 (0.973)
|
KNN
|
0.909 (0.667)
|
0.408 (0.447)
|
0.643 (0.667)
|
0.737 (0.750)
|
VAChT
|
RF
|
1.000 (0.911)
|
0.702 (0.915)
|
0.830 (0.956)
|
0.887 (0.957)
|
DT
|
0.975 (0.944)
|
0.953 (0.934)
|
0.976 (0.967)
|
0.976 (0.966)
|
SVM
|
0.998 (1.000)
|
0.991 (1.000)
|
0.995 (1.000)
|
0.991 (1.000)
|
KNN
|
0.998 (0.956)
|
0.953 (0.934)
|
0.976 (0.967)
|
0.977 (0.967)
|
Abbreviations: ACC: Accuracy; MCC: Matthew’s Correlation Coefficient; RF: Random Forest; DT: Decision Tree; SVM: Support Vector Machine; KNN: K-Nearest Neighbor; nAChR: Nicotinic Acetylcholinesterase Receptor; mAChR: Muscarinic Acetylcholinesterase Receptor; AChE: Acetylcholinesterase Enzyme; BuChE: Butyrylcholinesterase Enzyme; VAChT: Vesicular Acetylcholine Transporter. The values in parenthesis are belongs to the test set. The best model was selected based on the ensemble-AUC (Table S7)
Multi-Tasking of Array Classifiers and Performance. The first tasking of the built array model is predicting cholinergic activities of ‘out-of-set (neither training nor test set)’ molecules on nAChR, mAChR, VAChT, AChE, and BUChE (Figure 4). For the purpose, every cholinergic DTI classifier was already validated in the prior section. Thus, CWAs and none CWAs consisting of NPSs and designer drugs [19] are out of ChEMBL data [38], neither training nor test data. Cholinergic patterns of the CWAs and none CWAs were predicted to play the role of meta-predictors for the second tasking. The second tasking of the array model is judging the chemical warfare likeness of ‘out-of-set’ molecules. For this purpose, the discrimination between CWAs and none CWAs was learned by a convolutional neural network algorithm. Despite the difference in data size, our meta-predictors have the same property of binary pixel array with MNIST hand-written data (28 x 28 pixel).[39] The common property made us benchmark the image-based learning of MNIST data, particularly, convolutional neural network (CNN). Firstly, our meta-predictors were converted to the 2D array of 5 × 4 shape for CNN learning. After the investigation, the architecture of Figure 6A (see also Figure S9) was chosen for the best learner. As our expectation, the 2D array reliably detected CWAs from large NPS data. During the learning along with the increased epoch, accuracy and loss values reached to their optimal values and retained the values (Figure 6B). With the encouraging results, we tried to adjust the data imbalance between CWAs and non-CWAs through over-sampling and under-sampling (the removal of data showing duplicated array values). As shown in Figure 7, when imbalanced native data (Model 01) was compared with balanced over-sampled data (Model 03), statistical metrics showed the deviation with a slight decrease but the area under precision-recall curve (AUPR) values of Figure 7A were still comparable between native (imbalanced) and over-sampled data (balanced) to prove that these statistical values did not simply result from data imbalances. Matthews correlation coefficient (MCC), F1-score, and accuracy (Figure 7B) also supported that the SMOTE (over-sampling) confirmed the ability able to find CWAs.[40] Furthermore, the two types sampling allowed us to evaluate 2D or 3D array classifiers of different shapes. When we re-shaped the 2D array from [50 × 4] to [40 x 5], the detection ability steeply decreased to show this data is sequential. Meanwhile, wen we converted the 2D array into 3D arrays, surprisingly, image-based learning of [10 x 5 x 4] shape improved AUPR, MCC, and F1-score of the worst ‘Model 04’ and decreased the performance gap between different data (Figure 7). When the 3D array was reshaped into [5 x 10 x 4], the improvement of these statistical values was also retained.
Based on the statistical validation of Figure 7 and Table S8, the array classifiers are ready for CWA detection from NE chemicals. Obviously, this predictive model for chemical threats under the chemo-centric assumption is arguable due to the available data and impossible experimental validation. However, such a trial is not never only one. For example, OECD also developed the QSAR model toolbox and has provided it for risk assessment. [10] Because current information on the mechanism of CWAs enriches in AchE and cholinergic effect, this study only described cholinergic patterns to detect chemical threats. In the future, if data is updated, this methodology is applicable for other pharmacological effects of known harmful chemicals such as brain monoacylglycerol (MAG) lipase activity and endocannabinoid degrading enzyme, fatty acid amide hydrolase (FAAH), which are recently reported toxicity mechanism of organophosphorus pesticides. [2,16] Even if the MAG and FAAH inhibition of the insecticides were reported, such a trial will be more feasible after updating the data (of MAG or FAAH agents) as much as those of cholinergic agents.