The proposed approach is explained in this section. The problem formulation is given first, then each step is explained in detail.
Problem Formulation
Given \({\{\left({c}^{\left(i\right)},{p}^{\left(i\right)}\right),{a}^{\left(i\right)}\}}_{i=1}^{N}\) where \({c}^{\left(i\right)}\) is the\({i}^{\text{t}\text{h}}\)SMILES format of drug, \({p}^{\left(i\right)}\) is the \({i}^{\text{t}\text{h}}\)protein sequence and \({a}^{\left(i\right)}\) denotes the corresponding affinity value. Our approach uses the protein-protein interaction (PPI) and drug-drug interaction (DDI) networks as auxiliary knowledge. The main goal is to design a model which takes drug, protein, DDI network, and PPI network as input and then predicts the affinity value of the input drug pair as output.
In our approach, to learn a better model, we could encourage the model to produce the same feature space for similar drug-protein pairs and different feature space for dissimilar drug-protein pairs. In this paper, to consider this issue, we have utilized triplet loss which encourages clustering of feature space and similarity between samples. The overall schematic of the proposed approach is given in Fig. 1. In the following, more details of the proposed approach are given.
Node2Vec
This paper uses the DDI and PPI networks as auxiliary knowledge. To describe this knowledge, Node2vec is utilized. Node2vec provides a framework to learn a representation for each node from highly structured objects like the graph. In other words, it gets a graph as input and then maps each node of a graph to an embedding space such that the representation reflects the local structure of the node with its neighborhood.
In the proposed approach, the PPI network is shown as a graph in which each node is a protein, and the edge between two proteins exists if there is an interaction between corresponding proteins. This network shown by \({G}_{PP}\) is fed into the Node2vec as input. Nod2vec learns a feature representation for each protein based on its network neighborhood. The output of Node2vec for the PPI network is shown by \({O}_{P}^{G}\in {\mathcal{R}}^{{n}_{p}\times {d}_{p}}\) where \({n}_{p}\) shows the number of unique proteins in the dataset and \({d}_{p}\) shows the dimension of feature representation of each protein. A similar computation is done for the DDI network. The DDI network shown by \({G}_{CC}\) is fed into the Node2vec as input. The output of Node2vec for the DDI network is shown by \({O}_{C}^{G}\in {\mathcal{R}}^{{n}_{c}\times {d}_{c}}\) where \({n}_{c}\) shows the number of unique drugs in the dataset and \({d}_{c}\) shows the dimension of feature representation of each drug.
Feature Descriptor
This section explains the feature descriptor networks for drug and protein sequences. We use 1D CNN architecture as a feature descriptor for both drug and protein sequences. The detailed architecture of the network is given in Fig. 1. CNN includes convolutional layers and pooling layers. 1D convolutional layer takes 1D sequence as input and then learns multiple filters (i.e., patterns, features) through one direction of the input (common horizontal stride). Pooling layer down-sample its input by reducing the resolution of the feature map, which leads to enlarging the receptive field. Assume that a protein sequence is shown by \(p\) into the 1D CNN. The output of CNN for input protein sequence is shown by \({o}_{p}^{I}\in {\mathcal{R}}^{1\times {d}_{p}}\) where \({d}_{p}\) shows the dimension of feature representation of each protein. Similarly, the output of feature descriptor (i.e., 1D CNN) for input drug molecule is shown by \({o}_{c}^{I}\in {\mathcal{R}}^{1\times {d}_{c}}\) where \({d}_{c}\) shows the dimension of feature representation of each drug.
The final feature descriptor for the input protein sequence is obtained by concatenating \({o}_{p}^{I}\) and \({O}_{P}^{G}\left(p\right)\):
$${o}_{p}=cocate({o}_{p}^{I}, {O}_{P}^{G}\left(p\right))$$
1
The final feature descriptor for the input drug sequence is obtained in a similar way by concatenating \({o}_{c}^{I}\) and \({O}_{C}^{G}\left(c\right)\):
$${o}_{c}=cocate({o}_{c}^{I}, {O}_{C}^{G}\left(c\right))$$
2
Finally, the feature descriptor of the paired drug and protein is computed as follows:
$${o}_{{d}_{p},{d}_{c}}=cocate({o}_{p},{o}_{c})$$
3
Triplet Loss Function
The training step in this study incorporates both similar and dissimilar drug-target pairs. It causes the model to learn better feature space representation with margin maximizations. In the training step, one similar pair and one dissimilar pair are retrieved for each drug-protein pair. Therefore, three pairs are fed into the method: an anchor pair (input pair), a positive pair (similar pair), and a negative pair (dissimilar pair). In this case, the triplet loss is also considered in addition to the main loss function (prediction loss error).
The triplet loss is defined as follows:
$${L}_{triplet}=\sum \text{m}\text{a}\text{x}(dist\left({o}_{{d}_{a},{t}_{a}},{F}_{{d}_{p},{t}_{p}}\right)-dist\left({o}_{{d}_{a},{t}_{a}},{F}_{{d}_{n},{t}_{n}}\right)+margin,0)$$
4
where \(({d}_{a},{t}_{a})\), \(({d}_{p},{t}_{p})\) and \(({d}_{n},{t}_{n})\) denote the anchor pair, the negative pair, and the positive pair, respectively. Also, \(dist(.,.)\) indicates a distance function which is defined as \({l}_{2}\) function.
Triplet loss encourages the anchor and positive sample representations to be similar together while the anchor and negative sample representations to be away from each other. The overall schematic of the triplet loss function is in Fig. 2. In the triplet loss, there is a \(margin\) which enforces different points at a distance greater than the margin do not contribute to the loss function. In the triplet loss function, the way of selecting negative samples is important. In our approach, the semi-hard negative sample technique is utilized. In this case, for each input pair, the following computation is done between each input pair and all other pairs in the batch:
$$dist\left({o}_{{d}_{a},{t}_{a}},{o}_{{d}_{p},{t}_{p}}\right)-dist\left({o}_{{d}_{a},{t}_{a}},{o}_{{d}_{n},{t}_{n}}\right)-margin$$
5
Then, the k top ones with maximum value are chosen as negative samples, and \(k\) triplet inputs are provided. Positive samples are selected randomly based on their labels. It should be noted that triplet loss optimization is done in the training step hence utilizing the label of samples is unrestricted.
The prediction loss for the classification task is a categorical cross-entropy. The final descriptor of the paired input is fed into the prediction network shown by \({N}_{pred}\) to produce the final label of the input sample. The prediction network is a multilayer perceptron that maps the input feature into interaction class labels. The prediction loss is defined as follows:
$${L}_{1}=\sum _{k}{{N}_{pred}\left({o}_{{d}_{i},{p}_{i}}\right)}_{k}\text{l}\text{o}\text{g}\left({a}_{i}^{k}\right)$$
6
At first, the proposed network is optimized using the task prediction function (i.e., categorical cross-entropy loss), and then it is tuned using the triplet loss function and task prediction loss. Iteratively, the process is repeated until convergence is achieved.
Experiments
In this section, the experimental results are given. The proposed approach is evaluated using the dataset provided by Wang et al. [25]. Their data have gathered six heterogeneous networks, including drug-drug interaction network, protein-protein interaction network, drug-protein interaction network, drug-disease association network, protein-disease association network, and drug side-effect association network. Drug-drug and protein-protein interaction networks are extracted from Drugbank Version 3.0 [26] and HPRD database Release 9 [27]. These networks include 708 unique drug nodes and 1512 unique protein nodes. Drug-drug and protein-protein interaction networks include 10036 and 7363 edges. Drug-target interaction network is extracted from Drugbank Version 3.0 [26] with 1923 edges. The drug-disease and protein-disease association networks are extracted from [28] with 199214 and 1596745 edges, respectively. The drug-side effect association network is also extracted from [29] with 80164 edges. All of these networks only provide binary edge weights.
In this paper, to evaluate the model, we have used a wide variety of performance measures, including accuracy, precision, recall, F1 score, the area under the receiver operating characteristic curve (AUC), and the area under the precision-recall curve (AUPRC). Since the applied dataset is imbalanced, utilizing measures like F1 score and AUPRC is essential. The imbalanced ratio of the dataset is high, and hence we need to use a technique to learn a more reliable model with more generalization ability. To handle this issue, similar to Wan et al.[30], the positive samples are oversampled ten times, and the negative samples are under-sampled by randomly selecting the samples such that they have the same size as the positive samples. Also, 10-fold cross-validation is used as an evaluation technique to have a fair comparison with the state-of-the-approaches.
In the proposed approach, hyper-parameter optimization is done using grid search. This optimization is done for the number of filters in drug and protein feature encoder networks search over [16, 32, 64, 128, 256], the size of filters for drug feature encoder network search over [4, 8, 12], and size of filters for protein feature encoder search over [8, 12, 16], learning rate search over [0.0001, 0.001, 0.01, 0.1]. Also, semi-hard samples are used for batch definition in triplet loss minimization. First, each input's distance from all the samples in the subset is calculated to select semi-hard negative samples. Next, we select k samples with a larger distance from the anchor than the positive but still produce a positive loss. The hyper-parameter optimization for \(k\) is done over [3, 5, 7].
In order to evaluate the contribution of each component of the proposed approach, an ablation study is performed. In this case, two variants of the proposed approach are created. The first version of the multimodal knowledge that includes protein-protein and drug-drug interaction networks is not considered. This version is shortly called TripletDTI-uni-modal. In the second version, the triplet loss is not considered during the optimization, which is called DTI multimodal. The obtained results of these two versions and the full proposed approach are given in Table 1.
Table 1
The obtained results on the dataset and its comparison with the state-of-the-art approaches
Algorithms | Precision | Recall | Accuracy | F1-score | AU-PR | AU-ROC |
DTI-multi-modal | 0.802 | 0.971 | 0.865 | 0.878 | 0.927 | 0.940 |
TripletDTI-uni-modal | 0.854 | 0.931 | 0.923 | 0.891 | 0.954 | 0.918 |
TripletMultiDTI | 0.931 | 0.950 | 0.942 | 0.940 | 0.994 | 0.993 |
We compare our approach to four state-of-the-art approaches: NeoDTI [30], DTINet [31], HNM [25], and MultiDTI [10]. All of these approaches utilize heterogeneous networks to learn a more effective model for DTI prediction. Table 2 compares the proposed approach with state-of-the-art approaches, and its results are given. As it is shown, the proposed approach gets higher performance in four measures, including AU-PR, AU-ROC, F1-score, and Recall. Most improvement in performance is achieved in the F1-score, which is 0.072 over the MultiDTI and NeoDTI.
Table 2
The obtained results on the dataset and its comparison with the state-of-the-art approaches
Algorithms | Precision | Recall | Accuracy | F1-score | AU-PR | AU-ROC |
NeoDTI | 0.928 | 0.716 | 0.969 | 0.868 | 0.961 | 0.947 |
DTINet | 1.0 | 0.048 | 0.524 | 0.090 | 0.914 | 0.932 |
HNM | NA | NA | NA | NA | 0.572 | 0.890 |
MultiDTI | 0.876 | 0.882 | 0.873 | 0.868 | 0.947 | 0.961 |
Our Approach | 0.931 | 0.950 | 0.942 | 0.940 | 0.994 | 0.993 |