This preprint is under consideration at Journal of Cheminformatics. A preprint is a preliminary version of a manuscript that has not completed peer review at a journal. Research Square does not conduct peer review prior to posting preprints. The posting of a preprint on this server should not be interpreted as an endorsement of its validity or suitability for dissemination as established information or for guiding clinical practice.
Learn more about In Review
Research article
Pavel Polishchuk
Univerzita Palackeho v Olomouci Lekarska fakulta
Corresponding Author
ORCiD: https://orcid.org/0000-0001-5088-8149
License:
This work is licensed under a CC BY 4.0 License. Read Full License
Interpretation of QSAR models is useful to understand the complex nature of biological or physicochemical processes, guide structural optimization or perform knowledge-based validation of QSAR models. Highly predictive models are usually complex and their interpretation is non-trivial. This is particularly true for modern neural networks. Various approaches to interpretation of these models exist. However, it is difficult to evaluate and compare performance and applicability of these ever-emerging methods. Herein, we developed several benchmark data sets with end-points determined by pre-defined patterns. These data sets are purposed for evaluation of the ability of interpretation approaches to retrieve these patterns. They represent tasks with different complexity levels: from simple atom-based additive properties to pharmacophore hypotheses. We proposed several quantitative metrics of interpretation performance. Applicability of benchmarks and metrics was demonstrated on a set of conventional models and end-to-end graph convolutional neural networks interpreted by the previously suggested universal ML-agnostic approach for structural interpretation. We anticipate these benchmarks to be useful in evaluation of new interpretation approaches and investigation of decision making of complex “black box” models.
Keywords
QSAR model interpretation, benchmark data set, synthetic data set, interpretability metrics, atom contributions, graph convolutional neural networks
Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9
Figure 10
Figure 11
This is a list of supplementary files associated with this preprint. Click to download.
Loading...
Badges
Prescreen
This manuscript passed our Prescreen assessment
Complete author information
Appropriate declaration statements
Potential risks to human health
Prescreen
Peer Review Timeline
CURRENT STATUS: Under Revision
Version 1
Posted 04 Mar, 2021
No community comments so far
Editorial decision: Major revision
On 28 Mar, 2021
Review #2 received
Received 27 Mar, 2021
Review #1 received
Received 24 Mar, 2021
Reviewer #2 agreed
On 07 Mar, 2021
Reviewer #1 agreed
On 06 Mar, 2021
Review #? received
Received 06 Mar, 2021
Reviewers invited
Invitations sent on 27 Feb, 2021
Submission checks complete
On 25 Feb, 2021
Editor assigned
On 24 Feb, 2021
Editor invited
On 24 Feb, 2021
First submitted
On 23 Feb, 2021
Metrics
Comments: 0
PDF Downloads: ...
HTML Views: ...
Subject Areas
Physical Chemistry
Learn more about our company.
This preprint is under consideration at Journal of Cheminformatics. A preprint is a preliminary version of a manuscript that has not completed peer review at a journal. Research Square does not conduct peer review prior to posting preprints. The posting of a preprint on this server should not be interpreted as an endorsement of its validity or suitability for dissemination as established information or for guiding clinical practice.
Learn more about In Review
Research article
Pavel Polishchuk
Univerzita Palackeho v Olomouci Lekarska fakulta
Corresponding Author
ORCiD: https://orcid.org/0000-0001-5088-8149
Badges
Prescreen
This manuscript passed our Prescreen assessment
Complete author information
Appropriate declaration statements
Potential risks to human health
Prescreen
Peer Review Timeline
CURRENT STATUS: Under Revision
Version 1
Posted 04 Mar, 2021
No community comments so far
Editorial decision: Major revision
On 28 Mar, 2021
Review #2 received
Received 27 Mar, 2021
Review #1 received
Received 24 Mar, 2021
Reviewer #2 agreed
On 07 Mar, 2021
Reviewer #1 agreed
On 06 Mar, 2021
Review #? received
Received 06 Mar, 2021
Reviewers invited
Invitations sent on 27 Feb, 2021
Submission checks complete
On 25 Feb, 2021
Editor assigned
On 24 Feb, 2021
Editor invited
On 24 Feb, 2021
First submitted
On 23 Feb, 2021
Metrics
Comments: 0
PDF Downloads: ...
HTML Views: ...
Subject Areas
Physical Chemistry
License:
This work is licensed under a CC BY 4.0 License. Read Full License
Interpretation of QSAR models is useful to understand the complex nature of biological or physicochemical processes, guide structural optimization or perform knowledge-based validation of QSAR models. Highly predictive models are usually complex and their interpretation is non-trivial. This is particularly true for modern neural networks. Various approaches to interpretation of these models exist. However, it is difficult to evaluate and compare performance and applicability of these ever-emerging methods. Herein, we developed several benchmark data sets with end-points determined by pre-defined patterns. These data sets are purposed for evaluation of the ability of interpretation approaches to retrieve these patterns. They represent tasks with different complexity levels: from simple atom-based additive properties to pharmacophore hypotheses. We proposed several quantitative metrics of interpretation performance. Applicability of benchmarks and metrics was demonstrated on a set of conventional models and end-to-end graph convolutional neural networks interpreted by the previously suggested universal ML-agnostic approach for structural interpretation. We anticipate these benchmarks to be useful in evaluation of new interpretation approaches and investigation of decision making of complex “black box” models.
Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9
Figure 10
Figure 11
Loading...