A Machine Learning Protocol to Directly Translate Molecular Infrared or Raman Spectra into Molecular Formula

doi:10.21203/rs.3.rs-3709542/v1

Download PDF

Article

A Machine Learning Protocol to Directly Translate Molecular Infrared or Raman Spectra into Molecular Formula

https://doi.org/10.21203/rs.3.rs-3709542/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Molecular recognition usually adopts the molecular spectra library matching or the simulation-based ‘trial-and-error’ strategies. However, these two methods are largely limited by the low coverage rate, high construction cost and the time-consuming process. Here we developed TranSpec, a deep learning model based on the convolutional neural networks and multi-head attention mechanism, to directly ‘translate’ the molecular vibrational spectra into the simplified molecular input line entry system (SMILES) representations. Taking the QM9S dataset that includes the quantum chemistry simulated spectra of 130K molecules as the benchmark, we have demonstrated the greedy search (generating only one SMILES) can precisely identify 90%-100% functional groups, and provide about 60% correct SMILES based on the infrared (IR) or Raman spectra. To enhance the translation accuracy, we proposed several strategies such as leveraging the packed IR and Raman spectra as input, employing threshold search to generate more SMILES candidates, and filtering with the molecular mass. Finally, we demonstrated that TranSpec has good transferability through translation experimental infrared spectra and found that threshold searches (generating top 1 and top 10 SMILES candidates) correctly identified 21.8% and 55.9% of the molecules, respectively. The proposed TranSpec realized the direct interpretation of molecule spectra and paved a promising way to achieve fast and real-time molecular recognition.

Physical sciences/Chemistry/Theoretical chemistry/Structure prediction

Physical sciences/Chemistry/Theoretical chemistry/Method development

Vibrational spectroscopy including IR and Raman spectra play a crucial role in identifying the chemical compounds^1,2 due to the high sensitivity, non-destructive nature, and the capacity of providing molecular fingerprint. In practical application, the spectra library matching strategy^3,4 that compares the query spectrum and each entry in the reference spectral library is widely adopted. However, this method is largely limited by the high construction cost and low coverage rate, resulting the chemical compounds out of the spectral dataset cannot be identified. As another popular method, the ‘trial-and-error’ method⁵ adopts a cyclic self-consistent workflow that contains building the model, simulating the spectra, comparing with the target and adjusting the model. The complicated process requires deep chemical intuition, extensive experience and expensive calculations, which largely limits the recognition efficiency and accuracy.

Machine learning (ML) is superior at finding hidden correlations and has been exploited in the library matching and ‘trial-and-error’ method^6–9. For example, Zhang et al developed the deep-learning-based neural network to predict molecular fingerprints from the electron ionization − mass spectrometry and searches the molecular structure dataset with the predicted fingerprints¹⁰. Wang et al trained the ¹³C NMR spectra and SMILES contrastive learning network which enables matching ¹³C NMR spectra directly against a molecular structure library¹¹. The usage of the transformed index or cross-contract with structure library can partly overcome the low coverage of the spectral dataset, while the identification accuracy and efficiency are still limited. On the other hand, the molecule generating ML models were proposed to replace the artificial molecular design and efficiently improve the ‘trial-and-error’ method. For example, Tsuda et al. proposed a de novo molecular generator¹², and Priyakumar et al. employed the Monte Carlo tree search (MCTS) and a message passing neural network to generate the molecules¹³. Then the molecular recognition could be realized based on the comparison of the NMR spectra of the generated molecules with the input spectra. However, the recognition process is still based on quantum chemistry calculation, largely limiting the efficiency.

Transformer proposed in the article "Attention Is All You Need"¹⁴ have demonstrated remarkable success in various field such as image and speech recognition, natural language processing etc. The multi-head attention mechanism used in Transformer could not only improve the translation accuracy for very long sentence, but also provide precise and proper meaning of every particular word in different context. The IR and Raman spectra contain abundant information about the molecular structure including particular functional groups, local and non-local chemical environments. On the other hand, molecules can also be equivalently represented as text in line symbol format, such as the international chemical identifier (InChI)¹⁵, simplified molecular input line entry system (SMILES)¹⁶, SMILES arbitrary target specification (SMARTS), and so on. As a result, an alternative approach to identify the unknown compounds is to map the spectral sequence to the widely accepted molecular representations.

In the present work, we developed a deep learning algorithm to directly translate the IR or Raman spectra into SMILES representations, which is named as TranSpec hereafter (illustrated as Fig. 1b). Taking QM9S ¹⁷and the experimental IR spectra dataset from NIST Chemistry WebBook¹⁸ as the benchmark, we firstly use the convolutional neural network^19–21 and linear layers to extract the feature vectors from the molecular vibrational spectra. Then the multi-head attention mechanism combined the feed-forward network are used to export the probable tokens one by one and finally output the SMILES. We found the greedy search²² that generate only one SMILES could precisely recognize about 60% molecules from QM9S dataset, while the threshold search²³ generates top 10 SMILES candidates could cover 90% molecules. Packing the IR and Raman spectra together and using the molecular mass as a filter can further improve the analytic accuracy. Furthermore, we have validated the TranSpec’s transferability and potential applications by recognizing the experimental IR spectra, identifying different functional groups and distinguishing the isomers and homologues.

TranSpec architecture. Inspired by the classic Transformer, we build in the TranSpec the encoder-decoder structure (illustrated as Fig. 1b) and utilize the multi-head attention mechanism^24,25 to translate the molecular spectra into the corresponding SMILES. As shown in Fig. 1b, the tokens are firstly positional encoded into a ${d}_{\text{m}\text{o}\text{d}\text{e}\text{l}}=256$ dimensional vector representation in the SMILES encoder. Here we use the Kekuler-style instead of canonical SMILES to reduce the kinds of tokens. Considering the coverage of QM9S database, only 16 tokens are used in the present work to represent the start (< BOS>), termination (< EOS>), padding (< PAD>), two kinds of special chemical bonds (<=>, <#>), four kinds of elements (< C>, <N>, <O>, <F>), two kinds of branches (< (>, <)>), and five kinds of ring or connections (< 1>, < 2>, < 3>, < 4>, < 5>).

In the spectra encoder, we firstly use 6 layers of convolutional neural networks and a fully connected linear layer to generate a ${d}_{\text{m}\text{o}\text{d}\text{e}\text{l}}\times {d}_{\text{m}\text{o}\text{d}\text{e}\text{l}}$ spectra feature matrix for any spectra input. Here the input could be single IR or Raman spectra, or packed IR and Raman spectra (named as IR&Raman hereafter). Finally, the spectral feature matrix produces the V and K vectors of the next decoder model.

$$\text{A}\text{t}\text{t}\text{e}\text{n}\text{t}\text{i}\text{o}\text{n}\left(Q,K,V\right)=\text{s}\text{o}\text{f}\text{t}\text{m}\text{a}\text{x}\left(\frac{Q{K}^{T}}{\sqrt{{d}_{K}}}\right)V$$

The tokens decoder is composed of a stack of $N=2$ identical layers. Each layer has three sub-layers. The first one is the masked multi-head self-attention²⁶ calculation on the SMILES vector representation:

$$\text{M}\text{u}\text{l}\text{t}\text{i}\text{H}\text{e}\text{a}\text{d}\left(Q,K,V\right)=\text{C}\text{o}\text{n}\text{c}\text{a}\text{t}\left({\text{h}\text{e}\text{a}\text{d}}_{1},\cdots {\text{h}\text{e}\text{a}\text{d}}_{h}\right){W}^{O}$$

$${\text{h}\text{e}\text{a}\text{d}}_{i}=\text{A}\text{t}\text{t}\text{e}\text{n}\text{t}\text{i}\text{o}\text{n}\left(Q{W}_{i}^{Q},K{W}_{i}^{K},V{W}_{i}^{V}\right)$$

where the projections are parameter matrices ${{W}_{i}^{Q}\in \mathbb{R}}^{{d}_{\text{m}\text{o}\text{d}\text{e}\text{l}}\times {d}_{q}}$, ${{W}_{i}^{K}\in \mathbb{R}}^{{d}_{\text{m}\text{o}\text{d}\text{e}\text{l}}\times {d}_{k}}$, ${{W}_{i}^{V}\in \mathbb{R}}^{{d}_{\text{m}\text{o}\text{d}\text{e}\text{l}}\times {d}_{v}}$ and ${{W}^{O}\in \mathbb{R}}^{{hd}_{v}\times {d}_{\text{m}\text{o}\text{d}\text{e}\text{l}}}$. In the present work we employ $h=$8 parallel attention layers, or heads, resulting ${d}_{q}={d}_{k}={d}_{v}=\frac{{d}_{\text{m}\text{o}\text{d}\text{e}\text{l}}}{h}=64$. The multi-head attention ensures the model could jointly attend to information from different representation subspaces at different positions, while the masked coding ensures the prediction at present depends only on already generated tokens. The second sub-layer is the key competitive neural network that utilize the multi-attention mechanism to extract the molecular structural information from the spectra features. As a result, the $Q$ vector is generated by SMILES self-attention, while the $V$ and $K$ vectors from the spectral feature matrix. The third is a simple, position-wise fully connected feed-forward network. We employ a residual connection around each of the three sub-layers, followed by layer normalization to moderate over-smoothing²⁷. That is, the output of each sub-layer is $\text{L}\text{a}\text{y}\text{e}\text{r}\text{N}\text{o}\text{r}\text{m}(\text{x}+\text{S}\text{u}\text{b}\text{l}\text{a}\text{y}\text{e}\text{r}(\text{x}\left)\right)$, where $\text{S}\text{u}\text{b}\text{l}\text{a}\text{y}\text{e}\text{r}\left(\text{x}\right)$ is the function implemented by the sub-layer itself.

Lastly, we apply a MLP²⁸ and a SoftMax²⁹function on the output of the token decoder and produce the probability distributions of the present token. All parameters in the network are optimized by minimizing the cross-entropy loss³⁰ between the output and the target sequence which is defined as:

$$CrossEntropyLoss=\sum _{n=1}^{N}\frac{{l}_{n}}{\sum _{n=1}^{N}{w}_{{y}_{n}}\bullet 1},{l}_{n}=-{\omega }_{{y}_{n}}\text{log}\frac{\text{e}\text{x}\text{p}\left({x}_{n,{y}_{n}}\right)}{{\sum }_{c=1}^{C}\text{e}\text{x}\text{p}\left({x}_{n,c}\right)}\bullet 1$$

where $x$ is the input, $y$ is the target, ${w}_{c}$ is the weight, $C$ is the number of classes, and N spans the minibatch dimension as well as d₁, …, d_k for the K-dimensional case.

Translation on theoretical spectra datasets. Taking the QM9S dataset that includes IR and Raman spectra of 130K molecules from quantum chemistry calculations as the benchmark, we firstly trained and optimized the TranSpec model. As illustrated in Fig. 1c, TranSpec successively outputs the tokens as well as the corresponding probabilities. Here we should set a threshold value for the probabilities, otherwise the number of the output SMILES will be infinite. As shown in Fig. 2a, a threshold value larger than 60% will output the most probable tokens at every step, which is usually called as the greedy search. It is well known that the greedy search would loss some global information, resulting the translation accuracy from the IR, Raman, and IR&Raman spectra to SMILES located at 55.3%, 57.9% and 63.8%, respectively.

Alternatively, decreasing the threshold value will produce more SMILES candidates (shown in Fig. 2a). For example, the average numbers of the produced SMILES candidates with the threshold of 20% are 3.2, 4.1 and 3.5 for IR, Raman, and IR&Raman spectra, while a threshold of 1% will produce averaged 358.9, 561 and 366.8 candidates. Including all candidates largely increases the computational cost and decreases the analytic efficiency. As a result, we rank the SMILES by multiplying the probabilities at every step to produce Top N candidates. We can regard the translation as successful if the right SMILES is included in the Top N candidates and some other observable properties (such as the mass, dipole moment³¹, elementary composition, etc) can be used to further filter the SMILES candidates. For example, providing the mass information could largely decrease the number of the candidates and improve the translation efficiency, as shown in Fig. 2b.

Figure 2c-e shows the TranSpec’s translation accuracy rate with varying the Top N candidates at threshold values of 20%, 5% and 1%. We can see from Fig. 2c-d that the translation accuracy from the Raman spectra is a litter higher than that from the IR spectra when the threshold is 20% and 5%. However, the translation accuracy from the IR and Raman spectra is almost the same when the threshold is 1% (shown in Fig. 2e). Infrared&Raman spectroscopy can further improve the translation accuracy. We also found that decreasing the threshold value could slightly decrease the translation accuracy rate of the Top 1 candidate, resulting a translation accuracy rate of 66.8%, 66.3% and 65.3% for the threshold value of 20%, 5% and 1%, respectively. However, using smaller thresholds will increase the translation accuracy rate if more candidates are included. For example, the translation accuracy rate of the Top100 candidate from Raman spectra is 77.8%, 92.3% and 93% with the threshold value of 20%, 5% and 1%, respectively. Using the mass information as the filter could further improve the translation accuracy, making the translation accuracy rate of the Top100 candidate from Raman spectra is 77.8%, 92.8% and 97% with the threshold value of 20%, 5% and 1%, respectively.

Translation on experimental spectra datasets. We also tested the TranSpec’s accuracy by translating the experimental IR spectra extracted from the national institute of standards and technology (NIST) Chemistry WebBook. Here only the 5624 IR spectra observed in the gas phase are used to keep the data’s uniformity. Compared to the 16 tokens used in the theoretical data, three element tokens (< Cl>, <Br>, <S>) are added and two ring or connection tokens are removed (< 4>, < 5>) for the experimental data. The translation accuracy on the experimental IR spectra using the Top 1 and 100 candidates with the threshold of 0.01 is only 8% and 39.9%. This low translation accuracy rate can be attributed to the small size of the experimental dataset. To further improve the translation accuracy, here we also use the molecular relative mass to filter the SMILES candidates. In this way the translation accuracy of Top 1 and 100 candidates with the threshold value of 1% on the experimental IR spectra can be increased to be 21.8% and 66%, respectively (As shown in Fig. 2f).

Table 1

Identification accuracy rate of the 12 most common functional groups using the TranSpec by translating the theoretical IR, Raman, IR&Raman and experimental IR spectra.
Functional Group Types	Functional Groups Identification Accuracy Rate using Different Spectra and Datasets (%)
Functional Group Types	Theo. IR	Theo. Raman	Theo. IR&Raman	Exp. IR
alkane	99.80	99.86	99.91	96.79
alkene	96.35	98.03	97.91	92.59
alkyne	98.92	99.57	99.79	100
aromatics	95.76	96.30	97.28	96.97
alcohols	96.95	96.74	98.36	100
aldehydes	94.83	94.21	96.84	60
ketones	90.43	88.88	90.25	75
esters	91.54	86.35	90.44	95.45
ether	96.89	93.87	97.33	94.34
amides	83.64	82.90	87.44	55.56
nitriles	99.34	99.12	99.56	100
amine	88.44	87.10	92.76	76

Identification of functional groups. Considering the greedy search could lose some global information, we examined the potential capability of TranSpec in identifying the different functional groups. The identification accuracy of the most common 12 functional groups in the QM9S and the experimental IR dataset are listed in Table 1. We use RDKit³² to determine if each compound has a specific functional group. As can be seen, 90% of the functional groups in the QM9S dataset can be identified. Furthermore, we found the identification of the alkane, alkene, alkyne and aromatics groups could be more easily realized using Raman rather than IR spectra. On the other hand, the identification of the alcohols, aldehydes, ketones, esters, ether, amides, nitriles and amine groups can be more easily realized using IR spectra. Packing the IR and Raman spectra could further enhance the identification accuracy for all groups except the alkene, ketones and esters. The identification accuracy of functional groups in the experimental IR spectroscopy ranges from 76% to 100%. This can be attributed to the small size of the dataset and the low coverage of the corresponding groups.

Distinguishing similar molecules from spectra.

Taking several molecules extracted from the verification and test set of QM9S for instance, we demonstrated the capability of TranSpec in identifying isomers and homologues. As shown in Fig. 3, the IR or Raman spectra of the four isomers or homologues are very similar. However, TranSpec can capture the fine distinction (emphasized in gray box in Fig. 3) and finally output the right SMILES using the greedy searching. From this point view, TranSpec can distinguish the molecules with similar structures from the very similar spectra.

Overall, TranSpec model can quickly and efficiently recognize the molecular species by directly translate the theoretical or experimental IR or Raman spectra into SMILES representations. The translation accuracy rate on the QM9S theoretical spectra datasets was found to be about 60%, while the threshold search can reach about 93% if the Top 100 candidates are included. We found the translation accuracy rate based on the experimental IR spectra is lower because of the low coverage and heterogeneity the dataset. To further increase the translation accuracy, we proposed the IR&Raman packing and mass-filtering strategy. TranSpec could realize the molecular species recognition, the functional groups identification, and isomers from homologues distinguishment, validating the possibility of real-time structural identification based on spectroscopic measurements.

Data Availability

All datasets used in this document are publicly available.

The QM9S dataset including the molecular SMILES, the optimized structures, and the IR and Raman spectra of the 133,885 different molecules is available at https://github.com/zouzihan/QM9S.

The experimental infrared spectroscopic data extracted from the National Institute of Standards and Technology Chemistry WebBook (NIST) is available at https://webbook.nist.gov/chemistry.

Code Availability

The TranSpec model and trained parameters are available from https://github.com/hutianqing/TranSpec. A TranSpec based SMILES predictor is also included in the code. All code is written using the Pytorch and RDKit libraries on python 3.9.

Acknowledgements

We acknowledge grants from the National Natural Science Foundation of China (22073053 (W.H.), 22025304 (J.J.), 22033007 (J.J.)), the Young Taishan Scholar Program of Shandong Province (tsqn201909139 (W.H.)), the Natural Science Foundation of Shandong Province (ZR2023MA089 (Y.Z)), the Program of New Collegiate 20 Items in Jinan (2021GXRC042 (W.H.), 202228031 (Y.Z.)) and the Qilu University of Technology (Shandong Academy of Sciences) Basic Research Project of Science, Education and Industry Integration Pilot (2023PY046 (Y.Z.)). The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.

Author Contributions

W.H. conceived the research and designed the TranSpec model. T.H, B.L. and Z.Z performed the training and prediction process. W.H., Y.Z., J.J. and Y.L. oversaw the project from concept to experimental design, implementation, theory, and analysis of the data. All authors contributed to the writing of the manuscript.

Kim, S., Lee, D., Liu, X., Van Neste, C., Jeon, S. & Thundat, T. Molecular recognition using receptor-free nanomechanical infrared spectroscopy based on a quantum cascade laser. Sci. Rep. 3, 1111 (2013).
Butler, H. J. et al. Using Raman spectroscopy to characterize biological materials. Nat. Protoc. 11, 664-687 (2016).
Shiferaw, G. A., Vandermarliere, E., Hulstaert, N., Gabriels, R., Martens, L. & Volders, P.-J. COSS: A fast and user-friendly tool for spectral library searching. J. Proteome Res. 19, 2786-2793 (2020).
Ju, Y., Neumann, O., Bajomo, M., Zhao, Y., Nordlander, P., Halas, N. J. & Patel, A. Identifying surface-enhanced raman spectra with a raman library using machine learning. ACS Nano 17, 21251-21261 (2023).
Kruse, K., Hug, C. B., Hernández-Rodríguez, B. & Vaquerizas, J. M. TADtool: visual parameter identification for TAD-calling algorithms. Bioinformatics 32, 3190-3192 (2016).
Huber, F., van der Burg, S., van der Hooft, J. J. J. & Ridder, L. MS2DeepScore: a novel deep learning similarity measure to compare tandem mass spectra. J. Cheminf. 13, 84 (2021).
McCarthy, M. & Lee, K. L. K. Molecule identification with rotational spectroscopy and probabilistic deep learning. J. Phys. Chem. 124, 3002-3017 (2020).
Skinnider, M. A., Wang, F., Pasin, D., Greiner, R., Foster, L. J., Dalsgaard, P. W. & Wishart, D. S. A deep generative model enables automated structure elucidation of novel psychoactive substances. Nat. Mach. Intell. 3, 973-984 (2021).
Ji, H., Xu, Y., Lu, H. & Zhang, Z. Deep MS/MS-aided structural-similarity scoring for unknown metabolite identification. Anal. Chem. 91, 5629-5637 (2019).
Ji, H., Deng, H., Lu, H. & Zhang, Z. Predicting a molecular fingerprint from an electron ionization mass spectrum with deep neural networks. Anal. Chem. 92, 8649-8653 (2020).
Yang, Z. et al. Cross-Modal retrieval between ¹³C NMR spectra and structures for compound identification using deep contrastive learning. Anal. Chem. 93, 16947-16955 (2021).
Zhang, J., Terayama, K., Sumita, M., Yoshizoe, K., Ito, K., Kikuchi, J. & Tsuda, K. NMR-TS: de novo molecule identification from NMR spectra. Sci. Technol. Adv. Mater. 21, 552-561 (2020).
Sridharan, B., Mehta, S., Pathak, Y. & Priyakumar, U. D. Deep reinforcement learning for molecular inverse problem of nuclear magnetic resonance spectra to molecular structure. J. Phys. Chem. Lett. 13, 4924-4933 (2022).
Vaswani, A. et al. Attention is all you need. Preprint at https://doi.org/10.48550/arXiv.1706.03762 (2017).
Heller, S. et al. InChI-the worldwide chemical structure identifier standard. J. Cheminf. 5, 1-9 (2013).
Weininger, D. SMILES, a chemical language and information system. 1. Introduction to methodology and encoding rules. J. Chem. Inf. Comput. Sci. 28, 31-36 (1988).
Zou, Z. et al. A deep learning model for predicting selected organic molecular spectra. Nat. Comput. Sci. 3, 957-964 (2023).
Linstrom, P. J., Mallard, W. G. The NIST Chemistry WebBook: A chemical data resource on the internet. J. Chem. Eng. Data 46, 1059-1063 (2001).
LeCun, Y. et al. Gradient-based learning applied to document recognition. P. IEEE 86, 2278-2324 (1998).
Li, Z., Liu, F., Yang, W., Peng, S. & Zhou, J. A survey of convolutional neural networks: analysis, applications, and prospects. IEEE T. Neur. Net. Lear. 33, 6999-7019 (2022).
Gu, J. et al. Recent advances in convolutional neural networks. Pattern Recogn. 77, 354-377 (2018).
Chickering, D. M. Optimal structure identification with greedy search. J. Mach. Learn. Res. 3, 507-554 (2002).
Wei, Z. & Hao, J.-K. A threshold search based memetic algorithm for the disjunctively constrained knapsack problem. Comput. Oper. Res. 136, 105447 (2021).
Niu, Z., Zhong, G. & Yu, H. A review on the attention mechanism of deep learning. Neurocomputing 452, 48-62 (2021).
Soydaner, D. Attention mechanism in neural networks: where it comes and where it goes. Neural Comput. Appl. 34, 13371-13385 (2022).
Lin, Z. et al. A structured self-attentive sentence embedding. Preprint at https://doi.org/10.48550/arXiv.1703.03130 (2017).
Zhang, K., Sun, M., Han, T. X., Yuan, X., Guo, L. & Liu, T. Residual networks of residual networks: multilevel residual networks. IEEE T. Circ. Syst. Vid. 28, 1303-1314 (2018).
Tolstikhin, I. O. et al. Mlp-mixer: An all-mlp architecture for vision. Advances in neural information processing systems 34, 24261-24272 (2021).
Zhu, D., Lu, S., Wang, M., Lin, J. & Wang, Z. Efficient precision-adjustable architecture for softmax function in deep learning. IEEE Trans. Circuits Syst. II Express Briefs 67, 3382-3386 (2020).
Zhang, Z., Sabuncu, M. Generalized cross entropy loss for training deep neural networks with noisy labels. Preprint at https://doi.org/10.48550/arXiv.1805.07836 (2018).
Roussy, T. S. et al. An improved bound on the electron’s electric dipole moment. Science 381, 46-50 (2023).
Landrum, G. RDKit: Open-source cheminformatics. https://www.rdkit.org (accessed 2021)

There is NO Competing Interest.

ExtendedDataFigures.docx

Download PDF

Version 1

posted

You are reading this latest preprint version

A Machine Learning Protocol to Directly Translate Molecular Infrared or Raman Spectra into Molecular Formula

Status:

Version 1

Abstract

Figures

Introduction

Results

Declarations

References

Additional Declarations

Supplementary Files

Status:

Version 1