From Machine Learning to Transfer Learning in Laser-Induced Breakdown Spectroscopy: the Case of Rock Analysis for Mars Exploration

With the ChemCam instrument, laser-induced breakdown spectroscopy (LIBS) has successively contributed to Mars exploration by determining elemental compositions of the soil, crust and rocks. Two new lunched missions, Chinese Tianwen 1 and American Perseverance, will further increase the number of LIBS instruments on Mars after the planned landings in spring 2021. Such unprecedented situation requires a reinforced research effort on the methods of LIBS spectral data treatment. Although the matrix effects correspond to a general issue in LIBS, they become accentuated in the case of rock analysis for Mars exploration, because of the large variation of rock composition leading to the chemical matrix effect, and the difference in morphology between laboratory standard samples (in pressed pellet, glass or ceramics) used to establish calibration models and natural rocks encountered on Mars, leading to the physical matric effect. The chemical matrix effect has been tackled in the ChemCam project with large sets of laboratory standard samples offering a good representation of various compositions of Mars rocks. The present work deals with the physical matrix effect which is still expecting a satisfactory solution. The approach consists in introducing transfer learning in LIBS data treatment. For the specific case of total alkali-silica (TAS) classification of natural rocks, the results show a significant improvement of the prediction capacity of pellet sample-based models when trained together with suitable information from rocks in a procedure of transfer learning. The correct classification rate of rocks increases from 33.3% with a machine learning model to 83.3% with a transfer learning model.


Introduction
It is generally considered that the matrix effects, both the chemical matrix effect 1 and the physical matrix effect, 2 represent a critical issue in analyses with laser-induced breakdown spectroscopy (LIBS) for either qualitative classification or quantitative determination. 3 Suitable solutions with respect to such consideration become primordially determinant for applications as important as LIBS analysis of rocks in Mars explorations, 4 where the targeted scientific goals, searching for the present and past water activities and the traces of the life, as well as the study of the Mars habitability, [5][6][7] rely, at least partially, on the reliability and the exactitude of the analytical data that one can extract from LIBS spectra recorded by LIBS instruments embarked on Mars rovers. 8 Certainly the diversity of chemical composition of Mars rocks has been studied by the precedent missions, the absence of real samples from Mars, except meteors, requires a large number of laboratory rock standard samples to be prepared with Earth natural rocks or by mixing pure chemical compounds, in order to cover the chemical variety of Mars rocks. It was the purpose of the sets of laboratory standard rock samples prepared and used by the ChemCam team for training and validation of Mars LIBS spectral data processing models. The number of involved samples was first 69, 9 they were further increased to 408 in order to offer a larger covering of the chemical and mineral compositions of Mars rocks. 10 It is useful and important to point out that all the laboratory rock standards were prepared in the form of pressed powder disks, glasses, and ceramics to minimize heterogeneity in the scale of LIBS observation of typically several hundred µm. Such sample preparation leads to obvious differences in surface morphological and physical properties between laboratory standards and real rocks analyzed by LIBS on Mars, differences from which physical matrix effect can result. With this concern, the effect of surface roughness on the hydrogen emission line has been investigated. 11 Our recently published work 12 observed and analyzed the performance of a machine learning-based model 13 trained with a set of pressed rock powder pellets for total alkali-silica (TAS) classification 14 of rocks in their natural state. Important degradation of the model prediction performance compared to the prediction for pellet samples has been observed. Such degradation prevents the models trained with laboratory standards from reliable predictions with LIBS spectra acquired in situ on real rock samples, a situation which can become crucial in an application as we mentioned above, of in situ LIBS analysis of Mars rocks, since we are not yet able to bring back samples from Mars.
In order to search a solution for the discussed issue, in this work, transfer learning has been introduced for LIBS spectral data treatment. Transfer learning is considered in machine learning when the knowledge gained while solving one problem is required to be applied to a different but related problem. 15 Its necessity comes from the fact that a major assumption in many machine learning and data mining algorithms is that the training and the modeltargeted data should share the same feature space and have the same distribution. 16 It is unfortunately not the case for the application scenario that we consider. Moreover, transfer learning has recently emerged as a new learning framework to address the problem of insufficient training data in an application (target domain) with the help of the knowledge learnt from a related application with the facility to get sufficient training data (source domain). 17 Such strategy fits the requirement of LIBS analysis of Mars rocks, where sufficient laboratory standards can be prepared as the source domain, while real Mars rock samples are not yet available as the target domain. According to the specific contents in the "knowledge" to be transferred, we can distinguish feature-representation-transfer, where parts of relevant features respectively from the both source and target domains are further selected for their low sensitivity to the difference between the two domains, to form a common set of transfer features contributing to the training of a transfer learning model. 18 Instance-transfer is another specification of transfer learning where data from the both source and target domains participate in the transfer learning model training, with a conditional testing of the relevance of each data from the source domain for its effect in improving the performance of the model during a cross-validation with data from the target domain. 18 A weight is then applied to each source domain data according to its efficiency in improving the performance of the model for predicting with target domain data.
More specifically, in our experiment, on the basis of the LIBS spectra from a set of laboratory standard samples in the form of pressed pellets, a machine learning-based multivariate model was trained and used to predict the concentrations of major oxides, SiO 2 , N a 2 O and K 2 O, with the spectra from the corresponding natural rock samples, concentrations necessary for the TAS classification of the rocks. The influence of the physical matrix effect was observed. A modified model training procedure was then applied with the adjunction in the training sample set, of a small number of natural rocks with certified concentrations of the major oxides. A transfer learning-based model was thus trained with the implementation of feature-representation-transfer and instance-transfer, and used to predict the concentrations of the major oxides for the rock samples. In the following, we will first briefly present the used samples, the experimental setup and protocol. We describe then the methods of machine learning-and transfer learning-based model trainings, before the performances of the both models being presented and compared in order to draw the conclusion of the work.  figure): S3, S7, S8, S11, S13, S14, S18 and S19. The rest of the 12 samples were used as validation samples (represented by blue dots in the figure).

Samples and Experimental setup Sample
In this work, 20 natural terrestrial rocks were used as samples for LIBS analysis. The rocks were first washed using alcohol and distilled water before any further preparation. All the rocks were analyzed in 2 different forms. Raw rocks: LIBS measurements took place on the natural surface of each rock. Pellets: a part of each rock was crushed and ground into powder by a laboratory mill and then sieved by a 300-mesh screen (grain size ¡50 µm). A binder (microcrystalline cellulose powder) with similar particle size was mixed with the rock powder at a weight ratio of 20%. One gram of the obtained powder was pressed under a pressure of 850 MPa for 30 minutes to form a pellet of 15 mm diameter and 2 mm thickness.
The composition, with especially the concentrations of major oxides, SiO 2 , N a 2 O, K 2 O, of each rock was determined by XRF with the pellets, which allowed presenting the rocks in a TAS diagram as shown in Figure 1. The samples were mounted on a 3D translation stage allowing recording replicate spectra on a sample surface with an ablation crater matrix, while keeping a constant distance between the focusing lens and the sample surface.

Spectrum acquisition.
For each sample, 50 replicate spectra were taken on 50 different ablation craters, and each crater received 25 successive laser shots. The emission spectra induced by the first 5 laser shots were removed in order to avoid surface contaminations, and those induced by the subsequent 20 laser shots were accumulated to produce a replicate spectrum. Such procedure also intended to reduce the difference in surface roughness between the 2 different forms of the samples. In total, 2000 spectra were recorded for the 20 rocks with the 2 different forms for each of the rocks. Typical spectra are presented in the inset of Figure 2 for a pellet and the corresponding rock sample, showing different line intensities of Si, Na and K. Such difference corresponds well to that observed for plasma images induced with the different types of samples.

Data Treatment method.
The general data treatment flowchart used in this work is shown in Figure 3. Several steps can be distinguished: pretreatment, feature selection, machine learning (ML) and transfer learning (TL) model trainings, and model validation. The LIBS spectra from the pellet samples were used as training data, while those from the rocks were separated into a training data set containing 8 samples and a validation data set containing 12 samples.

Data treatment
The pretreatment consisted in the following operations. i) Average in order to reduce experimental fluctuations and the effect of sample inhomogeneity: For each sample, the 50 raw spectra were averaged in a procedure where an averaged spectrum was calculated with a first group of randomly selected 30 spectra. The rest 20 spectra then replaced one by one, a spectrum in the first group, each time the new group of 30 spectra was averaged to generate 20 other average spectra. 21 average spectra were generated for each sample. ii) Baseline correction: an average spectrum was decomposed into a set of cubic spline of undecimated wavelet scales, the local minima were found, then the spline function was interpolated through the different minima to construct the spectral baseline which was removed. 20 iii) Normalization: Baseline-corrected average spectra were normalized with their respective total intensity (the area under the spectrum). iv) Standardization: Standard normal variate (SNV) transformation was respectively applied to the normalized baseline-corrected average spectra of the pellets (20×21=420 spectra) and the training set of the rock samples (8×21=168 spectra).
Within a given sample set, for each channel in a spectrum (22161 channels in total), the variation range of the intensity value over all the samples was transformed into a range with a mean value equal to 0 and a standard derivation (SD) equal to 1. The parameters determined in the standardization of the training set of the rock samples (the mean and the ) were applied to the validation set of the rock samples (12×21=252 spectra) by assuming a same statistical distribution of the data for all the rock samples. The ensemble of the above pretreatments generalized pretreated spectra.

Spectral feature selection.
SelectKBest algorithm 21 was respectively applied to the pretreated spectra of the pellets and the training set of the natural rocks, and successively for the 3 concerned oxides. Within a sample set, for each spectral channel, covariance was calculated between the channel intensity and the concentration of the concerned compound in the corresponding sample, over all the spectra of the sample set. A score was then calculated as a function of the covariance according to the definition given in Reference 13. Different scores, ρ i,j , were thus associated to the spectral channels, with 2 index and a value varying from 1 to 22161, which ranks the them from the lowest score to the highest one. Such procedure was applied to the 2 sample sets (i = 1: pellets, i = 2: calibration set of rocks) and the 3 concerned oxides (j = 1: . A feature selection procedure first identified 100 highest ranked spectral channels respectively for each of the 3 oxides in each of the 2 sample sets.
Pearson's correlation coefficient 22 related to the above mentioned covariance was calculated for the 6 groups of 100 selected features. The results showed that all the selected features had a Pearson's coefficient larger than 0.75.
As we can see in the Figure 3, the 3 groups of 100 features selected for the 3 oxides in the pellet sample set were directly used to respectively train the calibration models for the 3 oxides base on a back-propagation neural network (BPNN). The training algorithm which involved stochastic gradient descent (SGD) and mini-batch stochastic gradient descent (MSGD) optimization iterations, as well as cross validation with randomly generated statistical equivalent data configurations, has been presented in detail in Reference 13. We will not in this paper go into more detail about such training algorithm.
For the transfer learning model training, and according to the principle of feature-representation-transfer discussed above, an ensemble of common selected features was identified between the pellet and the training rock sample sets, by calculating a total ranking index ρ j = ρ 1,j + ρ 2,j . A feature selection procedure then retained the 100 highest ranked features according to the value of ρ j from the highest one to the lowest one, respectively for the 3 oxides. These groups of features were fed into the transfer learning model training algorithm. As an example, the results of feature selection for N a 2 O are shown in Figure   4, although similar behaviors can also be observed in the feature selections for the other 2 oxides. In the insets of Figure 4

Data formatting.
According to the above discussed principles of feature-representation-transfer and instance transfer in transfer learning, spectra from the pellet samples (the source domain) and those from the calibration set of the rock samples (the target domain), with respectively 100 common selected features, participated in the training process. All the 20 pellet samples with their average replicate spectra were initially involved in the training data set. These spectra were organized in a given data configuration where the replicate spectra for each sample were arranged in an arbitrary order. The efficiency of each pellet was tested within an iteration loop where the RET s with and without the spectra from the pellet were compared in order to decide the exclusion or the definitive inclusion of the pellet in the transfer learning model training sample set. It was why the ensemble of replicate spectra associated to one of the 20 pellets was indexed with which went from 1 to 20 ( Figure 5 (a)) for the test the efficiency of all the pellet samples during the training process. Eight chosen rock samples contributed to the transfer learning model training data set (S3, S7, S8, S11, S13, S14, S18 and S19 in Figure 1). In particular, they were used in a cross validation process during the optimization of the neural network. It was why that the spectra of this data set were first organized in different data configurations where each configuration j corresponded to a certain arrangement of replicate spectra for each rock (Figure 5 (a)). Data configurations were all statistical equivalent since the order of a replicate spectrum of a sample was a dummy index. The number of different data configurations were limited to 3 in this work because more configurations did not bring further improvement of the model as tested in the work. For a given configuration j, the spectra were further organized into 5 groups of replicates containing respectively 4, 4, 4, 4 and 5 spectra, respectively. A new index i was introduced to designate a group of replicate spectra of all the rock calibration samples as shown in Figure 5  Model training by optimization. and REC(ij + k) were calculated for the respectively optimized Models for test (ij − k) and (ik + k). These models were then tested using the ensemble of replicate spectra of the rest i th group, i = 1 for example, generating RET (j − k) and RET (j + k), together with the optimized Models for test (j − k) and (j + k). and RET (j + k).
-An outer loop for k = 1 to 20: in this loop the above discussed loop i and loop j were executed for each of the 20 pellet samples successively assigned as the k pellet. For a given pellet, RET (−k) and RET (+k) were compared. If an improvement was observed with the sample k, it was kept in the training sample set, otherwise it was removed. This loop generated a Model for test (k) for each considered pellet sample with the corresponding RET (k). The optimization process finally generated a Model for valuation with a minimized RET .

Model validation.
The resulted transfer learning model was validated by the pretreated spectra from the validation set of the rock samples (12 spectra) with the identified features according to the common selected features between the pellet sample set and the training set of the rock samples. The parameters assessing the performance of the model for prediction, REP and RSD were calculated. These parameters would indicate the performance of the model when used for predictions with LIBS spectra from unknown rock samples.

RESULTS AND DISCUSSION
Analytical Performances with the machine learning model. For the cross-validation optimization in the training process, similarly as for the transfer learning model training, 3 × 5-fold iterations were performed with 3 randomly organized pellet spectrum data configurations and 5 replicate groups of respectively 4, 4, 4, 4 and 5 spectra for each data configuration. A larger number of data configurations did not lead to a significant improvement of the model as shown by the tests in our work. The training process was executed for the 3 concerned compounds resulting in 3 prediction models respectively for SiO 2 , N a 2 O and K 2 O. The obtained results are shown in Figure 6. The extracted parameters for the assessment of the model performances are presented in Table 1.
In Figure 6 and Table 1, We can see that the machine learning calibration models trained with pellet samples present good performances in terms of the usual assessment parameters including r 2 , LOD, REC and RET . As we have pointed out in Reference 12, this indicates an efficient chemical matrix effect correction with machine learning. At the same time, we can remark a large degradation in the performance when the model was validation by the rock validation samples, in terms of REP and RSD. Figure 6 shows that the use of the pellet models for prediction with the spectra from the rock validation samples can lead to systematic bias, with a shift of the linear regression of the validation data with respect to the model as shown in Figure 6  In figure 7, we can only see 4 correct classifications (S1, S2, S5 and S10), corresponding to a correct classification rate of 33.3%. Analytical Performances with the transfer learning model.
In parallel to the results with the machine learning models and in order to review the improvements with the transfer learning models, the calibrations models resulted from transfer learning are shown in Figure 8. The extracted parameters for assessment of the model performances are presented in Table 2. In Table 2, we can see that although the transfer learning models present slightly lower calibration performance in terms of r 2 , the slope, LOD, REC and RET , the performance for prediction for rock samples are significantly improved, especially for REP . This means that the participation of the 8 rock samples in the training data set together with the retained pellet samples efficiently takes into account the physical matrix effect and reinforces the robustness of the models for prediction with rock samples.
Correspondingly in Figure 8, we can see significant reductions of bias and variance of the  The calibration models shown in Figure 8 were used to represent the validation rock samples in a TAS diagram. The obtained result is shown in Figure 9 using the same symbols as in Figure 7. We can see a much improved result conforming the good performance of the transfer learning models shown in Figure 8 and Table 2. A detailed counting shows 10 correctly classified validation rock samples. Only two samples were classified into a wrong field (S4 and S6), although they are very close to borders separating the correct and wrong fields. The rate of correct classification can thus be calculated to be 88.3%.

Notes
The authors declare no competing financial interest.