Data preparation
Machine learning data should be representative of the sample space, a universal set of all possible segregation sites in this work. While the segregation could happen anywhere inside a GB network common in a realistic alloy, the literature usually used bicrystals when preparing the data13,14,22, where the possible sites are limited within the single GB inside the configuration. Even though many such configurations can provide a variety of GBs containing various segregation sites, the obtained data is yet a sufficient sampling for a common GB network due to the exclusion of triple junctions that are more preferable segregation sites23. Here we used a polycrystalline configuration containing 40 randomly oriented grains for each alloy system studied here, with an average grain size of ~ 7 nm (Fig. 1a). Each atom in the GB network, including GBs, triple junctions, and vertices, is considered as a possible segregation site. In addition, the neighboring crystalline atoms of the GB network, where segregation also happens, are considered possible sites as well 14,24.
The local atomic environment is defined as the neighboring atoms within a sphere of radius \({r}_{cut}=3 \text{n}\text{m}\) centered at a segregation site (Fig. 1b). The segregation energy is calculated as \(\varDelta {E}_{seg}^{sol}=\left({E}_{site}^{sol}-{E}_{site}\right)-\left({E}_{sgl}^{sol}-{E}_{sgl}\right)\), with a positive value indicating a segregation tendency. Here \({E}_{site}^{sol}\) and \({E}_{site}\) are the respective total energy of the atomic configuration shown in Fig. 1b after and before the centered matrix atom is replaced by a solute atom, whereas \({E}_{sgl}^{sol}\) and \({E}_{sgl}\) are the respective total energy of a reference single crystal after and before one host atom is replaced by a solute atom 12. Details on how to calculate the segregation energy are presented in the Methods section. Figure 1c shows the distribution of the calculated Ni segregation energy within the GB network of a polycrystalline Ag. Most of the sites in the GB network, including the neighboring crystalline atoms, possess positive segregation energy and thus act as preferable segregation sites for Ni. But many sites inside the GB network also have negative segregation energy, showing an anti-segregation tendency.
Identification of PI features
The PI features were identified based on physical understanding. The first PI feature is the atomic volume at a segregation site usually calculated with Voronoi analysis of the nearest neighbors. This feature indicates the interatomic distance between the segregation site and the surrounding nearest neighbors and has a strong effect on the solute binding energy 9. Atomic volume suitable for the solute atom size tends to have high binding energy and is favorable for segregation, while that too small/large tends to decrease the binding energy and is not favorable for segregation. It is therefore highly expected and has been confirmed that the atomic volume is strongly relevant to the segregation energy22. Atomic volume only tells information about the nearest neighboring atoms. The information about the atoms further away can be described by a disorder factor. This PI feature measures how far the structure of a local atomic environment is away from a perfect crystalline structure or how close it is to a completely disordered structure, i.e. glassy state or amorphous structure. Both theoretical prediction and experimental observation showed that solute segregation and disordering transition promote each other at GBs 25–27. Atomistic simulations 24 also showed that amorphous GB structures have the highest solute composition, suggesting that the sites at highly disordered atomic environments tend to possess high segregation energy. The third PI feature is the atomic energy of the matrix atom at a site. Matrix atoms with higher energy are less stable and more likely to be replaced by solute atoms and tend to possess higher segregation energy. For comparison, SNAP parameters were also extracted from the local atomic environment 16. Designed to accurately model machine learning interatomic potentials, SNAP parameters are expected to give appreciable accuracy while modeling the solute segregation energy.
Performance of PI features
Random forest algorithm was first used to do machine learning on a AgNi alloy system, with Fig. 2 showing that the three PI features, used either separately or collectively, outperform the SNAP features significantly. Atomic volume per se already gives an RMSE of 0.056 eV (Fig. 2a), more accurate than similar modeling of an AlMg system that also used atomic volume as the only feature but with training data obtained from bicrystals 22. Adding a disorder factor into the feature set can reduce the RMSE to 0.0436 eV (Fig. 2b). Putting the three PI features together eventually reduces the RMSE to 0.0415 eV (Fig. 2c). In contrast, three SNAP features only give an RMSE of 0.0722 eV (Fig. 2d), even much higher than that based only on the atomic volume. The predicted statistical distribution of the segregation energy shifts towards a low-energy regime due to the poor performance. While increasing the number of SNAP features decreases the RMSE (Fig. 3a) and 20 SNAP parameters give an RMSE of 0.0456 eV (Fig. 2f) comparable with the results in the literature work 13,21, this RMSE is only as good as the RMSE based on atomic volume and atomic energy (Fig. 3a), still higher than that based on the atomic volume and disorder factor, not to mention the RMSE based on all PI features.
The PI features also outperform the SNAP features on eight other alloy systems in that the same number of PI features always gives lower RMSE than the same number of SNAP features (Fig. 3), with the extent depending on alloy systems. The atomic volume itself behaves as well as 2 SNAP features in AlTi alloy but 13 SNAP parameters in NiZr alloy; The RMSE of all PI features is only as low as three SNAP parameters in AgCu alloy but 18 SNAP parameters in AlNi alloy.
The PI features continue excelling when using support vector machine and linear regression algorithms, albeit with different performance (Fig. 4). In the AgNi alloy system for example, atomic volume itself behaves better than 4 SNAP features using random forest and linear regression but 11 SNAP features using support vector machine. All three features work better than 20 SNAP features using random forest and support vector machine algorithms but only better than 9 SNAP features while using linear regression algorithm.
It is worth noting that SNAP features were added to the feature set in the as-output order from atomistic calculations while investigating the accuracy versus the number of SNAP features (Fig. 3, S1, S2). One may argue that the performance of PI features might result from a possible weak correlation of the preceding SNAP features to the segregation energy. To rule out this possibility, we sorted the SNAP features from the greatest to the least relevance to segregation energy, as indicated by a correlation matrix heat map (Fig. S3), and redo the investigation for AgNi alloy. Results (Fig. S4) similar to Fig. 3a, S1a, and S2a are obtained for the three respective algorithms, indicating that the performance of PI features does not strongly depend on the order of SNAP features.
Since both types of features contain information on the segregation energy, it would be interesting to put them together in machine learning. Figures 3 and S1−S2 show that the addition of PI features, either separately or collectively, significantly improves the performance of SNAP features, in that the combined feature set requires much fewer SNAP features than pure SNAP parameters to reach the same accuracy. In AgNi alloy for example, using random forest algorithm, 2 SNAP features give an RMSE of ~ 0.095 eV (Fig. 3a), adding atomic volume can significantly reduce the RMSE to ~ 0.052 eV, comparable to that of 11 SNAP parameters. Further addition of atomic energy reduces the RMSE to a value lower than that from 20 SNAP features. It works even better by adding all three PI features to the 2 SNAP parameters. In addition, the lowest RMSE is always achieved by the addition of PI features, with only a few SNAP features in many cases (Fig. 5). For example, the lowest RMSE in AgNi alloy is achieved by adding PI features to 20 SNAP features with linear regression but only 3 SNAP features with random forest algorithm. PI features can even do the best without SNAP features while using a support vector machine algorithm.
Conventional wisdom in the literature to achieve accurate machine learning modeling is to include as many features as possible so that any possible information about the segregation energy is included in the training data 13,14,21. This strategy may not live up to the expectations since too many features might lead to overfitting, a manifestation of the curse of dimensionality. In this regard, the efficiency of PI features also manifests itself by hardly showing overfitting that SNAP features suffer a lot. Figures 3, S1, and S2 show that the RMSE usually decreases with the number of SNAP features, whether used separately or combined with the PI features. This decrease, however, is not monotonic sometimes, indicating an overfitting problem with too many features (Figs. 3c, 3i, S1c, S1d, and S1e). Overfitting is unclear in the literature due to the lack of comparative study but should be highly likely due to a large number of lumped features. In contrast, PI features almost have no overfitting, with the only exception in AlMg alloy while using support vector machine algorithm, that atomic volume plus atomic energy works slightly better than all PI features.
Source of redundant information
The performance of PI features over SNAP features strongly suggests that the information of segregation energy is densely compacted in the former as we expected, but spread more widely in SNAP features and probably in the many lumped features in the literature work. In other words, SNAP features contain more redundant information than PI features. On one hand, SNAP parameters are developed to model interatomic potentials in classical atomistic simulations and are supposed to keep the information of the atomic energy, the interatomic force, and the stress tensor of the central atom while extracted from a local atomic environment16, where the argument should also hold for SOAP parameters15. In the calculation of solute segregation energy, however, the only useful information is the energy term. Information about the interatomic force and the stress tensor is inherently retained in SNAP parameters as redundant information in the machine learning data. On the other hand, the segregation energy is calculated based only on equilibrium local atomic configurations. The interatomic forces are all zero or close to within the reference crystalline configuration and the atomic configurations of possible segregation sites. The force information stored in SNAP features is not only useless but also constant zero. Therefore, the local atomic configuration space used in the segregation energy calculations is only a special subset of the universal atomic configuration space that is supposed to be described quite accurately with SNAP or SOAP parameters.
The correlation between the local atomic environment, PI features, SNAP features, segregation energy, etc. can be visualized in a Venn diagram (Fig. 6). All of the closed curves are within the rectangle, meaning that all information is contained in the local atomic environment. Most of the force set is overlapped with the SNAP set, implying that SNAP features can be used to predict the interatomic forces. The segregation energy set is also largely overlapped with the SNAP set, explaining why SNAP features work well in the modeling of segregation energy. However, the overlap only accounts for a smaller area of the SNAP set. The other larger area does not contain information on the segregation energy and acts as redundant information. Therefore, using SNAP features to model segregation energy is like putting a fine timber to petty use, working well but wasting many feature functions and computational resources.
The merit of physics-informed feature identification
One natural strategy to reduce the redundant information is to perform dimensionality reduction, for example, using principal component analysis, on the SNAP features. The accuracy of 3 principal components (Fig. 2e) is higher than that of three SNAP features (Fig. 2d) but lower than that of the 20 SNAP features (Fig. 2f), a fair compromise between accuracy and feature dimension. However, the three principle components still underperform PI features significantly. The accuracy is even lower than that of the atomic volume itself, not to mention all PI features. This inferior performance can be attributed to possible information loss during the feature extraction and subsequent dimensionality reduction process. First, extracting SNAP features from the local atomic environment may lose information, disordering state (Fig. S5) as an example, about segregation energy, resulting in errors in machine learning, even though the SNAP parameters are developed to model interatomic potentials. Dimensionality reduction can condense more information onto the principle components but cannot recover the information already lost when extracting SNAP features. Secondly, dimensionality reduction from SNAP features to principle components will also lose information, as confirmed by that the accuracy of the three principal components is still lower than that of the 20 SNAP features.
In contrast, PI features are directly extracted from the local atomic environment. Despite the inevitable information loss, the information about segregation energy is largely retained by always identifying the important features based on physical understanding. Secondly, each separate feature affects the segregation energy through different physical mechanisms, considerably ensuring feature independence sought by a successful feature extraction method. For example, both atomic energy and disorder factor are weakly correlated with atomic volume (Fig. S3). Though the atomic energy and disorder factor are more strongly correlated, indicating that a disordered structure tends to have higher atomic energy, the independence is still there and the combination of the two features contains more information than each separately, as indicated by that all three PI features almost always give the best prediction of segregation energy. Thirdly, identified based on physical understanding, the PI features help pinpoint the important mechanism for the propensity of solute segregation, while the pure mathematical SNAP or SOAP features do not due to the lack of explicit physical meaning. Fourthly, the extraction of a few PI features requires much less computational resources than SNAP features since both feature extraction and further dimensionality reduction are computationally expensive. Lastly, extracted directly from the local atomic environment, new features (Fig. 6) can always be added to further improve the accuracy with additional computational cost only from the new features, while adding more SNAP features usually requires starting over the whole extraction process and at least doubles the already-high computational cost.
The above comparison showcases the merit of identifying features directly from the local atomic environment based on physical nature. Feature extraction is probably the biggest challenge for machine learning. Inappropriate features or too many ones lead to poor model performance and/or high computational cost. Fortunately, modeling of segregation energy is a physical problem. One should always start by identifying the most important features by making the most of the physical nature. This strategy is like the one used in first-principles calculations based on density functional theory: first calculating the exchange-correlation energy based on local electronic density and then improving the accuracy by further considering the gradient28,29. By making sure the inclusion of important features and adding more relevant features if necessary, one can quickly approach sufficient accuracy with the least number of features. Otherwise, recklessly relying on the power of machine learning will solve the problem but likely with reduced efficiency.
Outlook
The importance of integrating physics into machine learning has been demonstrated by the development of physics-informed machine learning algorithms (deep neural networks, kernel-based regression networks, etc.) while simulating multiphysics problems30. This work demonstrates the importance from a different perspective, i.e. feature identification, by showing that PI features have clear advantages over SNAP features. While the merit manifests itself on all of the nine alloy systems, PI features work best on the AgNi system, the interatomic potential of which happens to be the only potential developed by explicitly including solute segregation energy obtained through first-principles calculations in the potential fitting process 12. It is thus highly likely that the solute segregation energy in this system is more accurate than that in eight other alloy systems. We, therefore, attribute the best performance to the data authenticity for this system and expect better performance if all data is prepared through first principles that by far give the most accurate calculations.