3.1 Hyperparameter tuning
Prior to the progressive training, we primarily focused on optimizing the hyperparameters. The results of the comprehensive optimization are shown in Fig. 1. The optimization objectives are quantitively described as the objective values, which equal the average value of the training and testing errors. Several ANNs with arbitrary combinations of hyperparameters are evaluated in each trial, represented as blue triangles. The optimal ANNs with the lowest objective values over various trials are connected with the solid red line in Fig. 1(a), illustrating the trend of the optimization process. The optimization history shows that the objective values substantially fall by around 85% after about 150 optimization trials, indicating that the algorithm-driven optimization can successfully locate the optimal ANN with the best combination of hyperparameters in the high-dimension search space. The detailed optimization results based on specific hyperparameters are provided in Fig. 1(b), where the colors represent the number of trials (the darker the blue, the larger the number of trial). The learning rate is optimized to be 6.05⋅10− 3, weight decay is optimized to be 1.53⋅10− 5, layers are optimized to be 3, and the Adam optimizer is the best one.
3.2 Progressive training
Following the identification of the best ANN with the best combination of hyperparameters, the next stage is progressive training, which focuses on uncovering the information hidden in the training dataset. The R-value was used in this work to evaluate how well a regression model fit the data. The better the model is fitted, the closer the R-value is to 1. The C11 of the BCC phase was chosen for demonstration, as indicated in Fig. 2. The vertical axis displays the computed value from the first-principles calculations, while the horizontal axis displays the predicted values. Orange circles represent the training data, whereas green circles represent the testing data. We also included the data that hasn't yet been used for training and testing, which is represented by the open grey circles. Meanwhile, the training data, testing data, and entire dataset are all fitted using linear regression, and these fitted lines are depicted as dashed lines in orange, green, and black. The starting training data only contained five unary systems, as illustrated in Fig. 2(a). As expected, the R-value for training, i.e., RTr, is 1.0000, the R-value for testing, i.e., RTe, is just 0.8348, while the R-value for the whole dataset, i.e., Ra, is only 0.6548, indicating that the present model is far from reality. However, with the addition of the binary system data in the second step training, as shown in Fig. 2(b), not only were the unary and binary systems well-fitted (RTr = 0.9986), but the testing ternary systems' performance (RTe= 0.9115) was also noticeably enhanced as compared to the first step. For the entire dataset (Ra=0.8664), there is still some room for improvement. Figure 2(c) presents further training results with the ternary system data added to the training data. The R-value for training data, testing data, and the whole dataset are even more close to 1 (RTr = 0.9971; RTe = 0.9411; Ra = 0.9411), demonstrating that the existing model can be successfully extrapolated to quaternary systems and was well fitted by unary to ternary systems. Finalizing the model required covering all of the original data in the training dataset, the outcomes are displayed in Fig. 2(d) and achieved optimal fitting results (Ra = 0.9977).
As demonstrated individually in Fig. 3(a), (b), (c), and (d), we use the BCC Al-Fe, Fe-Co, Co-Cr, and Cr-Ni systems out of all the ten binary systems to illustrate the actual evolution progress through the progressive training. The orange circles with error bars represent the training data. The green dashed lines reflect the average prediction outcomes, while the green area shows the prediction confidence (PC). The four steps of progressive training, which range in hue from light green to dark green, are shown in these graphs. Only the pure elements are fitted during the first step of training, and hence, the related prediction intervals seem to be the widest, denoting the highest uncertainty. Both the unary and binary systems are fitted during the second training step. And it can be shown that across the subsequent steps of training, the fitting results for these unary and binary systems essentially remain unaltered, indicating that the training process of higher-order systems do not affect the lower-order ones that have already been trained. Additionally, as more data are included, the prediction intervals get smaller and smaller, indicating less and less prediction uncertainty. In general, it can be found that the progressive training used in the current work is similar to the CALPHAD approach.
3.2 Prediction
The final predicted results of various properties (\(\varDelta {H}^{f}\), C11, C12, and C44) for both FCC and BCC are shown in Fig. 4, with the compositions of the 10 binary systems in the Al-Co-Cr-Fe-Ni system. Different markers with error bars (\(\varDelta {H}_{error}^{f}\) = ±1 kJ/mol; Cij,error = ±10 GPa) represent different properties calculated from the first-principles calculations, while the dotted lines represent the final predicted results after the progressive training. The matching colormaps designate each color to describe a particular binary system. These graphs indicate that the computed results from the first-principles calulations and prediction results from the progressive learning are highly consistent after progressive training. Additionally, adopting the dropout strategy avoids overfitting.
Additionally, as shown in Fig. 5, we used the valence electron concentration (VEC) 55–57 to illustrate the predicted properties for high-order systems. Since the concentration interval for each element in the prediction dataset is 0.02 at.%, over 300,000 predicted data points were gathered and presented in these graphs. The system's VEC value may be characterized by Eq. (9):
$$VEC=\sum _{i=1}^{n}{C}_{i}{\left(VEC\right)}_{i}$$
9
,
where Ci and (VEC)i stand for the atomic fraction and VEC value of element i, respectively. The corresponding VEC value of each element can be found in Table 2. The enthalpy of formation (\(\varDelta {H}^{f}\)) as well as six elastic properties (bulk modulus, shear modulus, Young’s modulus, Pugh’s ratio, Poisson’s ratio, and Vickers hardness) for both FCC and BCC were investigated.
Table 2
The VEC value of each element in the Al-Co-Cr-Fe-Ni system
Element
|
VEC
|
Al
|
3
|
Co
|
9
|
Cr
|
6
|
Fe
|
8
|
Ni
|
10
|
Supplementary Materials on “High-entropy materials design by integrating the first-principles calculations and machine learning: a case study in the Al-Co-Cr-Fe-Ni system” |
Guangchen Liu1, Songge Yang1, Yu Zhong1,* |
1Mechanical and Materials Engineering Department, Worcester Polytechnic Institute, 100 Institute Rd, Worcester, MA, 01609, USA |
The bulk modulus describes the resistance to hydrostatic pressure, while the shear modulus describes the resistance against the deformations upon the shear stress. The Pugh's ratio, is often employed to estimate the ductile/brittle property of materials. The Young's modulus explains the response of the material to normal longitudinal stretching stress, and Poisson's ratio describes the behavior of the material during deformation (expansion or contraction). Out of all the properties, only the \(\varDelta {H}^{f}\), B, and G are shown in Fig. 5. The other predicted properties can be determined using the bulk and shear modulus, as described in Eq. (5–8), and are included in the supplementary documents (Figure A 1 and Figure A 2). Phase stabilities are represented by color using two sets of colormaps: the Blues and the Rainbow. For the Blues colormap, the darker the color, the more positive the value of \(\varDelta {H}^{f}\), indicating that the system is a more unstable system according to Eq. (1). The scatters using the Blues colormap represent the materials over the full composition range. It is discovered that the materials with high bulk and shear modulus also display significantly positive \(\varDelta {H}^{f}\) across the whole composition range, indicating that while having outstanding attributes, these candidates are highly unstable. It is noteworthy that that the discordant projections are due to the thermodynamically unstable FCC phase for the Cr rich corner (VECCr = 6). For the Rainbow colormap, the closer a color is to red, the more positive value there is for \(\varDelta {H}^{f}\). The scatters using the Rainbow colormap represent the materials with an elemental concentration of each range from 5 to 35 at.% based on the classic definition of high entropy alloys.
Figure 5 shows that for both FCC and BCC, the VEC of HEAs ranges from 5.8 to 8.6, and the VEC value of the lowest \(\varDelta {H}^{f}\) is 7.06. The \(\varDelta {H}^{f}\) of HEAs for FCC and BCC, respectively, range from − 24.86 to 10.46 kJ/mol and − 25.90 to 10.85 kJ/mol, as shown in Fig. 5(a) and 5(d). The bulk modulus of HEAs for FCC and BCC, respectively, range from 138.12 to 214.08 GPa and 151.98 to 224.81 GPa, as shown in Fig. 5(b) and 5(e). In general, as the VEC rises, the bulk modulus and the \(\varDelta {H}^{f}\) both rise. The shear modulus of HEAs for FCC and BCC, respectively, range from 78.92 to 117.77 GPa and 86.37 to 110.05 GPa, as shown in Fig. 5(c) and 5(f). In addition to the bulk modulus, the shear and \(\varDelta {H}^{f}\) modulus also rise when the VEC does, though the effects are less pronounced.
Following the establishment of the Al-Co-Cr-Fe-Ni system's whole compositions-properties map for both the FCC and BCC phases, a comprehensive property database covering the full composition range of the Al-Co-Cr-Fe-Ni system was developed. The database can provide detailed information, including phase stabilities and elastic properties under desired compositions. Additionally, the corresponding software program, HEA_ML, was developed to put this database into practice. The latest release of this software’s demo version can be downloaded through the link below: https://github.com/IMPDGroup/HEA_ML.
3.3 Analysis and screening
The property maps with the change of VEC values for FCC and BCC is shown in Fig. 5. Only about 10% of the total predicted data meet the HEA criteria. Further examination of these HEAs focusing on two crucial elastic properties, bulk modulus (B) and shear modulus (G), is shown in Fig. 6. Pugh's ratio (k), or B/G, is also depicted in these graphs with a critical value of 1.75 and is illustrated with black dashed lines, suggesting that materials tend to be brittle when k is less than 1.75 and ductile when k is larger 58. The colorful circles plotted in these graphs represent phase stabilities and elastic properties of HEAs. The distribution histograms of bulk and shear modulus for the FCC and the BCC are also plotted. Overall, it can be seen that each property follows a normal distribution. The bulk modulus for the FCC is mainly concentrated between 160 and 180 GPa, whereas the shear modulus is primarily concentrated between 85 and 95 GPa. The bulk modulus for the BCC mainly falls between 180 and 200 GPa, whereas the shear modulus primarily falls between 90 and 100 GPa. For both FCC and BCC phases, the majority of the HEAs are in the region where \(\varDelta {H}^{f}\) is negative, which means that the more negative the \(\varDelta {H}^{f}\), the more likely the HEAs will be stable and show up. Meanwhile, these HEAs are commonly observed above the dashed line, indicating that these HEAs also exhibit ductile behavior. Moreover, it can be found that the regions with large moduli tend to be highly thermodynamically unstable.
Additional screening of these HEAs was conducted to do the further investigation. The screening criteria include: (1) To guarantee that the HEAs are thermodynamically stable at 0 K, the \(\varDelta {H}^{f}\) should not be positive; (2) Since the bulk modulus varies more noticeably than the shear modulus, only the top 10% of the whole predicted HEAs dataset based on the bulk modulus should be reserved. Eventually, 2,461 candidates for FCC and 2,336 candidates for BCC are identified in the graphs on the left hand of Fig. 7, sorted by \(\varDelta {H}^{f}\). The stack plots for each candidate's compositions are shown in the lower regions, while the above regions display the corresponding phase stabilities and elastic properties. These data were then statistically analyzed using boxen plots, as shown in the graphs to the right. Different boxes use different percentile values, and the largest box uses the quartile values (25th and 75th percentile). For FCC, the concentrations for Al, Fe, Co, Cr, and Ni within the interquatile range are between 10 to 16 at.%, 8 to 22 at.%, 26 to 32 at.%, 10 to 18 at.%, and 24 to 32 at.%, respectively, with an average value of 13.18 at.%, 16.00 at.%, 28.77 at.%, 15.01 at.%, and 27.05 at.%, respectively. The bulk modulus, shear modulus, and \(\varDelta {H}^{f}\) within the interquatile range (from 25th to 75th percentile) are between 183.53 to 188.55 GPa, 96.02 to 104.90 GPa, and − 9.45 to -1.88 kJ/mol, respectively, with an average value of 186.42 GPa, 100.52 GPa, and − 5.92 kJ/mol. For BCC, the concentrations for Al, Fe, Co, Cr, and Ni within the interquatile range are between 12 to 16 at.%, 10 to 28 at.%, 20 to 32 at.%, 10 to 24 at.%, and 20 to 32 at.%, respectively, with an average value of 13.55 at.%, 19.06 at.%, 25.43 at.%, 17.08 at.%, and 24.88 at.%, respectively. The bulk modulus, shear modulus, and \(\varDelta {H}^{f}\) within the interquatile range are between 192.10 to 196.70 GPa, 94.58 to 97.74 GPa, and − 4.78 to -1.00 kJ/mol, respectively, with an average value of 194.86 GPa, 96.23 GPa, and − 3.23 kJ/mol. Through the analysis of these data, we discovered that for both FCC and BCC, the concentration of Fe fluctuates the most while Al varies the least, indicating that Fe has the least influence on the properties of HEAs while Al has the greatest influence. This result agrees well with Chuan’s work 59, which found that Fe has a neutral effect on both FCC and BCC phase while Al acts as a strong stabilizer especially for BCC phase. Unexpectedly, we also discovered that the ideal composition for both the FCC and the BCC falls into a rather close region on which future HEAs design should be heavily focused. Regarding the properties, it can be found that after the screening, the optimal HEAs candidates dominated by FCC exhibit better phase stability and more balance elastic properties than the optimal HEAs candidates dominated by BCC. Additionally, these elite HEAs candidates' elastic properties demonstrate that the materials have ductile behavior for both FCC and BCC.