In the present study, the genomic prediction performance of two linear parametric methods (BL and rrBLUP) and a non-parametric Machine learning method (LightGBM) were compared. The comparative analysis was carried out using the genomic selection approach in corn hybrids from crosses between inbred lines from the company LongPing High-Tech in the locations of Jataí-GO, Rolândia-PR and Sorriso-MT in Safrinha 2022. Two important traits of corn crops were evaluated in this study: Grain Yield (GY) and Grain Moisture (GM). It is worth mentioning that moisture at harvest is used as an estimator of physiological maturity (Sala et al., 2007), since all plots were harvested on the same date.
The criteria used to compare the methods were: Prediction accuracy, defined as the correlation between observed and predicted phenotypic data; T10Recall, which indicates the proportion of hybrids from the validation population that were among the 10% most productive for GY and 10% precocious for GM, considering both prediction and field evaluation; in addition to computational performance in training GY and GM models.
The prediction accuracies obtained from the BL, LightGBM and rrBLUP models for the two traits evaluated in the three locations (Jataí-GO, Rolândia-PR and Sorriso-MT) were medium-high, ranging from 0.386 to 0.680 for GY, and 0.653 to 0.781 for GM (Table 2). Similar results were found by Terraillon et al. (2023), which using data from corn hybrids evaluated within the same year obtained predictive accuracies that ranged from 0.300 to 0.710 for GY and 0.490 to 0.830 for GM.
The highest estimates of predictive accuracy for GY and GM were observed for Rolândia-PR. Sorriso-MT and Jataí-GO had the lowest average accuracy for GY and GM respectively. In general, for the two traits evaluated in the three locations, it was observed that with the highest heritabilities, the predictive accuracies were also higher. This behavior is expected and is in line with the work reported by Liu et al. (2018), in which, working with genomic selection for grain yield in corn, they found that heritability has a positive correlation with prediction accuracy. Also, in agreement with the results found here, Hayes et al. (2010) showed that by increasing heritability from 10–90%, predictive accuracy was also increased from 0.35 to 0.75.
Considering the three locations evaluated, it was observed that the highest prediction accuracies for GY were obtained by the additive-dominant model (BL), and the lowest accuracies were obtained by the additive model (rrBLUP). The average range between these two models in terms of predictive ability for GY was 0.050.
For GM, the BL model also had the highest predictive accuracies compared to the additive model (rrBLUP), but the difference was not statistically significant. These results indicate that the genetic effect of dominance considered in the BL model contributed more effectively to the prediction of GY. Very similar results were found in the study carried out by Ferrão et al. (2020), who comparing additive and additive-dominant models in the genomic selection of corn hybrids, found greater prediction accuracies from models incorporating the dominance effect, especially for the grain yield trait.
Because of heterosis in maize hybrids, the incorporation of non-additive effects in genomic prediction models can contribute considerably to increase prediction accuracy (Fritsche-Neto et al. 2021), but this superior performance also depends on the genetic architecture and complexity of the trait of interest. This was demonstrated by Bernardo (1996), who, when evaluating the prediction accuracy for different traits in corn, found that the inclusion of the dominance effect in the modeling was not consistent for all traits studied.
The influence of each model's genetic effects on genomic prediction is related to the heritability of the trait (Zhang et al. 2017). In this study, GM heritability values were higher than GY values, for all sites evaluated. This indicates that the proportion of additivity effect on the phenotypic variance of GM is greater than that in GY. This proportion may explain the smaller differences found between the predictive accuracies of the additive (rrBLUP) and additive-dominant (BL) models for GM, but greater differences for GY.
Another approach that has been applied in genomic selection studies is based on Machine Learning (ML). The algorithms of this approach can capture non-additive effects without requiring any assumptions about the genetic architecture of the evaluated trait. This is a result of its high power in capturing complex and non-linear patterns (Abdollahi‑Arpanahi et al. 2020; Wang et al. 2018). In this study, the genomic prediction performance of the ML method (LightGBM) was evaluated and compared to the two conventional statistical methods BL and rrBLUP. In terms of prediction accuracy, the LightGBM model was equivalent to the BL and rrBLUP models in predicting GM for Rolândia-PR (0.759) and Sorriso-MT (0.681). There was a statistical difference only for Jataí-GO (0.653), where LightGBM had a predictive performance equivalent to rrBLUP, but was inferior to the performance of the BL model (Table 2). For GY, its prediction accuracy was equivalent to the BL model for Rolândia-PR (0.685) and Sorriso-MT (0.404), differing only in the prediction of Jataí-GO (0.487). In this location, its performance was like the rrBLUP additive model. These results confirm that for traits containing complex gene action, such as GY, ML models can perform equivalently or even better than conventional statistical methods (Abdollahi-Arpanahi et al. 2020). In agreement with the results found here, Perez et al. (2022) also observed the superiority of ML models when compared to additive linear models for genomic prediction of traits with lower heritability.
Another very useful parameter in evaluating the predictive capacity of genomic prediction models is T10Recall, which indicates the percentage of correspondence between the data obtained in the prediction and the data observed in the field for the 10% of the most productive hybrids for GY, and the 10% earlier for GM. In practice, through this parameter we were able to estimate, for the models evaluated, the success rate of those hybrids that are truly superior in the field. For this parameter the results indicated that, in general, the LightGBM model was inferior to the parametric models (BL and rrBLUP) for Jataí-GO and Rolândia-PR, but this model was superior for Sorriso-MT (Table 3). The BL and rrBLUP models had very similar percentages for Jataí-GO and Rolândia-PR, in both traits (GY and GM). Given the above, there is no clear association between the results obtained for prediction accuracy and T10Recall, but it is worth highlighting the performance stability of the BL model compared to the other two methods for these two parameters.
Table 3
Percentages of hybrids that were above 10% most productive for GY, and above 10% earliest for GM, both in the field result (Observed) and in the prediction (Predicted). The higher the percentage, the greater the correspondence between Predicted and Observed values.
| Jataí-GO | Rolândia-PR | Sorriso-MT |
Models | GY | GM | GY | GM | GY | GM |
BL | 34% | 46% | 38% | 68% | 28% | 41% |
LightGBM | 32% | 40% | 36% | 62% | 28% | 43% |
rrBLUP | 33% | 45% | 37% | 66% | 25% | 39% |
In terms of computational efficiency, the LightGBM model stood out as the most efficient and fastest for the three locations evaluated (Jataí-GO, Rolândia-PR and Sorriso-MT), and in both traits. For the location with the largest amount of training data (Sorriso-MT), this model took less than 20 seconds to estimate the effects of markers for the two traits evaluated (Fig. 2). This excellent computational performance of LightGBM was expected and is justified by the way trees are built in this algorithm, which, unlike other gradient boosting algorithms, is based on leaves instead of a level-based structure (Shi et al. 2022).
Among the two parametric models, the additive model (rrBLUP) presented the best computational performance for training the GY and GM models, being considerably faster than the additive-dominant model (BL). Considering the location with the largest amount of training data (Sorriso-MT), the time difference between the two models was greater than 8,000 seconds (Fig. 2). The low computational performance of the BL model is associated with the method used in statistical inferences, which in this case is based on a Monte Carlo algorithm via Markov Chains (MCMC) (Yang et al. 2020).
Although BL has been largely surpassed by the other two methodologies (LightGBM and rrBLUP) in terms of computational performance for training GY and GM models, this inferiority is mitigated with the faster execution of GBV estimates. Once the model training and validation stage is completed, the time spent predicting the GBV of new selection populations becomes the biggest bottleneck. In this scenario, the BL model becomes the second fastest, taking around 31 seconds less than the rrBLUP model and 21 seconds more than the LightGBM model (Fig. 2). As genomic prediction projects in genetic breeding programs may require the prediction of a few million hybrids, computational time is an important parameter when choosing prediction models.
The BL model showed greater stability in terms of predictive ability. If computational time is not a limiting factor, this is the most suitable method to obtain greater prediction accuracies for situations like those found in this study. In a scenario with a high amount of data for model training, the additive method (rrBLUP) and Machine Learning (LightGBM) can be used to achieve better computational performance, however this use can imply a significant loss of predictive accuracy.
The heritability of traits influences the prediction accuracy of the models. Higher accuracies are obtained with higher heritabilities. The genetic effects resulting from smaller heritabilities can be better captured using additive-dominant and machine Learning models.