Background: The investigation of epistasis becomes increasingly complex as more loci are considered due to the exponential expansion of possible interactions. Consequently, selecting key features that influence epistatic interactions is crucial for effective downstream analyses. Recognizing this challenge, this study investigates the efficiency of Relief-Based Algorithms (RBAs) in detecting higherorder epistatic interactions, which may be critical for understanding the genetic architecture of complex traits. RBAs are uniquely non-exhaustive, eliminating the need to construct features for every possible interaction and thus improving computational tractability. Motivated by previous research indicating that some RBAs rank predictive features involved in higher-order epistasis as highly negative, we explore the utility of absolute value ranking of RBA feature weights as an alternative method to capture complex interactions. We evaluate ReliefF, MultiSURF, and MultiSURFstar on simulated genetic datasets that model various patterns of genotype-phenotype associations, including 2-way to 5-way genetic interactions, and compare their performance to two control methods: a random shuffle and mutual information.
Results: Our findings indicate that while RBAs effectively identify lower-order (2 to 3-way) interactions, their capability to detect higher-order interactions is significantly limited, primarily by large feature count but also by signal noise. Specifically, we observe that RBAs are successful in detecting fully penetrant 4-way XOR interactions using an absolute value ranking approach, but this is restricted to datasets with a minimal number of total features.
Conclusions: These results highlight the inherent limitations of current RBAs and underscore the need for enhanced detection capabilities for the investigation of epistasis, particularly in datasets with large feature counts and complex higherorder interactions.