Heap Based Optimizer Algorithm for Solving Feature Selection Problems in High-Dimensional Cancer Microarray Data

doi:10.21203/rs.3.rs-1402946/v1

PDF

Research Article

Heap Based Optimizer Algorithm for Solving Feature Selection Problems in High-Dimensional Cancer Microarray Data

https://doi.org/10.21203/rs.3.rs-1402946/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted 14 Mar, 2022

You are reading this latest preprint version

Feature selection (FS) is an important preprocessing step that has been commonly used in several fields to improve the performance of learning algorithms. In the field of medical data mining, a huge number of features are used in diagnosing disease, but these features have a lot of non-relevant weak correlations and redundant characteristics, which causes a number of problems that adversely affect diagnostic predictive accuracy. Work on FS has grown extensively many fields due to increased demand for methods that can reduce the dimensionality of data by choosing the best subset of features according to specific criteria in order to maximize prediction accuracy and minimize irrelevant features. In recent times, metaheuristics have been preferred over conventional optimization methods for solving FS problems in order to try to obtain an almost optimal solution in a finite time. Metaheuristics are general-purpose algorithms that can be used to solve almost any optimization problem because they generate “appropriate” solutions in a reasonable amount of time, which is especially useful when seeking to solve complicated problems. Many popular implementations have shown the utility of metaheuristics in different ways by contrasting their performance on well-known problems with that of other algorithms or applications .There are many metaheuristic algorithms in the literature such as those based on swarm intelligence, including particle swarm optimizations and ant colony optimization. The major objective of this research is to provide an increased degree of accuracy to resolve FS problems by conducting different experiments using a metaheuristic algorithm, namely the heap-based optimizer algorithm (HBO). The HBO is used with a k-nearest neighbor classifier in a wrapper to improve the FS process. The performance of the proposed method is evaluated and compared against seven approaches in the literature that are applied on nine high-dimensional data sets that contain, a low number of samples and multiple classes. The findings reveal that the HBO decreases the number of features for classification tasks, and is able to achieve high accuracy in two data sets as compared to the other approaches, the BHBO achieved the best convergence speed as compared to the competing methods. It is therefore concluded that the proposed HBO method can be used to optimize the FS process, whether in terms of classification accuracy or selection size.

Feature selection

heap based optimizer

high-dimensional data

metaheuristics

medical diagnosis.

An increase in dimensionality increases the number of potential cases in the data space, making the data scattered. Machine learning algorithms are applied to these high-dimensional problems, in which this number typically increases exponentially with regard to the number of features[1]. The extensive use of medical data a large quantity of the high-dimensional data. Such data hugely increases both the search area and time complexity that needs to be covered in the data processing stage, regression production, classification, and an entirely intractable clustering. In addition, high-dimensional data sets also include a lot features that are irrelevant or redundant, the presence of which degrades the efficiency of the classifier [2].

More significantly, not all characteristics are helpful. Irrelevant characteristics give misleading information that contributes to degradation in learning performance, unlike relevant characteristics that provide meaningful information about the learning process[3]. Also, the same or similar information about the learning task is supplied by redundant features. Therefore, FS has been suggested as a way to decrease the number of features by eliminating redundant and irrelevant features in order to enable the better handling of high-dimensional data sets[4–6].

Feature selection is often used to enhance many processes in machine learning. However, in a large search space, FS is a difficult task[7]. If the number of basic characteristics is n, the total number of potential subsets of characteristics is 2ⁿ, which grows exponentially with regard to the number of characteristics[8]. Feature selection is an NP-hard problem that makes it impossible to perform an exhaustive search [9].

In several applications, advancements in technology have resulted in larger and larger data sets in terms of the number of cases and the number of predictors. When the number of predictors is greater than the number of observations, a big problem arises in respect of the estimation of high-dimensional data[10, 11]. Some predictors can be redundant, irrelevant, or detrimental to model training in these circumstances. Related predictors do not participate in model training, whereas redundant predictors provide knowledge that has already been provided by other predictors[12]. These predictors, in particular, increase computation time and degrade the output of the classification/regression model unnecessarily. Therefore, selecting a smaller subset of predictors through finding the most-related predictors is crucial because it reduces the amount of time required for data collection and computing while also avoiding over fitting in prediction models[13]. Feature selection methods were developed to filter out obsolete and redundant predictors in order to obtain the smallest and most successful subset of predictors, which decreases computation time in addition to increasing prediction accuracy [14]

Despite some developments, the current FS methods for large feature sets are still insufficient[15]. They are technically feasible, but far from optimal, or they are optimum or nearly optimum but can not handle the computational complexity of FS problems of realistic, or real-world size[4]. Therefore, more research is needed to develop more efficient methods, and scientists are now trying to work out how to use another algorithm with the learning model (classifier) to improve the obtained results. One approach that has been employed is the use of a single metaheuristic algorithm as an optimizer for the selection of features, with a view to delivering really good results, and more effective computation [16–18].

Metaheuristic algorithms are a kind of random algorithm that is utilized to give the optimal solutions achievable. In a short amount of time, metaheuristic algorithms can find successful (near-optimal) solutions to difficult optimization problems. Metaheuristics are a type of estimate optimization algorithm that can help you get away from the local optimum and can be used in engineering problems[19]. Metaheuristic algorithms have recently been developed in many categories. There has not been a consistent and detailed definition of the word metaheuristic until recently, and different definitions have been suggested [20–22].

Thus far, several metaheuristics-based methods have been applied for FS, such as the monarch butterfly optimization algorithm [23], the genetic algorithm (GA) [24], the binary cuckoo search (BCS) algorithm [25], the hybrid whale optimization algorithm (HWOA) with simulated annealing (SA) [26], self-adaptive differential evolution for FS in hyper spectral image data [27], harmony search [28], Tabu search [29], and ant colony optimization for FS in intrusion detection [30].

As indicated above, there are many in-depth research studies that have been conducted in the field of metaheuristics to solve FS problems, and it is the aim of this study to contribute to this research area[31]. Therefore, in this study, a metaheuristic strategy for resolving FS is proposed that uses, the heap-based optimizer (HBO) algorithm. The binary heap is a popular heap implementation, a data structure for the technique for sorting the heap sort contents, was created in 1964 by W. J. Williams. Heaps are also important in some efficient graph algorithms, such as Dijkstra’s algorithm.

The aim of this research is to obtain increased accurate findings the method applies the HBO algorithm which forms as an optimizing for minimizing any unrelated attributes. Furthermore, this study aims to find near-optimal solutions and achieve a balance between exploration and exploitation in order to attain high convergence speed.

The rest of this paper is organized as follows: In Section 2, the related works for FS problems are described. Then, in Section 3, the HBO algorithm is discussed in detail. This is followed by a demonstration of the proposed method of using HBO for FS in Section 4. After that, in Section 5, the experimental results are presented and analyzed. Finally, in Section 6, some conclusions are drawn, as well as some suggestions for future enhancements.

Metaheuristic algorithms were utilized successfully to solve difficult optimization problems and present suitable solutions in an acceptable time. In this section, the most significant works on FS will be discussed. This problem is solved using a variety of metaheuristic algorithms. At first, Li et al.[32] proposed a dividing-based multi-objective evolutionary algorithm (DMEA-FS) for large-scale feature selection. To begin, four new objectives for exploring the optimal feature subsets are created. Besides that, in DMEA-FS, they build two wrapper frameworks for high accuracy and a filter for small computational costs. Second, two novel rapid convergence recombination approaches are developed. For specifically related variables, mapping-based variable splitting is demonstrated. Third, triangle-approximating decision-making dependent on minimum Manhattan distance is suggested for aiding the user’s determination with/without preference knowledge. The outcomes, efficiency, and decision-making capabilities of DMEA-FS are assessed, and its yields are compared to four approaches. The extensive tests show that W-DMEA-FS outperforms various state-of-the-art FS algorithms.

In another study, Wang, Zhang [33] suggested to minimize the volume of samples utilized in the evolutionary procedure, a selection strategy of representative samples, called K that indicates differential selection dependent on clustering, and a ladder-like sample utilization strategy is suggested within the system. An algorithm for selecting features, called fast multi-objective and bee colony FMABC-FS, is suggested by integrating into the system an improved bee Colony Algorithm particle update dependent model.As well as, Aljarah et al.[34] adopted two critical components by the modified slap swarm algorithm (SSA): the complex, time-varying approach and the best local solution. Multi-objective dynamic slap swarm algorithm (MODSSA) is comparable to MOSSA and other MOEA specifications, which include multi-objective particle swarm optimization (MOPSO), multi-objective evolutionary algorithm based on decomposition (MOEA / D), and non-dominated sorting genetic algorithm II (NSGA-II). The algorithm obtained important results over the 13 data sets, as for the average number of errors & g-mean. And where it is done rather than relying on one leader, it uses several leader slaps & several sub swarms.

Furthermore, Hegazy et al.[35] produced a new technique identified as improved slap swarm algorithm (ISSA) is being tried in the task of selection of features. The suggested optimizer has outperformed other classification accuracy optimizers, becoming the 2nd-fastest optimizer and the one that has chosen the least number of features. The results showed that the integration of the weight parameter inertia into the SSA algorithm achieves better efficiency than other algorithms. In another research, Mohmmadzadeh [36] introduced a new hybrid model with a mixture of WOA & flower pollination (FPA) to resolve the problem of the selection of features. Studies on ten sets of data set features taken from the UCI data repository were performed within the first phase. In the 2nd stage, by detecting spam emails, they tried to check the efficiency of the suggested model. Regarding the average selection size and classification accuracy, the suggested algorithm was able to execute more accurately than other simple metaheuristic algorithms.

In recent study, Tumar et al [37] implemented an Enhanced Binary Moth Flame Optimization (EBMFO) with Adaptive Synthetic Sampling (ADASYN) for software problems prediction. BMFO is used as a subset of the wrapper attribute while ADASYN improves the input dataset and fixes the unstable data set. EBFMO’s performance with V-shape (TFs D 3) outperforms other literature versions and other versions. The value of the feature selection algorithm for classification problems is clear from the results obtained. In addition, Agrawal et al.[38] proposed to increases the classical WOA’s capacity for discovery and exploitation. Like a variation operator, the approach uses quantum bit representation of population individuals and the quantum rotation gate operator. The effectiveness of the proposed approach is contrasted with traditional WOA and well-known swarm and quantum algorithms with 14 datasets from varied domains. The improved efficiency of the quantum approaches compared to the classical versions can be due to improved Q-bit representation and improved exploration and exploitation.

In another study, Abdel-Basset et al.[39] proposed a novel algorithm of grey wolf optimizer (GWO) combined with a two-phase mutation. The first mutation aims to minimize the number of selected features considering the account of high precision of classification. The 2nd step tries to incorporate more features using a limited likelihood to improve the precision of the classification. The results demonstrate how the mutation operator increases the flexibility and effectiveness of the algorithm. Also, Nourmohammadi-Khiarak et al.[40] produced a competitive algorithm to define an optimum subset of features, imperialist competitive algorithm with modified fuzzy min – max classifier (ICA-MFMCN) is given. Performance of the proposed classification ICA-MFMCN is approved and recognized a real credit set selected from a UCI dataset. Since the selection of features in data mining activities is epidemic, more existing application of algorithms on datasets in other regions can be a topic of research.

Furthermore, Chen, Li [41] suggested a hybrid variable neighborhood searching (VNS) and evaluation for distributing method (EDA) by the most elite population protocol to define the ideal subset of features. Variable neighborhood search is used to maximize periodicity, prevent premature convergence and save in the search process from the local optimum probabilistic model tries to pick the distribution for the possibility of viable solutions skewed in the direction of the optimum global. The suggested methodology was tested on China’s real-world credit and publicly accessible credit data sets. As well as, Meera and Sundar [42] implemented with the system using a GE Naïve Bayes variant would have an accuracy of classification in this article, for attainment. Minimum processing time for the query and database load reduction PSO, PSO-GE, and Grammatical Evolution (GE) these hybrid a methods were used. This method can aid users to get the necessary facts from a large transaction database in a simplified way the authors add.In addition, Alsaeedi, Albukhnefis [43] suggested Extended Particle Swarm Optimization (EPSO) for FS of high dimensional biomedical data EPSO model suggests a novel search mechanism that could theoretically improve PSO’s search process. The model proposed was compared with PSO and classification of all features. The suggested EPSO was outperformed other models experimentally and has proved to be a promising solution for optimization.

As mentioned, FS is a complex problem that usually requires intelligent ways to deal with datasets for the purpose of selection the most suitable attributes relating to the research problem. This problem is solved using a number of algorithms after presenting literature. The majority of these algorithms were used several times, either on their own or in combination with several other algorithms, or by improving the same algorithm aim of providing better results in this area. Furthermore, since no singular approach could be effectively implemented to solve any type of problem, no method is best in all instances because every method has benefits in particular areas of concern. Outcomes are dependent on the nature and scope of the problem.

As a response, a new strategy is used in this study to decrease repetitive and noisy characteristics in gene expression and create resolutions with more dependable findings in a reasonable amount of time. The suggested method relies on the HBO, which is one of the most recent metaheuristic algorithms. It has been employed in a number of applications and has proven to be capable of dealing with a wide range of challenges in engineering optimization problems. It also has the capacity to deal with the problems of real-world optimization, as well as a high degree of convergence and the ability to provide high-quality performance [44–49].

In 2020, researchers created a new metaheuristic approach in which the concept of the corporate rank hierarchy (CRH) of firms was mapped, and this approach was named the heap-based optimizer or HBO [50]. The purpose of using the heap data structure in FS is to simulate CRH. The heap is a tree-based nonlinear data structure that is commonly used to implement a priority queue. The heap data structure is utilized for CRH mapping, and the solutions are then arranged and listed in a hierarchical form that depends on their suitability. It should be noted that utilization of the hierarchy here is done in a very special way in the process of updating the algorithm [50].

People who work together for a public objective arrange themselves into a CRH to achieve the required work, efficiently and effectively. This concept inspired the development of a novel optimization algorithm, named the heap optimization algorithm (HOA), which assesses the suitability of search agents [50]. The mathematical modeling of the HOA is based on three pillars: the engagement of employees with their direct boss, peer relationships, and employee involvement. The authors of the initial study showed that the HOA was better than or comparable to algorithms in the literature. The main reason for its success is that it is inspired by social behaviors, where the abovementioned three forms of worker behavior are modeled by the suggested algorithm intelligently. Therefore, the suggested optimization approach could be revised and modified to deal with a variety of smart system problems in many fields. Hence, the HBO, which evolved from the HOA, can enable the resolution of a range of real-world resource planning issues, intelligent system design challenges, network optimization, robotics tracking preparation, packaging problems, and vehicle steering applications [51].

The present study decomposes the mapping of the whole HBO concept into four steps:

Step 1: Modeling the corporate CRH.

Step 2: Modeling mathematically the interactions between subordinates and boss.

Step 3: Modeling mathematically the interactions between colleagues.

Step 4: Modeling mathematically an employee’s self-commitment to completing a mission.

3.1 Modeling the corporate rank hierarchy

We decided to model the CRH using a heap data structure because of the design of the heap itself. The heap can be described as a tree-shaped data structure and has the following characteristics:

We consider the heap to be a complete tree when every level of the tree is full, excluding perhaps the final one, and the nodes in the final level in the extreme left, the tree is said to be complete.
In min–heap, every parent node’s key is either < = keys (children), the condition in max, each parent node’s key will be > = keys (children).

Hence, the population is described as the entire CRH.

A search agent is assigned to every formal designation. A search agent corresponds to a heap node through the implementation. The key (node) represents the search agent's suitability, and the value (node) is the index (search agent) in the population. Figure 1 illustrates the CRH modeling method using a heap data structure, where xi indicates the population’s ith search agent. The landscape of a supposed objective function is represented by the curvature of the objective space, and the search agents are represented on the suitability landscape as per their suitability. A heap is generated by utilizing the suitability of every search agent as a key. The nodes in the heap discover their places in the heap based on their suitability. It must be remembered that for minimization, min–heap must be utilized and for maximization, max–heap can be utilized.

In this thesis, for the first time, a wrapper-based FS approach dependent on the KNN classifier and HBO was developed to deal with high-dimensional, low-sample datasets, as recently suggested. The two main aims of the FS are: limiting the number of features that can be chosen and reducing the classification error rate compared the best binary variant's findings to those from other known methods after that S-shaped and V-shaped transfer functions (TFs) are compared and a sensitivity analysis is performed. Table 1 shows the definitions of the symbols used in HBO.

Table 1

The symbols used in the HBO.
Description	Symbol
Current iteration	(t)
Maximum number of iterations	T
Number of cycles	Γ
Number of cycles γ in T iterations	C
Random number between 0 and 1	( r )
Objective function that calculates the search agent’s fitness	F
k^th part of vector${ }_{{\lambda }}^{\to }$	${{\lambda }}^{\text{k}}$
Vector’s k^th component	${}_{\text{k}}{}^{\to }$
Dimension	D
Population’s i^th search agent	x_i
Probability value derived by TF	$\text{T}\left({\text{x}}_{\text{i}}^{\text{j}}\left(\text{t}\right)\right)$
j^th dimension of the i^th solution at iteration	${\text{x}}_{\text{i}}^{\text{j}}$
Dimension	j^th
Solution	i^th
New binary value	${\text{x}}_{\text{i}}^{\text{j}}\left(\text{t}+1\right)$)

3.2 Mathematical modeling of the relationship between subordinates and immediate boss

In this modeling step of an organization with a centralized structure, upper-level rules and policies are enforced, and subordinates obey their immediate supervisor. This behavior is represented by moving each search agent xi closer to its parent node B and using Eq. (1):

$\text{X}{}_{\text{i}}{}^{\text{k}}\left(\text{t}+1\right)={\text{B}}^{\text{k}}+\text{Y}{{\lambda }}^{\text{k}}\|{\text{B}}^{\text{k}}-{\text{X}}_{\text{i}}^{\text{k}}\left(\text{t}\right)\|$	(1)
Where vector ${ }_{{\lambda }}^{\to }$is denoted by the letter${{\lambda }}^{\text{k}}$, which is estimated using the formula in Eq. (2):

$${{\lambda }}^{\text{k}}=2\text{r}-1$$

$${\gamma }=\left|2-\frac{\left(\text{t} \text{m}\text{o}\text{d}\frac{\text{T}}{\text{c}} \right)}{\frac{\text{T}}{4\text{C}}}\right|$$

The high variance or high value of C helps search agents to avoid local optima at the cost of poor exploitation. We choose to compute the balanced value of C as shown in Eq. (4):

$$\text{C}=\left[\frac{\text{T}}{25}\right]$$

3.3 Mathematical modeling of the interactions between colleagues

Colleagues work together to perform official obligations. Coworkers in a heap are nodes of the same rank. As a consequence, the location of a population${ }_{{\lambda }}^{\to }$in relation to a colleague S_r shifts. This is reflected in Eq. (5), which includes a location updating function and permits the search agent to explore the area around

${\text{S}}_{\text{r}}^{\text{k}}$ if f(${\text{s}}_{\text{r}{ }_{ }}^{\to }$) < f ( ${{ }_{\text{x}}^{\to }}_{\text{i}}^{ }\left(\text{t}\right))$and permits it to also explore the region around${ \text{x}}_{\text{i}}^{\text{k}}$.

$$\text{x}{}_{\text{i}}{}^{\text{k}}\left(\text{t}+1\right)=\left\{\begin{array}{c}{\text{S}}_{\text{r}}^{\text{k}}+\gamma \lambda { }^{\text{k}}\left|{\text{S} }_{\text{r}}^{\text{k}}-\right. {\text{x}}_{\text{i}}^{\text{k}}\left(\text{t}\right)\left. \right|, f\left({{\text{S}}_{\text{r}}^{\to }}_{ }\right)<f{}_{{\text{x}}_{\text{i}}}{}^{\to }\left(\text{t}\right)\\ \\ {\text{x}}_{\text{i}}^{\text{k}}+\gamma \lambda { }^{\text{k}}\left|{\text{S} }_{\text{r}}^{\text{k}}-\right. {\text{x}}_{\text{i}}^{\text{k}}\left(\text{t}\right)\left. \right|, f\left({{\text{S}}_{\text{r}}^{\to }}_{ }\right)<f{}_{{\text{x}}_{\text{i}}}{}^{\to }\left(\text{t}\right)\\ \\ \end{array}\right.$$

3.4 Mathematical modeling of an employee’s self-commitment to completing a mission

This step depicts an employee’s self-contribution. The mapping of this step is kept very simple. However, here we propose slight variations to the modeling of this action through maintaining the employee’s prior role in the next iteration, as seen in Eq. (6):

$${\text{x}}_{\text{i}}^{\text{k}}\left(\text{t}+1\right)={\text{x}}_{\text{i}}^{\text{k}}\left(\text{t}\right)$$

In Eq. (5) x_i does not modify its location in the next iteration for its k^th design variable. This behavior enables us to control the pace at which a search agent changes. The following section illustrates how this equation can be used to guide exploration. The equation is used to calculate the parent index of node i is as in Eq. (7) [50].

$$\text{P}\text{a}\text{r}\text{e}\text{n}\text{t}\left(\text{i}\right) = ⌊\left(\frac{\text{i}-1}{2}\right)⌋$$

$$\text{c}\text{h}\text{i}\text{l}\text{d}\left(\text{i},\text{j}\right)=\text{d}⨯\text{i}-\text{d}+\text{j}+1$$

$$\text{d}\text{e}\text{p}\text{t}\text{h}\left(\text{i}\right)=⌈{\text{log}}_{4}(\text{d}⨯\text{i}-\text{i}+1)⌉-1$$

$$\text{p}1=1-\frac{\text{t}}{\text{T}}$$

$$\text{p}2=\text{p}1+\frac{1-\text{p}1}{2}$$

$\text{p}3=\text{p}2+\frac{1-\text{p}1}{2}$ =1 (12)

All these steps are represented as the following pseudocode.

Algorithm 1: Pseudocode of heap

Heapsort(A) {

BuildHeap(A)

for i <- length(A) downto 2 {

exchange A[1] <-> A[i]

heapsize <- heapsize − 1

Heapify(A, 1)

}

BuildHeap(A) {

heapsize <- length(A)

for i <- floor( length/2 ) downto 1

Heapify(A, i)

}

Heapify(A, i) {

le <- left(i)

ri <- right(i)

if (le < = heapsize) and (A[le] > A[i])

largest <- le

else

largest <- i

if (ri < = heapsize) and (A[ri] > A[largest])

largest <- ri

if (largest != i) {

exchange A[i] <-> A[largest]

Heapify(A, largest)

}}

Figure 2: Flowchart of heap-based optimizer algorithm

The HBO algorithm was introduced to overcome continuous optimization problems. Feature selection, on the other hand, is a binary problem where each solution is described through a binary vector of zeros and ones. Therefore, to adapt the HBO for the FS problem, a binary version of the HBO (BHBO) must be constructed. Several strategies for adapting continuous algorithms to binary search spaces have been developed in the literature. There are two types of binarization method. The first one involves the continuous binary alteration of the operator, which includes the redefinition of the basic real operators of metaheuristics equations as binary operators. In the second method, which is known as two-step binarization, the real operators are used without enhancement, while the produced continuous solutions are transformed into binary format using two additional phases. The first phase uses a TF to convert the continuous solution Rn to a middle probability vector [0, 1] n, where every element in the vector defines the possibility of altering the corresponding element in Rn to 1 or 0. Then, in the second phase, the middle solution is binarized using various binarization methods [52].

4.1 Binary heap based optimizer with transfer functions

Two major forms of TF have been identified in the literature depending on their shapes. The S-shaped TF which can be seen in Fig. 4. 3 A, was first presented by Kennedy and Eberhart (1997) to transform the particle swarm optimizer (PSO) to binary by utilizing Eq. (13) and Eq. (14). On the other hand, the V-shaped TF, which can be seen in Fig. 3, was introduced by Rashedi et al. (2010) to binarize the gravitational search algorithm (GSA) using Eq. (4.15). These two TFs each have two binarization rules (standard and complement). The typical law (i.e., Eq. (16)) is utilized through the S-shaped function in the second stage of the binarization method, whereas the complement law (i.e., Eq. (16)) is utilized through the V-shaped function to upgrade the components of the feature subset during the following iteration, where ~ denotes the complement of x_i^j (t), and T(x_i^j) denotes the probability value achieved by applying Eq. (15) (Thaher et al., 2020).

(A) (B)

Figure 3: Transfer functions: (A) S-shaped, (B) V-shaped (Thaher et al., 2020)

$$\text{T}\left({\text{x}}_{\text{i}}^{\text{j}}\left(\text{t}\right)\right)=\frac{1}{1+{\text{e}}^{-\text{x}{}_{\text{i}}{}^{\text{j}}\left(\text{t}\right)}}$$

$$\text{T}\left({\text{x}}_{\text{i}}^{\text{j}}\left(\text{t}\right)\right)=|\text{t}\text{a}\text{n}\text{h}(\text{x}{}_{\text{i}}{}^{\text{j}}\left(\text{t}\right)\left)\right|$$

$${\text{x}}_{\text{i}}^{\text{j}}\left(\text{t}+1\right)=\left\{\begin{array}{c}0 if rand<T\left({\text{x}}_{\text{i}}^{\text{j}}\left(\text{t}+1\right)\right)\\ 1 otherwise\end{array}\right\}$$

$${\text{x}}_{\text{i}}^{\text{j}}\left(\text{t}+1\right)=\left\{\begin{array}{c}\tilde{\text{x}}_{\text{i}}^{\text{k}}\left(\text{t}\right) r<T\left({\text{x}}_{\text{i}}^{\text{j}}\left(\text{t}+1\right)\right)\\ {\text{x}}_{\text{i}}^{\text{j}}\left(\text{t}\right) otherwise\end{array}\right\}$$

4.2 Binary heap based optimizer for feature selection problems

Features selection is a binary optimization problem where the search agents can only work for binary 0 and 1 values. Hence, any solution in this study is represented as a single dimensional vector, the length of which is determined by the number of features in the data set. Any cell of the vector may have one of two values, 1 or 0, where 1 indicates that the equivalent feature is chosen and 0 indicates that it is not [53].

Metaheuristics are designed to solve continuous optimization problems in general. However, several optimization problems, such as FS problems, have a binary search space. Therefore, to solve binary problems, the form of the metaheuristic algorithm must be changed. The binary version of the HBO is presented in the remaining part of this section.

The TF is the easiest ways in which to transform a continuous algorithm into a binary version. Its versatility stems from the fact that, with the exception of small modifications to certain operators, the use of a TF does not alter the structure of the original continuous algorithm. Transfer functions are used to measure the potential of converting a continuous value in a solution to a binary value. A binary variant of the HBO must be utilized because in the FS search spaces, the function is either chosen (1) or not chosen (0) [54].

In this study, for the first time, a wrapper-based FS that treats the BHBO as a search algorithm is suggested. As shown in Figure. 4, the KNN classifier is employed as an evaluator. The KNN classifier is used because it is one of the most fundamental and commonly utilized non-parametric classification techniques. Furthermore, it has been commonly used in the literature and has demonstrated competitive efficiency as compared to a number of FS methods. Presenting the solution and the function assessment must be taken into consideration when designing any optimizer. Suggested solutions must be arranged in a fitting manner depending on the type of the manipulated problem for this reason. It is a crucial step that determines the size of the search space and impacts the overall performance of the optimizer. The evaluation function, on the other hand, directs the search process through determining the standard of quality of the solution space. Since FS is a binary problem, each suggested solution has to be expressed as a 1-dimensional vector through N elements X = x1, x2, x3... x N, where N denotes the overall features.

A value of 0 or 1 is assigned to every variable in order to indicate whether the feature is chosen or not. The lowest number of selected features and the overall classification accuracy are used to assess the fitness of a feature subset. When we use a single-objective HBO, we use the fitness function in Eq. (17) to formulate these two goals.

$\downarrow \text{F}\text{i}\text{t}\text{n}\text{e}\text{s}\text{s}={\alpha }{\gamma }\text{ʀ}\left(\text{D}\right) + {\beta }\frac{\left|\text{R}\right|}{\left|\text{N}\right|}$ [52] (17)

where ${\gamma }\text{ʀ}\left(\text{D}\right)$is the classifier’s classification error rate, |R| is the optimizer’s number of selected features, and ${\alpha }\in \left[\text{0,1}\right],{\beta }=\left(1-{\alpha }\right).$ Two parameters are utilized to set the importance of efficiency of classification and the feature reduction rate [52]. Figure 4 shows proposed method.

A number of tests and experiments were run to determine the benefits, drawbacks, and effectiveness of the HBO, as well as its position in comparison to other optimizers when applied to a number of benchmark data sets from the high dimensional domain. To make a fair comparison, all the settings were matched to ensure that all the algorithms started with the same population. All of the tests and comparisons with other metaheuristic algorithms were conducted on a personal computer with an Intel® Core™ i5-1035G1 CPU at 1.19 GHz, a memory size of 16 GB and a windows 10 Professional N64-bit operating system.

The proposed wrapper-based FS approach dependent on the KNN classifier was used to determine the performance of the upgraded HBO, which are dependent on the created feature subsets. A hold-out technique was used to validate the findings, in which each data set was divided into two parts: 80% for training purposes and 20% for testing purposes. These divisions were replicated 30 times to produce meaningful results. As a result, 30 separate runs provide the final statistical findings. MATLAB 2015 was used to produce both results and analyses.

5.1 Parameter Settings

The parameters were set based on the recommendations in previous papers and related works in the FS field. Table 2 provides the optimizer internal parameters that were used for the proposed algorithm and the compared algorithms.

Table 2

Parameter Settings
Parameter Name	Value
α	0.99
β	0.01
Number of runs	30
Population size	40
No. of iterations	100
G0 (for GSA)	100
For GSA α	20
For HBO a	From 2 to 0
Ԛ min(BA)	0
Ԛ max (BA)	2
Loudness (for BA)	0.5
Pulse rate (BA)	0.5
For PSO ω	From 0.90 to 0.40
For (PSO) cp, cg	2
GA selection	Roulette wheel selection
Mutation probability in GA	0.01
Crossover probability in GA	0.90
Elite size in GA	2
K for KNN	5

5.2 Description of the Data Sets

Experiments were conducted on nine well-known, high-dimensional benchmark data sets that were obtained from the GEMS System website http://www.gems-system.org. The characteristics of these data sets are provided in Table 3. Thus, the robustness and performance of the enhanced optimizer can be determined by testing it on a variety of problems with various characteristics.

Table 3

Characteristics of the High-dimensional, Low-sample Data Sets
Data set	No. of samples	No. of features	No. of classes
11_Tumors	174	12533	11
14_Tumors	308	15009	26
Brain_Tumor1	90	5920	5
Brain_Tumor2	50	10367	4
DLBCL	77	5469	2
Leukemia1	72	5327	3
Leukemia2	72	11225	3
Prostate_Tumor	102	10509	2
SRBCT	83	2308	4

5.3 Results

This section focuses on investigating the sensitivity of the binary variant to the initial population and to the number of iterations. Note that the HBO does not have a specific user-defined initial parameter and is characterized by its random and variable nature that changes over time. Understanding the method for obtaining the best results for the BHBO method before comparing it with other methods is an important benefit of this experiment. Such an analysis can show whether or not the population size and number of iterations has a clear impact on performance.

Table 4 shows the accuracy rates obtained by the BHBO for different sets of common parameters. From the analysis of the ranking results in Table 3, it was found that the dual HBO with 20 agents and 100 iterations had the potential to achieve the best results for the cases, i.e., the nine benchmark data sets described above. The F-test (last row in Table 3) was used to reveal the best results as this is a popular method of assessing various methods in a comparative test.

Table 4

Average Classification Accuracy achieved by BHBO with multiple sets of common parameters
Iteration	50			100			150
Population	5	10	20	5	10	20	5	10	20
11_Tumors	0.689	0.689	0.701	0.735	0.758	0.804	0.643	0.666	0.747
14_Tumors	0.402	0.435	0.500	0.480	0.487	0.564	0.480	0.428	0.474
Brain_Tumor1	0.888	0.733	0.644	0.800	0.711	0.822	0.777	0.977	0.911
Brain_Tumor2	0.440	0.520	0.680	0.720	0.600	0.720	0.640	0.720	0.760
DLBCL	0.846	0.897	0.820	0.820	0.897	0.948	0.820	0.871	0.820
Leukemia1	0.833	0.750	0.833	0.833	0.944	0.916	0.833	0.833	0.777
Leukemia2	0.777	0.777	0.777	0.833	1.000	0.888	0.944	0.944	0.638
Prostate Tumor	0.862	0.725	0.823	0.803	0.823	0.862	0.862	0.857	0.784
SRBCT	0.714	0.857	0.833	0.833	0.880	0.904	0.880	0.738	0.785
Overall rank	4.720	4.720	5.560	3.890	5.670	3.440	5.110	6.390	5.500

Table 5 shows the average time, precision, recall and f-measure results obtained by the BHBO over 30 runs.

Table 5

Average results for time, precision, recall and F-measure
Data Set	Time	Precision	Recall	F-Measure	accuracy
11_Tumors	322.99	0.96	1.00	0.99	0.83
14_Tumors	1079.76	1.00	1.00	1.00	0.51
Brain_Tumor1	215.98	1.00	1.00	1.00	0.92
Brain_Tumor2	57.50	0.89	0.92	0.85	0.71
DLBCL	43.02	0.94	0.98	0.96	0.94
Leukemia1	64.30	0.95	1.00	0.97	0.92
Leukemia2	581.40	1.00	1.00	1.00	0.95
Prostate_Tumor	159.50	0.87	0.87	0.85	0.87
SRBCT	276.99	0.93	0.97	0.94	0.92

Furthermore, a boxplot is a type of chart that shows a five-number summation. The interquartile range indicates where the data's center section is located. The first quartile (the 25% point) as well as the third quartile (the 75% mark) are situated at the box's opposite ends. The chart's minimum value is on the left, while its highest position is on the right. A vertical line in the middle of the box denotes the median. A boxplot shows how tightly the data is clustered and if it is symmetrical. Any outliers' presence and locations are also revealed. Figure 5 depicts the boxplots that depict the distribution of HBO performance across 30 runs when applied to the 9 datasets.

Figure 5 shows how the HBO was able to close the gap between of min and max accuracy values, bringing them closer to the mean value. This is a strong indicator of the HBO approach's capacity to deliver accurate findings by striking a proper balance between local and global search.

Moreover, A swarm plot is another approach to visualize the distribution of a single attribute or the combined distribution of several variables. The position of each dot on the horizontal and vertical axis indicates values for an individual data point. A swarm plot displays all of the data points, which aids in better understanding the distribution. It also aids in comprehending how data is dispersed throughout a categorical characteristic as well as how the variable varies within a category.. All the 30 run results for each dataset have been clarified as a dot in the Fig. 6.

As seen in Fig. 6, we get a good idea of how the data (the 30 runs) for each dataset is distributed in terms of a numeric attribute.

One of the goals of this study was to speed up optimization by minimizing random selection and the exploration process in order to find the best solution in a shorter duration of time. Figures 7 illustrate the success of the HBO in finding the best solution for the nine data sets using enhanced convergence. From Figs. 7, it can be seen that the BHBO achieved the best convergence trends as compared to the competing methods, as its convergence curves obtained the lowest cost values in the final iterations of the optimization process. A total of 100 iterations were executed for the BHBO because accuracy did not increase after the 80th iteration. The convergence behavior, on the other hand, can be used to predict early convergence and local minimum deviation.

5.4 Comparison with Other Methods in the Literature

The performance of the BHBO was compared with that of other well-known optimizers for various metrics such as result accuracy. The algorithms from the literature were the BHHO, the GA, and binary versions of the GSA (BGSA), the ant-lion optimizer (BALO), the bat algorithm (BBA), the salp swarm algorithm (BSSA), and the particle swarm optimizer (BPSO). The average accuracy and the number of features selected using the BHBO, as well as the fitness values for each of these methods were obtained. The results presented in the tables below were based on 30 runs.

Initially, the BHBO was implemented with a population size of 20 in 100 iterations and the accuracy rate was measured. However, the BHBO did not achieve the desired results as it was found that it outperformed the BHHO only in two data sets. Therefore, the parameters were tuned further and it was found that by setting the population size to 40 and the iterations to 100, the BHBO outperformed the BHHO in four data sets, namely, 11_ tumor, DLBCL, leukemia 1, and leukemia 2. By comparing the results of BHBO with the rest of the other methods, we found that it achieved higher results in two data sets.

We noticed after performing the algorithm implementation that the data sets that contain a large number of samples and also contain a large number of classes, as in cases of 11_Tumors and 14_Tumors give less accuracy than the dataset that contains a small number of samples and a small number of classes, which represents the rest of the data used in the study

Table 6 shows a comparison of the BHBO and the abovementioned state-of-the-art algorithms in terms of classification accuracy. It can be seen from the table that the BHBO exceeded all these modern FS approaches in two data sets, namely 11_Tumors and DLBCL. In detail, the BHBO method outperformed six methods (GA, BSSA, BBA, BALO, BPSO, and BGSA) in the Brain_Tumor1 data set; and three methods (GA, BSSA, and BPSO) in the Brain_Tumor2 data set. It also achieved clear superiority over six methods (BHHO, GA, BSSA, BBA, BALO and BGSA) in the leukemia1 data set. Also investigated a distinction in accuracy on six methods, namely, the BHHO, GA, BSSA, BBA, BALO, and BPSO. In leukemia 2, the BHBO performed better than the BBA in 11_Tumors data set, in a data set, Prostate_Tumor, BHBO progressed in three methods namely BSSA, BALO, and BGSA. As shown in the table, the BHBO was ranked fourth, followed by the BHHO, BGSA and BPSO.

Table 6

Comparison of BHBO versus other optimizers in terms of average classification accuracy
Data set	BGSA	BPSO	BALO	BBA	BSSA	GA	SBHHO	BHBO
11_Tumors	0.7506	0.8162	0.7143	0.6286	0.6765	0.8162	0.8295	0.8319
14_Tumors	0.5122	0.5557	0.5942	0.4619	0.5322	0.6381	0.5412	0.5136
Brain_Tumor1	0.8889	0.8333	0.9444	0.9185	0.8333	0.7778	0.9796	0.9200
Brain_Tumor2	0.8852	0.6300	0.9000	0.8534	0.7000	0.6667	0.8074	0.7148
DLBCL	0.8750	0.8208	0.8750	0.9021	0.7792	0.9375	0.9396	0.9410
Leukemia1	0.8822	0.9844	0.9089	0.8778	0.8222	0.8444	0.8956	0.9241
Leukemia2	1.0000	0.9333	0.9333	0.5667	0.8689	0.8667	0.9156	0.9528
Prostate_Tumor	0.8556	0.8937	0.8587	0.8746	0.8191	0.9746	1.0000	0.8686
SRBCT	0.9706	0.9628	0.9412	0.8804	0.8863	0.8804	1.0000	0.9171
F_test	0.9637	0.9637	1.4422	0.6966	1.6097	1.6136	0.9637	-
Mean rank (F_test)	6.5	6.5	3	8	2	1	5	4
Overall rank	6	6	3	7	2	1	5	4

As shown in Fig. 8, the BHBO achieved the best accuracy rate, at 0.8426614, followed by BPSO, GA, BBA, and BSSA with accuracy rates of 0.8255778, 0.8224889, 0.7737778, and 0.7686333, respectively.

One of the important reasons for the high accuracy that was obtained by implementing the BHBO is that the best combined values were found for the parameters. By tuning the parameters the best population size was found to have a value of 40 and the best value for the number of iterations was found to be 100. This played a crucial role in improving the performance of the algorithm. The other reason that helped to improve performance is that the BHBO does not need a large number of parameters; when there are a lot of parameters it is difficult to determine the combined optimal values for all the parameters.

Table 7 shows the p-values of the BHBO and the other algorithms. In certain instances, the p-values reveal a substantial difference between the BHBO and the other optimizers. These findings suggest that the BHBO may make a more consistent trade-off between the core searching phases. Thus, it has a greater chance of escaping local optima and avoiding immature convergence disadvantages, resulting in higher performance in terms of accuracy.

Table 7

Wilcoxon Rank-sum Test of the P-values for Classification Accuracy
Dataset	BHBO versus
Dataset	BGSA	BPSO	BALO	BBA	BSSA	GA
11_Tumors	0.767097	0.593955	0.767097	0.109745	0.015156	0.374259
14_Tumors	0.767097	0.593955	0.767097	0.109745	0.015156	0.374259
Brain_Tumor1	0.767097	0.593955	0.767097	0.109745	0.015156	0.374259
Brain_Tumor2	0.767097	0.593955	0.767097	0.109745	0.015156	0.374259
DLBCL	0.767097	0.593955	0.767097	0.109745	0.015156	0.374259
Leukemia1	0.767097	0.593955	0.767097	0.109745	0.015156	0.374259
Leukemia2	0.767097	0.593955	0.767097	0.109745	0.015156	0.374259
Prostate_Tumor	0.767097	0.593955	0.767097	0.109745	0.015156	0.374259
SRBCT	0.767097	0.593955	0.767097	0.109745	0.015156	0.374259

On the other hand, Table 8 shows that the BHBO was unable to effectively identify feature that fit the problem as the number of features was found to be greatly increased. This also explains the increase in the number of specific features when the parameters were tuned in this algorithm, which greatly increased the accuracy. It can also be seen from the table that the BHBO method ranked seventh in terms of performance in the F-test.

Table 8

Comparison of BHBO versus Other Optimizers in Terms of Features Selected
Data set	BGSA	BPSO	BALO	BBA	BSSA	GA	BHHO	BHBO
11_Tumors	6239.9	6202.2	7083.6	5092.6	7160.6	5906.7	3651.4	9045.8
14_Tumors	7517.8	7486.0	11299.0	6063.4	9472.7	7235.1	6214.6	11233.7
Brain_Tumor1	2884.3	2822.6	2891.6	2404.8	2892.5	2670.6	2314.9	4100.7
Brain_Tumor2	5128.9	4953.0	6309.5	4210.9	5095.4	4815.8	3172.1	8004.5
DLBCL	2669.2	2538.8	3634.7	2193.7	2941.9	2459.1	1818.5	3856.2
Leukemia1	2635.5	2552.9	3511.0	2191.2	2772.2	2443.3	1584.8	3632.1
Leukemia2	5513.6	5394.4	5686.4	4567.7	5557.5	5221.6	3956.6	8110.7
Prostate_Tumor	5209.6	5070.4	6658.6	4201.0	5766.4	4942.5	3489.4	7487.8
SRBCT	1143.6	1071.1	1168.4	962.7	1193.2	1012.9	784.5	1706.7
F_test	2.31	2.32	1.11	3.52	1.48	2.41	3.79
Mean rank (F_test)	5	4	7	5	6	3	1	8
Overall rank	4	3	6	4	5	2	1	7

As shown in Fig. 9, the number of features increased when the parameters were tuned for this algorithm, which caused a greater increase in accuracy compared to the previous parameter settings. Hence, despite the greater number of features chosen when tuning the parameters, the BHBO was better in terms of accuracy than many of the other approaches that were tested.

Figures 10 and 11 represent how the solution was obtained by the implementation of the algorithm on nine data sets, DLCBL and SRBCT, respectively. In the figures, the value of 1 denotes that the algorithm selected a feature, while the value of 0 indicates that the algorithm did not choose a feature.

Table 9 contains the fitness results. As shown in this table, the BHBO performed better in terms of fitness score as compared to the BSSA, BBA, BALO, and BGSA in the case of 11-Tumors, while it outperformed the GA, BSSA and PSO in the case of the Brain_Tumor1 data set. Also, the BHBO outperformed the BPSO in Brain_Tumor2, outperformed the BSSA and BPSO in DLBCL, and outperformed the GA and BSSA in the case of Leukemia1. Furthermore, it outperformed three methods (GA, BSSA and BBA) in the case of Leukemia2, and outperformed the BSSA in Prostate_Tumor. In terms of overall classification accuracy, it can be seen that the BHBO was ranked in fifth place, followed by the BPSO and the BBA.

Table 9

Comparison of BHBO versus other optimizers in terms of average fitness values.
Dataset	BGSA	BPSO	BALO	BBA	BSSA	GA	BHHO	BHBO
11_Tumors	0.252	0.185	0.289	0.342	0.326	0.187	0.172	0.233
14_Tumors	0.488	0.445	0.409	0.511	0.470	0.363	0.458	0.532
Brain_Tumor1	0.115	0.170	0.060	0.037	0.170	0.225	0.024	0.115
Brain_Tumor2	0.119	0.371	0.105	0.037	0.302	0.335	0.194	0.357
DLBCL	0.129	0.182	0.130	0.065	0.224	0.066	0.063	0.131
Leukemia1	0.125	0.020	0.097	0.066	0.181	0.159	0.105	0.127
Leukemia2	0.005	0.071	0.071	0.400	0.135	0.137	0.087	0.109
Prostate_Tumor	0.148	0.110	0.146	0.098	0.185	0.030	0.003	0.171
SRBCT	0.034	0.042	0.063	0.072	0.118	0.123	0.003	0.124
F_test	1.029	0.991	1.470	0.619	0.497	1.659	1.022
Mean rank (F_test)	3	6	2	7	8	1	4	5
Overall rank	3	6	2	7	8	1	4	5

Figure 12 shows the fitness rates for all methods, from which it can be seen that the BHBO achieved a fitness rate of 0.21098289, followed closely by the BSSA with a fitness rate of 0.23441111.

The initial population of the BHBO was randomly generated. In the following steps, the population was changed toward the optimum solution depending on the fitness values. This step was repeated until the termination criterion was reached. The reason for the improvement in the fitness values is because the initial population influences the quality of the last solution and the number of iterations required to reach the optimal solution. The quality of the initial population was improved to discover solutions of superior quality and eliminate duplicates in order to arrive at the best solution.

5.7 Discussion

In this chapter, the efficacy of the BHBO was compared with that of the BGSA, BSSA, GA, BPSO, BBA and BALO methods in terms of accuracy, number of features, and fitness values when applied on nine high-dimensional data sets. The F-test results. The Wilcoxon ranking test for linked samples, and the p-values of the BHBO and the other algorithms were calculated and obtained for the three measures mentioned above.

This chapter presented the results of the proposed wrapper FS model using a KNN classifier and HBO algorithm when it was deployed to select proper features in nine high-dimensional data sets with a small number of samples. The evaluation of this method was performed depending on several criteria: classification accuracy, number of features selected, error rate, precision, recall, F-measure, and convergence speed. All the obtained results showed that the BHBO enhanced the search capability, and that it was better than most of the compared state-of the-art methods for FS problems.

The comparison of the BHBO with the other optimizers showed that it outperformed all the methods in terms of accuracy in two data sets, namely, 11_Tumors and DLBCL. In more detail, the results of the experiments showed that the BHBO method was capable of outperforming six methods in the Brain_Tumor1 data set and three methods in the Brain_Tumor2 data set. It also achieved clear superiority in terms of accuracy as compared to six methods in the leukemia1 data set, also investigated a distinction in accuracy on six methods. In leukemia2, Moreover, the BHBO performed better than the BBA in the 11_Tumors data set, in a data set, Prostate_Tumor, BHBO progressed in three methods. In terms of fitness, the BHBO performed better as compared to the BSSA, BBA, BALO, and BGSA in the case of 11-Tumors; the GA, BSSA and PSO in Brain_Tumor1; the BPSO in Brain_Tumor2; the BSSA and BPSO in DLBCL; the GA and BSSA in Leukemia1; the GA, BSSA and BBA in Leukemia2; and the BSSA in Prostate_Tumor.

In this study, a new method based on the heap structure was presented to solve FS problems in high dimensional datasets. HBO was upgraded and converted into a new binary HBO – the BHBO. Experiments were conducted on nine well-known, high-dimensional benchmark data sets. HBO was able to significantly and distinctly reduce the number of features in most nine datasets used. However, the applicability of this algorithm depends on the type of problem to be solved, that is, whether the degree of accuracy is decisive, or whether the size of the problem is the most important factor. The performance efficiency of the proposed method was evaluated and compared with seven recent FS methods, namely, the BGSA, BPSO, BALO, BBA, BSSA, GA and BHHO. Based on this comparison, HBO's advantages were notable in terms of further improving accuracy. Initially, the HBO did not achieve high accuracy in some data sets, after tuning the parameters, the results that it achieved were more accurate as compared to those generated by most of the previous methods. This indicates that the HBO has some superiority over the other methods. However, the HBO method has exemplified improvement in the FS process in two areas: accuracy rate and convergence speed.

As a future work, the suggested method could be hybridized with a variety of other search algorithms, So far, the HBO is up to date and there are many preparations, and visions to delve deeper into HBO's operations to manipulate and deal with real-world data sets.

Capobianco, E., High-dimensional role of AI and machine learning in cancer research. British journal of cancer, 2022: p. 1-10.
Pandey, A.C., D.S. Rajpoot, and M. Saraswat, Feature selection method based on hybrid data transformation and binary binomial cuckoo search. Journal of Ambient Intelligence and Humanized Computing, 2020. 11(2): p. 719-738.
Alweshah, M., et al., The monarch butterfly optimization algorithm for solving feature selection problems. Neural Computing and Applications, 2020: p. 1-15.
Abualigah, L. and A. Diabat, Chaotic binary group search optimizer for feature selection. Expert Systems with Applications, 2022. 192: p. 116368.
Alsahaf, A., et al., A framework for feature selection through boosting. Expert Systems with Applications, 2022. 187: p. 115895.
Aram, K.Y., S.S. Lam, and M.T. Khasawneh, Linear Cost-sensitive Max-margin Embedded Feature Selection for SVM. Expert Systems with Applications, 2022: p. 116683.
Balakrishnan, K., R. Dhanalakshmi, and U. Mahadeo Khaire, Analysing stable feature selection through an augmented marine predator algorithm based on opposition‐based learning. Expert Systems, 2022. 39(1): p. e12816.
Alweshah, M., et al., A hybrid mine blast algorithm for feature selection problems. Soft Computing, 2021. 25(1): p. 517-534.
Chandrashekar, G. and F. Sahin, A survey on feature selection methods. Computers & Electrical Engineering, 2014. 40(1): p. 16-28.
Momeni, H. and A. Ebrahimkhanlou, High-dimensional data analytics in structural health monitoring and non-destructive evaluation: a review paper. Smart Materials and Structures, 2022.
Bommert, A., et al., Benchmark of filter methods for feature selection in high-dimensional gene expression survival data. Briefings in Bioinformatics, 2022. 23(1): p. bbab354.
Mulaudzi, R. and R. Ajoodha. Demonstration that the Use of Feature Selection on High Dimensional South African Macroeconomic Data Results in Improved Performance. in Proceedings of Sixth International Congress on Information and Communication Technology. 2022. Springer.
Bhui, N. Ensemble of Deep Learning Approach for the Feature Selection from High-Dimensional Microarray Data. in Proceedings of the International Conference on Paradigms of Communication, Computing and Data Sciences. 2022. Springer.
Amini, F. and G. Hu, A two-layer feature selection method using genetic algorithm and elastic net. Expert Systems with Applications, 2021. 166: p. 114072.
Almomani, A., et al., Metaheuristic algorithms-based feature selection approach for intrusion detection, in Machine Learning for Computer and Cyber Security2019, CRC Press. p. 184-208.
Yusta, S.C., Different metaheuristic strategies to solve the feature selection problem. Pattern Recognition Letters, 2009. 30(5): p. 525-534.
Alweshah, M., et al., Coronavirus herd immunity optimizer with greedy crossover for feature selection in medical diagnosis. Knowledge-Based Systems, 2022. 235: p. 107629.
Alweshah, M., Solving feature selection problems by combining mutation and crossover operations with the monarch butterfly optimization algorithm. Applied Intelligence, 2021. 51(6): p. 4058-4081.
Kumar, S., et al., Improved crow search algorithm based on arithmetic crossover—a novel metaheuristic technique for solving engineering optimization problems, in Multi-Objective Combinatorial Optimization Problems and Solution Methods2022, Elsevier. p. 71-91.
Kassaymeh, S., et al., Salp swarm optimizer for modeling software reliability prediction problems. Neural Processing Letters, 2021. 53(6): p. 4451-4487.
Kassaymeh, S., et al. A Hybrid Salp Swarm Algorithm with Artificial Neural Network Model for Predicting the Team Size Required for Software Testing Phase. in 2021 International Conference on Electrical Engineering and Informatics (ICEEI). 2021. IEEE.
Alweshah, M., et al., Improved water cycle algorithm with probabilistic neural network to solve classification problems. Cluster Computing, 2020. 23(4): p. 2703-2718.
Alweshah, M., et al., The monarch butterfly optimization algorithm for solving feature selection problems. Neural Computing and Applications, 2020: p. 1-15.
Leardi, R., R. Boggia, and M. Terrile, Genetic algorithms as a strategy for feature selection. Journal of chemometrics, 1992. 6(5): p. 267-281.
Rodrigues, D., et al. BCS: A binary cuckoo search algorithm for feature selection. in 2013 IEEE International Symposium on Circuits and Systems (ISCAS). 2013. IEEE.
Mafarja, M.M. and S. Mirjalili, Hybrid whale optimization algorithm with simulated annealing for feature selection. Neurocomputing, 2017. 260: p. 302-312.
Ghosh, A., A. Datta, and S. Ghosh, Self-adaptive differential evolution for feature selection in hyperspectral image data. Applied Soft Computing, 2013. 13(4): p. 1969-1977.
Diao, R. and Q. Shen, Feature selection with harmony search. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2012. 42(6): p. 1509-1523.
Zhang, H. and G. Sun, Feature selection using tabu search method. Pattern recognition, 2002. 35(3): p. 701-711.
Mehmod, T. and H.B.M. Rais, Ant colony optimization and feature selection for intrusion detection, in Advances in machine learning and signal processing2016, Springer. p. 305-312.
Abu Khurma, R., et al., A review of the modification strategies of the nature inspired algorithms for feature selection problem. Mathematics, 2022. 10(3): p. 464.
Li, H., et al., A dividing-based many-objective evolutionary algorithm for large-scale feature selection. Soft computing, 2019: p. 1-20.
Wang, X.-h., et al., Multi-objective feature selection based on artificial bee colony: An acceleration approach with variable sample size. Applied Soft Computing, 2020. 88: p. 106041.
Aljarah, I., et al., A Dynamic Locality Multi-Objective Salp Swarm Algorithm for Feature Selection. Computers & Industrial Engineering, 2020: p. 106628.
Hegazy, A.E., M. Makhlouf, and G.S. El-Tawel, Improved salp swarm algorithm for feature selection. Journal of King Saud University-Computer and Information Sciences, 2020. 32(3): p. 335-344.
Mohmmadzadeh, H., A Combination Approach of Two Metaheuristic Algorithm for Optimal Feature Selection: Case Study Email Spam Detection. Preprints, 2020.
Tumar, I., et al., Enhanced binary moth flame optimization as a feature selection algorithm to predict software fault prediction. IEEE Access, 2020. 8: p. 8041-8055.
Agrawal, R., B. Kaur, and S. Sharma, Quantum based whale optimization algorithm for wrapper feature selection. Applied Soft Computing, 2020. 89: p. 106092.
Abdel-Basset, M., et al., A new fusion of grey wolf optimizer algorithm with a two-phase mutation for feature selection. Expert Systems with Applications, 2020. 139: p. 112824.
Nourmohammadi-Khiarak, J., et al., New hybrid method for feature selection and classification using meta-heuristic algorithm in credit risk assessment. Iran Journal of Computer Science, 2020. 3(1): p. 1-11.
Chen, W., Z. Li, and J. Guo, A VNS-EDA Algorithm-Based Feature Selection for Credit Risk Classification. Mathematical Problems in Engineering, 2020. 2020.
Meera, S. and C. Sundar, A hybrid metaheuristic approach for efficient feature selection methods in big data. Journal of Ambient Intelligence and Humanized Computing, 2020: p. 1-9.
Alsaeedi, A.H., et al., Extended Particle Swarm Optimization (EPSO) for Feature Selection of High Dimensional Biomedical Data, in arXiv preprint arXiv:2008.035302020.
AbdElminaam, D.S., et al., An Efficient Heap-Based Optimizer for Parameters Identification of Modified Photovoltaic Models. Ain Shams Engineering Journal, 2022. 13(5): p. 101728.
Shaheen, A.M., et al., Improved Heap-Based Optimizer for DG Allocation in Reconfigured Radial Feeder Distribution Systems. IEEE Systems Journal, 2022.
Abdelhamid, M., et al., An Adaptive Protection Scheme Based on a Modified Heap-Based Optimizer for Distance and Directional Overcurrent Relays Coordination in Distribution Systems. Mathematics, 2022. 10(3): p. 419.
Chakraborty, S., et al., HSWOA: An ensemble of hunger games search and whale optimization algorithm for global optimization. International Journal of Intelligent Systems, 2022. 37(1): p. 52-104.
Elsayed, S.K., et al., An improved heap-based optimizer for optimal reactive power dispatch. IEEE Access, 2021. 9: p. 58319-58336.
Ginidi, A.R., et al., A Novel Heap-Based Optimizer for Scheduling of Large-Scale Combined Heat and Power Economic Dispatch. IEEE Access, 2021. 9: p. 83695-83708.
Askari, Q., M. Saeed, and I. Younas, Heap-based optimizer inspired by corporate rank hierarchy for global optimization. Expert Systems with Applications, 2020. 161: p. 113702.
Shaheen, M.A., H.M. Hasanien, and A. Al-Durra, Solving of Optimal Power Flow Problem Including Renewable Energy Resources Using HEAP Optimization Algorithm. IEEE Access, 2021.
Thaher, T., et al., Binary Harris Hawks optimizer for high-dimensional, low sample size feature selection, in Evolutionary machine learning techniques2020, Springer. p. 251-272.
Arora, S. and P. Anand, Binary butterfly optimization approaches for feature selection. Expert Systems with Applications, 2019. 116: p. 147-160.
Mafarja, M., et al., Efficient hybrid nature-inspired binary optimizers for feature selection. Cognitive Computation, 2020. 12(1): p. 150-175.

PDF

Version 1

posted 14 Mar, 2022

You are reading this latest preprint version

Heap Based Optimizer Algorithm for Solving Feature Selection Problems in High-Dimensional Cancer Microarray Data

Status:

Version 1

Abstract

Figures

1. Introduction

2. Related Work

3. Heap-based Optimizer

3.1 Modeling the corporate rank hierarchy

3.2 Mathematical modeling of the relationship between subordinates and immediate boss

3.3 Mathematical modeling of the interactions between colleagues

3.4 Mathematical modeling of an employee’s self-commitment to completing a mission

4. Proposed Method

4.1 Binary heap based optimizer with transfer functions

4.2 Binary heap based optimizer for feature selection problems

5. Experimental Results

5.1 Parameter Settings

5.2 Description of the Data Sets

5.3 Results

5.4 Comparison with Other Methods in the Literature

5.7 Discussion

6. Conclusion

References

Status:

Version 1

\(\text{X}{}_{\text{i}}{}^{\text{k}}\left(\text{t}+1\right)={\text{B}}^{\text{k}}+\text{Y}{{\lambda }}^{\text{k}}\|{\text{B}}^{\text{k}}-{\text{X}}_{\text{i}}^{\text{k}}\left(\text{t}\right)\|\)	(1)
Where vector \({ }_{{\lambda }}^{\to }\)is denoted by the letter\({{\lambda }}^{\text{k}}\), which is estimated using the formula in Eq. (2):