DEHypGpOls: a genetic programming with evolutionary hyperparameter optimization and its application for stock market trend prediction

Stock markets are a popular kind of financial markets because of the possibility of bringing high revenues to their investors. To reduce risk factors for investors, intelligent and automated stock market forecast tools are developed by using computational intelligence techniques. This study presents a hyperparameter optimal genetic programming-based forecast model generation algorithm for a-day-ahead prediction of stock market index trends. To obtain an optimal forecast model from the modeling dataset, a differential evolution (DE) algorithm is employed to optimize hyperparameters of the genetic programming orthogonal least square (GpOls) algorithm. Thus, evolution of GpOls agents within the hyperparameter search space enables adaptation of the GpOls algorithm for the modeling dataset. This evolutionary hyperparameter optimization technique can enhance the data-driven modeling performance of the GpOls algorithm and allow the optimal autotuning of user-defined parameters. In the current study, the proposed DE-based hyper-GpOls (DEHypGpOls) algorithm is used to generate forecaster models for prediction of a-day-ahead trend prediction for the Istanbul Stock Exchange 100 (ISE100) and the Borsa Istanbul 100 (BIST100) indexes. In this experimental study, daily trend data from ISE100 and BIST100 and seven other international stock markets are used to generate a-day-ahead trend forecaster models. Experimental studies on 4 different time slots of stock market index datasets demonstrated that the forecast models of the DEHypGpOls algorithm could provide 57.87% average accuracy in buy–sell recommendations. The market investment simulations with these datasets showed that daily investments to the ISE100 and BIST100 indexes according to buy or sell signals of the forecast model of DEHypGpOls could provide 4.8% more average income compared to the average income of a long-term investment strategy.


Introduction
Forecasting of stock market trends is one of the challenging prediction problems because several factors such as social, economical, ecological and physiological factors, even local events and trends, can affect the behavior of stock markets. For this reason, the stock markets are subject to uncertainties (Subhi Alzazah and Cheng 2020), they involve high nonlinearity and show changing dynamics and trends Yoo et al. 2005). For this reason, in the literature, many computational intelligence methods have been used to overcome these complications. Research results have shown that automated stock market trend prediction, which is supported by computational intelligence, can reduce loss risks of the income and bring profit to the investor (Sheta et al. 2013). Some of computational methods, which have been employed to solve the stock market prediction problems, are ANN (Bisoi and Dash 2014;Akbilgic et al. 2014; Moghar and Hamiche 2020;Parray et al. 2020), regression models (Ananthi and Vijayakumar 2020;Huang 2012;Parray et al. 2020), deep learning (Ingle and Deshmukh 2021;Shen and Shafiq 2020), swarm intelligence (Bagheri et al. 2014), evolutionary computation (Sheta et al. 2013;Hsu 2011;Huang 2012), dynamical Bayesian factor graphs (Wang et al. 2015), and neuro-fuzzy systems (Rajab and Sharma 2019;Mahmud and Meesad 2016).
Recently, numerous studies have shown that machine learning algorithms could effectively solve data analysis, classification and detection problems in a widespread application areas; for instance the ranking and classification of DNA microarray data (Kanti Ghosh et al. 2021), the feature selection for character recognition (Kumar et al. 2018, the object recognition (Gupta et al. 2019), the finger vein recognition (Shaheed et al. 2022), and electronic nose (Ari and Alagoz 2022). A need for an initial user configuration of classical machine learning algorithms can reduce their autonomy and adaptation skills. On the other hand, manual tuning of user parameters can also be an exhaustive process, and it may not be feasible in cases of existence of large amounts of data or high complexity machine learning algorithms. For this reason, the hyperparameter optimization topic has been gaining more and more importance in recent years, particularly for the development of more intelligent, adaptive and autonomous systems that are capable of intelligently performing in daily life applications. Especially in the finance area, real-time or online investment systems require automated and optimal modeling of financial data to provide reliable predictions for decision making and risk management algorithms.
The hyperparameter optimization tunes a set of initial user configuration parameters of machine learning algorithms so that the well-tuned machine learning algorithms can yield the best models for a given datasets (Burke et al. 2003). Accordingly, hyperparameter optimization contributes to performance of data-driven modeling efforts (Solomatine et al. 2009). In other words, the best practical performance of the machine learning algorithms for a given dataset can be automatically achieved by using hyperparameter optimization techniques in real data-driven modeling applications. In the literature, some hyperparameter optimization techniques are based on the brute-force searching such as grid search and random search techniques (Liashchynskyi and Liashchynskyi 2019;Villalobos-Arias and Quesada-López 2021), also, Bayesian optimization (Cho et al. 2020), gradient-based optimization (Ollar et al. 2017), evolutionary optimization (Chatzimparmpas et al. 2021;Schmidt et al. 2019), stochastics multi-parameter divergence optimization (Ates and Akpamukcu 2021), etc.
Previously, GP has been implemented in stock market modeling and prediction problems (Karatahansopoulos et al. 2014;Sheta et al. 2013;Nikolaev and Iba 2002). An enhanced tree-based GP algorithm (GpOls) was suggested with an orthogonal least square (OLS) improvement (Madár et al. 2005), and it was effectively used for identification of dynamic systems (Madár et al. 2005;Wiese 2011). However, we observed that the hyperparameter optimization of GpOls can further improve model optimality for a given dataset and contribute to practical performance of the algorithm in real applications. Recently, the use of DE optimization algorithm for evolutionary hyperparameter optimization has been discussed for several machine learning algorithms and dataset (Schmidt et al. 2019). All these progresses become the main motivation to use the DE algorithm for hyperparameter optimization of the GpOls algorithm and develop an evolutionary hyperparameter optimal GpOls modeling framework. In the current study, authors implement a DEbased hyperparameter optimal GpOls algorithm for one day-ahead trend forecast of the ISE100 and BIST100 closing indexes. The proposed DEHypGpOls algorithm optimizes user configuration parameters of the GpOls algorithm by means of the DE algorithm. This allows the adaptation of the GpOls modeling agents to characteristic properties of modeling dataset and thus improves performance of resulting GpOls forecaster models. The finetuned GpOls algorithm was obtained automatically and its evolutionary optimal forecaster model can predict a-dayahead trend in the market index depending on seven other international stock market indices. The trend predictions of the forecast model were used to generate buy or sell signals for daily exchanges in the ISE100 and BIST100 indexes.
This study presents two main contributions: (i) The data-driven modeling performance of the GpOls algorithm is enhanced by enabling the differential evolution of GpOls agents within the hyperparameter search space of the GpOls algorithm. Thus, a DE-based hyper-GpOls (DEHypG-pOls) algorithm was suggested for the evolutionary optimal genetic programming modeling. (ii) An experimental application of the proposed DEHypGpOls algorithm is demonstrated for the daily trend prediction of stock market indices ISE100 and BIST100. New BIST100 trend datasets were composed of three consecutive time slots from the years 2014 to 2022. Performance analyses and market investment simulations of the DEHypG-pOls's forecaster model and the ANN forecaster models were carried out for these datasets.

A brief literature review for modeling and prediction of stock market parameters
The automated stock market prediction with intelligent computation has attracted interest of many investors and researchers because a dependable knowledge of stock movements by a fraction of a second can lead to significant profits (Subhi Alzazah and Cheng 2020). Therefore, data analysis speed and prediction reliability are also major concerns in stock market prediction systems. Several machine learning techniques and their hybrid utilizations were suggested to address these concerns. ANNs, their variants and their hybridizations with other algorithms have been widely preferred to solve the market parameter prediction problems: Bisoi and Dash have proposed a hybrid method that implements an infinite impulse filter-based dynamic neural network (DNN) and an unscented Kalman filter for stock price prediction (Bisoi and Dash 2014). In another study, an efficient and interpretable neural fuzzy system was presented for stock price prediction (Rajab and Sharma 2019). To improve accuracy in predicting the next day's closing price, they selected the most effective technical indicators using Pearson's correlation coefficient in preprocessing of data (Rajab and Sharma 2019). Kumar aimed for an efficient model to predict the future price of the stock market using technical indicators that were derived from historical data. An Elman neural network (ENN) was used for its ability to memorize historical information and a gray wolf optimization algorithm is used to optimize the ENN parameters (Kumar Chandar 2021). In another work, Yang et al. optimized neural networks in order to develop a stock market prediction model by using a Particle swarm optimization (Yang et al. 2021). Recently, a hybrid model based on the echo state neural network was used for prediction of stock price return volatility (Trierweiler Ribeiro et al. 2021). Alternatively, deep neural networks have been used for stock market prediction problems in recent years (Pang et al. 2020;Hiransha et al. 2018). In a recent study, the movement of NIFTY 50 index was predicted by using three different machine learning methods that are the support vector machine, the perceptron and logistic regression and high prediction accuracy was reported (Parray et al. 2020). This work also provides a comprehensive review of the literature and the definition of useful technical indicators.
A primary objective of the stock market investment is to reach a high revenue (Thakkar and Chaudhari 2021;Gandhmal and Kumar 2019). In the classic approaches, the stock markets were modeled by using stochastic models but the stochastic market model was not adaptive and they cannot adapt themselves for changing dynamics of the stock markets data (Lo and MacKinlay 1988). Uncertainty in local or global factors and chaotic behavior in daily investment trends causes unpredictable fluctuations of stock market indices, and this makes consistent and longterm forecasting of stock market parameters a difficult problem. In order to maintain the consistency of financial market models, adaptability of prediction models to changing market dynamics is necessary and evolutionary computation methods were suggested to enhance financial models (Nikolaev and Iba 2002). To benefit from adaptation ability of the evolutionary computation approaches in the finance market modeling, several computational schemes were implemented in computational finance; for instance, GP algorithms, learning classifier systems, multiobjective evolutionary algorithms (MOEAs), co-evolutionary optimization schemes, and evolutionary algorithms (Nikolaev and Iba 2002). Also, hybrid methods, which combine neural computation with evolutionary computation, were proposed for the market trend prediction problems (Akbilgic et al. 2014). Akbilgic et al. suggested a hybrid radial basis function neural networks (HRBF-NN) with model selection by using genetic algorithms. Akbilgic et al. gave a brief discussion on the predictability of daily trend of the market index and suggested consideration of markets as evolutionary and adaptive systems rather than a physical or dynamic system in order to obtain more consistent prediction models (Akbilgic et al. 2014). This observation becomes the main motivation of the current study. The GpOls algorithm is implemented for the modeling relations between daily trends of several international market indexes. To enable adaptation of the GpOls algorithm according to the changing dynamics of stock market trends, the DE algorithm is adopted to self-tuning of the GpOls algorithm. This hybridization of the DE and GpOls algorithms allows the realization of hyperparameter optimal GpOls agents and improves the data-driven modeling performance. Thus, the daily trend forecasting models of the GpOls algorithm can be automatically and progressively updated with the most recent data from a modeling horizon. These up-to-date models of optimal GpOls modeling agents can produce more consistent and accurate forecast models to estimate daily trends during a forecasting horizon of the market. Accordingly, the progressive model update allows adaptation of the proposed DEHypGpOls algorithm to the changing market dynamics in the modeling dataset. It is useful to provide taxonomy of the GP-based financial data modeling works in the literature. Table 1 lists some GP-based studies and their properties, which are useful for the data-driven adaptive modeling of the financial markets. Previously, non-optimal GP algorithms have been implemented for the solution of several financial market problems. GP algorithms were used for financial market modeling without hyperparameter optimization concern (Nikolaev and Iba 2002). Karatahansopoulos et al. demonstrated the application of gene expression programming (GEP) for modeling the Greek index (Karatahansopoulos et al. 2014). In another stock market prediction method study, GP was implemented to detect arbitrage opportunities in the London stock market index (FTSE-100) (Markose et al. 2001). Garcia-et al. used decision trees (DT) that were generated by GP in order to detect primary financial incident changes (Garcia-Almanza and Tsang 2006). In recent work, Ari et al. discussed use of an ensemble average GP model to represent stock exchange market trend data based on other international market data (Ari and Alagoz 2021).
2 Theoretical background and preliminaries 2.1 Why is hyperparameter optimization important for practical machine learning applications?
Hyperparameter optimization techniques aim to find an optimal set of hyperparameters of the algorithm in order to obtain the best machine learning model for a given dataset. In general, the hyperparameter optimization achieves the parametric optimality of machine learning tools for a training dataset, and it contributes to the practical performance of data-driven modeling methods by auto-tuning the user-defined parameters of algorithms. An important requirement for the optimization techniques, which are used in the hyperparameter optimization task, is fast convergence to optimal points. Otherwise, in cases that machine learning algorithms have high computational load or modeling dataset is too large or big data, the hyperparameter optimization task leads to severe rise of the computational load in the data-driven modeling task and becomes an impractical task for real-world applications. The well-tuned machine learning algorithms yield the best models for a given datasets (Burke et al. 2003).There are generally two tuning methods for user-defined parameters of the machine learning methods. These are manualtuning and automatic-tuning of parameters (Wu et al. 2019). In some cases, it is quite difficult to manually configure a set of the optimum parameter values in computational intelligence methods (Trujillo et al. 2020). In a data-driven intelligent system point of view, the hyperparameters of machine learning algorithms should be automatically tuned without a need for the expert knowledge on the dataset and/or manual configuration on the machine learning algorithms. Besides the requirement of optimal configured algorithms in data-driven modeling tasks, when the amount of data is quite high (e.g., big data) or a live data stream is used for real-time modeling, auto-reconfiguration of algorithms, namely hyperparameter optimization of algorithms, is a necessity in online real-world applications. This fact was plainly stated in the machine learning literature (Yang and Shami 2020). The automatic machine learning (AutoML) and adaptive deep neural networks need the optimal tuning of many parameters of algorithms (Feurer and Hutter 2019). In general, hyperparameter optimization offers the following plus points (Feurer and Hutter 2019): • System Autonomy and Adaptivity: The machine learning tasks require less human efforts or expertise when hyperparameters of machine learning tools are automatically configured. • Optimal System Performance: Parametric optimality of machine learning algorithms improves performances of intelligent systems. • Practical Effectiveness: Hyperparameter optimization maximizes the benefits and applicability of machine learning algorithms in real-world applications.
In the current work, authors have preferred an evolutionary hyperparameter optimization technique to increase the accuracy of GP-based stock market trend prediction models. In the previous work, authors demonstrated extracting a regression model of ISE100 index stock market by using ensemble average GpOls modeling method (Ari and Alagoz 2021). However, this approach cannot yield an optimal model for solutions of the day-ahead forecast problem. The main reason is that user parameters of GP algorithms were manually configured for a limited number of set and trial efforts, and it does not ensure the hyperparameter optimality of GP algorithms for the modeling dataset. In the current study, to alleviate this shortcoming, we proposed an evolutionary hyperparameter optimal GpOls algorithm, namely the DEHypGpOls algorithm.

A brief introduction of differential evolution algorithm
Differential evolution (DE) algorithm is an effective evolutionary optimization method that can iteratively improve candidate solutions for an optimization problem by considering values of a predefined objective function Price 1995, 1997;Cui et al. 2018). It implements essential genetic processes (mutation, crossover and selection) by using relatively simple formulas to form a new generation of candidate solutions and maintain the best candidates through generations by selecting them according to their fitness performance. A brief introduction of the DE algorithm was summarized as follows (Qin et al. 2009): A candidate solution of the DE algorithm is expressed in D-dimensional parameter space in the form of where the subscript G represents the generation number and the parameter N p is the population size of the candidate solution set. The D-dimensional parameter vector expres-ses a search space where the algorithm searches for an optimal solution. At the initialization stage of the DE algorithm, the individuals of the population are randomly distributed into the search space (Qin et al. 2009). Evolutionary processes, i.e., mutation, crossover and selection processes are employed to obtain new generation candidate solutions, and a group of the candidate solutions is selected according to their fitness values in order to improve generations with better fitting solutions. Fitness values are calculated according to a predefined objective function of the optimization problem. As a result of repeated improvement of generations based on evolutionary processes, the candidate solutions evolve towards the optimum solutions that are defined by the objective function (Qin et al. 2009).
To form an initial population of candidate solutions, candidate solutions are distributed uniformly into the entire search space by using uniform random numbers within the search ranges that are bounded by predefined upper and lower boundaries X min ¼ x 1 min x 2 min x 3 min ::: Qin et al. 2009). In the current study, the following formulations of DE algorithm are implemented for the genetic processing: (i) Mutation Process: The mutation process of the DE algorithm forms new candidate solutions of the population by using differences of the selected solution vectors. The new candidate solutions, which are represented by the vector set , are formed in the search space. Several mutation strategies have been suggested in the literature. We implemented the ''DE/rand/1'' strategy that formulated the mutation as follows (Qin et al. 2009).
where the subscripts r i 1 , r i 2 , r i 3 are randomly selected numbers of individuals from the population in the range of ½1; N p . The parameter S is the scale factor that is used to adjust length of the difference vector ðX r i 2 ;G À X r i 3 ;G Þ (Qin et al. 2009). The scaled difference vector S:ðX r i 2 ;G À X r i 3 ;G Þ determines the range of alterations in the mutation process of solutions. (ii) Crossover Process: After the mutation process, a crossover operation is performed with a probability rate of C r to form new individuals of the population (Qin et al. 2009).The new candidate solution of the crossover process are represented by the vector set U i; The parameter C r is the crossover rate that is used to adjust the frequency of crossover operations and new candidate solutions are formed by randomly selected element of the mutated solution V i;G (Qin et al. 2009).
(iii) Selection Process The selection process is an essential process of differential evolution that allows the maintenance of good candidate solutions through generations of evolution process. The objective function value of new individuals and old individuals are denoted by f ðU i;G Þ and f ðX i;G Þ, respectively, and the best fitting individuals for a minimization problem are selected according to objective function values as follows (Qin et al. 2009).

Genetic programming preliminaries and GpOls algorithm
GP is an evolutionary computation (EC) methodology that can automatically formulate an optimal solution of modeling, design and computer programming problems (Koza 1992). The GP has become a popular EC technique because of its success in applications and its potentials to generate effective automatic modeling solutions for problems from many science and engineering fields (López-López et al. 2019; Koza et al. 2004;Dal Piccol Sotto and De Melo 2014). In GP literature, the automatic problem solving ability of GP algorithms is widely utilized in symbolic regression problems for data-driven modeling works (Dal Piccol Sotto and De Melo 2014). The GP algorithms have been used in diverse application areas, for example production scheduling (Nguyen et al. 2017), optimal water reservoir-operating (Ashofteh et al. 2015), energy of residential buildings (Castelli et al. 2015;Kaboli et al. 2017;Tahmassebi and Gandomi 2018), educational technologies (Zafra and Ventura 2012), urban planning (Patnaik and Bhuyan 2016), geotechnical design (Keshavarz and Mehramiri 2015), hydrology (Shoaib et al. 2015), and medicine (De Falco et al. 2018). Also, GP has been widely utilized in many computer science problems such as classification problems (Tran et al. 2016;Kuo et al. 2007), computer vision(Liu et al. 2016, image processing (Shao et al. 2014;Liang et al. 2020), signal processing (Feli and Abdali-Mohammadi 2019), and artificial neural network design (Suganuma et al. 2017). Moreover, GP were used in the field of evolutionary hardware (Mora et al. 2019) and circuit design (Sikulova et al. 2014;Koza et al. 2004). One can find many application of GP in the field of economy [e.g., financial fraud detection (Li and Wong 2015) and green supplier selection (Fallahpour et al. 2016)]. Figure 1 shows an example expression tree of a GP model. This tree is a graphical expression of the population individual that represents a solution model The genetic processes such as crossover, mutation produce new GP expression trees, and those better fitting to solution of the problem, are selected to compose new generations of GP population individuals. During the repeating cycles of genetic processes, the expression trees are evolved towards better fitting trees, and they express more suitable candidate solutions for a problem.
GpOls algorithm is a tree-based GP algorithm that involves Orthogonal Least Square (OLS) calculation. The OLS adaption to the GP algorithm improves fitting of the nonlinear mathematical model to the modeling dataset (Madár et al. 2005). The GpOls algorithm was proposed for the identification of nonlinear input-output models by implementing a tree-based genetic programming technique (Madár et al. 2005). It was used for the nonlinear model identification in control application (Wiese 2011). A major advantage of GpOls algorithm is the production of accurate and interpretable models (Madár et al. 2005). Essentially, it implements an orthogonal least square algorithm to Fig. 1 An example GP tree structure that represents y ¼ ðx 1 þ x 2 Þ À ðx 1 þ x 3 Þ Â x 2 solution model estimate the contribution of each nonlinear model fragments (components), expressed by subtrees of expression trees, to a resulting linear-in model and thus yields a robust and interpretable models in the form of weighted sum of nonlinear model fragments (Madár et al. 2005). The linearin model of GpOls algorithm was formulated in the form of where F i ðxðkÞÞ stands for the nonlinear operator model fragments (functions or terminals) that are expressed by subtrees of expression trees (Madár et al. 2005). The parameter p i is the weights of subtree models. Input vector of this linear-in model was written by These methods can use an operator set that involves linear operators, i.e., {þ, À} and nonlinear operators, i.e., {Â, =}. In operation, the GpOls algorithm initially decomposes expression trees into the subtrees of nonlinear model fragments that are initiated with nonlinear operator nodes or a terminal node as described in Fig. 2. Each of these subtrees forms a nonlinear function or a terminal that is denoted by F i ðxðkÞÞ in the linear-in model formulation (Eq. (5)). Figure 2 shows an example decomposition of the expression tree (in Fig. 1) into subtrees of the nonlinear functions and terminals to compose a linear-in model. Whenever the root node is a nonlinear operator or a terminal, it forms an element of the function set F i ðxðkÞÞ to be considered in Eq. (5) (Madár et al. 2005).
In Fig. 2, the selected subtrees produce the functions F 1 ¼ x 1 , F 2 ¼ x 2 and F 3 ¼ ðx 1 þ x 3 Þ Â x 2 :. Then, the linear-in model can be written by To calculate p i weight coefficients, the least square method was used to fit a dataset. Eq. (5) was written in the vector form for whole modeling dataset as Madár et al. (2005) where the matrix F denotes the regression matrix, the parameter p is a vector of weights p i and e represents the error vector for this regression problem. It is solved by using an orthogonal least square technique that transforms the columns of the F matrix into a set of orthogonal basis vectors (Madár et al. 2005).

Problem statement
This study introduces a hyperparameter evolutionary optimal GpOls modeling scheme in order to obtain improved data-driven prediction models. For this purpose, the DE algorithm is implemented for evolution of GpOls modeling agents within the hyperparameter search space of GpOls algorithm. Figure 3 depicts a general view of the proposed DEHypGpOls scheme that enables evolution of GpOls modeling agents in order to yield better models that can represent a modeling dataset. The DEHypGpOls algorithm performs the following stages.
(i) Differential evolution algorithm configures hyperparameter code of GpOls agents within the hyperparameter search space of GpOls algorithm. (ii) Performances of the GpOls modeling agents are measured according to their data-driven modeling performance indices for a given modeling set. (iii) DE algorithm performs hyperparametric evolution of next-generation GpOls agents in order to increase their data-driven modeling performance. (iv) The best model from GpOls agents population, which has the best modeling performance, is selected as the best hyperparameter optimal model of the DEHypGpOls algorithm.
This evolutionary optimization process optimizes a hyperparameter code that expresses user-defined parameters of the classical GpOls algorithm. This process autonomously improves the modeling performance without a need of expert configuration on the GpOls algorithm. This perspective contributes to development of autonomous data-driven modeling algorithms for real-time applications. Experimental results in the Sect. 4 indicated that the data-driven modeling performance of the GpOls modeling agents could be improved by the DEHypGpOls algorithm. A main reason for this improvement is that the GpOls modeling agents evolve in the hyperparameter search space of the algorithm to yield optimal models that can better represent the modeling dataset. This modification allows the adaptation (self-tuning) of the GpOls algorithm according to the dataset content and it can contribute to the data-driven modeling performance of GpOls modeling agents via the DEHypGpOls algorithm. Steady decreases in MSE values of the model during the differential evolution of GpOls agents apparently validate this property.

DEHypGpOls: an evolutionary optimization scheme for hyperparameters optimal GpOls algorithms
The main focus of this study is to generate a-day-ahead forecast model that can predict the next-day behavior of the ISE100 index. Our tests showed that modeling performance of the GpOls algorithm depends on the configuration of hyperparameters of the algorithm. Therefore, authors adopted a DE algorithm to obtain evolutionary optimal-tuned GpOls programming agents. Accordingly, the GpOls algorithm itself is considered as programming agents that evolve in the search space according to a set of hyperparameter code. The DE algorithm optimizes the hyperparameters of the GpOls agents according to their data-driven modeling performance. Therefore, the GpOls programming agents are evolved to the best hyperparameter configuration that yields the best fitting model for a-day-ahead forecast of stock market index trend. In this way, the proposed evolutionary hyperparameter optimization minimizes human intervention and allows reaching an optimal model generation performance of the GpOls algorithm. These evolutionary hyperparameter optimization improves: (i) autonomy of the GpOls algorithm by adapting initial settings of the algorithm according to characteristic properties of dataset and modeling problem, (ii) model generation performance by selecting the best model from evolving GpOLs agents, (iii) practical effectiveness of GpOls by reducing the need for configuring by a human expert. Figure 4 shows a block diagram of the proposed DEHypGpOls method. According to this flow chart, each GpOls modeling agent is evolved to produce a model with a better Mean Square Error (MSE) performance. After completion of differential evolution of GP agents, the GP agents provide the best model, which fits to the modeling dataset with the lowest MSE. Then, the performance evaluation of the best model is carried out in a simulation environment according to a forecasting dataset.
According to the flowchart in Fig. 4, steps of the proposed DEHypGpOls algorithm can be written as follows: Step 1: Generate randomly initial hyperparameter solutions by using the DE algorithm.
Step 2: Run GpOls modeling agents and obtain datadriven models by using modeling dataset.
Step 3: Evaluate MSE performance of this model according to the modeling dataset.
Step 4: Produce next-generation hyperparameter solutions by applying genetic processes (mutation, crossover and selection) of the DE algorithm. Fig. 3 A schema that conceptualizes the evolution of GpOls modeling agents according to the twodimensional hyperparameter search space and generation of the best model with the highest performance GpOls agents Step 5: Select hyperparameter solutions that have lower MSE values and form a set of hyperparameter solutions that better represents the modeling dataset.
Step 6: If the stopping criteria is not met, turn to Step 2. Otherwise, select the best model with the lowest MSE value and stop the program.
In the stock market trend prediction application, dayahead forecasts of the ISE100 and BIST100 index trends are performed by using the current day trends of these markets and the other 7 world stock markets. Since stock markets are highly dynamic and uncertain, several unpredictable factors are effective, forecast models can represent market behavior for a limited time horizon, and they need adaptation to up-to-date conditions of market trends. To keep the model up-to-date, P numbers of previous days' data can be used to compose a modeling dataset to generate a prediction model. These P numbers of previous days are called the modeling horizon. The forecasting dataset is composed of L numbers of the following days' data. The L numbers of the following days in the near future are called the forecast horizon. This temporal modeling approach based on finite time-horizon modeling and forecasting stages is referred to as progressive model update in this study. Figure 5 shows a diagram for this finite time-horizon modeling and forecasting scheme. Performance of the best model is evaluated according to a forecasting dataset that is not involved in the modeling process. This model generation approach keeps the forecasting model up-to-date with data of the most recent P day period and enables consideration of the latest P day changing market dynamics in the forecasting model. A similar finite time-horizon solution has been implemented in the model predictive control since the 1980s. Current response of the control system is optimized by considering future time slots that are called prediction horizon (Kwon et al. 1982;García et al. 1989) and it is a very effective method for optimality of control system responses in changing environments and conditions. Similar approaches are also used for neural networks to keep the neural model up-to-date for changing training dataset, for instance, progressive neural network (Gideon et al. 2017), transfer learning (Zhuang et al. 2021  the performance evaluation of furcating models was performed by using the forecasting datasets and stock market investment simulations in the MATLAB environment. All experiments were conducted by using a PC that has a fourcore Intel(R) Core(TM) i7-5700HQ CPU 2.70 GHz and 16 GB RAM. The GpOls is an effective software tool that was developed to generate polynomial regression models in the MATLAB environment by using a tree-based GP algorithm with an OLS improvement (Madár et al. 2005). Table 2 introduces the hyperparameter code [C1 C2 C3 … C9 C10] that was used to optimize of GpOls agents by the DE algorithm. These hyperparameters can affect modeling performance of GpOls toolbox (Madár et al. 2005;Wiese 2011) The MATLAB DE tool, which was provided by Mostapha Kalami Heris from Yarpiz, was modified to optimize the hyperparameter code of GpOls agents. Figure 6 shows a pseudocode for the objective function implementation in order to optimize hyperparameters of GpOls agents. In performance tests, the ANN models have 2 hidden layers that have 5 neurons at the first hidden layer and 2 neurons at the second hidden layer. Similar to the DEHypGpOls forecasting model, the ANN regression model is used to estimate daily trend values, and these estimates are used to generate buy or sell signals (BS).
Dynamic and complex relations between international stock market indices were learned by forecasting models, and the modeling relation between many market trends can provide more correlation and contribute to accuracy of the long-term forecasts. Also, the progressive model update for the finite-time horizon modeling and forecasting enables the adaptation of forecasting models to the recently altering dynamics and relations in the stock markets.
The Istanbul Stock Exchange (ISE100) dataset (Akbilgic et al. 2014) and new BIST100 datasets (parts 1-3) were used in this experimental study. Table 3 introduces the parameters of the forecasting model. By using data in the table, a-day-ahead trend forecasting models were obtained by using the DEHypGpOls algorithm and ANNs. The next day trend forecast (y nþ1 ) is expressed as a function of the current day trends of SP (x 1;n ), DAX (x 2;n ), FTSE (x 3;n ), NIK (x 4;n ), BVSP (x 5;n ), EU (x 6;n ), EM (x 7;n ) and the forecasted trend (y n ). This a-day-ahead prediction model can be expressed in a recursive form according to the day index n as follows: y nþ1 ¼ f ðx 1n ; x 2n ; x 3n ; x 4n ; x 5n ; x 6n ; x 7n ; y n Þ ð 9Þ Figure 7 depicts a block diagram of an active stock market index forecast system that is composed of a  Fig. 6 A pseudocode to implement the objective function in Fig. 4 database for hosting market data, DEHypGpOls hyperparameter optimizer and an evolutionary optimal forecaster. To evaluate the prediction performances of the evolutionary optimal model within the time frame of forecasting horizon, the mean square error (MSE), the mean absolute error (MAE), relative absolute error (RAE) and the root mean square error (RMSE) are used: where the parameter y d;nþ1 is actual value of ISE100 index trend within forecasting horizon (ground truth data) and the parameter y nþ1 denotes the day-ahead forecasts of the optimal forecasting model. The accuracy of buy-sell recommendation was calculated by using true buy signal (TB), false buy signal (FB), true sell signal (TS) and false sell signal (FS) as follows

Experimental results and investment simulations
In the experimental study, the past 335 days daily trends of Stock Exchange (ISE100) dataset were used to compose a modeling set to generate an evolutionary optimal forecasting model (a modeling horizon with P = 335), and this model was used to predict the next 200 days daily trends of ISE100 index (a forecast horizon with L = 200). The DEHypGpOls algorithm produced a-day-ahead forecast model of ISE100 index trend depending on the other international index as y nþ1 ¼ 0:19683 Ã ðx 1;n À ððx 6;n À ððððy n À x 6;n Þ þ ðx 4;n þ x 5;n ÞÞ À x 7;n Þ À x 7;n ÞÞ À x 1;n ÞÞ þ 0:56828 Â 10 À4 ð15Þ Figure 8 shows decrease of MSE to reach the best GpOls agents when the optimizing hyperparameters of the GpOls agents. This decrease in MSE validates the adaptation of GpOls modeling agents according to the modeling set. The resulting evolutionary optimal model, which is given by Eq. (15), provided a buy-sell recommendation accuracy value of 68% for a forecast horizon of 200 days for the ISE dataset.
To analyze investment performance, a stock exchange market investment simulation was carried out by using the BIST100 dataset. In this simulation, the automated daily investment was performed according to the buy-sell signal of the forecasting models. The buy or sell signal (BS) is generated according to the sign of daily trend forecast as follows: where BS ¼ 1 stands for a buy signal and BS ¼ À1 stands for a sell signal. Figure 9a shows actual ISE100 trend values (y d;nþ1 ) and the forecasted ISE100 trend values (y nþ1 ) and Fig. 9b shows corresponding buy and sell signals that were generated according to Eq. (16). Figure 10 shows true buy-sell recommendation instances with a value of 1 and the false buy-sell recommendation instances with a value of 0. According to Fig. 10, there are several periods where the consecutive true investment recommendations are produced. The false recommendations are mostly very short-term (a few consecutive days long) and they are often followed by the longer true recommendation periods. The long true investment periods cause gradually growing the incomes in the market investment simulation. On the other hand, the false investment recommendations are rather collected around intervals, where the actual daily trend values are near to zero values. For high values of the actual trend, the investment recommendations tend to become true. For this reason, the true recommendations can be more dominating on the movement of income than the false recommendations and this contributes to increasing the revenue. Table 4 shows accuracy in buy-sell recommendations for forecasting models of the DEHypGpOls algorithm and the ANN for the ISE100 dataset (Akbilgic et al. 2014). Table 5 summarizes overall performances of these models for the same dataset.
To validate the performance evaluations, we generated new datasets that are composed of the daily trend of international stock market indexes similar to Akbilgic et al.'s ISE100 dataset. We collected data from different   time slots of publicly available stock exchange market records and composed consecutive BIST100 dataset series (part 1-3). A list of these datasets, some trend statistics and modeling and forecasting partitioning of datasets are summarized in Table 6. (Istanbul Stock Exchange (ISE) was combined to the Borsa Istanbul (BIST) and then this market index was renamed as the BIST100 on April 5, 2013.) For the BIST100 datasets, the DEHypGpOls algorithm produced the following day-ahead forecasting models. y1 nþ1 ¼ À0:089099 Ã ðððx 5;n À x 7;n Þ À x 3;n Þ À ðx 5;n þ ðx 1;n À ððx 6;n þ ðx 1;n þ x 3;n ÞÞ Ã y n ÞÞÞÞ þ 1:33451 Â 10 À4 ð17Þ y2 nþ1 ¼ 3:6034 Ã ððððx 5;n Ã x 3;n Þ Ã Àððx 5;n þ x 4;n Þ þ ðððx 7;n Ã Ày n Þ þ ðx 2;n þ x 4;n ÞÞ þ ðx 5;n Ã x 5;n ÞÞÞÞ Ã ðx 7;n þ y n ÞÞ Ã Àðððx 6;n þ ðððy n Ã x 5;n Þ þ ðy n À x 5;n ÞÞ þ x 1;n ÞÞ þ y n Þ þ ðððx 3;n Ã ðx 7;n À y n ÞÞ À ðx 4;n À ðx 7;n À x 2;n ÞÞÞ þ ðx 5;n Ã x 6;n ÞÞÞÞ ð18Þ y3 nþ1 ¼ 0:03205 Ã ðððððy n À ðx 3;n À x 1;n ÞÞ þ ðððx 4;n À x 5;n Þ À x 1;n Þ Ã x 3;n ÞÞ À x 6;n Þ À ððððy n À ðx 3;n À x 1;n ÞÞ þ ððx 3;n À x 1;n Þ Ã x 1;n ÞÞ À x 6;n Þ þ x 4;n ÞÞ Ã ððx 3;n À x 1;n Þ Ã ðx 2;n À x 1;n ÞÞÞ where y1 nþ1 is the forecast model for the BIST100 dataset (part 1), y2 nþ1 is the forecast model for the BIST100 dataset (part 2) and y3 nþ1 is the forecast model for the BIST100 stock market (part 3). Time slots of these dataset series and application periods are illustrated in Fig. 11. The modeling and forecasting dataset sizes for these model updates can be found in Table 6. Figure 12 shows daily income curves from the stock market investment simulations for the ISE100 dataset (Akbilgic et al. 2014). In the figure, incomes of an automated daily investment system, which uses buy-sell signals from a-day-ahead forecasting model of DEHypGpOls algorithm and a neural forecaster with ANN model, were compared with the income of a long-term investment strategy. To obtain more reasonable results, 0.1% trading fee per buying and selling operation was used in these market simulations and we disabled the short selling option (See ''Appendix'' for details of the stock exchange market investment simulation and a pseudo-code for the implementation of this simulation.) Table 7 shows accuracy values in buy-sell recommendations of these forecast models for the ISE100 and BIST100 datasets (part 1-3). Daily income curves are shown in Figs. 13, 14 and 15 for several investment strategies. These strategies are the long-term investment strategy, daily investment strategies according to buy-sell recommendations from the DEHypGpOls forecaster and the neural forecaster. Table 8 lists final incomes per 1 $ investment at the end of 200 days. Simulation results showed that daily investment according to buy-sell recommendations from DEHypGpOls models could provide more average income for these 4 different datasets. Under assumptions of the proposed market investment simulation, these simulation results reveals that the forecasting model of the DEHypGpOls algorithm may provide more income than the long-term investment strategy particularly in the accumulation phase of the stock market index, where index can goes up and down around a base line, because the short-term trading with more true buy-sell recommendations than the false recommendations can produce more income relative to the long-term investment strategy as in Figs. 12, 13 and 14. However, in case of markup cycles of the markets similar to the uptrend in Fig. 15, the long-term investment strategy becomes more advantageous compared to the short-term investments with the forecaster of the DEHypGpOls algorithm because the long-term strategy can continuously earn in the case of the consecutive positive daily trends in uptrend periods of the stock market as illustrated in Fig. 15. Here, trading fees and false sell recommendations of the DEHypGpOls forecaster can lead to losses in these uptrend periods, and these effects can reduce incomes of the DEHypGpOls forecasting model compared to incomes of the long-term investment in Fig. 14. Table 9 shows the regression performance measures (MSE, MAE, RAE, RMSE) for the BIST100 datasets.
The market investment simulations were conducted with four datasets. The simulation results shows that daily investments according to the evolutionary optimal Fig. 12 Daily income curves of the long-term investment strategy, daily investment strategies (the short-term strategy) based on the forecasters of the DEHypGpOls algorithm (y nþ1 ) and the ANN from the investment simulations for the ISE100 dataset

Conclusions
This study introduced an evolutionary hyperparameter optimal genetic programming framework for a-day-ahead trend forecasting for ISE100 and BIST100 datasets. The ISE100 and new BIST100 datasets (part1-part3) form a total of 2265 days of data. The forecast model of the DEHypGpOls algorithm can provide an average 57.87% accuracy in buy-sell recommendations for four different dataset time slots. The market investment simulations showed that daily investments according to the buy or sell signal of the forecasting model of DEHypGpOls could provide an average 4.8% more income compared to the average income of the long-term investment strategy for Fig. 15 Daily income curves of the long-term investment strategy, daily investment strategies (the short-term investment) based on the forecasters of the DEHypGpOls algorithm (y3 nþ1 ) and the ANN from the investment simulations for the BIST100 dataset (part 3) datasets from four different time slots. A 0.001 (the rate of 0.1%) trading fee per the daily buying or selling operations was applied, and the short-selling were disabled in the investment simulations. The trading fee causes losses in incomes in case of frequent buy-sell operations, and this can reduce incomes of short-term investments in the case of the DEHypGpOls and ANN forecasters. Some remarks of the study can be summarized as: • The hyperparameter optimization can improve modeling performances of the GP algorithms. The hyperparameter optimization should be considered as an integral part of the data-driven modeling tools. • The DE based evolutionary optimization of hyperparameter codes of GpOls agents can autonomously generate evolutionary optimal data-driven models without a need for the expert knowledge on the algorithms as well as the modeled data. This is a substantial property for the fully automated data-driven modeling systems. • Autonomous financial parameter forecasting tools can be helpful to reduce investment risks. However, performance of forecasting tools has a vital importance. Due to the high uncertainty and changes in market dynamics, forecaster models should be updated according to recent financial data. Consideration of several financial parameters enables to use correlation or relations between these parameters in modeling and this can help reducing negative effects of uncertainty. In the current study, relations between seven international stock market trends improve the modeling performance.
Primary aim of this research is demonstration of an evolutionary hyperparameter optimal GpOls tool for datadriven modeling applications. Authors preferred for financial parameter forecasting application because it introduces a very difficult data-driven modeling problem. Results in this study are based on basic assumptions of the market investment simulation model that was explained in the ''Appendix'' section. These performance results were not verified in a real investment environment, and authors do not recommend use of this algorithm in a real investment application to avoid possible loss risks.

Appendix
To implement stock exchange market investment simulation, the daily trend of stock index is expressed by considering 24 h nominal return rate as. trend n ¼ ðI n À I nÀ1 Þ I nÀ1 ; where I nÀ1 is the previous closing time index and I n is the current closing time index. (The daily trend data in BIST100 datasets were generated according to this formulation) Then, when the total capital is invested for an stock market index, the daily change in total capital (C n ) with respect to the daily trend of the stock index is calculated as Then, the current total capital is updated by adding daily change in total capital (DC n ) to previous total capital as C n C nÀ1 þ DC n ¼ C nÀ1 þ trend n C nÀ1 This recursive relation estimates the current total capital. When the total capital is kept in currency (no investment in the stock market), the total capital in currency does not change depending on the daily trend of the stock index.

C n C nÀ1
When an investor preference is limited to choosing one of these two states (the stock index investment for s n ¼ 1 and the currency investment for s n ¼ 0) in the market, the current total capital can be updated as C n C nÀ1 þ trend n C nÀ1 s n ¼ 1 C nÀ1 s n ¼ 0

& '
A trade commission (stock trading fee), which is charged per buying or selling operation, is applied as C n ð1 À T f ÞC n ; where T f is the commission rate per the operation. A pseudocode for the market investment simulation is given, bellow: Author contributions All authors contributed to the study conception and design. The analysis and code were done by DA and BBA. The first draft of the manuscript was written and then polished by DA and BBA. All authors read and approved the final manuscript.
Funding This study didn't receive any funding.
Data availability Enquiries about data availability should be directed to the authors.

Declarations
Conflict of interest The authors declare that they have no conflict of interest.
Ethical approval This article does not contain any studies with human participants or animals performed by any of the authors.
Informed consent Informed consent was obtained from all individual participants included in the study.