Artificial Neural Network for analyzing the chaotic time series motion: The case of the Lebanese GDP

In this paper, we propose to analyze the motion of the Lebanese GDP over the period 1950-2019. This macroeconomic aggregate reveals large fluctuations notably during the civil war period (1975-1990). By estimating the Lyapunov exponents with the Artificial Neural Network (ANN) procedure, we show that this series exhibits a strange attractor generated by a chaotic dynamic and we use the embedding procedure to shed in light the bizarre structure of such a series. Thus, the ANN method gives better results regarding prediction than other linear regression models and allows to fit with accuracy the chaotic motion followed by the Lebanese GDP in the phase space.


Introduction
The analysis of time series motion is crucial in the economics and financial field where forecasting plays a significant role in the economic policy of a country. Such an analysis is more complicated as series exhibit a chaotic dynamic as is the case in most developing countries. For example, Lebanon is a country known for its political instability, numerous conflicts, and high level of corruption that lead to large volatility of macroeconomic time series as the GDP. So, some models, as the Artificial Neural Network, are relevant for predicting the motion of series which present chaotic dynamic as the Lebanese GDP.
Chaos theory is primarily used in the meteorology fields (Lorenz, 1960(Lorenz, , 1972. The main insight behind this concept is that even a simple deterministic system can sometimes produce unpredictable situations notably when the system has a sensitivity to initial conditions in the short run. This theory involves the strange attractor concept to which the trajectories of a variable have a bizarre structure. The inclusion of the chaos theory in economic analysis is not recent. For example, Mandelbrot (1963) analyses the chaotic variation of speculative prices. Kesley (1988), using the overlapping generation model, assert that economics models involve chaos. Baumol and Benhabib (1989) present what is chaos, how it works, and whether it does or does not occur in economic phenomena. Viad and al., (2010), taking an example of chaos in exchange rates, show that chaos theory is related to the notion of nonlinearity. Verne and Verne (2019) analyze the chaotic dynamic of the Lebanese GDP growth rate by using the Lorenz model including a system of differential equations.
Thus, the chaos theory presents an advantage with respect to time series methods because the main macroeconomic aggregates (which reflect economic behaviors) are mostly governed by non-linear dynamics. In fact, nonlinearity is relevant in economics because it paves the way to the study of cyclic, non-periodic, and chaotic behavior. Therefore, for estimating the evolution of a chaotic time series, the neural networks models seem pertinent.
Like the chaos theory, the artificial neural network (ANN) approach does not primarily come from social sciences like business or economics. It's an information processing system that is inspired by the models of biological neural networks. This approach is an adaptative system that changes the internal information that flows through the network during the training phase (Gnana Sheela and Deepa, 2013). So, the ANN approach is largely used for forecasting the chaotic times series that mandatorily includes non-linearity as in economics where many variables show most of the time a nonlinear and even chaotic motion. The features of this approach consist of a strong capacity for nonlinear mapping, high accuracy for learning, and robustness. It is also potentially suitable for nonlinear chaotic series prediction as they can perform complex mappings between their input and output spaces (Li and Lin, 2016). This paper proposes to use the ANN method for analyzing the evolution of a chaotic time series as the Lebanese GDP per capita from 1950 to 2019 which exhibits large fluctuations notably during the civil war period (1975)(1976)(1977)(1978)(1979)(1980)(1981)(1982)(1983)(1984)(1985)(1986)(1987)(1988)(1989)(1990).
The rest of the paper is organized as follows. Section 2 presents the characteristics of the Lebanese GDP over the period 1950-2019 and carries out the phase space reconstruction procedure from the Lebanese GDP series. It analyzes the motion of the Lebanese GDP by using the ANN approach for estimating the Lyapunov exponents that indicate if the time series behavior is chaotic or not. It also uses the ANN method for predicting the embedded time series motion and compares this method with the linear model. Section 3 discusses the results and Section 4 concludes.

Characteristics of the Lebanese GDP
Lebanon is a country that has experienced several civil conflicts that have had a significant impact on the evolution of the Gross Domestic Product (GDP) especially between 1975 and 1990. Therefore, from 1950 to 2019, Lebanon has known stabilities periods (peace periods) and conflict periods. These unstable political situations can explain why the evolution of GDP growth seems chaotic. Indeed, Figure 1, exhibits the GDP motion in level.  This test is a tool for detecting the null hypothesis of independence and identically distributed (i.i.d) against an unspecified alternative. This test is a necessary but not sufficient condition to confirm the chaotic behavior of the time series. The BDS test indicates if the series is linear or not.
It employs the spatial correlation concept through the calculation of the correlation integral.
The correlation integral is computed as follows: N is the number of observations; m designates the embedding dimension and  indicates the maximum distance between two pairs of points (i,j) with 1  i  N and 1  j  N, in the mdimensional space. If the time series is i.i.d, we have: The BDS statistic test BDS test is a two-tailed test. We should reject the null hypothesis if the BDS test statistic is greater than the critical values (e.g., if =0.05, the critical value = 1.96).
For the Lebanese GDP, the results of this test are in Table 1.  Table 1 indicates that for all dimensions the statistic BDS exhibits values greater than the critical value (since the probabilities are inferior to the 0.05 threshold). We can reject the null hypothesis of independence and identically distributed (i.i.d). This suggests that the Lebanese GDP is nonlinearly dependent, which is a necessary but not sufficient condition of chaotic behavior.

The reconstruction procedure
To search the dynamic chaotic of the Lebanese GDP, we use the embedding procedure to analyze the time series in the phase space.
The embedding procedure, that has been developed by Takens (1981) in his famous theorem, is used to carry out the reconstruction of the series in the phase space.
Let { =1 }, be the time series of the Lebanese GDP over the 1950-2019 period in annual frequency.
The data have been centered to improve the efficiency when we use the ANN procedure to estimate the time series motion 1 We form a sequence for delaying vectors (Ruelle and Takens, 1971) by associating for each period a vector in a reconstructed phase space, whose coordinates satisfy the following equation.
The parameter is the reconstruction time delay (or lag), m the embedding dimension and M = n -(m -1), the number of phase vectors.
As in the reconstruction theorem, we assume that the dynamical system is uniformly in time, that is the data come from a series of values measured periodically in time. Moreover, in a prediction model, there is an attractor that warps the observed data in the phase space and provides precise information about the dynamics involved. Thus, such a model can be used to predict Yt from Yt -1 in the phase space and synthesize Yt using time series reconstruction.
The problem is to find the optimal value of m and for reconstructing the time series in the phase space. To resolve this problem, Chang et Tong (2001)  For selecting parameters m and and compute the Lyapunov exponents, we use the neural network proposed by McCaffrey et al. (1992) and Nychka et al. (1992) and then revisited by Gençay and Dechert (1992) and by Shintani andLinton (2003, 2004).
To obtain a consistent neural network estimator, based on robust estimation of the function v, we approximate the unknown nonlinear function v through a feed-forward single hidden layer network with a single output by: Where Ф0 ∈ (identity matrix) is the estimated network bias from input, h is the number of neurons (or nodes) in the single hidden layer, ̂0 are the estimated layers connection weights from input to hidden layer, Φ is the transfer function, which in our case is the logistic function, ̂ is the estimated network bias from the hidden layer, m is the embedding dimension, and ̂ are the estimated layers connection weights from hidden layer to output.
To obtain the estimation of the parameters, we use the least-squares method by minimizing the residual sum of squares (RSS): From equation (3), we obtain the partial derivatives as elements of the Jacobian matrix J: We estimate the best-fitted feed-forward single hidden layer for choosing the parameters m, , and h. The best 10 models are estimated by assuming that the embedding dimension ∈ {1: 4}, the lag ∈ {1: 4} and the hidden neurons (with one layer) ℎ ∈ {1: 10}.
For choosing the best model, we take the Bayesian Information Criterion (BIC) that is defined as follows: In Table 2, models are sorted from lowest to highest BIC values. In this case, the best-fitted neural net model has the following parameter set values m = 4, = 1, and h = 2. The parameter h represents the hidden neurons with one layer for our purpose (which is explained below).
The phase space reconstruction of the Lebanese GDP is displayed in Figure 2. where axes x = Yt, y = Yt -1, and z = Yt -2. This embedding procedure shows that the trajectories of a GDP have a bizarre structure being neither simple smooth, nor continuous curves but fractals (Puu, 1997). Fractals (Mandelbrot, 1982) could be an indefinite set of unconnected points or a smooth curve with mathematical discontinuity, or a curve that is fully connected but discontinuous everywhere. So, trajectories of the variable exhibit a fractal structure meaning that even though GDP exhibits chaotic fluctuations in a certain period, as was the case between 1975-1990 during the civil war, a strange attractor does exist in the long run that pushes GDP to regain regular growth. Indeed, in this Figure

The Lyapunov exponents for analyzing the motion of the Lebanese GDP
The Lyapunov exponent measures the sensitivity of the series to the initial conditions. It quantifies the rate of separation of infinitesimally close trajectories. Depending on the initial conditions, this rate can be positive (divergence) or negative (convergence). More precisely, in the phase space reconstruction procedure, the Jacobian-based method fits a model to the data based on approximations of the trajectories in the reconstructed state space. Moreover, the Jacobian matrices of the estimated dynamic system are used to compute the Lyapunov exponents (Sandubete and Escot, 2021).
For estimating the Lyapunov exponents, we use the neural network which is an algorithm inspired by the human brain function to perform a particular task. This algorithm operates calculations through a process of learning.
Thus, using such a procedure, we estimate the following equation knowing that the number of dimensions m = 4 and the lags = 1.

̂= lim
Where eigk is the kth largest eigenvalue provided by the Jacobian matrix estimated above (equation We must calculate the eigenvalues from the Jacobian matrix to know the Lyapunov Exponents whose values indicate if the attractor is reduced to: -Stable fixe point: All the exponents are negative.
-Limit cycle: On exponent is zero and the remaining ones are all negative.
-Strange attractor generated by chaotic dynamic: At least one exponent is positive.
When the Lyapunov exponents are computed, we test their statistical significance by putting the following hypotheses: H0: ̂> 0 against the alternative hypothesis H1: ̂≤ 0. The rejection of the null hypothesis indicates the lack of chaotic behavior. For accepting or rejecting the null hypothesis, we compute the statistic as follows: ∅ is the variance of the kth Lyapunov exponent estimator and , , the quadratic spectral Kernel function.
Regarding the Lebanese GDP, we obtain the following Lyapunov exponents with their t-ratio and p-value. We can see that the first Lyapunov exponent is significantly positive. Indeed, we cannot reject the null hypothesis because the p-value (= 1.00) is larger than the critical threshold (generally fixed at five percent). On the contrary, we can reject the null hypothesis for the other Lyapunov exponents which are significantly negative. Consequently, the Lebanese GDP is seen as a strange attractor generated by a chaotic dynamic. Thus, the ANN method can be used for predicting the Lebanese GDP motion.

The Artificial Neural Network method for predicting the Lebanese GDP motion
The ANN for predicting Lebanese GDP motion is a set of connected input (Xt) /output (Yt) units, called neurons, in which each connection has a weight (w), playing the role of synapsis, associated with it. In the learning phase, the network learns by adjusting the weights to predict the correct class label of the given inputs. The inputs and output neurons are included in layers, called input layer and output layer respectively. The learning process is represented by the hidden layers h also including several hidden neurons. Hidden layers represent nonlinearity and interaction between input variables. More precisely, input neurons receive external signals which are fed by the network. These signals are moderated by some weights (Shahriary and Mir, 2016). According to this moderation, in each output, inputs are collected, and afterward, they are passed from the activation function which can take several forms (generally, linear, logistic, or hyperbolic tangent form).
The input layer receives only information and acts as an independent variable while the output layer acts as a dependent variable. The hidden layer (we can have one layer or several hidden layers) plays an important role regarding the accuracy of the prediction of the time series motion.
Indeed, it can influence the error on the neurons to which their output is connected. The stability of the neural network is estimated by error. The minimal error reflects better stability, and the higher error reflects the worst stability (Sheela et Deepa, 2013, p. 1).
The ANN uses two kinds of networks for predicting the motion of a non-linear or chaotic time series: The feedforward network and the feedback network. The first one is a not recursive network, that is, neurons in this layer are only connected with neurons in the next layer and they do not form a cycle. Feedback networks contain cycles. Signals travel in both directions by introducing loops in the network. For our purpose, we choose the feedforward network that is the most common in the economic model because of its conceptual complexity and computational efficiency (Marcek, 2006;Falat et al., 2015).
The ANN is an appropriate structure to build a nonlinear model of chaotic time series prediction.
For our purpose, we have three inputs xi = {Yt-1, Yt-2, Yt-3} and one output Yj. h1 and h2 are the hidden neurons included in one hidden layer, h. Thus, the number of neurons in the input layer xi, equals the number of embedding dimensions minus one (m -1), the hidden layer, h = 2, and the output layer Yt = 1. Technically, we obtain the following relation: We choose the hyperbolic tangent function as activation function Ф, which is the most used in the ANN model, so: With wij, the synapsis weights, that is, the link weight from the input layer to the hidden layer and bj which is the bias playing the role of the intercept in multiple regression models.
Our neural networks can be represented in Figure 3.

Figure 3: The neural network applied on the chaotic time series
When the hidden neuron takes a single variable as an input, it shows that the variable has a nonlinear main effect on the output variable. If the hidden neuron takes several variables as an input, as is the case in Figure 3, then the variables as an interactive effect on the output variables (Briesch and Rajagopal, 2010).
This Figure can be summarized on the equation form as follows: Most of the time, the ANN procedure involves the process of training, validation, and testing. The validation approach involves randomly dividing the available set of observations into two parts, a training set, and a validation set or hold-out set for testing the model. The model is fit on the training set, and the fitted model is used to predict the responses for the observations in the validation set. The resulting validation set error rate (using Mean Square Error in the case of a quantitative response) provides an estimate of the test error rate. The lower the error test, the best of the fitted model. In fact, the "training" data set is the general term for the samples used to create the model, while the "test" or "validation" data set is used to qualify performance.

Results and discussion
Regarding the Lebanese GDP over the 1950-2019 period, we split the data into a training set and testing set. The training set represents 70% of the data and the testing set, 30%. We estimate the relationship (14) with the hyperbolic tangent function as the activation function described in (15).
By applying the ANN method for predicting the Lebanese GDP growth rate, we obtain the following relationships described in Figure 4.

Figure 4: The neural networks: Results regarding the Lebanese GDP
Steps show the number of iterations (called also "epoch") needed to adjust initial weights (assigned randomly) in the network to reduce the error (representing the Sum of Squared Error) which is relatively weak.
By writing Figure 4 in the equation form, we obtain: We notice that an interactive effect does exist since both hidden neurons take three inputs.
Moreover, the weights regarding synapsis linking the input neurons to hidden neurons are much larger for the first hidden neuron (h1) than for the second (h2). This means that the first hidden neuron plays the most important role regarding the accuracy of the prediction of the Lebanese Yt neuron on the GDP Yt is : exp(-0.27) = 0.76 against exp(-1.10) = 0.33 regarding the h2 neuron.
These results from the ANN model can be compared with other procedures such as the multiple regression model where the dependent variable Yt is regressed on the lagged variables displayed in Figure 4. It's like to estimate an autoregressive model with three lags (AR (3)). By using the generalized least square method, we obtain the following results:    Indeed, the R-square between the embedded observed and predicted series is around 0.83. Figure 2, the center of the Figures represents relative stability in the GDP evolution.

As in
Moreover, the further the points are from the center, the chaotic the GDP is.

Conclusion
The Lebanese GDP analyzed in the phase space presents a strange attractor generated by a chaotic dynamic. Indeed, the Brock, Dechert, Scheinkman, and Lebaron (BDS) test indicates that, over the 1950-2019 period, the Lebanese GDP follows a nonlinear dynamic that paves the way to the study of chaotic behavior. Such a behavior can be apprehended by estimating the Lyapunov exponents that give, regarding the Lebanese GDP, the optimal dimension and delay of the time series reconstructed in the phase space. So, to analyze the chaotic behavior of the Lebanese GDP in the space phase, the embedding procedure is used, and the series is reconstructed with four dimensions and one delay. Thus, for predicting the motion of this embedded series, the ANN model gives the best results compared with the multiple regression model because the MSE, measuring the quality of fit, exhibits lower values than that resulting from the linear model. However, it is true that other nonlinear models like the neural network autoregressive model (NNAR model) can be efficient as well for predicting and forecasting time series but its use, for analyzing with accuracy the chaotic dynamics of a time series for which an embedding procedure has been realized, doesn't seem well adapted.

Availability of data and materials
Not applicable