Recently, there is growing interest in using machine learning and data analysis methods to study the stock market. The main objective is often predicting the direction of a stock trading.
The S&P 500 index follows the weighted performance of the 500 most valuable companies in the US. It is often used by economists to measure the health of the US stock market and the overall economy [1]. The massive adoption of this index has resulted in an increased interest by retail and institutional investment leading to exponential growth in trading of ETFs and options on the index. Modeling the performance of the index is increasingly valuable.
Multiple factors influence the price of a stock. Among these are the earnings season, wars, political news, and traders' psychology [2]. There are two main approaches to predict stock performance: fundamental analysis and technical analysis. Fundamental analysis focuses on how a company is performing and the success of the relevant industry and market. Information such as earnings per share, dividend yield, price-to-earnings ratio, and fair market value of the stock are important metrics in fundamental analysis [3].
In order to predict possible price movements, some traders use technical analysis to find trends or patterns. This discipline employs past data such as price movement and volume in order to identify trading opportunities [4]. Although past performance does not correlate highly with future returns, people’s emotions in the market do repeat themselves. Technical analysts create compound features using historical data. These technical indicators may find recurring patterns within the stock [5].
Plenty of machine learning and artificial intelligence papers propose models that could accomplish this feat. Some models have used machine learning, comparing the results against technical analysis indicators such as Bollinger bands. Key insights from this line of research are that there are no statistical differences between the two approaches. However, it is worth noticing that both outperform the index return by almost 1-fold [6]. Other models look at the relationship between social sentiment and the stock price to draw conclusions of future movements in price [7].
Brunhuemer et al. evaluated a machine learning model that focuses on trading straddles of buying put and call options at the same strike price. They focus on volatility and other technical analysis data to feed information to the machine learning model. However, they did not find any significant signs of outperforming their previous trading strategy [8]. Nevertheless, this shows that predicting volatility is as useful as predicting a specific market direction.
Machine learning techniques such as support vector regression, decision trees, k-nearest neighbor, random forest and multilayer perceptron have been used to predict the stock market. Techniques such as MLP and LSTM have showed promising results [9]
In this paper, a combination of technical analysis and machine learning methods has been used to create models that can improve the likelihood of accurately forecasting the absolute movement of the S&P 500 index. The predictions focus on large versus small changes in the index. An absolute change of less than 1% is coded as a small change (Group 0), while a change above 1% is coded as a large change (Group 1). The reasoning for this is that S&P 500 options have a high likelihood of yielding over 100% daily return when the index moves more than 1% in either direction.