For the out-of-sample forecast, generally forecasting accuracy increases with moneyness and time to maturity. From one-week to maturity to one-year to maturity the average RMSE decreases from 0,7357 to 0,2417 for the GARCH model, 0,7303 to 0,2521 for LSTM and 0,8224 to 0,2473 for Random Forest. A decline of respectively 67,16%, 65,47% and 69,92%. The daily change in the implied volatility, computed as the absolute value of the average daily change for each maturity, declines by 76,21% when the maturity increases from one week to one year. The reduction in RMSE is therefore declining with longer maturities as expected beforehand. The daily change in the implied volatility is also lower for options ATM and options close to ATM than OTM options. It is also lower for call options than put options with the same option delta (negative risk-reversal).
5.1 ATM and OTM Options Summary
The results for an ATM put and an OTM put and call for each of the five specific times to maturity are presented in Table 5.1. The first column indicates the options level of moneyness and time to maturity, the three following columns the forecasting accuracy of the AR(1)-GARCH(1,1) model, the three mid columns the results for the LSTM model, and the three rightmost columns the results for the Random Forest model.Table 5.1. Forecast performance for OTM put/call options and ATM put options
Table 5.1
First column indicates the level of moneyness measured in delta for the different maturities. Delta 50 is the ATM option, and delta 5 is the OTM put and call option. The highlighted value indicates the best fitted value for that particular option for MSE, RMSE and MAE, respectively.
|
AR(1)-GARCH(1,1)
|
LSTM
|
RANDOM FOREST
|
|
MSE
|
RMSE
|
MAE
|
MSE
|
RMSE
|
MAE
|
MSE
|
RMSE
|
MAE
|
1 week put 5
|
0,6262
|
0,7913
|
0,5056
|
0,6362
|
0,7976
|
0,4974
|
0,8277
|
0,9098
|
0,5973
|
1 week put 50
|
0,4895
|
0,6997
|
0,4632
|
0,4763
|
0,6901
|
0,4475
|
0,6944
|
0,8333
|
0,5492
|
1 week call 5
|
0,5947
|
0,7712
|
0,4813
|
0,5917
|
0,7692
|
0,2833
|
0,7820
|
0,8843
|
0,5583
|
1 month put 5
|
0,2561
|
0,5061
|
0,2729
|
0,2645
|
0,5143
|
0,2767
|
0,2670
|
0,5167
|
0,3055
|
1 month put 50
|
0,1500
|
0,3873
|
0,2709
|
0,1537
|
0,3920
|
0,2308
|
0,1842
|
0,4292
|
0,2640
|
1 month call 5
|
0,2519
|
0,5019
|
0,2518
|
0,2519
|
0,5019
|
0,2513
|
0,2806
|
0,5297
|
0,2782
|
3 months put 5
|
0,1706
|
0,4131
|
0,2002
|
0,1792
|
0,4233
|
0,2056
|
0,1777
|
0,4215
|
0,2346
|
3 months put 50
|
0,0773
|
0,2781
|
0,1540
|
0,0850
|
0,2915
|
0,1610
|
0,0888
|
0,2980
|
0,1757
|
3 months call 5
|
0,1387
|
0,3725
|
0,1772
|
0,1413
|
0,3759
|
0,2278
|
0,1642
|
0,4052
|
0,1999
|
6 months put 5
|
0,1324
|
0,3639
|
0,1641
|
0,1392
|
0,3731
|
0,1679
|
0,1443
|
0,3799
|
0,1963
|
6 months put 50
|
0,0506
|
0,2248
|
0,1191
|
0,0550
|
0,2345
|
0,1269
|
0,0634
|
0,2518
|
0,1445
|
6 months call 5
|
0,0975
|
0,3123
|
0,1433
|
0,1022
|
0,3197
|
0,1449
|
0,1137
|
0,3372
|
0,1649
|
1 year put 5
|
0,1135
|
0,3369
|
0,1421
|
0,1215
|
0,3486
|
0,1436
|
0,1399
|
0,3740
|
0,1714
|
1 year put 50
|
0,0367
|
0,1916
|
0,0972
|
0,0398
|
0,1995
|
0,0992
|
0,0404
|
0,2010
|
0,1124
|
1 year call 5
|
0,0770
|
0,2776
|
0,1195
|
0,0815
|
0,2855
|
0,1232
|
0,0841
|
0,2900
|
0,1359
|
The benchmark AR-GARCH model is performing superior for both put and call options compared to the machine learning methods for options with longer maturities. For all options with maturities of three months or more, the AR-GARCH model outperforms the machine learning models. The Random Forest model has the lowest forecast performance of the three models for all options across the different maturities. On shorter maturities, i.e., for options with one week and one month to maturity, the LSTM and AR-GARCH model are the best-fitted models. For an OTM put option with one week to maturity, the GARCH model is the best fitted with an RMSE of 0,7913 whilst the LSTM model has an RMSE of 0,7976. The Random Forest model performs significantly poorer with an RMSE of 0,9098, an increase of 14,97% compared to the GARCH model. It is somewhat surprising that the GARCH outperforms the LSTM model for this particular option, considering this is the most volatile of the fifty-five options. The LSTM performs better in terms of MAE, meaning it is not as robust to outliers in the test set as the GARCH model. When we perform a Diebold-Mariano (DM) test for this option we see that there are no significant differences between LSTM and GARCH or LSTM and RF. However, the AR-GARCH is significantly better than the RF. According to the DM test, AR-GARCH is significantly better than RF for one-week options. AR-GARCH is significantly better than LSTM for the ATM option, but not for OTM options when time to expiration is one week.
5.2 One Week to Maturity
For all other options with a maturity of one week, the LSTM model outperforms the benchmark models on RMSE and MAE. An exception is a put option with a delta of 35, where the benchmark GARCH model has an MAE 1,96% lower than the LSTM, and a put option with a delta of 10, for which both models have an RMSE of 0,7635, a forecast accuracy 7,85% better than the Random Forest model. On average, for all options with one week to maturity, the LSTM outperforms the GARCH model with 0,75% in RMSE and 2,77% measured by MAE, whilst LSTM is 11,07% and 14,99% lower than the Random Forest in terms of RMSE and MAE, respectively. These findings show that the benchmark AR-GARCH model is not significantly poorer than the LSTM model at shorter maturities, whilst the LSTM outperforms the Random Forest model considerably. For the one week to maturity there is no significant difference between LSTM and RF, according to the DM test.
5.3 One Month to Maturity
When time to maturity increases to one month, the results fluctuate more. For the OTM and ATM put and call options depicted in Table 5.1, the benchmark AR-GARCH model seems to deliver the best forecast accuracy of the three models measured by RMSE. The LSTM performs equally well for the OTM call option and beats the benchmark AR-GARCH at MAE for the OTM put option. When comparing RMSE for all levels of moneyness, the benchmark AR-GARCH model performs on average 0,78% better than the LSTM model. However, the LSTM is, on average, 1,56% more accurate measured by MAE. The benchmark AR-GARCH model captures the outliers, i.e., significant sudden changes in the volatility, better than the LSTM model. It is also interesting that the LSTM model outperforms both benchmark models in terms of RMSE and MAE for OTM call options, i.e., options with a delta of 25 and lower. Comparing the LSTM to the Random Forest model, the RMSE and MAE are 4,97% and 10,82% lower for the LSTM model. The LSTM model outperforms the Random Forest model more often for call options than for put options.
According to the DM test, LSTM is significantly better than the RF for OTM call option, but not significantly better for ATM and OTM put options. Comparing LSTM to AR-GARCH model, the same applies here. LSTM is significantly better for OTM call option, and there is no significant difference for ATM and OTM put options. When comparing AR-GARCH to RF, RF is significantly poorer for ATM and OTM put options, and there are no significant differences between OTM call options.
Figures 5.4, 5.5 and 5.6, plot the forecasted values for ATM one month to maturity put options for the LSTM, RF and AR-GARCH model. The plot shows that the RF model has problems with the extensive shocks in implied volatility, especially around March 2020, when the COVID-19 pandemic had its outbreak worldwide. The RF overestimates the peaks from COVID-19 shocks, whereas the LSTM model underestimates these shocks. All through the test period, which stretches from the end of September 2018 to august 2021, the AR-GARCH fits the rapid changes in implied volatility better than the machine learning models, especially around the extensive shocks, whereas the implied volatility rises significantly.
5.4 Longer Maturities
For all options with a time to maturity of three months and longer, the simple benchmark AR(1)-GARCH(1,1) model proved superior to the more complicated machine learning models across all moneyness levels. Due to non-stationarity at a 5% significance level, the first difference is applied for all options when the maturity surpasses three months. The benchmark AR-GARCH model is better than the LSTM model with increasing maturity. At the same time, the Random Forest comes closer to the LSTM with increasing maturity, measured in average RMSE. However, LSTM still outperforms the Random Forest for all moneyness levels except for a three-month put option with a delta of five and a call option with a delta value of 35. On average, the difference in RMSE between the Random Forest and LSTM declines from 11,07% for one-week options to 2,80% for a one-year option. When comparing the MAE for the LSTM and Random Forest model, the differences are more significant, varying from the lowest for the three-month option at 8,24% to 14,99% for the one-week option. Interestingly, the MSE increases between the two models as the time to maturity increases with respectively 10,08% at six months and 11,25% when the time to maturity increases to one year, while the difference in RMSE decreases. When conducting the DM test on the longer maturities, the results indicate no statistically significant difference between the forecasts. This result is expected as the day-to-day changes in implied volatility decrease as maturity increases.
5.5 Other Findings
The distribution for the changes in implied volatility has high peaks and fat tails. As mentioned in section 3, the distribution changes with time to maturity. When regressing in-sample, we assume that the residuals follow a normal distribution. This precondition is applied for the in-sample regressions for the benchmark AR(1)-GARCH(1,1) model. Performing the same regressions assuming Student t-distribution, the average RMSE declined by 1,36% for one-week to maturity options, making the Student t-distributed model the superior model for forecasting compared to the benchmark AR-GARCH model. For options with one month to maturity, the Student t-distributed model performs 0,24% better than the benchmark AR-GARCH model. The t-distribution fits better for data with mean clustering and fat tails than the normal distribution. Our findings support that the t-distributed models fit the data better for shorter maturities. The benchmark AR(1)-GARCH(1,1) model with normal distribution is better than the t-distribution for the first-order integrated options with a maturity of three months and more. We note that the in-sample goodness of fit decreases with maturity for the Student t-distributed model compared to the normal distribution model used as the benchmark.