In the stock market, accurate prediction of stock price movement direction can effectively increase the profits for investors. However, the stock price is an extremely complex dynamic system with strong fluctuation, proper selection of technical indicators can potentially improve the accuracy of the direction prediction. We propose a novel sparse least squares support vector machine (LSSVM) by combining recursive feature elimination (RFE) and Relief via a weight parameter. Specially, the benefit if this hybrid is three fold: (1) accounting for any intrinsic correlations among the features, (2) more effective prediction due to the sparse framework capable of removing some “noise” features completely; and (3) simultaneously select technical indicators according to the feature ranking and accounts for possible interactions and possible non-linear effects among the features. Three stock datasets from the liquor and spirits concept are analyzed to demonstrate the superiority of our proposed new framework providing sparse solutions resulting in more accurate predictions and higher returns among all seven considered classifiers.