Machine and deep learning applications play a dominant role in the current scenario in the agriculture sector. To date, the classification of fruits using image features has attained the researcher’s attraction very much from the last few years. Fruit recognition and classification is an ill-posed problem due to the heterogeneous nature of fruits. In the proposed work, Convolution neural network (CNN), Recurrent Neural Network (RNN), and Long-short Term Memory (LSTM) deep learning methods are used to extract the optimal image features, and to select features after extraction, and finally, use extracted image features to classify the fruits. To evaluate the performance of the proposed approach, the Support vector machine (SVM) unsupervised learning method, Artificial neuro-fuzzy inference system (ANFIS), and Feed-forward neural network (FFNN) classification results are compared, and observed that the proposed fruit classification approach results are quite efficient and promising.