In the present world, there are many songs over the internet. But the information retrieval on these songs can be complicated. This paper intends to classify songs based on emotions using deep learning. We propose a strategy to recognize the emotion present in a song by classifying their spectrograms, which contains both time and frequency information. According to human psychology, neurons within a sub pop- ulation of our brain did not react the same way for all the emotions.So only specific neurons need to be triggered for identifying an emotion. Dif- ferent deep learning and machine learning algorithms are implemented to build music emotion recognizer. The main objective of this study is to study about the features which are important for audio file ,to de- velop a music emotion classifier using deep learning algorithm and also to validate the model.The datasets are split into training and testing sets, models are trained with training data set. The accuracy of Artifi- cial Neural Network (ANN) model is 79.7% ,K-Nearest Neighbor (KNN) model is 78.26% and logistic regression for gender classification is 81%.