In today’s world suspicious or unusual activities express threat and danger to others. For the prevention from various security issues an automatic video detection system is very important. The study objective is to create an intelligent system that will take a video stream as input and detect what kind suspicious activity is happening in that particular video to reduce the time that consume on watching video. It is difficult to consecutively monitor cameras videos that recorded in public places for the detection any abnormal event so an automatic video detection system is needed for that purpose. For that purpose, deep learning-based model is the best approach. In this work we use three models Convolutional neural network (CNN) model GRU model and ConvLSTM model. These models are trained on the same dataset of 6 suspicious activities of humans that are (Running, Punching, Falling, Snatching, Kicking and Shooting). The dataset consist of various video related to each activity. Different deep learning techniques are applied in the proposed work that are preprocessing, data annotation model training and classification. The frames are extracted from the source video and then features are calculated through model known as Inception v3 which is a variant of Convolutional Neural Network. On the same dataset the CNN model attains 91.55% accuracy the ConvLSTM model attain 88.73% accuracy and the GRU model attain 84.01% accuracy. The performance of proposed models are evaluated using confusion matrix, f1-score, precision, and recall. The proposed model proved better than other models in terms of performance and accuracy. The findings of this study prove helpful unusual event by examining the abnormal behaviour of person.