The detection of events using the video data is getting popular due to the detailed information made available from the video data for multiple instances. This popularity is increasing the use of number of devices and amount of data generated from various sources, which makes the manual detection of abnormal events highly complex, and the recent research demands highly timely and highly accurate automation process. Thus, this work proposes a three-phase solution to address this problem as using a hybrid segmentation process for object detection with 97% accuracy, detection of the objects by using a pre-trained machine learning model with 98% accuracy and detection of the motion using predictive regression model with mean time of 58 ns. This proposed work has demonstrated benchmarking outcomes and showcases highly accurate detection process for making video-based surveillance safer and better choice.