Computer vision-based fire flame detection system has recently begun to replace traditional fire detection systems. The conventional imaging-based system uses color models and temporal analysis algorithms to detect and segment fire in an image or a frame. Color model algorithms could not effectively distinguish between fire and fire-like objects in real-time videos. Besides, temporal analysis between structures has not yielded higher detection accuracies. This work proposes a lightweight convolutional neural network (CNN) combined with Gaussian distribution threshold probability to detect and segment the fire in a frame sequence. The proposed hybrid work extracted the features and automatically classified the frames using simple network architecture, and the Gaussian probability threshold method improves the detection of fire region localization. Experimental performance of the proposed work increases the detection accuracy and better false alarm performance. This paper can predict the severity of the fire, area, and centroid, fire probability. The proposed work using low-resolution frames will surely help the real-time implementation system.