Intelligent Detection and Assessment of Damaged Buildings using UAV Imagery and YOLOv8

doi:10.21203/rs.3.rs-3143110/v1

Download PDF

Research Article

Intelligent Detection and Assessment of Damaged Buildings using UAV Imagery and YOLOv8

https://doi.org/10.21203/rs.3.rs-3143110/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

To improve disaster response and recovery, there is an increasing need for precise and rapid assessment of seismic building damage in urban communities through large-scale optical sensing. Recently, unmanned aerial vehicles (UAVs) and advanced object detection algorithms have been increasingly adopted for rapid and accurate damage assessment. In this study, a method for post-disaster building location and damage assessment was developed using deep learning techniques. Specifically, we investigated the You Only Look Once (YOLOv8) object detection algorithm and its potential use for processing optical images obtained from UAVs. To achieve this, we consider a RescueNet image dataset that comprises 4494 images with a resolution of 3000 by 4000 pixels captured by UAVs in the aftermath of Hurricane Michael's landfall near Mexico Beach, Florida which we utilized 1603 images related to our project in this article. According to our experiments with different model configurations and data processing pipelines, object detection accuracy for damage assessment is 85% for buildings in different locations. The proposed method was qualitatively found to have a very good performance on densely distributed small buildings, demonstrating the potential of computer vision in UAV datasets for large-scale disaster management.

Building damage assessment

disaster management

unmanned aerial vehicles

UAV imagery

YOLOV8

object detection

Building damage assessment is essential for effective disaster management response, and recovery. Accurately assessing building damage after natural allows for the efficient allocation of resources and the prioritization of rescue operations. Traditional methods of damage assessment can be time-consuming and require human expertise, making them less effective in urgent situations. However, recent advances in unmanned aerial vehicles (UAVs) and object detection algorithms have opened up new possibilities for more efficient and accurate damage assessment. Deep learning algorithms and computer vision technology are becoming a critical component of prospective solutions to these challenges. Computer vision techniques, such as object detection algorithms, enable the rapid analysis of large amounts of visual data, making them a powerful tool for building damage assessment. By automating the analysis of UAV imagery, these techniques can significantly reduce the time and effort required for damage assessment, enabling a more effective response to natural disasters. Overall, image processing techniques have the potential to revolutionize building damage assessment, enabling more efficient and accurate disaster management and response. In this paper, we present a novel approach that utilizes UAV imagery and investigates the different versions of the YOLOv8 object detection algorithm for building damage assessment. YOLOv8 utilizes a Convolutional Neural Network (CNN) for object detection by leveraging its deep learning capabilities. The network is designed to process input images in a grid-like fashion, dividing them into smaller regions, and then applying convolutional layers to extract features at various scales and levels of abstraction. These extracted features are then used to predict bounding boxes and class probabilities for different objects present in the image. By using a single CNN architecture, YOLOv8 achieves real-time object detection with impressive accuracy, making it a powerful tool for tasks such as identifying damaged buildings in UAV imagery. We perform a thorough investigation of available datasets. We used RescueNet as the basis and developed an object detection dataset. Using the information gleaned from our labeling and classification efforts, we created a new dataset with the specific aim of facilitating object detection and building damage assessment. This dataset was carefully curated to ensure that it contained all the necessary information and features required for the tasks at hand. We made sure to include all relevant information that would assist in identifying and assessing damage to buildings, such as the location, extent, and severity of the damage. Then we utilized the YOLOv8 algorithm trained on a large annotated UAV image dataset from Rescuenet Our evaluation shows that YOLOv8x outperforms other versions of YOLOv8 and state-of-the-art methods for building damage assessment. Not only will our proposed method allow decision-makers to accurately locate the damage and prioritize rescue operations, but it will also provide a valuable labeled image dataset that can be used to inform future studies and facilitate the development of more accurate and effective object detection and damage assessment tools. This has the potential to dramatically improve the efficiency of disaster response activities, lowering the time required to repair devastated areas.

Cha et al.[1] developed a CNN-based concrete fracture detection algorithm. The study extensively compared the suggested method to Canny and Sobel edge detection techniques. The CNN-based model performed better in detection and generalization. Dorafshan et al. [2] compared DCNN and edge detection concrete fracture detecting techniques. Their study tested these two technologies for concrete fracture detection. DCNN-based models outperformed edge detection methods in their study. This study impacts concrete fracture detection. Recently, machine learning-based crack detection methods have gained popularity. Manually inspecting big concrete structures is costly, time-consuming, and error-prone. Kim et al. investigated SURF and DCNN fracture detection algorithms for concrete surfaces in their study [3]. CNN or SVM with local or global features improved detection accuracy significantly. CNN-based damage detection algorithms were found to be more accurate and robust than IPT-based approaches. Instead of constructing new CNN models, some researchers train certain parameters to increase performance. Kim et al. [4] trained a CNN to recognize cracks, joints, edges (multiple lines), joints and edges (single lines), intact surfaces, and plants using transfer learning on a fine-tuned AlexNet model. The scientists found that their strategy improved non-crack visualization due to crack-like features in non-crack portions of input photos. Atha and Jahanshahi's[5] study examined how image size and color space affect CNN-based model identification accuracy. The study also tested CNN models for steel structural corrosion detection. Their results showed that 128x128 pixels, RGB or YCbCr color space, the VGG16 CNN model, and transfer learning yield the best detection performance. In 2020, Chow[6] presented an AI-based method for identifying concrete deterioration in diverse circumstances. Three modules detected, extracted, and classified damage. The convolutional self-encoding module initially estimated picture damage distribution. Threshold analysis determined the damage region. Finally, transfer learning classified the damaged region. This hybrid technique may identify various types of concrete damage by eliminating light, shooting distance, and angle. Jang et al.[7] suggested combining visual and infrared thermal imaging pictures to improve item detection. A CNN model trained on mixed pictures was tested for detection reliability. Visual and infrared thermal-making images can provide complementing information that neither imaging modality can capture, improving object detection. Combining these two imaging modalities produces images with color and thermal information, which can help detect items that are hard to see with one modality. They tested this method by training a CNN model on mixed pictures and assessing its detection reliability. The CNN model trained on mixed images beat the visual-only model, suggesting that merging visual and infrared thermal imaging images can increase item detection reliability. Researchers have employed the IC (image classification-based) algorithm for local, global, and disaster loss assessment. Yeum et al.[8] suggested an IC-based approach to identify collapsed structures after disasters. This technique automatically classifies post-disaster photographs into collapsed buildings, damaged buildings, irrelevant images, and undamaged buildings. This strategy boosts picture summarizing efficiency in post-earthquake data collecting. Transfer learning (TL) technique helped Gao et al. [9] train their CNN-based global damage detection model. This method enhances training efficiency and reduces data-driven overfitting. Despite TL's benefits, hyperparameter adjustment during training is difficult. Thus, selecting hyperparameters during training is still important. Xiong and colleagues used the IC model with a UAV platform to detect worldwide damage in [10]. They determined the worldwide damage status of 66 multi-story buildings in the old Beichuan County. Researchers used the IC-based global damage detection algorithm to analyze a structure's macro damage state. Ghosh et al. developed a model to detect disaster-related building damage[11]. The study sought to develop a tool to swiftly and correctly assess damage and aid in healing. The researchers used Faster R-CNN, a deep learning model that classifies images using convolutional neural networks, to do this. The model learned to identify surface fractures, spalling, severe damage with exposed rebars, and badly buckled rebars from photos of damaged buildings. This type of damage frequently indicates more serious structural flaws that must be addressed to maintain building and occupant safety. Recovery operations can be prioritized and coordinated more efficiently by properly identifying damage kind and location, thereby saving lives and decreasing property damage.in [12] This work created a two-stage machine learning system for post-earthquake building location and damage assessment. Satellite remote sensing (SRS) optical pictures are densely scattered, tiny, and imbalanced, therefore the method accounts for them. A modified You Only Look Once (YOLOv4) object detection module and an SVM-based classification module are used. In the first stage, network architecture, training hyperparameters, and anchor boxes optimize the YOLOv4 model to extract and fuse multiscale characteristics from densely scattered building SRS images. This study employed 386 1024 × 1024 SRS optical photos of the 2017 Mexico City earthquake before and after. The technique achieves 95.7% location accuracy of post-earthquake buildings and 97.1% binary classification accuracy for damage assessment.

One of the most popular computer vision algorithms is the You Only Look Once (YOLO) object detection algorithm. YOLO is a deep neural network that predicts bounding boxes and class probabilities for each cell in a grid of input images. Bounding boxes, object classes, confidence scores, and class probabilities are the output. Self-driving cars, security systems, and drone surveillance use YOLO because of its speed and precision. To make YOLO more powerful and adaptable, several research hurdles remain. The algorithm has trouble detecting small items and can give false positives when objects overlap. The YOLO algorithm has had various versions with different features and enhancements. Anchor boxes helped YOLOv2 handle object scales and aspect ratios. YOLOv3, on the other hand, employed a feature pyramid network (FPN) to detect objects of varying sizes and included "swish activation," which enhanced network accuracy. YOLOv4, the latest algorithm, added a spatial pyramid pooling (SPP) module, a novel activation function, and CutMix, a new data augmentation technique. These enhancements make YOLOv4 one of the most accurate and efficient object identification algorithms. YOLO has improved with each edition, but more study is needed to improve it. Attention processes are being investigated for object detection. Attention methods can help the system focus on specific visual regions and better detect things. YOLO-based algorithms for 3D point clouds, utilized in autonomous driving and robotics, are also of interest.YOLOv8 provides improved architecture and developer experience. YOLOv8 detects objects in photos using CNNs. The YOLO CNN extracts edges and textures from the input image to detect objects. YOLO does object categorization and bounding box regression with one CNN. This makes object detection faster and more efficient than methods with numerous phases or task-specific models. YOLO trains its CNN on a large collection of annotated photos with object labels and bounding boxes. This lets the CNN detect objects of varied sizes, shapes, and orientations. YOLO's CNNs have revolutionized real-world object identification with rapid and precise detection. Figure 1 illustrates CNN-based object detection. First in conducting our research, we undertook a thorough investigation of the available images related to our study topic. We carefully analyzed and scrutinized each image, taking note of important features and patterns that could inform our understanding of the subject matter. After an exhaustive analysis, we were able to label all the available images and classify them into three distinct classes. The process of labeling and classification was a rigorous one that required a lot of attention to detail and careful consideration of various factors. We considered the different attributes of each image, including its size, color, and content, among other things, in order to accurately classify it into the appropriate category. Our method detects buildings with no damage, moderate damage, and catastrophic loss in post-disaster images.

3.1 Dataset

In this section, we showcase how we leveraged the Rescue-Net dataset, which is a high-resolution UAV semantic segmentation benchmark, to evaluate the extent of natural disaster damage. Our team employed Roboflow¹ for image annotations and sorted the images into three categories: no damage, moderate damage, and destruction, as shown in Fig. 2. Our dataset included 1370² raw images, which we enhanced with pre-processing and augmentations to enhance our method's accuracy. The outcomes demonstrate that our strategy is highly effective in precisely evaluating natural disaster damage. With Rescue-Net and Roboflow, we were able to process large image datasets more efficiently and accurately, indicating that this method holds great promise as a tool for future disaster response efforts.

3.2 Image Pre-Processing

To enhance the efficiency of the YOLOv8 method for this dataset, we altered the size of the initial images. Specifically, we resized them from their original dimensions of 3000 by 4000 pixels to 1280 by 1280 pixels. This change was implemented to minimize computational demands and maximize efficiency. Our experimentation has demonstrated that this resizing did not result in any notable decrease in the accuracy of the YOLOv8 method for the Rescue-Net dataset, and it also had the added benefit of reducing training duration and memory usage.

3.3 Image Augementation

Image augmentation is a widely used method in machine vision to enhance the accuracy of computer vision models. This technique involves creating new images from existing ones by applying a range of transformations, such as flipping, rotation, zooming, brightness adjustment, and contrast modification. This approach enables us to considerably enlarge the dataset and avoid overfitting, enhancing the model's generalization. In this work, we used various image augmentation methods to enhance the detection accuracy. We employed 90° rotations, clockwise and counterclockwise rotations, within the range of -15° and + 15°, horizontal and vertical shearing of up to 15°, and brightness adjustments within the range of -25% and + 25%. These techniques helped us create a final dataset of 6265 images that included variations of the original dataset, improving its diversity. Figure 3 displays some sample images after the augmentation process. Our experimental results showed that the accuracy of our machine vision model improved considerably

when trained on this augmented dataset, which demonstrates that image augmentation is a vital technique for improving the performance of computer vision models.

3.4 Models

We utilized the latest version of the YOLO algorithm, namely YOLOv8 for building damage assessment and object detection. YOLOv8 is a modified version of the original YOLOv4 algorithm, which uses a more efficient backbone architecture and introduces various modifications to improve object detection accuracy.

By using all the extensions of YOLOv8, we can achieve even better results in object detection tasks. We trained and evaluated YOLOv8 on this dataset using different variants of the algorithm, including YOLOv8s, YOLOv8m, YOLOv8n, YOLOv8l, and YOLOv8x.

The paper's experimental findings reveal that the YOLOv8l and YOLOv8x models achieve the best accuracy on large image datasets. YOLOv8l is optimal for high-performance computing systems as it has a larger model size and higher computational requirements. Conversely, YOLOv8x is tailored to deliver high accuracy with a more compact model size, making it more efficient for real-time object detection applications. The models' performance was assessed on a dataset, and the outcomes are displayed in Figures 7-10.

We trained YOLOv8x for 20 and 30 epochs. As anticipated, the increased number of epochs resulted in higher accuracy. The results are demonstrated in Figure 7, which illustrates the performance of our model in terms of accuracy. These figures demonstrate the improvement achieved through increased training and underscore the importance of thorough model training for achieving optimal results.

In Figure 8, the different versions of YOLOv8 are compared based on the number of epochs and their mean average precision (mAP) scores at two different intersections over union (IoU) thresholds, 50 and 95. The graph shows the performance of each version of YOLOv8 across different epochs, allowing us to observe how the models perform over time. The mAP score, which measures the accuracy of object detection, is an important metric to evaluate the performance of object detection models. The comparison reveals that there is a trade-off between the number of epochs and the mAP score, as increasing the number of epochs can lead to higher mAP scores but also increases the risk of overfitting. Overall, the comparison of YOLOv8 versions provides valuable insights into the performance of different models and can help researchers and practitioners choose the best model for their specific use case.

The suggested UAV-based automatic building damage detection approach using several machine vision algorithms has been tested and found to accurately detect and classify different forms of building damage. To improve damage detection and evaluation, the study emphasizes selecting the right algorithm for each type of damage. The results indicate that the proposed approach can help decision-makers and emergency responders estimate building damage quickly and accurately after natural disasters or other environmental conditions. This study used YOLOv8, a cutting-edge object detection system that uses a single neural network to predict bounding boxes and class probabilities from complete photos in real-time. On the same dataset as the, most versions of YOLOv8 performed well, averaging 85% precision. YOLOv8x and YOLOv8l had the highest accuracy, with 87% and 89%, respectively. This emphasizes the necessity of choosing the right algorithm for the job, as different versions perform differently depending on image complexity and type. This study suggests that YOLOv8 can detect building deterioration and aid disaster response and management. Moving forward we will explore different object detection and segmentation algorithms and design more accurate and efficient building damage detection methods. Specifically, we recommend the following research directions:

1. Hyperspectral and thermal imaging: The study suggests employing these methods to detect building damage. These strategies might be tested in numerous disaster circumstances and integrated with machine vision algorithms for more accurate and efficient damage detection.

2. Real-time damage assessment systems: This study's technique could be used to construct real-time damage assessment systems for disaster response and management. Future research could design and test such systems in real-world settings.

3. UAV photos can also locate catastrophe victims as well as building damage. UAVs' high-resolution photographs can help rescuers locate survivors and identify obstructions. Future research could build machine vision algorithms for victim detection and assessment utilizing UAV photos. These systems may evaluate photographs in real-time, identify casualties, and aid rescuers. Such technology could speed up catastrophe rescues and save more lives. To ensure accuracy and dependability in real-world circumstances, building and implementing such systems requires significant testing and validation.

Conflicts of interest :The authors have no conflicts of interest to declare that are relevant to the content of this article.

Competing Interest :The authors have no relevant financial or non-financial interests to disclose

AUTHOR’S CONTRIBUTION: All three authors had an equal role in writing the article and have cooperated in all stages

Availability of data and materials: Dataset is available on request

Cha YJ, Choi W, Büyüköztürk O (2017) Deep learning-based crack damage detection using convolutional neural networks. Computer‐Aided Civil and Infrastructure Engineering 32(5):361–378
Dorafshan S, Thomas RJ, Maguire M (2018) Comparison of deep convolutional neural networks and edge detectors for image-based crack detection in concrete. Constr Build Mater 186:1031–1045
Kim B, Cho S (2019) Image-based concrete crack assessment using mask and region‐based convolutional neural network. Struct Control Health Monit, 26(8), e2381
Q.Lin T, Ci L, Wang SK (2022) and Y.Wang. Transfer Learning for Improving Seismic Building Damage Assessment. Remote Sens, 14(1),
Atha DJ, Jahanshahi MR (2018) Evaluation of deep learning approaches based on convolutional neural networks for corrosion detection. Struct Health Monit 17(5):1110–1128
Chow JK, Su Z, Wu J, Li Z, Tan PS, Liu KF, …, Wang YH (2020) Artificial intelligence-empowered pipeline for image-based inspection of concrete structures. Autom Constr 120:103372
Jang K, Kim N, An YK (2019) Deep learning–based autonomous concrete crack evaluation through hybrid image scanning. Struct Health Monit 18(5–6):1722–1737
Yeum CM, Dyke SJ, Ramirez J (2018) Visual data classification in post-event building reconnaissance. Eng Struct 155:16–24
Gao Y, Mosalam KM (2018) Deep transfer learning for image-based structural damage recognition. Computer‐Aided Civil and Infrastructure Engineering 33(9):748–768
Xiong C, Li Q, Lu X (2020) Automated regional seismic damage assessment of buildings using an unmanned aerial vehicle and a convolutional neural network. Autom Constr 109:102994
Ghosh Mondal T, Jahanshahi MR, Wu RT, Wu ZY (2020) Deep learning-based multi‐class damage detection for autonomous post‐disaster reconnaissance. Struct Control Health Monit, 27(4), e2507
Wang Y, Cui L, Zhang C, Chen W, Xu Y, Zhang Q (2022) A two-stage seismic damage assessment method for small, dense, and imbalanced buildings in remote sensing images. Remote Sens 14(4):1012

https://roboflow.com/
If you use this dataset in a research paper, please cite it using the following BibTeX:
@misc{ rescuenet-yolo8_dataset, title = { RescueNet-YOLO8 Dataset },
type = { Open Source Dataset }, author = { KIOS }, howpublished = { \url{ https://universe.roboflow.com/kios-alpsc/rescuenet-yolo8 } },
url = { https://universe.roboflow.com/kios-alpsc/rescuenet-yolo8 },
journal = { Roboflow Universe }, publisher = { Roboflow },
year = { 2023 }, month = { apr }, note = { visited on 2023-04-07 },
}

No competing interests reported.

Download PDF

Version 1

posted

You are reading this latest preprint version

Intelligent Detection and Assessment of Damaged Buildings using UAV Imagery and YOLOv8

Status:

Version 1

Abstract

Figures

1 Introduction

2 Related Works

3 Proposed Method

3.1 Dataset

3.2 Image Pre-Processing

3.3 Image Augementation

3.4 Models

4 Results

5 Conclusion and future works

Declarations

References

Footnotes

Additional Declarations

Status:

Version 1