Vision-Based UAV Distribution Line Inspection Using Deep Learning

Regular inspection of distribution line is an important link to maintain the normal operation of distribution network. Using unmanned aerial vehicle (UAV) instead of manpower can save the cost of inspection. With the universal application of vision sensor in UAV and the rapid development of deep learning, Convolutional Neural Networks (CNN) is applied to the detection of power line in UAV visible images. In view of the lack of application environment inspection methods for distribution line, a vision-based UAV distribution line inspection method using deep learning and a dataset for the deep learning method of distribution line inspection task are proposed in this paper. The method proposed predicts distribution line area through the encoder-decoder structure network firstly. Image processing operation and sampling clustering are used to remove the interference. Finally, the UAV tracking direction of distribution power line is calculated according to the detected distribution line. The method can reach an inspection speed of nearly 77ms per frame, the range of heading deviation error can reach (-1.52°, 1.36°), the tracking rate nearly 100%. Through the test of network and inspection method on dataset, the results show that the method proposed in this paper can be effectively, quickly and accurately applied to UAV distribution line inspection.


Introduction
With the rapid development of power system scale, a large number of power line have been put into operation. Distribution network is the link between the power plants and the points of consumption, occupies a considerable proportion. Power failure will cause serious economic losses and bring unexpected consequences [1,2]. Therefore, the distribution line needs to be inspected and maintained to ensure its normal operation.
Regular visual inspection of distribution power line is one of the most important part to maintain the normal operation of distribution network [3]. The traditional methodologies for inspecting power lines include field surveys and airborne surveys. Manual inspection is inefficient and potentially dangerous [4]. Some scenes are inaccessible to humans, e.g. a dam on a river in a tricky, or power grid in thick forests [5]. An emerging option out of all the airborne surveys is to employ a UAV [6]. High resolution images can be obtained by a small UAV that could be used in line inspections and many other applications [7]. The cost of using UAV for inspection incurs approximately one third of that of using helicopter for inspection [8][9][10]. Also, it is possible to overcome the low efficiency and safety problems of manual inspection [11]. In addition, it can observe the transmission line more closely and obtain more detailed images of power components, thus greatly improving the inspection accuracy.
As a lot of research on vision technology have been made and the advantage of vision sensors, the realization of UAV power line inspection based on vision has become the research hotspots of power grid inspection [12]. The most basic but the most important problem in the vision-based UAV power proposed to detect line segments by morphological filtering an edge map image in the local criterion and group the line segments into whole power line in the global criterion. In [14] proposed a novel object-based Markov random field with anisotropic weighted penalty method to distinguish the power line segments. And an envelope-based piecewise fitting method to fit the power line. With the rapid development of deep learning and the disadvantages of versatility and stability of traditional image processing methods [15][16][17]. Deep learning methods are gradually being used to detect power line [2,18,19]. Hui al et. achieve autonomous navigation of drones based on transmission towers by using faster R-CNN [20] deep learning framework to detect tower and fully convolutional network (FCN) [21] to detect power line [22]. Saurav al et. use a nested U-Net segmentation architecture to automatic autonomous visual power line segmentation [23]. Currently, the distribution line detection by using deep learning lacks the comprehensive consideration of detection speed and accuracy in application.
And there is almost no public dataset for deep learning distribution line detection.
In this article, a vision-based UAV distribution line inspection method using deep learning is proposed. The main contributions are as follows: 1) A dataset for deep learning method of testing distribution line detection task is proposed. 2) A visual based inspection method for UAV distribution line is proposed. 3) Deep learning is used to segment the power line in UAV aerial images, and compared with the existing methods, the detection accuracy and speed are significantly improved.

Method
An overview of the method we adopted in the whole work is shown in the Fig. 1. The visible image of distribution line captured by UAV is input into the deep learning network to obtain the prediction results of the power line area in the image. Through the steps of image processing, sampling clustering and deviation calculation of the prediction results, the heading deviation and roll deviation for controlling the UAV line patrol are obtained.

Model Architecture
The advanced light-weight network model is used for semantic segmentation. The whole network adopts encoder-decoder structure. MobileNetV2 [24] and atrous spatial pyramid pooling (ASPP) [25] as an encoder module to capture contextual information, which is suitable for mobile devices. A simple and effective structure as decoder module to refine the segmentation results along object boundaries.
The architecture in the encoder module is MobileNetV2, which is specifically tailored for mobile and resource constrained environments. This network pushes the state of the art for mobile tailored computer vision models, by significantly decreasing the number of operations and memory needed while retaining the same accuracy. The basic idea of Depthwise Separable Convolutions is to replace a full convolutional layer with a depthwise convolutional layer and a pointwise convolutional layer. The depthwise convolutional layer performs filtering by applying a single convolutional filter per input channel. The pointwise convolutional layer responsible for building new features through 1×1 convolutional kernel computes linear combinations of the input channels. Bottleneck Residual block is used as the basic building block in the network to achieve a balance between feature extraction and computing speed.  The structure of decoder module is similar to the decoder module which is proposed by DeepLabv3+ [25]. This simple yet effective decoder module can successfully recover object segmentation details.
The output features from encoder module are computed with output stride = 8. Firstly, the bilinear upsampling with a factor of 2 is performed on the output feature map of the encoder module, and then concatenate it with the low-level features from the encoder module that have the same spatial resolution. The deep information with low spatial resolution after multiple downsampling provides the context semantic information. The shallow information provides more precise segmentation information. The segmentation of target object will be more precise and accurate by concatenating the shallow information with the deep information. Different from the decoder module of DeepLabv3+, the number of channels for low-level features is not reduced but expanded. Because of the low-level features which is connected in the network only contains 24 channels, will not outweigh the importance of the output features from encoder module. Besides, it is not conducive to refine the segmentation results along object boundaries through the low-level features with too few channels. Then, the concatenated features are applied a few convolutions with a 3×3 convolutional kernel before the bilinear upsampling with a factor of 4 is performed. Finally, the features are mapped to the desired class by a 1×1 convolution layer. The output of the decoder module is also the output of the whole network, which shows the class of each pixel.

Image Processing
Simple but effective image processing operation is adopted for the prediction results from network to ensure the efficiency and improve the accuracy of results. The adjacent regions of the prediction results from network are connected by using morphological close operation. The morphological close operation is to dilate the image I firstly as: where [ , ] stand for the pixel coordinate of image, and [ , ] stand for the coordinate of kernel.
Then, erode the dilated image D as: The prediction regions of the same power line in the prediction results are connected, and the edge of the prediction area is smoothed without significantly change. Although the larger kernel of the closed operation, the more connected the whole image is, we hope that the prediction region belonging to the same power line in one connected region. It is not conducive to cluster the power line if connecting the prediction regions that are not belong to the same power line. Therefore, it is not proponent to select the kernel with a large size. In the experiment, the size of kernel is selected by 9×9.

Sampling Cluster
It will obtain the mask of the power line by performing semantic segmentation on the image. But there is no instance segmentation for power line, so the pixels belonging to power line in the prediction result have the same label. Density based spatial clustering of applications with noise (DBSCAN) [26] algorithm is used to cluster the set of power line pixel coordinates.

Deviation Calculation
According   To generate the ground-truth mask from the images, we use the Labelme image annotator which is publicly available free to coast. The annotation for each image come in a jason format. We manually annotate the distribution power lines along the edge of each distribution power line without distinction.
In many cases, the distribution power lines are blocked by tall trees. We annotate the ground-truth even through the middle of distribution power line is occluded partially. However, some distribution power lines, which are almost completely occluded or hard to identify, are not annotated. This way, the network will learn to predict distribution power line location even they are occluded partially. where TP refers to the number of present in the image and correctly detected. FP denotes the number of not present in the image but detected, and FN is the number of present in the image but not detected.
In order to evaluate the method proposed, we also provide the test video of the method in the dataset. with the distance between the heading deviation calculated in a frame and the marked deviation is less than 1/2 of the distance between the two farthest distribution lines. Therefore, the calculated power line trend is in the area of power line. The higher the ratio of the number of frames successfully tracked to the number of frames extracted from the video is, the better the method is.

Network Comparison
In order to verify the effectiveness of network proposed, two groups of experiments were carried out.
The training parameters are shown in the Table 1. The backbone network utilizes the MobileNetV2 pre-training model. All proposed experiments were performed in the PyTorch deep learning framework on GTX GeForce 1080Ti GPU, executed under Linux operating system. Table 1 The training parameters of network.

Parameters Value
Batch size 8 Epoch 200 Num classes 2 Weight decay rate 0.0005 Momentum rate 0.9 Initial learning rate 0.01 The comparison with the state-of-the-art network is shown in the Table 2. The size of all input are 3*600*480. The Intersection over Union (IoU) rate can be significantly improved by adding ASPP module to the backbone network. And it will not make the processing speed much slower. Compared with the model with ResNet [28] or Dilated Residual Networks (DRN) [29] as the backbone network, the processing speed is increased by 7-1000 times. Compared with the lane line state-of-the-art network Special CNN (SCNN) [30] and the nested U-Net [31] adopted in [23], the IoU rate and processing speed are significantly improved.   Table 3 The IoU rate and processing speed for different output stride and low-level features channels parameters of the network structure selected.

Method Test
We conduct experiments on test videos in two environments to verify the accuracy of the proposed vision-based UAV distribution line inspection method by using deep learning. In order to get the best output stride and low-level features channels parameter training, the network model trained with different output stride and low-level features channels parameters for each video is tested.  The results of the two test videos of the method are shown in

Conclusion
In this paper, a vision-based UAV distribution line inspection method using deep learning is proposed for UAV distribution network line inspection. Firstly, the deep learning network with encoder-decoder structure is used to quickly and accurately predict the distribution line area in the picture. Then, for the network prediction results, effective image processing operations and sampling clustering are used to remove the interference and determine the distribution line in the image. Finally, the UAV tracking direction of distribution network power line is calculated according to the detected distribution line. The dataset of deep learning method for testing distribution line detection task is proposed, and the test video and evaluation criteria are provided. In the experiment, the network comparison and method test results show that the method proposed in this paper is effective, fast and accurate for UAV distribution line inspection. The IOU rate of the network model in the proposed method on the dataset can reach 86.08%, and the processing speed can reach 11ms. The tracking rate of the proposed method on the test video can be guaranteed to be more than 99%.
In the future work, we will focus on the decision-making calculation of UAV patrol line under the condition of multi-line intersection and strengthen the application of the method in project practice. Availability of data and materials The datasets generated during the current study are available from the corresponding author on reasonable request.