Regular inspection of distribution line is an important link to maintain the normal operation of distribution network. Using unmanned aerial vehicle (UAV) instead of manpower can save the cost of inspection. With the universal application of vision sensor in UAV and the rapid development of deep learning, Convolutional Neural Networks (CNN) is applied to the detection of power line in UAV visible images. In view of the lack of application environment inspection methods for distribution line, a vision-based UAV distribution line inspection method using deep learning and a dataset for the deep learning method of distribution line inspection task are proposed in this paper. The method proposed predicts distribution line area through the encoder-decoder structure network firstly. Image processing operation and sampling clustering are used to remove the interference. Finally, the UAV tracking direction of distribution power line is calculated according to the detected distribution line. The method can reach an inspection speed of nearly 77ms per frame, the range of heading deviation error can reach (-1.52°, 1.36°), the tracking rate nearly 100%. Through the test of network and inspection method on dataset, the results show that the method proposed in this paper can be effectively, quickly and accurately applied to UAV distribution line inspection.