Weld Defect Detection and Image Defect Recognition using Deep Learning Technology

—Welding defects not only bring several economic losses to enterprises and individuals but also threatens peoples lives. We propose a deep learning model, where the data-trained deep learning algorithm is employed to detect the weld defects, and the Convolutional Neural Networks (CNNs) are utilized to recognize the image features. The Transfer Learning (TL) is adopted to reduce the training time via simple adjustments and hyperparameter regulations. The designed deep learning-based model is compared with other classic models to prove its effective- ness in weld defect detection and image recognition further. The results show this model can accurately identify weld defects and eliminates the complexity of manually extracting features, reach-ing a recognition accuracy of 92.54%. Hence, the reliability and automation of detection and recognition is improved signiﬁcantly. Actual application also veriﬁes the effectiveness of TL in weld defect detection and image defect recognition. Therefore, our research results can provide theoretical and practical references for efﬁcient automatic detection of steel plates, cost reduction, and the high-quality development of iron and steel enterprises.


I. INTRODUCTION
A S the economy and society develop speedily, China has entered industrial modernization, and its Gross Domestic Product (GDP) has been continuously increasing [1]. The steel industry is vital in industrial modernization. Steel exists in every corner of society, and the advancement of steel technology can indicate the level of industrialization of a country [2]. However, the recent development of steel technology in China is slow because most steel enterprises are at the stage of exchanging output for development; hence, the investment in technological innovation is little, which significantly hinders Chinas steel industry [3]. Chinas iron and steel products are gradually being eliminated due to the continuous optimization of iron and steel technologies in developed countries, causing losses of the iron and steel industry. Given the current development model, emphasizing ecological and environmental protection, the production capacity of the iron and steel industry is surplus, the demand declines, and the competition within the industry becomes severe; therefore, high-quality growth of steel enterprises has become the only way to develop the iron and steel industry [4]. Steel plate, the primary steel material, is widely utilized in various industries, such as aviation, shipbuilding, chemicals, and automobile [5]. The quality of the steel plate determines the quality of downstream products. However, while producing the steel plates, impurities, scratches, and cracks often appear on the surface of the steel plates due to raw materials, production equipment, and production processes [6]. Usually, welding technology is utilized for connection and repair. However, if the welding is under qualifying, welding cracks will occur, leading to severe accidents, such as steel plate rupture and boiler explosion, which not only brings economic losses to enterprises and individuals but also threatens peoples lives [7]. Therefore, the defect detection for steel welds is particularly important.
At present, steel welds are usually detected by nondestructive methods, in which X-ray is the common most method [8]. X-ray weld detection is divided into radiographic film detection and digital detection according to the images. The former is highly subjective and prone to misjudgments [9], while the latter utilizes computer algorithms for image recognition and detection [10]. The weld defect detection methods have been researched extensively. Hou et al. (2019) developed a model based on Deep Convolutional Network (DCN) to extract deep features from X-ray images directly; the classification capabilities of traditional methods and this model were compared using different datasets; the model had an accuracy of 97.2%, which was much higher than traditional feature extraction methods [11]. Shevchik et al. (2020) proposed a method that could detect defects in process instability in real-time based on a deep Artificial Neural Network (ANN); finally, the quality classification confidence was between 71% and 99%, revealing excellent application values [12]. Ajmi et al. (2020) provided a comparative evaluation method of deep learning network performance for different combinations of parameters and hyperparameters and added an enhanced learning method to the dataset, which increased the model accuracy by approximately 3% [13]. Ajmi et al. (2020) also applied Machine Learning (ML) and image processing tools to traditional crack detection and proposed a novel classification method based on deep learning network using data enhancement for random image transformation on the data; it turned out that the model had the best performance in a short time [14]. Hence, current studies mostly focus on using deep learning for weld defect detection.However, a stable and efficient automatic detection system is never established. Most steel enterprises still adopt traditional manual sampling methods, which have many subjective factors and a low detection efficiency, causing quality problems of steel plates.
Therefore, the previous studies are summarized, and the current problems in steel plate production are analyzing using the production line status of a steel mills workshop as an example. The five major weld defects are explored, including inclusions, scratches, scars, roll marks, and bubbles. The deep learning algorithm recognizes and detects weld defects; the images of weld defects are processed, whose features are extracted via Convolutional Neural Networks (CNNs). Then, Transfer Learning (TL) is adopted to shorten the training time via simple adjustments and hyperparameter regulations. The results can lay a foundation for efficient automatic detection of steel plates, eliminating technology constraints, reducing the operation costs, and high-quality development of iron and steel enterprises.

A. Analysis of steel plates surface defects
The hot-rolled steel plates will have some surface defects due to the production process and the steel billets; such defects are divided into steel defects and process defects according to the causes [15]. The structure of surface defects is shown in Figure 1. Scars are metal flakes with irregular shapes that attach to the surface of the steel strip.This defect can cause problems such as metal peeling or holes during subsequent processing and utilization.Bubbles are irregularly distributed round or elliptical convex hull defects on the surface of the steel strip, which can cause problems such as delamination or low welding during subsequent processing and utilization [16]. Inclusions are lumpy or elongated inclusive defects in the slab exposed on the surface of the steel strip after the inclusions or slag inclusions are rolled. Such defects will cause holes, cracks, and delamination during subsequent processing. Iron oxide scale is a kind of surface defect formed by pressing an iron oxide scale into the surface of the steel strip during the hot rolling process.This defect will affect the surface quality and coating effect of the steel strip. Roll marks are irregularly distributed convex and concave defects on the surface of the steel strip, which can cause folding defects in the rolling process. Edge cracking is a phenomenon in which one or both sides of the steel strip edges are cracked along the length direction, which may cause problems such as interruption of the strip during the subsequent processing and utilization [17]. Scratches are linear mechanical damages on the surface of the steel strip lower than the rolled surface. The scratched iron sheet is difficult to eliminate by pickling after oxidation, which is easy to cause breakage or cracking.Scrapes are mechanical damages on the surface of the steel strip in the form of points, strips, or blocks. The iron oxide scale at the scrapes is challenging to remove by pickling. Problems such as bending and cracking may be caused [18].  manual detection methods and non-destructive detection methods. The manual inspection is based on the visual inspection and manual experience, which requires on-site observations in harsh environments, causing considerable damages to the health of the staff member; besides, solely relying on workers experience often causes problems, such as missed inspections, making it difficult to guarantee the quality of the steel plates [19]. Traditional non-destructive detection is divided into eddy current detection, infrared detection, and magnetic leakage detection. Eddy current detection is suitable for detecting defects on the surface and lower layer of the steel plate, which requires a more extensive current guarantee. Hence, it consumes much energy, and the surface of the steel plate must be at a constant temperature, making it unsuitable for industry requirements [20]. Infrared detection adds induction coils in the production process of industrial steel plates. If the steel billets pass by, the induced current will be generated on the surface; if a defect is found, the current will increase, which is an excellent way to detect the defects. However, infrared detection can only be utilized in products with lower detection standards, and fewer types of defects can be detected [21]. Magnetic leakage detection is based on a proportional relationship between the volume of steel defects and the magnetic flux density. After calculating the density of magnetic leakage, the defect location and area of?the steel can be calculated; however, this detection method is disadvantageous for surface detection [22]. As science and technology advances, a machine vision detection technology is proposed, which uses lasers and charge-coupled components to detect the surface of steel plates after digitization effectively.
2) Deep learning detection technology: As deep learning technology advances continuously, it gradually presents apparent advantages in image recognition and classification. CNNs can extract image features. The detection ability of neural networks is improved by continuously increasing the  [23]. Such an improvement can effectively avoid the subjectivity and inefficiency in the manual extraction process. Research on recognizing the weld defect images is various; especially, deep learning technology processes and organizes a single feature to quantify the abstract features according to the extraction principle, thereby using these features to classify and recognize images [24]. Figure 2 shows the difference between traditional machine learning and deep learning processes. Deep learning can obtain high-dimensional image features from the input data, fit the features, and finally, utilize the learning method of weights to increase the accuracy of classification prediction. However, while analyzing welds of industrial steel plates, deep learning cannot learn due to the lack of complete datasets; in addition, current research mostly focuses on improving the weld recognition ability; however, a complete automatic detection system is never built, increasing the difficulty in actual industrial applications [25].
C. Image preprocessing 1) Image denoising: Image processing is the basis for effective weld detection. Here, the image preprocessing aims to ensure that the image quality meets the requirements of deep learning. The weld images, provided by some enterprises, are observed, revealing a problem that currently, some images often appear dispersive white or black noise particles during digitization; meanwhile, during the radiography process, the exposure intensity will also cause problems, such as image contrast and grayscale degradation [26]. Therefore, image denoising is necessary. The typical image denoising methods include mean filtering, median filtering, Gaussian filtering, bilateral filtering, and wavelet filtering, among which median filtering can interfere with uniform pulses effectively, enabling it to maintain edge information after processing effectually. However, such processing will cause the gray level to decrease. The Gaussian filtering can process the details very well, but the images must conform to the Gaussian function distribution. The bilateral filtering can retain the edge information; nevertheless, the processing of other noises is inexplicit. The wavelet filtering has a good time-domain performance but low processing effect on the frequency band [27]. Randomly, an image during processing is chosen and processed by these above five denoising methods to find the optimal processing method. Peak Signal-to-Noise Ratio (PSNR) and Mean Squared Error (MSE) are employed for evaluation. Specifically, the equations are as follows: In (1) and (2),f (i, j)is the gray pixel value of the image after denoising,f 0 (i, j) is the gray pixel value of the input image, and MN is the image pixel.
2) Image enhancement: In welds, unreasonably adjusting the window width will cause image contrast reduction. Especially, edges of the defects are usually difficult to recognize, affecting the subsequent processing of the images. Therefore, image enhancement technology is employed to improve the contrast of effectually [28]. First, images undergo grayscale processing; a specific histogram is obtained after statistical analysis. Then, the processed image is stretched according to its size to make its average gray value as the segmentation standard so that the average grayscale will increase after processing. Finally, a dual-peak gray distribution image is obtained. The Sin function is used for nonlinear transformation and image stretching, which is shown in (3): In (3),f (i, j)is the gray value after transformation,f 0 (i, j)is the gray value before the transformation, a is the lowest gray value before the transformation, and b is the highest gray value before the transformation.

D. Deep learning neural networks 1) Convolution Neural Networks:
CNN is a useful supervised deep learning model. It is a feed-forward neural network whose artificial neurons can respond to some surrounding units within the coverage [29]. CNNs are widely applied to image recognition, including AlexNet, Visual Geometry Group Nets (VGGNets), and other models that reduce the recognition error rate of CNNs on a typical ImageNet dataset. Generally, deep learning models have at least three hidden layers. As the number of hidden layers increases, the models parameters will also increase, thereby increasing the complexity of the model, providing the possibility to complete more complicated tasks. If a shallow model in which each layer of the network is fully connected is adopted for image classification and recognition, this model will contain many parameters. In the case of multiple hidden layers, the parameters contained in the model will exhibit explosive growth, causing adverse impacts on space occupation, iterative calculation, and convergence speed   [30]. Figure 3 shows the structure of CNN.
In CNN, first, the cross-entropy loss function for the classification error of the i-th sample(x i , y i )is defined as: The output of a single sample(x i , y i )after passing the network isf (x),and the corresponding sample loss value is: The backpropagation rule of CNN updates the weight of each neuron, making the overall error function of the model continuously decrease. The convolution process is defined as follows: In (6),l is the number of convolutional layers in the model,k l ij is the number of convolution kernels,b l j is the additive bias,f is the activation function, and M j is the input image. The convolutional collection layer is defined as: In (7),down(·)represents the data collection function,β l j and b l j represent the product bias and additive bias, respectively,f is the activation function.
2) Transfer Learning: TL can meet the end-to-end needs in practical applications, with more expressive features. It is a deep learning method that uses existing knowledge to solve the problems in different but related domains. TL has become another popular research direction in deep learning [31]. Compared with traditional machine learning methods, TL directly improves the learning effect on different tasks, focuses on applying good source domain task knowledge to different but related target problems, and enables computers to learn by analogy without relying on big data for initial learning in every field. The training of traditional machine learning models requires labeled data from various fields, while data in different fields do not have TL performance on the same model. TL can utilize existing knowledge to learn new knowledge. Figure 4 shows the comparison of the two learning models. TL can organically utilize the knowledge in the source domain to better model the target domain under the condition of changes in data distribution, feature dimensions, and model output conditions [32].

E. Deep learning-based image defect-recognition model
For the recognition of welding images, the neuron structure of multiple hidden layers in the Deep Neural Networks (DNNs) can be adopted to analyze, decode, aggregate, abstract, and improve the input features according to the relevance of the iterative update of the training experience, thereby reducing the weight of the variable parts and increasing that of the constant parts in the data distribution. In different recognition tasks, some of the underlying features (such as color, texture, edges, and corners) in the data have common visual patterns. Therefore, a deep learning-based image defectrecognition model is built. The specific structure is shown in Figure 5. The first step is to pre-train the model via large challenging image datasets in relevant fields. The publicly trained model parameters can be utilized directly to save time cost and computing resources. Then, the model is transferred. Parameters of the convolutional layer and the pooling layer are retained as the frozen layer based on the model pre-training. The fully connected layer of the model and the size of the input image are changed to meet the model input requirements and defect-recognition types. The data are retained; finally, the classification and recognition tasks are completed. The last step is fine-tuning the model. After fine-tuning, the model can fit excellently and extract image features during training. The model is initialized while the target domain is trained. The Backpropagation Neural Network (BPNN) and Stochastic Gradient Descent (SGD) algorithm are employed to fine-tune the model and correct the parameters. Afterward, a model with excellent generalization ability is obtained until the training is completed or the condition of terminating model training is reached. Finally, the Softmax classifier is used to classify and output the recognition results.

III. RESULTS AND DISCUSSION
A. Experimental data and performance evaluation 1) Experimental environment and data: This experiment is based on Linux Ubuntu 16.04 operating system, Inter(R) Fig. 6: Some images in the ImageNet database Core(TM) I5-2400 Central Processing Unit (CPU) @3.10GHz, using python language to implement under the TensorFlow framework. The number of iterations is set to 250 according to the model accuracy and training time. The SGD algorithm is adopted. The momentum parameter is 0.9, the Batch-Size is 20, and the learning rate is 1e-4. Table I shows the configuration of the experimental environment. The VGG16 model is employed for TL. The pre-trained model comes from the ImageNet dataset, the worlds largest database for image recognition, containing 15 million images and covering the images of all objects in life [33]. Some of the images in the ImageNet database are chosen, consisting of 1000 different types, such as scratches and defects on the surface of various animals, plants, buildings, and objects. Figure 6 illustrates the detailed information of chosen images. The ratio of the training dataset to the test dataset is 8:2.
2) Model performance evaluation: Accuracy(ACC) is the comparison indicator of model performance evaluation, representing the proportion of processed samples correctly classified as positive samples [34].The calculation of ACC is (8) Figure 7 shows the image effects and the quantization results processed by the filtering methods. The median filtering can present a clear edge area of the steel plate defect, while other denoising algorithms cannot. Furthermore, in all the images, especially in the fifth image, the highest PSNR reaches 50.31dB, indicating that the effect of median filtering is better than other methods. Figure 8 demonstrates the weld image and grayscale histogram before and after the Sin function transformation. The gray value of the image after the Sin function transformation increases significantly, and the contrast image is enhanced considerably, showing the effectiveness of the image enhancement technology applied. Figure 9 presents the comparison results of different training models performances. As the amount of data increases, the defect detection performance of the model is continuously improving, and the training set and the test set show the same trend, indicating the correctness of this training process.Among different algorithm models, the proposed CNN model presents the best performance, whose average ACC is above 92%, followed by the RNN model because of its multiple input processes that can reduce the model loads.The above results show that the weld defect detection and recognition model based on deep learning technology has excellent performance. The highest defect detection ACC reaches 98.75%, which effectively improves the automation degree of industrial weld detection and recognition.

IV. CONCLUSIONS
A deep learning-based weld defect detection and image defect recognition system is established based on summarizing previous research. This system aims to recognize five major weld defects: inclusions, scratches, scars, roll marks, and bubbles. The fitting problem during weld defect detection is solved by image segmentation using image denoising and enhancement. The problem of insufficient training data in DNNs is solved by applying an improved activation function adaptive pooling method to the model. The recognition problem of DNNs on the image dataset of weld defect detection is solved by TL, which reduces the model training time through simple adjustment and hyperparameter regulations. The established system can effectively overcome the shortcomings of traditional methods and has strong robustness and high recognition accuracy in detecting weld defects from images. The research results can provide a theoretical basis for the automatic detection and recognition of weld defects in steel processing enterprises. CNNs are utilized to break through the bottleneck of manual extraction of defect features, effectively improve the automation of defect recognition, and achieve a high recognition accuracy. Nevertheless, several shortcomings are found. First, since there are no high-precision industrial sample datasets, if the images of industrial weld defect samples can be utilized to build a high-quality database, the accuracy of the model can be improved effectively. Second, given the increasing number of industrial datasets, building more complex neural network models is necessary. This requires using the Graphic Processing Unit (GPU) distributed optimization algorithms to improve the efficiency of algorithm operation. Finally, although the content of TL is introduced, it is an addition to CNNs. Therefore, the problem of sample appearance caused by TL needs to be solved. In the future, these aspects will be researched and analyzed more deeply, in an effort to help steel enterprises master the automatic detection technology of weld defects, get rid of technology monopoly, and truly realize the high-quality development.
V. REFERENCES