Visual Sensing And Controlling of The Keyhole In Robotic Plasma Arc Welding

With an advantage of "single-sided welding and double-sided forming", plasma arc welding (PAW) has a great application potential in modern industrial production. The welding quality can be guaranteed by sensing and controlling of the keyhole. However, it is difficult to make an on-line observation on the back of base metal, and realize a dynamic registration of the visual sensor and welding torch. In this study, it has investigated the relationship between the welding condition and image feature of keyhole. Image processing is designed to obtain the feature image and conduct a template matching of the keyhole. The target feature of weld zone will be extracted and processed in real time. Besides, it has designed a digital controller for the welding robot and power source in this study, and discussed control method to stabilize the keyhole and achieve good welding quality. Eventually, experiments are conducted to inspect the comprehensive performance of the welding


Introduction
With the advantages of large energy density, good arc directionality, and strong penetration ability, the PAW can penetrate 8-10mm medium-thick stainless steel at one time even without groove and no filler wire. Wu have proposed a keyhole effect to achieve "single-sided welding, double-sided forming" [1]. Liu and Wu have studied the dynamic behavior of keyhole which can determine the weld quality to achieve strong welded metal joint [2,3]. Kuril and Trushnikov have shown that keyhole in the PAW depends on the welding condition, such as welding current, pilot gas and welding speed [4,5]. Li have simulated the influence of welding parameters in PAW. It is sensitive to the change of parameters and may become unstable due to the behavior of molten metal [6]. Prasad have pointed that the uncontrolled variation may lead to the keyhole closure or collapse of the weld pool [7]. As a result, it is difficult to maintain a stable keyhole by optimizing the process parameter window in PAW.
Because of high energy density and concentration of the plasma arc, welding position is just below the electrode hiding in the torch centre. Due to the compression effect and high ionization degree of the plasma arc, Li and Wu have verified that PAW is available to obtain a uniform weld bead with high welding quality. And It has broad application prospect in heavy machinery, storage and transportation tanks, pressure vessel pipelines, aerospace shipbuilding and so on [8,9]. However, the PAW system is relatively complex. The process parameters are numerous and coupled, which need the skilled technicians and high requirements for the control system. In addition, due to the large size and complex structure of the welding torch, Liu and Zhang have shown that the observability of the heat-affected area is poor [10,11].
In order to observe and control the keyhole behavior directly and accurately, some researchers have utilized visual sensor in their studies. Wu and Liu studied the dynamic variation of keyhole exit and its inclination extent. The extent was indicating the penetration and key-holing ability of plasma arc [12,13]. Medellín proposed a practical method based on a circular shape model of the weld crosssection area to evaluate the weld quality [14]. A synchronous visual sensing system was built by Jia with triple CCD cameras. Image registration was conducted and showed keyhole entrance and topside weld pool behaved closely related to the keyhole exit evolution process [15]. Yamane applied convolutional neural network (CNN) to identify a possible gap in the image of weld pool [16]. Zhang proposed a controlled-pulse strategy to improve the stability and dynamics of keyhole [17]. The relationship between process parameter and image features of keyhole has been investigated in these studies. However, there is still a lack of effective adaptive control method using closed-loop feedback approach to keep the keyhole stable and realize seam tracking in the meantime.
Authors have realized seam tracking by image processing of the weld pool in prior study. The study can be divided into three parts. First, it monitored weld zone by utilizing visual sensor before the welding torch. Through the boundary detection of gouging domain of keyhole, it realized the movement and height control of the welding torch in PAW [18]. In the subsequent study, author installed the visual sensor behind the welding torch. By detecting the triangle groove or a reserved gap in the welding direction, it successfully achieved the seam tracking [19]. Furthermore, author realized pattern matching and boundary detection of keyhole. It accomplished the adaptive control of weld pool behaviour [20].
In this study, fundamental experiment is conducted to investigate the relationship between the process parameters and feature image of the keyhole by utilizing high speed video camera. It is found that target feature from shield cap to the edge of keyhole entrance in feature image varies with the flowrate of plasma gas. According to this phenomenon, control method to stabilize the keyhole has been discussed. Author tries to design a digital controller to conduct an adaptive control of keyhole by regulating gas flowrate. This study will be of important significance in PAW to analyse the welding process, design of visual control system, and so on.

Visual PAW system
The visual PAW system built in this study is shown in the Fig.1(a). Main components consist of a six-axis industrial robot, a visual sensing system with CMOS camera, computers for welding control and image processing, a digital controller (dsPIC) for the power source characteristic, and the data acquisition device collecting process parameters. Moreover, the welding robot with its movement controller connects to the control PC through the Ethernet to realize the robot position setting and human-computer interaction. The welding conditions of the power source are set on the DSPIC, and data is transmitted through I/O serial communication. It prevents the electric noise of welding power from entering the control port by utilizing isolation amp.
(a) Robotic welding control system (b) Relationship between weld zone and visual sensing system In the visual sensing system shown in the Fig.1(b), there is a CMOS camera equipping with 950nm filter intercepting all the visible light, especially the arc light, down to near infrared region (NIR). It focuses on the optical characteristics of the heat-affected zone of the base metal. In order to observe the keyhole entrance, the camera and welding torch are installed on the same side of the base metal. They are fixed on the same jig and move together in the welding. Furthermore, the camera is installed at a vertical angle of 47° in this system to make a clearest observation through the keyhole during the welding process. The waveform of shutter is synchronized with the welding current, and the shutter opens for 3 ms at each current trough. It is controlled by the dsPIC to eliminate the effect of strong arc light on the weld pool image.
The resolution of CMOS camera is 1280×1024 pixels and the field of view (FOV) is 18×15 mm 2 for each frame of image. The pixel accuracy is 14.3 / with 1 mm corresponding to about 70 pixels in the taken image. The groove width is 12 mm narrower than the 18mm of horizontal FOV. In this case, the CMOS camera can concentrate on the keyhole entrance during the PAW. Moreover, in order to correct lens distortion and make an accurate measurement of the keyhole size, camera calibration is adopting before the welding experiment to obtain intrinsic matrix and extrinsic matrix.
Image processing software called Halcon13 containing over 2000 library operators has been utilized.

Basic experiment investigating the effect of welding conditions
In this study, the base metal is SS400 mild steel with V-type groove. Both plasma gas (PG) and shield gas ( Fig.2(b), it shows an approximate linear variation along with the PG flowrate. However, when welding current approaches 245A, the linear relationship will disappear and the keyhole will become unstable. It is considered that the pressure of plasma arc on molten metal becomes small, if heat input continues to increase. At this moment, PG flowrate has less influence on the size of keyhole, and the balance of arc pressure, gravity and surface tension of weld pool will be broken. Once welding current exceeds 250A, vibration of weld pool will happen. The weld bead will become intermittent and humping. Relationship between PG flowrate and average value of target feature approximately calculated in steady state is shown in the Fig.3.  The size of keyhole is limited by the arc pressure, and the surface tension of molten metal. If target feature continues to increase with the arc pressure under high welding current, it can't afford to build the bridge between the iron plates of butt-weld. Once the distance increases to the critical value, it is difficult to maintain the keyhole's shape and probably results in burning through. Therefore, the target feature serves as a significant factor to ensure the stability of keyhole.
In the actual welding process, it has utilized CMOS camera to observe the weld zone, filter the strong arc light and realize a real-time image processing. In order to investigate the flowrate's influence on the keyhole, step change (from 2.0 to 4.0 L/min) of flowrate is taken with a constant current (235A).
The appearance of weld bead and keyhole before and after the step change is shown in the Fig.6(a). It is confirmed that the target feature increases with PG flowrate, as well as keyhole size. Increase in the height of back bead is accompanied by a decrease in the face bead, which appears to be a concave shape. Penetration of the base metal has been raised by PG flowrate.
(a) Influence on the appearance of weld bead and keyhole (b) Step response of target feature is corresponding to about 70 pixels. It shows a transient response of the target feature following the step change of PG flowrate. According to the experiment result, it can be approximated by a first-order system with time delay. Therefore, it is available to realize the maintenance of keyhole and weld bead through real-time control of PG gas flow in PAW.

Image processing for keyhole maintenance and seam tracking
In the PAW, the weld pool image is a complex result making a coupling of the intense light of plasma arc and the heat radiation of heat-affected zone with high-temperature. It is significant to design an image processing to effectively extract the keyhole feature from the image containing the arc state, brightness level of the weld zone, and some other information hidden in the weld pool. In the built system, welding torch starts to move after the keyhole is formed. In order to design the image processing program, the first thing is to make sure whether the keyhole is formed. It will indicate that the welding has entered an initial stable state.
Above all, the feature image will be pre-processed to perform a binarization processing.
Considering that the weld pool has not yet completely formed, when the welding just starts. The length of weld pool will be shorter than in the steady state. At this moment, the back of weld pool, i.e. the bottom area of the feature image, is with low brightness. Due to approximate symmetry, it has set a detection area at the bottom left of feature image to determine the welding state, shown in the Fig.7(a).
If the average brightness of detection area is below the set threshold, it will consider that the PAW hasn't entered a steady state. Cycle detection will be adopted until the welding reaches a stable state, and then subsequent image processing will be carried out. By this means, the images captured at the beginning and end of welding can be excluded to improve the image processing efficiency.
(b) PAW in a steady state Because the electrode is in the geometric center of the welding torch and the plasma arc generated by the electrode is highly concentrated, the underside of the electrode can be considered as the welding position. In this way, the photographed image will include shield cap. It is able to calibrate the welding position by detecting the geometric center of the shield cap, which is also the center of the welding torch. Therefore, it sets another detection area at the upper left to identify the shield cap, shown in the Fig.7(b). And lowest coordinates of the welding torch will be calculated to determine the electrode position, i.e. welding position in the horizontal direction.
Because the brightness of keyhole and possible gap is low, and they also have obvious shape characteristics. Therefore, template matching can be used to quickly identify them to improve the image processing efficiency. In PAW process, when the keyhole and weld pool are basically formed, the template matching will start to work. In order to reduce the effect of singularity with high brightness and enhance the contrast of keyhole and gap, the current image is combined with the previous two to generate a composite image with lowest brightness of each pixel. Representative samples of feature image under several welding conditions will be captured through basic experiments for template matching in advance. Then, the region of interest (ROI) including a steady-state keyhole or a possible gap will be obtained as the template models. Through similarity calculation with the template models, template matching is adopted to separate and extract the ROI from the weld pool images acquired during welding experiments. And the subsequent image processing will be performed only on the ROI rather than the entire feature image, which can save processing time and improve the efficiency.
Template matching has adopted normalized cross correlation (NCC) in this study. It will calculate the similarity between the template model and target image. And if the similarity degree meets the threshold requirement, it will extract the similar regions in ROI. Similarity degree is given by In the equation above, the pixels of the template image are × . The template matching of keyhole is shown in the Fig.8(a). Since keyhole has a relatively specific shape, it can be easily recognized in the steady state without multi-stage processing. However, welding defects such as a gap or some other disturbance may exist, and it will cause the deformation of keyhole.
Therefore, to identify the keyhole accurately, it has utilized several template models of the unstable keyhole in the basic experiments. In this way, it will reduce the image processing errors and robustness can be expected. According to the keyhole identification, the size and boundary curve of keyhole in the feature image can be obtained at the same time. As a disturbance, the gap may exist in the weld zone due to the welding defects or placement of base metal. A large gap will cause the deformation of keyhole and destroy the force balance in the weld pool. Moreover, since the welding control in this study is based on the image processing, it is necessary to avoid the processing mistake caused by possible gap. Hence, a detection area traversing the feature image is set below the shield cap, shown in the Fig.8(b). The regions with low brightness can be obtained by binarization. Through comparing the brightness of these regions with the central column in feature image, area with smallest difference and an intense brightness change on the edge will be temporarily identified as a gap.
A different similarity standard robust to the linear average brightness change is required, and the study has taken a double check of the gap. Then, it will carry out the template matching of the gap in the previous detected area, shown in the Fig.8(c). The matching score will be recorded when it is above the set threshold. Furthermore, it will calculate the transformation matrix based on the vector difference between template model and target image. The template model is represented through translation and linear transformation, and affine transformation. It will take template matching at least three times with some other models, and the biggest matching score will be utilized to determine whether the target image contains a gap.
In addition, the weld line can be obtained by image processing to detect the horizontal center of the gap, or extract the shape feature of the weld pool introduced in the author's previous study in the But if the welding condition is not adjusted properly, it is difficult to maintain a stable keyhole and will lead to the keyhole closure. As a result, it is significant to maintain the stability of characteristic, i.e. keyhole size and target feature, in the ROI by adjusting the welding conditions. Thus, ROI can be retrieved from the changing feature image even if any disturbance exists. Eventually, the image processing results will be extracted and sent to the robot control PC and digital controller for power source, respectively. Through real-time processing in the PAW, the keyhole maintenance and seam tracking can be achieved at the same time.

Controller design for welding robot and power source
In the joint space of the welding robot, it can conduct a path planning to ensure the end effector, i.e. plasma torch, passing through the start and end point of the weld line. However, the trajectory between the two points may be unknown, it will depend on the weld line and the robot kinematics characteristic. In the visual robotic system, path planning of the operating space is carried out by visual sensor in the Cartesian coordinates, and the trajectory of plasma torch can be represented by a series of nodes. Moreover, to find the vector of joint variables which can achieve the desired movement of the plasma torch, it needs to solve the inverse kinematics and path planning problem of welding robot.
In this study, the control PC will calculate the coordinate conversion relationship from the image point to the space point. Then, based on the target feature obtained by image processing, the controller will estimate the desired pose of end effector. In this way, it will perform inverse kinematics calculation and transmit the pulse signals based on position information to the robot. Shan and Guo has designed the state feedback controllers and studied the relationship between movement pulse signal and end effector location in the robot kinematics [21,22]. Based on the position information, the required rotation angle of joint variables is given to each robot motor, and a desired trajectory of plasma torch will be obtained. However, in the practical operation, there are external forces and torques like inertial force and gravity, which will affect the dynamic performance of the welding robot. Even if the deviation from the target position is correctly calculated, it is difficult to obtain an ideal operation.
Therefore, a digital controller for the welding robot is designed in this study.
The block diagram of digital controller designed in this study is shown in the Fig. 9.
20 Fig.9 Block diagram of digital control for the welding robot and power source

digital controller for welding robot
Control of the PAW robot is performed by feeding a movement pulse signal from the robot control PC to the manipulator. Since strong inertia force exists in each joint axis of the welding robot, plasma torch is unable to reach the target position immediately when the controller receives a movement pulse.
Therefore, it's necessary to consider the time delay of the robot motion in PAW system. Fig.10 Indicial response of torch axis to a control pulse.
Based on the robot kinematics experiment, the characteristic curve of the system response has been summarized in this study. The 1st and 2nd order delay indicial response to the movement pulse in the torch motion are shown in the Fig. 10. Since the second-order response curve is closer to the step response, the robot control system in this study will be approximated by a second-order delay system. When a pulse signal is sent to the control port, it will be integrated in the robot controller and the movement of plasma torch will base on this integrated value. Hence, the dynamic behavior of the welding robot expressed in the form of transfer function ( ) is shown as follows where ( ) is the current position with respect to the weld line, and ( ) is the operating value of robot controller. Due to the Fig. 10, the damping ratio = 0.8, and the undamped natural frequency = 40rad/s are approximated in the designed robot welding system.
Since < 1 , the singularity 1 , 2 can be calculated by making the denominator of Eq. (2) equal to zero, which 1 , 2 = − ± = √1 − 2 . Then, the Eq. (2) By adopting the z-transform for the transfer function ( ), it is able to process the discrete-time signal and acquire the following equation The desired response for the robot movement is illustrated in the Fig.11, Where is sampling period with respect to the interval of image processing, and k is the number of iterations. Due to the Eq. (9) In the Eq. (8)

Experiment result and discussion
This study establishes a model of keyhole feature and process parameters. Through accurately obtaining the feature information of weld pool and effectively adjusting the welding conditions, it is available to stabilize the keyhole and achieve a quality control. Based on the influence of welding conditions investigated by basic experiment, it will evaluate the established model, image processing, and the control system.
Change of the welding condition is easy to cause a weld defect, such as incomplete fusion or burn through in the welding process. It has utilized the base metal with a 0 − 2 gap in this study, shown in the Fig. 12. Therefore, the plasma gas is easy to escape from the possible gap affecting the arc pressure, and making an unbalance in the welding pool. In this way, it's available to inspect the comprehensive performance of control system with a varying disturbance. The research goal is to verify whether the built system can maintain the stability of the keyhole feature to guarantee the welding quality. Welding speed is 2.5mm/s, and the length of base metal adopted for the one-pass welding experiment is 30cm. The image feature of keyhole has been obtained to analyse the experiment result.
The gap feature obtained by image processing is shown in the Fig.13. As a result, the detected gap width is smaller than the actual reserved under 2mm. This is due to the influence of the fluidity and inertia of molten metal on both sides of the gap. And when the actual width is less than about 1mm, molten metal can fill into the gap in front of the keyhole, so that the gap cannot be detected during this period.
26 Fig.13 Relationship between the detected gap width and the actual gap width Plasma arc couldn't penetrate the base metal immediately after the welding started, and the target feature wasn't detected by the visual sensor at this moment, because the keyhole hadn't formed. The welding torch only started to move 4 seconds after the plasma arc generated a relatively stable keyhole.
Since the flow of molten metal in the weld pool hadn't been stabilized after the torch started moving,  Since the plasma gas is easy to escape from the gap with an increase of its size. The gap may easily cause a deformation of the keyhole and destroy the force balance of arc pressure, gravity and surface tension in the weld pool. On the other hand, the diameter of heat-affected zone remains almost invariant due to a constant welding current, so the gap will lead to a decrease of the molten metal in the weld pool. It can be found from the experiment results that in order to keep the target feature constant, i.e., to maintain the shape of the keyhole, the flowrate of plasma gas has gradually decreased since the gap size became larger.
The welding results are shown in the Fig.15. Above all, by inspecting the appearance of base metal on both sides, the weld bead shows no obvious welding defects such as incomplete fusion or burn through. By utilizing a flaw detector to inspect the weld zone after the experiments, and no blowhole is found inside the weld bead. It has utilized laser cutting machine to obtain the cross section of the weld beads in order to analyze the welding quality, shown in the Fig.16. It can be concluded that PAW has achieved a good welding result. By comparing the cross-sections of weld bead at each stage of the welding process, it is found that the plasma arc always penetrates the base metal, but the crosssectional area as well as the height of the weld bead decreases due to the width increase of the reserved gap.
(a) Surface of weld bead (b) Back of weld bead

Conclusions
In this study, it aims at the visual inspection and quality control requirements of the PAW process.
And it has investigated the evolution process and response rule of keyhole behavior and weld pool feature, when welding condition dynamically changed. As a result, the built PAW system can effectively maintain the keyhole and guarantee the welding quality. The main results of this study are summarized as follows: 1. Author has investigated the relationship between the welding condition and image feature of the keyhole by utilizing high speed video camera, and the influence of welding condition on the weld bead has been analyzed according to the basic experiment.
2. Author has designed image processing program to obtain the feature image of the weld pool and conduct a template matching of the keyhole. The target feature of keyhole and the weld pool has been extracted and processed for the welding process control.
3. Author has designed a digital controller for the welding robot and power source, and discussed the control method to stabilize the keyhole and achieve a good welding quality in this study.
4. The target feature of keyhole is successfully controlled by the designed system in the welding process. The performance of the welding control system and image processing are validated in the real-time welding experiments.