Fast plug-in capacitors polarity detection with morphology and SVM fusion method in automatic optical inspection system

Defect detection is a critical element in the PCB manufacturing process. Different from surface mount PCB, the components on the plug-in PCB are usually installed manually, resulting in significant errors. We make contributions in the following two aspects: (1) a framework and measurement method of a light source and make a cheap and efficient lighting system; (2) a fusion algorithm based on machine learning and morphology for polarity detection of plug-in capacitors. The capacitor is detected using SVM and fused with the polar coordinate expansion method. The AOI system and the proposed fusion algorithm have been applied to the production line, with an accuracy of 99.73%\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$99.73\%$$\end{document} and a missed detection rate 0.12%\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$0.12\%$$\end{document}.


Introduction
Due to the variety, large size, and irregular shape of plug-in components, it is difficult for high-speed automatic optical inspection (AOI) equipment to detect such components [1] in production process. In addition, due to cost and other factors, many factories in the PCB assembly line still use manual visual inspection of component defects. However, manual visual inspection has the fatal disadvantages of high error rate and low efficiency, and high labor cost. In the PCB with plugin components, electrolytic capacitors are the most frequently used polar components, which have a very great detection demand. With the rapid development of machine vision, the design of a simple structure, low cost, and simple method for rapid detection of capacitor polarity system, instead of manual detection, has application value.
The main works of this paper are: (1) develop an AOI system for capacitor polarity defect detection, propose the framework and measurement method of a light source and B Jiang Lurong jianglurong@zstu.edu.cn 1 make a cheap and efficient lighting system; (2) propose two effective capacitor polarity detection methods from machine learning and image morphology and fuse the two detection methods for this AOI system. In detail, the first method refers to machine learning, extracts the histogram of oriented gradients (HOG) feature of the capacitor image, and trains the extracted feature information with support vector machine (SVM) to judge the polarity. The second method refers to image morphology, and converts capacitive images into polar coordinates. A new fusion algorithm with low complexity and great accuracy has been developed by combining the two methods.
The remainder of this paper is organized as follows. Section 2 introduces relevant works. Section 3 takes an overview of our AOI system. Then, it introduces the light source structure, the image capture, and the detection method. Section 4 introduces two polar detection algorithms with SVM and polar coordinate conversion methods. Section 5 provides the algorithm fusion process, experimental results, and field detection results in the factory. At last, Sect. 6 concludes our work.

Related Works
The critical technology of capacitive polarity recognition is the polarity detection algorithm with the image. Because the pin configuration of the capacitor dictates that polarity detec- Lin et al. [2] proposed a method for detecting the existence of capacitors on PCB using the YOLO algorithm. The recognition time of a single component can be as long as 294 milliseconds (219-710ms). Susa et al. [3] proposed a method to detect the capacitor on the circuit board using YOLO V3. with the accuracy of 93.33%. Fan et al. [4] proposed a three-stage capacitor search algorithm based on YOLO target search to realize the recognition and location of plugin capacitors. Then, they use the hybrid feature comparison algorithm to detect the polarity of capacitors. Li et al. [5] also used YOLO V3 to identify capacitive components' existence and achieved an average accuracy of 93.07%. The average processing time of a single resistor is 2.66-4.29s. Fan et al. [1] used based on Haar-like feature and AdaBoost classifier to recognize and classify plug-in capacitors with eight directions. He et al. [6] gave a more comprehensive overview of the defect detection of the generative adversarial networks. Zeng et al. [7] proposed an atrous spatial pyramid poolingbalanced-feature pyramid network (ABFPN) improving the detection performance of tiny defects on PCB. In addition, the defect detection methods applied to other products can also be used for reference. Tao

Overview of System
The AOI system consists of a lightbox, an industrial camera, an industrial computer, an interactive screen, a speaker, an alarm lamp, and other equipment, as shown in Fig. 1. This system is installed on the conveyor belt of the printed circuit board assembly (PCBA) line and placed in front of the wave soldering furnace Fig 2 shows a prototype that we build. Fig 3  depicts the detecting procedure of the AOI system. First, the system tracks and captures the PCB. Then, the system uses the template to locate and collect the capacitor target on PCB. Finally, the capacitor image is delivered to the detect polarity section, where fusion algorithm is used.

Illuminance Evaluation Method
The rectangular area under the field of vision of the industrial camera can be uniformly divided into 8 × 8 checkerboards [11]. The illumination core image capture region is the center 6 × 6 area. A luminance meter was used to measure the luminous intensity in each grid. Place the checkerboard 10 cm below the lightbox, as indicated in Fig. 4a, to ensure that it is horizontally aligned with the lightbox. The luminous intensity recorded in the grid in row i and column j is denoted as L i j (in Lux) in Fig. 4b. The ratio of the minimum illumination to the average illumination on a given surface is referred to as illumination uniformity.
For the entire area within 8×8, the illumination uniformity is U global is: For the core image capture area within 6 × 6, the illumination uniformity is U core is: where the closer U global or U core is to 1, the more uniform the light distribution in that area is.

Light source design and evaluation
The detection performance of the entire system is influenced by the luminous intensity, stability, and homogeneity of the light source system [12,13]. In terms of AOI systems, LED light sources may generally perform as well as or better than metal halide and quartz halogen lamps [14]. As a result, the system uses civil-led rectangular light panels, which provide lighting that roughly satisfies the criteria while lowering production costs.
As shown in Fig. 5, the lightbox structure is separated into the square box (a1) and inclined box with 75 • to the horizontal plane (a2). There are three lighting schemes: top light (b1), four sides light (b2), and five sides light (b3). Table 1 shows the luminous intensity measurement results for two types of boxes and three lighting schemes. Finally, we find out that the light source with square box (a1) and five sides light (b3) has the best luminous uniformity result. Based on this scheme (a1 & b3), we made a prototype for image capture, as shown in Fig. 2.

Template making
Before defect detection of capacitors, a template for detection must be created, which comprises the PCB diagram, calibration mark, frame coordinates, and compensation angle of each capacitor. The capacitor polarity can be adjusted to a horizontal state by rotating the image in the template.
To simplify the classification type of polarity recognition, it is set that after image rotation, the capacitor's polarity is left as positive. As shown in Fig. 6, when making the template, the compensation angles of the capacitors with upper, lower, left, and right polarities are −90 • , 90 • , 0 • , and 180 • , respectively. Before the polarity detection of a single capacitor, rotate the capacitor with the corresponding compensation angle. Finally, the multi-classification problem of polarity detection of capacitor can be transformed into a two-classification problem through the above rules.

Image matching and tracking
The pixel of the industrial camera employed in this project is 12 million (4096 × 3000). Considering the light quality and the size of the circuit board, we set the area surrounded by abcd as the best acquisition area, in which the position of ab is set according to the conveyor belt, and cd is located between 1/4 and 3/4 of the picture respectively, as shown in Fig. 7a.
When the PCB enters the camera's field of view, the camera tracks the PCB according to the template as shown in Fig. 7b. In the preprocessing stage the region of interest (ROI) coordinate data of the capacitor to be measured is obtained according to the template as shown in Fig. 7c. In ROI, template matching is carried out by the correlation coefficient

Detection Based on SVM and HOG features
In image matching and recognition applications, histogram of oriented gradient (HOG) statistics [15] is a practical feature extraction method. HOG is highly suitable for feature  extraction of the plug-in capacitors because of the inherent mode of shape and polarity characteristics.
In AOI system, we use SVM as one of the classifiers. The SVM [16,17] is a binary linear classifier. The principle of SVM is to use the kernel function to map data to a high-dimensional feature space, solve the hyperplane appro-priately partitioning the two categories of data and maximize the separation interval between the two types of data. As a result, using SVM to detect the polarity of a plug-in capacitor offers clear benefits. The flow of SVM method based on the HOG features [18] is shown in Fig. 8.

Coordinate conversion
In two-dimensional space, polar and Cartesian coordinates are extensively utilized.
The polarity of the plug-in capacitor is consistent with the white mark on the plastic film on the top of the capacitor, as shown in Fig. 9. The capacitor's plastic film often presents oval or other irregular shapes, causing errors in capacitor inner circle identification. Therefore, it is unrealistic to directly obtain the polarity characteristics of the sector by extracting a ring.
A circle in the polar coordinate system can be mapped into a rectangle in the Cartesian coordinate system [20]. as shown in Fig. 10. The equation for converting polar coordinates is as follows:

Morphological processing and polarity determination
After the capacitor image is transformed from polar coordinate system to Cartesian coordinate system, the circular capacitor pattern is also transformed into a long strip pattern The capacitor polarity direction can be determined by comparing the average bright pixel distribution of the strip capacitor map in the horizontal direction using the mapping relationship. This approach can easily and quickly produce polarity results. However, the detection error rate will increase due to the interference of letters and other marks to ensure the accuracy of detection, it is necessary to extract the polar feature region. The traditional image algorithm is used to extract and correct the boundary of the strip capacitor diagram, remove the characters and noise on the surface of the capacitor, and finally determine the ROI area and compare it to obtain the capacitor polarity.

Fusion Algorithm and Evaluation
This section uses a computer equipped with Intel Core i5 2.9Ghz CPU and 6GB GPU for experiments. Please refer to these experimental environments for a better comparison.

Evaluation Criteria
The result of polarity detection is classified into two categories: positive (left) and negative (right). In the binary classification issue, the actual category of the sample and the category predicted by the learner can be separated into true positive (TP), false positive (FP), true negative (TN), and false negative (FN). This paper evaluates the performance of  Further more, we define missed detection rate (MDR) in terms of the actual condition of PCB assembly. MDR indicates the proportion of capacitor that is actually negative but the detection result is positive. Such capacitor defects and errors are usually found in the circuit functional test after welding.

Optimized Fusion of Polarity Detection
We investigated and spot-checked 30 batches of PCBs in a factory and counted the errors of manually assembled capacitors. As shown in Table 2, the average error rate of capacitor polarity in manual assembly is 3.75%. This paper uses 20400 pcs of capacitor data sets in the training stage. The training set has 16320 pcs (80%) samples, and the test set has 4080 pcs (20%) samples. In order to ensure that the training model has the same ability to predict positive and negative cases, the proportion of positive and negative samples in the training set is set to 1:1 (8160 pcs: 8160 pcs). We can adjust the number and proportion of positive and negative samples by rotating the samples in advance. The test set has an unbalanced sample distribution with negative samples accounting for about 3.75% (153 pcs)to imitate the actual defect distribution in manual assembly. The data set is tested by a fivefold cross-validation method, as shown in Fig. 12.
We provide a combined method of SVM and polar coordinate to produce a good performance in both indicators and time consumption, as illustrated in Figs. 13 and 14. Experimental results of D1-D5 are shown in Fig. 15. We can find

Test Results and Discussion
We installed the prototype of AOI system in the factory and tested three batches of PCBs (WeNP3, AoPu, DaDo). We preserved the capacitor images using KNN, VGG13, VGG16, ResNet18, and ResNet50 as shown in Table 5  Table 5 show that the proposed fusion detection method is effective in actual detection. Accuracy, recall, F1, and MDR have average values of 99.73%, 99.85%, 99.86%, and 0.12%, respectively. It only takes 7.84 ms to identify the polar of a single capacitor, which is more than enough time to satisfy the demands of practical production. The data show that our suggested method outperforms ResNet, VGG, YOLO, and KNN in terms of accuracy and time complexity.

Conclusion
In order to improve the performance of PCB defect detection, an AOI system for capacitor polarity defect detection is developed which is used to detect the polarity defect caused by the manual inserting of plug-in capacitors. The PCB defect detection experiment on the assembly line of an electronic enterprise shows that the fusion algorithm proposed is significantly enhanced compared with the single machine learning method in the detection index, and the accuracy of the developed fusion algorithm in the detection of plug-in capacitor polarity defects is as high as 99.73%. The AOI system has been applied to the production line. In the future, it will continuously accumulate data and iteratively improve the identification effect in the operation process.
Author Contributions He Jiawang and Jiang Lurong designed the study, performed the research, and wrote the paper. Jiang Lurong is the corresponding author. Zhang Suoming collected the images and labeled them. Li Renwang provided the guidance of algorithm. Xu Changguo participated in the design of the experiment. Liu Xinxia and Shen Yongjian designed the hardware of image acquisition device.
Funding This research was supported by the Zhejiang Postdoctoral Science Foundation (254824), National Natural Science Foundation of China (61602417).

Data Availability
The data set used or analyzed during the current study are available from the corresponding author on reasonable request.

Declarations
Ethical approval: Not applicable.