A Simple, Precise, and High-Speed Die Edge Detection Framework for the Semiconductor Industry

– This paper presents an automated high-speed die edge detection method that is meant to be implemented in non-destructive die-level defects inspection in the semiconductor industry. This paper specifically aims to achieve high accuracy (or yield) of greater than 99.95% with stable performance within a short computation time (i.e., less than 30 ms), provided the targeted unit-per-hour (UPH) is at 30,000. To demonstrate the applicability of the proposed method, the proposed method is validated using three production units (i.e., Production units A: 6000 units; Production unit B: 3500 units; Production unit C: 4000 units) and is benchmarked to two common edge detection methods that are widely implemented in the semiconductor industry nowadays (i.e., cross-correlation and normalized cross-correlation methods). The results obtained show that the proposed method is capable to perform die edge detection with promising accuracy and stable performance by achieving 100.0% yield in all three production units and outperformed the benchmarking methods. Also, the overall computation time (considering die edge detection and die rotation) of the proposed method is short, which is approximately 30 ms.


Introduction
The semiconductor industry is one of the most competitive and fast-evolving industries. In order to fulfill the high quality and consumption requirements, the product quality must be at an excellent level with mass production. Defective products are proven to impinge the manufacturing process and increase cost in general [1]. An automated optical inspection (AOI) system is a non-destructive approach that is widely implemented in the semiconductor industry typically for quality assurance purposes [2,3]. Die edge detection is the first and most fundamental step in defects detection and dies coordination in an AOI system. Faulty edge detection can result in underkill (false negative) and overkill (false positive) which contributes to major end-product quality issues. In most cases, faulty edge detection can return the AOI system a wrong die coordinate (i.e., center coordinate) which results in a wrong picking center. Thus, a "flying die" phenomenon could occur whereby the die is detached from the picking collet during the high-speed machine motion.
For application in an AOI system, the edge detection method must fulfill the following criteria: (a) the accuracy must be high and performance must be stable, and (b) the computation time must be low. In most of the cases, the accuracy requirement (or commonly known as yield) for edge detection is at 99.95% to 99.99%. In terms of computation time, the unit-per-hour (UPH) requirement could be varying from 5,000 UPH to 50,000 UPH dependent on the machine capability. The higher the UPH, the better the throughput of the AOI system, the higher the machine price. Taking an AOI system with 30,000 UPH as an example, the total allowable time to process one die is 120 ms. Thus, the ideal computation time for edge detection would be 1/4 of the total allowable time (i.e., approximate 30 ms).
Edge detection is performed by identifying the discontinuities of the pixel intensities in an input image. Sobel [4,5], Prewitt [6,7], and Canny [8][9][10] are some common operators and filters which can effectively locate the boundaries of an object in the input image. To date, many researchers proposed various effective edge detection methods but with a tradeoff of high computation complexity which are not suitable for high-speed application. Chao and Tsai in [11] proposed an improved anisotropic diffusion method to detect defects on low contrast surface of a glass object. The anisotropic diffusion approach or commonly known as Perona-Malik diffusion [12] is a famous edge detection approach that can retain important lines and edges features while minimize the image noise. In [11], the authors combined the conventional anisotropic diffusion with an adaptive smoothing process to improve the defects detection process on the glass object. The reported computation time for the proposed method is approximately 300 ms. In [13], the same authors proposed another edge detection method by combining the conventional anisotropic diffusion with particle swarm optimization (PSO). The proposed method aims to detect defects on liquid-crystal display (LCD) in low contrast backlight. By using the improved method, the overall computation time is reduced to 60 ms. However, the algorithm is only applicable to non-textured surfaces which greatly limit its application in the semiconductor industry as the die material could possibly black-coated with textured surface. Chang et. al. proposed a simple edge detection method based on thresholding to identify cracks on touch panel [14]. The proposed method can achieve high accuracy (i.e., 94.0 to 99.0%) but with a tradeoff of long computation time.
It is worthwhile to remark that the template matching method [15] is of the most common methods implemented in the semiconductor industry for edge detection purposes [3,[16][17][18][19] [20]. The feature-based template matching approach has better connection in terms of control points and features between the input image and the template image [20]. Cross-correlation and normalized cross-correlation methods are two typical examples in template matching approach. In crosscorrelation method, the similarity is calculated basically by computing the sum of pair wise multiplications on the particular pixels in both the images. The larger the value, the greater the similarity. The normalized cross-correlation method is an improved method of the conventional cross-correlation method.
The main improvements lie within the ability to omit the changes in global brightness, such that regions with consistent bright and dark pixels will not affect the output of the calculation. Also, the output of the normalized cross-correlation method is always range from -1 to 1. For a better understanding in the cross-correlation and normalized cross-correlation methods, please refer to [21] and [22], respectively. Nevertheless, the template matching method can result in faulty edge detection which lead to an undesirable quality issue (i.e., overkill). Faulty edge is usually occurred when the edge detection is performed on die with markings, especially die with inconsistent marking location (refer to Figure 1 (a) and (b)). The overall similarity metric returns from the template matching method is predominant by the inconsistent marking location which act as the prominent features in template matching method. This similarity metric is then lead to a faulty edge detection (refer to Figure 1 (c)).

Fig. 1
In this paper, an automated high-speed non-destructive die edge detection method is proposed. The proposed method aims to be implemented in an AOI system for non-destructive optical inspection on die-level in a real manufacturing production line, typically in the semiconductor industry. Based on the recent literature, the various edge detection methods are either effective with long computation time [11][12][13][14] or short computation time with a trade-off of unstable accuracy and performance stability [16][17][18][19]. The main objectives of this paper are to address the research gap by proposing a die edge detection method which could achieve promising accuracy with stable performance within a short computation time. The proposed method consists of two main stages: (a) Stage 1: learn phase and (b) Stage 2: execution phase. The remainder of this paper is organized as follows: Section 2 detailed the proposed AOI system (i.e., image acquisition and detailed algorithm of the proposed method); Section 3 presents the experimental validation, and Section 4 remarks the conclusion of this paper.

The proposed AOI system
The proposed edge detection method first starts with the image acquisition stage, then followed by Stages 1 (learn phase) and 2 (execution phase), respectively. In image acquisition stage, the image of the die is captured and is stored in bitmap format (.bmp). In Stage 1, the user is required to select a non-defective die from the production unit to perform the learn phase. The main purpose of the learn phase is to compute the mean intensity difference (u ̅ ) which is an essential parameter in the next stage. It is important to remark that only one unit is required to perform the learn phase for each production unit. Stage 2 is the core image processing algorithm stage whereby termed as the execution phase. The images captured in image acquisition stage are used as input in this stage for edge detection and die rotation purposes. Figure 2 shows the overall block diagram of the stages involved in this paper. The methodology of each stage is detailed in the subsequent subsections.

Image acquisition Stage
In a real manufacturing production line, the imaging environment and lighting configuration can substantially affect the accuracy of die edge detection [3,23]. Figure 3 shows the architecture and hardware setup of the proposed AOI system. A Toshiba CMOS monochrome camera (2.2MP) is used to perform the image acquisition. The camera is mounted to a Navitar 12x zoom motorized lens together with a C-mount coupler, a mini adapter, and a 0.5x lens attachment. An external coaxial light (red) is fixed under the lens and followed by the low angle ring light (blue). The external coaxial light (red) and low angle ring light (blue) are hold in position using the mechanical arms with specific working distance (WD). The WD of the lens, external coaxial light (red), and low angle ring light (blue) are 165.0 mm, 56.0 mm, and 14.0 mm, respectively. The die is placed in a pocket of a carrier tape, whereby the carrier tape is hold by a bracket in a static position (refer to Figure 4).  In this paper, the die is the object of interest and shall be prominent from the background. This could be done by finetuning the camera and light settings of the imaging configuration, which included the camera exposure time, camera gain, intensity of the external coaxial light (red), and intensity of the low angle ring light (blue). Table 1 shows the camera and light settings of the image configuration.

Table 1
The algorithm of the proposed method is developed and execute using a Window 7 Professional personal computer with Intel (R) Core (TM) i7-6700 CPU @ 3.4GHz and equipped with installed memory (RAM) of 8.00 GB. The images captured in this stage are in gray scale with a resolution of 1088x1088 at a calibration value of 4.1360µm/ pixel.

Stage 1: Learn phase
Stage 1 (learn phase) starts by selecting a non-defective unit as the input image. The die and background are then characterized into two distinct clusters using an improved K-Mean clustering method, namely K-Mean with histogram-based initialization [24]. This method can effectively reduce the computation time of the conventional K-Mean by providing the initial values based on the histogram analysis of the input image. The final centroid can be calculated with shorter iterations as compared to the conventional K-Mean. Also, the K-Mean with histogrambased initialization can eliminates inappropriate initial values which may lead to center redundancy, dead center, and possibility of trapping in the local minima [25].
Next, hole filling is performed on the output obtained from the K-Mean with histogram-based initialization. Figure 5 shows the output of the learn phase. The die and background are now white and black in color, respectively (refer to Figure 5 (c)). The coordinates of the detected die edge (i.e., edge of the white object) is used to calculate the mean intensity difference (u ̅ ) in the input image using Equation 1 (refer to Figure 5 (d)).
where denotes the intensity of the foreground transition coordinate (refer to Figure 5 (d)); denotes the intensity of the background transition coordinate (refer to Figure 5 (d)); and E denotes the total number of transition coordinates (i).  Figure 6 shows the detailed flow chart of the proposed method: Stage 1 (learn phase) and Stage 2 (execution phase). The u ̅ obtained in Stage 1 is used in this stage for die edge detection purposes. In detailed algorithms of the execution phase is as follows:

Stage 2: Execution phase
Step 1: Landscape analysis in x and y-axis. Calculate the intensity difference for each pixel line in x and y-axis respectively. The pixel is set as foreground if and only if the intensity difference is greater than u ̅ which obtained from Stage 1.
Step 2: Grouping the coordinates. The coordinates of the foreground pixel are then characterized into four groups in order to identify which edge the specific coordinate is belonging to. This is done by calculating the minimum Euclidean Distance from four seed points, namely CoordinatesL and CoordinatesR (for x-axis); CoordinatesT and CoordinatesB (for y-axis). The coordinates of foreground pixel with minimum Euclidean Distance to the CoordinatesL, CoordinatesT, CoordinateR, and CoordinatesB are respectively termed as group points left, top, right, and bottom. The CoordinatesL, CoordinatesT, CoordinateR, and CoordinatesB are calculated using Equations 2 to 5, respectively. Figure 7 illustrates the coordination of each seed point. Step 3: Outlier elimination. The detected edge (i.e., foreground pixels in Step 1) may consist of outlier if the die is a defective unit with defects such as chipping, foreign material, and/ or burr edge. These defects can potentially impinge the die edge detection. In order to eliminate the outlier, the line of best fit must first be calculated for each group point (i.e., group points left, top, right, and bottom) by using Equations 6 to 8. Figure 8 illustrates the line of best fit of a specific edge.
where denotes the slope, b denotes the coefficient of , and denote the arbitrary coordinate ( , ) for each r, denotes the y-intercept, ̅ denotes the mean of the x-values, ̅ denotes the mean of the y-values, and n denotes the total number of r.

Fig. 8
Next, calculate the perpendicular distance (d) of an arbitrary coordinate (xr, yr) from the line of best fit using Equation 9. Eliminate the outlier using the boxplot method [26] and recalculate the line of the best fit using Equations 6 to 8. This line of best fit is optimized and is termed as optimized line of best fit in the subsequence section.
Step 4: Calculation of die corners. In this step, the die corners are determined based on the optimized line of best fit in Step 3. The die corners located at top left, top right, bottom right, and bottom left are termed as die corners P, Q, R, and S, respectively. The die conners P, Q, R, and S can be calculated by determining the intersection of two optimized lines of best fit as follows: Step 5: Calculation of die center. An accurate die center is important for die picking purposes in an AOI system. The die center can be calculated by averaging the x and y coordinates of the detected edge. Equation 11 shows how the die center is computed. The die edge detection is now completed.
Step 6: Die rotation. Die rotation is not an essential step in die edge detection. However, it is important to remark that the die rotation can greatly reduce the complexity for majority of the defect detections. The defect detection algorithms can be implemented without considering the orientation and/ or angle correction of the input die. This can substantially reduce the computation time in the subsequent defect detection steps. In order to perform the die rotation, the rotation angle and the direction of rotation must first be determined. Figure 9 illustrates how the die rotation is performed. From this figure, the rotation angle from the Coordinate E (the original arbitrary die corner) to the Coordinate G (the rotated arbitrary die corner) on the Coordinate F (the die center) can be calculated by using Equations 12 to 14, where the Coordinate E is obtained from Step 4 (in Equation 10), the Coordinate G is calculated using Equation 15, and the Coordinate F from Step 5 (in Equation 11). The direction of rotation is determined by using Equation 16.
where ⃗⃗⃗⃗⃗ denotes the vector from E to F, ⃗⃗⃗⃗⃗ denotes the vector from F to G, ‖ ⃗⃗⃗⃗⃗ ‖ denotes the magnitude of vector ⃗⃗⃗⃗⃗ , ‖ ⃗⃗⃗⃗⃗ ‖ denotes the magnitude of vector ⃗⃗⃗⃗⃗ , w denotes the width of the die, and h denotes the height of the die.

Experimental Validation
To validate the applicability of the proposed method in a real manufacturing production line, three production units namely production units A, B, and C are used. The die size of the production unit A is 1.3 x 1.3 mm with a total of 6000 units. The die size of production units B and C are 2.0 x 1.6 mm, whereby the total units are 3500 and 4000, respectively. The validation criteria are set based on the requirements as stated in Section 1 whereby the proposed edge detection method is evaluated in terms of: (a) the die edge detection accuracy and stability and (b) the computation time. The minimum edge detection accuracy (i.e., yield) is set as 99.95% and the computation time for edge detection shall be not more than 30 ms in general, provided the target UPH is at 30,000. For benchmarking purposes, the proposed method is compared to two common edge detection methods: (a) cross-correlation and (b) normalized cross-correlation methods. In order to established a fair comparison, the input images for crosscorrelation and normalized cross-correlation methods are captured using the same imaging environment, lighting configuration, as well as the same camera and light settings as in image acquisition stage in Section 2.1. All three methods are executed using the same computer environment. Figure 10 shows the selected results of die edge detection using the three methods (i.e., cross-correlation, normalized cross-correlation, and the proposed method). The overall die detection accuracy (i.e., yield) is depicted in Table 2. Table 3 shows the quality issues due to faulty die edge detection for correlation, normalized correlation, and proposed methods. Table 4 shows the comparison in mean computation time for all three methods.
In Figure 10, a sample of edge detection result is selected from each production unit. Based on the results, the crosscorrelation and normalized cross-correlation methods show faulty edge detections. The faulty detected die edges located beyond the real die edge (Figure 10 (a), (b), (e), (i), and (j)) can potentially result in the "flying die" phenomenon and overkill in defect detections in the subsequent steps. The gap between the faulty detected die edges and the real die edge can be characterized as chipping and/ or crack which leads to overkill in defect detection. For faulty edge detection by which the detected edge is located within the real die edge as shown in Figure 10 (f), the return die center is incorrect and can lead to a die picking issue in the later step. Also, the area between the faulty detected edges and real die edge would be characterized as defects such as foreign material and/ or burr edge. By comparing the edge detection obtained from the normalized cross-correlation method and the proposed method as in Figure  10 (j) and (k), the proposed method (Figure 10 (k)) shows better sensitivity and outperformed the normalized cross-correlation method with better accuracy in die edge detection. The superior die edge detection found in the proposed method can be explained by Step 3 in Stage 2 (execution phase). The optimized line of best fit is found proficient and contribute to better die edge detection in an AOI system.  Table 2  Table 3   Table 4 From Table 2, for production units A, B, and C, the crosscorrelation method shows faulty die edge detection in 12, 8, and 8 units, respectively, and the normalized cross-correlation method shows faulty die edge detection in 5, 2, and 1 units, respectively. Amongst the three methods, the accuracy or yield of the normalized cross-correlation method and the proposed method are comparative. However, it is important to remark that only the yields of the production units A, B, and C obtained from the proposed method achieved the minimum yield requirement (i.e., 99.95%). Based on the validation, the proposed method shows better accuracy and stable performance in die edge detection.
Computation time is another important factor in die edge detection in a real manufacturing production line. Based on Table 3, "flying die" phenomenon occurred in the crosscorrelation and normalized cross-correlation methods, whereas, none found in the proposed method. From Table 4, the proposed method shows shorter computation time in production unit A and the cross-correlation method shows shorter computation time in production units B and C. However, it is worthwhile to remark that the die edge detection (without die rotation) of the proposed method is not more than 30 ms which fulfills the criteria as aforementioned, provided the target UPH is at 30,000. Considering the computation time with die rotation, the computation time obtained from the proposed method is very encouraging. The die rotation can potentially reduce the computation time required for the majority of defect detection algorithms in the later steps by eliminating any orientation and/ angle related calculations.
Overall, the proposed method can be applied to wide variations of die regardless of the materials and/ or the presence of marking. Different production unit may reflect lights in a different intensity which lead to a different value in u ̅ . Therefore, to ensure a suitable u ̅ is used in Stage 2 (execution phase), it is important to determine the suitable u ̅ for a different production unit. In Stage 2, the execution phase analyzes the input image in both the x and y-axis and optimizes the edge detection by removing the outlier which could be a result of chipping, foreign material, and/ or burr edge. The optimized line of best fit improves the sensitivity of the edge detection method which can be seen in Figure 10 (k). Based on the image processing algorithms in Stage 2 (execution phase), the proposed method is invariant to rotation, scaling, and/ or presence of marking on the die which may or may not be consistent in location. As the marking is not a factor that contributes to edge detection in the proposed method, the faulty edge condition as reported in the template matching method can be eliminated. The applicability of the proposed method is validated using three production units with considerable high in quantity. The main purpose is to demonstrate that the proposed method is capable to maintain stable performance with promising accuracy (i.e., greater than 99.95%) in die edge detection. Also, the mean computation time recorded is relatively low (i.e., 30 ms) which reflects that the proposed method is suitable for high-speed applications.

Conclusion
An automated high-speed non-destructive die edge detection method for the semiconductor industry is proposed in this paper. The proposed method aims to be implemented in an AOI system in a real manufacturing production line to perform the non-destructive optical inspection on die for quality assurance purposes. The proposed method consists of two main stages: Stage 1 (learn phase) and Stage 2 (execution phase). In image acquisition stage, the imaging environment and lighting configuration have been proposed. The images captured in this stage are used as input to Stages 1 and 2. In Stage 1, a nondefective unit is selected to use as an input to the learn phase. The main purpose of Stage 1 is to compute the u ̅ which is an essential parameter in the next stage. In Stage 2, the core image processing algorithm is performed. The images captured in image acquisition stage are used as input in this stage. The main algorithms in Stage 2 are: (1) landscape analysis in the x and yaxis, (2) grouping the coordinates, (3) outlier elimination and calculation of optimized line of best fit, (4) calculation of die corners, (5) calculation of die center, and (6) die rotation. The proposed method is validated using three production units namely production units A, B, and C. The proposed method is evaluated in terms of accuracy or yield of the die edge detection, performance stability, and the computation time. To demonstrate the superiority of the proposed method, the proposed method is benchmarked to two common die edge detection methods in the semiconductor industry (i.e., crosscorrelation and normalized cross-correlation methods). Based on the experimental validation, the proposed method found outperformed the cross-correlation and normalized crosscorrelation methods by achieving promising yields in die edge detection of 100.0% for all three production units. The promising yields obtained also proven the stability of the proposed method in die edge detection. In terms of computation time, the die edge detection of the proposed method is not the shortest but comparative to the cross-correlation method (refer to Table 4). Nevertheless, taking UPH of 30,000 as the target, the mean computation time of the die edge detection of the proposed method for all three production units is acceptable as it is not more than 30 ms. Furthermore, when considering the computation time of the die rotation, the proposed method remarks a mean computation time of 29 (± 2) ms, 30 (± 3) ms, and 29 (± 2) ms for production units A, B, and C, respectively. The obtained mean computation time (with die rotation) are very encouraging as the die rotation is expected to facilitate the defect detection algorithms in the later steps. Overall, the proposed method demonstrates its proficiency and applicability in die edge detection with promising accuracy or yield, stable performance with short computation time. By incorporating the proposed method in an AOI system for the semiconductor industry, the overall defect detection algorithms could be improved and the "flying die" phenomenon which potentially leads to quality issues can be eliminated.

Funding
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.