In addition to the lens and light-sensitive CMOS, the lighting condition has a considerable influence on the quality of the image captured by the CMOS sensor. The intensity values of the monochrome image represented by the 36 × 36 pixels have a considerably smaller range than the theoretical range, namely from 0 (black) to 255 (white). Image blurring is inevitable because of uneven brightness, inconsistent flatness in the targeted speckle pattern, and different degrees of loss of focus. Therefore, a procedure including several image processing methods, which is described in the following sections, is proposed to improve the image quality, specifically for DIC.
4.1 Optimal optical focus length
The focal length of the optical sensor was determined experimentally according to Gordon’s contrast definition after performing edge detection by using the Sobel–Feldman operator, which convolutes each pixel in a two-dimensional image [28]. Fig. 2 displays three characters: (a) the letter “M” in Calibri font, (b) the letter “m” in Calibri font, and (c) the symbol “·.” Each letter with a font size of 4 points was utilized as a target in edge detection for contrast comparisons at various focal lengths. Table 1 lists the resulting contrast for each character at different sensor heights. The results indicate that the image had the highest edge contrast when the sensor height was 2.0 mm. The focal length of the optical lens was 2.6 mm at this height. With the aforementioned optimal height, clear edges or high sharpness can be guaranteed in an image.
4.2 Strategy for image stability
Theoretically, a CMOS sensor can capture more than 10,000 images per second; however, fewer than 60 images per second could be fully transferred to the microcontroller unit used in this study because of limitations in the serial peripheral interface. Moreover, the images captured by the CMOS sensor in an extremely short period are susceptible to environmental noise, which leads to intensity drifting in each pixel. To avoid unstable or incomplete images, the resulting intensity of each pixel in the final image used in the subsequent DIC was determined according to the predominant intensity through the use of a voting scheme. With this voting scheme, pixels with abnormal intensity caused by factors such as time drifting, uneven brightness, or incomplete image transfer can be filtered out efficiently. Consequently, a more stable image can be obtained for later image processing and DIC. In this study, the final intensity of each pixel of an image was determined pixel by pixel from 50 images by employing the aforementioned voting scheme.
4.3 Pixel intensity compensation
An LD on the CMOS sensor illuminates the image of the surface texture; however, the light appears to be uneven in the images (Fig. 3), with the upper part of the images being darker than their lower part. To overcome this problem, an intensity compensation strategy is proposed. For a speckle pattern that is carefully designed with randomly but evenly distributed speckles, the averaged intensity of each row of pixels in the CMOS sensor should be approximately the same if the light from the LD is evenly distributed on the surface. In the proposed compensation strategy, the pixel intensity is compensated several times by using moving masks of various sizes (Fig. 3). The intensity of a pixel located inside the mask, which is represented by the red window in Fig. 3 and whose width is equal to that of the captured image, is compensated by the amount indicated above each image. Notably, the upper part of an image is compensated to a considerably greater extent than its lower part is. The aforementioned procedure may be repeated several times until the averaged intensity of pixels in each row is sufficiently similar, which ensures that the brightness is evenly distributed in an image. Fig. 4 displays the intensity of each row of an image before and after the intensity is compensated. The upper part of the image appears brighter than the original image. Nevertheless, the brightness adjustment performed using the nonlinear compensation technique did not increase the image contrast. The procedure described in this section simply corrects the uneven exposure of the original image captured by the CMOS sensor. The method for increasing the image contrast is described in the following section.
4.4 Image interpolation and contrast adjustment
The CMOS sensor in the optical mouse has a fixed lens; thus, the function of optical zoom is not available to magnify the light before it reaches the digital sensor. To increase the sensitivity of displacement sensing, the image interpolation that achieved the most accurate approximation of a pixel’s intensity according to the values of surrounding pixels was adopted. The problem of image distortion induced by interpolation can be minimized assuming that no image rotation is involved. Other concerns are that the contrast may decrease and halos may appear after interpolation; however, the image quality can be considerably improved using a suitable interpolation algorithm combined with contrast adjustment. In this study, bicubic interpolation involving the closest 4 × 4 neighborhood of known pixels was adopted. The image acquired by the CMOS sensor was interpolated three times. First, a pixel was interpolated into four small subpixels to halve the pixel size (i.e., close to 15 mm). Subsequently, each subpixel was continuously interpolated into nine smaller subpixels with a size of 5 mm each. Finally, each 5-mm subpixel was interpolated into 25 considerably smaller subpixels to reduce the final pixel size to approximately 1 mm.
Starting from an image consisting of 30 × 30 pixels with an original pixel size close to 30 mm, images were enlarged and interpolated using bicubic interpolations. The final images comprised 900 × 900 subpixels, with a subpixel size close to 1 mm, after three consecutive bicubic interpolations, namely double, triple, and then quintuple interpolation, as illustrated in the top panel of Fig. 5a. However, the contrast of the resulting images after three consecutive bicubic interpolations became extremely poor. A solution to this problem is provided in the following section.
The speckle pattern was designed to be chess-like and either in black or white. An image with favorable contrast increases the image discrimination and the resulting displacement calculated using DIC. To increase the image contrast, the values of the input image intensity were intentionally manipulated pixel by pixel to not only saturate them at low and high intensities for bipolarization but also to reduce the intensity uncertainty caused by diffraction.
Fig. 5a illustrates graphs of the output gray level versus the input gray level. A change in contrast is indicated by a change in the slope of the line denoted by the gamma value. With a gamma value of more than 1, a dark pixel with a low intensity becomes even darker, whereas a pixel with a high intensity changes to become considerably whiter. Thus, an image with an extremely narrow low contrast with a gray level of 30–60, as indicated in the far left diagram, can be mapped to a gray level ranging from 5 to 200 to become a higher-contrast image, as shown in the rightmost diagram in Fig. 5a (three consecutive gamma adjustments). Fig. 5b reveals the change in grayscale distribution before and after each contrast adjustment, where the vertical axis represents the total pixels in percentage. The pixel intensity distribution indicated by the dash-dot line becomes more bipolarizing after three consecutive adjustments compared with the original pixel intensity distribution represented by the solid line. Contrast adjustment was conducted once after each image interpolation, as illustrated in the top panel of Fig. 5a. Three adjustments with gamma values of 1.2, 1.5, and 1.7, respectively, were conducted in this study.
After thorough image interpolation, contrast bipolarization, and edge detection, the resulting image had higher sharpness, which increased the sensitivity in sensing the displacement through DIC.