A Novel Method for Mass Measurement by Quantitative Image Deformation

Mass of an object is an important characteristic for quality assessment. However, in some cases, it is hard to measure the mass of objects with instruments directly. In this paper, we proposed a novel method based on image processing to measure the mass of an object by analyzing the deformation degree in a grid pattern. In the spatial field, thin plate spline algorithm was adopted to calculate the minimum deformation bending energy in order to give a quantitative analysis of the weight; In frequency domain, the Fast Fourier Transform algorithm was used to calculate the spectrum within a deformation frequency area before and after the change of grids, from which the relationship between weight and spectrum was investigated. Two different equations evaluated by the above two methods were proposed in order to calculate the mass of an object. Both of them showed a high level of explanatory power of R 2 ( R 2 =0.9833 and R 2 =0.9698, respectively). The equations were then used to determine the estimated mass. Estimated and measured values were plotted against each other. A high correlation ( R 2 =0.9833 and R 2 =0.9698, respectively) was found between actual and calculated mass. Finally, Bland-Altman plot was introduced to access the agreement of the calculated mass and the actual mass. The average bias was -54.408g and -0.007g for spatial domain method and frequency domain method, respectively. Theoretical analysis and experiments were performed to verify the effectiveness of our approaches.


Introduction
Generally, physical characteristics are the most important parameters in judging the quality of an object. The significant physical properties of an object are color, volume, mass, shape, size, etc. These properties can be estimated by using several sources of information. For instance, an object's size and shape can be judged based on visual as well as haptic cues. Though Ernst and Banks' research [1] indicated that human observers integrate visual and haptic size information in a statistically optimal fashion, in the sense that the integrated estimate is most reliable, it is hard for a computer to complete this because that means more sensors are required.
With the rapid development of robot and computer image vision technology, the image processing method makes it possible to identify and analyze the physical properties of an object, especially in agriculture [2][3][4] Xu et al. [5] used machinevision technology to obtain shape, size, and color of strawberries and separate them into different grades. The correct percentages were 90%, 95%, 88.5% for each characteristic, respectively. M. Omid et al. [6] used two cameras to give perpendicular views of the fruit and calculated their volumes by dividing the fruit image into a number of elementary elliptical frustums. The volumes computed showed good agreement with the actual volumes.
Although image processing approaches have been well developed for properties such as shape and volume measurement, applications to the analysis of the measurement of object mass are relatively rare. Current research on object mass measurement using image processing method mainly focuses on fruit mass measurement. A series of studies on the relationship between geometry and mass have been carried out for different fruit species. For example, for apples [7], mangos [8], apricots [9], and citrus fruits [6]. In these studies, weight of fruit was estimated by linear and non-linear models using different geometric characteristics such as minor diameter, area and volume. However, these approaches cannot evaluate the mass of an object with an irregular shape.
The objective of this study is to develop an image processing-based method which could estimate the mass of an object without considering its geometric characteristics. The experiments in this paper were conducted on a pair of images with grids on the tensile belt before and after the placement of weight sets. Our experimental results show that the method provides a valuable alternative to the traditional methods for the measurement of mass.

Theory
In this section, we give a background concerning the main theories of the thin plate spline and the Fourier transform.
Let(x i , y i )denote the coordinates of reference points in the plane, and v i refers to the values of target function at the location(x i , y i ), with i = 1, 2, ..., n.
In two-dimensional space, the bending energy functional of the thin-plate spline interpolation function is in the form of integral: The TPS interpolation function which minimizes I(f ) is given by Where U (r ij ) = r 2 ij logr 2 ij , r ij = (||(x i , y i ) − (x j , y j )||). Note that f (x, y) is divided into two parts: affine transformation parameterized by and non-affine warping specified by w. To make the f (x, y) have square-integrable second derivatives, we have the following constraints: All together, these constraints are equivalent to the symmetric linear system of equations: O denotes a 3×3 zero matrix, o is a 3×1 column vector of zeros, a refers to a column vector with element a 1 , a x , a y and w, v , are vectors formed with w i and v i , respectively. For convenience, we will denote the (n + 3) × (n + 3) matrix of this system by L. And L −1 n refers to the upper left n × n sub-block of L −1 , and then it can be shown that

2-Dimensional Discrete Fourier Transform (2D-DFT)
The Discrete Fourier Transform (DFT) decomposes a function of time into the frequencies that make it up and plays an important role in various applications in signal processing and image processing. For example, image transformation [22][23][24], image reconstruction [25][26][27], image recognition [28][29][30], etc. In the image processing field, the Discrete Fourier transform study a signal in both time and frequency domains simultaneously, and the signal can be better understood through this way. Let M and N be positive integers, the 2D-DFT of the function f (x, y) can be calculated by Returning the Fourier spectrum F (u, v) at the frequencies u and v for u = 1, 2, ..., M − 1 and v = 1, 2, ..., N − 1. Combined with Euler's formula, the Fourier transform can be generally rewrote in the form of a real and an imaginary parts as: Where a and b are the Fourier coefficients at the frequencies u and v. The power spectrum is then estimated by: And the logarithmic power spectrum can be defined as to avoid the result of P (u, v) being a negative value, we add a correction parameter C (C is a positive integer), then the corrected power spectrum can be calculated by In our study, the value of C is 100. Since it is a very time-consuming task to calculate DFT directly, DFT is computed efficiently with Fast Fourier transform (FFT) [31] in this paper. Fast Fourier transform can be regarded as a fast version of DFT expressed by the formula Where u = 1, 2, ..., M − 1.

Materials and methods
In this paper, our proposed method includes several components: data acquisition, Feature calculation by image processing, determination of correlation formula, and computation of estimated mass. These steps are further elaborated in the following sub-sections. Figure 1 summarizes the procedures involved.

Data acquisition
The required materials mainly include a tensile belt with good elasticity, a camera, a red pen, a series of weights with different values. First, we use a red pen to draw a series of uniform red dots on the central area of the tensile belt. Next, fix a small iron stick on the tensile belt with needlework and tie a string of a certain length below it to hang weights. Figure 2 shows the treated tensile belt. The role of the iron stick is to make sure that tensile belt will not wrinkle during the stretching process because only one point is stressed. Finally, a thumbtack is used to fix the tensile belt to a test bench with a certain height. After placing the camera in a fixed position and adjusting the shooting angle, images of tensile belt are captured by the camera in 512×512 pixels and stored on the PC for further analysis. Figure  3 shows the deformation process of the tensile belt with different weights.

Feature calculation
The image processing aspect of this study consists of two parts: spatial domain analysis and frequency domain analysis. In both experiments, the original images are resized to the same size in 256×256 pixels.
In spatial domain experiment, the coordinates of the marker points on images are recorded manually, where the coordinates of the marker points in the 0g image is denoted as p, and p ′ is the coordinates of the marker points corresponding to other weights. Our algorithm written in Matlab 2014a is developed for calculating the bending energy based on TPS algorithm. Entering the values of p and p ′ into the program will generate the corresponding values of bending energy.
In frequency domain experiment, background segmentation is firstly undertaken in order to eliminate the impact of background on results. Initially, an image of the background (tensile belt without marker points) is captured, from which the standard deviation values (denoted as σ) of the image can be calculated and saved in a database by determining the R, G and B values of all of the pixels in the background image. Then the R, G and B values of all the pixels in the marker point image are found and compared with those of the background image. If the difference between the two sets of R, G and B values are higher than 2σ, then the pixel will be regarded as marker point, else background. The marker point image after segmentation is finally inputted into the program to calculate the power spectrum.
The necessity of the background segmentation procedure is indicated in Fig.4: Figure 4a and 4c show the power spectrum images with different weights before background segmentation. The images in Fig. 4b and 4d represent the result of the FFT algorithm after the background removal procedure (using the 2σ difference between the two RGB values of foreground and background color). This experiment clearly demonstrates the improvement of the spectrum image quality after the background segmentation procedure, and the value of power spectrum is calculated within a red circle region which has been proven to have the largest change with grid deformation [32].

Statistical analysis
To validate the effectiveness of the proposed image processing methods, initially, the bending energy values and power spectrum values corresponding to different weights are recorded as illustrated in Table 1 and 2. Specifically, to decrease the spectrum spreading effect, the area is chosen as the red circles with radius of 3 unit pixels in Fig. 4b and 4d, where the center point is located at the maximum spectrum point with 0g weight, and the final frequency feature is the sum of the power spectrum values in this area. In order to reduce unexpected error, both of the experiments have been undertaken for ten times to calculate the average of experimental values as the final result.
The correlations between weight and bending energy, weight and power spectrum can be obtained through the scatter plot. According to the linear relation from the experiments, we can calculate the value of mass. Finally, the Bland-Altman approach [33] is used to plot the agreement between the calculated and measured mass.

Results and Discussion
The correlations between weight and bending energy, weight and power spectrum are shown in Fig. 5 and Fig. 6, respectively. Figure 5 presents that the TPS minimum bending energy value increases as the mass of weights on tensile belt increases, while Fig. 6 shows that the power spectrum value decreases as the weights on tensile belt increases. The experimental results are consistent with our expected conjecture. Accordingly, the following correlation formulas, derived based on the collected data on mass and computed bending energy and power spectrum by image processing methods were obtained: Bending energy=0.0003Mass+0.0486 Power spectrum=-3.5565Mass+1168.7 The results of comparison between predicted (spatial domain and frequency domain) and actual mass of weights are shown in Fig. 7 and 8, respectively. The coefficient of determination (R 2 ) for spatial domain method and frequency domain method were 0.9833, and 0.9698. The R 2 values can be interpreted as the proportion of the variance in the image processing estimated attributable to the variance in the actual measurements. The higher the R 2 values, the closer the image processing results are to the actual results. The proposed image processing methods yielded above 96% accuracy in estimating weights mass. Finally, the Bland-Altman plot is introduced to access the agreement of the image processing methods and the traditional method. In these figures, the dash lines indicate the 95% limits of agreement while the solid lines show the average difference. The Bland-Altman plot for comparison of mass computed by spatial domain method and measured with balance is shown in Fig. 9. The result shows agreement with the 95% limits of agreement being −222.888 to +114.071 g and the average bias is -54.408g. The Bland-Altman plot for frequency domain method versus the traditional method is shown in Fig. 10, showing that excellent agreement with the 95% limits of agreement being −33.715 to +33.701 g and the average bias is -0.007g.

Conclusions
In this paper, two image processing techniques for estimating the mass of an object were presented and discussed. In the spatial domain and frequency domain experiments, thin plate spline algorithm and FFT algorithm were adopted to analyze the images with grid points respectively. The spatial domain experiment uses the manual acquisition of experimental raw data, while the frequency domain experiment uses the automatic acquisition of experimental raw data. The experimental results showed that the mass of an object is linearly proportional to the minimum bending energy value of TPS while is inversely proportional to the sum of the spectral values of the central region of the FFT spectrogram. In summary, the results obtained in two different ways are ultimately consistent with the theoretical predictions, and mass is able to be measured efficiently by image processing methods.