In this section, the details of the leaf image dataset and suggested image processing methods with a novel LBHPG feature extraction technique are discussed in suitable subheadings.
2.3 Research Framework
The framework of the proposed leaf species identification system is shown in Fig. 2.1. The proposed work is divided into five phases: image acquisition; image preprocessing; image segmentation; feature extraction; and classification. In the image acquisition phase, various leaf image datasets are developed. In the preprocessing phase, the query leaf image is resized to have a standard and similar size for all. In the image segmentation phase, leaf area as a foreground area is segmented from the background using a fast-adaptive fuzzy C-means clustering (FAFCM) technique. Feature extraction consists of extracting the different shape and texture features from leaves using simple and morphological features using the LBP, HOG, and LBHPG methods.
These features become the input vector to KNN, PNN, and SVM in the classification stage that classifies the leaf species based on the extracted features.
The steps performed are outlined as follows:
II) Image Pre-processing includes a) Image Transformation, b) Image Segmentation, and c) Image Binarization. III) Feature extraction-a) Geometric and Morphological based Feature Extraction b) LBP-Texture based feature extraction c) HOG-Shaped-based feature extraction d) LBHPG-Feature optimization-based feature extraction IV) Classification-a) K-Nearest Neighbour (KNN), b) Probabilistic Neural Networks (PNN), c) Multiclass Support Vector Machine (SVM).
2.3.1 Processing Steps
The description of the process and steps involved are:
a. Image Acquisition
The purpose of this step is to obtain the image of a whole plant leaf from a defined image database so that analysis towards classification can be performed. The public as well as a self-developed image are used for the implementation of the algorithm. Plant leaf image databases of soybean, potato, and tomato were collected from plant village image databases [7]. While creating a self-developed soybean image database, the diseased soybean leaves are placed on a white background to remove background complexity. Then the plant leaf image is captured using a high-resolution mobile camera. Figure 2.2 shows acquired images of soybean, potato, and tomato healthy leaf samples from a defined image database.
b. Image Pre-Processing
Image preprocessing consists of Image Transformation, Image Segmentation, and Image Binarization.
c. Image Transformation
In image transformation, the input RGB leaf image is first converted into HSV image using the Equation (1) shown in Figure 2.3.
d. Leaf Image Segmentation
To extract leaf features, the input image used consists of background, which may affect the feature characteristics. The segmentation process consists of RGB to HSV color space conversion [8] or better contrast-based pixel identification for foreground and background. The background of the leaf is separated by using a threshold-based masking technique as shown in Fig. 2.4. The Fast Adaptive Fuzzy C-Means clustering (FAFCM) technique [9] is applied, which partitions the leaf object into 2 clusters, foreground, and background, as shown in Fig. 2.5. The leaf segmentation is achieved by a threshold algorithm with hue, saturation, and value features as follows:
The pixel value is within the range of 0 to 255, which is first converted into 0 to 1 using 255 as the dividing factor. The maximum contrast information Cmax and the minimum contrast information Cmin are obtained by selecting the minimum amongst red, green, and blue values. The range ∆ is the difference between Cmax and Cmin. The ratio of ∆ and Cmax is calculated to get the value of sensitivity. Based on these sensitivity values, the foreground and background pixel identification is done.
The pixel belongs to foreground if S > 0.2; (S- sensitivity),
The image data type is set to unsigned integer 8 bit to get values within 0 to 255.
I1(scan (H > s)) = (0….255)- Make foreground pixel on
I2(scan (S > s)) = (0….255)- Make foreground pixel on
I3(scan (V > s)) = (0….255)- Make foreground pixel on
Here, the (I) color image is shown in Fig. 2.5, where the foreground corresponds to the leaf region, and black pixels correspond to the background.
e. Fast Adaptive Fuzzy C-Means clustering (FAFCM)
The basic fuzzy c-means clustering (FCM) algorithm is sensitive to noise. The robustness of FCM is improved by using spatial information for image segmentation [9]. This improvement gives better segmentation but also increases computational complexity due to spatial information while calculating the distance between pixels within local spatial neighbours and clustering centroids. This issue can be solved by using an improved FCM algorithm based on morphological reconstruction and membership filtering (FRFCM) that is significantly faster and more robust than the FCM proposed in this research.
The modified objective function of these algorithms is given as follows:
$${J}_{m}= \sum _{i=1}^{N}\sum _{k=1}^{c}{u}_{ki}^{m} {‖{x}_{i}- {v}_{k}‖}^{2}+ \sum _{i=1}^{N}\sum _{k=1}^{c}{G}_{ki}$$
(10)
Where, f = fx1; x2; x Ng represents a grayscale image, xi is the grey value of the ith pixel, \({v}_{k}\) represents the prototype value of the kth cluster, and uki denotes the fuzzy membership value of the ith pixel with respect to cluster k. U = [\({u}_{ki}^{}\)] cN represents the membership partition matrix. N is the total number of pixels in the image f, and c is the number of clusters. The parameter m is a weighting exponent on each fuzzy membership that determines the amount of fuzziness of the resulting classification. The fuzzy factor Gki is used to control the influence of neighboring pixels on the central pixel. The Fast Adaptive Fuzzy C-Means clustering (FAFCM) technique is applied, which partitions the leaf object into 2 clusters, foreground, and background, as shown in Fig. 2.5.
f. Leaf Image Binarization
After that, the segmented image is converted into a binary image by using the global thresholding technique shown in Figure 2.6.
g. Feature extraction and description
Feature extraction refers to taking measurements, geometric or otherwise, of possibly segmented, meaningful leaf object regions in the image. The features vector describes the characteristics of the plant leaf captured in the images. In this research, a total of 9 features were extracted, out of which 6 for geometric and morphological shape-based features [10] and 3 for Local Binary Pattern (LBP) with Histogram of Oriented Gradients (HOG) formed a new developed Local Binary Histogram Pattern of Gradient (LBHPG) feature.
h. Local Binary Histogram Pattern of Gradient (LBHPG) feature
The LBHPG method is a combination of LBP and HOG. In this research, it was determined that when LBP is combined with the Histogram of oriented gradients (HOG) descriptor, the performance of detection is considerably improved on defined plant leaf image datasets.
i. Local Binary Pattern (LBP)
The texture feature extraction is done by using the LBP algorithm [11]. LBP features are extracted. The radius parameter is set to four pixels, and the binary number generator is set to eight pixels. The LBP-based feature extraction consists of an encoding mechanism in which at a time, a window of particular pixels, such as a 3 x 3 window, is considered for the process. The value of each pixel is subtracted from the center pixel value to get a positive and negative difference. The positive difference is considered as 1 and the negative as 0, along with the exact 0 as 0. After obtaining such 0 and 1 values, all values are collected in a clockwise manner to generate a binary number. The corresponding decimal value is obtained from this binary number to label the pixel. These numbers are called LBP codes. The example of the LBP operator is illustrated as shown in Fig. 2.7, and the results of LBP are shown in Fig. 2.8.
The LBP feature value for a pixel at (ip, ic) is calculated as,
$$LBP P, R(xc,yc)={\sum }_{P=0}^{P-1}s\left(ip-ic\right){2}^{P}$$
(11)
Where ic and ip denote the grey level of pixel at the center of the window selected and the count of pixels in the radial region with radius R. The functional value of s (x) for binarization is calculated as,
$$s\left(x\right)=\left\{\begin{array}{c}1, if x\ge 0 \\ 0, if x<0\end{array}\right.$$
(12)
$${\left(LBP\right)}^{Ri }P,R=\text{min}\{\text{ROR}LBPP,{R}^{i}|,i=\text{0,1},\dots .,P-1\}$$
13
The P number of pixels provides 2P values in 2P patterns using the operator LBP (P, R), given by,
LBP algorithm:
- Select a window of 16x16 pixels.
- Perform the subtraction process of pixels by selecting 8 pixels from the window.
- Perform positive and negative values based on 0 and 1 labelling.
- Collect 8 labelled values to generate a binary number.
- Convert binary numbers to decimal values to label the pixels.
j. Histogram of oriented gradients (HOG)
The occurrence of gradient orientations is counted to obtain HOG features. HOG feature extraction has a variety of applications in image processing when applications such as object recognition are developed [12]. The process for computing HOG is given stepwise in the algorithm given below.
HOG Algorithm
Step 1. Normalize the leaf image for gamma and color.
Step 2. Compute gradients.
$$gp=g\left(x,y\right)=\sqrt{{\varDelta x}^{2}+{\varDelta y}^{2}}$$
(14)
where x and y are the locations of pixels and \(\varDelta\)is the vector gradient.
The gradient direction is given by:
$${\theta }\text{p}={\theta }(\text{x},\text{y})=arctan\frac{\varDelta x}{\varDelta y}$$
(15)
Step 3. Use weighted voting to build spatial and orientation cells.
Step 4. Perform contrast normalization for overlapping spatial blocks.
Step 5. Use histogram gradients to assemble the features’ structure.
k. Local Binary Histogram Pattern of Gradient (LBHPG)
obtained the LBHPG features from combining the LBP texture feature with the HOG shape-based feature image [12] shown in Fig. 2.9.
LBHPG algorithm:
Step 1: Convert the RGB input image to HSV color space.
Step 2: Perform threshold segmentation to discard the background pixel information.
Step 3: Apply FAFCM for correct foreground segmentation.
Step 4: Extract geometrical features such as area, perimeter, minor and major axis length, convex area, axial ratio.
Step 5: Apply LBP to get binary texture features and obtain an LBP image.
Step 6: Extract HOG features from LBP image to obtain LBHPG features
l. Classification
In the classification step, 9 orthogonalize features are concatenated into a feature vector, which is then classified. The feature vector is used as input to KNN, PNN and SVM classifier to classify soybean, tomato, and potato plant leaf images.
m. Performance Analysis
The performance of the proposed LBHPG (leaf recognition model) is measured by a confusion matrix. The confusion matrix is used to measure true positive (TP), true negative (TN), false positive (FP), false negative (FN), sensitivity, specificity, precision, recall, and accuracy [13]. The performance is also analyzed using receiver operating characteristics (ROC) [14].