This section covers the working of proposed model, algorithm, working, dataset and comparison parameters.
3.1 Citrus fruit disease dataset
Figure 1 shows the samples of images of citrus fruit. As target objects, healthy fruit and fruit exhibiting five kinds of typical blemishes like HLB, black-spot, melanose, canker, and scab were extracted.
Samples of each peel state are represented by Fig. 1. (Fig. 1A) represents the healthy peel condition of the citrus sample. (Fig. 1B)HLB (CandidatusLiberibacterasiaticus) decreases the size of citrus fruits and does not color the fruit properly i.e. it remains green and with a bent central core, the fruit is skewed, inducing fruit deformity and cracking (Fig. 1B) [23]. There may be botched seeds in the affected fruit and they have a salty, bitter taste. As several fruits fall prematurely from infected trees, HLB also decreases citrus yields. Black spots (Fig. 1C) in fruits with a diameter of 0.12 to 0.4 are small, circular, and dangerous spots [21]. The presence of Citrus Black Spot symptoms can differ and cause aesthetic abnormalities on the fruit crust. The usual lesion, identified as a hard spot, begins with tiny red brick spots with black borders that grow in size and produce tissue necrosis at the center of the lesion [24].
Melanose (Fig. 1D) is precipitated by Diaporthecitri and is distinguished by dispersed raised blotches of brown to black color. The disease produces a gradual scar in the fruit which is barely capable of affecting the total yield of fruit production but causes noticeable blemishes which reduce the viability of the fruit destined for the producing market [25]. The crop fungal infections that are commonly present in different regions producing citrus cultivars are Citrus scab (Fig. 1E). Generally, the scab incidence seems to be more severe in-plane regions with frequent wetting compared to tropical areas. Scab damages can be identified less than a week after getting an infection of the fruit [26]. The pathogens often occur with a gritty and irregular appearance as tiny dots. The fruit spot diameter of Canker (Fig. 1F) is around 1–10 mm and is covered by water-dipped and yellow curve-like blemishes [27]. Table 2 represents the number of the image samples for each type of disease and its proportion available in the dataset.
Disease Type
|
Number
|
Proportion
|
Table 2
Count and Proportion of Citrus Samples
Canker
|
370
|
14.4
|
Scab
|
409
|
16
|
Melanose
|
280
|
10.9
|
Greening
|
440
|
17.2
|
Black Spot
|
390
|
15.2
|
Healthy
|
667
|
26
|
3.2 Proposed PFDI Model
The proposed PFDI model is based on Context data fusion with Faster-CNN in edge computing environment.
3.2.1 Edge Computing Platform
Encouraging the remarkable resource consumption and delay in the processing time of the complex deep learning mechanism there is a much need of optimized mechanism for many applications. To detect or classify diseases present in citrus fruits in multi-classification manner, it's essential to advance the existing deep learning framework employed in the automatic disease detection system. All image samples collected from the various online platforms, while other services such as pre-processing and computations in the edge nodes make use of different clusters [28].
Figure 2 shows the working of Edge computing architecture of proposed model. The complete framework consists of four major modules, each corresponding to specific tasks. First module is consisting of collector nodes. Web services or edge computing are deployed second module, third module consist of prediction process and model pruning with quantization and user applications are deployed in module 4. The third module serves as a bridge between platform-based local and remote functions. The data collection is carried out locally from public domain, however after it has been set off, edge services are incorporated for processing and computational task [29].
Different clusters were created analogously to provide broader coverage of potential application scenarios. The platform's applications and services are accessible to users through module four, which is the last module. The layer is therefore in charge of giving the user full access to all services offered by the edge computing platform. This module offers access to the computer vision application via virtual network computing, evaluating and analyzing the outcome. The public dataset consist of citrus fruit diseases of different types is send to the local collector node. The detection layer contains a deep CNN model with magnitude pruning and quantization, an internet connection is established. The remote access of deep model output is provided by the visualization layer to [30]. Comparing the edge learning servers to the most advanced cloud computing architecture, the edge learning servers reduce the workload of the network infrastructure. The image samples can be pre-processed to eliminate some challenges due to low contrast, such as light effects and flickering, etc. The preprocessing phase serves a critical role as minimal disparity images decrease the lesion segmentation's accuracy in the field of image processing [31].
We need to normalize to the equivalent image dimension in order to standardize the training data because all of the images in the training data are sparse and have different dimension sizes. Hence, the size of 256 X 256 has been used for all images in the training set. The deep learning based technique can also be employed by the edge learning servers for data augmentation to obtain new data without any labeling costs. As well, the data augmentation would be used to enhance the capabilities of the designed Faster-CNN model [32]. Therefore, 8 standard were included to supplement our training set and test set i.e. horizontal flip, vertical flip, brightness, rescaling, shear, zca-whitening, rotation, height, and width shift [33]. Initially, fruit samples collected were 2556 including six classes which reached to 20,448 after applying the eight function of data augmentation. Table 3 shows the samples of citrus fruits obtained after performing different data augmentation operations on the dataset.
As the edge servers are near to the client devices, the communication latency between the client devices and the edge servers is substantially lower than that of the cloud server. Model used for training. To offer end users consistent service, the cloud could be installed on edge servers, and image data can be continually uploaded to the cloud in order to modify the system. The classification process of citrus fruit diseases is briefly explained in Fig. 3. The major steps are (a) image pre-processing, (b) data augmentation (c) Model implementation and training (d) Pruning (e) Quantization (d) Performance evaluation. As seen in Fig. 3, each phase is made up of a series of steps. The detail of each process is given below.
3.2.2 Magnitude based Pruning with Polynomial Decay Based Sparsity on Faster-CNN Model
By using deep learning model have a huge number of trainable weights, where learning can indeed be obtained. Contribution of all the weights and feature vectors to model performance is not equivalent. In this paper proposed model is pruned using magnitude-based pruning which removes insignificant weights after each epoch. This approach compares the exact size of the weight with a certain threshold value δ. Input vector x and weight vector w within the neuron is multiplied [34]. If the weights in the vector are set to zero, the outcome will always be zero that defines in Eq. 1. This, in effect, ensures that the neuron no longer contributes to model performance. Table 4 represents the configuration of the proposed pruned sequential Convolutional neural network model.
$$\left({w}_{t}\right)=\{{w}_{t}: if\left|{w}_{t}\right|>\delta 0 : if\left|{w}_{t}\right|<\delta$$
1
Layer (type) | Output Shape | Param # |
---|
Table 4
Configuration of Proposed Pruned CNN Model
Prune._.low._.magnitude._.conv2d | (None, 256, 256, 32) | 1762 |
Prune_low_magnitude_activation | (None, 256, 256, 32) | 1 |
Prune._.low._.magnitude._.conv.2d._1 | (None, 256, 256, 32) | 36930 |
Prune._.low._.magnitude._.activation | (None, 256, 256, 32) | 1 |
Prune._.low._.magnitude._.max _ pool | (None, 256, 256, 32) | 1 |
Prune._.low._.magnitude._.dropout | (None, 256, 256, 32) | 1 |
Prune._.low._.magnitude._conv2d_2 | (None, 256, 256, 32) | 73794 |
Prune._.low._.magnitude._.activation | (None, 256, 256, 32) | 1 |
Prune._.low._.magnitude._.con2d._.3 | (None, 256, 256, 32) | 73794 |
Prune._.low._.magnitude._.activation | (None, 256, 256, 32) | 12 |
Prune._.low_.magnitude._.max._.pool | (None, 256, 256, 32) | 1 |
Prune._.low_magnitude_dropout | (None, 256, 256, 32) | 1 |
Prune_low_magnitude_flatten | (None,28224) | 1 |
Prune_low_magnitude_dense | (None,512) | 28901890 |
Prune_low_magnitude_activation | (None,512) | 1 |
Prune_low_magnitude_dropout | (None,512) | 1 |
Prune_low_magnitude_dense_1 | (None,5) | 5127 |
Prune_low_magnitude_activation | (None,5) | 1 |
At every step, all the connections of the network are iteratively pruned by setting up the levels of step sparsity\(L\) using polynomial decay function.Step sparsity\(L\) level is represented by the Eq. (2)
$$L={s}_{t}+\left({s}_{i}-{s}_{t}\right)\left(1-\frac{j-i}{i-e}\right)$$
2
Where targeted sparsity level is represented by\({ s}_{t}\); initial sparsity level is\({s}_{i}\), initial iteration is i, e is the end iteration, and the current iteration is represented by j. The threshold of significant connections is calculated using the sparsity level step. The threshold value is the total weights multiplied by the value at the position step sparsity level. When the total absolute values every weight filters is below the threshold limit then all the mask values belonging to its weight are adjusted to 0 as shown in Eq. 2. Then the dot product sets the pruned weights as zero. Using polynomial decay-based sparsity, more or less sparsity can be used with increasing or decreasing speed, as training progresses [35]. Initially, we set the model to be 50–80% sparse, increasingly getting sparser to eventually 80% and 90%. We begin at 0 and end at end step. Finally, the prune_low_magnitude functionality which generates the prunable model will be executed from our initial baseline Faster-CNN model and the defined pruning_params. The requirements to digitize the system, some numbers are allocated to various pieces of hardware. The requirements the maximum number is offered with good resources.
3.2.3 Edge Learning Platform
Figure 4 shows the edge learning frame work used by the proposed technique.
For every six classes including scab, HLB, melanose, black spot, canker, and healthy samples of citrus fruits, the training, validation, and test sample comprise of randomized input samples data with 8:2 ratios. For training the model one set s used as training and validation another as the test set to evaluate the model training. Furthermore, an optimizer (SGD) controls the gradient steps for every dimension of the loss feature with 0.9 momentum magnitude. The model training is done on the image samples using stochastic gradient descent having 0.0001 as the learning rate, batch dimension is 64, and epoch range is 20. Furthermore, output layer soft-max having categorical cross-entropy loss is used [36].
3.2.4 Quantization Process
Quantization lowers the model's representation that enables faster processing and smaller memory space. By ensuring that all weights and activations are quantized latency, processing, and power consumption can be boosted [37]. Quantization may be done during training that is called pre-quantization and post-quantization is done after training. In pre-quantization, the fixed point training approach is used for the direct training of the floating-point model. Whereas, Post quantization is used for the conversion of fixed-point representation from a pre-trained floating-point model and inferences can be taken from the fixed-point computation. In this paper, post-training quantization was adopted after pruning the proposed model. Fixed-point weights quantization has been applied that compressed the weights matrix from 32-bit to 8-bit floating-point values. Furthermore, Flat Buffer protocols are used as the basis for this transformation, which helps in bypassing a lot of the traditional expensive file parsing and un-parsing which leads to slower execution.
3.2.5 Context Data Fusion
Multiple data sources are combined through the procedure of "data fusion," which results in statistical information which is more reliable, precise, and beneficial than the whole the data supplied by any one source alone. In the proposed model the data a combination of early and late data fusion is applied. The classification data from the two components, colour as well as NIR visual imagery, is combined in late fusion [38]. Each multi-model M generates initiatives for the (NM, R) terrain. Such region key points are coupled to create a singular set of (Ns*Pr = M*NM, Pr), region propositions, which combines the two modules. The prth proposed terrain of the Mth, model type is then set a score (ScoreM, Pr). Eq. 3 shows the score model formulation.
$$\text{S}\text{c}\text{o}\text{r}\text{e}\text{M}=\sum { }_{M=1}^{NM} (\text{S}\text{c}\text{o}\text{r}\text{e}\text{M}, \text{P}\text{r})$$
3
Similar in the early fusion data process the input layer of faster-CNN has number of Channels 4 (one NIR and three RGB).
3.2.6 Transfer Learning
It takes a lot of time and materials to train Faster-CNNs, particularly for networks with many layers. We employed two transfer-learning strategies known as "fixed feature extraction process" and "fine-tuning process" to prevent having to retrain a whole network on citrus fruit. The simpler option is a fixed feature extraction method that further utilizes a channel that has been received training on a previously untrained category or entity to identify it or categories it. To identify fruits in this instance, we utilized weight that was received training and managed to learn on citrus fruits. The trained weights of the network that use the same architectural features are transferred directly to the main channel we just want prepare with updated information through fine tuning. In order to prepare the network for the detection of citrus fruits, that have fewer training sets, we utilized the network's weights that had been trained on fruit including its base [37].