Application of knowledge distillation method with dynamic adjustment of temperature parameters in pest classification

doi:10.21203/rs.3.rs-4691672/v1

Download PDF

Research Article

Application of knowledge distillation method with dynamic adjustment of temperature parameters in pest classification

https://doi.org/10.21203/rs.3.rs-4691672/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

In recent years, the output of China's four major crops has declined due to pests and diseases. This situation poses a serious challenge to food security. Therefore, timely detection and prevention of diseases is essential. First, we use data enhancement techniques to augment the data to improve the generalization ability of the model. Secondly, to reduce the model parameters and facilitate the deployment at the terminal, we use the knowledge distillation method. Finally, a method of dynamically adjusting the parameter T according to the loss value (DYTKD) is proposed to improve the performance of the model further. The experiment shows that knowledge distillation can reduce the number of parameters while making the accuracy of the student model as close as possible to the teacher model 98.94%. Meanwhile, data augmentation can also improve the accuracy of the model by 6.83%. Compared with the basic knowledge distillation method, the accuracy of DYTKD was increased by 1.3% without changing the student network and other parameters, and the accuracy of pest identification and classification was effectively improved. Among 1342 pest pictures, 1221 were correctly identified and accurately classified. Our codes are available at https://github.com/wln130221/DYTKD.

Data augmentation

knowledge distillation

Dynamic temperature

Image recognition and classification

In agricultural production, pest control is essential to ensure stable crop yields. In recent years, especially in the planting process of China's four major staple grains - rice, wheat, corn, and potato, the frequent occurrence of major pests and diseases such as wheat scab and stripe rust, rice sheath blight, and corn spot disease has brought great challenges to agricultural production. These diseases not only directly threaten the safety of grain and oil production, but also may lead to a significant decline in crop yield and serious damage to quality, which in turn hurts the overall development of the agricultural industry. It is predicted that by 2024, the pest and disease situation of these food crops will be even more severe. It is estimated that the area of major pests and diseases of wheat, rice, corn, potatoes, and other food crops will reach a staggering 2.04 billion mu nationwide, an increase of 15% over the average level in 2023. This means that more than 70% of food crop-producing areas will be threatened by these pests and diseases, and agricultural production is facing unprecedented risks and challenges. To cope with this critical situation and improve agricultural production efficiency, timely detection and early prevention of plant diseases have become crucial.

With the rapid development of artificial intelligence and computer technology, deep learning algorithms have shown strong potential in many fields, such as agriculture and forestry, medicine, finance, and other fields. Deep learning has become a popular research direction and has achieved excellent results in the fields of natural language processing (NLP), image processing, and object detection. In the field of pest identification, we try to use deep learning algorithms and computer vision technology to realize the automatic identification and classification of pest and disease images, which greatly improves the accuracy and efficiency of pest and disease identification.

Among the many deep learning models, convolutional neural networks (CNNs) have performed well in the field of pest and disease identification due to their unique advantages. At present, a variety of CNN-based pest identification methods have been proposed, such as AlexNet, VGGNet, ResNet [1–3], etc., and these models have achieved excellent results in the field of pest and disease identification. For example, Cheng et al.[4]achieved remarkable results with the application of improved AlexNet in the classification of strawberry pests and diseases, with an accuracy rate of 98%. At the same time, the work of Alatawi et al.[5]also revealed the great potential of the VGGNet model in plant disease identification.

With the deepening of research, it is found that the expressive ability of the model can be further improved by designing deeper neural network structures, to extract more discriminating depth features. However, this also poses a problem: a deeper network structure means more model parameters and higher computational complexity, as well as higher requirements for storage devices and computing resources. This makes existing deep learning models face certain challenges in practical applications, especially when deployed on end-user mobile devices with limited resources.

To solve this problem, researchers have begun to explore how to reduce the complexity and computational cost of the model while ensuring the accuracy of the model[6–7]. Among them, the proposal of a knowledge distillation (KD) method[8] has attracted extensive attention. This approach guides the learning process of a smaller, less parameter-rich model (the student model) by utilizing a larger, parameter-rich model (the teacher model). During the training process, the teacher model "distills" the knowledge and feature information it has learned into the student model so that the student model has lower complexity and computational cost while maintaining high accuracy. To let the students better imitate the teacher, we also set the temperature parameter T to be dynamically adjusted according to the predicted value output by the student model and the loss value of the true label. This method can not only simplify the model structure on the premise of ensuring the accuracy of the model but also improve the feasibility and generalizability of the model in practical application. Therefore, the pest classification method based on knowledge distillation provides a new idea and method for solving practical problems in the identification of crop diseases and pests.

The rest of this article is organized as follows: First, we reviewed the work in Section 2. Then, in the third section, we will elaborate on the details of the ideas and models presented. We then present the experimental results and comparisons in Section 4 and finally summarize our work in Section 5.

2.1 The development process of crop pest and disease identification

In the early stages of crop pest identification, manual on-site inspection, measurement, and identification are mainly relied upon. This method is highly dependent on the experience and expertise of the observer, so it is highly subjective. Different experts may come to different conclusions on the same issue. In addition, manual identification is inefficient, time-consuming, and labor-intensive, making it difficult to cope with large-scale agricultural production needs.

To improve the efficiency and accuracy of identification, researchers have gradually introduced various detection instruments. For example, Li et al. [9]developed a citrus pest detector. However, these testing instruments still need to be operated manually, the degree of automation and intelligence is generally low, and the detection results are easily affected by external environmental factors (such as temperature, light, etc.), which limits their wide application.

In recent years, with the rapid development of image processing and pattern recognition technologies, experts and scholars have begun to apply these technologies to the identification of crop diseases and pests[10]. At this stage, the color, texture morphology, and other features extracted from the image, combined with recognition technologies such as linear classifiers, are used to classify and identify various crop diseases and pests, which promotes the development of agricultural informatization and precision.

Duarte-Carvajalino et al.[11] aimed to identify potato diseases and used the collected datasets to train machine learning algorithms such as multilayer perceptrons, convolutional neural networks (CNNs), support vector machines, and random forests. The results show that the performance of the convolutional neural network is slightly better than that of other machine learning algorithms. There are many commonly used CNNs, each with its unique advantages[12], such as VGG, ResNet, GoogleNet, and DenseNet. [13–16] provides detailed research and experimental results on the role of CNN in pest and disease identification, which provides an important reference for further research and application.

With the continuous progress of science and technology, more and more scholars at home and abroad have begun to use deep learning technology to identify crop diseases and pests.[17] Through automated and efficient image processing technology, target features can be accurately extracted from a large number of crop pest and disease images, thereby improving the accuracy and efficiency of identification. These technologies not only reduce the dependence on manual experience but also greatly improve the speed and reliability of pest and disease identification, providing strong support for agricultural production.

In general, crop pest and disease identification technology has undergone a significant development process from early manual detection to the use of image processing and pattern recognition technology, and then to the widely used deep learning technology. With the continuous maturity of deep learning technology, the identification of crop diseases and pests will become more intelligent and efficient, providing strong technical support for the modernization and precision of agriculture.

3.1 Data preprocessing

Data augmentation, as a key technology in the field of deep learning, is a method of extending a finite dataset by generating more equivalent data, aiming to increase the number and diversity of training samples, thereby improving the generalization ability and performance of the model. Li et al.[18] demonstrated through research that data augmentation significantly improves knowledge distillation performance on image classification and object detection tasks even though the teacher model lacks comprehensive information about augmented samples. With data augmentation, the over-reliance of the model on specific features can be reduced, thus avoiding overfitting. In deep learning, a network often has millions of parameters that require large amounts of data to train effectively. However, in some application scenarios, such as crop identification, the difficulty, and complexity of data collection are relatively high due to the influence of various factors on the growing environment of crops, and it is usually difficult to obtain enough data to support network training. Using raw data directly can lead to unbalanced sample classification and affect the training process of the model. Since the training goal is to improve the classification accuracy on the dataset, the category with a large sample size tends to dominate the training process due to its large cumulative training error, resulting in the algorithm biasing towards this category on unbalanced data. To improve the accuracy of the model and avoid overfitting the model, we can use data augmentation techniques to augment the sample[19]. In the application of crop identification, the growth environment of crops is affected by many factors, such as temperature and humidity, light, rainfall, soil, etc., and these objective conditions make data collection more difficult and complex. Therefore, data augmentation becomes an effective means to solve this problem.

Common data augmentation methods include:

Add noise: Introduce random noise into the input data to simulate real-world noise interference, making the model more robust to random changes in the data.

Modify the intensity and contrast of the image: Enhance the adaptability of the model by adjusting the brightness, contrast, and color of the image to generate different viewing angles.

Geometric transformation: including rotation, clipping, and translation, by changing the geometric properties of the image, more training samples are generated. [20–21]

For example, an example of a data augmentation method is shown in Fig. 1, showing the leaves of four crop diseases, and the original image undergoes a series of transformations, from left to right, after 45° rotation, adjusting the brightness of the image, Gaussian blurring, color enhancement, and saturation increase. These newly generated images are treated as different samples in the network, which improves the recognition accuracy of the model and effectively mitigates the overfitting phenomenon. Through these methods, the model can better learn and identify diseases in complex and variable crop growth environments, and improve the overall classification performance and robustness.

3.2 knowledge distillation

In 2015, Hinton et al. first proposed the Knowledge Distillation technique[6] to solve the problem of increasing parameters as models become more complex. Knowledge distillation involves training the student network to mimic the output of the teacher's network and mimic its internal characteristics or decision-making processes. The purpose of this technique is to enhance the performance of smaller models while making them behave close to larger models while reducing computational costs and memory requirements. The knowledge distillation system consists of three key components: knowledge, refining algorithms, and teacher-student architecture. Figure 2 illustrates the flowchart of knowledge distillation, with a "teacher-student" structure at its core.

There are three different forms of knowledge in knowledge distillation: response-based knowledge, feature-based knowledge, and relationship-based knowledge. In this paper, we will use the logits of the last output layer of the teacher model as teacher knowledge[22]. This method is known as response-based knowledge distillation, and its main idea is to directly mimic the final predictions of the teacher model. Response-based knowledge distillation is a simple and effective model compression method that has been widely used in different tasks and applications.

The Logits of the last output of the teacher model are used as the basis for comparing the output of the student model and the teacher model. By testing Logits, the model can measure the certainty or confidence of its predictions before applying the Softmax function to obtain the final probability. Specifically, the probability of the Softmax function is calculated as follows:

$$\:\begin{array}{c}{p}_{i}=\frac{exp\left(\frac{zi}{T}\right)}{\sum\:_{j}exp\left(\frac{zj}{T}\right)}\left(1\right)\end{array}$$

Here, zi is the logit of class i, and the importance of the hyperparameter T is introduced to control the importance of each soft target, where T represents the "temperature" of knowledge distillation. When T = 1, it corresponds to the normalized exponential function. As the temperature parameter T increases, the probability distribution of the Softmax function becomes smoother, conveying finer details about the relationship between the different classes in the teacher model.

3.3 Dynamically adjust the temperature

The soft tags described by Hinton et al. contain valuable dark knowledge from the teacher model. Thus, the loss function is extended to include the standard cross-entropy loss between the predictions of the student model and the true labels, as well as the difference between measuring the softening probability of the teacher's model (obtained by a higher temperature Softmax) and the corresponding predictions of the student model.

The loss value is used to measure the difference between the model prediction and the true label. The smaller the loss value, the more accurate the model prediction. At the same time, the T-parameter is an important hyperparameter in the model, which may affect the complexity, learning speed, or generalization ability of the model. Figure 3 is a flow chart of the network for training students. In the process of traditional knowledge distillation training, we use static T parameters, which cannot adjust the training state in time. [23] It has been shown that the traditional fixed temperature setting may not be sufficient to capture the full knowledge of the teacher model during distillation. So this paper proposes a method to dynamically adjust the temperature parameter T according to the value of the student's predicted value and the cross-entropy loss of the real label, to expect the model performance to achieve better training results.

To implement this method, we only need to complete the following steps in the training process: first, set an initial parameter value T, secondly, calculate the cross-entropy loss of the prediction of the student model and the real label, and then set a threshold, and when the loss value is greater than the threshold, increase the parameter T appropriately; When the loss value is less than the threshold, the parameter T is lowered appropriately, and finally, we recalculate the distillation loss value with the T parameter. The combined loss function for distillation loss and student loss is determined as follows:

$$\:\begin{array}{c}L\left(x;W\right)=\alpha\:\times\:CE\left(y,\sigma\:\left({z}_{s};T=1\right)\right)+\left(1-\alpha\:\right)\times\:KL\left(\sigma\:\left({z}_{t};T=\tau\:\right),\sigma\:\left({z}_{s},T=\tau\:\right)\right)\left(2\right)\end{array}$$

In this case, x represents the input data, W is the set of weight parameters for the student model, and y is the corresponding true class label. We introduce cross-entropy loss (CE) as part of supervised learning, which aims to optimize the accuracy of the student model for direct label prediction. Meanwhile, using a Softmax function σ regulated by a "temperature" parameter T, we transform the logits (Zt) of the teacher model into a smoother probability distribution, a process called "softening" the objective. The student model's logits (Zs) try to match these softened targets, which goes beyond simple label prediction to learn the teacher model's internal understanding and representation of the data.

To further improve the performance of the student model, we design a comprehensive loss function to minimize the comprehensive loss function, which not only encourages the student model to learn to classify correctly, but also encourages it to learn the deep feature representation and generalization ability of the teacher model when dealing with complex data.

4.1 Dataset and preprocessing

The Rice Leaf Disease Images dataset was used in the experiment, including Bacterial blight, Blast, Brown spot, and Tungro. There were 4700 images in JPEG format. The size is 224x224 pixels. Table 1 shows our experimental condition configuration.

To further enhance the generalization ability of the model, reduce the risk of overfitting, and improve the robustness of the model in the face of complex and variable environments, we introduce a data augmentation strategy. This strategy includes not only basic brightness and contrast adjustments, but also advanced techniques such as noise addition and image rotation to simulate real-world disturbances such as illumination changes, shooting Angle differences, and natural noise. Through these measures, the size of the original dataset was successfully doubled to 9400 images, and each image was carefully scaled and normalized, which laid a solid foundation for the subsequent training process.

Table 1

Experimental Configuration
Experimental Environment	Value
Programming language Deep learning framework GPU GPU acceleration tool	Python 3.8 PyTorch 1.11.0 RTX 3080 CUDA:11.3

To quantitatively evaluate the actual effect of data augmentation strategies, we design a series of rigorous comparison experiments. While keeping the key elements of the network architecture (ResNet34 as the teacher model and ResNet18 as the student model for knowledge distillation) and the optimization algorithm unchanged, we use the original dataset and the enhanced dataset for training respectively. The experimental results show that after using the enhanced dataset, the recognition accuracy of the model is significantly improved, from 90.98–97.81%, with an increase of 6.83 percentage points. Figure 4 visually shows this significant performance improvement, which further confirms the excellent contribution of data augmentation in improving the robustness and accuracy of the model.

In addition, we use more detailed evaluation metrics, including accuracy, precision, and recall, to fully understand the performance of the model.

$$\:\begin{array}{c}Accuracy=\frac{TP+TN}{TP+TN+FP+FN}\left(3\right)\end{array}$$

$$\:\begin{array}{c}\text{P}\text{r}\text{e}\text{c}\text{i}\text{s}\text{i}\text{o}\text{n}=\frac{TP}{TP+FP}\left(4\right)\end{array}$$

$$\:\begin{array}{c}Recall=\frac{TP}{TP+FN}\left(5\right)\end{array}$$

By comparing the TP (true examples), FP (false positive examples), TN (true negative examples), and FN (false negative examples) data of the four pest categories before and after data augmentation in Table 2, we found that the data augmentation strategy significantly reduced the misclassification (for example, the FP of Bacterial blight decreased from 87 to 4, and the FP of Bacterial blight decreased from 87 to 4. Tungro's TP increased from 154 to 307), which greatly improved the accuracy of the model.

Table 2

Performance comparison before and after data enhancement (after knowledge distillation)
Original data					Enhance data
Class	Bacterial blight	Blast	Brown spot	Tungro	Bacterial blight	Blast	Brown spot	Tungro
TP	546	306	212	154	359	237	296	307
FP	87	23	14	0	4	10	19	0
FN	30	54	40	0	25	3	4	1
TN	679	959	1076	1188	844	982	913	924
Accuracy(%)	91.28	94.26	95.98	100.0	97.65	98.94	98.13	99.92
Precision(%)	86.26	93.01	93.81	100.0	98.90	95.95	93.97	100.0
Recall(%)	94.79	85.00	84.13	100.0	93.49	98.75	98.67	99.68

4.2 Experimental Results

Taking the Resnet model as an example, we use the pre-trained Resnet34 as the teacher model on the rice dataset, and the accuracy can reach 98.94%. To further compress the model, we use knowledge distillation technology to pass the logits of the teacher output layer as knowledge to the lightweight student model Resnet18, and the accuracy is 97.81%. Compared with the Resnet18 model without distillation, the performance has been significantly improved. Figure 5(a) shows the comparison chart of teacher-student model accuracy.

Table 3

Comparison of accuracy, precision, and recall of DYTKD
Model	Accuracy	Precision	Recall	Parameters(M)
Teacher: ResNet34 Student: ResNet18	98.94% 93.83%	99.09 95.12	98.79 92.46	21.8 11.2
KD DYTKD	97.81% 99.11%	97.7 99.15	97.94 99.04	11.2 11.2

To verify the effectiveness of dynamically adjusting the parameter T, the module of dynamically adjusting T is introduced without changing the network structure, and other hyperparameters, Table 3, Table 4 and Fig. 5(b) are the comparison of the experimental results. The experimental results show that our proposed DYTKD method can fully let students learn the knowledge in the teacher network, and generalize well to the test data set. Not only the accuracy and precision are improved, but the number of true examples (TP) also increased from 367 to 382.

Figure 6(a) is the confusion matrix of the student model on the test set after knowledge distillation. It performs well in classifying these pests and diseases, but there is still room for improvement, especially in reducing misjudgment, among which there are 9 times that are Bacterial blight but predicted as Blast. Figure 6(b) is the confusion matrix of our proposed DYTKD method. Compared with the previous confusion matrix, the performance of the model has been further improved with higher accuracy and fewer misjudgments.

Table 4

Comparison of TP, FP, TN, and FN of DYTKD
teacher								student
Class		Bacterial blight		Blast		Brown spot	Tungro	Bacterial blight	Blast		Brown spot	Tungro
TP		384		234		294	307	382	180		288	306
FP		11		2		0	0	60	3		7	6
FN		0		6		6	1	2	60		12	2
TN		837		990		932	924	788	989		925	918
Accuracy(%)		99.11		99.35		99.51	99.92	94.97	94.89		98.46	99.35
Precision(%)		97.22		99.15		100.0	100.0	86.43	98.36		97.63	98.08
Recall(%)		100.0		97.50		98.0	99.68	99.48	75.0		96.0	99.35
KD								DYTKD
Class	Bacterial blight		Blast		Brown spot		Tungro	Bacterial blight		Blast	Brown spot	Tungro
TP	367		234		298		306	382		236	296	307
FP	6		9		12		0	7		3	1	0
FN	17		6		2		2	2		4	4	1
TN	842		983		920		924	841		989	931	924
Accuracy(%)	98.13		98.78		98.86		99.84	99.27		99.43	99.59	99.92
Precision(%)	98.39		96.30		96.13		100.0	98.20		98.74	99.66	100.0
Recall(%)	95.57		97.50		99.33		99.35	99.48		98.33	98.67	99.68

Table 5

Teachers and students share the same architectures. ∆represents the performance improvement over the KD.
Teacher(%) Student(%)	Resnet34 95.14 Resnet18 91.77	Resnet50 94.84 Resnet18 91.60	Resnet101 94.72 Resnet34 90.67	Resnet152 94.77 Resnet34 89.93
KD(%)	94.88	93.76	93.07	93.17
DYTKD(%)	95.65	94.93	93.90	93.96
△(%)	+ 0.77	+ 1.17	+ 0.83	+ 0.79

To further verify the effectiveness of the DYTKD method, we also carried out comparison tests of different networks and replaced the teacher-student network with different network structures, as shown in Table 5. The experimental results show that the performance of DYTKD is improved compared with KD in each time.

In this paper, the recognition and classification performance of crop pests and diseases is significantly improved by effectively transferring the knowledge of large models to small models and combining data augmentation techniques. Taking the rice dataset as an example, our experimental results show that the loss threshold is set to dynamically adjust the temperature parameter during the training process. When the student predicted and true label cross-entropy loss values are large, it means that the student's student ability is weak, and we appropriately adjust to the appropriate temperature. Compared with the static temperature parameter in the traditional knowledge distillation method, the performance of the model is improved. Through this method, it provides strong technical support for the accurate identification and classification of crop pests and diseases.

Author Contribution

Wang participated in the experiments, data analysis visualization and manuscript writing;Zhao gives final approval for what will be published.All authors reviewed the manuscript

Data Availability

The datasets generated during and/or analysed during the current study are available in the [GitHub] repository, [https://github.com/wln130221/DYTKD]

Krizhevsky A, Sutskever I, Hinton G E. Imagenet classification with deep convolutional neural networks[J]. Advances in neural information processing systems, 2012, 25.
Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[J]. arxiv preprint arxiv:1409.1556, 2014.
He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 770–778.
Dong C, Zhang Z, Yue J, et al. Classification of strawberry diseases and pests by improved AlexNet deep learning networks[C]//2021 13th International Conference on Advanced Computational Intelligence (ICACI). IEEE, 2021: 359–364.
Alatawi A A, Alomani S M, Alhawiti N I, et al. Plant disease detection using AI based vgg-16 model[J]. International Journal of Advanced Computer Science and Applications, 2022, 13(4).
Guan H, Fu C, Zhang G, et al. A lightweight model for efficient identification of plant diseases and pests based on deep learning[J]. Frontiers in Plant Science, 2023, 14: 1227011.
Xu Y, Gao Z, Zhai Y, et al. A CNNA-Based Lightweight Multi-Scale Tomato Pest and Disease Classification Method[J]. Sustainability, 2023, 15(11): 8813.
Hinton G, Vinyals O, Dean J. Distilling the knowledge in a neural network[J]. arxiv preprint arxiv:1503.02531, 2015.
Li Z, Hong T, Wang J, et al. Development and experiment of Panonychus citri infestation fast detector[J]. Transactions of the Chinese Society of Agricultural Engineering, 2014, 30(14): 49–56.
Guan Z X, Yao Q, Yang B J, et al. Application of digital image processing technology in recognizing the diseases, pests, and weeds from crops[J]. Scientia Agricultura Sinica, 2009, 42(7): 2349–2358.
Duarte-Carvajalino J M, Alzate D F, Ramirez A A, et al. Evaluating late blight severity in potato crops using unmanned aerial vehicles and machine learning algorithms[J]. Remote Sensing, 2018, 10(10): 1513.
Hasan R I, Yusuf S M, Alzubaidi L. Review of the state of the art of deep learning for plant diseases: A broad analysis and discussion[J]. Plants, 2020, 9(10): 1302.
Liu B, Zhang Y, He D J, et al. Identification of apple leaf diseases based on deep convolutional neural networks[J]. Symmetry, 2017, 10(1): 11.
**e X, Ma Y, Liu B, et al. A deep-learning-based real-time detector for grape leaf diseases using improved convolutional neural networks[J]. Frontiers in plant science, 2020, 11: 751.
Saleem M H, Khanchi S, Potgieter J, et al. Image-based plant disease identification by deep learning meta-architectures[J]. Plants, 2020, 9(11): 1451.
Prashanthi V, Srinivas K. Plant disease detection using Convolutional neural networks[J]. International Journal of Advanced Trends in Computer Science and Engineering, 2020, 9(3).
Sourav M S U, Wang H. Intelligent identification of jute pests based on transfer learning and deep convolutional neural networks[J]. Neural Processing Letters, 2023, 55(3): 2193–2210.
Li W, Shao S, Qiu Z, et al. Multi-perspective analysis on data augmentation in knowledge distillation[J]. Neurocomputing, 2024, 583: 127516.
Zhao S, Sun X, Gai L. Data enhancement and multi-feature learning model for pest classification[J]. Journal of Intelligent & Fuzzy Systems, 2023 (Preprint): 1–13.
Liu G, Nouaze J C, Touko Mbouembe P L, et al. YOLO-tomato: A robust algorithm for tomato detection based on YOLOv3[J]. Sensors, 2020, 20(7): 2145.
Wu H, Wiesner-Hanks T, Stewart E L, et al. Autonomous detection of plant disease symptoms directly from aerial imagery[J]. The plant phenome journal, 2019, 2(1): 1–9.
Mirzadeh S I, Farajtabar M, Li A, et al. Improved knowledge distillation via teacher assistant[C]//Proceedings of the AAAI conference on artificial intelligence. 2020, 34(04): 5191–5198.
Chi Z, Zheng T, Li H, et al. Normkd: Normalized logits for knowledge distillation[J]. arxiv preprint arxiv:2308.00520, 2023.

No competing interests reported.

Download PDF

Editorial decision: Revision requested
15 Jul, 2024
Reviews received at journal
15 Jul, 2024
Reviews received at journal
10 Jul, 2024
Reviewers agreed at journal
10 Jul, 2024
Reviewers agreed at journal
10 Jul, 2024
Reviewers invited by journal
10 Jul, 2024
Editor assigned by journal
10 Jul, 2024
Submission checks completed at journal
09 Jul, 2024
First submitted to journal
05 Jul, 2024

You are reading this latest preprint version

Application of knowledge distillation method with dynamic adjustment of temperature parameters in pest classification

Status:

Version 1

Abstract

Figures

1. Introduction

2. Related Work

2.1 The development process of crop pest and disease identification

3. Methods

3.1 Data preprocessing

3.2 knowledge distillation

3.3 Dynamically adjust the temperature

4. Experiment

4.1 Dataset and preprocessing

4.2 Experimental Results

5. Conclusions

Declarations

Author Contribution

Data Availability

References

Additional Declarations

Status:

Version 1