Apple leaf disease identification via improved CycleGAN and convolutional neural network

The identification of apple leaf diseases is crucial to reduce yield reduction and timely take disease control measures. Employing deep learning for apple leaf disease identification is challenging because of the limited availability of samples for supervised training and the serious class imbalance. Hence, this paper proposes an accurate deep learning-based pipeline to solve the problem of limited data sets on farms and reduce the partiality due to serious class imbalance. Firstly, an improved cycle-consistent adversarial networks (CycleGAN) is used to generate synthetic samples to improve the learning of data distribution and solve the problems of small data sets and class imbalance. Secondly, ResNet is trained as a baseline convolutional neural network classifier to classify apple leaf diseases. The experimental results show that ResNet has the highest recognition accuracy on the test set, reaching 97.78%, and the classification accuracy is significantly improved by the generated synthetic samples (+ 14.7%). In addition, the experiment results of t-distributed stochastic neighbor embedding (t-SNE) and visual Turing test visually confirmed that the images generated by improved CycleGAN have much better quality and are more convincing.


Introduction
Apple, as one of the main economic crops in China, is a kind of fruit that can be stored and sold all year round (Musacchi and Serra 2017). However, the disease of apple leaves caused significant production and economic losses, as well as the quality and quantity decline (Liu et al. 2018). Therefore, disease identification is crucial. Traditional disease identification and detection are mainly carried out in the field by experienced fruit farmers. However, this method has a time and space lag, coupled with the lack of professional skills learning and assistance, making it difficult for the majority of fruit farmers to achieve scientific control, modern control and efficient control (Xiaonan et al. 2020). Therefore, comprehensive research on apple leaf disease identification based on machine vision plays an important role in disease control and prevention.
With the development of computer vision and image recognition technology, image recognition based on deep learning has been widely applied in various detection fields [such as pedestrian detection (Zhou et al. 2019), face recognition (Deng et al. 2019) and biomedical image analysis (Zhou et al. 2019)] and has achieved great success in non-contact, real-time and low-cost. Compared to traditional pattern recognition, the appearance of convolutional neural network (CNN) greatly improves the accuracy of recognition, reduces the cost and improves the speed (Fuentes et al. 2017). In Kamal et al. 2019;Thenmozhi and Reddy 2019;Too et al. 2019;Zhang et al. 2019), various CNN-based models have been applied in the field of plant leaf disease identification, indicating that the model based on deep learning has become a popular method in plant leaf disease identification. However, sufficient training images are an important requirement for CNN-based models to improve generalization capabilities. If the training data set is not extensive enough, overfitting problems will occur (Srivastava et al. 2014). In the field of apple leaf disease identification, the collection of disease images is time-consuming and laborious, and there is a lack of disease images. Therefore, the lack of training images is the main factor hindering the further improvement of apple leaf disease identification accuracy.
The researchers attempted to use traditional data augmentation techniques such as flipping, clipping, rotation, Gaussian noise, translation and affine transformation to augment large of images. The essence of this method is to obtain a new image with the same semantics as the original image without increasing the diversity of the image, so it cannot essentially solve the diversity problem. Generative adversarial networks (GANs) (Mirza and Osindero 2014), as an unsupervised training method, can acquire information that is back-propagated from the discriminator, just like human learning image features. Therefore, many researchers have applied GAN to many fields, such as denoising (Schawinski et al. 2017), reconstruction ) and image synthesis (Huang et al. 2018). In this paper, a novel and robust data augmentation method is proposed to enhance the data of apple leaf disease images to overcome the overfitting problem faced by the recognition model. The data augmentation method proposed in this paper can not only provide sufficient and high-quality images of apple leaf disease for different training models, but also balance the data set.
The remainder of the paper is organized as follows: Section 2 presents related work, Sect. 3 gives the materials and the different methodologies used followed by experimental results in Sect. 4, and finally, Sect. 5 gives the conclusions and future directions.

Related work
In recent years, traditional machine learning methods have achieved remarkable results in identifying apple leaf diseases. Table 1 shows the results of some recent studies on the identification of plant diseases. The shape, texture and color characteristics of the leaf images of plant diseases are used to identify the classes of apple leaf diseases. Chuanlei et al. (2017) used three methods, namely RGA (region growing algorithm), GA-CFS (genetic algorithm and correlation-based feature selection) and SVM, to realize the identification of apple disease images according to the extracted color feature, shape feature and texture feature of apple disease leaf images. However, this data set is collected under controlled conditions and cannot be applied to scenes with uneven illumination. In addition, Shi et al. (2017) proposed an apple disease identification method based on two-dimensional subspace learning dimension reduction (2DSLDR), which improved the effect of disease identification and overcame the problem of feature extraction and selection in traditional disease recognition methods. However, the traditional machine learning methods mentioned above all require manual extraction of lesion characteristics, which is limited for diseases with similar disease characteristics. In the scene with large noise, such as uneven illumination, it is difficult to obtain high identification accuracy.
With the continuous development of deep learning, some studies have applied convolutional neural networks to plants, such as wheat (Lu et al. 2017), cucumber (Ma et al. 2018) and cassava (Ramcharan et al. 2017). Experiments on different plant data sets have shown that deep learning is significantly better than traditional machine learning methods. In addition, researchers have applied it to apple identification and detection. Bi et al. (2020) proposed an apple leaf disease identification method based on MobileNet. This method not only can be easily deployed on mobile devices, but also greatly reduces the burden of disease identification on experts. Based on the DenseNet-121 deep convolutional network, Zhong and Zhao (2020) proposed three methods of regression, multi-label classification and focus loss function to identify apple leaf diseases. The experimental results show that this method is better than the traditional multi-classification method based on the cross-entropy loss function under the condition of data imbalance. The identification accuracy reached 93.71%. In these experiments, the methods used to solve the overfitting problem were either traditional data augmentation methods or pre-trained models. Each image contains only one disease, and there is no conversion between data classes. When there is an imbalance between the categories of the data set, the traditional data augmentation method can only be used to compensate for the difference in the data volume between the categories. Moreover, this method cannot increase the diversity of images, so it cannot solve the diversity problem in essence.
It is well known that deep learn-based approaches require large amounts of data. However, collecting sufficient data sets is unreliable and time-consuming. In agriculture, due to the difference in light conditions and seasonal growth of morphology, the annotation dependence on the size of the data set increases. In order to solve this problem, methods of synthesizing data sets have been studied in recent years. The purpose of data augmentation is to increase the size of the data set (Kukačka et al. 2017). This is a method widely used in various fields. Zhang et al. (2021) designed a multi-feature fusion Faster R-CNN (MF3 R-CNN) to solve the problem of insufficient soybean leaf images in complex scenes. However, this method of combining images manually with Photoshop is labor-intensive and time-consuming. The emergence of generative adversarial networks provides a new way of thinking for researchers (Goodfellow et al. 2014). Goodfellow et al. compared the recognition accuracy of different augmentation methods, such as C-DCGAN (Mirza and Osindero 2014), rotation and translation, etc., and non-augmentation methods. The final result proves that the effect of using GAN to generate images is better than traditional data augmentation methods. Besides, Wu et al. (2020) used deep convolution generative adversarial network (DCGAN) to generate tomato leaf images, and GoogLeNet deep learning networks were trained to identify tomato leaf diseases. Hu et al. (2019) proposed conditional deep convolutional generative adversarial networks (C-DCGAN) for augmenting tea disease spot samples, and VGG16 deep learning networks were trained to realize disease spots identification. However, this method converts noise into an image, that is, noise-to-image, which produces images of poor quality and contains a lot of noise. Furthermore, this method is only suitable for generating more similar images from the same domain and is not suitable for solving the problem of imbalance between data set categories. Another state-of-the-art way of using GAN to generate images is image-to-image, which forms an image pair that had some relation with each other. Qu et al. (2019) presented an enhanced pix2pix dehazing network to solve the problem of an image-to-image translation and generate haze-free images without using the physical scattering model. The experimental results demonstrated that the model is better than the other methods. This method requires the input image to be geometrically paired from the two domains. However, in most cases, the one-to-one correspondence of image pairs is not satisfied. Zhu et al. (2017) proposed an innovative network called cycle-consistent adversarial networks (CycleGAN), which used unpaired image-to-image translation. In the same year, Yi et al. (2017) proposed DualGAN, which eliminated the need for paired data for image-to-image translation. Besides, Tian et al. (2019) used CycleGAN to solve the problem of insufficient image data. Chen et al. (2022) proposed a framework combined with you only look once version four (YOLOv4) and CycleGAN to improve the quality of surface defect detection for golden diamond pineapples. These image-to-image translation networks all share the same framework with two generators (G X2Y and G Y2X ) doing opposite transformations, which can be seen as a kind of dual learning and two discriminators D X and D Y that predict whether an image belongs to that domain or not. Generators (G X2Y , G Y2X ) translate an image from one domain to another. A discriminator for each domain (D X , D Y ) judges if an image belongs to that domain or not.
So far, there have been few studies in the field of plant diseases based on unpaired image-to-image translation. Moreover, existing state-of-the-art methods have problems such as unstable training. It is difficult to use gradient training to make CycleGAN converge to a balance point. Therefore, in this paper, an improved CycleGAN is proposed to generate images of apple leaves to solve the problem of unbalanced data sets and small data sets that are not enough to train convolutional neural networks. In this paper, we add the labels of the training data to the GAN training to present more visually compelling synthetic images on the unaligned data set. The main work of this paper is to use some metrics and recognition systems to comprehensively evaluate the quality of the generated image data and the performance of the proposed method. First of all, a method for qualitative and quantitative indicators to analyze the quality of the generated pictures and the performance of the generative adversarial networks is given. Secondly, the effects of different training modes, data set composition and size on identification performance are analyzed in detail. Finally, some images on the Internet are used as test sets to evaluate the generalization ability of the proposed model. The main contributions of this paper are summarized as follows.
• An improved CycleGAN model for generating apple leaf disease images is proposed. In order to solve the problem of GAN training instability, the classification labels of the training data are added to the GAN training, and the discriminator is changed from one to two. Then, using the model to balance and expand the training samples, an automatic identification framework for apple leaf diseases was constructed. • An accurate deep learning-based pipeline is proposed for synthetic augmentation of apple leaf disease datasets and apple leaf disease identification in a data deficient and class unbalanced environment. For the purpose of balancing the data set, healthy leaves are converted into disease-carrying leaves using two models, health-to-scab and health-to-rust. • The GAN-based hybrid dataset is established, which provides a new contribution to the diagnosis of apple leaf diseases. The initial images of apple leaf disease selected from the PlantVillage dataset and the generated apple leaf disease images are mixed into the training set. Using the synthesized dataset, the recognition performance of the classification model based on ResNet has obtained a higher accuracy rate, which is better than the initial dataset.

Dataset
A total of 1977 apple leaf images in openly and freely dataset collected from the PlantVillage project (Hughes and Salathe 2015), which are distributed in three different classes, were selected for this study (Fig. 1). Among them, there are 1354 healthy apple leaves, 415 apple scabs leaves, and 208 apple cedar rust leaves. Specifically, apple scab spots are round in shape, with light color in the middle and dark brown around. This disease thrives in areas with a multitude of high humidity during spring. Because there is no treatment for infected trees, early identification and prevention are crucial for its control. While apple cedar rust spots are dark gray and occupy a larger volume. This disease can affect a wide range of hosts across apple. Seriously, with a large enough infection, the foliage of the leaves will be negatively affected and likely cause defoliation in the canopy. This is an imbalance between the categories. Each image should be the same size as the input of the neural network (GoogLeNet, AlexNet and ResNet 224 9 224 pixels, VGG 299 9 299 pixels) and in the RGB color space and JPG format. As is known to all, the accuracy of deep learning in recognition will be seriously affected when the dataset is unbalanced among categories (Pulgar et al. 2017). Therefore, considering the problem of overfitting caused by too many parameters and too little data, the improved model based on CycleGAN was used to generate synthetic images and achieve the purpose of balancing the data set. For the purpose of balancing the data set, healthy leaves are converted into disease-carrying leaves using two models, health-to-scab and health-to-rust. Then mix the generated images with part of the original images as input data for the identification network. The preprocessing of apple leaf dataset is shown in Fig. 2. We expanded the apple scab and apple cedar rust categories by 1000, respectively. The implementation details of data augmentation are detailed in 3.2.1. The data set is divided into training set (including validation set) and test set, as shown in Table 2.

Fusion of improved CycleGAN and CNN for apple leaf disease identification
The main work of this paper is to identify apple disease from synthetically generated image data. At the same time, this paper is based on the CycleGAN model, which is (1) apple_healthy.
(3) apple_cedar rust combined with semi-supervised learning to generate higher-quality images and balance data sets on unpaired data sets. The main problems solved are as follows: (a) solving the problem of insufficient data and (b) addressing the imbalance between data categories. The main process of apple leaf disease identification is shown in Fig. 3. Our proposed pipeline contains two components: a synthetic data augmentation module with improved CycleGAN and a disease identification system with a convolutional neural network for supervised learning. Based on CycleGAN's excellent unpaired sample translation capability, we designed a novel CycleGAN model structure to generate hard-to-collect samples. In this work, we added the classification label of training data into the training of GAN and proposed an improved CycleGAN semi-supervised model to solve the unstable problem of GAN training.

Improved CycleGAN model for generating apple leaf disease images
We first introduced the image-to-image CycleGAN network. Then, in view of the problem that the traditional CycleGAN algorithm is prone to instability in the process of leaf image synthesis, we propose an improved Cycle-GAN semi-supervised model.

The basic CycleGAN network
Since the apple leaf images used in this paper were collected at different shooting times and with different shapes, the conditions of one-to-one correspondence were not available. Therefore, a cycle suitable for training mismatched data was used in this paper to generate the network model CycleGAN. Compared with the traditional generative adversarial network GAN, CycleGAN is no longer a traditional noise-to-image translation, but an image-to-image translation. It can Fig. 2 The preprocessing of apple leaf dataset transform healthy leaves into diseased ones, so as to achieve the purpose of balancing the data set. The advantage of CycleGAN over other image-to-image translation methods, such as Pix2pix, is that the input is no longer required to provide a pair of images, meaning that two images entered can be without any relationship. The principle of CycleGAN is shown in Fig. 4. X and Y, respectively, represent two sample spaces. Firstly, input a pair of images X and Y from different styles and generate two discriminators and two generators. Secondly, Cycle-GAN translates samples in X space into samples in Y space. In this work, the mapping from X to Y is represented by the function, where G corresponds to the generator in GAN. For the images generated at this stage, discriminator in GAN is also needed to determine whether they are real images. According to the generator and discriminator here, a GAN discriminator loss function can be constructed, which can be expressed as: Next, the CycleGAN training process needs to convert samples in Y space into samples in X space, set the mapping from X to Y as G Y2X , and construct a GAN discriminant loss function, the expression is: CycleGAN trains two mappings at the same time.
According to the cycle consistency principle, the loss function is defined as: Therefore, the total loss function is composed of three parts: X to Y generation against the discriminator loss function, Y to X generation against the discriminator loss function and cycle consistency loss function, which is denoted by: where k is the weight ratio of cycle consistency loss and adversarial loss. The weight of the generator is optimized according to the loss of each training by repeatedly training the two discriminators until the maximum number of iterations is reached (Fig. 5).

The improved CycleGAN network Although
CycleGAN proposed a method to learn the image translation from source domain X to target domain Y in the case of no paired images, the discriminator is easy to defeat the generator under unsupervised conditions, and the problem of unstable GAN training is very likely to occur. In order to solve this problem, this paper adds the labels of the training data to GAN training and proposes an improved Cycle-GAN semi-supervised model. The specific improved method is to change the discriminator from the original one to two: one to judge whether the image is true or false, and one to judge whether the leaf image has disease (Fig. 6). For an ordinary classifier, suppose that there is a total of k types of data for image classification. The classifier model takes data x as input and outputs a k-dimensional vector. After softmax, the class with the largest classification probability is calculated. In supervised learning, the best results are achieved by minimizing category labels and predicting distribution cross-entropy. GAN can be used in the semi-supervised learning field to learn the styles and features of different images from unlabeled data with the help of GAN. As long as the input data are known to be real data, then the probability of output of real images can Fig. 4 Health-to-scab and health-to-rust models Fig. 5 The principle of CycleGAN be maximized. The improved total loss function is as follows:

CNN model for apple leaf disease identification
After aforementioned model of improved CycleGAN balancing and enlarging training samples, we constructed an automatic identification framework for apple leaf disease. The input here is the data mixed by the original apple leaf images and the apple leaf images generated by improved CycleGAN. Compared with other identification algorithms, convolutional neural networks have the advantage of greatly reducing the learning time and reducing the memory requirements for network operations, thereby allowing the construction of a more powerful neural network. Nevertheless, it has been difficult to train deeper neural networks because of problems such as vanishing gradient and degradation. ResNet tries to address both of these issues. Residual neural network (ResNet) was proposed by He et al. and won the championship in the 2015 ILSVRC (Russakovsky et al. 2014) (ImageNet Large Scale Visual Recognition Challenge). The main contribution of ResNet is to discover the ''Degradation'' and invent the ''shortcut connection'' for the degradation phenomenon, which greatly eliminates the difficulty of neural network training with excessive depth. The main idea of ResNet is to increase the direct connection channel on the network. Although the previous network structure is a nonlinear transformation, ResNet allows to retain a certain percentage of the output of the previous network layer, as shown in Fig. 7. The whole structure is called ''bottleneck design'', which can reduce the number of parameters. The ''curves'' shown here are shortcut connections. One of the biggest benefits of using a 1 9 1 network is that it can greatly reduce the amount of calculation. The first 1 9 1 convolution reduces the 256-dimensional channel to 64 dimensions, and finally it is restored by 1 9 1 convolution. In this work, ResNet-50 is represented by image identification.

Implementation details
In our implementation, TensorFlow was used as deep learning framework in Python to build the network model, and dual graphics processing unit (GPU) was used to accelerate the experimental process. The experiments were conducted on a server which contained two Tesla P100 processors (16 GB memory) with the Centos operating system. Additional configuration parameters are listed in Table 3.
To validate the performance of the proposed two-stage method, three experiments are carried out.
Firstly, the performance of the improved CycleGAN is verified by qualitative and quantitative measurements of  Secondly, some experiments were carried out to verify the performance of the classification model, such as comparing five mainstream classification models and comparing them with traditional data augmentation methods. After obtaining the best model, we also fine-tuned certain parameters of this pre-trained model to obtain the best result. After that, the state-of-the-art methods used to study apple leaf diseases identification were compared with our methods.
Finally, T-SNE and Turing tests were performed to intuitively verify the quality and performance of the images generated by the improved CycleGAN.

Image quality evaluation
In this paper, qualitative and quantitative methods are used to evaluate the quality of generated images, namely General Overviews, GAN-train and GAN-test. Furthermore, t-Distributed Stochastic Neighbor Embedding (t-SNE) (Laurens and Hinton 2008) is used to compare the distribution of images generated by improved CycleGAN with the distribution of real images. Visual Turing test (Salimans et al. 2016) by plant experts is also used to evaluate the appearance of the generated images.

GAN-train and GAN test
GAN-train is to train the classification network with the images generated by GAN and then to classify the real images with the trained network. GAN-train reflects the diversity of generated images. GAN-test means that the classification network is trained with real images, and the images generated by GAN are used for testing. If the result of the test is similar to that of real images, it proves that GAN generated images are highly similar to real images. The process of GAN evaluation is shown in Fig. 8.
There are also some common indicators used to evaluate the performance of GAN, such as Inception Score (IS) and Frechet inception distance (FID). However, IS can only reflect the diversity of generated images, instead of comparing the output samples with ground truth samples. Therefore, it cannot reflect whether the generated images approximate the real images. Similarly, the FID method first assumes a Gaussian distribution of data, then calculates their mean and variance, and then finds the FID value. The disadvantages are: (a) inability to capture subtle changes; (b) according to the increase of FIS value, it is impossible to judge whether it is caused by the insufficient diversity of GAN-generated samples or the great difference between GAN-generated samples and ground-truth samples. Therefore, GAN-train and GAN-test indexes were selected as GAN evaluation indexes in this work.
We compare the performance of the proposed method with some advanced models, such as DCGAN, pix2pix and CycleGAN. Since the image-to-noise translation method only enhances the images of apple leaves with disease, for the sake of fairness, we only evaluate the images with disease and ignore the images with health when we use the GAN-train and GAN-test evaluation. Table 4 shows the final results of each model comparison.
As shown in Table 4, images generated by improved CycleGAN achieves a GAN-train accuracy of 96.67% and GAN-test accuracy of 85.76%, high-lighting their high image quality as well as diversity. The value of GAN-train is generally higher than GAN-test, because the number of original images is small enough to train the deep convolutional neural network. The identification network framework we use here is ResNet.

The generated images
As Fig. 9 shows, some examples of training can be seen. From left to right are original images, the images generated by DCGAN, pix2pix, the original CycleGAN, and the improved CycleGAN. Under the same training settings, we can see that although DCGAN and pix2pix can produce obvious lesions, the generated image quality is low, and the  Bold values are the best result obtained in the test experiments edges are blurred. Moreover, the pattern breakdown of images generated based on DCGAN is serious, which means that the generator crashes and the sample types generated are limited. The image generated with Cycle-GAN does not have this phenomenon. Observation of the images generated by CycleGAN showed that although the quality of the images generated was high, the lesions were not obvious. However, the improved CycleGAN model trained in the semi-supervised way not only produces highquality images, but also has obvious lesions, which is more suitable for generating apple leaf disease images.

t-Distributed stochastic neighbor embedding
To better visualize the distribution of apple leaf images generated by improved CycleGAN and further analyze the performance of the proposed model, t-distributed stochastic neighbor embedding (t-SNE) was applied to this work. First of all, in order to test whether different classes of generated images have different features, we randomly selected 200 generated images of each class and used a total of 400 images to test the results (Fig. 10). It can be clearly seen from Fig. 10 that the data of different labels are distributed far and wide. It can be concluded that the improved CycleGAN successfully captures the subtle features of real images, which can be used to train the classification network, and successfully realizes the conversion between healthy leaves and diseased leaves.
In addition, in order to intuitively verify whether the generated images have the same semantic information as the original images, we randomly selected 400 images in the same class, of which 200 are the original images and 200 are the generated images. The results are shown in Fig. 11. As can be seen from Fig. 11, in the same class, the distribution of real images and the distribution of synthetic images are almost integrated, and the two have similar distributions. Not only that, the images generated by the proposed method fill in the distribution of the original images coverage.

Visual Turing test
In order to quantitatively evaluate the quality of the generated images, we proposed a visual Turing test to three botanists without giving image information and background information: 50 real images and 50 generated images were selected from each class of apple leaf images with disease, and a total of 200 images were tested. The problems faced by botanists are as follows: (a) confirm whether the following images are real or synthetic (Accu-racy1); (b) determine the disease class to which the following images belong (Accuracy2). To make the results of the visual Turing test reliable, we compared the highquality generated images mixed with the low-quality real images, and set the size of all the images to 100 9 100. The experimental results are shown in Table 5 and Fig. 12. The accuracy of the three raters in judging whether the image is a real image (Accuracy1) was only 53.17%, and the accuracy in judging the image class (Accuracy2) reached 96%. This result clearly shows that the generated images successfully capture the features of the real data, and even for botanists, the images generated by our model are indistinguishable from the real images of apple leaves with diseases. Accuracy1 and accuracy2 denote the successful identification rate between the real/generated images and between the different classes of images, respectively.

The identification performance of five classification models
In order to show the robustness of the proposed data augmentation method, we incorporate a recognition model on the basis of the generating model. The generated images and the original images are used as the input of the recognition model. The training set contains the generated images and the original images, and all the images in the test set are from the initial data set. 3797 apple leaf images were used to train ResNet, 180 apple leaves were regarded as test samples as shown in Table 2. To compare the performance of different recognition models, we set up several different experiments. Initialized by ImageNet-pre-trained weights, five popular image classification models Xception,ResNet) are used to train and test these data sets. The results of different pre-trained models are shown in Table 6. We used an initial learning rate of 0.001 and then reduced it by 0.5 every 512 iterations. Stochastic gradient descent (SGD) with momentum of 0.9 was used as the optimization method.
The observation results show that the data augmentation method proposed in this work has good performance for different classes of models. Secondly, under the same experimental conditions, ResNet showed better performance, with the recognition accuracy reaching 97.78%. Finally, we can also observe from the precision, recall and F1 scores reported in Table 6 that our model and the stateof-the-art models are not affected by the data imbalance, indicating that the use of GAN plays a role in balancing the data set. The confusion matrices of different models on test set are shown in Fig. 13. The graph presents the overall accuracy associated with a single class. The threshold is set to 0.5, and the proportion of accurately predicted images for each class is displayed in detail. Compared with other models, ResNet achieves a higher test accuracy, improving the test accuracy of label2 (97%) and label3 (97%).
In order to show the convergence of ResNet for identify apple leaf diseases during the training process, figure on training and validation loss versus epoch is shown in Fig. 14. Figure 14 shows that two curves declined rapidly at the first 5 epochs and began to converge after about 10 epochs.
In order to determine the optimal parameters, different accuracies were achieved in the case of different hyperparameters by training ResNet as shown in Table 7, where the highest accuracy of 97.78% was obtained in the case of optimal hyperparameters combination with an iteration of 1024 and a batch size of 32.

The identification performance of classic and synthetic augmentation technique
This experiment uses the following training samples to train ResNet: (a) non-augmented training samples; (b) training samples augmented by traditional methods (flipping, rotation, Gaussian noise); (c) training samples augmented by improved CycleGAN. The result of the recognition is shown in Fig. 15. These results show that the use of data augmentation methods can effectively solve the overfitting problem caused by the small data set and the imbalance between the data set categories. When using the data augmentation method, the training samples augmented by the improved CycleGAN are significantly better than  Fig. 12 Visual Turing test results (by botanists for classifying real (R) vs generated (G) images). R-R: a real image is recognized as a real image; R-G: a real image is recognized as a generated image; G-R: a generated image is recognized as a real image; G-G: a generated image is recognized as a generated image. Accuracy indicates the botanists' successfully classification rate of real/generated images the traditional data augmentation method, and the accuracy is increased by 14.7%.

Results of the proposed method and the state-of-art methods
In order to verify the quality of the proposed method, we compared this method with the state-of-the-art methods. proposed a new deep convolutional neural network incorporating ROI to identify apple leaf diseases. Although compared to the most advanced method, they used fewer images and the method is more novel, but the accuracy rate obtained is only 84.3%. In conclusion, in the process of disease identification, not only the accuracy rate, but also the number of samples should be considered. How to use the minimum number of samples to achieve the highest accuracy is the goal we need to achieve.

Conclusions and future work
This paper presents an accurate deep learning-based pipeline to solve the problem of limited data sets on farms and reduce the partiality due to serious class imbalance. First, the improved CycleGAN model was proposed to expand and translate the original images. Then, the generated images and the original images are mixed together as the input of the classic convolutional neural network, and finally the identification accuracy of apple leaves is 97.78%. Compared with not using any data augmentation methods or using traditional data augmentation methods, the performance of the proposed model is superior. In  In future work, we will focus our research on images that remove edge information and background information to further improve the performance of model generation and identification [such as the research of Nazki et al. (2020) and Janarthan et al. (2020)]. In short, we strive to develop new methods to achieve continuous performance improvement. Apple leaf disease identification via improved CycleGAN and convolutional neural network 9785