SeptoSympto: A high-throughput image analysisof Septoria tritici blotch disease symptoms using deep learning methods

doi:10.21203/rs.3.rs-3111942/v1

Download PDF

Research Article

SeptoSympto: A high-throughput image analysisof Septoria tritici blotch disease symptoms using deep learning methods

https://doi.org/10.21203/rs.3.rs-3111942/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 01 Feb, 2024

Read the published version in Plant Methods →

You are reading this latest preprint version

Background

Quantitative, accurate, and high-throughput phenotyping of crop diseases is needed for breeding programs and plant-pathogen interaction investigations. However, difficulties in the transferability of available numerical tools encourage maintaining visual assessment of disease symptoms, although this is laborious, time-consuming, requires expertise, and rater dependent. Deep learning has produced interesting results for plant disease evaluation, but has not yet been used to quantify the severity of Septoria tritici blotch (STB) caused by Zymoseptoria tritici, a frequently occurring and damaging disease on wheat crops.

Results

We developed a Python-coded image analysis script, called SeptoSympto, in which deep learning models based on the U-net and YOLO architectures were used to quantify necrosis and pycnidia, respectively. Small datasets of different sizes (containing 50, 100, 200, and 300 leaves) were trained to create deep learning models and to facilitate the transferability of the tool, and five different datasets were tested to develop a robust tool for the accurate analysis of STB symptoms. The results revealed that (i) the amount of annotated data does not influence the good performance of the models, (ii) the outputs of SeptoSympto are highly correlated with those of the experts, with a similar magnitude to the correlations between experts, and that (iii) the accuracy of SeptoSympto allows precise and rapid quantification of necrosis and pycnidia on both durum and bread wheat leaves inoculated with different strains of the pathogen, scanned with different scanners and grown under different conditions.

Conclusions

Although running SeptoSympto takes longer than visual assessment to evaluate STB symptoms, it allows the data to be stored and evaluated by everyone in a more accurate and unbiased manner. Furthermore, the methods used in SeptoSympto were chosen to be not only powerful but also the most frugal, easy to use and adaptable. This study therefore demonstrates the potential of deep learning to assess complex plant disease symptoms such as STB.

Septoria tritici blotch

wheat

image analysis

deep learning

U-net

YOLO

Septoria tritici blotch (STB), caused by the ascomycete fungus Zymoseptoria tritici, is a major and persistent threat for wheat growing in temperate regions [1]. It is considered as the most prevalent and yield-reducing disease in Europe [2], causing up to 50% yield losses under conditions favorable for disease development [3]. The primary methods for controlling this disease involve the use of fungicides and host resistance genes. Nearly 70% of fungicides applied in Europe are devoted to control STB, which is worth one billion euro [4]. However, these strategies are not sustainable and need to be diversified [5]. STB therefore remains highly studied [6], especially to develop disease management through agro-ecological approaches [7].

STB symptoms are quite complex as they display a large diversity of shapes, sizes and colors for the necrosis, as well as for the pycnidia occuring on these lesions. They are also very dependent on the environment and on Z. tritici isolates and wheat genotypes [8]. When Z. tritici infects wheat, leaf lesions appear only after a long symptomless period of more than ten days post inoculation and progressively form necrosis that may contain pycnidia, the asexual fruiting bodies [9]. Because of this long latency period, it can be difficult to differentiate between disease symptoms and leaf senescence, which represents an issue to accurately evaluate disease severity. Furthermore, for the study of disease incidence and epidemics, it is essential to obtain a quantification of pycnidia. Accurate high-throughput phenotyping is also necessary to study alternative control methods that are more quantitative than fungicide based control [10]. In particular, resistance QTLs associated with the percentage of necrosis and pycnidium production have already been identified [11], but it is more difficult to identify QTLs associated with small phenotype variation without accurate symptom detection tools. Therefore, for such studies on STB, the complexity of symptoms prompts the development of image-based phenotyping. Being increasingly used for experiments, this type of phenotyping could allow dynamic monitoring of symptoms and can also be used in breeding programs.

However, visual evaluation by raters or experts is the most commonly used method to assess STB severity, especially in the field [12]. This type of evaluation has been found to represent challenges in effectively quantifying the severity of the disease, as described in Bock et al. 2020 [13]. Notably, the evaluation results are significantly influenced by the rater’s level of experience, thus making disease assessment a challenging task to young researchers. Various aspects are considered to correct for rater biases, such as tools that facilitate concerted evaluation [14] or by proposing quantitative molecular evaluation [15], with the ultimate goal of achieving a standardized interlaboratory evaluation at the international level.

Among the tools available for phenotyping wheat disease symptoms, only few methods are available to evaluate symptoms related to STB ([16]). Some are based on hyperspectral remote sensing, they are available for field evaluation but do not allow an evaluation of pycnidia production [17, 18]. A batch processing macro in ImageJ image analysis software has been developed to automatically measure lesions and pycnidia [19, 20]. Lesions are detected by color thresholding in RGB color space to measure the green leaf area, and pycnidia are identified using the 'Find Maxima' function which produces a binary mask. Another method for detecting pycnidia has been developed in Python and is based on determining contours of constant brightness in the vicinity of each pycnidia [21]. More recently, a tool has been developed to detect the yellow halo present in lesion perimeters, that also detects lesions based on a transformation of the original image into six color spaces, a random forest classifier for thresholding followed by a post-processed filter [22]. Surprisingly deep learning methods have not been yet developed for a quantification of STB symptoms despite the creation of a tool to classify the principal foliar diseases of wheat [23, 24]. However, despite efforts to create tools to detect STB symptoms, the most preferred solution remains a visual analysis of symptoms by experts. This is mainly due to the perception of these tools as more time consuming, difficulty of transferring these technologies or their possible lack of accuracy in new lab environment. Indeed, the tools developed to detect STB severity seem to be too specific to the training sets used to develop them and therefore unsuitable or difficult to use with other datasets obtained under different conditions. Therefore, it is critical to develop a method for quantifying STB severity that is user-friendly, high-throughput, accurate, unbiased, and easily adaptable.

Despite disease incidence (e.g. number of attacked plants) being fairly easy to assess visually, estimating disease severity (e.g. level of disease of individual leaf) requires complex cognition [25]. This explains the complexity of developing accurate and robust phenotyping tools, which are necessarily based on an initial human assessment. Machine learning, which is artificial intelligence capable of automatic adaptation with minimal human intervention, and in particular deep learning, which is a subset of machine learning using highly complex neural networks, could produce results close to expert assessments. Comparisons of image analysis methods for other plant diseases than STB have shown that machine learning outperforms ImageJ in measuring the severity of cassava fire blight [26]. Furthermore, in Sujatha et al. 2021 [27], deep learning methods outperformed machine learning in the case of citrus disease detection. Deep learning is based on an artificial neural network, mimicking the human brain, for data analysis and feature learning [28]. Deep learning has shown very promising results in the field of computer vision, including object detection, semantic segmentation, image classification, etc. [29]. Developing a deep learning tool for STB symptom quantification seems to be a promising method that could correctly distinguish necrosis from senescence and quantify pycnidia by being more accurate and more easily adaptable than currently available methods. However, to our knowledge, deep learning has not yet been used to quantify STB symptoms.

Deep learning architectures used for plant disease detection [30] are based on convolutional neural networks (CNNs). The U-Net architecture [31] is a deep learning segmentation architecture with high performance in image segmentation [32], making it a widely used architecture. U-Net is relatively simple to implement and does not require much training data to be effective, which is a great advantage for its use in biology. It has been used to distinguish subtle differences of growth patterns between plants exposed to various types of gamma radiation in Arabidopsis and has shown superior performance for semantic segmentation [33]. Models based on U-net architectures have also been used to detect leaf blight and brown spot in maize, target spot, brown spot and smut disease in cucumber and have shown high performance (F1 score higher than 85) for both U-net and improved U-net architectures [32]. There is also YOLOv5 (You Only Look Once version 5) architecture [34] which is a deep learning architecture designed for real-time object detection and which can perform global optimization and improve detection speed while maintaining high accuracy. Bacterial spot disease in pepper plants was detected using YOLOv5 and provided higher accuracy and speed compared to previous versions of YOLO [35]. It was also used to detect tomato diseases and pests and provided better accuracy and speed of detection compared to the results obtained with other architectures such as Faster R-CNN, Mask R-CNN and SSD [36]. The U-net and YOLOv5 architectures could therefore allow more accurate quantification of objects such as pycnidia and polygons such as necrosis respectively.

In this work, we tested if deep learning techniques allow to accurately distinguish necrosis caused by STB from leaf senescence symptoms and to precisely quantify STB necrosis and pycnidia on various datasets. We developed an image analysis script, named SeptoSympto (https://github.com/maximereder/septo-sympto), in which deep learning models based on the U-net and YOLO architectures were trained on small datasets to allow a quick, quantitative and accurate phenotyping of STB symptoms obtained under controlled conditions.

Material

Each leaf is taken from a plant, pasted to an A4 sticky sheet, placed in a clear pouch and scanned directly afterwards. All the scanned images were taken at 1200 dpi and in TIFF format to run correctly the SeptoSympto script. The images selected for the datasets were chosen to represent a maximum diversity of symptoms and an equal distribution of each wheat line in the datasets. Different datasets were used to train and evaluate the models (Table 1). In order to represent the maximum of possible parameters influencing symptom development, the datasets originated from three laboratories, four growth conditions, two wheat species and a large set of Z. tritici genotypes.

Table 1

Information about datasets used to develop SeptoSympto.
Dataset	Usage	Leaf number	Plant species	Varieties	Zymoseptoria strains	Growth conditions	Scanner	Institute
1	Model training	375	Triticum aestivum	19	IPO9415	Greenhouse 16h/8h photoperiod, at 24°C/20°C and with 250µmol/s/m²	Epson Perfection V370 Photo	1
2	Model evaluation	40	Triticum aestivum	19	IPO9415	Greenhouse 16h/8h photoperiod, at 24°C/20°C and with 250µmol/s/m²	Epson Perfection V370 Photo	1
3	Model evaluation	50	Triticum aestivum	13	IPO9415	Growing chamber 16h/8h photoperiod, at 24°C/20°C and with 250µmol/s/m²	Epson Perfection V370 Photo	1
4	Model evaluation	50	Triticum turgidum	18	P1	Growing chamber 16h/8h photoperiod, at 24°C/20°C and with 250µmol/s/m²	Epson Perfection V370 Photo	1
5	Model evaluation	115	Triticum aestivum	3	IPO9415	Growing chamber 16h/8h of photoperiod, at 21°C/18°C and with 400µmol/s/m²	Epson Perfection V750 Pro	2
6	Model evaluation	55	Triticum aestivum	1	Descendants of a biparental population (Parental strains: INRA09-FS0813 & INRA09-FS0732)	Growing chamber 16h/8h of photoperiod at 22°C/18°C and with 300µmol/s/m²	CanoScan 9000F MarkII	3
Six different datasets were used to create (dataset 1) or evaluate (dataset 2 to 6) the deep learning models. All the scanned images contain only wheat leaves inoculated with Z. tritici and were taken at 1200 dpi in TIFF format. The datasets consist of different varieties, strains, growth conditions and scans used to evaluate the models.

Data annotation

Individual leaf images were annotated on Roboflow, an online platform for annotating data to train computer vision models (https://roboflow.com/ or https://docs.roboflow.com/). Two independent projects, one based on semantic segmentation and the other based on object detection, were created to annotate necrosis and pycnidia respectively by an expert. 375 leaves were annotated with a polygon class for necrosis, and 240 leaves with a rectangle class for pycnidia. The annotated data were then exported in compatible formats (binary masks in PNG format for the necrosis project and coordinate tables in txt format for the pycnidia project).

Model training

From the annotated data, necrosis detection was trained with the U-net architecture [31] and pycnidium detection with the YOLO (You Only Look Once) architecture [37] (Fig. 1A). Pycnidia were approximated by rectangles thanks to an object detection model using the most powerful architecture to date: YOLOv5. The choice of this architecture for pycnidium detection is based on its ability to perform fast and accurate detections in real time. YOLOv5 is an object detection model that divides an image into a grid and predicts the objects present and their approximate position as rectangles. This architecture is considered very powerful and efficient for object detection. In this context, due to the small size and the large number of pycnidia to be identified, pycnidia are approximated by rectangles to facilitate their detection and annotation. In order to detect necrosis, we used semantic segmentation, which is an image analysis method that classifies each pixel of the image according to the object or class of objects to which it belongs. In this case, the U-net architecture was chosen. The U-net architecture is specifically designed for biomedical image segmentation and is known for its ability to produce accurate results with a limited number of training samples. Unlike pycnidia, necroses cannot be approximated by rectangles. Semantic segmentation therefore allows for the identification of more complex shapes in the form of polygons.

All the training was conducted using Python as a programming language on a computer equipped with an Intel Core i5 processor and two T4 graphics processing units (GPU), each with 16Go of memory. Necrosis models are using a U-Net implementation for semantic segmentation, implemented with the Tensorflow library (https://github.com/maximereder/unet). Pycnidium models are using YOLOv5 architecture for object detection, implemented with the Pytorch library [34]. For both, they were obtained from 50, 100, 200 or 300 training images and 20% validation set. The input size of models is an image size of 304 x 3072 pixels. Models are trained with 16 images per batch on 200 epochs initially. During training, if the training metrics remain constant over time, we stop training the model. The output models are in .h5 format for necrosis, and in .pt format for pycnidia.

Image analysis

Necrosis and pycnidia are detected thanks to the SeptoSympto script, which cuts each leaf to be able to analyze them with deep learning models (Fig. 1B).

To run the image analysis, a folder must contain files called “images_input”, “csv_input”, “models”, “outputs”, “tools” and the script SeptoSympto available on https://github.com/maximereder/septo-sympto. The images in TIFF format and scanned at 1200 dpi are either directly usable by SeptoSympto if there are only horizontally scanned leaves on the image, or pre-cut and renamed with the XnView software if there are writings on the scanned images in order to have only horizontal leaves present on the input images. These images are stored in the file called “images_input”. A file in csv format containing information for each image and the name of each image in the first column can be added in the file called “csv_input”. The deep learning-based models used for necrosis and pycnidia detection, respectively necrosis-model-375.h5 and pycnidia-model.pt, are stored in “models” file. They are implemented using the OpenCV and TensorFlow libraries. The “tools” file contains a python script called “metrics.py” to calculate metrics for segmentation models and can also store other optional scripts.

The first function, in SeptoSympto script, detects each leaf that has a minimum area of 50000 pixels and a color range between [0, 35, 65] and [255, 255, 255] using the function **cv2.findContours**. The detected leaves are then cut with a rectangle encompassing the whole leaf, resized to a fixed size of 304 x 3072, renamed with the file name and the leaf number of the image and saved in a file called “cropped”.

The second function, **predict_necrosis_mask**, predicts the probability of each pixel to be in necrosis class. To get a binary mask, we apply thresholds to keep only detected ones of more than 300 pixels and with a perimeter to area ratio of less than 0,8. The function returns a tuple containing the image with the necrosis contours drawn, the total area of necrosis and the necrosis number.

The third function, **predict_pycnidia**, uses the pycnidia model to predict rectangles and confidence from the analyzed image, and only retains pycnidia coordinates for those with a confidence level above the threshold of 0.3 and with a maximum number of pycnidia prediction per leaf of 10000. The function returns a tuple containing the image with the pycnidia contours drawn, the total area of pycnidia and the pycnidia number.

After running the image analysis, we obtain the cropped images and output folder. In output, we have the result csv file containing both data imported and data calculated. We obtain the cropped images, the images with pycnidia and necrosis contours drawn and the measurements containing the leaf area, necrosis area, necrosis number, pycnidia area and pycnidia number in a csv file, saved in files called “cropped”, “images_output” and “results” respectively.

Model evaluation

Before implementing models in SeptoSympto script, several models trained with different sized datasets were tested and compared using metrics and expert observations. The metrics used are precision, recall and F1 for necrosis and pycnidia models:

$${P}{r}{e}{c}{i}{s}{i}{o}{n}=\frac{{T}{P}}{{T}{P}+{F}{P}}$$

$$Recall=\frac{TP}{TP+FN}$$

$$F1 score=2\frac{Precision*Recall}{Precision+Recall}=\frac{2 TP}{2 TP+FP+FN}$$

with TP: true positive, FP: false positive and FN: false negative

The results obtained from best models were then compared to expert evaluations using Spearman correlation. Expert assessments were made by visually measuring the area of necrosis present on the total leaf surface and the area of necrosis containing pycnidia.

Script conception and model validation

Before implementing the final models (one for necrosis and one for pycnidia) in the SeptoSympto script, four models for necrosis and three for pycnidia were trained from different size subsets of dataset 1. Three metrics were used to evaluate the models: precision, recall and F1 score. High precision indicates minimal false positive detection, suggesting that the model detects a majority of positives that are well predicted. High recall indicates maximum detection of true positives and minimum false negatives, suggesting that the model misses a minimum of true positives. F1 is the harmonic mean of recall and precision, the higher it is, the better the model performs. The model metrics obtained with training datasets remained similar regardless of the size of datasets used for training (Table 2).

Table 2

Metrics of models for necrosis and pycnidia.
A Pycnidia
Model	Leaf number in training dataset	Leaf number in validation dataset	Epochs	Precision	Recall	F1
P0	50	10	149	0.56	0.24	0.34
P1	100	20	156	0.61	0.35	0.45
P2	200	40	183	0.56	0.25	0.34
B Necrosis
Model	Leaf number in training dataset	Leaf number in validation dataset	Epochs	Precision	Recall	F1
N0	50	10	10	0.92	0.83	0.87
N1	100	20	23	0.94	0.87	0.91
N2	200	40	19	0.91	0.85	0.88
N3	300	75	20	0.95	0.85	0.90
The model trainings were carried out on a small number of leaves and epochs. The resulting models were evaluated using different metrics: precision, recall and F1. High precision indicates minimal false positive detection. High recall indicates maximum detection of true positives. F1 is the harmonic mean of recall and precision. (A) Four models (N0, N1, N2 & N3) for necrosis detection were trained on training and validation datasets of different sizes, using a segmentation based deep learning architecture called U-net. (B) Three models (P0, P1 & P2) for pycnidia detection were trained on training and validation datasets of different sizes, using an object detection based deep learning architecture called YOLOv5.

The N3 model, which is used in the SeptoSympto script for detecting necrosis, has a precision of 0.95, a recall of 0.85 and a F1 score of 0.90 (Table 2), suggesting that our model is adapted to detect necrosis. After only 20 training cycles (epochs), the loss function, which is the summation of the errors made by the model for each example in the training set, does not decrease anymore but has reached a low value (Additional file 1). This demonstrates that the model makes few errors when comparing the expert annotations with the model predictions and that there was therefore no need for further training. For the P2 model, which is the one implemented in the SeptoSympto script for detecting pycnidia, precision is 0.56, recall is 0.25, and F1 is 0.34 (Table 2). Due to the large number of pycnidia that can be detected on leaves, the metrics remain quite low even with 183 training cycles. Indeed, it is much more difficult to have metrics with high values if they are calculated on a large number of detected objects. However, the loss function curves for the training and validation sets are strongly decreasing (Additional file 1). This shows that the model makes fewer errors at each iteration. The metrics analyzed show therefore very little variation between the models tested with different sizes of training datasets, suggesting that the number of annotated leaves used to create the models could be lower. Nevertheless, the trained models with more annotated leaves (models N3 and P2) were chosen for the SeptoSympto script because they allow a better representation of the diversity of observable symptoms and thus a better generalization of the models for inter-laboratory use. Despite the model metrics, the observation of SeptoSympto outputs (Fig. 2), using the N3 and P2 models, seems to correctly and precisely detect pycnidia and necrosis regardless of the quantity (absence of symptoms, necrosis without pycnidia, pycnidia density scale, necrosis size) and diversity (necrosis and pycnidia colors) of symptoms.

Script evaluation

In order to evaluate the performance of the SeptoSympto script, we performed an analysis on two datasets obtained independently of the one used to create the models: dataset 2 containing 40 leaves and dataset 6 containing 55 leaves (Table 1).

Dataset 2 was evaluated by the SeptoSympto script and by two independent experts. The correlation between experts was 74% for necrosis and 95% for pycnidia. The evaluations of these two experts were compared with the outputs of SeptoSympto (Fig. 3). We obtained high correlations for necrosis (94% correlation for expert 1 and 75% for expert 2) and for pycnidia (83% correlation for expert 1 and 80% for expert 2) between the manual evaluations and the script outputs. The correlations between SeptoSympto and the experts are of the same order of magnitude as those obtained between the experts, which was expected since deep learning models are trained on data annotated by another independent expert (expert 3).

To compare the different tools available for STB phenotyping, we submitted dataset 6 to the tool developed by Steward & McDonald 2014 and Stewart et al. 2016 [19, 20], to SeptoSympto script and to a visual evaluation (Fig. 4). Between the two tools, a correlation of 55% is obtained for the number of pycnidia and 59% for the necrosis area. However, the correlations between SeptoSympto and expert 1 is higher (76% for necrosis and 94% for pycnidia) compared to the ones between ImageJ and expert 1 (45% for necrosis and 57% for pycnidia). It should be noted that expert 1 is independent of the expert 3 who developed the tools. Therefore, this indicates that SeptoSympto is a more accurate tool to quantify septoria symptoms than the previous one [19].

Script transfer

A major problem with phenotyping based on machine learning techniques is overfitting. Indeed, it is possible that the model is too specific to the training set. The analyses presented above show that our script does not exhibit any overfitting since it detects necrosis and pycnidia very well on datasets that were not used for training the models. However, dataset 2 was obtained under the same conditions (same varieties, strain, growing conditions and scanner) as the training set. In order to develop a tool that everyone can use, we tested whether SeptoSympto performed as well on datasets from different conditions (species, varieties, strains, growing conditions & scanner) as the dataset used to create the models.

We first evaluated dataset 3 obtained with the same varieties, strain and scanner but the plants were grown under different growing conditions. The correlations between SeptoSympto outputs and expert assessments were 95% for expert 1 and 90% for expert 2 for necrosis, and 58% for expert 1 and 81% for expert 2 for pycnidia (Additional file 2). The correlation between the experts was 95% for necrosis and 80% for pycnidia. These results are similar to those obtained with dataset 2. Hence, SeptoSympto seems to be adapted to datasets from different growing conditions.

With dataset 4, we tested whether SeptoSympto was still performing as well on leaves of another species: durum wheat inoculated with another strain but grown under the same conditions as dataset 3 and scanned with the same scanner. The correlations between SeptoSympto results and those of the expert manual scoring are still high (Fig. 5A). Indeed, a correlation of 96% for expert 1 and 92% for expert 2 regarding necrosis (with a correlation of 88% between the experts), and a correlation of 69% for expert 1 and 71% for expert 2 regarding pycnidia (with a correlation of 89% between the experts) were scored. Thus, SeptoSympto seems to perform well on datasets from bread and durum wheat species, and from STB symptoms caused by different strains of Z. tritici.

To go further, we tested whether the script was sensitive to image capture from different laboratories. Dataset 5 is composed of 115 leaves from three varieties already present in datasets 1, 2 and 3, inoculated with the same strain as these three datasets, but grown under different growing conditions and taken with a different scanner. The correlations obtained between SeptoSympto and expert 1 results are 83% for necrosis and 81% for pycnidia (Fig. 5B). Dataset 6 is composed of 55 leaves from the same variety inoculated with different isolates, for which images were obtained with a different scanner but at the same resolution. The correlations between SeptoSympto and expert 1 do not decrease (correlation of 76% for necrosis and 94% for pycnidia) (Fig. 4), suggesting that SeptoSympto is able to analyze images from different scanners. This supports that SeptoSympto is a robust tool for accurate analysis of STB symptoms from different experimental conditions.

With the emergence of artificial intelligence, high-throughput phenotyping is growing exponentially, both on visible spectrum images and on other spectral ranges, as it allows the acquisition of high-quality phenotyping data very quickly, easily, and reproducibly. However, the phenotyping of STB symptoms in wheat is still very much done visually, which is laborious, inaccurate, time-consuming and requires expertise [38]. We have therefore developed an image analysis script using deep-learning techniques and allowing high-throughput phenotyping of necrosis and pycnidia caused by STB with pre-trained convolutional neural networks. As annotation is longer and pycnidium detection is less efficient in conventional machine learning, we used here deep learning architectures to facilitate annotation and enable more accurate quantification of STB symptoms from scanned wheat leaves, notably for pycnidium detection. This developed script uses two models trained on small datasets: one model trained with the U-net architecture for necrosis detection by semantic segmentation and another one trained with the YOLO v5 architecture that distinguishes pycnidia by object detection. This script was evaluated on 6 datasets, containing images obtained with 3 different scanners at 1200 dpi, 38 varieties of bread wheat or durum wheat inoculated with different Z. tritici strains and grown under different growing conditions.

To maximize the performance of our models, sources of variability in image capture were minimized with the same image resolution, fixed image backgrounds and no leaf curvature. In addition to source variability, methodological and dataset issues can also be encountered when using deep learning methods [28]. The training datasets used for learning necrosis and pycnidium detection models are small, so that the models can be easily recreated. When the training dataset is small, it reveals the effectiveness of the detection method used [39]. Therefore, the architectures used were carefully chosen to best fit our problems. Also, to facilitate the adaptation and transferability of this script, no augmentation was performed on the data but a kind of cross-validation was done on several datasets with expert notations in order to avoid a common problem with deep learning models: overfitting [39]. Here, model metrics are low, especially for pycnidia, but correlations between script outputs and expert ratings are high and do not decrease when changing datasets, even for datasets completely different from the training and validation datasets, which shows the performance of our models as well as the absence of overfitting.

Although the time required to annotate the images (1 hour for 10 leaves) and to learn the model (1 to 3 hours depending on the number of leaves) is long, it remains quite low for deep learning models since we use small amounts of annotated data for the model learning. With the image analysis tools, there is also an additional image acquisition time compared to the visual evaluation, as leaves have to be collected, pasted and scanned (60' for 80 leaves). However, the execution time of the script to cut the images and detect necrosis and pycnidia (1'45'' for 10 leaves) is almost similar to a visual evaluation (2’22” for 10 leaves). With a visual evaluation, we obtain quantitative data stored in a table. This requires a little storage space, but it is no longer possible to have access to the raw data (the leaf observation). In addition, visual evaluation requires expertise and makes it possible to obtain only the areas of leaves containing necrosis or of necrosis containing pycnidia, but not the number of pycnidia. The evaluation with SeptoSympto, on the contrary, allows to keep the scans of leaves, according to the chosen outputs of the script. Moreover, the image analysis tool makes it possible to obtain more precise data, in particular the number of pycnidia that may be a better proxy for assessing sporulation capacity than the leaf area containing pycnidia which is assessed visually. A better detection of this pycnidia number could allow a better evaluation of the aggressiveness of STB strains [40]. Detection with SeptoSympto also avoids the notator effects. Indeed, an estimation variability of the leaf area covered by pycnidia can be observed between experts depending on whether they take into account the density or the size of pycnidia within this area. Here, the raters have at least 10 years of expertise on STB. As the disease is difficult to phenotype, due to the wide variety of necrosis that can be observed and the small size of pycnidia, non-experts can also accurately study the severity of STB with SeptoSympto tool. This tool is intended to be easily usable by all and transferable to all, in order to standardize interlaboratory evaluation of STB symptoms.

The key issues for the deployment of new high throughput phenotyping methods are their transferability between laboratories and their acceptance by a majority of researchers. The SeptoSympto tool was developed with the aim of being efficient on a maximum of different datasets scanned at 1200 dpi but also to be easily adaptable to other datasets. We have therefore developed a tool that is as frugal as possible [41], in order to build the most accurate models possible using the least amount of resources (small amount of data needed to train the models, and no need to use large computing resources), which allows for easier adaptability. Indeed, SeptoSympto is an end-to-end script coded in Python using deep learning models. It runs quickly and offers several options to choose the input and output files. In addition, we have shown that this script performs well compared to visual evaluation on six different datasets containing leaves all inoculated with Z. tritici only. The methods used to develop the script were chosen to be not only the most powerful but also the most frugal and easy to use. Models are learned on small datasets and can be trained on computers with different processors and graphic cards to facilitate learning new models. All this allows to easily annotate images and train necrosis and pycnidium detection models.

Depending on the applications (plot management, rate phenotyping or epidemiological survey), tools based on image analysis to detect plant diseases can be developed at various detection scales: at the level of the leaf, the plant, or cover in order to quantify, identify or detect early the disease. Regarding SeptoSympto, it is a leaf phenotyping tool allowing the accurate quantification of STB severity for necrosis and pycnidia. Even if the models obtained for SeptoSympto allow precise detection and script transferability, we could create an open-access database with annotated data to train the models on larger datasets with leaves scanned by many laboratories to allow the use of a single script at the international level which is as robust as possible. In addition, a next step could be to adapt this script to the detection of STB symptoms observed in the field, which are very different from the symptoms observed under control conditions and where other diseases could be observed on the same leaves. This adaptation of the script to the field data would be easily achieved by using a portable scanner and training new models.

The SeptoSympto tool is a script coded in Python that uses deep learning models to detect STB symptoms. The models used have been trained on a small amount of data to facilitate the transferability of the tool and with the U-net and YOLOv5 architectures to quantify necrosis and pycnidia respectively. This script was created in order to develop an easily adaptable and common tool between laboratories to assess STB symptoms. The tool effectively allows accurate and rapid quantification of necrosis and pycnidia on durum and bread wheat leaves inoculated with different strains, grown in different conditions and scanned with different scanners.

Acknowledgements

Not applicable.

Authors' contributions

LM, CLP, TCM, CS participated to the data acquisition. LM, AS, AD, CS evaluated the data. LM, MR annotated the images and developed the script. LM analyzed the data and prepared the figures and tables. LM, EB wrote and edited the manuscript. MR, AS, TCM, JBM, CS reviewed the manuscript. JBM, EB supervised the work. All authors have read and approved the final manuscript.

Funding

This study was supported by the French National Research Agency under the Investments for the Future Program, referred as ANR-20-PCPA-0006.

Availability of data and materials

The datasets used in this study are available from the corresponding author on reasonable request.

Ethics approval and consent to participate

Not applicable.

Consent for publication

Consent and approval for publication were obtained from all authors.

Competing interests

The authors declare that they have no competing interests.

Authors’ informations

Laura Mathieu

Affiliation: PHIM Plant Health Institute, INRAE, CIRAD, Montpellier University, IRD, Institut Agro, Montpellier, France

E-mail: [email protected]

ORCID number: 0009-0003-4083-1298

Maxime Reder

Affiliation: PHIM Plant Health Institute, INRAE, CIRAD, Montpellier University, IRD, Institut Agro, Montpellier, France

E-mail: [email protected]

Ali Siah

Affiliation: BioEcoAgro, Junia, Lille University, Liège University, UPJV, Artois University, ULCO, INRAE, Lille, France

E-mail: [email protected]

ORCID number: 0000-0002-7011-9686

Aurélie Ducasse

Affiliation: PHIM Plant Health Institute, INRAE, CIRAD, Montpellier University, IRD, Institut Agro, Montpellier, France

E-mail: [email protected]

Camilla Langlands-Perry

Affiliation: BIOGER, Paris-Saclay University, INRAE, Palaiseau, France

E-mail: [email protected]

ORCID number: 0000-0003-2635-6905

Thierry C. Marcel

Affiliation: BIOGER, Paris-Saclay University, INRAE, Palaiseau, France

E-mail: [email protected]

ORCID number: 0000-0002-7452-4013

Jean-Benoît Morel

Affiliation: PHIM Plant Health Institute, INRAE, CIRAD, Montpellier University, IRD, Institut Agro, Montpellier, France

E-mail: [email protected]

ORCID number: 0000-0003-1988-956X

Cyrille Saintenac

Affiliation: GDEC, UCA, INRAE, GDEC, Clermont-Ferrand, France

E-mail: [email protected]

ORCID number: 0000-0002-3791-301X

Elsa Ballini

Affiliation: PHIM Plant Health Institute, INRAE, CIRAD, Montpellier University, IRD, Institut Agro, Montpellier, France

E-mail: [email protected]

ORCID number: 0000-0002-2571-2069

Orton ES, Deller S, Brown JKM. Mycosphaerella graminicola: from genomics to disease control. Molecular Plant Pathology. 2011;12:413–24.
Jørgensen LN, Hovmøller MS, Hansen JG, Lassen P, Clark B, Bayles R, et al. IPM Strategies and Their Dilemmas Including an Introduction to www.eurowheat.org. Journal of Integrative Agriculture. 2014;13:265–81.
Fones H, Gurr S. The impact of Septoria tritici Blotch disease on wheat: An EU perspective. Fungal Genetics and Biology. 2015;79:3–7.
Torriani SFF, Melichar JPE, Mills C, Pain N, Sierotzki H, Courbot M. Zymoseptoria tritici: A major threat to wheat production, integrated approaches to control. Fungal Genetics and Biology. 2015;79:8–12.
McDonald BA, Mundt CC. How Knowledge of Pathogen Population Biology Informs Management of Septoria Tritici Blotch. Phytopathology®. 2016;106:948–55.
Dean R, Van Kan J a. L, Pretorius ZA, Hammond-Kosack KE, Di Pietro A, Spanu PD, et al. The Top 10 fungal pathogens in molecular plant pathology. Molecular Plant Pathology. 2012;13:414–30.
Brzozowski L, Mazourek M. A Sustainable Agricultural Future Relies on the Transition to Organic Agroecological Pest Management. Sustainability. 2018;10:2023.
Schilly A, Risser P, Ebmeyer E, Hartl L, Reif JC, Würschum T, et al. Stability of Adult-plant Resistance to Septoria tritici blotch in 24 European Winter Wheat Varieties Across Nine Field Environments. Journal of Phytopathology. 2011;159:411–6.
Brennan CJ, Benbow HR, Mullins E, Doohan FM. A review of the known unknowns in the early stages of septoria tritici blotch disease of wheat. Plant Pathology. 2019;68:1427–38.
Xiao Q, Bai X, Zhang C, He Y. Advanced high-throughput plant phenotyping techniques for genome-wide association studies: A review. J Adv Res. 2022;35:215–30.
Brown JKM, Chartrain L, Lasserre-Zuber P, Saintenac C. Genetics of resistance to Zymoseptoria tritici and applications to wheat breeding. Fungal Genet Biol. 2015;79:33–41.
El Jarroudi M, Kouadio AL, Mackels C, Tychon B, Delfosse P, Bock CH. A comparison between visual estimates and image analysis measurements to determine septoria leaf blotch severity in winter wheat. Plant Pathology. 2015;64:355–64.
Bock CH, Barbedo JGA, Del Ponte EM, Bohnenkamp D, Mahlein A-K. From visual estimates to fully automated sensor-based measurements of plant disease severity: status and challenges for improving accuracy. Phytopathology Research. 2020;2:9.
Del Ponte EM, Pethybridge SJ, Bock CH, Michereff SJ, Machado FJ, Spolti P. Standard Area Diagrams for Aiding Severity Estimation: Scientometrics, Pathosystems, and Methodological Trends in the Last 25 Years. Phytopathology. 2017;107:1161–74.
Fraaije BA, Lovell DJ, Coelho JM, Baldwin S, Hollomon DW. PCR-based Assays to Assess Wheat Varietal Resistance to Blotch (Septoria Tritici and Stagonospora Nodorum) and Rust (Puccinia Striiformis and Puccinia Recondita) Diseases. European Journal of Plant Pathology. 2001;107:905–17.
Kumar D, Kukreja V. Deep learning in wheat diseases classification: A systematic review. Multimed Tools Appl. 2022;81:10143–87.
Yu K, Anderegg J, Mikaberidze A, Karisto P, Mascher F, McDonald BA, et al. Hyperspectral Canopy Sensing of Wheat Septoria Tritici Blotch Disease. Frontiers in Plant Science [Internet]. 2018 [cited 2023 Feb 2];9. Available from: https://www.frontiersin.org/articles/10.3389/fpls.2018.01195
Anderegg J, Hund A, Karisto P, Mikaberidze A. In-Field Detection and Quantification of Septoria Tritici Blotch in Diverse Wheat Germplasm Using Spectral–Temporal Features. Frontiers in Plant Science [Internet]. 2019 [cited 2023 Mar 31];10. Available from: https://www.frontiersin.org/articles/10.3389/fpls.2019.01355
Stewart EL, McDonald BA. Measuring Quantitative Virulence in the Wheat Pathogen Zymoseptoria tritici Using High-Throughput Automated Image Analysis. Phytopathology®. 2014;104:985–92.
Stewart EL, Hagerty CH, Mikaberidze A, Mundt CC, Zhong Z, McDonald BA. An Improved Method for Measuring Quantitative Resistance to the Wheat Pathogen Zymoseptoria tritici Using High-Throughput Automated Image Analysis. Phytopathology®. 2016;106:782–8.
Yates S, Mikaberidze A, Krattinger SG, Abrouk M, Hund A, Yu K, et al. Precision Phenotyping Reveals Novel Loci for Quantitative Resistance to Septoria Tritici Blotch. Plant Phenomics [Internet]. 2019 [cited 2023 Feb 2];2019. Available from: https://spj.science.org/doi/full/10.34133/2019/3285904
Anderegg J, Kirchgessner N, Kronenberg L, McDonald BA. Automated Quantitative Measurement of Yellow Halos Suggests Activity of Necrotrophic Effectors in Septoria tritici Blotch. Phytopathology®. 2022;112:2560–73.
Long M, Hartley M, Morris RJ, Brown JKM. Classification of wheat diseases using deep learning networks with field and glasshouse images. Plant Pathology [Internet]. [cited 2023 Feb 6];n/a. Available from: https://onlinelibrary.wiley.com/doi/abs/10.1111/ppa.13684
Genaev MA, Skolotneva ES, Gultyaeva EI, Orlova EA, Bechtold NP, Afonnikov DA. Image-Based Wheat Fungi Diseases Identification by Deep Learning. Plants. 2021;10:1500.
Bock CH, Chiang K-S, Del Ponte EM. Plant disease severity estimated visually: a century of research, best practices, and opportunities for improving methods and practices to maximize accuracy. Trop plant pathol. 2022;47:25–42.
Elliott K, Berry JC, Kim H, Bart RS. A comparison of ImageJ and machine learning based image analysis methods to measure cassava bacterial blight disease severity. Plant Methods. 2022;18:86.
Sujatha R, Chatterjee JM, Jhanjhi N, Brohi SN. Performance of deep learning vs machine learning in plant leaf disease detection. Microprocessors and Microsystems. 2021;80:103615.
Liu J, Wang X. Plant diseases and pests detection based on deep learning: a review. Plant Methods. 2021;17:22.
Voulodimos A, Doulamis N, Doulamis A, Protopapadakis E. Deep Learning for Computer Vision: A Brief Review. Computational Intelligence and Neuroscience. 2018;2018:1–13.
Saleem MH, Potgieter J, Arif KM. Plant Disease Detection and Classification by Deep Learning. Plants. 2019;8:468.
Ronneberger O, Fischer P, Brox T. U-Net: Convolutional Networks for Biomedical Image Segmentation. In: Navab N, Hornegger J, Wells WM, Frangi AF, editors. Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. Cham: Springer International Publishing; 2015. p. 234–41.
Zhang S, Zhang C. Modified U-Net for plant diseased leaf image segmentation. Computers and Electronics in Agriculture. 2023;204:107511.
Chang S, Lee U, Hong MJ, Jo YD, Kim J-B. High-Throughput Phenotyping (HTP) Data Reveal Dosage Effect at Growth Stages in Arabidopsis thaliana Irradiated by Gamma Rays. Plants. 2020;9:557.
Jocher G. YOLOv5 by Ultralytics [Internet]. 2020 [cited 2023 Feb 6]. Available from: https://github.com/ultralytics/yolov5
Mathew MP, Mahesh TY. Leaf-based disease detection in bell pepper plant using YOLO v5. SIViP. 2022;16:841–7.
Wang X, Liu J, Liu G. Diseases Detection of Occlusion and Overlapping Tomato Leaves Based on Deep Learning. Frontiers in Plant Science [Internet]. 2021 [cited 2023 Jun 25];12. Available from: https://www.frontiersin.org/articles/10.3389/fpls.2021.792244
Redmon J, Divvala S, Girshick R, Farhadi A. You Only Look Once: Unified, Real-Time Object Detection. 2016 [cited 2023 Feb 6]. p. 779–88. Available from: https://www.cv-foundation.org/openaccess/content_cvpr_2016/html/Redmon_You_Only_Look_CVPR_2016_paper.html
Bock CH, Poole GH, Parker PE, Gottwald TR. Plant Disease Severity Estimated Visually, by Digital Photography and Image Analysis, and by Hyperspectral Imaging. Critical Reviews in Plant Sciences. 2010;29:59–107.
Hasan RI, Yusuf SM, Alzubaidi L. Review of the State of the Art of Deep Learning for Plant Diseases: A Broad Analysis and Discussion. Plants. 2020;9:1302.
Suffert F, Sache I, Lannou C. Assessment of quantitative traits of aggressiveness in Mycosphaerella graminicola on adult wheat plants. Plant Pathol. 2013;62:1330–41.
Evchenko M, Vanschoren J, Hoos HH, Schoenauer M, Sebag M. Frugal Machine Learning [Internet]. arXiv; 2021 [cited 2023 Jun 25]. Available from: http://arxiv.org/abs/2111.03731

No competing interests reported.

Additionalfile1.pdf
Additional file 1: Loss curves of model training by epochs for necrosis and pycnidia. (A) Loss curves by epochs obtained for training the N3 necrosis model, with 300 leaves in the training dataset (red dots) and 75 leaves in the validation dataset (blue dots). (B) Loss curves by epochs obtained for training the P2 pycnidia model, with 200 leaves in the training dataset (red dots) and 40 leaves in the validation dataset (blue dots).
Additionalfile2.pdf
Additional file 2: Correlations between expert ratings and SeptoSympto results for necrosis and pycnidia detection in dataset 3. Comparisons between the assessments of two experts and SeptoSympto for the measurements of necrosis and pycnidia on 50 bread wheat leaves scanned with the Epson Perfection V370 Photo scanner were calculated using Spearman correlations. Dataset 3 was obtained under different growing conditions, with the same Z. tritici strain and 13 same varieties from the dataset used for the model trainings.

Download PDF

Journal Publication

published 01 Feb, 2024

Read the published version in Plant Methods →

Editorial decision: Major revision
31 Aug, 2023
Reviews received at journal
26 Aug, 2023
Reviewers agreed at journal
31 Jul, 2023
Reviews received at journal
21 Jul, 2023
Reviewers agreed at journal
03 Jul, 2023
Reviewers invited by journal
03 Jul, 2023
Editor assigned by journal
27 Jun, 2023
Submission checks completed at journal
27 Jun, 2023
First submitted to journal
26 Jun, 2023

You are reading this latest preprint version

SeptoSympto: A high-throughput image analysisof Septoria tritici blotch disease symptoms using deep learning methods

Status:

Journal Publication

Version 1

Abstract

Background

Results

Conclusions

Figures

Background

Materials and methods

Material

Data annotation

Model training

Image analysis

Model evaluation

Results

Script conception and model validation

Script evaluation

Script transfer

Discussion

Conclusion

Declarations

References

Additional Declarations

Supplementary Files

Status:

Journal Publication

Version 1