RotatedStomataNet: a deep rotated object detection network for directional stomata phenotype analysis

Stomata act as a pathway for air and water vapor during respiration, transpiration and other gas metabolism, so the stomata phenotype is important for plant growth and development. Intelligent detection of high throughput stoma is a key issue. However, current existing methods usually suffer from detection error or cumbersome operations when facing densely and unevenly arranged stomata. The proposed RotatedStomataNet innovatively regards stomata detection as rotated object detection, enabling an end-to-end, real-time and intelligent phenotype analysis of stomata and apertures. The system is constructed based on the Arabidopsis and maize stomatal data sets acquired in a destructive way, and the maize stomatal data set acquired in a nondestructive way, enabling one-stop automatic collection of phenotypic such as the location, density, length and width of stomata and apertures without step-by-step operations. The accuracy of this system to acquire stomata and apertures has been well demonstrated in monocotyledon and dicotyledon, such as Arabidopsis, soybean, wheat, and maize. And the experimental results showed that the prediction results of the method are consistent with those of manual labeled. The test sets, system code, and its usage are also given (https://github.com/AITAhenu/RotatedStomataNet).


Introduction
Stomata play an important role in the growth of plants and some basic life activities.Whether it is a monocotyledon or a dicotyledon, the state of the stomata (open or close) affects the survival state of the plant itself.Recent studies have shown that when plants are stressed, they close their stomata, leading to venous embolism and plant water supply system collapse (Murphy et al. 2016).Plants adapt to changes in their environment by adjusting the size of their apertures to change their rates of photosynthesis, respiration, and transpiration (Berger et al. 2000;Hetherington et al. 2003;Qian et al. 2013, Zuo et al. 2005).Therefore, stomatal shape and behavior have been identified as direct indicators of plant health and growth environment (Beerling et al. 1993;Ortega-Farias et al. 2012).Thus, the study of stomatal behavior has always been a hot topic in the field of botany.
However, the study of stomatal phenotypic traits has been limited by the trouble of obtaining data.Usually, stomatal images are obtained by microscope, and then these images are manually labeled by the professional image processing software ImageJ, so as to obtain the characteristics of stomatal length, width, stomatal density, aperture and so on, so as to lay a foundation for the subsequent study of stomatal behavior.However, this processing method will cost a lot of manpower and time, and also makes researchers unable to quickly and timely obtain a large number of data, which greatly limits the relevant research work.
For a long time, researchers have been investigating methods into automatically process stomatal images to obtain the morphology of related phenotypic cells.
As early as the 1980s, a group of scholars began to study the automatic stomata measurement through digital image processing.Individual stomata are acquired by using inverse Fourier transform and Hamming filter (Omasa et al. 1984).
With the development of microscopy and image processing techniques, a large number of processing methods have emerged in the last 20 years.In 2008, it was proposed to use microscopy image processing techniques to observe the structure of stomata and then achieve the classification of different tomato types based on the stomatal structure (Sanyal et al. 2008).In 2014, the stomata of wheat were identified by template matching, whereas when using other species templates needed to be recreated and parameters modified (Lage et al. 2014).Later, cascade object detection learning algorithms were proposed to obtain stomatal numbers and grapevine stomatal morphological parameters by binarization and skeletonization methods (Jayakody et al. 2017).However, the disadvantages are that the model could not be widely applied to a variety of plants, and this is a large error in obtaining stomatal morphology, and the performance was aperture when stoma contour was not obvious.
In recent years, with the development of deep convolutional network and its superior performance in image recognition, many excellent methods have emerged in stomatal image processing.For the study of Arabidopsis epidermal cells, a highthroughput imaging method was established to quantify intact epidermal tissue morphology, and an algorithm was developed for object detection and quantification in microscopic images based on Acapella (Saloman et al. 2010).For wheat and poplar studies, the density and size of their stomata were predicted by semantic segmentation to calculate the maximum conductivity of the stomata (Gibbs et al. 2021).Combining stomatal region detection and image segmentation of stomata built the stomatal detection tool DeepSmomata and fully automated this process (Toda et al. 2018).The stomata were counted by DCNN (Diffusion-Convolution Neural Network) (Fetter et al. 2019) and a stomatal counter was built to facilitate researchers to upload and calculate their own data.Based on the object detection method, the identification of stomata using Faster R-CNN (Faster Region Convolutional Neural Network) was proposed, and then the acquired stomatal features were processed by CV (Computer Vision) model to obtain the surface features of the aperture (Li et al. 2019;Liang et al. 2021).Later, the original automatic detection method was improved using Mask R-CNN (Mask Region Convolutional Neural Network) deep semantic segmentation network and better measurement results were obtained (Song et al. 2020).
To systematically obtain phenotypic data of stomata, an end-to-end stomatal detection and identification framework based on feature weight transfer learning and YOLOv4 (You Only Look Once version 4) network was developed (Yang et al. 2021), which can locate stomata based on horizontal detection boxes and obtain phenotypic data such as the number, length, and width of stomata.To accelerate the selection of drought-tolerant oil palm breeders, MobileNet was used as a template to automatically detect stomata in oil palm to obtain the density of stomata in drought-tolerant oil palm (Kwong et al. 2021).
To dynamically detect the behavior of stomata in wheat leaves, the SBOS system was constructed based on the U-Net network (Sun et al. 2021).In a recent study, a tool for plant stomatal localization and phenotypic data collection was developed based on Mask RCNN (Sai et al. 2023), called StomaAI (SAI), which and assessed the accuracy of the tool by comparing its prediction results with manual measurements by botanical experts.Fall Armyworm detection system (Kasinathan et al. 2023) is also proposed based on Mask R-CNN to detect pests for crop quality and safety.These intelligent methods are proposed to provide many facilities for the study of plant growth conditions.However, all detection methods are based on horizontal detection boxes or use image segmentation methods.These methods are tedious in the preparation of data sets or in the acquisition of stomatal phenotype data and difficult to achieve high accuracy in the high-throughput acquisition of stomatal phenotype data.
In this paper, an end-to-end objected automated porosity detection system, RotatedStomataNet, is proposed and built on the R 3 det network (Yang et al. 2021).It is designed to automatically detect the number of plant stomata and apertures size at high throughput, and obtain phenotypic data such as length, width, and density of the object.The system can automatically detect monocotyledonous plant stomatal images or dicotyledonous plant stomatal images in single or batch, and optionally save the output stomatal or aperture phenotype data (including localization map, aspect ratio data, density, aperture ratio, etc.).The main interface of the system is shown in Fig. 1.This system can easily and efficiently detect the phenotypic data of stomata and apertures, which helps to promote the research and development of plant stomata by.

Materials
A total of three data sets are used in the system training phase of this study, including microscopic images of Arabidopsis obtained by the transparent tape adhesion method, microscopic images of maize obtained by the nail polish blotting method, and nondestructive images of maize taken directly with a handheld microscope.
To obtain microscopic images of Arabidopsis stomata (destructively images), first, pull a 3-4cm piece of transparent plastic tape with the sticky side facing up and lay it flat on the lab bench.Second, place the top surface of the Arabidopsis positioning sampling material onto the tape and then fold the tape over and stick it onto the underside of the leaf surface.
Pinch the tape with your fingers to ensure full contact between the tape and both surfaces of the leaf.Third, tear open the folded tape and stick the tape with the epidermis onto a clean glass slide without a cover slip.Observe and capture images directly under a microscope.
To obtain microscopic images of maize (destructively images), first, select a maize leaf and apply nail polish evenly onto its surface.Let it air-dry naturally for 3-8 minutes until the interface between the nail polish layer and the leaf appears to be slightly detached.Then use tweezers to grab the edge of the dried nail polish layer and peel it off.Second, lay the nail polish layer, which is in contact with the leaf, flat in a drop of uniform glycerol layer and cover it with a cover slip.Third, use fingers to cover the cover slip with filter paper and gently press it to flatten the nail polish layer with stomata in the glycerol layer.Squeeze out any air bubbles and use filter paper to dry the glycerol around the cover slip.Observe and capture images under a microscope.Both data sets are captured using a ToupCam s500-GS microscope.Stomatal images of maize obtained in a nondestructive manner, i.e., directly on the maize leaves with a handheld microscope (ProScope HR5, Monocotyledonous nondestructively, Fig. 2).Fig. 2 Imaging devices and acquired nondestructive maize data set.All images are annotated by RolabelImg to obtain the XML tag file corresponding to each image, and the manually labeled data are used as "Manually labeled data".In order for the system to better learn the feature information of stomata and apertures, each image is segmented, and the system will be trained and evaluated on the segmented data sets.A summary of the stomatal data sets used in system training and evaluation is shown in Table 1, and the stomatal data sets used in the testing phase are shown in Table 2.The stomata and apertures of Arabidopsis (Dicotyledonous Arabidopsis destructively ".) and maize (Monocotyledonous maize destructively and Monocotyledonous maize nondestructively were submitted on maize data set are closed) are schematically shown in Fig. 3 (These three types of test data sets have been given on Electronic Supplementary Material).

Methods
The innovation of this work is the first use of a rotated object detection network to identify objects such as plant stomata and apertures size and to obtain phenotypic data on stomata and apertures.RotatedStomataNet, an end-to-end, highthroughput stomatal detection and analysis system, was also built.The system is divided into monocot maize (destructively image or nondestructively image) and dicot Arabidopsis (destructively image, and because there is no nondestructively image data set of Arabidopsis, so this system can't directly detect the nondestructively image of dicots), and the data type for detection can be divided into single image detection mode and batch image detection mode, i.e., it provides users with a convenient and multi-choice tool for plant stomatal detection and identification, and achieves high-throughput acquisition of phenotypic data of plant stomata.In addition, the method is the first end-to-end, high-throughput acquisition of density information and phenotypic data of stomata and apertures in images without two steps (detection first, processing later).The method no longer uses classical digital image processing methods, avoiding the influence of image noise and complex background information on the detection results during the recognition process.In terms of acquiring plant stomatal phenotypic data, the system is more convenient and better fits the shape of stomata than the object detection method using horizontal frames.The training set (labeled data sets) of this system is processed in a shorter process than the method using image segmentation techniques for stomatal phenotypes.This method can also be quickly adapted to different stomatal sizes of crops with better results (Table 4, Fig. 6c).This has important implications for the future extension of the method to other plants.The overall flowchart of this paper is shown in Fig. 4a.

Composition of RotatedStomataNet
In our work, RotatedStomataNet is a rotated object detection system used as a method to achieve stomata and apertures detection and phenotype acquisition.Plant stomata exhibit irregular distribution, uncertain aspect ratio and multiple angles on leaves.The rotated object network is able to locate the object at multiple angles and effectively separate it from the background.The rotated detection frame also fits the boundaries of the stomata better compared to the horizontal object detection method.Therefore, RotatedStomataNet aims to solve the stomatal distribution problem and achieves a multiselective end-to-end detection of single or multiple stomatal images and outputs phenotypic data such as stomata, and the network structure is shown in Fig. 4b.
From Fig. 4b, we can see that the training stage of the system consists of two network modules, namely the feature extraction module and the feature refinement module (FRM).In the feature extraction module, the Feature Pyramid Network (FPN) is used as the backbone network to extract features of the object for predicting the object's category and position, and generating classification loss and regression loss (0. , 0. ).
In the feature refinement module, the current refined boundary box position is overlaid on the feature map through bidirectional convolution to obtain new features.At this stage, the IoU (Intersection over Union) manner (Fig. 4c) is used to measure the difference between the detected box and the ground-truth box in the regression branch, and only the boundary box with the highest score at each feature point is kept to improve detection speed.Then, the feature vector corresponding to the feature map is obtained using the coordinates of the five feature points (one center and four corners), and classification loss and regression loss (0. , 0. ) are generated.The more accurate feature vector is obtained using bilinear interpolation (Fig. 4d), and after traversing the feature points, the entire feature map is reconstructed,  the trained weight file is used for detection.The hardware environment used in this study is the PC with an Intel i7 processor, a256GB SSD hard drive, and an Intel(R) Xeon(R) Gold 5218R CPU @ 2.10GHz.

Evaluation indexes of the object detection system
The accuracy of the system in identifying stomata or apertures is the primary indicator in evaluating the performance of the system.We evaluate the detection performance of the model using various metrics including precision, recall, F1 score ( 1 ), average precision (AP), and mean average precision (mAP).
Here,   indicates the number of stomata or apertures correctly detected as a region,   indicates the number of stomata or apertures that are incorrectly identified as stomata or apertures (i.e., false detections),   indicates the number of correct background regions as background regions, and   indicates the number of stomata, or apertures that are incorrectly identified as background resions (i.e., missed detections), where  is the number, indicates the current object.
To evaluate the accuracy of the system in obtaining stomatal phenotype data objectively, it is desirable for professional researchers to obtain a reliable set of manually measured data for comparison with the data obtained by the system.
Considering the possible errors introduced by the non-standard unit conversion when using ImageJ software to obtain the surface of the stoma, it is finally decided to use pixels as the unit of measurement data and to use RolabelImg to obtain manually labeled data (i.e.,   ,   ).Here,   and   represent the manually labeled data and the length or width values of the predicted stomata or apertures of the RotatedStomataNet (  is the length value, and   represents the width value), and define the aspect ratio as ℎ/ℎ  =   /  .
To determine whether the phenotypic data predicted by RotatedStomataNet are consistent with those measured by researchers, the CCC (consistency correlation coefficient, ranging from -1 to 1) metric is used for evaluation.Considering that plant stomata and apertures are elliptical in the image, the long axis of the ellipse is usually called the length of the stomata and the short axis of the ellipse denotes the width of the stomata.Our detection box is the outer rectangle of stomata or apertures, i.e., we assume that the length and width information of stomata and their apertures are the length and width of their outer rectangles.Based on this assumption, apart from the parameters used to evaluate the apertures detection and counting capabilities mentioned above, length accuracy ( ℎ ), width accuracy ( ℎ ), average length accuracy (.ℎ ), average width accuracy (.ℎ ), mean square error (MSE), and root mean square error (RMSE) are introduced to evaluate the accuracy of the detected stomata and aperture phenotype data.And the calculation formula is as follows: In Eq. 9, the means of  and  are expressed as   and   , the standard deviations are expressed as   and   , and the covariance between them is expressed as  (,) .

Imaging device for acquiring nondestructive data sets of stomata
An imaging device (Fig. 2) is designed to acquire image data of plant leaf stomata in real time.The ProScope HR5 is fixed on a stand so that it irradiated and photographed the growing plant leaves directly, and transferred the photographic data to a computer.

Detection average precision (AP) of stomata and apertures.
In this study, the test sets obtained in a destructive manner (microscopic images of Arabidopsis (Arabidopsis thaliana) obtained by the transparent tape adhesion method and microscopic images of maize obtained by the nail polish blotting method) are first predicted, and the results obtained are shown in Table 3 and Fig. 6a.
As can be seen from Fig. 6a, the constructed system can accurately detect stomata and apertures, and can adapt the distribution angle of stomata and apertures so that the detection box fits the edges of stomata and apertures completely.
From the Table 3, the recall of Arabidopsis stomatal system is 97.1%, the precision is 98.2%, the AP (average precision) is 90.7%, the recall of Arabidopsis apertures is 92.3%, the precision is 97.3%, and the AP reaches 87.8%, and the F1 scores for these two types of objects are 0.976 and 0.947, respectively.The system has a recall of 99.6%, a precision of 97.1%, an AP of 90.7%, and an F1 score of 0.983.Fig. 6b shows the labeled of the same object by the manually labeled box (green) and the RotatedStomataNet detection box (blue) as an example of a single stomatal map of Arabidopsis, and it can be seen that the detected box and the manually labeled box almost overlap and fit closely to the edge of the object.The system is trained using the Arabidopsis and maize data sets, respectively, and then tested on the test set, and good results are obtained.To verify the generalization of the system for stomata detection of different species, the soybean stomata are directly detected using weights trained on the Arabidopsis (dicotyledonous plants) data set, and the wheat stomata are detected using weights trained on the maize (monocotyledonous plants) data set, and it can be seen that the system can detect stomata and apertures in the data set.The detection box is also very suitable for the stomatal profile and aperture size, and for different images, the system can also detect the object well (Table 4, Fig. 6c).

Measurement accuracy and error of prediction results and manually labeled results
For the data predicted by RotatedStomataNet, data that satisfies   > 0.55 (represents width or length, Eq.10 and Eq.11) is filtered out.Predictions are made for 15 randomly selected destructive Arabidopsis images and 25 destructive maize images to obtain their phenotypic data and to calculate the measurement accuracy of length and width (The manually labeled data is compared with the RotatedStomataNet predicted data).Histograms are plotted for the predicted accuracy of the detected Arabidopsis stomata and apertures as well as the length and width data of the maize stomata (Fig. 7a).It can be seen that the measurement accuracies are all above 85%.And the prediction accuracy, mean square error and root mean square error and root mean square error of length and width are calculated and plotted (Table 5).There are 169 stomata and 149 apertures of Arabidopsis and 268 stomata of maize selected and recorded.The CCC map and error map are shown in Fig. 7b, c, d, e, f and Fig. 8, respectively.As shown in Table 3, the length and width accuracy of maize stomata are 93.64% and 93.44%, respectively.The length and width accuracy of Arabidopsis stomata are both above 95%, and the length and width accuracy of stomata are 95.01%and 95.31%, respectively, and the length and width accuracy of apertures are 88.34% and 87.58%, respectively.
The mean square error (MSE) for each type of object is also very small.The MSE for the length and width of Arabidopsis stomata are 1.2052 and 1.0420, respectively, while the MSE for the length and width of its aperture are 1.0142 and 0.3756, respectively.The MSE for the length and width of maize stomata are 6.4921 and 5.5028, respectively.The largest MSE is for the length of maize stomata, while the smallest is for the width of Arabidopsis stomata.The root mean square error (RMSE) is largest for the length of maize stomata, which is 2.5480, while the RMSE for the width of Arabidopsis apertures is the smallest at 0.6129.It can be reasonably concluded that the predicted data from RotatedStomataNet can effectively replace manually labeled data, saving the cost of human labor during plant stomata research.Fig. 7b, c, d, e, f and Fig. 8 present a comparative analysis between manually labeled phenotypic data of Arabidopsis and maize and the predicted data from RotatedStomataNet.These figures include consistency correlation coefficients, relative errors, and counting fitting graphs.Fig. 8a represents the count of Arabidopsis stomata, while Fig. 8b depicts the count of Arabidopsis aperture sizes.The fitting graph for maize stomata count can be found in Fig. 7b.It can be seen that the CCC values for maize stomatal width and length data are 0.824 (Fig. 7c) and 0.845 (Fig. 7d), and the relative errors are -0.242(Fig. 7e) and 0.559 (Fig. 7f), respectively.The CCC values for Arabidopsis stomatal width and length data are 0.870 (Fig. 8c) and 0.900 (Fig. 8g), and the relative errors are -0.344(Fig. 8e) and -0.227 (Fig. 8i), respectively.The CCC values for Arabidopsis apertures width and length data are 0.831 (Fig. 8d) and 0.869 (Fig. 8h), and the relative errors are 0.077 (Fig. 8f) and -0.033 (Fig. 8j), respectively.It can be observed that the prediction for Arabidopsis stomatal length is the best, with a CCC value of 0.900 and a relative error of 0.824, indicating that the overall RotatedStomataNet predicted data for this object is slightly lower than the manually labeled data.The worst prediction performance is for maize stomatal width, with a CCC value of 0.824 and a relative error of -0.242, indicating that the overall RotatedStomataNet predicted data for this object is slightly higher than the manually labeled data.The prediction for Arabidopsis stomata is generally better than that for maize, which may be due to several reasons: first, due to the growth characteristics of maize itself, there is a large difference in stomatal morphology.Secondly, some maize epidermal cells used in this experiment had bubbles in the cover glass during imaging, which are not completely removed, resulting in impurities in the captured microscope images and affecting the imaging quality.Finally, the training data set for maize is relatively small, and if a larger data set is used for training, the system's prediction results will be improved.
Table 5 Accuracy of length and width of stoma and aperture tests.

Performance evaluation of the system for detecting stomata and apertures of destructive data sets
A boxplot (Fig. 9a, b, c, d) is generated to compare the differences in density (Fig. 9a), width-length values (Fig. 9b), area (Fig. 9c), and aspect ratio (Fig. 9d) between the manually labeled data and the RotatedStomataNet predicted data for Arabidopsis and maize.No significant differences are found based on the statistical analysis in the plot.The predicted values for the length, width, and density of Arabidopsis stomata are slightly higher than the manually labeled values.The predicted aspect ratio of the Arabidopsis apertures is more concentrated around the median than the manually labeled values.
The predicted density of maize stomata is more concentrated around the median than the manually labeled values, with no significant difference in the median.The prediction performance of RotatedStomataNet system can replace the manually labeled of stomata and apertures in experiments, saving a lot of labor costs in biological research and facilitating the study of stomata and their growth and development by biologists.

Performance evaluation of the system for detecting stomata and apertures of nondestructive data sets
In the previous study, a lot of effort is put into the detection of plant stomata in microscope images, while in fact, the method of acquiring microscope stomatal images is more complicated and the change of plant stomatal size cannot be observed in real time.Therefore, we again trained and predicted the stomatal images of maize obtained in a nondestructive  way (directly on the leaves of the plant, as in Fig. 2), and obtained phenotypic data such as the length and width and density of stomata.The RotatedStomataNet system is found to be well adapted to this task and nondestructive to obtain real-time, nondestructive stomatal images to better support research in areas such as plant physiology and ecology.A detailed plot of the results of the RotatedStomataNet system for stomatal detection of different plants is shown in Fig. 9e.

Errors of the detection boxes
It can be seen from the experimental results that our system has a good performance on the data sets of different types of stomata and is suitable for most stoma detection.Moreover, due to the absence of traditional image processing methods in our system, there is a lower requirement for the quality of the images being detected.
Meanwhile, our system also has errors in some object detections, as shown in represent the prediction error of apertures, mainly due to small aperture size and abnormal aperture shape.The reason for the errors is generally that the length and width of the predicted box are too different from the actual object, or the predicted angle is too high.In the future, the accuracy of the system also needs to be improved.
This paper takes advantage of the excellent performance of deep convolutional network and the current advanced object detection network, which does not need to set parameters according to specific objects, and is relatively excellent in terms of computing speed.After testing, the average time cost of our system for each object is about 0.409s (Windows 10, environment, PyTorch, Quadro RTX 5000), should be the fastest stoma detection measurement network at present, which will greatly improve the research speed of researcher.

Comparison with the State-of-the-Art
Table 6 An accuracy comparison of several typical stomatal detection models is shown (using the Arabidopsis dataset as an example).Other existing methods for detecting plant stomata, trained and predicted on Arabidopsis data sets, and compared the predictions with the methods used in this paper.The CenterNet, Faster R-CNN, SSD (Single Shot MultiBox Detector) network and YOLOv4 network are included to obtain the detection accuracy of each model.

Stoma
Among them, the CenterNet can only obtain positioning information, and the accuracy of the predicted apertures and its location are 73.33% and 0.43%, respectively, indicating poor ability to locate small objects such as apertures.The Faster R-CNN and SSD network also have low precision in locating small objects like apertures, but their accuracy in locating stomata is above 85%.The Faster R-CNN is only used to locate the location of the stomata and then segment the detected stomata using the CV model to obtain phenotypic data (Li et al. 2019;Liang et al. 2021).
Therefore, it becomes challenging to simultaneously obtain both the location and phenotype data of apertures.The YOLOv4 network can obtain the location information and phenotype data of stomata and apertures, with an accuracy rate of 93.54% and 30.19%, respectively.It can be seen that the accuracy of apertures location is also relatively low.The YOLOv4 network was only used to obtain position information on monocotyledon stomata and did not involve the location of their pore size and the phenotype of dicotyledons (Yang et al. 2021).Moreover, the phenotype data of the detected stomata is only based on the length and width obtained from the horizontal bounding box, which has large errors compared with the actual data.Fig. 10e shows the histogram of the performance of different network models in predicting the stomata and apertures of Arabidopsis.
It can be seen that the proposed system is able to obtain the localization accuracy and phenotype data of both stomata and apertures in one step, with an average precision of 89.3% for both stomata and apertures (Fig. 10e, Table 6).The localization accuracy of small objects such as apertures can reach up to 87.8%, which is higher than most other methods.
The localization accuracy of stomata is also above 90%.Therefore, our system is easy to use and has higher accuracy than most other tools.
In a recent study, Na, S. et al. ( 2023) developed a stomata image detection application, SAI, for Arabidopsis and barley data sets.We used the SAI application to predict our Arabidopsis test set and compared the results with our RotatedStomataNet system and manually labeled data (Fig. 10b, c, d).From this, we can see that the SAI tool can adapt to our data set, but there are still some missed detections (indicated by purple boxes on the objects)

Conclusions
This article creates the RotatedStomataNet system that enables end-to-end, batch, rotated, and real-time detection of plant stomata and apertures phenotype data, with the ability to output and save prediction results according to user requirements.
The system is based on the characteristics of maize stomata in monocotyledonous plants and Arabidopsis stomata in dicotyledonous plants, and has obtained results consistent with manually labeled.At the same time, the system can also detect wheat and soybean stomata microscope images with excellent generalization performance.In addition, the system can detect nondestructive maize stomata images in real-time, indicating that it is convenient, adaptable, and efficient, and can save a lot of manual labor costs.Which is of great significance to biologists studying plant stomatal features and growth characteristics and has strong practical implications.
Currently, although real-time detection of maize stomatal images is achieved as much as possible, the task of real-time detection of plant stomata is still almost exclusively focused on images.Real-time monitoring of the growth process of stomata and apertures size in video would better assist biologists in their research work if it is possible.For real-time monitoring of plant stomatal growth processes, our team is focusing on and has achieved some preliminary results.
Preliminary results on the detection of stomatal changes are given in a further supplemental video (Movie S1: Stomatal diameter changes in peanut after Abscisic Acid (ABA) induction).This video shows the changes of ABA-treated peanut stomata taken with a handheld microscope.In addition, there is still room to improve the accuracy of rotated object detection.These areas deserve further exploration.

Key Message
Innovatively, we consider stomatal detection as rotated object detection and provide an end-to-end, batch, rotated, realtime stomatal density and aperture size intelligent detection and identification system, RotatedeStomataNet.

Fig. 4 Fig. 5
Fig. 4 Method flowchart and network structure.(a) is a flowchart (Arabidopsis stomata diagram as an example).(b) is the network structure diagram in the algorithm.(c) is an example of an IoU.(d) is the diagram of bilinear interpolation.

Fig. 5a and
Fig. 5a and 5d show the change curve of the overall loss function value of the algorithm during the iterative process of training.In this paper, we utilize pre-training weights obtained from the DOTA data set for 24 iterations.These weights are subsequently employed as initial weights for the network during the training phase.During the training, SGD (Standard Stochastic Gradient Descent Solver) is used with an initial learning rate of 0.004, a momentum of 0.9, and a weight decay rate of 0.0001.To improve the detection performance of the system, each image is cut into multiple images of size 800*800 for training.For all training sets, 100 images are trained each time, and the loss function value is output once, with a total of 100 epochs trained to obtain (⌊  100 ⌋ × 100) ( representing the number of images used for training) loss values.Finally,

Fig. 6
Fig. 6 Compare the original images with the detection images.(a) is the original plots of Arabidopsis and maize stomata (top) with the effect predicted in RotatedStomataNet (bottom).(b) is the original plots of Arabidopsis stomata and apertures (top) with the effect predicted in RotatedStomataNet (bottom, blue boxes are the RotatedStomataNet detection box, green boxes are the manually labeled box).(c) is the original plots of soybean and wheat stomata (top) with the effect predicted in RotatedStomataNet (bottom)).

Fig. 7
Fig. 7 Measurement accuracy and error of prediction results and manually labeled results.(a) is the mean average accuracy of length and width of stoma and aperture tests.(b-f) are the scatterplots and error plots of maize manually labeled data and system prediction data (counts (b), stomatal width (c, e) and length (d, f)).

Fig. 9
Fig. 9 Box plot-comparison of manually labeled data and RotatedStomataNet prediction data of Arabidopsis stomata and apertures and maize stomata for density (a), width-length value (b), area (c), and aspect ratio (d).(e) shows the stomatal detection results for different plants.
Fig. 10a.In the Fig. 10a, the first three examples (from left to right) show the prediction errors of stomata, where the yellow arrow indicates the actual width, the red arrow indicates the actual length, the purple arrow indicates the predicted width, and the blue arrow indicates the predicted length.The first example in Fig. 10a represents the error caused by predicted size, and the second and third examples describe the length-width error caused by predicted angle error.The last three examples (from left to right)

Fig. 10
Fig. 10 Errors of the detection boxes and detection results for different models.(a) has some examples of prediction errors in bounding boxes (From left to right, the first three examples represent prediction errors in stomata, the last three examples represent prediction errors in apertures).(b) is the manually labeled data.(c) is the RotatedStomataNet prediction data.(d) is the SAI prediction data.(e) is the detection average precision bar chart for different models.

Table 1
Summary metrics for stomata and apertures dataset used in system training and evaluation.

Table 2
Summary of datasets used for system testing.

Table 3
Prediction performance in RotatedStomataNet system.

Table 4
Soybean and wheat dataset detection average precision.