Wheat Posture Acquisition System Based on Infrared Tracer Robot Design and Experiment

Background:To investigate the effect of drought stress on wheat posture. Methods:An image acquisition system based on an infrared tracing robot was developed and a graphical user interface (GUI) software was designed to simplify the operation control of the robot. In this experiment, three genotypes of wheat, Ruihuamai 523, Jimai 22 and Xumai 33, were grown in indoor pots, and the images of wheatposture from owering stage to maturity were collected to extract morphological parameters such as plant height, stem width and leaf inclination angle. Results:The experimental results showed that the deviation of linear trajectory was less than 3 mm when the robot traveled in a straight line at 0.4 m/s in the greenhouse; the image acquisition eciency was about 18 images/min;the collected pictures of drought-stressed and control potted wheat groups can be used for posture assessment; the accuracy of wheat plant height and stem width with manual acquisition was 84.6% and 79.2%, respectively. After statistical analysis, it was concluded that drought stress and genotype and other inuencing factors had no signicant effect on plant height and stem width, but had a greater effect on leaf inclination angle. Conclusions:Therefore, this system can be used for posture collection of wheat.


Introduction
As a globally important food crop [1], wheat has important phenotypic traits such as plant height, stem width and leaf inclination that can effectively re ect the growth, stress and nutrition conditions of the crop [2].The external trait of wheat growth is the posture of wheat [3], and like animal posture [4], the posture of wheat undergoes dramatic morphological changes throughout the reproductive period, including the number of leaves, tillers and ears of wheat, etc. This is a major reason for the variation in crop posture when plants change their growth curvature to allow their aerial organs to resist the groundward motion that occurs when gravity is applied [5]. Wheat plants are sensitive to drought stress, and any degree of drought in any period can cause wheat yield reduction or even crop failure [6]. Under normal growth conditions, postural changes such as leaf curl in wheat are mainly caused by drought stress. Therefore, collecting images of drought wheat postures, identifying the degree of drought stress on wheat plants, and quantifying this change through posture assessment, can then analyze the physiological state of plants, which is of great signi cance for drought prevention and mitigation [7].
However, in the eld, the large number of samples is not conducive to gesture recognition. Therefore, we adopted pot planting method, set the potted wheat in the drought group and the control group, and obtained the posture picture of the wheat by collecting the side view of the potted wheat.
With the development of computer vision [8], image processing technology has been widely applied, and image acquisition is rstly required before image processing, and crop image acquisition is mainly done through computers, networks, various sensors, etc. to acquire and transmit crop growth images [9]. In crop monitoring, most of the previous work has used xed-wing aircraft with cameras and airborne LiDAR for crop monitoring and saving raw point cloud detail information [10], but phenotype imaging techniques such as UAV platforms focusd on the canopy scale and cannot acquire plant images in the vertical plane.
Raspberry Pi-driven phenotypic imaging techniques were also applied, which could use low-cost microcomputers and cameras to acquire high-throughput plant image data [11].The current crop plant height acquisition is mainly estimated by two methods: depth cameras [12] and 3D point cloud models [13], which are less accurate and cannot obtain lateral phenotypic traits of the crop. In terms of high-throughput phenotyping platforms [14], indoor crop phenotyping platforms have the advantage of accurate and real-time regulation of various environmental factors and simulation of different crop growth environments compared to outdoor phenotyping techniques. Depending on the application scenarios, indoor phenomics research facilities can be used to achieve self-service and scale-up research goals by collecting and analyzing phenotypic information of plants under controlled environmental conditions. These phenotype collection platforms can be divided into large and small and medium-sized indoor plant phenotyping devices, with the former including conveyor and track type, etc., and the latter generally being desktop or gantry structures [15]. However, desktop phenotyping equipment can only collect phenotypic data from the top of the crop and requires manual crop sample replacement, which is ine cient. Conveyor type delivers crop samples by conveyor belt and can collect phenotypic data from multiple angles of the crop, which is e cient but expensive. Gantry type is e cient but bulky and in exible. The traditional crop phenotype information collection is mainly based on manual collection, which is characterized by low e ciency, small sample size, large error and poor adaptability, which seriously restricts the modern crop phenomics research.Therefore, the development of an image acquisition robot that can accurately and quickly acquire images of crop poses and estimate parameters such as crop plant height is important to grasp crop growth status, trends, and crop yield estimation.
Based on the above analysis, this paper developed a greenhouse crop phenotype picture collection system based on infrared tracing robot, using Raspberry Pi development board as the core controller, realizing indoor tracing walking and xed-point photography with the help of infrared sensor and USB camera, using Wi-Fi module to realize the interaction with the remote monitoring terminal of robot operation status, and storing pictures. Three groups of potted wheat with different genotypes of Ruihuamai 523, Xu mai 33 and Jimai 22 cultivated indoors were used for the study, and the infrared tracing robot was used to collect images of wheat from owering stage to maturity stage without damage, and a comparison of the posture of each genotype wheat drought stress group and control group was obtained. Morphological parameters such as plant height and stem width were extracted after image pre-processing as well as Matlab pixel comparison algorithm, and wheat plant height and stem width of the three genotypes were measured manually by the experiment, and the predicted values of the image acquisition system were compared with the manually measured values for analysis. Finally, statistical analysis was performed to analyze the signi cance of two in uencing factors of treatment method and genotype with plant height, stem width and leaf inclination factors by ANOVA and LSD post hoc analysis.

Test technology roadmap
Robot structure and function design The schematic diagram of the crop image acquisition robot structure is shown in Fig. 2. The overall structure of the robot mainly consists of an image acquisition platform and a vehicle chassis located at the upper end [16]. In the image acquisition platform, the USB camera performs image acquisition, the camera bracket provides support for the camera and reserves two degrees of freedom for steering, and two servos are connected to the bracket to provide steering power in the direction of two degrees of freedom, respectively, so as to realize multi-angle crop phenotype image acquisition. The vehicle chassis consists of two metal plates, a robot controller (Raspberry Pi 4B), and a Raspberry Pi extension with an integrated motor drive board [17], with an infrared sensor at the front end under the chassis, L-shaped DC geared motors and wheels at both ends, and a DC power supply in the middle. As shown in Fig. 3, the image acquisition robot control system can achieve the following functions: i) it can control the trajectory guidance unit to make the robot smoothly complete the near-linear and turning trajectory motion along the black tape guide magnetic strip on the pre-laying foam board [18]; ii) it controls the robot to accurately complete the functions of xed-point parking, turning the rudder, and photographing the xed position crop; iii) it controls the Wi-Fi module to achieve automatic storage of image data after image acquisition is completed; iv) it can automatically adjust camera parameters such as exposure according to the GUI.

Experimental design
The test materials were selected from three wheat genotypes (Ruihuamai 523, Xumai 33, and Jimai 22) cultivated in a greenhouse at the Baima base of Nanjing Agricultural University, and subjected to drought stress from April 21, 2021. 8 pots of each genotype were selected, including 4 pots of drought stress (DS) and 4 pots of control (CK). Twenty-four pots of wheat were discharged on both sides of the runway, with 12 pots placed on one side, 80 cm from the center of the runway, and 1 m between two adjacent pots of wheat.
The runway is xed with black tape to allow the robot to follow the trail forward through the tracing module, and the loop tape at the end point allows the robot to return automatically. Black tape is attached across the line of each pot of plants for the location where the robot stops to take pictures. When the landmark where the crop is recognized, the power unit is turned off to stop forward travel, the camera system is turned on and the phenotypic images of the crop at that location are captured from multiple angles on the side. After the data acquisition is nished, the robot continues to move forward to capture phenotypic images of the crop, in this way and again.Green markers were placed on the black background cloth behind the wheat potted plants [19] to facilitate identi cation and analysis of parameters such as plant height and stem width [20].
After the wheat entered the owering stage on April 7, 2021, each pot of wheat was cleared of excess plants and well-developed single plants were retained. Then the pose image acquisition was started, each pot of wheat was taken at 90°, 95°and 100°from the horizontal reference position of the helm, respectively. The working schematic of the robot image acquisition system is shown in Fig. 4.
Robot trajectory method design Figure 5 shows the trajectory logic diagram of the indoor image acquisition robot. When the middle two infrared probes (X1 and X2) of the four tracing infrared sensors are directly above the black guide line, the robot travels straight and the motors on both sides run at the same speed. When the robot recognizes the landmarks according to the infrared probe side paths in the forward process, it will make the corresponding turning action [21]. And when the sensor recognizes the landmarks, the four infrared probes output low level and the robot stops and takes pictures of the crops on both sides to collect phenotypic images. Due to inertia, when the robot recognizes the landmark, it will drive past the landmark for a short distance before stopping, and the tractor sensor will again be above the black guide line, and the robot will continue to move forward after collecting data.
The robot is susceptible to environmental factors in the process of moving straight ahead, and its trajectory is in nitely close to the black guide line laid with respect to the guide line, showing an "S-like" shape. In order to ensure the running accuracy of the robot along the guide line, the width of the guide line should be as close as possible to the distance between the infrared probes X1 and X2. Because the distance between the two infrared sensors in the middle of the sensor is 15.7mm, we uses 16mm wide black tape to lay the tracing route.

Automatic crop image acquisition software interface
The robot crop image acquisition interface contains manual control mode, camera parameter settings, etc. The python language programming inside Raspberry Pi 4B is used to realize the control of indoor crop phenotype acquisition robot movement, tracing and photo function. The opencv library function is called to control the USB camera for crop phenotype acquisition and image processing. To satisfy users who do not know programming, this paper uses the tkinter library function to design the graphical user interface (GUI) [22]. As shown in Fig. 6, the software interface shows the robot motion control on the left side and the camera control on the right side.
The motion control part is mainly to control the robot's forward, backward and stop, as well as to control the servo rotation of the camera head. In the camera control, the exposure and brightness parameters of the camera are selected by sliding the slider; the location of the photo is input in the storage path; the control software adopts multi-thread programming to ensure that the camera is turned on and the robot's motion control does not con ict with each other; The control interface has "Auto" for the robot to enter the automatic tracing and photographing mode, at which time the robot cannot be controlled manually; "Quit" for the robot to exit the automatic tracing and photographing mode, at which time the robot can be controlled manually.
Extraction process of crop plant height and stem width The extraction of phenotypic parameters such as plant height and stem width can verify the acquisition accuracy of phenotypic robots. The main extraction steps are image pre-processing, image calibration separation and morphological parameter extraction.In order to distinguish the crop image to be processed from the background crop image pre-processing is required rst [23]. The pre-processing stage performs grayscale processing of the image based on the input RGB image, converts the image RGB tricolor component to HSV, and then extracts the wheat plant part by HSV color component, followed by an expansion operation to ll the holes after color extraction, and nally eliminates the noise after the expansion operation to obtain a new binary image [24].
Then target recognition is performed, the crop is identi ed with the calibration by judging the area, and the number of pixel points is calculated by delineating the connected domain.By calculating the ratio of the actual diameter of the calibration to the number of pixel points to obtain the scale factor K1 [25], the rectangle of the connected domain will be identi ed, and the number of pixels in the long rectangle will be summed to N1, and the height of the wheat plant will be K1×N1. When processing the stem width, the rectangle of the connected domain to calculate the stem width should be manually selected, and the wide pixel points in the new rectangle will be summed to N2, and the stem width of the wheat will be K1×N2. Figure 7 shows the image pre-processing and crop height and stem width extraction process.

Statistical analysis
To verify the accuracy of the image acquisition system and the relationship between each parameter and variables such as genotype. In this paper, wheat posture such as plant height, stem width, and leaf inclination were used as dependent variables, different wheat genotypes as xed factors, and treatment methods as random variables, according to a general linear model for ANOVA [26]. To determine the difference in posture between the drought-stressed and control groups, the means were compared with the mean of the LSD (P < 0.05).

Analysis of navigation accuracy test results of the tracking robot
The indoor image acquisition robot detects the position relationship between the center of the vehicle and the black guide line through four tracing infrared sensors, and adjusts the speed of the drive wheels on both sides of the vehicle to achieve automatic tracking function. In this paper, the motion control of the system is tested on the hard surface, and two different motion state tests of straight line and traveling turn are designed to test the motion speed and running position control effect of the platform respectively to verify the reliability of its operation [27]. The motion trajectory of the robot at three speeds of 0.2 m/s, 0.4 m/s, and 0.6 m/s was recorded by the marker pen. The distance between the robot and the black guide line was measured every 10 cm for 20 times at each speed, and the measurement results are shown in Table 2. The deviation of the image acquisition robot speed response is shown in Fig. 8. The analysis shows that the slower the driving speed, the higher the robot's navigation accuracy. Considering the operation e ciency, the 0.4 m/s running speed is selected without affecting the quality of photos. The robot acquisition mode is set to automatic acquisition mode, when setting the travel speed of the tracer robot to 0.4m/s, the rotation time of the servo is 1.2s, the interval time of continuous shooting is 0.2s, the image transmission time is about 0.5s, the camera acquires images about 10s for 3 times, and the e ciency of individual camera image acquisition is about 18 images/min. The test results show that the system can acquire crop images stably and reliably.

Wheat leaf pose acquisition and analysis
To verify the feasibility of the system, the system was tested at the Baima base of Nanjing Agricultural University. From April 7, 2021 to May 7, 2021, the system was used to collect the lateral view images of wheat from owering to maturity, during which 2556 images were collected and 96 samples were extracted. After image enhancement, the different poses of each genotype of wheat were arranged as shown in Fig. 9(a). It was observed that different genotypes of wheat collected by this system showed different posture performance, the top one leaf of Ruihuamai 523 grew horizontally, the top one leaf of Jimai 22 grew upward, and the top one leaf of Xumai 33 had a tendency to grow downward. The enhanced RGB images were clearer, so this dataset can be used for posture assessment and classi cation of wheat [28].
After manual calibration of the angle extraction, it was found that the mean values of the top one leaf angle of three genotypes, two treatments and four sets of replicated trials, namely,Ruihuamai 523, Jimai After 5 days of drought stress, the lateral views of potted wheat collected by the crop image acquisition robot are arranged according to genotype and treatment after image enhancement. To maintain the control variable principle, the same row shows wheat posture with different genotypes for the same treatment, while the same column shows wheat posture with different treatments for the same genotype.
To show the degree of leaf curl more clearly, a local magni cation was added to the leaf part of the image after drought stress treatment for each genotype, as shown in Fig. 9(b).

Image processing and parameter extraction results
In this paper, image processing is mainly studied by pre-processing the collected images for wheat plant height and stem width measurement, and then the accuracy of the gestalt acquisition system is obtained by comparing the predicted plant height test from the collected wheat pictures with the real measurement test of wheat plant height in the greenhouse [29].
In this paper, to extract the predicted value of wheat plant height, rstly, the side view of wheat is preprocessed by graying, binarization, color segmentation, image enhancement, and other functions in MATLAB; then the area judgment is used to identify the wheat plant with the calibration connected domain, as shown in Fig. 10(a); nally, by identifying the four edge points in the green part of the image, the distance between the four edge points is calculated, and the median distance obtained is the length of the connected domain rectangle, which is the predicted value of wheat plant height, and the resultant interface is shown in Fig. 10(b).In this paper, we specify that the measured portion of the single wheat plant height is the vertical distance from the top of the wheat ears to the top of the pot. The actual measurement of wheat stem width is the average of the two values from the spike to one leaf and from one leaf to two leaves, as shown in Fig. 10(c).
Similarly, by identifying the four edge points in the green part of the image, the distance between the four edge points is calculated, and the minimum value of the distance obtained is the width of the connected domain rectangle, which is the stem width of the wheat image.The image processing process and stem width result display interface are shown in Fig. 11.

Accuracy analysis of image acquisition system
In order to check the image acquisition accuracy of the crop image acquisition robot more intuitively, a linear regression analysis was performed on all extracted values of wheat plant height and stem width against the actual ground measurements. Due to the shooting angle, the image will have a certain linear tilt deformation, which is not conducive to the extraction and recognition of target features by the computer; meanwhile, in the vision measurement system, the image will have different degrees of geometric distortion in geometric position, size, shape, and orientation due to the lens manufacturing accuracy, which will affect the measurement accuracy of the vision measurement system [30]. Therefore, in this paper, the predicted values of plant height and stem width were multiplied by the corresponding correction coe cients at owering and maturity stages, and then the predicted values were compared with the measured values to obtain the tted images as shown in Fig. 12.
As can be seen from Fig. 12, the R2 is 0.7151 and 0.6278, respectively, indicating that the plant height acquisition values of the three genotypes are closer to the image extraction values, and the tting functions are signi cantly positively correlated with high accuracy [31]; indicating that the method of acquiring morphological parameters of a single wheat plant in a certain area is feasible based on the automatic crop image acquisition system of the infrared tracer robot.

Results of statistical analysis
The results of SPSS main effects ANOVA for wheat morphological parameters are shown in Table 3. The results showed that the actual measured values of plant height and stem width were signi cantly correlated with the predicted values, indicating that this picture collection system has some accuracy and stability. Drought stress did not signi cantly affect the plant height and stem width of wheat plants at owering, indicating that wheat plants do not change signi cantly in growth parameters such as plant height and stem width after owering. Plant height was signi cantly correlated with stem width, and the results indicate that it is feasible to use a gestalt acquisition robot to take high-de nition digital images for rapid estimation of plant height and stem width of winter wheat [32]. In addition, the top one and three leaf inclination angles were signi cantly correlated with genotype, indicating that different wheat species behave differently posture. The wheat plant height, stem width, leaf one inclination, and leaf three inclination were subjected to LSD post hoc analysis with the genotypes as well as the treatments, respectively, and the height of the histogram represents the mean value of its parameters, and the error bars represent the deviation between the actual measured and predicted values of this parameter, with different letters on the histogram indicating signi cant differences (p≤0.05) determined by LSD. The results were obtained as the signi cant variance histogram shown in Figure 13.
From Fig. 13, it was found that the in uence factors such as drought stress and genotype had no signi cant effect on plant height and stem width, while the leaf inclination angle was more in uential. In fact, winter wheat mainly develops spikelets from owering stage, and its plant height and stem width will not change signi cantly, while the leaf inclination angle may change due to drought stress, then the results of this statistical analysis are consistent with the growth pattern of wheat.

Conclusion
In this paper, an indoor crop pose acquisition robot is developed for the purpose of acquiring indoor crop phenotype image acquisition and exploring the purpose of crop growth patterns under different growth conditions through later processing and analysis of the images. The main work and results of this thesis are as follows.
(1) Determination of the overall scheme of the indoor crop phenotype collection robot and the hardware and software design of the control system. In this paper, we propose the overall mechanism design of the indoor crop phenotype collection robot, using Raspberry Pi 4B as the controller, using infrared tracking sensor to realize the robot's automatic guidance function on the indoor ground, and using USB camera to realize the crop phenotype image collection. The robot is remotely controlled and monitored by remote login via PC or laptop. The control program and human-computer interaction interface of the indoor crop phenotype acquisition robot were written based on Python on Linux operating system.
(2) Development of crop image processing and phenotype parameter extraction algorithms and wheat pose picture acquisition and dataset production. In this paper, the image preprocessing was performed by imcrop, rgb2gray, imdilate, bwareaopen and other functions in MATLAB, the parameters such as plant height and stem width of the collected images were predicted by pixel point calculation, and the predicted morphological parameters of wheat were linearly tted with the real measurements to obtain the accuracy of the crop pose acquisition system. In this experiment, 2556 lateral view images of wheat from owering to maturity were collected using this system, and it was observed that theposture of different wheat genotypes varied, so it could be used forposture assessment and classi cation of wheat.
(3) SPSS statistical analysis. In this paper, wheatposture such as plant height, stem width, and leaf inclination were used as dependent variables, different wheat genotypes as xed factors, and treatment methods as random variables, and ANOVA was performed by general linear model with SPSS software. As well as a post hoc comparison of the means of each parameter with LSD (P ≤ 0.05).  Schematic diagram of the image acquisition robot structure 1.DC reducer motor; 2.Body steel plate; 3.USB Camera; 4.USB camera;5.Raspberry Pi 6.Raspberry Pi expansion driver board; 7.Infrared sensor