Estimation of the chlorophyll concentration in sorghum using three high throughput phenotyping imaging techniques


 BackgroundLeaf chlorophyll content plays an important role in indicating plant stresses and nutrient status. Traditional approaches for the quantification of chlorophyll content mainly include acetone ethanol extraction, spectrophotometry and high-performance liquid chromatography. Such destructive methods based on laboratory procedures are time consuming, expensive, and not suitable for high-throughput phenotyping. High throughput imaging techniques are now widely used for nondestructive analysis of plant phenotypic traits. In this study three imaging modules, namely, RGB, hyperspectral, and fluorescence imaging, were used to estimate chlorophyll content of sorghum plants in a greenhouse environment. Color features, spectral indices, and chlorophyll fluorescence intensity were extracted from these three types of images, and regression models were built to predict leaf chlorophyll content (measured by a handheld leaf chlorophyll meter) from the image features. ResultsModels that included two additional variables, DAS (day after sowing) and SLW (specific leaf weight), were also investigated to improve the prediction of chlorophyll. R2 for chlorophyll concentration for multiple linear models at various color components were 0.77 for R, 0.79 for G, 0.70 for B. To obtain additional spectral information, color component H, S, and I were calculated after color spaces being transformed. The result of HSI space showed that R2 for chlorophyll concentration for multiple linear models were 0.67 for H, 0.88 for S, 0.77 for I. The R2 values for different hyperspectral index like the ratio vegetation index (RVI), the normalized difference vegetation index (NDVI), modified chlorophyll absorption ratio index (MCARI) between 0.77 and 0.78. R2=0.79 was obtained with fluorescence image. Partial least squares regression (PLSR) was employed to using the selected vegetation indices computed from different imaging data to estimate the chlorophyll concentration for sorghum plants. Among all the imaging data, chlorophyll content was predicted with high accuracy (R2 from 0.84 to 2.92, RPD from 2.49 to 3.58). ConclusionAccording to the Akaike's Information Criterion (AIC) error function, the model was better fitted based on images, DAS and SLW than that based on images and DAS. This study indicated that the accuracy for chlorophyll estimation was increased by the image traits combined with DAS and SLW. High throughput imaging provides a simple, rapid, and nondestructive method to estimate the leaf chlorophyll concentration.


6
In this paper, we present our work for non-destructive estimation of leaf chlorophyll content in sorghum using image-based traits derived from several imaging modules. We used a mini diversity panel of 15 sorghum genotypes exhibiting large variations in physical and physiological traits. The objectives of this study were to: (i) develop and validate an image analysis-based approach for nondestructive measurement of chlorophyll for individual plants, (ii) investigate how the chlorophyll estimation could be improved by including other auxiliary variables including DAS (day after sowing) and SLW (specific leaf weight), and (iii) to evaluate the potential use of the high through phenotypic images as a rapid tool to estimate spatial distribution of chlorophyll concentration within sorghum.

2.1Plant material and growth conditions
Fifteen different sorghum genotypes (20 plants each, 300 plants total) were used in this study, and the goal was to create a large variation in plant leaf property to validate HTPP image-based measurement.
The experiment was conducted at the Greenhouse Innovation Center of the University of Nebraska-Lincoln starting Jan 2019. Data collection occurred in April of 2019.
The temperature in the greenhouse was regulated between 25 and 27゜C during daytime and 20-22゜C during the night time. Relative humidity was maintained at ∼60%. The daily light intensity resulting from natural sunlight and the supplemental LED peaked at ∼350 μmol/m 2 /s photosynthetically active radiation. The supplemental LED had a photoperiod set to 12 hours. The pots used were 25.72 cm in diameter and 23.18 cm in height, with a capacity of 8.52 L. The pot substrate was made by mixing Fafard germination soil and water. The visual differences between DAS were pronounced (Fig. 1). Figure. 1 Photos of the planted sorghum with different DAS (days after sowing).
The experiment was designed as a two-factorial design, replicated five times, with the water and nutrition regime as the two main factors. For the 20 plants in each genotype, 5 plants were randomly selected and assigned to one of the four treatment combinations: drought (D) and high nutrition condition (HN), drought (D) and low nutrition condition (LN), well-watered (HW) and high nutrition condition (HN), well-watered (HW) and low nutrition condition (LN). The estimated volumetric water content of soil for the drought group (maintain 5230 g of pot weight by adding water daily) was 30% and for the well-watered group was approximately 70% of field capacity. During the growth, the plants were watered everyday to the targeted soil water content. The plants were irrigated with two different nutrient solutions twice per week and high-nutrition regime and low-nutrition treatment were as shown in Table 1.

High throughput imaging collection
High throughput images were collected on the sorghums prior to destructive sampling of plant leaf tissues. The sampled leaves were under stresses that affects chlorophyll content. This greenhouse was equipped with a high-throughput plant phenotyping system (Scanalyzer 3D, LemnaTec GmbH, Aachen, Germany) (Ge et al., 2016). The research collected visible (RGB), hyperspectral, and fluorescence images by using three imaging chambers. The imaging modules and their main parameters in these chambers are shown in Table 2. The hyperspectral imaging chamber is illuminated by two banks of halogen lamps (35W, color temperature 2600 K), located on the ceiling above the plant and the other on the wall behind the imaging system. The chambers are designed to permit the imaging of plants up to a nominal height of 2.5 m. During test, the intact plants were loaded onto the conveyer belt and transported to conduct imaging scanning. From each plant, the 3 leaf samples were chosen except flag leaf (which is leaf 1). Leaf 2, 3 and 4 from the plant were cut at the stem and immediately weighed for fresh weight. Leaf Area (LA) of leaf 2, 3 and 4 was determined with a leaf area meter (LI-3100C, LI-COR Biosciences, Lincoln, USA).
Chlorophyll content was estimated with a portable chlorophyll concentration meter (MC-100, Apogee Instruments, Inc., Logan, UT). The MC-100 measures the chlorophyll (Chl) content as a fast, compact and easy approach. It was calibrated to measure chlorophyll concentration in leaves using the sensor's build-in sorghum calibration with unit of Chl was µmol/m 2 . Three sampling areas of approximately 64 mm 2 (circle with 9 mm diameter) were taken from the same leaf for the determination of chlorophyll. That is, every leaf was estimated at the tip, middle and base sections to account for inleaf variability, and the average of the nine spot measurements were regarded as the plant's chlorophyll content value from that plant. The harvested plant leaves were then placed in a walk-in oven at 50°C for 72 hours, followed by the measurement of dry weight.

Phenotyping image processing and data analysis
Image processing of the RGB, hyperspectral, and fluorescence images was done by using Matlab R2017a (MATLAB and Image Processing and Computer Vision Toolbox Release 2017a, The MathWorks, Inc., Natick, Massachusetts, United States). The major task of image processing was to extract plant pixels from RGB, hyperspectral, and fluorescence images from which image-based plant phenotypes can be derived.
In this research, chlorophyll concentration meter MC 100 measuring the amount of Chl in leaf 2, 3 and 4. However, the cameras captured images of the entire plant including all leaves and the stem.
In order to match what the chlorophyll meter measure to the images, efforts were made to confine the analysis on leaves 2, 3 and 4 of each plant image only and calculate their projected area. In the present study, the specific leaf image is defined as the leaf image amount (for example, greenness of image) per total projected area(pixel number).

Visible imaging acquisition
Ten 2454ⅹ2056 resolution RGB images were taken of every plant: ten side view images from every 36 degrees at a horizontal rotation. In order to compare the estimation between chlorophyll content with color features, image processing technique was used and the color components of red (R), green (G) and blue (B) in RGB space and hue (H), saturation (S) and intensity (I) in HSI space were determined. During the color analysis, the HSI space was calculated by using the RGB space to increase the contrast between plant region and background region. Color spaces RGB and HSI can be transformed from one to another easily as illustrated in 'equations (1) -(3)'. A schematic diagram of the image processing procedure is shown in Figure 2.
Segmentation of these images was done by calculating a color index for each pixel and then using a threshold to derive a segmented image. The color index 3*S/(H+S+I) (where H, S, and I denote the hue, saturation and intensity components) was found to be effective in transforming HSI images to a single band images, because this index emphasized the saturation component in HSI pixels, and minimized the effect of non-consistent illumination among different images. A universal threshold of 0.75 was used to segment plant pixels from background.
The resulting image is a binary image, using white and black to distinguish the plant and background regions, respectively. But the binary image was found to contain noise in the form of isolated noise as well as vertical stripes near the edge of the image. Since the frames of the chamber were located in a fixed position for all images from greenhouse, the stripes were eliminated by using the reference image. And then area opening operation was applied to remove the small objects that contains fewer than 200 pixels from the binary image.  Figure. 2 The sequential steps in segmentation of plant pixels from the background A feature-region-based image matching method was proposed, in order to match chlorophyll content from MC 100 in leaf 2, 3 and 4 more efficiently and accurately. From statistics, we evaluated this criterion (5 to 35%) applied to all images to finally realize limitation of leaf 2, 3 and 4. In order to find an appropriate range, which contains the second leaf, the third leaf, and the fourth leaf, the random sampling method is used to infer the population parameters. The percentage of the position tall is recorded, which is the length from the top of the whole sorghum to the recorded position divided by the entire sorghum's height. The range is divided into several subranges, with a gap width of 5%. If the position is located less than 2.5% tall, the recorded position will be 0; If the position is located between 2.5% and 7.5% tall, the recorded position will be 5%; If the position is located between 7.5% and 12.5% tall, the recorded position will be 10%. The majority of the percentages are concluded to indicate the positions in the population. For each image, two positions are recorded, the top position of the second leaf and the bottom position of the fourth leaf. In this experiment, 300 images are randomly sampled from 3000 images, with 300 sorghum samples in total and ten angles for each sample. Positions from sampling analysis suggest that the majority percentages for the second leaf's top position is 5% and the majority percentages for the bottom position of the fourth leaf is 35%.
Therefore, from statistics, we evaluated this criterion (5 to 35%) applied to the entire dataset to finally realize limitation of leaf 2, 3 and 4. It was determined the total height of the plant, and set the two boundaries at 5% and 35% of the total height as the specific region. The specific region which contained just the three leaves elements was limited and a new matrix was formed. Then we acquired the extraction of the area with leaf 2, 3 and 4 to calculate the H, S, I value respectively, and use the same limitation area to suit the original RGB image to acquire the R, G, B value. Specific leaf image 13 of hue (H) component was calculated as the total Hue value of image divided into the pixel number.
Specific leaf image of saturation (S), intensity (I), red (R), green (G) and blue (B) component are in the same way.
The total pixel count of the plant from ten side views were then averaged as plant Projected Area (PA, or equivalently, pixel count). The number of pixels inside the plant region was counted in each of the ten side views, and then averaged to give the projected shoot area. This is not the actual aboveground surface area but the average of the areas of the image projected in ten planes. There are many cases when a mature plant's leaves are overlapping, appearing behind one another in side view images (Golzarian et al., 2011). Figure 3 shows ten different binary images converted from RGB from 0 to 360 degree. The ten orthogonal views (ten side views from 36 rotational difference) provides a means of correction of plant area for those overlapping leaves, corrects for hidden areas in the other views and gives a robust representation of plant area overall. Figure. 3 Ten different binary images converted from RGB from 0 to 360 degree.

Hyperspectral imaging acquisition
The hyperspectral image chamber consisted of a total of 243 image bands, with a spectral sampling resolution of 5 nm per band. The raw image was captured in a BIL (Band Interleaved by Line) format to acquire and store the original hyperspectral data. Then the individual spectral bands were then extracted from the BIL file using a custom MATLAB function to build 243 hyperspectral images (Pandey et al., 2017). The plant image cubes were individually processed to extract the spectrum of pixel intensities. The segmentation of plant pixels in the hyperspectral images was achieved by making use of the rapid increase in reflectance of vegetation (Ni et al., 2020). The imagery was calibrated to reflectance (Jiang et al., 2020). The following procedures were used to process the hyperspectral images. First, intensity of images at band 35 (705 nm) and band 44 (750nm) were used to normalize and generate new increased intensity; and then we got a function of the sum of those new intensities.
Second, the new intensities were applied to separate the plant pixels well from the non-plant pixels in optical methods all provided reliable estimates of relative leaf Chl. Empirical models to predict chlorophyll content are largely based on reflectance regions where the absorption is saturated at higher chlorophyll. Indices formulated with 705nm and 750nm bands would have higher accuracy in estimating chlorophyll content (Gitelson, et al. 1994). Reflectance index (Chl NDVI = (R750-R705)/ (R750+R705)) were commonly used in the literature (Richardson, 2002). According to every index of structure form and principle, hyperspectral index can be divided into the following types: normalized spectral index, ratio spectral index, and multi-band spectral index. Algorithms like the ratio vegetation index (RVI), the normalized difference vegetation index (NDVI), modified chlorophyll absorption ratio index (MCARI) have been used to measure canopy cover and chlorophyll content of plants.
These ratios and algorithms are positively correlated to total chlorophyll levels in plants and each of them represents normalized spectral index, ratio spectral index, and multi-band spectral index.

Fluorescence imaging acquisition
Fluorescence imaging captures the image that the red band is mainly emitted from photosystem. With

Data analysis
From the visible image, primary colors red (R), green (G) and blue (B), was recorded. Spectral parameters such as hue (H), saturation (S) and luminosity (L) were estimated from RGB values.
Specific leaf image was calculated by dividing the total component of image by total pixel number.
Hyperspectral and fluorescence image was processed in the same way. Specific leaf weight (SLW) is the oven-dry mass, divided by its one-sided area of fresh leaf (unit, g cm -2 ).
In this analysis, DAS (days after sowing) is measured from the date of planting. The visual appearances were confirmed by DAS, because DAS represent the growth stage development. Previous research reported that the first period occurring in seeding stage matched with Chl's slow rise. The second critical period occurred in active jointing-booting growth stage and matched with Chl's fast increase. The third period is filling to maturing stage matched with Chl's slow decrease (Ali et al., 2015;Earl et al., 1999). DAS also showed a significant (p<0.01) positive correlation with the macronutrient content (Zandonadi et al., 2016). Therefore, chlorophyll content could be written as a linear form of DAS and high-throughput image. The MC 100 value has already been found to provide the most accurate estimation of chlorophyll content in good correlation with leaf chlorophyll content extracted through organic solvent method (Padilla et al., 2018). S, and I parameters were chosen from color space because it corresponds better to how people experience color than the RGB parameter set (Stevens, 2002;Mack et al., 2018). Image information such as, hue, saturation and intensity color coordinates were also considered to study the relationship of color coordinates with chlorophyll content. Using the H, S and I components, the visible ''greenness'' of three leaves can be quantified and can be compared. A significant correlation was observed between the S parameter with chlorophyll content, while weaker correlation was observed with I parameter (Figure.7). Leaves hue of image analysis showed liable correlation with leaf chlorophyll content measured after pigment extraction, but this linear relationship was worse than saturation. Figure. 6 The correlations between the Chl values of measured by MC100 with predicted by R, G and B component and DAS. For Chl estimation in sorghum, figure 7 showed the correlation between hue, intensity and MC 100 reading (R 2 = 0.61 and 0.57, respectively).Saturation gave a better Chl detection results with R 2 = 0.85. As with Chl, HSI color model is found to achieve better fitting than RGB color model.  It can be seen that normalized spectral index, ratio spectral index, and multi-band spectral index can predict the chlorophyll content well.

Correlation of fluorescence image with chlorophyll content
Data were subjected to simple regression analysis, with value from MC 100 chlorophyll meter as the value for coefficients for each variable, the importance for each vegetation index is sorted. The larger the absolute value of the coefficient is, the more indispensable the corresponding variable is. Therefore, it indicated that the hue information is not necessary for constructing the PLSR regression with the component number of 9. The statistics to evaluations for each model are listed in table 3.  Table 3 shows that PLSR regression models guarantee that the R 2 is greater than 0.84 and RPD is larger than 2.49, which indicates that PLSR regression models provide an accurate way to predict chlorophyll's content. It can be seen form figure 10 that the PLSR Model 2 (with DAS and feature 'Hue' reduction) has improved the R 2 compared with the PLSR Model 1 (without DAS) and PLSR Model 2 (with DAS).
Comparing the performance among different models, it can be concluded that (1). When more useful features are considered, the performance of the model becomes better.
(2). Reducing unnecessary features in the model, the performance of the model becomes better.

Discussion
In order to reduce the bias, we put forward a predictive model based on the specific leaf weight (SLW).
SLW and single-leaf apparent photosynthesis (AP) have been shown to be positively correlated in field studies (Buttery et al., 1981). Sampling for SLW as a predictor of AP is not widely employed because it takes much time and energy and thus is not practical for evaluating large populations (Thompson et al., 1996). SLW is defined as the leaf dry weight per one-side area and it is sensitive to plant nitrogen status, light climate and several other stresses (Field et al., 1986), so it is a key variable involved with or related to physiological processes occurring in the functioning of canopies. The observations across the images showed that chlorophyll content can be estimated as a multiple linear function of color component and SLW (Table 4). Therefore, chlorophyll content could be written as a linear form of image, DAS and SLW, i.e. Chl = b0 + b1R + b2DAS+b3SLW. As the multiple linear model based on DAS and SLW proved to be better than the multiple linear models we considered, we compared our proposed model with the linear model described in table 4.
Since the resulting square of the correlation coefficient from two multiple linear models were given in has improved the R 2 and RPD.  Overall, these results confirmed the idea that the SLW, which was used as an additional input for predicting chlorophyll content in high throughput phenotypic image, plays a vital role in reducing the error. This can be seen in tables determination coefficient of the model was increased by involving SLW. Using SLW as parameters could help to improve the prediction accuracy of the model.
As can be seen from table 3, the values of error function used to minimize the differences between the experimental and predicted data, all the models based on DAS and SLW exhibited high R 2 value and low values of AIC, thereby confirming satisfactorily chlorophyll content for models tested. This difference between the linear regression and multiple linear regression might have been due to introducing variable SLW, one of indicators of leaf thickness. The influence of leaf thickness on regression model contributed to better estimation of chlorophyll content by the chlorophyll meter. Leaf thickness changes according to leaf age and growth environment (Gratani et al., 2000;Knapp et al., 1998). Also, it has been demonstrated that reflectance increases and transmittance decrease with an increase in leaf thickness (Yamamoto et al., 2002). Thus, it is hypothesized that leaf thickness is one of the factors that determines chlorophyll content under different conditions (water conditions, and nutrition treatment).
Increasing SLW may improve leaf apparent photosynthesis. Pettigrew reported that plants grown under dryland production had a 12% increase in SLW, and he speculated that these leaves may have been denser or thicker than leaves of irrigated plants (Pettigrew, 2004). In response to drought, waterstressed plants had 12% more chlorophyll than well watered plants ( Da et al., 2011). Campbell, R. J analyzed the relationship between the SPAD-501 (SPAD) meter and total extracted chlorophyll (TChl) for leaf sets grown under greenhouse and field conditions, and found big difference. It has been suggested that the disparity in the models between experiments may partly be due to differences in leaf thickness. Field-grown leaves are typically thicker than greenhouse-grown leaves, and this is supported by the higher SLW values for the field-grown leaves (Campbell et al., 1990). The estimation of Chl with the SPAD over time may be confounded by changes in SLW. Peng et al also demonstrated that thick leaves increased SPAD readings and thicker leaves (i.e. higher SLW) absorbed red light more than infrared light in leaves with similar chlorophyll content on the basis of leaf area (Peng et al., 1993). SLW is in general an indicator of leaf thickness and the degree of mesophyll development within a leaf blade. The extent of mesophyll development largely determines the photosynthetic capacity of a leaf. Thus, SLW can potentially be used as an indirect measure of the photosynthetic characteristics of a leaf (Jurik, 1986 (Esfahani et al., 2008).
The SLW data revealed stressed plants had thicker and denser leaves, which may have led to more chlorophyll per leaf and consequently SLW is an important contributing variable for predicting chlorophyll.
The objective of the present study is to develop a generalized method to estimate the chlorophyll content of sorghum from its high throughput phenotypic image. We have developed a method that significantly reduces the bias in chlorophyll content estimation of stressed plants. We have demonstrated that models that uses mixed variables of plant image's greenness and SLW achieves this reduction and therefore the method we proposed can be used to compute more accurately the chlorophyll content of sorghum regardless of whether or not they are water and nutrition stressed.
As easy method for determining the chlorophyll content is using portable chlorophyll meter. Even in vivo chlorophyll determination can be made using SPAD-502 meter that makes nondestructive and rapid measurements of leaf chlorophyll based on spectral transmittance properties of leaves (Madeira et al., 2003). However, chlorophyll meter provides that data only in arbitrary units rather than the actually amounts of chlorophyll per unit of leaf tissue.
Recently the image processing techniques have been used for remote sensing studies concerning plant monitoring projects. It is precisely because of by far easier acquisition of images and the facility of being available as real-time database that image processing has been captured the attention of researchers as alternative strategy. However, reports on the in-vivo analysis of chlorophyll content from high throughput phenotyping facility cannot be found in the literature. The use of the imaging techniques in vivo characterization of leaf chlorophyll content at the plant level would provide information about the usefulness of the technology in non-destructive phenotyping, stress detecting, ranking, and selection of plants.
It seems that high throughput phenotypic image provides a simple, rapid, and nondestructive method to estimate the leaf chlorophyll concentration, and could be reliably exploited to predict the exact stress in sorghum. The present work demonstrated the potential for real time estimation of chlorophyll content by high throughput image analysis and DAS.
There is no facility available for the direct measurement of SLW. Currently, SLW is calculated as the ratio of leaf dry weight to fresh leaf area. Although there is not an instrument to directly and accurately measure leaf thickness, the measurement of leaf thickness could be nondestructive and relatively easier than the measurement of SLW. As technology advances, a device for measuring leaf thickness could be developed and incorporated into the image analysis to provide chlorophyll content more accurately.

Conclusions
In this study, a robust and accurate method has been developed for rapid and noninvasive determination of chlorophyll content of leaves of sorghum using visible, hyperspectral and fluorescence based image analysis. The correlation relationship was improved by the spectral properties combination of DAS and SLW. It can be seen that adjustment of phenotypic image values for SLW increases the accuracy of the prediction. An image analysis method based on SLW may be an alternate choice for the real time prediction of chlorophyll content of plants. The potential of the imaging system in predicting chlorophyll has been discussed. It is concluded that imaging techniques can be a powerful tool for low-cost, nondestructive and high-throughput analysis of chlorophyll concentration.