Weeds are the biggest threat to crop growth, and leaf age and central area of weeds are important phenotypic traits of weeds. They have an important role in understanding the morphological structure of weeds, guiding precision for target weeding and reducing the use of herbicides. However, it is still a substantial challenge to obtain weed types, leaf age and central area in the complex field conditions of light changes, variation in appearance of plants, leaf occlusion. The latest developments in deep learning provide new tools for solving challenging computer vision tasks.
In this study, we present a weed phenotype segmentation method based on Mask R-CNN that obtains weed types, leaf age and central area in the complex field conditions. By shooting three different angles of the main weeds (Solanum nigrum, Barnyard grass, and Abutilon theophrasti Medicus) in the field through mobile devices, two datasets (data enhancement and without data enhancement) were produced and used as input to the network, two backbone networks was tested, namely ResNet50 and ResNet101, and the detection results and instance segmentation results of the model were evaluated. The results showed that data enhancement can improve the performance of the model. In the case of data enhancement, the F1 value with ResNet101 as the backbone network was 0.9214, the mAP scores were 0.6932 and 0.5244 (for IOU thresholds of 0.5 and 0.7, respectively), the mIOU reached 0.585, and the best segmentation performance example was obtained. Furthermore, the weed image taken from the top view angle compared to the other two angles achieved the highest detection accuracy.
Mask R-CNN can achieve accurate segmentation of weeds to obtain the types, leaf age and central area of weeds. Data enhancement and the weed image taken from the top view angle can help to improve the performance of the model. This dataset and research results may provide important resources for the development of precision agriculture in the future.