Recipient Organization
TEXAS TECH UNIVERSITY
(N/A)
LUBBOCK,TX 79409
Performing Department
(N/A)
Non Technical Summary
Emerging pests and diseases, along with resource scarcity, pose threats to the sustainability of the U.S. cotton industry. Effective decision-making frameworks based on accurate and timely information are necessary to address these challenges. Machine vision technologies are transformative in agriculture, increasing the speed, precision, and scale at which plant traits can be measured and analyzed. This project aims to upgrade CottonSense, a high-throughput phenotyping technology developed by Texas Tech University that tracks, counts, and distinguishes various cotton fruiting structures throughout the growing season under field conditions. Using RGB-D cameras on a ground-based platform, CottonSense captures and processes 2D and 3D data, enabling detailed phenotypic trait extraction. The system has proven effective in precisely predicting cotton yields. Our goal is to expand its capabilities to detect pest and disease damage, differentiate cotton plants from weeds, identify boll size and location, and assess leaf and canopy architecture. Additionally, we will develop a comprehensive, curated image database hosted on an accessible cloud platform. By integrating advanced imaging and data analysis technologies, this project will enhance agronomic decision-making through efficient, real-time crop monitoring, enabling the early identification of potential pest and plant health issues before they affect productivity. This project will also enhance the cotton breeding program through fast and detailed phenotyping, enabling more accurate selection of desirable traits for development of superior cotton varieties. This project represents a groundbreaking approach to transforming agriculture through machine vision applications development for cotton.
Animal Health Component
35%
Research Effort Categories
Basic
35%
Applied
35%
Developmental
30%
Goals / Objectives
As CottonSense was designed and developed to successfully identify healthy cotton leaves and fruit structures, it can equally be successfully expanded to identify abnormalities in plant structure development resulting from insect infestation, disease, and herbicide damage. Moreover, it can be equipped with the ability to distinguish cotton plants from weeds, identify boll size and position within the canopy, and assess leaf and canopy characteristics.Our major goal is to upgrade the CottonSense high-throughput phenotyping system and to expand its capabilities to meet the following specific objectives:Detect tissue damage attributed to pests and diseases commonly found in West Texas cotton production.Detect herbicide damage and differentiate weeds from cotton. Detect boll size and position within the canopy.Detect leaf shape variations and canopy architecture.Doing so will enhance management practices, provide foresight, and consequently, reduce yield penalties and damages to cotton production. Also, as a natural byproduct of accomplishing these tasks, the proposed project will develop a widely accessible, curated, representative, and annotated image database that will complement and enhance the larger database spanning the US Cotton Belt such as those hosted by the Cotton Incorporated (Cotton Cultivated), the National Cotton Council of America (Cotton Crop Databases), and the USDA National Agricultural Library Digital Collections.
Project Methods
I. Data Collection The primary source of data for this project will be images captured from field and greenhouse plants. These images will be obtained through various means, including direct field imaging and controlled environment settings in greenhouses. Additionally, leveraging existing online repositories of agricultural images such the USDA National Agricultural Library (NAL) Digital Collections, Wikimedia Commons - Agriculture Category, and Bugwood.org - University of Georgia's Center for Invasive Species and Ecosystem Health, will supplement our dataset. This multi-source approach ensures a diverse and comprehensive collection that accurately represents various features and conditions including:Tissue damage on different plant parts - patterns that can be linked to specific disease based on plant part, color, shape, and extent.Different growth stages from seedling establishment to maturityHerbicide damageGeneral weed classificationBoll size and boll position within the canopy - provide insights into the variations in fiber maturity, quality, and on generalized yield estimates prior to harvestLeaf shape and canopy architecture variations - provide insights into the interactions between cotton morphology and environmentDuring field imaging, a sufficient number of high-resolution RGB-D cameras will be mounted on an RTK GPS-equipped tractor platform. This existing data collection platform has been used on commercial and breeding fields and has been optimized to efficiently collect data while minimizing wind effects, color saturation from direct sunlight, and interference from plants in adjacent rows. The cameras will be positioned at an appropriate height and angle to ensure full capture of the entire canopy as the plant grows throughout the season. The platform that will be used for this proposal will be designed in such a way that cotton producers could attach it to their present equipment. In greenhouse settings, images will be captured with the use of high-resolution cameras strategically placed within the greenhouse to document plant parameters, ensuring consistent and controlled image acquisition. This facilitates detailed analysis of greenhouse-grown plants under various conditions. For both experimental setups, image data collection will be done on a weekly basis from seedling establishment to crop maturity.Once collected, the images collected from the field and greenhouse will undergo thorough organization and annotation. Organizing involves categorizing images based on the factors and conditions mentioned above, facilitating easy navigation and retrieval of specific images based on user needs.II. AnnotationAnnotation is equally crucial, as it involves adding detailed metadata to each image. A team of annotators will use software such as MATLAB Image Labeler to label images, specifically identifying the plant features and conditions described in the data collection section of the Materials and Methods. Each annotator will be assigned a unique set of images to ensure there is no overlap in the labeling task. Each annotated image set will be reviewed by another annotator to check for any inconsistencies or errors. This peer review process ensures that any discrepancies are identified and corrected promptly.III. Model Development, Training, and EvaluationThe annotated dataset will be used to train the model, optimizing parameters to minimize classification errors. The model will be validated using a subset of dataset to ensure it generalizes well to new, unseen images. This validation dataset will not be used to train the model; instead, it will serve as a benchmark to check the model's performance and adjust parameters accordingly.Fruit identification and plant tissue damage detection using machine vision and learning approaches are fundamentally similar problems. As such, developing new models for CottonSense to detect tissue damage, identify possible causes, identify leaf shape and canopy structure, and differentiate cotton plants from weeds will proceed in the same manner as the fruit identification and enumeration models were developed. We will gather and curate the training dataset and annotate the data under the supervision of problem domain experts who have a lot of knowledge and experience in model training. During the development of models for CottonSense, we devised a unique approach to dataset annotation that we now employ routinely in all our projects. This approach begins with the annotation of a small volume of data, which is then used to train a preliminary model. The generated model is then integrated into the annotation tool and is used to make predictions that are subsequently scrutinized and, if necessary, edited by the human annotators. This approach has several important advantages including:Streamlines the often cumbersome and tedious process followed by human annotators.Increases the accuracy and consistency of the annotations by offering a second opinion to the annotator.Offers a qualitative and quantitative assessment of the model's performance as observed by the human annotator.The training data that is produced in this round of model-assisted annotations is used to train the next version of the model. This process is repeated iteratively until an adequate amount of data is annotated and a well-performing model is developed.The architecture of the deep network that will be used to develop the models for various tasks will be selected based on a systematic evaluation of the available architectures. Drawing from our experience with CottonSense, we will begin with the Faster R-CNN architecture (Ren et al., 2016) for object detection and the Mask R-CNN architecture (He et al., 2017) for semantic and instance segmentation. We have had a great deal of success with these particular architectures in other projects, which is the reason for trying them first. Nevertheless, we remain open to exploring newer architectures, such as transformer-based models (Dosovitskiy et al., 2020; Radford et al., 2019) to assess their suitability for our current needs in this proposal.The model's performance will be assessed using Average Precision (AP), a standard evaluation metric for object detection and segmentation networks. AP is calculated as the area under the precision-recall curve. This provides a single metric that summarizes the model's ability to make correct detections across various thresholds (Bolouri et al., 2024; Everingham et al., 2015). The higher the AP, the better the model's performance.IV. Development of a Curated Image DatabaseA curated database storing images and their related metadata will be established. This will be a living database as it will allow constant image uploading from researchers, farmers, and crop consultants. This structured upload mechanism will help refine the system and enhance the user interface. It will also incorporate real-time feedback from the agricultural community so that the application will be continuously improved to better meet the needs of its users. This iterative process ensures that the application remains relevant and effective, providing valuable insights and tools for optimizing crop management and farming practices.The curated image database will be hosted on a scalable cloud platform, leveraging AWS (Amazon Web Services) infrastructure for scalable object storage (Amazon S3), relational database management (Amazon RDS), and serverless computing (AWS Lambda). This approach will ensure widespread accessibility, scalability, and reliability, supporting global agricultural research initiatives. Integration into a larger database spanning the US Cotton Belt area will further enhance its utility, facilitating comprehensive analysis across diverse geographic regions and advancing agricultural production.