ADVANCING INTEGRATED WEED MANAGEMENT RESEARCH WITH COMPUTER VISION TECHNOLOGY

• Sponsoring Institution: National Institute of Food and Agriculture
• Project Status: COMPLETE
• Funding Source: AFRI COMPETITIVE GRANT
• Reporting Frequency: Annual
• Accession No.: 1030067
• Grant No.: 2023-67012-39351
• Cumulative Award Amt.: $225,000.00
• Proposal No.: 2022-09741
• Project Start Date: Mar 1, 2023
• Project End Date: Feb 28, 2025
• Grant Year: 2023
• Program Code: [A1112]- Pests and Beneficial Species in Agricultural Production Systems

Recipient Organization
UNIVERSITY OF DELAWARE
(N/A)
NEWARK,DE 19717

Performing Department
(N/A)

Non Technical Summary
Commodity crop production is threatened by the evolution of weed biotypes that are resistant to multiple herbicide modes of action. These multiple herbicide resistant (MHR) weeds have rendered many common herbicide chemistries ineffective, causing severe yield losses in commodity crops and leading farmers to increase the number and volume of herbicide products used in their weed management programs. Over the long-term, however, sole dependency on herbicides for weed management will only exacerbate the MHR epidemic. Integrated weed management (IWM) programs that incorporate non-chemical strategies alongside effective chemical control options can reduce selection pressure for MHR . Site-specific weed management programs that use precision technologies to target individual weeds with effective chemical and/or non-chemical controls can also reduce selective pressure by accounting for the inherent spatial heterogeneity of weed populations and simultaneously detecting and preventing escapes of MHR weeds. Developing effective precision IWM strategies that emphasize prevention through multiple complementary tactics is critical for reducing the rate of herbicide resistance evolution and spread.Measuring the success of an IWM program is not based on the efficacy of each individual tactic in a given growing season, but rather on the collective ability of multiple tactics to manage weed populations at levels that prevent interference with crop production over multiple years. Thus, for effective IWM to be developed and adopted, there is a need for ways to monitor weed populations with higher spatial and temporal resolutions than is provided by traditional methods such as manual scouting and visual weed control ratings.Recent advances in computer vision (CV) technology have enabled low-cost sensing systems for estimating weed biomass and density at the species level. Information from these systems can provide high spatial resolution maps of weed populations that can assist with evaluating both short- and long-term effects of weed management tactics individually and in combinations. Demographic information about weed populations (i.e., seedling emergence, mortality, fecundity) obtained from CV systems can also be combined with data on the typical variation in environmental conditions at a site to predict the long-term impacts of combining multiple weed management strategies, thereby reducing the resources needed to directly monitor these effects. Utilizing CV technology to monitor weed populations at the species level will accelerate the development and adoption of effective precision IWM strategies, thereby combatting the MHR epidemic and improving the overall sustainability of weed management.Getting Rid Of Weeds (GROW; www.growiwm.org) is a national research and extension network of weed scientists, agronomists, computer scientists, and agricultural engineers collaborating to develop precision IWM technologies. The GROW team has previously developed Weeds3D, a low-cost CV platform for weed mapping. Weeds3D consists of a GoPro Camera, an Android tablet for camera control, image upload, and displaying weed maps, and a cloud-computing based image processing pipeline. Weeds3D is capable of generating 3D point cloud reconstructions of weed and crop plants at the species level that allow for the estimation of individual plant biomass. While this technology has many potential applications for precision weed monitoring and management, it has yet to be used as a tool for data collection in applied IWM research.This project will test the Weeds3D system's ability to accurately estimate weed demographic data (i.e., weed density, biomass, and fecundity at the species level) in a multi-state field experiment assessing the integration of cover crops (CCs) and residual herbicides for MHR weed control.At each research site three cover crop management treatments (No cover/fallow, cereal rye terminated 2 weeks prior to soybean planting, and cereal rye terminated at or soon after soybean planting) and two herbicide treatments (POST-only, and PRE-residual followed by POST) will be utilized in a factorial randomized complete block design to create a gradient of IWM intensity. Cover crop performance, weed demographics, and soybean yield will be measured using both traditional sampling methods (i.e., quadrat samples of density, biomass, yield) and the Weeds3D system. Data from quadrat sampling will be used to validate the weed density and biomass estimates produced by Weeds3D, and both datasets will be analyzed to evaluate the efficacy of the IWM treatments on weed populations and soybean yield. Results from this experiment will be used to inform best practices for managing cover crops and integrating them with effective herbicide programs to manage MHR weeds and slow resistance development and spread.In addition to evaluating the impacts of a promising IWM strategy on weed population dynamics within a growing season, the proposed project will also develop digital infrastructure and statistical approaches for predicting the long-term impacts of IWM strategies by parameterizing predictive weed population models with demographic estimates generated by the Weeds3D system. In short, this will involve developing simulation models to predict how MHR weed populations respond to IWM tactics over longer periods of time, and integrating these models with the existing Weeds3D image processing software and database through a new application programming interface, also developed as part of the project effort. These combined efforts will result in an improved Weeds3D system with a seamless data processing and analysis pipeline that will allow researchers with minimal programming or modeling experience to quickly and easily collect videos of research trials, extract information about weed demographics at the species level, and predict the long-term impacts of IWM tactics being tested in the trials. By facilitating the data collection and analysis process and thereby reducing the time, labor, and other resources necessary for IWM research, these tools promise to greatly accelerate the development and testing of IWM strategies and thereby provide more options for the sustainable management of MHR weeds.

Animal Health Component

50%

Research Effort Categories

Basic

(N/A)

Applied

50%

Developmental

50%

Classification

Knowledge Area (KA)	Subject of Investigation (SOI)	Field of Science (FOS)	Percent
216	2300	1140	50%
402	7299	2080	50%

Knowledge Area
216 - Integrated Pest Management Systems; 402 - Engineering Systems and Equipment;

Subject Of Investigation
7299 - Research equipment and methods, general/other; 2300 - Weeds;

Field Of Science
2080 - Mathematics and computer sciences; 1140 - Weed science;

Keywords

ai

integrated weed mgt

computer vision

cover crops

multiple herbicide resistant weeds

Goals / Objectives
Project Goal 1. Use computer vision (Weeds3D) and traditional field sampling methods to evaluate the in-season impact of integrating CCs and residual herbicide programs on weed suppression and cash crop performance across broad environmental gradients.Objective 1a: Develop and revise field experiment protocols.Objective 1b: Coordinatefield experiment management and data collectionwith collaborators in seven states (DE, GA, IL, KY,MD, NC, PA).Objective 1c: Collect and process video data with Weeds3D system to obtain weed density, biomass, and fecundity estimates for each species present in experimental transects.Objective 1d: Collect weed density, biomass, and fecundity data using standard quadrat sampling methods.Objective 1e: Validate weed demographic data estimates from Weeds3D using data from quadrat samples.Project Goal 2. Predict long-term effects of CC management and residual herbicides on weed communities, including multiple herbicide resistant weeds, using population models created from CV data.Objective 2a: Develop Weeds3D database API to enable seamless data flow between collection and analysis.Objective 2b: Develop weed population simulation models from Weeds3D demographic data.Objective 2c: Use population model results to evaluate long-term effects of cover crops and residual herbicides on weed communities and herbicide resistance development under varying management scenarios.

Project Methods
Field Experiment (Project Goal 1).The Weeds3D system will be used to monitor weed populations in an IWM field experiment carried out at two field sites in each of seven states (DE) over two cash crop growing seasons, for a total of 28 site-years of data. The experiment will use a factorial design with three levels of CC management and two levels of herbicide inputs as treatments. The cash crop will be soybean at all sites in both years. CC management levels will be a bare fallow control, a standard termination timing two weeks prior to cash crop planting, and a planting green termination timing at or up to two days after cash crop planting. The bare fallow treatment will receive a burndown herbicide application at the same time as the standard CC termination timing. Herbicide treatments will be a POST-only herbicide program when weeds reach height-based management thresholds, and a program that includes a PRE-residual herbicide application at cash crop planting followed by the same POST program.This treatment design will produce a gradient of IWM intensity through the interaction of CC and herbicide treatments. It will also provide substantial environmental gradients due to differences in soil types and weather conditions across site years. Weed population response to management and environmental gradients will be measured through both traditional weed population sampling methods (i.e., seedling emergence counts, dry biomass samples, seed production measurements) and the Weeds3D system at times with biological and management relevance during the growing season: 1) at CC termination/burndown, 2) at soybean emergence, 3) three and six weeks after soybean emergence to capture emerging weed cohorts, 4) at soybean physiological maturity, and 5) at soybean harvest. These sampling timings will allow us to capture differences in weed seedling emergence, growth, mortality, and fecundity under varying management conditions that will be used to parameterize life-cycle based population models. CC biomass at termination, soybean stand count at growth stage V3, soybean biomass at growth stage R5, and soybean yield will also be measured to be used as explanatory variables in weed population models. Environmental data will be obtained from the NOAA Climate Data Online database (ncei.noaa.gov/cdo-web) and the NRCS Web Soil Survey Database (nrcs.usda.gov/wps/protal/nrcs/main/soils/survey) and verified by researchers at each site.Cereal rye (Secale cereale L.) will be planted as a winter CC at all sites in Fall 2022. Data collection will begin at CC termination in Spring 2023 and will conclude with the second year of soybean harvest in Fall 2024. All sites are currently part of the GROW research network and have participated in prior research and development of the Weeds3D system. Thus, all sites already have the necessary Weeds3D hardware, have experience using the Weeds3D system for data collection, and will receive any Weeds3D software updates automatically from the GROW technical support team.Data Processing Pipeline for Modeling Weed Population Dynamics (Project Goal 2). Video collected with the Weeds3D system will be processed using existing algorithms for 3D point cloud reconstructions of weed and crop biomass using thestructure-from-motion technique that accounts for canopy height and density. Weed density and biomass estimates will be generated by this image analysis pipeline and will be validated using empirical data from the field experiment. New empirical relationships for estimating weed fecundity will be integrated into the current image analysis pipeline using weed biomass data from the field experiment and seed production data from a previous GROW experiment (Lazaro et al. 2022).A new data flow pipelinefor building predictive population simulation models from these estimates will be developed using the open-source R PROSPER package (von Redwitz and de Mol 2020). PROSPER was specifically developed for modeling MHR weed populations using demographic parameters and provides functions for introducing experimental data directly into simulation models. Models are built by defining an initial weed population structure (i.e., seedbank density and weed species biology, including herbicide resistance traits) and any number of selection functions (i.e., factors such as climate, competition, or weed management interventions) that determine the likelihood of an individual with a given set of traits proceeding to the next lifecycle or selection step. Initial weed population structure and selection functions for CC weed suppression and the effects of residual herbicides will be calculated based on estimates generated by the Weeds3D data collection over the course of each field season. The modeling data flow will be enabled by the development of an application programming interface (API) that will allow data to be pulled directly from the existing Weeds3D database into the PROSPER modeling environment. Additional model parameters and selection functions, such as the proportion of individual plants exhibiting HR traits in a population of a given species, and weed species life cycle characteristics such as seedbank mortality and annual germination rates will be obtained from site managers or published literature as needed. Population dynamics over multiple years will be simulated using initial population demographics and probabilities of individual weeds surviving and reproducing under a given management regime. Outputs of population models built with PROSPER will include year-to-year weed population growth or decline as influenced by weed demography and IWM practices, and the prevalence of herbicide resistance genes in the population if initial gene frequencies and susceptibility of resistant biotypes to management interventions are provided.All of the data flow infrastructure developed for this project will be hosted in a public Github repository, thereby making it easily accessible and customizable for other researchers and potential end users.Future plans for the Weeds3D system include developing a web application for researchers to easily access and analyze data generated by the system. The Weeds3D API and population modeling workflow developed for this project will eventually be integrated into this application, facilitating their use by other researchers.Evaluation: Success of the project will be evaluated based on the successful completion of data collection from the field experiment testing delayed CC termination and residual herbicide programs, the creation and validation of weed population models using demographic information generated by the Weeds3D computer vision system, and the deployment of an open-access web application connecting the current Weeds3D image processing pipeline to a user-friendly interface for evaluating IWM research trials using predictive population models. Progress on each of these objectives and the underlying activities associated with each will be assessed periodically during meetings between the PD and the Primary and Collaborating Mentors. Additional feedback on project outcomes will be obtained from GROW advisory board members when the PD presents updates at semi-annual GROW meetings, and from GROW collaborators during monthly meetings scheduled to promote communication about planning and executing the field experiment. All feedback will be used to revise research objectives and experimental protocols as needed to ensure successful completion of the project.

Progress 03/01/23 to 04/11/24

Outputs
Target Audience:The primary audience for this project is weed scientists who are most likely to utilize the digital tools for weed demographic data collection and analysis developed by the project. The secondary audience for this project is soybean farmers who will benefit from new information on the integration of cover crops and residual herbicides for weed management in soybean crops generated by the field experiment component of the project. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided?Training activities: Researchers at seven participating sites were trained to use the Weeds3D system for data collection in Q1 of 2023 by PD Law. Video tutorials were created to demonstrate the data collection process and are available for further training sessions or review by anyone conducting the protocol on an ongoing basis. Professional development activities: In addition to the professional experience gained by coordinating a multi-state field experiment, PD Law gained professional development through several research, extension, and outreach activities in the first year of this project. PD Law attended and presented preliminary results from this project at two national scientific conferences (Agronomy Society of America and Weed Science Society of America) and also gave a presentation on the Weeds3D technology to an audience of USDA-ARS scientists at an annual planning meeting for the LTAR Croplands Common Experiment group. He also contributed to the Getting Rid Of Weeds (GROW; www.growiwm.org) Outreach and Extension team throughout the year, which included authoring one extension publication, contributing to several others, presenting to stakeholder audiences at the Indiana CCA Conference and the Ohio Organic Grains Conference, and helping lead a Crop Production Field Tour on the Delmarva Peninsula for Environmental Protection Agency staff. How have the results been disseminated to communities of interest?Preliminary reseach results were communicated to scientific audiences via oral or poster presentations at the Agronomy Society of America Annual Meeting (October 2023), the Northeastern Weed Science Society Annual Meeting (January 2024), and the Weed Science Society of America Meeting (January 2024). An overview of the Weeds3D technology and its use for mapping and monitoring weed populations was included in an invited keynote presentation at the Ohio Organic Grain Conference (January 2024) that was attended by over 250 organic grain farmers, industry professionals, and extension specialists. A website providing an overview of precision weed management technologies (www.growim.org/what-is-precision-weedmanagement) was developed and published with the GROW Outreach and Extension team. What do you plan to do during the next reporting period to accomplish the goals? Nothing Reported

Impacts
What was accomplished under these goals? Project Goal 1: Substantial progress has been made on all objectives for this project goal. A shared protocol for field experiment management and data collection was developed during Q1 and Q2 and was successfully carried out at the seven participating locations in Q2 and Q3 of 2023, resulting in 14 site-years of data collected (2 field sites at each location). Data from the first field season have been processed, preliminary analyses have been conducted, and the first year of resultspresented at regional and national conferences. We are currently reviewing the 2023 protocol and making revisions for 2024 based on feedback from researchers at each of the participating sites. Project Goal 2: Work on the objectives for this project goal started in Q4 of 2023 as preliminary data from the first year of the field experiment became available. We are currently reviewing the PROSPER weed population model and are preparing the demographic data collected for each weed species of interest to parameterize the model. Once models are parameterized we will be able to simulate various management and herbicide resistance scenarios to make longer term population projections.

Publications

• Type: Conference Papers and Presentations Status: Published Year Published: 2024 Citation: Law, E.P., Menalled, U.D., Skovsen, S.K., Kutugata, M. (2024). Unlocking the Power of Sensor-Based Data Collection in Weed Ecology. Weed Science Society of America Annual Meeting, January 22-25, San Antonio, Texas.
• Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Law, E.P., Bagavathiannan, M.V., Basinger, N.T., Everman, W., Gage, K., Ginn, D., Haramoto, E., Leon, R., Lindquist, J., Little, R., Miller, E., Mirsky, S.B., Rubione, C., VanGessel, M.J., Wallace, J. (2023). Evaluating Integrated Weed Management Outcomes with Computer Vision Technology. ASA-CSSA-SSSA International Annual Meeting, October 29 - November 1, St. Louis, Missouri.
• Type: Conference Papers and Presentations Status: Published Year Published: 2024 Citation: Hoffer, G., Wallace, J., VanGessel, M., Basinger, N., Hager, A., Haramoto, E., Law, E., Everman, W., Lindquist, J., Gage, K., Miller, E. (2024). Delayed cereal rye termination influences weed recruitment: A regional perspective. Northeastern Weed Science Society Annual Meeting, January 7 - 11, Boston, Massachusetts.

Progress 03/01/23 to 02/29/24

Outputs
Target Audience:The primary audience for this project is weed scientists who are most likely to utilize the digital tools for weed demographic data collection and analysis developed by the project. The secondary audience for this project is soybean farmers who will benefit from new information on the integration of cover crops and residual herbicides for weed management in soybean crops generated by the field experiment component of the project. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided?Training activities: Researchers at seven participating sites were trained to use the Weeds3D system for data collection in Q1 of 2023 by PD Law. Video tutorials were created to demonstrate the data collection process and are available for further training sessions or review by anyone conducting the protocol on an ongoing basis. Professional development activities: In addition to the professional experience gained by coordinating a multi-state field experiment, PD Law gained professional development through several research, extension, and outreach activities in the first year of this project. PD Law attended and presented preliminary results from this project at two national scientificconferences (Agronomy Society of America and Weed Science Society of America) and also gave a presentation on the Weeds3D technology to an audience of USDA-ARS scientists at an annual planning meeting for the LTAR Croplands Common Experiment group. He also contributed to the Getting Rid Of Weeds (GROW; www.growiwm.org) Outreach and Extension team throughout the year, which included authoring one extension publication, contributing to several others, presenting to stakeholder audiences at the Indiana CCA Conference and the Ohio Organic Grains Conference, and helping lead a Crop Production Field Tour on the Delmarva Peninsula for Environmental Protection Agency staff. How have the results been disseminated to communities of interest?Preliminary reseach results were communicated to scientific audiences via oral or poster presentations at the Agronomy Society of America Annual Meeting(October 2023), the Northeastern Weed Science Society Annual Meeting (January 2024), and the Weed Science Society of America Meeting (January 2024). An overview of the Weeds3D technology and its use for mapping and monitoring weed populations was included in an invited keynote presentation at the Ohio Organic Grain Conference (January 2024) that was attended by over 250 organic grain farmers, industry professionals, and extension specialists. A website providing an overview of precision weed management technologies (www.growim.org/what-is-precision-weed-management) was developed and published with the GROW Outreach and Extension team. What do you plan to do during the next reporting period to accomplish the goals?Project Goal 1 will be accomplished by repeating the field experiment and collecting and an additional 14 site-years worth of datain 2024. Scripts for cleaning and analyzing data that were developed during the first year of the project will expedite these processes for the incoming data in 2024. Once all data have been obtained at the second soybean harvest, analyses will be finalized and manuscripts prepared for submission by the end of the grant performance period in February 2025. Work on Project Goal 2 is accelerating now that a full season of field data are available. During the next reporting period we will parameterize PROSPER models for weed species of interest. We will then generate management scenarios and herbicide resistance profiles based on literature review,field experiment outcomes, and expert knowledge. Themodelling environment will be ready for simulations by the end of Q3 2024 so that incoming data from the current field season can be incorporated into the analysis. Results from the project will be communicated to communities of interest at regional and national scientific conferences, local and regional extension meetings, and scientific and extension publications throughout this reporting period and continuing beyond the end of the performance period.

Impacts
What was accomplished under these goals? Project Goal 1: Substantial progress has been made on all objectives for this project goal. A shared protocol for field experiment management and data collection was developed during Q1 and Q2 and was successfully carried out at the seven participating locations in Q2 and Q3 of 2023, resulting in 14 site-years of data collected (2 field sites at each location). Data from the first field season have been processed, preliminary analyses have been conducted, and the first year of results presented at regional and national conferences. We are currently reviewing the 2023 protocol and making revisions for 2024 based on feedback from researchers at each of the participating sites. Project Goal 2: Work on the objectives for this project goal started in Q4 of 2023 as preliminary data from the first year of the field experiment became available. We are currently reviewing the PROSPER weed population model and are preparing the demographic data collectedfor each weed species of interest to parameterize the model. Once models are parameterized we will be able to simulate various managementand herbicide resistance scenarios to make longer term population projections.

Publications

• Type: Conference Papers and Presentations Status: Published Year Published: 2024 Citation: Law, E.P., Menalled, U.D., Skovsen, S.K., Kutugata, M. (2024). Unlocking the Power of Sensor-Based Data Collection in Weed Ecology. Weed Science Society of America Annual Meeting, January 22-25, San Antonio, Texas.
• Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Law, E.P., Bagavathiannan, M.V., Basinger, N.T., Everman, W., Gage, K., Ginn, D., Haramoto, E., Leon, R., Lindquist, J., Little, R., Miller, E., Mirsky, S.B., Rubione, C., VanGessel, M.J., Wallace, J. (2023). Evaluating Integrated Weed Management Outcomes with Computer Vision Technology. ASA-CSSA-SSSA International Annual Meeting, October 29 - November 1, St. Louis, Missouri.
• Type: Conference Papers and Presentations Status: Published Year Published: 2024 Citation: Hoffer, G., Wallace, J., VanGessel, M., Basinger, N., Hager, A., Haramoto, E., Law, E., Everman, W., Lindquist, J., Gage, K., Miller, E. (2024). Delayed cereal rye termination influences weed recruitment: A regional perspective. Northeastern Weed Science Society Annual Meeting, January 7 - 11, Boston, Massachusetts.