Source: KANSAS STATE UNIV submitted to
WHEAT YIELD PREDICTION AND ADVANCED SELECTION METHODOLOGIES THROUGH FIELD-BASED HIGH-THROUGHPUT PHENOTYPING WITH UAVS
Sponsoring Institution
National Institute of Food and Agriculture
Project Status
EXTENDED
Funding Source
Reporting Frequency
Annual
Accession No.
1011391
Grant No.
2017-67007-25933
Project No.
KS1011847
Proposal No.
2016-06713
Multistate No.
(N/A)
Program Code
A1142
Project Start Date
Nov 15, 2016
Project End Date
Nov 14, 2018
Grant Year
2018
Project Director
Poland, J. A.
Recipient Organization
KANSAS STATE UNIV
(N/A)
MANHATTAN,KS 66506
Performing Department
Plant Pathology
Non Technical Summary
Plant breeding programs must evaluate thousands of candidate varieties to identify and deliver the best high-yielding new varieties to farmers. Novel tools are needed to accelerate this process to meet the growing demands for food, feed and fiber. One promising approach is to utilize the rapidly advancing technology of unmanned aerial vehicles (UAV). Through this project we will implement UAVs outfitted with cutting-edge imaging tools to rapidly assess field trials in wheat breeding programs and extract precise measurements from the aerial images of important plant traits relating to plant health and yield. We will evaluate thousands of field plots of candidate varieties in the Kansas State University and International Wheat Research Center (CIMMYT) breeding programs and use the 'big-data' generated to develop yield prediction models to assist breeders with identifying and selecting the best candidate varieties. We will also use Deep Learning to automatically measure important traits from UAV captured images in ways that are consistent with what an expert breeder would do in the field. This approach will provide an 'eye-in-the-skies' to give breeders additional information for quickly identifying the best new varieties out of thousands of field plots.These approaches using UAVs for rapid measurement of large field trials in wheat breeding developed through this project will be implemented in powerful and breeder-friendly software. These tools will enable breeders to more effectively and quickly identify superior new varieties and deliver them to farmers. The rapid development and delivery of high-yielding varieties is a critical part of maintaining stable food supplies and obtaining global food security.
Animal Health Component
0%
Research Effort Categories
Basic
50%
Applied
50%
Developmental
(N/A)
Classification

Knowledge Area (KA)Subject of Investigation (SOI)Field of Science (FOS)Percent
2011549108180%
2067210208020%
Goals / Objectives
Goal: The overall goal of this project is to accelerate genetic gain in wheat breeding programs through the implementation of field-based high-throughput phenotyping (HTP) combined with genomic prediction modeling. Within this goal, the specific objectives for the project focus on a step change in field-based high-throughput phenotyping approaches with unmanned aerial vehicles (UAVs).Specific Objectives:Develop and deploy robust and scalable unmanned aerial systems with a range of spectral and thermal imaging capabilities; including standardized protocols for routine collection of field-based HTP measurements using UAVs over large wheat breeding nurseriesComplete foundational work to develop and validate integrated genomic and physiological models for yield prediction; including novel statistical modeling of hyperspectral dataImplement novel approaches using deep learning on 'breeder-trained' datasets to score developmental and other important agronomic traitsDevelop robust and efficient analysis pipelines that can facilitate community adoption of field-based HTP and enable in-season selection decisionsAssess optimal selection strategies for breeding programs that are implementing HTP and genomic profiling through simulation of potential genetic gain from all possible combinations of resource allocation
Project Methods
To develop relevant HTP platforms and analysis pipelines, we will work directly with wheat breeding programs in the U.S. and at the International Wheat and Maize Improvement Center (CIMMYT). We will deploy low-cost small UAVs equipped with RGB, multispectral, hyperspectral, and thermal cameras to capture high-resolution images over the entire season for wheat field plot trials.We will develop pipelines for image and data handling to ensure file and data integrity and enable rapid processing. We will implement robust database structures in MySQL and PostGIS, including multiple layers of file checking, to ensure integrity and rapid processing of thousands of images. In the image analysis pipelines, we will generate geo-rectified orthomosaic images from which plot level boundaries will be overlaid and plot-level data extracted for spectral values and vegetation indexes.For yield prediction we will develop and test multiple levels of models including 1) physiological models with HTP, 2) whole-genome prediction models, and 3) genomic + HTP models. For each level of prediction model, we will evaluate the prediction accuracy through cross-validation and forward prediction to new environments.The improved algorithms will be made publicly available through project websites and GitHub. To disseminate the application of UAV-based HTP for yield prediction and selection decisions in breeding, we will develop analysis packages as plugins for open-source QGIS with a PostGIS database.

Progress 11/15/16 to 11/14/17

Outputs
Target Audience:The initial target audience for this project are research scientists and plant breeders in the U.S. and internationally.We have made presentations at international meetings, workshops, and publication of scientific papers in peer-review journals. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided?Through the first year of this project, we have contributed to the training and professional development of two graduate students and two postdoctoral research associates. Each of these students and postdocs have attended multiple research conferences and workshops, given poster and oral presentations and developed professional networks. Presentations: (Presenter Xu Wang) Wang, X. & Poland, J. (March 2017). sUAV phenotyping at Kansas State University. NSF Workshop in Field-based High Throughput Phenotyping. Maricopa, AZ (Presenter Daljit Singh) Singh, D., Wang, X., Kumar, U., McGahee, K., Lucas, M. &Poland, J. (2016). Unmanned Aerial System Based High-Throughput Phenotyping of Wheat Breeding Nurseries.Crop Science Society of America Annual Meeting, Phoenix, AZ (Presenter Daljit Singh)Singh, D., Wang, X., Kumar, U. & Poland, J. (2017)Genetic Analysis of Crop Lodging in CIMMYT Wheat Using UAV Based High-throughput Phenotyping.National Association of Plant Breeders, Davis, CA (Presenter Daljit Singh) Singh, D. (2016).High Throughput Phenomics: Towards rapid wheat cultivar development for South Asia.Board of International Food and Agricultural Development (BIFAD) Annual Public Meeting. Des Moines, Iowa (Presenter AtenaHaghighattalab)Haghighattalab, A. & Poland, J. (2017).Application of Moving Circular Spatial Adjustment (MCSA) to increase the accuracy of Spatial Adjustment of Field Trials. Phenome, 2017. Tuscon, AZ (Presenter Daljit Singh)Singh, D., Kumar, U., Wang, X., Lucas, M., Singh, R., Schinstock, D., & Poland, J. (2016). Phenotypic assessment of CIMMYT wheat varieties using a small Unmanned Aerial System. National Association of Plant Breeders Raleigh, NC. (Presenter Xu Wang) Wang, X., Singh, D., Haghighattalab, A., Mcgahee, K., Schinstock, D., & Poland, J. (2016). Canopy Height and GNDVI Extraction from sUAS Captured Images. Phenotypic Prediction: Image Acquisition and Analysis Workshop at Iowa State University Ames, IA. (Presenter Xu Wang) Wang, X., & Poland, J. (2016). Field-Based High-Throughput Phenotyping in Wheat. Kansas Extension Agricultural Agent Update Meeting Manhattan, KS. (Presenter Xu Wang)Wang, X., Lucas, M., Schinstock, D., & Poland, J. (2016). Development and Implementation of a Small Unmanned Aerial Systems Based Phenotyping Pipeline for Plant Breeding Programs. ASA, CSSA and SSSA International Annual Meetings, Phoenix, AZ. How have the results been disseminated to communities of interest? Nothing Reported What do you plan to do during the next reporting period to accomplish the goals?Develop and deploy robust and scalable unmanned aerial systems with a range of spectral and thermal imaging capabilities; including standardized protocols for routine collection of field-based HTP measurements using UAVs over large wheat breeding nurseries We have tested and deployed improved platforms for multispectral and thermal imaging. We will continue deployment of these platforms over field sites in Kansas, Mexico and India during 2017-18 field season. Pending operational permits, we will extend UAV deployment in Pakistan and Bangladesh We will complete testing of large M600 UAV with combined camera payload of Headwall Hyperspec camera, FLIR thermal camera, Micasense RedEdge, and Sony RGB camera. Generate complete datasets of hyperspectral (400 - 1000nm) for breeding nurseries Complete flight coverage of all CIMMYT and KSU wheat breeding program plots at multiple timepoints throughout growing season Complete video imaging of breeding plots during heading Complete foundational work to develop and validate integrated genomic and physiological models for yield prediction; including novel statistical modeling of hyperspectral data Test combined genotype x environment (GxE) models using spectral data from South Asia and CIMMYT, Mexico field trials Implement novel approaches using deep learning on 'breeder-trained' datasets to score developmental and other important agronomic traits Develop processing pipeline for plot-level image extraction from UAV video data Train convolutional neural networks using UAV extracted images to score heading percentage in wheat Test feature extraction for image processing to count number of heads Develop robust and efficient analysis pipelines that can facilitate community adoption of field-based HTP and enable in-season selection decisions Complete and deploy fully automated processing pipeline for plot-level extraction of vegetation index values from raw UAV images Develop and test improved processing pipelines for stitching and geo-rectification of images from video stream Assess optimal selection strategies for breeding programs that are implementing HTP and genomic profiling through simulation of potential genetic gain from all possible combinations of resource allocation We will initiate simulation modeling of optimized selection strategies for early generation selection from small plots with varying levels of selection accuracy based on UAV derived yield predictions

Impacts
What was accomplished under these goals? Impact To realize a new level of yield potential, breeding programs must increase the speed of developing new varieties (increase the rate of genetic gain) by evaluating larger populations, making more accurate selections, and decreasing the length of the breeding cycle. Genomic advancements during the past decade have enabled genomic prediction and selection of complex traits on larger number of breeding lines and at early stages in the breeding cycle. At the same time, however, phenotyping of breeding lines under field conditions has seen minimal advancement and is a critical bottleneck for evaluating large populations. Through this project we are applying novel developments in remote sensing with unmanned aerial vehicles (UAVs) analyzed with machine vision and deep learning to make improved yield directly within breeding programs in the US and internationally at the International Maize and Wheat Research Center (CIMMYT). We have tested and optimized UAV platforms that can be deployed in field crop breeding programs. Using the rich image datasets from the UAVs, we are working to generated in-season yield predictions by combining genomic information with multiple levels of proximal sensing and deep learning on tens of thousands of breeding lines. These optimized selection strategies using the full array of genomic and phenotypic information will be assessed and delivered to breeders. To disseminate these advancements beyond the immediate project team across the breeding community, we are developing user-friendly software tools to analyze UAV imagery, along with efficient algorithms to generate breeder-ready predictions for selection. Progress on specific goals and objectives: Develop and deploy robust and scalable unmanned aerial systems with a range of spectral and thermal imaging capabilities; including standardized protocols for routine collection of field-based HTP measurements using UAVs over large wheat breeding nurseries We have tested and deployed an improved UAV platform, the DJI M100. We have also completed testing of different cameras including Micasense RedEdge and NIR converted DJI X3. Multiple M100 units with RedEdge cameras were put into service this year, making hundreds of flight missions across the project locations in Kansas, India and Mexico. We have generated a combined 100,000+ plot level observations from across these locations. We have tested a larger UAV platform, the DJI M600, with integrated differential correction GPS (RTK-GPS) for precise positioning, longer flights and heavier cameras. We have obtained and completed initial testing with Headwall Hyperspectral camera (400 - 1000 nm) We have tested and obtained initial datsets for thermal cameras including DJI ZENMUSE XT and FLIR Vue. We have generated completed datasets for thermal imaging for further processing. Complete foundational work to develop and validate integrated genomic and physiological models for yield prediction; including novel statistical modeling of hyperspectral data using data from previous manned aircraft flights, we have tested multiple modeling approaches for incorporating spectral measurements to genomic selection models we have developed and validated an approach to measure lodging in plots from Digital Elevation Models (DEMs) derived from UAV imaging. We have validated these UAV derived lodging measurements and the impact of the measured lodging on yield Implement novel approaches using deep learning on 'breeder-trained' datasets to score developmental and other important agronomic traits Using previously collected image datasets from the ground phenotyping platform, we have trained convolutional neural networks to score awned / awnless and percentage heading (0-100%) in wheat directly from images By applying neural network scoring for percent heading scores to time-series imaging we can identify heading date as the midpoint where a given plot is 50% headed Using the M600 platform with DJI X5R ultra HD (4K) resolution video camera, we have collected hundreds of thousands of images of wheat plot throughout heading stages for the next phase of deep learning Develop robust and efficient analysis pipelines that can facilitate community adoption of field-based HTP and enable in-season selection decisions We have focused effort to develop fully automated pipelines to process new UAV image data directly to plot level measurements extracted from the images We have improved database performance and transferred all plot level information to geospatial data types We have developed initial automated pipeline for pre-processing all UAV image data from RedEdge cameras We have developed initial automated for identification of ground control points, image registration and image stitching to geo-referenced orthomosaic combined with plot coordinate overlay to extract plot level data

Publications

  • Type: Journal Articles Status: Published Year Published: 2017 Citation: Sun, J., J. E. Rutkoski, J. A. Poland, J. Crossa, J.-L. Jannink and M. E. Sorrells (2017) Multitrait, random regression, or simple repeatability model in high-throughput phenotyping data improve genomic prediction for wheat grain yield. The Plant Genome 10(2). DOI: 10.3835/plantgenome2016.11.0111
  • Type: Journal Articles Status: Published Year Published: 2017 Citation: Haghighattalab, A., J. Crain, S. Mondal, J. Rutkoski, R. P. Singh and J. Poland (2017) Application of geographically weighted regression to improve grain yield prediction from unmanned aerial system imagery.� Crop Science. DOI: 10.2135/cropsci2016.12.1016
  • Type: Journal Articles Status: Published Year Published: 2016 Citation: Haghighattalab, A., L. Gonz�lez P�rez, S. Mondal, D. Singh, D. Schinstock, J. Rutkoski, I. Ortiz-Monasterio, R. P. Singh, D. Goodin and J. Poland (2016) Application of unmanned aerial systems for high throughput phenotyping of large wheat breeding nurseries. Plant Methods 12(1): 1-15. DOI: 10.1186/s13007-016-0134-6
  • Type: Conference Papers and Presentations Status: Other Year Published: 2016 Citation: Poland J., D. Singh, A. Haghihattalab J. Rutkoski, A. Fritz, X. Wang, J. Crossa, R. Singh (2017) Application of UAVs to Increase Genetic Gain, National Association of Plant Breeders, Davis CA, Aug 2017
  • Type: Conference Papers and Presentations Status: Other Year Published: 2017 Citation: Poland J., D. Singh, A. Haghihattalab, A. Fritz, X. Wang, J. Crossa, R. Singh (2017) Accelerating Wheat Breeding with Genomic Selection and High Throughput Phenotyping, National Agriculture Research Council - Pakistan, Islamabad, Pakistan, April 2017
  • Type: Conference Papers and Presentations Status: Other Year Published: 2017 Citation: Poland J., S. Mondal, J. Crossa, R. Singh (2017) Accelerating Wheat Breeding with Genomic Selection, Wheat Research Institute, Faisalabad, Pakistan, April 2017
  • Type: Conference Papers and Presentations Status: Other Year Published: 2017 Citation: Poland J., S. Battenfield, J. Rutkoski, A. Fritz, X. Wang, J. Crossa, R. Singh (2017) Accelerating Wheat Breeding with Genomic Selection and High Throughput Phenotyping, University of Maryland, Department of Plant Science, Feb 2017
  • Type: Conference Papers and Presentations Status: Other Year Published: 2017 Citation: Poland J., A. Haghihattalab, S. Battenfield, J. Rutkoski, A. Fritz, X. Wang, J. Crossa, R. Singh (2017) Increasing Yield Prediction Accuracy for Wheat Breeding Through Combined Genomic Selection and Field-based High Throughput Phenotyping, PHENOME 2017, Tuscon AZ, Feb 2017
  • Type: Conference Papers and Presentations Status: Other Year Published: 2017 Citation: Poland, J., X. Wang, R. Pless, H. Wang (2017) Application of Artificial Neural Networks and Real Neural Networks to Field-based High Throughput Phenotyping, Plant and Animal Genome - National Plant Genome Initiative Workshop, San Diego CA, Jan 2017
  • Type: Conference Papers and Presentations Status: Other Year Published: 2017 Citation: Poland J., S. Mondal, A. Haghihattalab, S. Battenfield, J. Rutkoski, A. Fritz, X. Wang, J. Crossa, R. Singh (2017) Genomic Selection and High Throughput Phenotyping for Wheat Improvement, Bangladesh Agriculture Research Institute, Jamalapur Bangladesh, Jan 2017
  • Type: Conference Papers and Presentations Status: Other Year Published: 2016 Citation: Poland, J., X. Wang, R. Pless, H. Wang (2017) Deep learning for high-throughput phenotyping of complex trait in wheat. ASA-CSSA International Annual Meetings, Phoenix AZ, Nov 2016
  • Type: Conference Papers and Presentations Status: Other Year Published: 2017 Citation: Poland, J. Haghighattalab, A., Wang, X. & Singh, D. (2017) Field-Based High Throughput Phenotyping using Proximal and Remote Sensing Technologies. Nebraska NRIC Predictive Crop Design: Genome-to-Phenome Conference. Lincoln, NE
  • Type: Conference Papers and Presentations Status: Other Year Published: 2016 Citation: (Presenter Daljit Singh)�Singh, D., Kumar, U., Wang, X., Lucas, M., Singh, R., Schinstock, D., & Poland, J. (2016). Phenotypic assessment of CIMMYT wheat varieties using a small Unmanned Aerial System. National Association of Plant Breeders Raleigh, NC.
  • Type: Conference Papers and Presentations Status: Other Year Published: 2016 Citation: (Presenter Xu Wang) Wang, X., Singh, D., Haghighattalab, A., Mcgahee, K., Schinstock, D., & Poland, J. (2016). Canopy Height and GNDVI Extraction from sUAS Captured Images. Phenotypic Prediction: Image Acquisition and Analysis Workshop at Iowa State University Ames, IA.
  • Type: Conference Papers and Presentations Status: Other Year Published: 2016 Citation: (Presenter Xu Wang) Wang, X., & Poland, J. (2016). Field-Based High-Throughput Phenotyping in Wheat. Kansas Extension Agricultural Agent Update Meeting Manhattan, KS.
  • Type: Conference Papers and Presentations Status: Other Year Published: 2016 Citation: (Presenter Xu Wang)�Wang, X., Lucas, M., Schinstock, D., & Poland, J. (2016). Development and Implementation of a Small Unmanned Aerial Systems Based Phenotyping Pipeline for Plant Breeding Programs. ASA, CSSA and SSSA International Annual Meetings, Phoenix, AZ.