Source: KANSAS STATE UNIV submitted to
WHEAT YIELD PREDICTION AND ADVANCED SELECTION METHODOLOGIES THROUGH FIELD-BASED HIGH-THROUGHPUT PHENOTYPING WITH UAVS
Sponsoring Institution
National Institute of Food and Agriculture
Project Status
EXTENDED
Funding Source
Reporting Frequency
Annual
Accession No.
1011391
Grant No.
2017-67007-25933
Project No.
KS1011847
Proposal No.
2016-06713
Multistate No.
(N/A)
Program Code
A1142
Project Start Date
Nov 15, 2016
Project End Date
Nov 14, 2019
Grant Year
2019
Project Director
Poland, J. A.
Recipient Organization
KANSAS STATE UNIV
(N/A)
MANHATTAN,KS 66506
Performing Department
Plant Pathology
Non Technical Summary
Plant breeding programs must evaluate thousands of candidate varieties to identify and deliver the best high-yielding new varieties to farmers. Novel tools are needed to accelerate this process to meet the growing demands for food, feed and fiber. One promising approach is to utilize the rapidly advancing technology of unmanned aerial vehicles (UAV). Through this project we will implement UAVs outfitted with cutting-edge imaging tools to rapidly assess field trials in wheat breeding programs and extract precise measurements from the aerial images of important plant traits relating to plant health and yield. We will evaluate thousands of field plots of candidate varieties in the Kansas State University and International Wheat Research Center (CIMMYT) breeding programs and use the 'big-data' generated to develop yield prediction models to assist breeders with identifying and selecting the best candidate varieties. We will also use Deep Learning to automatically measure important traits from UAV captured images in ways that are consistent with what an expert breeder would do in the field. This approach will provide an 'eye-in-the-skies' to give breeders additional information for quickly identifying the best new varieties out of thousands of field plots.These approaches using UAVs for rapid measurement of large field trials in wheat breeding developed through this project will be implemented in powerful and breeder-friendly software. These tools will enable breeders to more effectively and quickly identify superior new varieties and deliver them to farmers. The rapid development and delivery of high-yielding varieties is a critical part of maintaining stable food supplies and obtaining global food security.
Animal Health Component
0%
Research Effort Categories
Basic
50%
Applied
50%
Developmental
(N/A)
Classification

Knowledge Area (KA)Subject of Investigation (SOI)Field of Science (FOS)Percent
2011549108180%
2067210208020%
Goals / Objectives
Goal: The overall goal of this project is to accelerate genetic gain in wheat breeding programs through the implementation of field-based high-throughput phenotyping (HTP) combined with genomic prediction modeling. Within this goal, the specific objectives for the project focus on a step change in field-based high-throughput phenotyping approaches with unmanned aerial vehicles (UAVs).Specific Objectives:Develop and deploy robust and scalable unmanned aerial systems with a range of spectral and thermal imaging capabilities; including standardized protocols for routine collection of field-based HTP measurements using UAVs over large wheat breeding nurseriesComplete foundational work to develop and validate integrated genomic and physiological models for yield prediction; including novel statistical modeling of hyperspectral dataImplement novel approaches using deep learning on 'breeder-trained' datasets to score developmental and other important agronomic traitsDevelop robust and efficient analysis pipelines that can facilitate community adoption of field-based HTP and enable in-season selection decisionsAssess optimal selection strategies for breeding programs that are implementing HTP and genomic profiling through simulation of potential genetic gain from all possible combinations of resource allocation
Project Methods
To develop relevant HTP platforms and analysis pipelines, we will work directly with wheat breeding programs in the U.S. and at the International Wheat and Maize Improvement Center (CIMMYT). We will deploy low-cost small UAVs equipped with RGB, multispectral, hyperspectral, and thermal cameras to capture high-resolution images over the entire season for wheat field plot trials.We will develop pipelines for image and data handling to ensure file and data integrity and enable rapid processing. We will implement robust database structures in MySQL and PostGIS, including multiple layers of file checking, to ensure integrity and rapid processing of thousands of images. In the image analysis pipelines, we will generate geo-rectified orthomosaic images from which plot level boundaries will be overlaid and plot-level data extracted for spectral values and vegetation indexes.For yield prediction we will develop and test multiple levels of models including 1) physiological models with HTP, 2) whole-genome prediction models, and 3) genomic + HTP models. For each level of prediction model, we will evaluate the prediction accuracy through cross-validation and forward prediction to new environments.The improved algorithms will be made publicly available through project websites and GitHub. To disseminate the application of UAV-based HTP for yield prediction and selection decisions in breeding, we will develop analysis packages as plugins for open-source QGIS with a PostGIS database.

Progress 11/15/17 to 11/14/18

Outputs
Target Audience:Target audiences: The initial target audience for this project are research scientists and plant breeders in the U.S. and internationally. Efforts: We have made presentations at international meetings, workshops, and publication of scientific papers in peer-review journals. Changes/Problems:We have faced initial problems with collecting useable data from the Headwall Nano hyperspectral camera. The data output is very sensitive to tuning the correct camera settings and flight parameters. To date, we have generated several useable datasets, but the quality for plot-level data extraction is not sufficient for further analysis. We are currently working with another group at KSU who is deploying the same camera and with technical reps from Headwall to optimized the flight parameters and camera settings. This is being done currently (July - Sept 2018) in off-season on sorghum field trials. We have faced issues with collecting useable data from FLIR VUE Pro R thermal camera. The pixel values of the same plot shown in continuous images have huge differences. We are currently trying to implement radiometric calibration upon the raw thermal images. This is being done off season on sorghum field trials. What opportunities for training and professional development has the project provided?Through the current project year, we have contributed to the training and professional development of two graduate students and two postdoctoral research associates. Each of these students and postdocs have attended multiple research conferences and workshops, given poster and oral presentations and developed professional networks. Xu Wang Xu Wang, Jesse Poland Improving Genomic Prediction with High-Throughput Phenotyping 2018 Coalition for Advancing Digital Research & Education Oklahoma State University, Stillwater, OK NIFA IWYP 4/18/2018 Xu Wang Xu Wang, Jesse Poland An Automated Image Processing Pipeline for High-Throughput Phenotyping with Unmanned Aerial Systems NIFA FACT Workshop: Big Data Analytics in Plant Breeding and Satellite Data and Modeling Workshop Arlington, Virginia NIFA IWYP 2/26/2018 Daljit Singh D. Singh, V. Tomar, X. Wang, U. Kumar, R. Singh, and J. Poland Phenomics-enabled Genetic and Physiological Dissection of Early Ground Cover in Wheat. National Association of Plant Breeders University of Guelph, Guelph, Canada USAID, NSF 8/9/2018 Daljit Singh Daljit Singh, Jesse Poland Phenomics-enabled genetic dissection of lodging in wheat through unmanned aerial systems. Dupont Plant Breeding Symposium Cornell University, Ithaca USAID, NSF 3/9/2018 Daljit Singh Daljit Singh, Jesse Poland Drone-based high-throughput phenotyping reveals genetic architecture of lodging in wheat. Dupont Plant Breeding Symposium Iowa State University, Ames USAID, NSF 3/2/2018 Atena Haghighattalab UAS Symposium: Applications in Precision Agriculture AAG 2018 New Orleans, LA April 2018 How have the results been disseminated to communities of interest?Continue...... 2.2. Hyperspectral prediction modeling To assess the utility of using aerial hyperspectral reflectance phenotypes within genomic selection to predict wheat grain yield, we deployed a manned aircraft equipped with a hyperspectral camera to phenotype five differentially managed treatments grown in four breeding cycles, totaling 20 site-years. During each flight, canopy reflectance was collected at 62 narrowband wavelengths between 400 and 850 nm. Hyperspectral reflectance-derived relationship matrices were calculated by taking the cross product of the reflectance data for all wavelengths. These relationship matrices were used within single-kernel and multi-kernel GBLUP models, the later using markers or pedigrees to estimate the genetic main effects and hyperspectral reflectance to estimate genotype-by-environment interactions, to predict grain yield within and across site-years. Prediction accuracies using the hyperspectral relationship matrices within single-kernel models performed similar to models using genomic markers or pedigrees to model genetic relationships. Overall, multi-kernel models integrating both hyperspectral reflectance and genomic markers or pedigrees increased prediction accuracy, but the increases were marginal or negligible for some site-years. These results show that genetic relationship matrices derived from aerial hyperspectral reflectance phenotypes may be a useful alternative to genomic markers/pedigrees for predicting grain yield within genomic selection, and that the greater accuracies may be achieved by integrating both genomic marker/pedigree information and hyperspectral reflectance phenotypes within multi-kernel prediction models. 3. Implement novel approaches using deep learning on 'breeder-trained' datasets to score developmental and other important agronomic traits 3.1. High-throughput, high-resolution imaging from UAV Using the M600 platform with DJI X5R ultra HD (4K) resolution video camera, we have collected hundreds of thousands of images of wheat plot throughout heading stages for the next phase of deep learning. We have developed image processing pipelines to extract high-resolution plot-level images for individual field plots from 4K video stream. For testing of connecting genotype-to-phenotype for the deep learning from UAV imaging during the 2017-18 wheat field season, we completed high-resolution imaging of winter wheat association mapping panel at 21 timepoints. For initial testing, we examined two datasets for imaging dates at the end of the season (May 31 and June4, 2018) to score awned/awnless. We extracted 8,602 individual image frames covering individual plots from each data sets and extracted over 94,000 images at 512x512 pixels from these frames to train the convolutional neural networks (CNNs). For a first validation we successfully trained the CNN to score awned / awnless phenotype in wheat directly from UAV images achieving 99% accuracy for individual crop images on a test datasets. CNN for percentage heading (0-100%) is in progress. 3.2. Image Embedding We are also exploring novel approaches to understand a more complete set of visual differences captured in the image data. We train a convolutional neural network to map images to a feature space using the criteria that images from the same plot should be closer together than images from different plots. Initial results show that phenotypes like awned/awnless are clustered automatically even though the algorithm was not explicitly given this meta-data. Work is ongoing to understand more completely the potential of this approach.? 4. Develop robust and efficient analysis pipelines that can facilitate community adoption of field-based HTP and enable in-season selection decisions We have focused efforts to develop fully automated pipelines to process new UAV image data directly to plot level measurements extracted from the images. From this, we have improved database performance and transferred all plot level information to geospatial data types including geo-referenced polygons for plots in the CIMMYT breeding program. 4.1. Automated processing pipeline for multi-spectral imaging We have developed the initial fully-automated pipeline for pre-processing UAV image data from the MicaSense RedEdge(-M) camera, including renaming, geospatial grouping, and radiometric calibration. The automated pipeline generates geo-rectified orthomosaic photos of wheat breeding field from UAV image data, including identification of ground control points for camera position optimization and photogrammetry processing UAV images. The image processing is followed by automated pipeline for extraction plot-level traits (e.g. vegetation indices and canopy height) from orthomosaic photos of wheat breeding field. 5. Assess optimal selection strategies for breeding programs that are implementing HTP and genomic profiling through simulation of potential genetic gain from all possible combinations of resource allocation 5.1. Yield prediction of small plots The irrigated bread wheat breeding program at CIMMYT utilizes a "selected bulk" scheme between the F3 and F5 generations in which one spike from each selected plant is harvested and bulked with spikes from the same cross to form the next generation. At the F6 stage, individual spikes are selected and promoted individually to the F7 which are sown as unreplicated double or triple rows within 0.7-1m plots in Ciudad Obregón. As yield measurements on these plots are unreliable and unfeasible to record, entries are selected visually for uniformity, disease resistance, and other agronomic characteristics. In an attempt to improve selection accuracy at the F7 stage, we are developing an aerial high-throughput phenotyping pipeline that would provide the breeder with quantitative canopy reflectance measurements for each genotype. During the 2016-17 and 2017-18 breeding cycles, sets of approximately 1000 genotypes were sown in two field tests: as unreplicated 0.7m plots and as twice-replicated yield trial plots. An unmanned aerial vehicle equipped with a multispectral sensor was flown over both field tests at multiple time points throughout the growing season. In both field tests, the heritability of aerial NDVI exceeded that of grain yield. Correlations between NDVI measurements upon the unreplicated 0.7m plots and grain yield estimates of the twice-replicated yield plots ranged from 0.35-0.43 in 2016-17 and 0.25-0.42 in 2017-18. These preliminary results indicate that canopy reflectance may be a useful selection tool at the F7 stage. What do you plan to do during the next reporting period to accomplish the goals?1. Develop and deploy robust and scalable unmanned aerial systems with a range of spectral and thermal imaging capabilities; including standardized protocols for routine collection of field-based HTP measurements using UAVs over large wheat breeding nurseries We have tested and deployed improved platforms for multispectral and thermal imaging. We will continue deployment of these platforms over field sites in Kansas, Mexico and India during 2017-18 field season. Complete flight coverage of all CIMMYT and KSU wheat breeding program plots at multiple timepoints throughout growing season Pending operational permits, we will extend UAV deployment in Pakistan and Bangladesh We will complete testing of large M600 UAV with combined camera payload of Headwall Hyperspec camera, FLIR thermal camera, Micasense RedEdge, and DJI X3 camera. Generate complete datasets of hyperspectral (400 - 1000nm) for KSU and CIMMYT breeding nurseries Complete video imaging of genetic populations and breeding plots during heading Complete foundational work to develop and validate integrated genomic and physiological models for yield prediction; including novel statistical modeling of hyperspectral data Test combined genotype x environment (GxE) models using spectral data from South Asia and CIMMYT, Mexico field trials Deployment of hyperspectral imaging from UAV (DJI M600 with Headwall Nano hyperspec) to transition from manned aircraft for hyperspectral data collection Test prediction models using UAV generated hyperspectral imaging Implement novel approaches using deep learning on 'breeder-trained' datasets to score developmental and other important agronomic traits Develop automated processing pipeline for plot-level image extraction from UAV video data Train convolutional neural networks using UAV extracted images to score heading percentage in wheat Test feature extraction for image processing to count number of heads Develop imaging embedding approaches for scoring phenotypes Develop robust and efficient analysis pipelines that can facilitate community adoption of field-based HTP and enable in-season selection decisions Develop and test improved processing pipelines for image extraction from video stream Develop the processing pipeline for plot-level extraction of canopy temperature values from UAV images Assess optimal selection strategies for breeding programs that are implementing HTP and genomic profiling through simulation of potential genetic gain from all possible combinations of resource allocation We will initiate simulation modeling of optimized selection strategies for early generation selection from small plots with varying levels of selection accuracy based on UAV derived yield predictions pilot study comparing small plot yield prediction with visual selections to assess selection gain and practical utility cost-benefit analysis of the UAV high-throughput phenotyping pipeline relative to current selection methods

Impacts
What was accomplished under these goals? Impact: To realize a new level of yield potential, breeding programs must increase the speed of developing new varieties (increase the rate of genetic gain) by evaluating larger populations, making more accurate selections, and decreasing the length of the breeding cycle. Genomic advancements during the past decade have enabled genomic prediction and selection of complex traits on larger number of breeding lines and at early stages in the breeding cycle. At the same time, however, phenotyping of breeding lines under field conditions has seen minimal advancement and is a critical bottleneck for evaluating large populations. Through this project, we are applying novel developments in remote sensing with unmanned aerial vehicles (UAVs) analyzed with machine vision and deep learning to make improved yield predictions directly within breeding programs in the US and internationally at the International Maize and Wheat Research Center (CIMMYT). We have tested and optimized UAV platforms that can be deployed in field crop breeding programs. Using the rich image datasets from the UAVs, we are working to generated in-season yield predictions by combining genomic information with multiple levels of proximal sensing and deep learning on tens of thousands of breeding lines. These optimized selection strategies using the full array of genomic and phenotypic information will be assessed and delivered to breeders. *How have the results been disseminated to communities of interest? To disseminate these advancements beyond the immediate project team across the breeding community, we are developing user-friendly software tools to analyze UAV imagery, along with efficient algorithms to generate breeder-ready predictions for selection. Progress on specific goals and objectives: 1. Develop and deploy robust and scalable unmanned aerial systems with a range of spectral and thermal imaging capabilities; including standardized protocols for routine collection of field-based HTP measurements using UAVs over large wheat breeding nurseries 1.1. Deployment of UAVs in KSU and CIMMYT breeding programs We deployed DJI M100 UAVs with Micasense RedEdge(-M) multi-spectral cameras in the K-State wheat breeding program locations across Kansas, the CIMMYT wheat breeding program at Obregon, Mexico, and at three locations in India. During the project year, we generated over 150,000 plot level observations for full yield plots and over 140,000 observations for small plots from across these locations. 1.2. High-resolution imaging from UAV To generate extremely high resolution (e.g. sub-mm pixel resolution) from a UAV, we completed testing and deployment of the DJI M600 with high-resolution 4K video camera recording RAW image frames at 24p. The M600 with integrated differential correction GPS (RTK-GPS) for precise positioning can be navigated directly over the plots at low altitude of 5m for extremely high resolution imaging. Using the X5R video imaging, we completed flights at 21 time points during the growing season in Kansas for a winter wheat association mapping panel to generate image dataset for scoring heading date and awned/awnless. 1.3. Hyperspectral imaging from UAV We continue with initial testing with Headwall Nano Hyperspectral camera (400 - 1000 nm) mounted on DJI M600 UAV. Pilots and technicians in the KSU group have completed additional training for operation. 1.4. Thermal imaging from UAV We have tested and obtained initial datasets for thermal cameras including DJI ZENMUSE XT and FLIR Vue Pro R. We have generated complete datasets for initial testing of thermal imaging for processing and plot level data extraction. 2. Complete foundational work to develop and validate integrated genomic and physiological models for yield prediction; including novel statistical modeling of hyperspectral data Using data from previous manned aircraft flights, we have tested multiple modeling approaches for incorporating spectral measurements to genomic selection models. We have developed and validated an approach to measure lodging in plots from Digital Elevation Models (DEMs) derived from UAV imaging. We have validated these UAV derived lodging measurements and the impact of the measured lodging on yield. A manuscript detailing this approach and findings has been submitted and is under review. 2.1. Estimation and yield prediction with early season ground cover Assessment of early season ground cover through conventional methods is time-consuming and subjective. We deployed an image-based supervised classification procedure to estimate digital ground cover from multi-time-point UAV imagery. Early season imaging was completed at multiple timepoints across three locations of advanced yield trials in India for the CIMMYT breeding program. A total of 39 multi-spectral datasets were generated and processed through the automated image processing pipeline. A total of 1.1 million unique data points (vegetation indexes and elevation for ~2650 plots at 39 time points ) was processed to explore the genetic and physiological underpinnings of ground cover in wheat. The digital measurements showed considerable variation in ground cover over the course of field season and among field plots. The digital ground cover assessment showed strong genetic correlations with visual estimates of ground cover (r = 0.85 - 0.99) suggesting that both estimates capture the same underlying genetic variation. In 5 out of 7 environments, the digital ground had higher repeatability (broad-sense heritability) than visual ratings. For a component trait for predicting grain yield, we examined the correlations between yield and the visual or digital assessment of ground cover. In 6 out of 7 comparisons, the digital measures had higher correlation to yield than did the visual assessment. We observed a positive effect of increasing thermal times on the relationship of ground cover and grain yield at our testing locations. Overall, these results highlight the superiority of digital ground cover measurements over conventional ground cover measurements. The accurate ground cover estimates will provide an important component trait for improving in-season yield prediction and for estimates of breeding values from multi-variate models.

Publications

  • Type: Journal Articles Status: Published Year Published: 2018 Citation: Belamkar, V., Guttieri, M. J., Hussain, W., Jarqu�n, D., El-Basyoni, I., Poland, J., Lorenz, A. J., & Baenziger, P. S. (2018). Genomic Selection in Preliminary Yield Trials in a Winter Wheat Breeding Program. G3 (Bethesda, Md.), g3.200415.2018. https://doi.org/10.1534/g3.118.200415
  • Type: Journal Articles Status: Published Year Published: 2018 Citation: Elbasyoni, I., Lorenz, A. J., Guttieri, M., Frels, K., Baenziger, P. S., Poland, J., & Akhunov, E. (2018). A Comparison Between Genotyping-by-sequencing and Array-based Scoring of SNPs for Genomic Prediction Accuracy in Winter Wheat. Plant Science. https://doi.org/10.1016/J.PLANTSCI.2018.02.019
  • Type: Journal Articles Status: Published Year Published: 2018 Citation: Juliana, P., Singh, R. P., Singh, P. K., Poland, J. A., Bergstrom, G. C., Huerta?Espino, J., Bhavani, S., Crossa, J., & Sorrells, M. E. (2018). Genome wide association mapping for resistance to leaf rust, stripe rust and tan spot in wheat reveals potential candidate genes. Theoretical and Applied Genetics, 131, 14051422. https://doi.org/10.1007/s00122-018-3086-6
  • Type: Journal Articles Status: Published Year Published: 2018 Citation: Crain, J., Mondal, S., Rutkoski, J., Singh, R. P., & Poland, J. (2018). Combining High-Throughput Phenotyping and Genomic Information to Increase Prediction and Selection Accuracy in Wheat Breeding. The Plant Genome, 11(1), 0. https://doi.org/10.3835/plantgenome2017.05.0043
  • Type: Journal Articles Status: Other Year Published: 2018 Citation: Crain, J., Reynolds, M., & Poland, J. (2017). Utilizing High-Throughput Phenotypic Data for Improved Phenotypic Selection of Stress-Adaptive Traits in Wheat. Crop Science, 57(2), 648659. https://doi.org/10.2135/cropsci2016.02.0135
  • Type: Journal Articles Status: Published Year Published: 2018 Citation: Wang, X., Singh, D., Marla, S., Morris, G., & Poland, J. (2018). Field-based high-throughput phenotyping of plant height in sorghum using different sensing technologies. Plant Methods, 14(1), 53. https://doi.org/10.1186/s13007-018-0324-5
  • Type: Conference Papers and Presentations Status: Other Year Published: 2018 Citation: Wang, X., Singh, D., Haghighattalab, A., Brown, R., & Poland, J. (2018). An Automated Image Processing Pipeline for High-Throughput Phenotyping with Unmanned Aerial Systems. NIFA FACT Workshop: Big Data Analytics in Plant Breeding and Satellite Data and Modeling Workshop, Arlington, VA.
  • Type: Conference Papers and Presentations Status: Other Year Published: 2018 Citation: Wang, X., & Poland, J. (2018). Improving Genomic Prediction with High-Throughput PhenotypingNo Title. 2018 Coalition for Advancing Digital Research & Education, Oklahoma State University, Stillwater, OK.
  • Type: Conference Papers and Presentations Status: Other Year Published: 2018 Citation: Poland J., D. Singh, A. Haghihattalab J. Rutkoski, A. Fritz, X. Wang, J. Crossa, R. Singh (2017) Application of UAVs to Increase Genetic Gain, National Association of Plant Breeders, Davis CA, Aug 2017


Progress 11/15/16 to 11/14/17

Outputs
Target Audience:The initial target audience for this project are research scientists and plant breeders in the U.S. and internationally.We have made presentations at international meetings, workshops, and publication of scientific papers in peer-review journals. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided?Through the first year of this project, we have contributed to the training and professional development of two graduate students and two postdoctoral research associates. Each of these students and postdocs have attended multiple research conferences and workshops, given poster and oral presentations and developed professional networks. Presentations: (Presenter Xu Wang) Wang, X. & Poland, J. (March 2017). sUAV phenotyping at Kansas State University. NSF Workshop in Field-based High Throughput Phenotyping. Maricopa, AZ (Presenter Daljit Singh) Singh, D., Wang, X., Kumar, U., McGahee, K., Lucas, M. &Poland, J. (2016). Unmanned Aerial System Based High-Throughput Phenotyping of Wheat Breeding Nurseries.Crop Science Society of America Annual Meeting, Phoenix, AZ (Presenter Daljit Singh)Singh, D., Wang, X., Kumar, U. & Poland, J. (2017)Genetic Analysis of Crop Lodging in CIMMYT Wheat Using UAV Based High-throughput Phenotyping.National Association of Plant Breeders, Davis, CA (Presenter Daljit Singh) Singh, D. (2016).High Throughput Phenomics: Towards rapid wheat cultivar development for South Asia.Board of International Food and Agricultural Development (BIFAD) Annual Public Meeting. Des Moines, Iowa (Presenter AtenaHaghighattalab)Haghighattalab, A. & Poland, J. (2017).Application of Moving Circular Spatial Adjustment (MCSA) to increase the accuracy of Spatial Adjustment of Field Trials. Phenome, 2017. Tuscon, AZ (Presenter Daljit Singh)Singh, D., Kumar, U., Wang, X., Lucas, M., Singh, R., Schinstock, D., & Poland, J. (2016). Phenotypic assessment of CIMMYT wheat varieties using a small Unmanned Aerial System. National Association of Plant Breeders Raleigh, NC. (Presenter Xu Wang) Wang, X., Singh, D., Haghighattalab, A., Mcgahee, K., Schinstock, D., & Poland, J. (2016). Canopy Height and GNDVI Extraction from sUAS Captured Images. Phenotypic Prediction: Image Acquisition and Analysis Workshop at Iowa State University Ames, IA. (Presenter Xu Wang) Wang, X., & Poland, J. (2016). Field-Based High-Throughput Phenotyping in Wheat. Kansas Extension Agricultural Agent Update Meeting Manhattan, KS. (Presenter Xu Wang)Wang, X., Lucas, M., Schinstock, D., & Poland, J. (2016). Development and Implementation of a Small Unmanned Aerial Systems Based Phenotyping Pipeline for Plant Breeding Programs. ASA, CSSA and SSSA International Annual Meetings, Phoenix, AZ. How have the results been disseminated to communities of interest? Nothing Reported What do you plan to do during the next reporting period to accomplish the goals?Develop and deploy robust and scalable unmanned aerial systems with a range of spectral and thermal imaging capabilities; including standardized protocols for routine collection of field-based HTP measurements using UAVs over large wheat breeding nurseries We have tested and deployed improved platforms for multispectral and thermal imaging. We will continue deployment of these platforms over field sites in Kansas, Mexico and India during 2017-18 field season. Pending operational permits, we will extend UAV deployment in Pakistan and Bangladesh We will complete testing of large M600 UAV with combined camera payload of Headwall Hyperspec camera, FLIR thermal camera, Micasense RedEdge, and Sony RGB camera. Generate complete datasets of hyperspectral (400 - 1000nm) for breeding nurseries Complete flight coverage of all CIMMYT and KSU wheat breeding program plots at multiple timepoints throughout growing season Complete video imaging of breeding plots during heading Complete foundational work to develop and validate integrated genomic and physiological models for yield prediction; including novel statistical modeling of hyperspectral data Test combined genotype x environment (GxE) models using spectral data from South Asia and CIMMYT, Mexico field trials Implement novel approaches using deep learning on 'breeder-trained' datasets to score developmental and other important agronomic traits Develop processing pipeline for plot-level image extraction from UAV video data Train convolutional neural networks using UAV extracted images to score heading percentage in wheat Test feature extraction for image processing to count number of heads Develop robust and efficient analysis pipelines that can facilitate community adoption of field-based HTP and enable in-season selection decisions Complete and deploy fully automated processing pipeline for plot-level extraction of vegetation index values from raw UAV images Develop and test improved processing pipelines for stitching and geo-rectification of images from video stream Assess optimal selection strategies for breeding programs that are implementing HTP and genomic profiling through simulation of potential genetic gain from all possible combinations of resource allocation We will initiate simulation modeling of optimized selection strategies for early generation selection from small plots with varying levels of selection accuracy based on UAV derived yield predictions

Impacts
What was accomplished under these goals? Impact To realize a new level of yield potential, breeding programs must increase the speed of developing new varieties (increase the rate of genetic gain) by evaluating larger populations, making more accurate selections, and decreasing the length of the breeding cycle. Genomic advancements during the past decade have enabled genomic prediction and selection of complex traits on larger number of breeding lines and at early stages in the breeding cycle. At the same time, however, phenotyping of breeding lines under field conditions has seen minimal advancement and is a critical bottleneck for evaluating large populations. Through this project we are applying novel developments in remote sensing with unmanned aerial vehicles (UAVs) analyzed with machine vision and deep learning to make improved yield directly within breeding programs in the US and internationally at the International Maize and Wheat Research Center (CIMMYT). We have tested and optimized UAV platforms that can be deployed in field crop breeding programs. Using the rich image datasets from the UAVs, we are working to generated in-season yield predictions by combining genomic information with multiple levels of proximal sensing and deep learning on tens of thousands of breeding lines. These optimized selection strategies using the full array of genomic and phenotypic information will be assessed and delivered to breeders. To disseminate these advancements beyond the immediate project team across the breeding community, we are developing user-friendly software tools to analyze UAV imagery, along with efficient algorithms to generate breeder-ready predictions for selection. Progress on specific goals and objectives: Develop and deploy robust and scalable unmanned aerial systems with a range of spectral and thermal imaging capabilities; including standardized protocols for routine collection of field-based HTP measurements using UAVs over large wheat breeding nurseries We have tested and deployed an improved UAV platform, the DJI M100. We have also completed testing of different cameras including Micasense RedEdge and NIR converted DJI X3. Multiple M100 units with RedEdge cameras were put into service this year, making hundreds of flight missions across the project locations in Kansas, India and Mexico. We have generated a combined 100,000+ plot level observations from across these locations. We have tested a larger UAV platform, the DJI M600, with integrated differential correction GPS (RTK-GPS) for precise positioning, longer flights and heavier cameras. We have obtained and completed initial testing with Headwall Hyperspectral camera (400 - 1000 nm) We have tested and obtained initial datsets for thermal cameras including DJI ZENMUSE XT and FLIR Vue. We have generated completed datasets for thermal imaging for further processing. Complete foundational work to develop and validate integrated genomic and physiological models for yield prediction; including novel statistical modeling of hyperspectral data using data from previous manned aircraft flights, we have tested multiple modeling approaches for incorporating spectral measurements to genomic selection models we have developed and validated an approach to measure lodging in plots from Digital Elevation Models (DEMs) derived from UAV imaging. We have validated these UAV derived lodging measurements and the impact of the measured lodging on yield Implement novel approaches using deep learning on 'breeder-trained' datasets to score developmental and other important agronomic traits Using previously collected image datasets from the ground phenotyping platform, we have trained convolutional neural networks to score awned / awnless and percentage heading (0-100%) in wheat directly from images By applying neural network scoring for percent heading scores to time-series imaging we can identify heading date as the midpoint where a given plot is 50% headed Using the M600 platform with DJI X5R ultra HD (4K) resolution video camera, we have collected hundreds of thousands of images of wheat plot throughout heading stages for the next phase of deep learning Develop robust and efficient analysis pipelines that can facilitate community adoption of field-based HTP and enable in-season selection decisions We have focused effort to develop fully automated pipelines to process new UAV image data directly to plot level measurements extracted from the images We have improved database performance and transferred all plot level information to geospatial data types We have developed initial automated pipeline for pre-processing all UAV image data from RedEdge cameras We have developed initial automated for identification of ground control points, image registration and image stitching to geo-referenced orthomosaic combined with plot coordinate overlay to extract plot level data

Publications

  • Type: Journal Articles Status: Published Year Published: 2017 Citation: Sun, J., J. E. Rutkoski, J. A. Poland, J. Crossa, J.-L. Jannink and M. E. Sorrells (2017) Multitrait, random regression, or simple repeatability model in high-throughput phenotyping data improve genomic prediction for wheat grain yield. The Plant Genome 10(2). DOI: 10.3835/plantgenome2016.11.0111
  • Type: Journal Articles Status: Published Year Published: 2017 Citation: Haghighattalab, A., J. Crain, S. Mondal, J. Rutkoski, R. P. Singh and J. Poland (2017) Application of geographically weighted regression to improve grain yield prediction from unmanned aerial system imagery.� Crop Science. DOI: 10.2135/cropsci2016.12.1016
  • Type: Journal Articles Status: Published Year Published: 2016 Citation: Haghighattalab, A., L. Gonz�lez P�rez, S. Mondal, D. Singh, D. Schinstock, J. Rutkoski, I. Ortiz-Monasterio, R. P. Singh, D. Goodin and J. Poland (2016) Application of unmanned aerial systems for high throughput phenotyping of large wheat breeding nurseries. Plant Methods 12(1): 1-15. DOI: 10.1186/s13007-016-0134-6
  • Type: Conference Papers and Presentations Status: Other Year Published: 2016 Citation: Poland J., D. Singh, A. Haghihattalab J. Rutkoski, A. Fritz, X. Wang, J. Crossa, R. Singh (2017) Application of UAVs to Increase Genetic Gain, National Association of Plant Breeders, Davis CA, Aug 2017
  • Type: Conference Papers and Presentations Status: Other Year Published: 2017 Citation: Poland J., D. Singh, A. Haghihattalab, A. Fritz, X. Wang, J. Crossa, R. Singh (2017) Accelerating Wheat Breeding with Genomic Selection and High Throughput Phenotyping, National Agriculture Research Council - Pakistan, Islamabad, Pakistan, April 2017
  • Type: Conference Papers and Presentations Status: Other Year Published: 2017 Citation: Poland J., S. Mondal, J. Crossa, R. Singh (2017) Accelerating Wheat Breeding with Genomic Selection, Wheat Research Institute, Faisalabad, Pakistan, April 2017
  • Type: Conference Papers and Presentations Status: Other Year Published: 2017 Citation: Poland J., S. Battenfield, J. Rutkoski, A. Fritz, X. Wang, J. Crossa, R. Singh (2017) Accelerating Wheat Breeding with Genomic Selection and High Throughput Phenotyping, University of Maryland, Department of Plant Science, Feb 2017
  • Type: Conference Papers and Presentations Status: Other Year Published: 2017 Citation: Poland J., A. Haghihattalab, S. Battenfield, J. Rutkoski, A. Fritz, X. Wang, J. Crossa, R. Singh (2017) Increasing Yield Prediction Accuracy for Wheat Breeding Through Combined Genomic Selection and Field-based High Throughput Phenotyping, PHENOME 2017, Tuscon AZ, Feb 2017
  • Type: Conference Papers and Presentations Status: Other Year Published: 2017 Citation: Poland, J., X. Wang, R. Pless, H. Wang (2017) Application of Artificial Neural Networks and Real Neural Networks to Field-based High Throughput Phenotyping, Plant and Animal Genome - National Plant Genome Initiative Workshop, San Diego CA, Jan 2017
  • Type: Conference Papers and Presentations Status: Other Year Published: 2017 Citation: Poland J., S. Mondal, A. Haghihattalab, S. Battenfield, J. Rutkoski, A. Fritz, X. Wang, J. Crossa, R. Singh (2017) Genomic Selection and High Throughput Phenotyping for Wheat Improvement, Bangladesh Agriculture Research Institute, Jamalapur Bangladesh, Jan 2017
  • Type: Conference Papers and Presentations Status: Other Year Published: 2016 Citation: Poland, J., X. Wang, R. Pless, H. Wang (2017) Deep learning for high-throughput phenotyping of complex trait in wheat. ASA-CSSA International Annual Meetings, Phoenix AZ, Nov 2016
  • Type: Conference Papers and Presentations Status: Other Year Published: 2017 Citation: Poland, J. Haghighattalab, A., Wang, X. & Singh, D. (2017) Field-Based High Throughput Phenotyping using Proximal and Remote Sensing Technologies. Nebraska NRIC Predictive Crop Design: Genome-to-Phenome Conference. Lincoln, NE
  • Type: Conference Papers and Presentations Status: Other Year Published: 2016 Citation: (Presenter Daljit Singh)�Singh, D., Kumar, U., Wang, X., Lucas, M., Singh, R., Schinstock, D., & Poland, J. (2016). Phenotypic assessment of CIMMYT wheat varieties using a small Unmanned Aerial System. National Association of Plant Breeders Raleigh, NC.
  • Type: Conference Papers and Presentations Status: Other Year Published: 2016 Citation: (Presenter Xu Wang) Wang, X., Singh, D., Haghighattalab, A., Mcgahee, K., Schinstock, D., & Poland, J. (2016). Canopy Height and GNDVI Extraction from sUAS Captured Images. Phenotypic Prediction: Image Acquisition and Analysis Workshop at Iowa State University Ames, IA.
  • Type: Conference Papers and Presentations Status: Other Year Published: 2016 Citation: (Presenter Xu Wang) Wang, X., & Poland, J. (2016). Field-Based High-Throughput Phenotyping in Wheat. Kansas Extension Agricultural Agent Update Meeting Manhattan, KS.
  • Type: Conference Papers and Presentations Status: Other Year Published: 2016 Citation: (Presenter Xu Wang)�Wang, X., Lucas, M., Schinstock, D., & Poland, J. (2016). Development and Implementation of a Small Unmanned Aerial Systems Based Phenotyping Pipeline for Plant Breeding Programs. ASA, CSSA and SSSA International Annual Meetings, Phoenix, AZ.