Progress 05/15/24 to 05/14/25
Outputs
Target Audience:The target audience isscientists in fields of agricultural high-throughput phenotyping and computer vision as well as peanut breeders and geneticists. Changes/Problems:
Nothing Reported
What opportunities for training and professional development has the project provided?Three graduate students have participated part time in the project - Javier Rodriguez-Sanchez, Lizhi Jiang, Prasanna Kharel. How have the results been disseminated to communities of interest?One manuscript has been submitted for publication. What do you plan to do during the next reporting period to accomplish the goals?Continue developing phenotyping tools and applying them to the MAGIC population.
Impacts
What was accomplished under these goals?
Objectives relevant to Year 3 activities Complete construction of a 16-way peanut MAGIC population through speed breeding. Develop and validate high throughput phenotyping platforms for yield related traits. Perform high throughput phenotyping on the MAGIC core set Progress on MAGIC population development MAGIC population 16-way crosses were completed and harvested in summer 2022. Eighteen genotypes were included as founder parents in MAGIC population development. The parents were selected to represent diverse genetic backgrounds as well as to combine one or more beneficial traits of each parent into progenies of the MAGIC population. A total of 420 F1 hybrid seeds were obtained. The F1 seeds were equally divided for advancement in two locations: one half at the University of Georgia Tifton Campus and one half at the Hudson Alpha Institute for Biotechnology, Huntsville, Alabama. Out of the total F1 hybrids, 210 F1 hybrids were planted in the greenhouse at the UGA Tifton Campus in fall 2022 using speed breeding techniques, although only 157 F1s produced one or more F2 seeds. A total of 811 F2 seeds were planted in the greenhouse for transplanting to the field in spring 2023 out of which 657 seeds germinated representing 153 F1 families. F3 seeds were harvested from each F2 plant in the fall 2023. Out of the total 657 F2 lines that produced F3 seeds, 349 lines representing all 153 F1 families were selected to advance in the greenhouse. These lines were selected as an attempt to equalize each parents' contribution to the population. A single F3 seed from each selected F2 line was germinated in peat pots and densely transplanted in tubs in the greenhouse to produce F4 lines to be planted in summer 2024. In total, 333 F3 lines were advanced in the field in Summer 2024 to yield F4 seeds. The field was sprayed with insecticide for thrips control to manage TSWV infection. The remaining 308 F2 lines that were not advanced in the greenhouse in fall 2023 were advanced in the field in summer 2024. A total of 270 F5 lines were sent to Puerto Rico for advancement including 16 parents. Only those lines that produced more than 10 F5 seeds were sent for advancement. An additional 36 lines that produced less than 10 seeds were planted in the GH. These pods have been shelled and will be planted for plot-level phenotyping in summer 2025. DNA was extracted from 2-way, 4-way, 8-way, and 16-way F1 individuals and parental lines. A total of 877 samples were sent for sequencing at HudsonAlpha Institute for Biotechnology, Huntsville, AL. Low-coverage short-read RipTide library sequencing was conducted, and the samples were genotyped for SNPs using the Khufu pipeline developed by HudsonAlpha. Forty-three sub-samples representative of the MAGIC population development at different stages were sent for deeper coverage sequencing, which will help better understand the recombination events at each stage of development. The SNP variant calling is based on the pangenome graph developed through another project using the founder lines of the MAGIC population. Progress on high-throughput phenotyping UAV-based high throughput phenotyping of yield-related traits. The UAV-based phenotyping pipeline has been described for publication in a submitted manuscript entitled "Aerial Imagery and Segment Anything Model for Architectural Trait Phenotyping to Support Genetic Analysis in Peanut Breeding." The target traits include canopy height, growth habit, and mainstem prominence. Models were trained and the high-throughput phenotyping output traits were used for QTL mapping in test populations along with the manually derived traits. The results were consistent with conventional QTL mapping methods, demonstrating that UAV-based phenotyping provides reliable trait data for genetic studies in peanut breeding. To collect preliminary data on phenotyping of the MAGIC population core set, 1800 F4 individuals and parents were transplanted in the field in 2024, 2 plants per rep with 3 reps per line. At 6 weeks after transplanting (WAT), the traits of canopy height, maximum canopy width (in any direction), and perpendicular canopy width were collected along with RGB aerial images. At 8 WAT, traits collected were TSWV infection, main stem prominence, and growth habit along with RGB aerial images. At 10 WAT: canopy height, maximum canopy width (in any direction), perpendicular canopy width; RGB aerial images. At 12 WAT: RGB aerial images. At 14 WAT: TSWV infection and multispectral aerial images. These data are being analyzed with the above described pipeline to test correlation with manually collected data for certain traits. Peanut plant 3D reconstruction In 2024, a new round of data collection was conducted on peanut plants of the same genotypes previously sampled in 2023. The data acquisition took place indoors. To support the peanut plants during imaging, a supporting frame was suspended from the ceiling. For each plant, 360-degree panoramic videos were recorded using a smartphone (iPhone 11), covering both leafy and defoliated conditions. A handheld stabilizer was employed throughout the filming process to minimize camera shake and ensure video clarity. In total, data were collected from 108 plants across 36 genotypes, with three individual plants sampled per genotype. For 2D peanut pod counting using the peanut plant dataset collected in 2023, we employed YOLOv8x for pod segmentation. On this dataset, the model achieved a precision (P) of 0.857, recall (R) of 0.823, mAP@0.5 of 0.887, and mAP@0.5:0.95 of 0.521. We selected 36 peanut plants of different genotypes for pod counting, resulting in a mean absolute percentage error (MAPE) of 19.28% for leafy plants and 12.26% for defoliated plants. We then applied the YOLOv8x model trained on the 2023 data to perform pod counting on the 2024 dataset, which included 108 peanut plants. The counting errors for leafy and defoliated plants were 30.95% and 26.32%, respectively, indicating a decline in accuracy. To address this, we plan to annotate a subset of the 2024 data to fine-tune the YOLOv8x model in the next step. The 2023 image data also were used to conduct 3D segmentation of peanut plants using the FruitNeRF model. Pods on both leafy and defoliated plants were successfully segmented, with improved segmentation performance observed on defoliated plants. Future research will include annotating approximately 100 new samples from the 2024 dataset, focusing primarily on labeling the pods to fine-tune the YOLOv8x model, which was initially trained on the 2023 data, to improve its testing accuracy on the 2024 dataset. The fine-tuned YOLOv8x model also will be re-evaluated on the 2024 dataset for pod detection and counting, and FruitNeRF will be used to perform 3D segmentation on peanut plants from both the 2023 and 2024 datasets, enabling pod counting and evaluation.
Publications
|
Progress 05/15/23 to 05/14/24
Outputs
Target Audience:Scientists in fields of agricultural high-throughput phenotyping and computer vision and peanut genetics. Changes/Problems:
Nothing Reported
What opportunities for training and professional development has the project provided?Three graduate students have participated part time in the project - Javier Rodriguez-Sanchez, Lizhi Jiang, Prasanna Kharel How have the results been disseminated to communities of interest? Through scientific meetings What do you plan to do during the next reporting period to accomplish the goals?Continue developing phenotyping tools and advancing MAGIC population.
Impacts
What was accomplished under these goals?
?Progress on MAGIC population development MAGIC population 16-way crosses were completed and harvested in summer 2022. Eighteen genotypes were included as founder parents in MAGIC population development. The parents were selected to represent diverse genetic backgrounds as well as to combine one or more beneficial traits of each parent into progenies of the MAGIC population. A total of 420 F1 hybrid seeds were obtained. The F1 seeds were equally divided for advancement in two locations: one half at the University of Georgia Tifton Campus and one half at the Hudson Alpha Institute for Biotechnology, Huntsville, Alabama. Out of the total F1 hybrids, 210 F1 hybrids were planted in the greenhouse at the UGA Tifton Campus in fall 2022 using speed breeding techniques, although only 157 F1s produced one or more F2 seeds. A total of 811 F2 seeds were planted in peat pots in the greenhouse for transplanting to the field in spring 2023 out of which only 657 seeds germinated representing 153 F1 families. F3 seeds were harvested from each F2 plant in the fall 2023. Out of the total 657 F2 lines that produced F3 seeds, 349 lines representing all 153 F1 families were selected to advance in the greenhouse. These lines were selected as an attempt to equalize each parents' contribution to the population. A single F3 seed from each selected F2 line was germinated in peat pots and densely transplanted in tubs in the greenhouse to produce F4 lines to be planted in summer 2024. The remaining 308 F2 lines that were not advanced in the greenhouse in fall 2023 will be transplanted in the summer 2024 for advancement. DNA was extracted from 2-way, 4-way, 8-way, and 16-way F1 individuals and parental lines and genotyped using RipTide sequencing and Khufu for SNP calling. The half at HudsonAlpha has been advanced twice from F1. At each advancement, representative seeds for advancement and harvest were planted from one plant per family. Currently the F4 generation is growing in the greenhouse and growrooms at HudsonAlpha in Hunstville and represents 209 families. A total of 139 families were included in our HudsonAlpha wiregrass education program and were genotyped using low coverage whole genome sequencing and low coverage long read (PacBio HIFI) sequencing. Those families will mature in June/July 2024. Progress on high-throughput phenotyping UAV-based high throughput phenotyping of yield-related traits. Accurate identification of the experimental field was accomplished using SAM (Segment Anything Model). SAM is a prompt-based image segmentation model capable of generating high-quality masks of objects, offering zero-shot generalization to unfamiliar objects. By applying SAM's auto-mask generator to the orthomosaic image, we initially obtained a set of 3072 potential masks, each identifying objects within the field. Assuming the field of interest is roughly centered in the orthomosaic image, we filtered these masks based on location and size to narrow down the set to 37 masks. Ranking these masks based on their quality (i.e., predicted Intersection over Union (IoU) and stability score) enabled us to select the most suitable one for our experimental field. Finally, fitting a rectangle of minimal area allowed us to determine the orientation of the field relative to a local Cartesian coordinate system, with the X-axis aligned with the row direction and the Y-axis perpendicular to it. Once the field was identified, we proceeded to extract it from the orthomosaic for further processing. To enhance height estimations, we implemented a preprocessing step for height normalization. This new methodology along with implementation of the multi-point prompt version of the SAM model enabled us to enhance height estimations, increasing the R2 value from 0.697 to 0.787 compared to the previous approach. To estimate more complex traits such as growth habit (GH) or main stem prominence (MP), we used a neural network architecture based on CNNs and a variation of the LeNet-5 architecture, comprising 2 convolutional layers, 3 fully connected layers, and 2 average pooling layers. We split our datasets into 80/20 for training and testing, further splitting the training dataset into 80/20 for validation. The MP classifier achieved an accuracy of 70%, with weighted average precision, recall, and F1-score values of 0.71, 0.70, and 0.71, respectively. Meanwhile, the GH classifier achieved an average accuracy of 84%, with weighted average precision, recall, and F1-score values of 0.85, 0.84, and 0.84, respectively. These results demonstrate the effectiveness of employing a neural network architecture based on CNNs for estimating complex traits in crop phenotyping. We achieved promising results in accurately predicting growth habit (GH) and main stem prominence (MP) traits that can facilitate more robust phenotyping in crop breeding programs. PeanutNeRF: 3D Radiance Field for Peanut plant 3D reconstruction and detection Replicate plants from each of the MAGIC parents were collected at the time of digging for data collection in the lab, which ensured that there was no complex background for imaging. A total of 34 genotypes and 107 plants were collected. A cloth hanger hanging from the ceiling was used to open up and support peanut plants. Light tubes above and below were used to enhance the brightness for imaging. A mobile phone (iPhone 11) was used to take 360-degree videos around the peanut plants before and after defoliation. Multi-view 2D images can be extracted from the captured videos of peanut plants for further study. Neural Radiance Fields (NeRF) was used to generate 3D point clouds. In this study, the trained generated 3D model can provide architectural traits such as the branch length of peanut plants. The main steps involve initially extracting a certain number (e.g., 150 images) of multi-view images from the video. Subsequently, the Nerfacto or Splatfacto models provided by Nerfstudio were used for training and obtaining the 3D model. Nerfacto produces point clouds, while Splatfacto generates 3D Gaussian splatting models. YOLOv8x was used for segmenting peanut pods in 2D images. Roboflow was utilized to annotate the polygons of objects. It is a 2D data annotation tool capable of automatically extracting the required information from the raw image data provided by developers and converting it into formats directly usable by YOLOv8. A total of 220 images were annotated, with each peanut pod marked using a polygonal annotation. The dataset was split into a ratio of 7:2:1 for training, validation, and testing, respectively. After training the model, we obtained the following test results: Precision (P) = 0.857, Recall (R) = 0.823, mAP@0.5 = 0.887, and mAP@0.5-0.95 = 0.521. This indicates that a relatively high segmentation result was achieved. Each pod was predicted with a mask, and by counting the number of predicted masks, the number of pods for each peanut plant was determined. A 3D Gaussian splatting model also was tested using Segment Any 3D Gaussians (SAGA). SAGA is an innovative 3D interactive segmentation method that combines the foundational 2D segmentation model with the 3D Gaussian splatting model. Through meticulously designed contrastive training, SAGA efficiently incorporates multi-scale 2D segmentation results into 3D Gaussian point features, thereby enabling multi-scale 3D segmentation. This approach was used to segment small, immature pegs.
Publications
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2023
Citation:
Farah Saeed, Jin Sun, Peggy Ozias-Akins, Ye Juliet Chu, Changying Charlie Li. 2023. PeanutNeRF: 3D Radiance Field for Peanuts. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Pages 6254-6263.
|
Progress 05/15/22 to 05/14/23
Outputs
Target Audience:scientists in fields of agricultural high-throughput phenotyping and computer vision Changes/Problems:
Nothing Reported
What opportunities for training and professional development has the project provided?Three graduate students have participated part time in the project -Javier Rodriguez-Sanchez, Farah Saeed, Prasanna Kharel How have the results been disseminated to communities of interest?CVPR Agricultural Vision Workshop - paper presented 2023 Artificial Intelligence in Agriculture conference - poster presented and won 2nd place What do you plan to do during the next reporting period to accomplish the goals?Continue developing phenotyping tools and advancing MAGIC population.
Impacts
What was accomplished under these goals?
Year 1 NIFA project progress report Genetic dissection of yield-related traits enabled by high throughput phenotyping and whole genome sequencing of a peanut MAGIC population Peggy Ozias-Akins, Changying Li, Josh Clevenger, Corley Holbrook, Ye Chu Students: Javier Rodriguez-Sanchez, Farah Saeed, Prasanna Kharel Objectives relevant to Year 1 activities Complete construction of a 16-way peanut MAGIC population through speed breeding. Develop and validate high throughput phenotyping platforms for yield related traits. Progress on MAGIC population development MAGIC population 16-way crosses were completed and harvested in summer 2022. Nine cross combinations yielded 420 F1 seeds which were split between the UGA Tifton Campus and HudsonAlpha for advancement. The UGA half was grown in the greenhouse under speed breeding conditions (dense planting and 22h light) to allow advancement prior to the field 2023 season. Twenty-five seeds did not germinate. Of the 185 seeds that did germinate, one or more seeds was harvested from 161 lines. These seeds are being treated to break dormancy and will be planted or transplanted to the field in May 2023. A few of the lines that have been slow to develop remain in the greenhouse. Progress on high-throughput phenotyping UAV-based high throughput phenotyping of yield-related traits. Field phenotyping for plant breeding involves analyzing crop traits at the plot level as a crucial aspect for crop improvement. However, accurate plot segmentation from orthomosaic images is a complex and time-consuming task. Traditionally, this has been done by manually marking polygonal regions around every visible plot, but this approach can be prone to errors and is impractical for fields with thousands of plots. Semi-automatic approaches that use a grid of cells based on the number of rows and columns planted in the field can be more efficient, but it may not work well for breeding fields planted without precision machinery. To address these challenges, a fully automated AI-based plot segmentation approach was implemented. First, the vegetation pixels in the orthomosaic image were classified using the Excess Green minus Excess red index (ExG-ExR) (Meyer and Neto, 2008). These pixels were then added in the vertical direction (i.e., top to bottom). A peak detection algorithm allowed us to roughly identify each column of plots in the field. Working on each segmented column, we added the vegetation pixels in the horizontal direction. Using another peak detection algorithm, we located the center point of each individual plot. Using the detected plot centers as "seeds", a watershed algorithm (Meyer, 1991) was used to obtain the boundary for each plot which was stored as geospatial vector data using the shapefile file format. By applying this shapefile to the orthomosaic image we can extract growth and health canopy-related traits such as ground canopy cover and TSWV rating at the individual plot level. In addition to the ortho image, we also applied the shapefile to the dense point cloud generated during the orthomosaicking process. By leveraging the dense point cloud data and the shapefile boundary information, we were able to accurately analyze the morphological traits at the plot level. These traits, including plot height, plot volume, growth habit, and mainstem prominence, are essential for understanding the physical characteristics of the plants in the field and can provide valuable insights for agricultural research. In order to assess the accuracy of our methodology for plot height estimation, we conducted a quantitative evaluation by comparing the estimated heights to manually measured ground truth values for 50 plots in the field. Our findings revealed a significant correlation between the estimated and ground truth plot heights, with a strong linear relationship (R2=0.697) as shown. These results demonstrate the effectiveness of our methodology in accurately estimating plot heights and provide confidence in the reliability of our approach for field phenotyping research. Based on our initial analysis of the dense point cloud, we have observed that this approach has great potential for analyzing more complex morphological traits such as growth habit and main stem prominence. Our analysis has revealed distinct visual differences in shape for plants with different growth habits, as illustrated. These differences can potentially be exploited for classifying and identifying plants based on their morphological characteristics using advanced deep learning techniques. These preliminary findings demonstrate the power and versatility of point cloud data in enabling more comprehensive and accurate analysis of plant phenotypes. Our next steps will be focused on automating the entire data analytics workflow using deep learning techniques for estimating traits and relating them with final yield. By leveraging this data and utilizing advanced data analytics, we might obtain new insights into the characteristics of the crop, ultimately leading to more informed and effective practices in peanut breeding. PeanutNeRF: 3D Radiance Field for Peanut plant 3D reconstruction and detection Our paper (Saeed et al 2023) submitted to the CVPR Agricultural Vision Workshop was accepted. The paper presents a novel framework for conducting phenotypic analysis of peanuts using 3D radiance fields. We propose a combination of 2D and 3D data analysis to overcome the limitations of using only 2D images from RGB cameras for plant trait estimation. The framework first captures 2D data of a scene containing peanut plants, which is easily accessible and requires less human supervision. Then, we train 3D radiance fields using NeRF to obtain an implicit representation of the scene where they could sample 3D point clouds. The proposed framework has several advantages over traditional methods. It provides more accurate and efficient ways to analyze plant traits in peanuts, which can help plant breeders identify and analyze suitable plant traits to enhance crop yield. Additionally, it addresses the limitations of using only 2D images from RGB cameras for plant trait estimation, such as occlusion and the absence of depth information. Overall, this paper presents an innovative approach to phenotypic analysis in plant breeding that has significant potential for improving crop yield through more accurate and efficient identification and analysis of suitable plant traits. In future studies, we aim to perform detection from reconstruction covering a 360-degree view of the plant. In addition to pod detection, we aim to detect the thin parts including pegs recovered from the 3D reconstruction. References Meyer, F. (1991). An optimal algorithm for the watershed line. In 8th Congress of Pattern Recognition and Artificial Intelligence. 2, 847-857, Lyons, France. Meyer, G., and Neto, J. C. (2008). Verification of color vegetation indices for automated crop imaging applications. Comput. Electron. Agric. 63, 282-293. Farah Saeed, Jin Sun, Peggy Ozias-Akins, Ye Chu, Changying Li. PeanutNeRF: 3D Radiance Field for Peanuts. CVPR workshop 2023.
Publications
- Type:
Conference Papers and Presentations
Status:
Accepted
Year Published:
2023
Citation:
Farah Saeed, Jin Sun, Peggy Ozias-Akins, Ye Chu, Changying Li. PeanutNeRF: 3D Radiance Field for Peanuts. CVPR workshop 2023
|
|