Progress 01/15/21 to 01/14/25
Outputs
Target Audience: Our target audience includes the following groups: 1. Faculty and scientists working in plant disease management, metagenomics, plant pathology, and agricultural automation and robotics. 2. Graduate and undergraduate students in agriculture-related disciplines, particularly those focused on plant disease management and automation. 3. Agricultural producers, farmers, and crop advisors involved in plant production within greenhouses and controlled environments. 4. State commodity board and CALS advisory board members, including high-level executives, business owners, and state policymakers. Changes/Problems:
Nothing Reported
What opportunities for training and professional development has the project provided? The project has provided various opportunities for training and professional development across its different research objectives. Research Objective One (Autonomous System for Plant Sample Extraction/Collection and Image Collection): In the first year, an undergraduate intern, Sohan Kulkarni, received training in the design of the end-effector for selective leaf sampling, exploring both rigid and compliance-based mechanisms. Also in the first year, Dominic Guri, a Ph.D. student, received hands-on robotics experience, with training focused on motion planning and scheduling of robotic tasks for automated image data collection. In the second year, Simi Asher, a graduate intern, was trained in the implementation of various deep learning-based approaches to 3D reconstruct plant models using multiple stereo image pairs and comparing them to traditional methods. Also in the second year, Dominic Guri, a Ph.D. student, continued to gain hands-on robotics experience, with training mostly involving motion planning and scheduling of robotic tasks for automated image data collection pipeline. In the third year, Sohan Kulkarni, a freshman in Mechanical engineering, designed the early version of the end-effector as part of his research credit. Another undergraduate, Srecharan Selvam, Mechanical engineering, and a senior high school student, Ben Tarr, together designed the microneedle dispensing system and its software. Simi Asher, a robotics student, implemented deep learning approaches for 3D plant model reconstruction and compared them to traditional methods. Dominic Guri, a Ph.D. student, worked as a coordinator and gained hands-on robotics experience, also working with other graduate and undergraduate students in motion planning and automated image data collection. Research Objective Two (Microfluidic Device for Sequencing-Based Pathogen Detection): In the first year, a graduate student, Thomas Hadlock, was trained in DNA sequencing pipeline development and microfluidic device design and fabrication. He was also trained in nanopore sequencing and data analysis protocols. Also in the first year, an undergraduate student, Katie Orr, received training in microfluidic device fabrication and sample processing procedures for operating the semi-automated process. In the second year, two graduate students, Thomas Hadlock and Jacob Neice, were trained in DNA sequencing pipeline development and microfluidic device design and fabrication. They also received training in nanopore sequencing and data analysis protocols. In the third year, two graduate students, Thomas Hadlock and Jacob Neice, continued to be trained in DNA sequencing pipeline development, microfluidic device design and fabrication, nanopore sequencing, and data analysis protocols. Research Objective Three (Imaging-Based Disease Detection in a Greenhouse Setting): In the first year, a postdoc associate, Missi Zhang, was trained in plant pathology and hyperspectral imaging. Also in the first year, a postdoc associate, Wei Xing, was trained in using machine learning to analyze hyperspectral data. Two undergraduate interns (in Li Lab) were trained in developing phenotyping robots for greenhouse producers. In the second year, a postdoc associate, Missi Zhang, was trained in plant pathology and hyperspectral imaging. One master student, Upasana Sivaramakrishnan (in Li Lab), was trained in developing phenotyping tools for analyzing hyperspectral data and a 3D LiDAR model of tomato seedlings. One Ph.D. student, Victor Wang, was trained to develop a 3D model of tomato seedlings using a 3D LiDAR scan and compared the plant height between infected and healthy plants. In the third year, a postdoc associate, Missi Zhang, received training in plant pathology and hyperspectral imaging. Dr. Zhang completed her training and left the project in August 2023. Two master students in Li Lab, Shadab Haque and Adwait Kaundanya, were trained in developing phenotyping tools. Shadab focused on analyzing 3D LiDAR models and 2D images, while Adwait focused on building a lower-cost gantry system for automatic data capture. One master student, Chhayank Srivastava, was trained to develop a robotic arm with LiDAR for pruning stressed leaves from a model plant. Two undergraduate students, Sam Gratta and Samuel Wang, were trained in collecting 2D and 3D images of tomato seedlings under stress and measuring plant height to determine plant wilting using computer vision and manual measurements. They also developed Python scripts for automatic phenotype extraction. In summary, the project has provided diverse training opportunities for undergraduate and graduate students, as well as postdoctoral associates, in areas such as robotics, AI, computer vision, plant pathology, hyperspectral imaging, DNA sequencing, microfluidic device design and fabrication, and data analysis.? How have the results been disseminated to communities of interest? The results of the project have been disseminated to communities of interest through various channels, including seminars, class lectures, demonstrations, publications (journal articles and conference papers), and outreach events. Here's a breakdown of the dissemination activities based on the project aims and reporting periods: From Aim 1 at CMU (Robotics and AI for Agriculture): Teaching: The fundamentals of the robot system design were included as an example in the course 16765 Robotics and AI for Agriculture at CMU during the 2023 spring semester. Co-PI Kantor and collaboration Abhisesh Silwal also included the robot design and AI concepts as a case study in this course from 2021 through 2024. The collected dataset was made available to students for class projects. Demonstrations: The system design and automated data collection pipeline were demonstrated to several aspiring high school students and their mentors exploring robotics projects for STEM initiatives. The system was also demonstrated to several participants from Kubota Inc. and Bloomfield Robotics Inc., who are exploring robotic applications in Closed Environment Agriculture (CEA). Additionally, demonstrations were given to undergraduate freshmen, faculty, researchers from various academic institutions, government officials from the USDA, senators, and officials from the mayor's office at Pittsburgh. Publications: The research methodologies and findings are currently being written for publication in a relevant scientific journal. From Aim 2 and 3 at Virginia Tech (Pathogen Detection and Imaging): Meetings and Conferences: The project team attended the APS (American Phytopathological Society) meeting and the CPS (Cyber-Physical Systems) PI meeting in 2022, 2023, and 2025. Poster presentations were used to demonstrate research progress and outcomes at both meetings. The PI delivered a plenary presentation at the American Society of Plant Biologists (ASPB) annual conference, addressing an audience of nearly 3,000 attendees. ASPB is the largest plant biology society in the United States. The PI also gave a presentation at the AAAI fall conference in the session of Using AI to Build Secure and Resilient Agricultural Systems. Outreach Events: Two outreach events were hosted: the Virginia Governor's School of Agriculture and the Virginia 4-H Summer Camp. These events involved more than 50 (in the first instance) and 30 (in the second instance) high school students combined. Demonstrations at these events included microscopic observations of infected plant samples, microbiome sequencing of infected plants, and hyperspectral imaging of infected plants. A new method was developed to teach students about hyperspectral imaging using colored crayon drawings of leaves, highlighting the quantitative measurements of colors beyond human vision. Summer Research Experiences for Undergraduates (REU): Two summer REU students were hosted and conducted research related to the project goals. They were trained in plant care, sensor data collection, automation, and AI algorithms, focusing on plant height measurement and wilting phenotype determination. They presented their results as a poster. Key Publications: A journal article, "Hyperspectral Imaging Analysis for the Early Detection of Tomato Bacterial Leaf Spot Disease," by X. Zhang, BA. Vinatzer, and S. Li, is published with Scientific Reports (2024).? A conference paper, "Towards an AI-Driven Cyber-Physical System for Closed-Loop Control of Plant Diseases" was published in the proceedings of 2024 AAAI Fall Symposia What do you plan to do during the next reporting period to accomplish the goals?
Nothing Reported
Impacts
What was accomplished under these goals?
Year 1 Accomplishments (Reporting period ending 01/14/2022) The major goals for the first year focused on the initial design and establishment of the project's core components. Objective 1 (Autonomous System for Plant Sample and Image Collection): The project successfully designed and assembled a tomato gantry and collected temporal RGB color images of tomato plants from germination to late vegetative growth. This dataset included top-view stereo pair images of two batches of tomato plants taken at different heights autonomously by the robot at precise locations. Objective 2 (Microfluidic Device for Sequencing-Based Pathogen Detection): The project successfully designed and fabricated a microfluidic device for LAMP PCR and nanopore library preparation. They also developed a two-module semi-automated device capable of performing both LAMP amplification and nanopore library preparation. Furthermore, they designed a single-lane, single-module device for the same purpose and developed primer sets targeting multiple gene regions for pathogen identification and variant detection. Objective 3 (Imaging-Based Disease Detection in a Greenhouse Setting): The initial goal was to establish visual signatures of infected plants using spectral imaging. The project established several experimental protocols for plant inoculation, quantification of pathogen load, and hyperspectral imaging. They identified a host plant variety (Early Girls) and selected highly virulent, antibiotic-resistant pathogens. Protocols for plant inoculation, disease progression monitoring, pathogen quantification, and hyperspectral scanning were developed. They also collected hyperspectral data to investigate the influence of leaf size, pathogen infection, and abiotic stress on tomato leaf reflectance, and tested PCR protocols for pathogen identification. Additionally, CRISPR-Cas12a was successfully developed as an additional method for target pathogen detection. Year 2 Accomplishments (Reporting period ending 01/14/2023) The second year saw advancements in the robotic system, microfluidic device, and image-based disease detection. Objective 1 (Autonomous System for Plant Sample Extraction): The project achieved a preliminary demonstration of the entire robotic operation, including imaging, 3D reconstruction, computing grasping points, executing motion planning to grasp, and dropping the sample to a dedicated location. They also designed and fabricated a novel end-effector with 3D printed grippers including micro needle loading, uploading, and clamping mechanisms for secure grasping and DNA extraction. A custom and novel leaf grasping algorithm was formulated to optimally select secure grasping points using multiple layers of information like graspable area, point cloud normal, and signed distance function. Objective 2 (Microfluidic Device for Sequencing-Based Pathogen Detection): The project successfully designed and fabricated a multi-channel microfluidic device for LAMP PCR and nanopore library preparation. They designed an assortment of multiplexed LAMP primers for rapid diagnostics and identification of genomic variations at the single nucleotide level. A microneedle-based extraction method was developed for integration into a robotic arm for sample extraction. Additionally, a two-module semi-automated device for LAMP and nanopore library preparation was developed, as well as a multi-lane, single-module device capable of processing four separate samples simultaneously. Software for automated sample handling within the microfluidic device was also adapted. Objective 3 (Imaging-Based Disease Detection in a Greenhouse Setting): The project collected hyperspectral imaging data for infected and healthy plant leaves. They compared multiple machine learning models in their accuracy in classifying infected and non-infected leaves and tested methods using vegetation indexes (VI), finding useful VIs and machine learning models that could classify infected samples as early as 2 hours post-inoculation. They found that plant height and distance to the hyperspectral camera had a strong impact on the spectral data. They also started collecting 3D point cloud data with the goal of using a 3D model to correct for changes in hyperspectral scans. A manuscript titled "Early Detection of Bacterial Spot Disease of Tomato with Hyperspectral Imaging" was under preparation. Year 3 Accomplishments (Reporting period ending 01/14/2024) The focus in the third year included further development and testing of the robotic system, microfluidic device optimization, and advanced imaging techniques for stress detection. Objective 1 (Autonomous Robot System for Scouting and Sampling): Significant progress was made in developing both hardware and software aspects of the robotic system. This included completing the development of a custom gantry-like robot system (T-REX) for automated image acquisition. They finalized the design and development of a novel end-effector for physical leaf sample extraction, incorporating features to minimize plant damage. They also developed an AI-based vision algorithm to understand plant semantics and guide the robot for optimal leaf sample extraction. The robot platform (T-REX) was equipped with a custom-built end-effector and an active light camera, utilizing digital servo motors and running on ROS. The novel leaf grasping algorithm was further developed, using 2D semantic segmentation of leaves and 3D information from merged point cloud data to propose grasping locations. A method for selecting the "easiest" leaf to grasp using a Pareto front approach was also implemented. Objective 2 (Microfluidic Device for Sequencing-Based Pathogen Detection): The project successfully designed and optimized a microfluidic device capable of producing multi-region LAMP amplified libraries from low sample inputs generated by microneedle extraction. Significant protocol optimization was performed to adapt this device to low-input field-extracted samples. They also demonstrated that the device could produce amplified multi-region (avrRpt2 and rpoD) libraries from microneedle extracts on inoculated tomato plants. Objective 3 (Imaging-Based Disease Detection in a Greenhouse Setting): Two methods for plant segmentation and stress detection within a crowded canopy were tested and developed. The first method used top-down LiDAR and side-view RGB images to determine the wilting phenotype (change in plant height due to stress). The second method used LiDAR and a robotic arm to isolate the foreground and optimized the robot arm's movement to locate pruning locations on a stressed leaf model. Protocols for whole-plant scans using LiDAR were developed. They explored the analysis of plant 3D models using top-down LiDAR and side-view RGB cameras, using both manual measurements and 2D/3D segmentation (including foundation AI models like Segment Anything Model and Grounding DINO) for data analysis. Due to challenges with maintaining disease-conducive environments and the departure of a postdoc, the focus shifted to using salt/water-stressed plants as a model for developing computer vision tools to segment stressed plants in a crowded canopy. A manuscript on hyperspectral imaging analysis for early detection of tomato bacterial leaf spot disease was under review. Year 4 Accomplishments (Reporting period ending 01/14/2025) Objective 1 and 3: two papers were published in peer-reviewed journals. A master's student was trained to build a mini replica of the T-REX robot using off-the-shelf components. The student completed a master's thesis, and a paper is currently in preparation for ASABE, with the abstract already accepted for presentation at ASABE 2025.
Publications
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2024
Citation:
Moving towards closed-loop control systems for plant production with AI and robotics. Song Li, ASPB annual symposium, Plenary Talk, 2024
- Type:
Peer Reviewed Journal Articles
Status:
Published
Year Published:
2024
Citation:
Zhang, X., Vinatzer, B.A. & Li, S. Hyperspectral imaging analysis for early detection of tomato bacterial leaf spot disease. Sci Rep 14, 27666 (2024). https://doi.org/10.1038/s41598-024-78650-6
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2024
Citation:
Silwal, A., Zhang, X. M., Hadlock, T., Neice, J., Haque, S., Kaundanya, A., Lu, C., Vinatzer, B. A., Kantor, G., & Li, S. (2024). Towards an AI-Driven Cyber-Physical System for Closed-Loop Control of Plant Diseases. Proceedings of the AAAI Symposium Series, 4(1), 432-435. https://doi.org/10.1609/aaaiss.v4i1.31828
- Type:
Conference Papers and Presentations
Status:
Submitted
Year Published:
2025
Citation:
Development of Open-Source Gantry-Plus Robot Systems for Plant Science Research. Adwait Kaundanya, Sanchari Kundu, Abhisesh Silwal, George Kantor, and Song Li
- Type:
Theses/Dissertations
Status:
Published
Year Published:
2024
Citation:
Development of Open-Source Gantry-Plus Robot Systems for Plant Science research. Kaundanya, Adwait Anand; Electrical Engineering; Ha, Dong S.; Li, Song; South, Kaylee Anne; Zeng, Haibo. 2024-12-19
|
Progress 01/15/23 to 01/14/24
Outputs
Target Audience:1. Faculty and scientists working in plant disease management metagenomics, plant pathology, and agriculture automation and robotics. 2. Graduate and undergraduate students working in agriculture-related disciplines with a focus on plant disease management and automation. 3. Agricultural producers, farmers, and crop advisors with a focus on plant production in greenhouses and controlled environments. 4. State commodity board and CALS advisory board members including high-level executives, business owners, and state policymakers. Changes/Problems:Research objective three: The post-doctoral assistant funded on this project has found a job as a data scientist in a research institute and left the project in 2023. The challenge is that we don't have enough time to train another person who can perform plant inoculation with a plant pathogen. To solve this problem, we shifted our focus to use salt/water stressed plants as our model instead of diseased plant. The wilting phenotype is one phenotype that consistently observed in many cases of infected plants and is easy to induce by withholding water or add salt to the water. With this change, we can focus on development computer vision tools that allows us to segment stressed plants from a crowded canopy, which is a crucial step towards control of plant disease via the removal of infected plants from a population automatically. What opportunities for training and professional development has the project provided?Research objective one: Through various objectives in this project, research exposure, opportunities for training and professional development were provided at multiple levels. At an undergraduate level, Sohan Kulkarni, a freshman in Mechanical engineering, designed the early version of the end-effector as a part of his research credit. Another undergraduate, Srecharan Selvam, Mechanical engineering, and a senior highschool student Ben Tarr together designed the microneedle dispensing system and its software. At a graduate level, Simi Asher, a robotics student implemented various deep learning-based approaches to reconstruct 3D plant models using multiple stereo image pairs and comparing them to their traditional counterparts. Similarly, Dominic Guri, a Ph.D student, worked as a coordinator and used this robotic system to gain hands-on robotics experience as well as worked with other grad and undergraduate students. His training mostly involves motion planning and scheduling of robotic tasks such as the automated image data collection pipeline. Research objective two: Two graduate students, Thomas Hadlock and Jacob Neice, have been trained in DNA sequencing pipeline development and microfluidic device design and fabrication. They have also been trained in nanopore sequencing and data analysis protocols. Research objective three: A postdoc associate, Missi Zhang has been trained in plant pathology and hyperspectral imaging. Dr. Zhang finished her training and left the project in August 2023. Two master students were trained in Li Lab, Shadab Haque and Adwait Kaundanya have been trained in developing phenotyping tools. Shadab focused on analyzing 3D LiDAR model and 2D images of tomato seedlings. Adwait focused on building a lower cost gantry system by learning from the CMU group (aim1) to capture plant data automatically. One master student, Chhayank Srivastava, was trained to develop a robotic arm with LiDAR to prune stressed leaves from a model plant. Two undergraduate students, Sam Gratta and Samuel Wang were trained in collecting 2D and 3D images of tomato seedlings under stress and measure plant height to determine plant wilting phenotype using computer vision tools as well as manual measurments. How have the results been disseminated to communities of interest?From Aim 1 at CMU. The results have been disseminated to communities of interest mostly through: Seminars: Co-PI Kantor shared research methodologies and findings through seminars at multiple institutions including Oregon State University, Iowa State University, Virginia Tech etc. Class lectures: Co-PI Kantor along with collaboration Abhisesh Silwal included the robot design and AI concepts as a case study in one of the lectures on the course 16765 Robotics and AI in Agriculture from 2021 through 2024. Demonstrations: We have demonstrated the system design and automated data collection pipeline to several aspiring high school students, undergraduate freshmen seeking degrees in robotics, faculties and researchers from various academic institutions across the country, government officials from the USDA, senators, and officials from the mayor's office at Pittsburgh. Publications: The research methodologies and findings are currently being written for publication in a relevant scientific journal. From Aim 2 and 3 at Virginia Tech. We have hosted two outreach events: Virginia governor school of agriculture and Virginia 4H summer camp. In both events, we demonstrated microbiome sequencing of infected plants, and hyperspectral imaging of infected plants. These two outreach activities involved more than 30 high school students combined. During these events, we developed a new method to teach students the power of hyperspectral imaging. Instead of using real plants, we asked the students to draw plant leaves using color crayons. More specifically, we asked the students to use (1) different shade of green, and (2) purple and violet crayons. These fake "leaves" were then scanned by hyperspectral imaging platforms. The key take home message is that hyperspectral scans can provide quantitative measurements of colors that are difficult to quantify by human vision. For example, although some color appears "purple" to human eyes, some purple color included violet/blue light (430nm) but other purple color included both blue and red light (450nm + 700nm). This activity helped the participants understand the importance of quantitative assessment of leaf colors and how such an approach can be used to determine plant health status.? We also hosted two summer REU students and conducted summer research related to the project goals. These students were trained in plant care, sensor data collection, automation and AI algorithms. Both students were involved in measurement plant height using multiple sensors and to determine the plant wilting phenotype. They also developed python scripts to extract plant phenotypes automatically and used the manually measured phenotypes to correlate with automatically measured plant heights. They presented their results as a poster presentation at the end of the summer session. What do you plan to do during the next reporting period to accomplish the goals?Our plan for next reporting period includes the following activities: Research objective one: No more activities. Research objective two: No more activities. Research objective three: Collect canopy level 3D LiDAR, hyperspectral and RGB images stressed plants and develop computer vision methods to segment stressed plants from non-stressed plants in a crowded canopy. Write an overview manuscript in the format of a perspective paper to summarize the discoveries and challenges for building an integrated system including the technologies developed in all three research objectives and submit it before the end of this project. Write a grant to continue the development of the robotic platform for automatic plant tissue collection and DNA sequencing. Bayer has released a research project call for automatic plant tissue collection in the greenhouse (due on April 30th 2024). The goal of this Bayer call is closely related to the research objectives in this CPS project, indicating industrial interest in our research direction.
Impacts
What was accomplished under these goals?
The goal under objective one is to develop an autonomous robot system for scouting plants in a greenhouse environment and use imaging-based techniques to sense and extract leaf samples to study the presences and spread of diseases. To achieve this goal, we have made significant progress in the development of both hardware and software aspects of the robotic system. We have completed the development of a custom gantry like robot system to fully automate the laborious task of acquiring temporal images of plants, completed the design and development of a novel end-effector to physically extract leaf samples, and the AI-based vision algorithm to understand plant semantics to guide the robot to extract leaf samples at the most optimal locations. The following paragraph further details our methods and accomplishments. A. Hardware Development: Robot platform: Plants grown in a greenhouse environment with modern techniques such as hydroponic systems usually have flat beds as a base for plants to grow and growth lights are usually hung from the above. The design of the robot system also emulates a similar setting with a workspace of 3 meter by 1.5 meter flat test bed with timed growth lights. This gantry-like robot actually is a large robot arm in disguise with three prismatic and three revolute joints to achieve six degrees of freedom pose. This system named T-REX (Robot of EXtracting Tomato leaf samples) is equipped with a custom built end-effector (described below) and also a custom active light camera to image the plants. The robot uses digital servo motors as actuators and runs on ROS (robot Operating Software). Currently, the robot traverses to each known location of the plants and acquires images of plants using six pre-programmed poses that later is used for 3D modeling and image-based AI analytics. Novel end-effector: While the T-REX platform provides actuation to reach the location for sampling and image-based data collection, it is the end-effector that physically interacts with the plants. The design of the end-effector takes into account that plants/ leaves are deformable, and highly prone to damage. To minimize damage to surrounding leaves as well as to unintentionally push the plants away, the motion planner approaches the plant from the top and tries to grasp the leaf by clamping the leaf from its top and bottom surfaces. Unlike conventional robot grippers like the parallel-jaw actuator, this design requires less robot joint actuation for grasping that could potentially avoid singularities or unnatural looking grasping poses. The tip of the end-effector houses a microneedle array that is pushed against the leaf surface to destructively extract DNA samples. Micro needle dispensing mechanism: The section of the end-effector that houses the microneedle array is a thin rectangular plastic where microneedle arrays are glued to. The section where the microneedle array sticks has laser etched tabs that break away when an external force is applied. The dispensing system consists of several of the rectangular microneedle housing and automatically reloads the tip of the end-effector for end-to-end sampling without human intervention. B. Software: Novel leaf grasping algorithm: To guide the robot to physically grasp a leaf requires the semantic understanding of the location of the plant and its parts such as leaves with respect to adjacent plants in the robot's workspace. To enable this capability, we leverage both 2D spatial and 3D information. The 2D information contains semantic segmentation of instances of leaves and using deep learning algorithms and the 3D information is extracted from the merged point cloud data using multi-view stereo. The pipeline first uses instance segmentation and 3d information to generate proposals of grasping locations for each individual leaves. This part of the pipeline automatically generates large training samples for heatmap based regression model to identify optimal grasping locations. However, as plants have indeterminate growth, some of the leaves are very hard to reach. The next step includes selecting the most "easy" leaf to grasp to increase the chances of successful sampling. This capability uses pareto front to select leaves that are maximally away from the surrounding leaves and minimally away from free unoccluded regions of the workspace. The goal of objective two is to develop microfluidic device for sequencing-based pathogen detection. Our achievement in this year is the successful design and optimization of a microfluidic device that can produce multi-region LAMP amplified libraries from low sample inputs generated by microneedle extraction. Significant protocol optimization was performed this year to adapt this device to low-input field-extracted samples. The goal of objective three is to develop imaging-based disease detection in a greenhouse setting. The ultimate goal is to develop a model of disease spreading in a greenhouse. In the third year of this project, we tested and developed two methods for plant segmentation and stress detection within a crowded canopy.First method is to use top-down LiDAR and side-view RGB images to determine the wilting phenotype, which is the change of plant height due to stress. The second method is to use LiDAR and a robotic arm to isolate the foreground (leaf and stems) from the background, and to optimize the movement of the robotic arm to locate a pruning location for the stressed leaf (as indicated by a painted spot on a fake leaf).
Publications
- Type:
Journal Articles
Status:
Under Review
Year Published:
2024
Citation:
Hyperspectral Imaging Analysis for the Early Detection of Tomato Bacterial Leaf Spot Disease, X Zhang, BA Vinatzer, S Li, Scientific Report, Under review, 2024
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2023
Citation:
Early Stress Detection in Tomato Plants using Computer Vision and 2D/3D Imaging. Samantha Gratta, Samuel Xiang, Shadab Haque, and Song Li. Poster presentation for 2023 Summer REU Solving Problems with Data Science at Virginia Tech
|
Progress 01/15/21 to 01/14/22
Outputs
Target Audience:Our audience include the following categories: 1. Faculty and scientists working in plant disease management metagenomics, plant pathology, and agriculture automation and robotics. 2. Graduate and undergraduate students working in agriculture-related disciplines with a focus on plant disease management and automation. 3. Agricultural producers, farmers, crop advisors with a focus on plant production in greenhouses and controlled environments. 4. State commodity board and CALS advisory board members including high level executives, business owners and state policy makers. Changes/Problems:A major problem we are facing is that the changing greenhouse conditions (ambient temperature and daytime length vary among seasons) have a great impact on disease spreading in our testing greenhouse. It is challenging to maintain a disease-conducive environment to create consistent disease epidemics in our testing greenhouse. To solve this problem, we will adjust our inoculum concentration and irrigation programs for different seasons. What opportunities for training and professional development has the project provided?Research objective one: An undergraduate intern, Sohan Kulkarni, has been training in the design of the end-effector to selectively sample leaves from the tomato plants. He is exploring conventional rigid mechanisms as well as the current state-of-the-art compliance-based mechanisms in the design of novel end-effectors that interact with soft and flexible leaf samples. Similarly, Dominic Guri, a Ph.D student, is training on this robotic system to gain hands-on robotics experience. His training mostly involves motion planning and scheduling of robotic tasks such as the automated image data collection pipeline. Research objective two: A graduate student, Thomas Hadlock, has been trained in DNA sequencing pipeline development and microfluidic device design and fabrication. He has also been trained in nanopore sequencing and data analysis protocols. An undergraduate student, Katie Orr, has been trained in microfluidic device fabrication and sample processing procedures in order to operate the semi-automated process. Research objective three: A postdoc associate, Missi Zhang has been trained in plant pathology and hyperspectral imaging. A postdoc associate, Wei Xing, has been trained in using machine learning to analyze hyperspectral data. Two undergraduate interns (in Li Lab) have been trained in developing phenotyping robots for greenhouse producers. How have the results been disseminated to communities of interest?We have published two papers and one conference proceedings related to our work on using imaging for disease detection. What do you plan to do during the next reporting period to accomplish the goals?Our plan for next reporting period includes the following activities: Research objective one: Complete systems integration with 6 DoF robot arm with novel end-effector Perception system to identify sampling locations Complete systems demonstrating sample retrieval and delivery Research objective two: Expand and optimize the primer set to maximize the number of genes regions of interest that can be targeted and sequenced for each library Expand the device from single to multi-lane to prepare libraries composed of numerous samples in each run. Advance integration with sample extraction platform Optimize device platform for increased automation, specifically targeted at limiting human involvement with syringe pumps and magnetic handling steps Research objective three: Finish writing the manuscript related to the hyperspectral imaging signature for infected plants Collect canopy level hyperspectral and RGB images for infected plants. Develop methods to distinguish infected plants from non-infected plant using images. Monitoring disease spread using time lapse photography. Develop CRISPR based methods for pathogen detection.
Impacts
What was accomplished under these goals?
The goal of objective one is to develop autonomous system for plant sample collection and image collection. Our achievement in the first year is the successful design and assembly of a tomato gantry and collection oftemporal RGB color images of tomato plants from germination to late vegetative growth. The goal of objective two is to develop microfluidic device for sequencing-based pathogen detection. Our achievement in the first year is the successful design and fabrication of microfluidic device for LAMP PCR and nanopore library preparation. The goal of objective three is to develop imaging-based disease detection in a greenhouse setting. The ultimate goal is to develop a model of disease spreading in a greenhouse. In the first year of this project, our initial goal is to establish visual signatures of infected plants using spectral imaging. Towards this goal, we have established several experimental protocols (see product section) on inoculation of plants, quantification of pathogen load, and hyperspectral imaging. These results will allow us to correlate disease progress stages/pathogen loads with visual signatures. We also expect to develop spectral imaging-based diagnosis methods for asymptomatic plants.?
Publications
- Type:
Journal Articles
Status:
Published
Year Published:
2021
Citation:
Identifying Optimal Wavelengths as Disease Signatures Using Hyperspectral Sensor and Machine Learning. Xing Wei, Marcela Johnson, David Langston, Hillary Mehl and Song Li. Remote Sensing. 2021, 13(14), 2833; https://doi.org/10.3390/rs13142833
- Type:
Journal Articles
Status:
Published
Year Published:
2021
Citation:
Detection of Soilborne Disease Utilizing Sensor Technologies: Lessons Learned from Studies on Stem Rot of Peanut. Xing Wei, Marcela Aguilera, Rachael Walcheck, Dorothea Tholl, Song Li, David B. Langston, Jr., and Hillary L. Mehl. Plant Health Progress. 2021. 22(4): 436-444. https://doi.org/10.1094/PHP-03-21-0055-SYN
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2022
Citation:
Interactive Deep Learning for Exploratory Sorting of Plant Images by Visual Phenotypes. Huimin Han, Ritvik Prabhu, Timothy Smith, Kshitiz Dhakal, Xing Wei, Song Li, and Chris North. Proceedings of NAPPN annual conference. Jan-30-2022. https://doi.org/10.1002/essoar.10508768.2
|