Source: VIRGINIA POLYTECHNIC INSTITUTE submitted to
COLLABORATIVE RESEARCH: CPS: MEDIUM: EARLY STAGE PLANT DISEASE DETECTION VIA ROBOTIC SAMPLING AND ON SITE METAGENOMIC SEQUENCING
Sponsoring Institution
National Institute of Food and Agriculture
Project Status
EXTENDED
Funding Source
AFRI COMPETITIVE GRANT
Reporting Frequency
Annual
Accession No.
1025458
Grant No.
2021-67021-34037
Project No.
VA-Li
Proposal No.
2020-11360
Multistate No.
(N/A)
Program Code
A7302
Project Start Date
Jan 15, 2021
Project End Date
Jan 14, 2025
Grant Year
2021
Project Director
Li, S.
Recipient Organization
VIRGINIA POLYTECHNIC INSTITUTE
(N/A)
BLACKSBURG,VA 24061
Performing Department
School of Plant Sciences
Non Technical Summary
Plant diseases pose an increasing threat to the nation's food supply and biosecurity in a changing environment. There is a growing risk of plant pathogens spreading through shipments of stock plants between production facilities, and from production facilities to growers or to retailers and consumers. Recent failures to prevent plant disease emergence and spread in the US has resulted in major economic losses (20-40% of total yield) for growers. One central challenge to preventing accidental pathogen dissemination and disease outbreaks is that many plant diseases are difficult to detect at an early stage and infected (possibly asymptomatic) plants can spread pathogens undetected when being shipped from one location to another. Once an emerging disease takes foothold in a new environment, eradicating such disease is extremely challenging.As our model system, we focus on disease detection and control in transplant facilities, which produce seedlings that will later be planted in production fields. These facilities are an amplifier of diseases in the agriculture production chain because of the use of greenhouses with limited environmental control and high plant density (average 800 plants per square meter, growing on plastic trays), that facilitate disease spread. A typical transplant facility grows over 500,000 plants at a time; therefore, manual scouting of the facility is time consuming and ineffective. Poor training in plant pathology and tight schedules of employee limit sensitivity of disease detection and spatial-temporal resolution. In summary, the transplant industry is a major source of disease outbreaks and can thus be a key point for disease control in the agriculture supply chain. To reduce the contribution of transplant facilities to disease outbreaks, we plan to develop a novel disease detection and control CPS to provide closed-loop disease control at a very early stage.This project includes three main objectives. First, an open-source robotic gantry system will be developed to automate the process of greenhouse scouting, plant sampling and plant removal to prevent disease spread. Second, a microfluidic device will be implemented to automate the process of library preparation from bacterial DNA extraction, amplification to metagenomic sequencing. Third, controlled experiments will be performed to determine the best methods for disease detection via nanopore, meta-genomic sequencing, machine learning, and computer vision. Different control strategies will be tested to determine the best approach that integrates data from metagenomic sequencing and the analysis of images of infected plants.The immediate goal of our project is to reduce the spread of plant diseases in the agriculture production chain by focusing on reducing disease instances in the transplant facilities and greenhouses. Our approach is to reduce the loss of plant products by optimization of disease detection and control strategies through robotics and automated genetic sequencing. In the long run, our project will contribute to the improvement of automation and reduction of the labor cost in agriculture. Our system, if successful, will reduce the loss of plants due to diseases for greenhouse operations and in plant production systems that utilizes transplant facilities.
Animal Health Component
0%
Research Effort Categories
Basic
50%
Applied
50%
Developmental
(N/A)
Classification

Knowledge Area (KA)Subject of Investigation (SOI)Field of Science (FOS)Percent
2121460110030%
2161460110030%
4044010104040%
Knowledge Area
212 - Pathogens and Nematodes Affecting Plants; 216 - Integrated Pest Management Systems; 404 - Instrumentation and Control Systems;

Subject Of Investigation
1460 - Tomato; 4010 - Bacteria;

Field Of Science
1040 - Molecular biology; 1100 - Bacteriology;
Goals / Objectives
The transplant industry is a major source of disease outbreaks and can be a key point for disease control in the agriculture supply chain. To reduce the contribution of transplant facilities to disease outbreaks, the major goal of this project is to develop a novel disease detection and control CPS (Cyber-Physical System) to provide closed-loop disease control at a very early stage. We envision a novel disease detection and pathogen identification system that combines computer vision and real-time gene sequencing to provide closed-loop disease detection, pathogen identification, and control to reduce pathogen spread. In objective 1, we will develop a robotic system which can constantly scout large production greenhouses to perform real-time disease detection by imaging followed by sampling of plants that may already be infected. In objective 2, we will develop a pathogen identification method using a high-throughput, mobile device whereby sample preparation, sequencing, and data analytics will all be automated. In objective 3, we will integrate the results from image-based disease detection and sequencing-based pathogen identification with temperature and humidity data from environmental sensors and develop a model to guide the implementation of disease control strategies.
Project Methods
EffortsFor objective 1, we have four sub-objectives including in field plant perception, manipulator planning and control, manipulator design and system integration. We will develop 3D models of plant architecture using a stereo vision camera and perform sematic labeling of plant parts using neural networks. For manipulator planning and control, we will focus on using existing approaches and improve reliability of the methods in difficult field conditions. For manipulator design, we will develop tools to rapidly design simple robotic manipulators from modular components on a gantry system for greenhouse applications. We will use good engineering practices to integrate the gantry and end-effector hardware and control systems. We will use the robot operating system (ROS) for software integration. The robotic scanning and sampling system will be integrated with sample preparation and sequencing. For objective 2, we will apply a simplified version of our published device to conduct DNA release and purification from bacterial cells. A suspension of bacteria collected by washing the leaves will be processed in our device. We will apply short electric pulses to electrically lyse bacterial cells to release genomic DNA within minutes with high efficiency. We will take advantage of several modules developed in our prior publications and apply them to nanopore library preparation and automation of the process using a portable liquid handling system. The microfluidic system will be fully automated by programming the operation of solenoid valves, a heating/cooling system, and a programmable infusion/withdrawal syringe pump. On board PCR will be performed and the prepared library can be removed from the microfluidic system and loaded onto the sequencer for sequencing by the liquid handling robot.For objective 3, we will use inoculated tomato samples to collect metagenomic data and use convolutional neural networks to classify four categories of plants: S (susceptible and healthy), E (exposed to pathogen but healthy), I (infected without detectable symptom), and D (infected with detectable symptom). The experiment will be setup in a growth chamber with control of light, humidity and temperature to simulate different environmental conditions. Plants will grow in pots and putting in trays similar to real greenhouse production. Time course observation of plants will be taken by regular RGB cameras every hour. This experiment will provide key parameters for our disease model including pathogen growth curve and infection rate. Disease model parameters will be estimated using classical MCMC approaches. Imaging will have two different modes: a surveillance mode before the first "D" plant emerges and a model-guided mode afterwards. Plants will be chosen for sequencing based on imaging and model predictions. We will use inoculation experiment to test several control strategies and use simulation to explore the parameter space with a manageable number of parameter combinations. Results will be evaluated in both growth chamber experiments and greenhouse experiments. Evaluation Whole project evaluation. The final goal is to reduce the number of infected plants with a fully integrated CPS. To evaluate this, we will compare the trays managed by the CPS and by a human. The total fraction of plants that are infected and the propagation of disease overtime will be used to quantify the effectiveness of the CPS. Evaluation for aim 1. The overall objective of Aim 1 is to produce a functional gantry robot with a manipulator capable of taking leaf samples from a matrix of tomato seedlings planted in a growth tray. Specific metrics for success in this aim are: (1) Use an open-source gantry system with sufficient localization accuracy to address individual plants in a tray, and sufficient ease of use that it can be replicated at both CMU and VT, and can be teleoperated remotely. (2) A perception and plant modeling system of sufficient accuracy to resolve individual plants, identify their leaves, and identify suitable locations for collecting physical samples. (3) Gantry end effector capable of reaching plant leaves and collecting physical samples. (4) End-to-end operation that provide integrated plant tissue sampling at a rate of one plant per minute. Evaluation for aim 2. (1) Conduct DNA library preparation on a microfluidic platform using 10 ng or less DNA; (2) Collect and release bacterial DNA with an efficiency of >90%; (3) Integrate various components for microfluidic control into a portable platform. Evaluation for aim 3. The major goal is to collect parameters related to disease propagation, and to use the parameters to determine the control strategy. The first year will focus on inoculation experiments to determine the optimal imaging and sequencing protocols such that we can accurately identify plants that are in the "Infected(I)" and "Exposed(E)" categories. Because we are using inoculated plants, we can use typical machine learning evaluation metrics such as accuracy and F1-score to evaluate the experimental outcomes. The second year will focus on modeling where we will compare the control strategies using simulations and comparing the simulated results with experimental outcomes. The evaluation metrics will be the total fraction of plants that are infected at the end of each experiment. The evaluation of the third year will be similar to the experiment in the second year but in a field setting.

Progress 01/15/23 to 01/14/24

Outputs
Target Audience:1. Faculty and scientists working in plant disease management metagenomics, plant pathology, and agriculture automation and robotics. 2. Graduate and undergraduate students working in agriculture-related disciplines with a focus on plant disease management and automation. 3. Agricultural producers, farmers, and crop advisors with a focus on plant production in greenhouses and controlled environments. 4. State commodity board and CALS advisory board members including high-level executives, business owners, and state policymakers. Changes/Problems:Research objective three: The post-doctoral assistant funded on this project has found a job as a data scientist in a research institute and left the project in 2023. The challenge is that we don't have enough time to train another person who can perform plant inoculation with a plant pathogen. To solve this problem, we shifted our focus to use salt/water stressed plants as our model instead of diseased plant. The wilting phenotype is one phenotype that consistently observed in many cases of infected plants and is easy to induce by withholding water or add salt to the water. With this change, we can focus on development computer vision tools that allows us to segment stressed plants from a crowded canopy, which is a crucial step towards control of plant disease via the removal of infected plants from a population automatically. What opportunities for training and professional development has the project provided?Research objective one: Through various objectives in this project, research exposure, opportunities for training and professional development were provided at multiple levels. At an undergraduate level, Sohan Kulkarni, a freshman in Mechanical engineering, designed the early version of the end-effector as a part of his research credit. Another undergraduate, Srecharan Selvam, Mechanical engineering, and a senior highschool student Ben Tarr together designed the microneedle dispensing system and its software. At a graduate level, Simi Asher, a robotics student implemented various deep learning-based approaches to reconstruct 3D plant models using multiple stereo image pairs and comparing them to their traditional counterparts. Similarly, Dominic Guri, a Ph.D student, worked as a coordinator and used this robotic system to gain hands-on robotics experience as well as worked with other grad and undergraduate students. His training mostly involves motion planning and scheduling of robotic tasks such as the automated image data collection pipeline. Research objective two: Two graduate students, Thomas Hadlock and Jacob Neice, have been trained in DNA sequencing pipeline development and microfluidic device design and fabrication. They have also been trained in nanopore sequencing and data analysis protocols. Research objective three: A postdoc associate, Missi Zhang has been trained in plant pathology and hyperspectral imaging. Dr. Zhang finished her training and left the project in August 2023. Two master students were trained in Li Lab, Shadab Haque and Adwait Kaundanya have been trained in developing phenotyping tools. Shadab focused on analyzing 3D LiDAR model and 2D images of tomato seedlings. Adwait focused on building a lower cost gantry system by learning from the CMU group (aim1) to capture plant data automatically. One master student, Chhayank Srivastava, was trained to develop a robotic arm with LiDAR to prune stressed leaves from a model plant. Two undergraduate students, Sam Gratta and Samuel Wang were trained in collecting 2D and 3D images of tomato seedlings under stress and measure plant height to determine plant wilting phenotype using computer vision tools as well as manual measurments. How have the results been disseminated to communities of interest?From Aim 1 at CMU. The results have been disseminated to communities of interest mostly through: Seminars: Co-PI Kantor shared research methodologies and findings through seminars at multiple institutions including Oregon State University, Iowa State University, Virginia Tech etc. Class lectures: Co-PI Kantor along with collaboration Abhisesh Silwal included the robot design and AI concepts as a case study in one of the lectures on the course 16765 Robotics and AI in Agriculture from 2021 through 2024. Demonstrations: We have demonstrated the system design and automated data collection pipeline to several aspiring high school students, undergraduate freshmen seeking degrees in robotics, faculties and researchers from various academic institutions across the country, government officials from the USDA, senators, and officials from the mayor's office at Pittsburgh. Publications: The research methodologies and findings are currently being written for publication in a relevant scientific journal. From Aim 2 and 3 at Virginia Tech. We have hosted two outreach events: Virginia governor school of agriculture and Virginia 4H summer camp. In both events, we demonstrated microbiome sequencing of infected plants, and hyperspectral imaging of infected plants. These two outreach activities involved more than 30 high school students combined. During these events, we developed a new method to teach students the power of hyperspectral imaging. Instead of using real plants, we asked the students to draw plant leaves using color crayons. More specifically, we asked the students to use (1) different shade of green, and (2) purple and violet crayons. These fake "leaves" were then scanned by hyperspectral imaging platforms. The key take home message is that hyperspectral scans can provide quantitative measurements of colors that are difficult to quantify by human vision. For example, although some color appears "purple" to human eyes, some purple color included violet/blue light (430nm) but other purple color included both blue and red light (450nm + 700nm). This activity helped the participants understand the importance of quantitative assessment of leaf colors and how such an approach can be used to determine plant health status.? We also hosted two summer REU students and conducted summer research related to the project goals. These students were trained in plant care, sensor data collection, automation and AI algorithms. Both students were involved in measurement plant height using multiple sensors and to determine the plant wilting phenotype. They also developed python scripts to extract plant phenotypes automatically and used the manually measured phenotypes to correlate with automatically measured plant heights. They presented their results as a poster presentation at the end of the summer session. What do you plan to do during the next reporting period to accomplish the goals?Our plan for next reporting period includes the following activities: Research objective one: No more activities. Research objective two: No more activities. Research objective three: Collect canopy level 3D LiDAR, hyperspectral and RGB images stressed plants and develop computer vision methods to segment stressed plants from non-stressed plants in a crowded canopy. Write an overview manuscript in the format of a perspective paper to summarize the discoveries and challenges for building an integrated system including the technologies developed in all three research objectives and submit it before the end of this project. Write a grant to continue the development of the robotic platform for automatic plant tissue collection and DNA sequencing. Bayer has released a research project call for automatic plant tissue collection in the greenhouse (due on April 30th 2024). The goal of this Bayer call is closely related to the research objectives in this CPS project, indicating industrial interest in our research direction.

Impacts
What was accomplished under these goals? The goal under objective one is to develop an autonomous robot system for scouting plants in a greenhouse environment and use imaging-based techniques to sense and extract leaf samples to study the presences and spread of diseases. To achieve this goal, we have made significant progress in the development of both hardware and software aspects of the robotic system. We have completed the development of a custom gantry like robot system to fully automate the laborious task of acquiring temporal images of plants, completed the design and development of a novel end-effector to physically extract leaf samples, and the AI-based vision algorithm to understand plant semantics to guide the robot to extract leaf samples at the most optimal locations. The following paragraph further details our methods and accomplishments. A. Hardware Development: Robot platform: Plants grown in a greenhouse environment with modern techniques such as hydroponic systems usually have flat beds as a base for plants to grow and growth lights are usually hung from the above. The design of the robot system also emulates a similar setting with a workspace of 3 meter by 1.5 meter flat test bed with timed growth lights. This gantry-like robot actually is a large robot arm in disguise with three prismatic and three revolute joints to achieve six degrees of freedom pose. This system named T-REX (Robot of EXtracting Tomato leaf samples) is equipped with a custom built end-effector (described below) and also a custom active light camera to image the plants. The robot uses digital servo motors as actuators and runs on ROS (robot Operating Software). Currently, the robot traverses to each known location of the plants and acquires images of plants using six pre-programmed poses that later is used for 3D modeling and image-based AI analytics. Novel end-effector: While the T-REX platform provides actuation to reach the location for sampling and image-based data collection, it is the end-effector that physically interacts with the plants. The design of the end-effector takes into account that plants/ leaves are deformable, and highly prone to damage. To minimize damage to surrounding leaves as well as to unintentionally push the plants away, the motion planner approaches the plant from the top and tries to grasp the leaf by clamping the leaf from its top and bottom surfaces. Unlike conventional robot grippers like the parallel-jaw actuator, this design requires less robot joint actuation for grasping that could potentially avoid singularities or unnatural looking grasping poses. The tip of the end-effector houses a microneedle array that is pushed against the leaf surface to destructively extract DNA samples. Micro needle dispensing mechanism: The section of the end-effector that houses the microneedle array is a thin rectangular plastic where microneedle arrays are glued to. The section where the microneedle array sticks has laser etched tabs that break away when an external force is applied. The dispensing system consists of several of the rectangular microneedle housing and automatically reloads the tip of the end-effector for end-to-end sampling without human intervention. B. Software: Novel leaf grasping algorithm: To guide the robot to physically grasp a leaf requires the semantic understanding of the location of the plant and its parts such as leaves with respect to adjacent plants in the robot's workspace. To enable this capability, we leverage both 2D spatial and 3D information. The 2D information contains semantic segmentation of instances of leaves and using deep learning algorithms and the 3D information is extracted from the merged point cloud data using multi-view stereo. The pipeline first uses instance segmentation and 3d information to generate proposals of grasping locations for each individual leaves. This part of the pipeline automatically generates large training samples for heatmap based regression model to identify optimal grasping locations. However, as plants have indeterminate growth, some of the leaves are very hard to reach. The next step includes selecting the most "easy" leaf to grasp to increase the chances of successful sampling. This capability uses pareto front to select leaves that are maximally away from the surrounding leaves and minimally away from free unoccluded regions of the workspace. The goal of objective two is to develop microfluidic device for sequencing-based pathogen detection. Our achievement in this year is the successful design and optimization of a microfluidic device that can produce multi-region LAMP amplified libraries from low sample inputs generated by microneedle extraction. Significant protocol optimization was performed this year to adapt this device to low-input field-extracted samples. The goal of objective three is to develop imaging-based disease detection in a greenhouse setting. The ultimate goal is to develop a model of disease spreading in a greenhouse. In the third year of this project, we tested and developed two methods for plant segmentation and stress detection within a crowded canopy.First method is to use top-down LiDAR and side-view RGB images to determine the wilting phenotype, which is the change of plant height due to stress. The second method is to use LiDAR and a robotic arm to isolate the foreground (leaf and stems) from the background, and to optimize the movement of the robotic arm to locate a pruning location for the stressed leaf (as indicated by a painted spot on a fake leaf).

Publications

  • Type: Journal Articles Status: Under Review Year Published: 2024 Citation: Hyperspectral Imaging Analysis for the Early Detection of Tomato Bacterial Leaf Spot Disease, X Zhang, BA Vinatzer, S Li, Scientific Report, Under review, 2024
  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Early Stress Detection in Tomato Plants using Computer Vision and 2D/3D Imaging. Samantha Gratta, Samuel Xiang, Shadab Haque, and Song Li. Poster presentation for 2023 Summer REU Solving Problems with Data Science at Virginia Tech


Progress 01/15/22 to 01/14/23

Outputs
Target Audience:Our audience include the following categories: 1. Faculty and scientists working in plant disease management metagenomics, plant pathology, and agriculture automation and robotics. 2. Graduate and undergraduate students working in agriculture-related disciplines with a focus on plant disease management and automation. 3. Agricultural producers, farmers, crop advisors with a focus on plant production in greenhouses and controlled environments. 4. State commodity board and CALS advisory board members including high level executives, business owners and state policy makers. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided?Research objective one: A graduate intern, Simi Asher, has been training in the implementation of various deep learning-based approaches to 3D reconstruct plant models using multiple stereo image pairs and comparing them to their traditional counterparts. Similarly, Dominic Guri, a Ph.D student, is training on this robotic system to gain hands-on robotics experience. His training mostly involves motion planning and scheduling of robotic tasks such as the automated image data collection pipeline. Research objective two: Two graduate students, Thomas Hadlock and Jacob Neice, have been trained in DNA sequencing pipeline development and microfluidic device design and fabrication. They have also been trained in nanopore sequencing and data analysis protocols. Research objective three: A postdoc associate, Missi Zhang has been trained in plant pathology and hyperspectral imaging. One master student, Upasana Sivaramakrishnan (in Li Lab) has been trained in developing phenotyping tools for analyzing hyperspectral data and a 3D LiDAR model of tomato seedlings. One Ph.D. student, Victor Wang, was trained to develop a 3D model of tomato seedlings using a 3D LiDAR scan and compared the plant height between infected and healthy plants. How have the results been disseminated to communities of interest? From Aim 1 at CMU. Teaching: The fundamentals of the design of the robot system was included as an example in the course 16765 Robotics and AI for Agriculture at CMU during the 2023 spring semester. The collected dataset was also made available to students for class projects. From Aim 1 at CMU. Demonstration: We recently demonstrated the system design and automated data collection pipeline to several aspiring high school students and their mentors exploring robotics projects for STEM initiatives. From Aim 2 and 3 at Virginia Tech. we have attended APS meeting and CPS PI meeting in 2022. In both meetings, we used poster presentations to demonstrate our research progress and outcomes. From Aim 2 and 3 at Virginia Tech. We have hosted two outreach events: Virginia governor school of agriculture and Virginia 4H summer camp. In both events, we demonstrated the methods we used for early plant disease diagnosis, including microscopic observations of infected plant samples, microbiome sequencing of infected plants and hyperspectral imaging of infected plants. These two outreach activities involved more than 50 high school students combined. What do you plan to do during the next reporting period to accomplish the goals?Our plan for the next reporting period includes the following activities: Research objective one: Complete and thorough systems integration test with 6 DoF robot arm and the end-effector Evaluate systems performance on several metrics such as extraction accuration, grasping point detection etc. Complete systems integration with other goals under objectives 2&3. Research objective two: Continued expansion and optimization of the primer set to maximize the number of genes regions of interest that can be targeted and sequenced for each library. Optimization of multi-lane device operational protocols to enhance consistency. Advance integration with sample extraction platform. Research objective three: Submit the manuscript related to the hyperspectral imaging signature for infected plants. Collect canopy level 3D LiDAR, hyperspectral and RGB images for infected plants and compare to control plants. Develop methods of 3D segmentation of combined 3D+hyperspectral data of infected plants. The goal is to segment infected plants from health plant in a crowded canopy. Each aim plans to submit a paper separately and PI Li plan to organize an all together paper and a review paper towards the end of this project.

Impacts
What was accomplished under these goals? The goal of objective one is to develop an autonomous system for plant sample extraction. Our achievement in this round is the preliminary demonstration of the entire robotic operation including imaging, 3D reconstruction, computing grasping point, executing motion planning to grasp, and dropping the sample to a dedicated location. The goal of objective two is to develop a microfluidic device for sequencing-based pathogen detection. Our achievement this year is the successful design and fabrication of a multi-channel microfluidic device for LAMP PCR and nanopore library preparation. We have designed an assortment of multiplexed LAMP primers that accomplish rapid diagnostics as well as identify genomic variations to the single nucleotide level. We have developed a microneedle-based extraction method that can integrate into a robotic arm for sample extraction. The goal of objective three is to develop imaging-based disease detection in a greenhouse setting. The ultimate goal is to develop a model of disease spreading in a greenhouse. In the second year of this project, we collected hyperspectral imaging data for infected and healthy plant leaves. We compared multiple machine learning models in their accuracy in classifying infected and non-infected leaves. We also tested methods using vegetation indexes (VI). We found useful VI and machine learning models that can classify infected samples as early as 2 hours post-inoculation. We found that the plant height and distance to the hyperspectral camera have a strong impact on the spectral data. We also started to collect 3D point cloud data and the goal is to use a 3D model to correct for the changes in hyperspectral scan.

Publications


    Progress 01/15/21 to 01/14/22

    Outputs
    Target Audience:Our audience include the following categories: 1. Faculty and scientists working in plant disease management metagenomics, plant pathology, and agriculture automation and robotics. 2. Graduate and undergraduate students working in agriculture-related disciplines with a focus on plant disease management and automation. 3. Agricultural producers, farmers, crop advisors with a focus on plant production in greenhouses and controlled environments. 4. State commodity board and CALS advisory board members including high level executives, business owners and state policy makers. Changes/Problems:A major problem we are facing is that the changing greenhouse conditions (ambient temperature and daytime length vary among seasons) have a great impact on disease spreading in our testing greenhouse. It is challenging to maintain a disease-conducive environment to create consistent disease epidemics in our testing greenhouse. To solve this problem, we will adjust our inoculum concentration and irrigation programs for different seasons. What opportunities for training and professional development has the project provided?Research objective one: An undergraduate intern, Sohan Kulkarni, has been training in the design of the end-effector to selectively sample leaves from the tomato plants. He is exploring conventional rigid mechanisms as well as the current state-of-the-art compliance-based mechanisms in the design of novel end-effectors that interact with soft and flexible leaf samples. Similarly, Dominic Guri, a Ph.D student, is training on this robotic system to gain hands-on robotics experience. His training mostly involves motion planning and scheduling of robotic tasks such as the automated image data collection pipeline. Research objective two: A graduate student, Thomas Hadlock, has been trained in DNA sequencing pipeline development and microfluidic device design and fabrication. He has also been trained in nanopore sequencing and data analysis protocols. An undergraduate student, Katie Orr, has been trained in microfluidic device fabrication and sample processing procedures in order to operate the semi-automated process. Research objective three: A postdoc associate, Missi Zhang has been trained in plant pathology and hyperspectral imaging. A postdoc associate, Wei Xing, has been trained in using machine learning to analyze hyperspectral data. Two undergraduate interns (in Li Lab) have been trained in developing phenotyping robots for greenhouse producers. How have the results been disseminated to communities of interest?We have published two papers and one conference proceedings related to our work on using imaging for disease detection. What do you plan to do during the next reporting period to accomplish the goals?Our plan for next reporting period includes the following activities: Research objective one: Complete systems integration with 6 DoF robot arm with novel end-effector Perception system to identify sampling locations Complete systems demonstrating sample retrieval and delivery Research objective two: Expand and optimize the primer set to maximize the number of genes regions of interest that can be targeted and sequenced for each library Expand the device from single to multi-lane to prepare libraries composed of numerous samples in each run. Advance integration with sample extraction platform Optimize device platform for increased automation, specifically targeted at limiting human involvement with syringe pumps and magnetic handling steps Research objective three: Finish writing the manuscript related to the hyperspectral imaging signature for infected plants Collect canopy level hyperspectral and RGB images for infected plants. Develop methods to distinguish infected plants from non-infected plant using images. Monitoring disease spread using time lapse photography. Develop CRISPR based methods for pathogen detection.

    Impacts
    What was accomplished under these goals? The goal of objective one is to develop autonomous system for plant sample collection and image collection. Our achievement in the first year is the successful design and assembly of a tomato gantry and collection oftemporal RGB color images of tomato plants from germination to late vegetative growth. The goal of objective two is to develop microfluidic device for sequencing-based pathogen detection. Our achievement in the first year is the successful design and fabrication of microfluidic device for LAMP PCR and nanopore library preparation. The goal of objective three is to develop imaging-based disease detection in a greenhouse setting. The ultimate goal is to develop a model of disease spreading in a greenhouse. In the first year of this project, our initial goal is to establish visual signatures of infected plants using spectral imaging. Towards this goal, we have established several experimental protocols (see product section) on inoculation of plants, quantification of pathogen load, and hyperspectral imaging. These results will allow us to correlate disease progress stages/pathogen loads with visual signatures. We also expect to develop spectral imaging-based diagnosis methods for asymptomatic plants.?

    Publications

    • Type: Journal Articles Status: Published Year Published: 2021 Citation: Identifying Optimal Wavelengths as Disease Signatures Using Hyperspectral Sensor and Machine Learning. Xing Wei, Marcela Johnson, David Langston, Hillary Mehl and Song Li. Remote Sensing. 2021, 13(14), 2833; https://doi.org/10.3390/rs13142833
    • Type: Journal Articles Status: Published Year Published: 2021 Citation: Detection of Soilborne Disease Utilizing Sensor Technologies: Lessons Learned from Studies on Stem Rot of Peanut. Xing Wei, Marcela Aguilera, Rachael Walcheck, Dorothea Tholl, Song Li, David B. Langston, Jr., and Hillary L. Mehl. Plant Health Progress. 2021. 22(4): 436-444. https://doi.org/10.1094/PHP-03-21-0055-SYN
    • Type: Conference Papers and Presentations Status: Published Year Published: 2022 Citation: Interactive Deep Learning for Exploratory Sorting of Plant Images by Visual Phenotypes. Huimin Han, Ritvik Prabhu, Timothy Smith, Kshitiz Dhakal, Xing Wei, Song Li, and Chris North. Proceedings of NAPPN annual conference. Jan-30-2022. https://doi.org/10.1002/essoar.10508768.2