Progress 01/01/24 to 12/31/24
Outputs
Target Audience:1. Target Audience In the following we provide a summary of scientists, postdoctoral researchers, graduate students, and undergraduate students participating in the project during this reporting period. They worked on five different AI research themes: field robot motion control, scheduling, manipulation, sensing, and pesticide residual analysis. They have also contributed to other activities associated with education, outreach, and dissemination. Four faculty members from the University of Central Florida (UCF) and 1 faculty from the Washington State University (WSU) who is affiliated with AgAID. Four graduate students and 1 postdoctoral researcher from the Department of Mechanical and Aerospace Engineering (MAE) at UCF. Four graduate students from the Department of Computer Science (CS) and the Center for Research in Computer Vision (CRCV) at UCF. Two graduate students from the NanoScience Technology Center (NSTC) at UCF and one non-student volunteer (Master Degreefrom NJIT). Two graduate students and 1 administrator (as a manager) from the Department of Biological Systems Engineering at WSU. Five undergraduate student researchers from UCF. Since the project just got started, the research progress was disseminated within a limited range of audience. 1 conference paper is published in 2024 ASABE Annual International Meeting, Anaheim, CA. 1 conference abstract (poster) is accepted to "2024 Southeastern Regional Meeting of the American Chemical Society". We hosted 4 high school students via the UCF Campus Connect program in the summer of 2024. We hosted 1 UG researcher via the UCF Computer Vision REU program in the summer of 2024. The activities of the project are disseminated to the general public via the website and news. https://mae.ucf.edu/ucfexpandingai/ UCF news: www.ucf.edu/news/ucf-researchers-lead-project-to-develop-ai-driven-technologies-for-agriculture/. Co-PI Dr. Karkee affiliated with AgAIDhas frequent communication with growers. We organized a one-day symposium on Oct. 3, 2024. We plan to attend the 2024 Florida Ag Expo. The PI attended the "AgAID Annual Review and All-Team" Meeting (September 11-13, 2024, Corvallis, Oregon) and presented an overview of this ExpandingAI project. We will attend the NSF ExpandAI leadership workshop (EXAIL) on Oct. 7-10, 2024. 2. Impacts or Expected Impacts In Year 1, we have already seen growing interests among faculty and students at UCF in utilizing AI technologies to help agricultural operations. Many undergraduate students submitted their application to join our undergraduate research, although we can only select four to participate in the Year 1 program. As mentioned in the proposal, if the partnership program is successful, the five directions on which our AI research focuses are expected to enhance the efficiency of specific agricultural field operations; more students and researchers/educators with a diverse background will show interests in conducting collaborative research to address emerging and challenging agricultural engineering problems; and we will widely disseminate our research progress via publications, symposiums, courses and other means. 3. Efforts Students with a diverse background were recruited and participated in this program. Among them 15.8% are female and 26.3% are with Hispanic background. Changes/Problems:
Nothing Reported
What opportunities for training and professional development has the project provided?The PI/co-PIs plan to attend the 2024 NSF EXAIL, and are expected to obtain training from peer AI institutes. The PI learned a lot in the AgAID annual review meeting hosted in Oregon State University in Sept. 2024. As can be seen in other sections, we provided training opportunities for about 5 undergraduate students, 13graduate students, and 1 postdoctoral researcher. They learned agricultural engineering problems, started to get familiar with AI technologies, and practiced writing and presentation skills.In addition, students at WSU were provided with the opportunity to share the results with tree fruit growers, learn from them and present in extension/outreach events. How have the results been disseminated to communities of interest?In year 1, we have disseminated our research and education activities via websites, 2024 ASABE conference, 2024 Southeastern Regional Meeting of the American Chemical Society, and various outreach activities for undergraduate and high school students. We attended or participated in the following list of events. 2024 ASABE Annual International Meeting, Anaheim, CA. The UCF Campus Connect program in the summer of 2024. The UCF Computer Vision REU program in the summer of 2024. The "AgAID Annual Review and All-Team" Meeting (September 11-13, 2024, Corvallis, Oregon) The UCF AI/Ag symposium on Oct. 3, 2024. Also, we plan to attend the following events (after the report is submitted but before Dec. 31, 2024) 2024 Florida Ag Expo as we planned. Students will present a poster at the "2024 Southeastern Regional Meeting of the American Chemistry Society". The NSF ExpandAI leadership workshop (EXAIL) on Oct. 7-10, 2024. What do you plan to do during the next reporting period to accomplish the goals?The following tasks are planned for Year 2 (Jan. 1, 2025 - Dec. 31, 2025). 1. Research The following research tasks are planned for Year 2. "Motion Control": The asymptotic stability of the new "actor-critic" algorithm in a receding horizon framework will be proven and it will be robust with respect to numerical errors from discretization. Then the algorithm will be validated in simulation and be prepared for hardware demonstration in Year 3. "Scheduling": A neural network will be constructed for the "critic" part of the scheduling algorithm. The algorithm will then be validated in a simulated harvesting scenario. "Sensing": We will develop specialized text-to-image diffusion models to generate various images of crops and agricultural scenarios, thereby providing additional high-quality synthetic data to support a range of downstream tasks, further improving the robustness and performance of the foundation model. We will tailor thefoundational models for specific applications, including plant phenotyping and crop disease detection. These tasks aim to enhance the robustness and adaptability of the models, driving further advancements in AI-driven agricultural technologies. "End-effector": We will build on the demonstration framework created during the first year to collect datasets and learn manipulation policies for robotic fruit harvesting. One of the primary focuses of our work will be the need to perform the work at high speed and on an autonomous platform, with comparatively limited computational capabilities. Some of the challenges we need to investigate are: how can the current, very high performance, but comparatively slow segmentation models be distilled to a fast, specialized model that can support a robot in real-time? How can the algorithms developed for tabletop robotic manipulation be adapted to the 3D settings of fruit picking from a plant? How can the robot learn to perform real-time visual reasoning about the dynamics of a plant? Solutions to these problems can significantly enhance the speed, reliability and robustness of agricultural robots. "Pesticide residue analysis": To support AI research tool development, more pesticide applied leaf images will be collected and processed. We will focus on demarketing pesticide residue coverage on citrus leaves affected by foliar disease (such as citrus canker) to minimize artifacts. We will also collect similar data on apple leaves in collaboration with WSU team. At the WSU side, more images will be collected, and vision system models will be further trained, and evaluated for accurately delineating target objects such as fruit and flowers. A distance estimation module will be developed and tested to localize the target flowers and obstacles. The orchard navigation system will be further advanced. 2. Training and Education: An "independent study" or "directed research" type course will be offered to graduate students at both UCF and WSU, focusing on AI research and technologies and how they are utilized to assist agricultural operations to enhance farming efficiency. The second cohort of undergraduate and potentially graduate students, with a diverse background, will be recruited to participate in the research program and also in the summer exchange program. These students will also be involved in a year-long training program. We will continue to find opportunities for undergraduate students and potentially high school students to participate in AI/Ag research via programs such as the UCF Campus-Connect program and the UCF Computer Vision REU program. The PI/co-PIs and faculty involved in this research will attend PI meetings to further enhance their research collaboration and education. We will continue to train students and postdoctoral researchers in conducting experiments, writing reports, and presenting their research findings. 3. Outreach and Dissemination: The project website will be regularly updated to showcase the project activities. We will submit journal and conference papers to disseminate research results and will encourage collaborative publications among UCF/AI and AgAID researchers. We will attend professional events to disseminate our research results and discuss possible collaborations.
Impacts
What was accomplished under these goals?
1. Major Activities Conducted for Objectives 1 and 3: The project aims to develop AI driven methods for five areas. Dr. Xu's group developed the first version of a custom "actor-critic" simulation framework for optimal motion control of an agricultural field robot. Subspace optimization is used in the "actor" network to generate trajectory controls for a robot, while uncertainties, e.g. numerical errors from discretization, are adjusted in the "critic" network. Dr. Xu's group conducted a comprehensive survey study about different scheduling algorithms that have been used in agricultural applications. An in-house developed algorithm has been included in the "actor" part of the scheduling algorithm framework. Dr. Chen's group focused on geospatial foundational models, conducting extensive surveys and collecting data from diverse sources to train these models. Building on prior work in geospatial modeling, the team successfully adapted the models for agricultural applications, such as plant disease classification. Dr. Chen also worked with Dr. Santra's team to create computer vision algorithms for pesticide residue analysis, further advancing precision agriculture technologies. Dr. Bölöni's group performed a comprehensive study of algorithms for the control of robotic manipulators driven by computer vision. The team assembled a small robotic platform for the study of the various of computer vision systems harvesting. The team developed a variety of demonstration collection systems including those based of gamepad (Microsoft X360-based) controller, keyboard-based controllers, as well as computer vision-based gesture controllers based on hand pose detection. For the latter, solutions based on YOLO V8 (https://docs.ultralytics.com/)) and Mediapipe® were implemented and compared. Dr. Santra's group developed experimental setup and protocols for foliar application of two model commercial pesticides (copper and oxytetracycline) on citrus plants. A portable 3D printed sample compartment prototype was fabricated, enabling a more controlled environment for image data acquisition under field conditions. Pesticide residue coverage area was determined using these digital images. Pesticide residue coverage analysis was done using a specialized Mask Region CNN. This CNN was pre-trained on the MS COCO dataset and fine-tuned training with acquired datasets in laboratory and field conditions. Dr. Karkee's group worked on smart and robotic technologies with optimized human-robot-canopy interactions. This task focused on using AI technologies and novel manipulation mechanisms in partnership with the ongoing AgAID institute. In perception, this module focuses on selecting, testing, and integrating the appropriate camera. Two cameras have been tested for the application. Their respective dependencies and ROS® packages have been configured to work. SLAM-based localization has been achieved using a LIDAR sensor. Multiple vision models were assessed for the application(s). YOLO V7/V8 (https://docs.ultralytics.com/) and Masked RCNN have been tested for apple detection and flower thinning, respectively. The undergraduate students formulated problems based on what they learned in the summer exchange program, as collaborations between UCF/AI and AgAID researchers. Problem 1 is about the kinematic modeling of a vehicle used in the WSU Center for Precision & Automated Agricultural Systems, as well as the output model from the vehicle global coordinate to the focal plane of the camera installed on the WSU vehicle. Problem 2 is about the problem formulation about how to control the end effector by precisely estimating the height of the target fruit. Problem 3 is about a pesticide residue detection progress primarily consisting of data collection via Galaxy A7116U® smartphone. Data was collected on grape vines, and apple trees, and images will be processed to train an AI model to recognize said residues. We had one undergraduate participated in the UCF CRCV REU program, where she worked on developing vision algorithms for fruit detection and height estimation to support robotic picking. 2. Major Activities Conducted forObjectives 2 and 3: We are using a wide range of activities to attract students, researchers and educators with a diverse background to work in agriculture engineering and AI problems, and to collaborate with AgAID researchers. A team of 4 UG students were selected from a large application pool from UCF. They participated in the year-long undergraduate research program at UCF, and three of them continued to the end of 2024. The same team of 4 UG students from UCF participated in the summer exchange program at WSU (4 weeklong). Among these four UG students, one is female and one has Hispanic background. 13graduate students are involved, 2 are female, and 4has minority background. More faculty members, and many undergraduate and graduate students showed interest in conducting research in the field of robotics and AI for agricultural applications. 1 UG student (female) did a 12-week summer REU program. 4 high school students were introduced to the topic of "AI for end effector designs", among them two are female and 1 has minority background. 3. Major Activities Conducted for Objectives 3 and 4: We willwidely disseminate our research progress via publications, symposium, and courses, etc. A course syllabus was developed focusing on AI technologies for Agricultural Applications. This graduate level course in the form of "independent study" or "direct research" will be delivered in Spring 2025 and offered to students at both UCF and WSU. A 1-day workshop/symposium was held at UCF on Oct. 3, 2024. 13 talks were delivered, and 33 researchers and students attended, among them five attended online. We have 1 conference paper in ASABE, and 1 conference poster is accepted. The PI attended the "AgAID Annual Review and All-Team Meeting" in Corvallis, Oregon. Three of the PI/CoPIs will attend EXAIL in Pittsburg in Oct. 2024. We plan to attend the UF GCREC organized Florida Ag Expo in November 2024. 4. Data collected The research mostly focused on problem formulations, discussion with AgAID researchers, and developing methodologies. Therefore, large scale data collection hasn't started yet. However, Dr. Karkee's group and Dr. Santra's group have started the process of data collection for their respective research thrusts. Dr. Karkee's group has collected data for training the machine vision model for preliminary testing of end-to-end learning for robotic manipulation using a soft manipulator. His group also collected data to develop a SLAM-based technique to map orchard environment and perform robotic fruit movement. Dr. Santra's group has collected pesticide applied citrus leaf images in laboratory and field settings. These images were used to develop a method for the assessment of pesticide residue coverage on citrus leaf surface using machine learning tools. 5. Summary statistics and discussion of results We haven't achieved any significant results yet. However, the progress of research, education, and dissemination is going forward as planned, and we anticipate having some results next year. 6. Key Outcomes or Other Accomplishments Realized. A change in knowledge: The AI methods are still under development. The processing of using these technologies by researchers at AgAID and/or other stakeholders has not started. Via a range of activities, more students at UCF are exposure to agricultural robotic applications and how AI can help agricultural operations. A change in action: Each PI/coPI is developing their AI algorithms for specific agriculture tasks. We started several education and dissemination activities, like symposium, summer exchange program, etc. A change in condition: In Year 1, there is no change in conditions.
Publications
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2024
Citation:
A. Rivera Palma, Y. Xu, L. Tituana, and M. Fritts, Scheduling of robotics or machinery operations in agricultural fields: a review, ASABE Annual International Meeting, Marriott Anaheim, CA, July 28-31, 2024
- Type:
Other
Status:
Accepted
Year Published:
2024
Citation:
(Poster) G. Diracca, A. Basavaraju, E. Davidson, and S. Santra, Development of a digital analysis tool powered by machine learning for pesticide leaf residue coverage estimation: a strategy towards precision agriculture, 2024 Southeastern Regional Meeting of the American Chemical Society, Undergraduate Research Symposium (Poster), Oct. 23-26, 204, Atlanta, GA. Accepted.
|