Progress 04/01/23 to 03/31/24
Outputs
Target Audience:The target audience of our research includes: * Members of the research community in computer vision and machine/deep learning. We attended the 18th International Symposium on Visual Computing in October 2023(http://www.isvc.net) * Start-up companies and other industries in precision agriculture (we have been collaborating with Dr. Dimitris Zermas, Principal Scientist at Sentera) * Practitioners in the application of computer vision and machine/deep learning for precision agriculture. * University students at the undergraduate and graduate levels. Changes/Problems:
Nothing Reported
What opportunities for training and professional development has the project provided?The project supports a PhD student who has been working on the project since January 2023. The student is closely mentored by the PI and has gained important experience in terms of performing research, writing papers, presenting at conferences, and reviewing papers. The student is co-advised by an industrial collaborator (Dr. Dimitris Zermas from Sentera), gaining important feedback about the practical aspects of his research. How have the results been disseminated to communities of interest?A conference paper was published/presented at the 18th International Symposium on Visual Computing (ISVC'23). A special track on "precision agriculture" was also organized at the ISVC'23 in collaboration with colleagues from other academic institutions and industries. We will be submitting a journal paper and one more conference paper in 2024. What do you plan to do during the next reporting period to accomplish the goals?In relation to goals (2) and (3), we plan to further extend the robustness of the above algorithms to different corn fields using domain adaptation and synthetic data augmentation methods as discussed earlier. We also plan to investigate attention-based mechanisms to provide a strong prior for detecting nitrogen-deficient regions in an image. This could be very useful for automatic image annotation and/or improved segmentation. We will also investigate the application of the developed algorithms to different but related problems such as weed detection.
Impacts
What was accomplished under these goals?
The major development of the period is the development and testing of a new Deep Learning (DL) methodology for whole image nitrogen deficiency classification in corn plants from aerial imagery; this relates to goals (2) and (3) of the project. The new method was designed to detect nitrogen deficiencies regardless of the growth stage of corn which is a more challenging problem than early-stage detection. This is also more practical since nitrogen deficiency can occur at any stage. By designing a generic detector, we can collect training data from any stage, therefore, improving overall performance and robustness. Transfer learning has been an integral part of our approach leading to significant performance improvements. Currently, our emphasis is on improving the robustness of the detector in different corn fields to further improve its generality and robustness. In this context, we have been investigating domain-adaptation methods based on CycleGAN models. The main idea is to map the data from different corn fields to a "target" domain both for training and testing. This would allow us to train the detector with data from different fields once they have been transformed into a common domain, effectively increasing the size of training data. Data from a new field can then be processed once they have been transferred to the same target domain. Concurrently with the domain adaptation approach, we have also been investigating synthetic data generation methods for data augmentation and adaptive attention mechanisms based on class activation maps (CAMs) as a way to improve classification accuracy by guiding the DL models to more effectively focus on the nitrogen deficient regions in an image.
Publications
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2023
Citation:
Aminul Huq, Dimitris Zermas, and George Bebis, Identification of Abnormality in Maize Plants from UAV Images Using Deep Learning Approaches", 18th International Symposium on Visual Computing (ISVC'23), Lake Tahoe, Nevada, USA, October 16-18, 2023.
|
Progress 04/01/20 to 03/31/21
Outputs
Target Audience:The target audience during this period of development primarily related to the communities (researchers and technologists) in precision agriculture. This is a subset of the overall aimed audience which in the later years of the project will also include - among others - end users of the technology (e.g., farmers and precision agriculture specialistis). The audience targeted during this period has been reached via research presentations and associated discussions in conferences (e.g., ICRA2021:https://www.ieee-icra.org/ - although remotely due to the effects of the global coronavirus pandemic) and through direct communication (e.g., with researchers of the PhenoRob cluster of excellence of the University of Bonn in Germany:https://www.phenorob.de/). Changes/Problems:
Nothing Reported
What opportunities for training and professional development has the project provided?The project has so far contributed into the training of (post)graduate-level researchers in the domain of precision agriculture. This has gone beyond the researchers specifically involved in the project and has helped to educate four members of our University of Nevada, Reno team in parts of the problems relevant to robotized precision agriculture. How have the results been disseminated to communities of interest?So far the main community of interest has related to the researchers and technologists in precision agriculture. Relevant work of our team was presented at IEEE ICRA 2021, while we further keep maintaining and updating the open-source code repositories of our team providing path planning functionality for robotic systems. Our path planners are compatible with the Robot Operating System (ROS) and thus allow for seamless integration to diverse platforms as long as a waypoint guidance functionality is implemented. What do you plan to do during the next reporting period to accomplish the goals?Our goals for the next reporting period are the following: In path planning: Further develop our path planning algorithms with the goal to account for the terrain underlying crops and thus allow optimal and robust path planning for ground robots. Further develop our path planning algorithms to incorporate information for areas with nutrientdeficiencies over which the planner should guarantee higher quality of sensor observations Publish the relevant results (e.g., in the upcoming deadline for IEEE ICRA 2022 and/or the IEEE/RSJ IROS 2022 conferences) and possibly release open-source code. In perception for precision agriculture: Further progress in our ability to associate multi-modal data coming from multispectral imaging and depth observations both for the purposes of robust mapping but also to enable new ways for inference regarding nutrient deficiencies. Publish the relevant results (e.g., in the upcoming deadline for IEEE ICRA 2022 and/or the IEEE/RSJ IROS 2022 conferences) and possibly release open-source code. Dissemination to further communities: We intend to publish,present and broadly disseminate to further communities - for example by attending specialized showcases (e.g., "Farms.com Precision Agriculture Conference & Ag Technology Showcase").
Impacts
What was accomplished under these goals?
Our main accomplishments in this period primarily relate primarily to the goal of: 1) The introduction of a collaborative robots-based and plant-centric framework which is a paradigm shift in crop management. The objective is to use robotics assets to materialize this plant-centric framework. Both aerial and ground robots will be integrated into a system-of-systems fashion. In particular, we have built upon years of research within our team in exploration and coverage path planning in order to progress in developing a new framework on autonomous path planning for precision agriculture applicable to both aerial and ground robots (and their collaboration). Specifically, we have built upon the ideas ofthe following previous work of members of our university: A. Bircher, M. Kamel, K. Alexis, H. Oleynikova, R. Siegwart,"Receding Horizon "Next-Best-View" Planner for 3D Exploration", IEEE International Conference on Robotics and Automation 2016 (ICRA 2016), Stockholm, Sweden. Open-Source Git Repo:https://github.com/ethz-asl/nbvplanner T. Dang, F. Mascarich, S. Khattak, C. Papachristos, K. Alexis, "Graph-based Path Planning for Autonomous Robotic Exploration in Subterranean Environments", IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2019, Macau, China T. Dang, M. Tranzatto, S. Khattak, F. Mascarich, K. Alexis, M. Hutter, "Graph-based Subterranean Exploration Path Planning using Aerial and Legged Robots", Journal of Field Robotics, November, 2020, https://doi.org/10.1002/rob.21993, Videos: https://youtu.be/SNMsKAnCQkw , https://youtu.be/W9lgdmDg6UM. Open-Source Git Repo: https://github.com/unr-arl/gbplanner_ros Mihir Rahul Dharmadhikari, Tung Dang, Lukas Solanka, Johannes Brakker Loje, Dinh Huan Nguyen, Nikhil Vijay Khedekar, and Kostas Alexis, "Motion Primitives-based Path Planning for Fast and Agile Exploration using Aerial Robots", IEEE International Conference on Robotics and Automation (ICRA) 2020, May 31 - June 4 2020, Paris, France. Video: https://youtu.be/ZvUedi5mzN8. Open-source Git Repo:https://github.com/unr-arl/mbplanner_ros and the recent contribution of a core member of our project team: Mihir Rahul Dharmadhikari, Harshal Deshpande, Tung Dang, Kostas Alexis, "Hypergame-based Adaptive Behavior Path Planning for Combined Exploration and Visual Search", IEEE International Conference on Robotics and Automation (ICRA), May 30-June 5, 2021, Xi'an China. Video:https://youtu.be/Nfo-RQ_RCFg In order to develop innovative path planning algorithms that allow a robotic system to ensure full exploration and mapping of an area given no prior knowledge of the environment. At the same time, we have incorporated the use of metrics for nutriend deficiency creating planning behavior adaptation to ensure high-quality sensor observations in the most critical areas of the crop. Our preliminary results in the domain relate to our under-review submission: M. Dharmadhikari, M. Kulkarni, K. Alexis, "Path Planning for Nutrient Deficiency-aware Attentive Crop Monitoring Using Aerial Robots", IEEE Aerospace Conference, March 5-12, Yellowstone Conference Center in Big Sky, Montana, 2022 (under review). Importantly, during these developments we maintain the goal to release stable versions of such software contributions as open-source code, analogously to previous practice from some of our team members as demonstrated in some of the previously mentioned publications. Beyond the above, during this period we also worked on an algorithm to allow the multi-modal association of camera images coming from different modalities. Specifically, also based on a synergistic effort we have developed an algorithm that allows to align images in the visual spectrum with images encoding Near InfraRed (NIR) information provided by multispectral cameras tailored to precision agriculture. To achieve this goal we exploit progress of the computer vision community in the domain of multi-modal image alignment and specifically employ the Mattes Mutual Information Metric to guide the registration process.
Publications
- Type:
Conference Papers and Presentations
Status:
Accepted
Year Published:
2021
Citation:
Mihir Rahul Dharmadhikari, Harshal Deshpande, Tung Dang, Kostas Alexis, "Hypergame-based Adaptive Behavior Path Planning for Combined Exploration and Visual Search", IEEE International Conference on Robotics and Automation (ICRA), May 30-June 5, 2021, Xi'an China. This publication was published by one of the members working in this project but relies on work activities this person conducted prior to joining the project and thus the NIFA project is not acknowledged. This publication is included in the list as the core underlying functionality (graph-based path planning combined with behavior mode-switching using a hypergame formulation) is the one we also use to develop path planning for crop coverage that accounts for information of nutrient deficiency. In addition, the first author of this publication conducts research in the domain of this project after he arrived at the University of Nevada, Reno (Fall 2021).
- Type:
Conference Papers and Presentations
Status:
Under Review
Year Published:
2022
Citation:
M. Dharmadhikari, M. Kulkarni, K. Alexis, "Path Planning for Nutrient Deficiency-aware Attentive Crop Monitoring Using Aerial Robots", IEEE Aerospace Conference, March 5-12, Yellowstone Conference Center in Big Sky, Montana, 2022 (under review).
|