Progress 03/15/21 to 03/14/22
Outputs
Target Audience: The following target audiences were reached: 1. Researchers: This includes graduate students, undergraduate students, faculty, and postdoctoral associates in the performing team. It also includes other researchers who are not a part of this project but in the community, with whom we communicated through papers and presentations. These researchers were from diverse fields of agriculture, computer sciecne, and engineering. 2. Farmers: We interacted with a number of farmers to discuss with them technologies and potentials of mechanical weeding. 3. Press: We interacted with several journalists to help them communicate the potential of mechanical weeding with small robots 4. Stakeholders: We communicated with government and industry stakeholders about the potential of mechanical weeding agbots. This includes startup companies, more established companies, program managers in goverment funding organizations, and politicians. 5. Students: We incorporated material from this course into various undergraduate and graduate courses. We also invited and communicated with K-12 students who were interested in learning more about agriculture and the role of technology in agriculture. Changes/Problems:
Nothing Reported
What opportunities for training and professional development has the project provided?Graduate students: Graduate students worked on key aspects of the research. They were provided opportunity and mentoringto lead papers and research activities. Undergraduate students: Undergraduate students helped with key research aspects and were provided research mentorship and opportunity to co-author papers. Postdocs: Postdocs were provided with training and mentoring to lead teams of researchers. In particular, Dr. Andres Baqueros is being trained to lead his own lab as he is interested in an academic position. Faculty: Faculty directed students and postdocs to drive research program objectives, and worked together to collaboratively solve problems that lie at the intersection of disciplines. The ASA/CSSA/SSSA International Annual Meeting, Salt Lake City, UT (ISU) American Society of Plant Biologists (ASPB) Biology Summit (Virtual) (ISU) Neural Information Processing Systems (NeurIPS) Conference (Virtual) (ISU, UIUC) North American Plant Phenotyping Network Annual Conference (Hybrid) (ISU) Association for the Advancement of Artificial Intelligence (AAAI) Conference (Virtual)(ISU, UIUC) RF Baker Plant Breeding Symposium (Hybrid) (ISU) American Geophysical Union Fall Meeting (OSU) Our project team members took part in demonstrations/talks including: Plant Phenomics and Artificial Intelligence to Glean Information from Plant Sensing Technologies, Genetics, Genomics and Bioinformatics Symposium, ASAS Midwest Section Meeting, March 2022. (ISU) Role of Interpretable Machine Learning in Cyber-Agricultural Systems, International Conference on Digital Technologies for Sustainable Crop Production (DIGICROP), March 2022. (ISU) A Cyber-Physical Systems Approach to Agricultural Sustainability, Japanese Society of Agricultural Informatics (JSAI) Annual Conference, May 2021. (ISU) Application of Machine Learning and Artificial Intelligence in Plant Breeding, Tri-Societies Meetings, Salt Lake, City, UT (ISU) Soybean Breeding at ISU, Iowa Soybean Association, Ankeny, IA (ISU) Artificial Intelligence (AI), Human Cognition, and Ethics,CRAI-CIS Seminar Series, Department of Computer Science, Aalto University, Helsinki, Finland (GMU) Building an Infrastructure to Help Scientists Handle Big Data, Arizona Science (National Public Radio - Episode 297) Tucson, AZ (UofA) Coupling Vegetation Biophysics and Ecohydraulics for Improved Simulation of Land-Atmosphere Carbon-Water-Energy Exchange, American Geophysical Union Fall Meeting (OSU) Our project team members provided Education and Outreach opportunities including: Fundamentals of Plant Physiology- Online (ISU) Uni-High FarmBot Educational Outreach, AGORA Day, Champaign, IL (UIUC) COntext Aware LEarning for Sustainable CybEragricultural SystemsPoster presentation at ISU Day, Des Moines, IA(ISU) Additionally, students (including multiple women and underrepresented minority students) associated with the COALESCE project had the opportunity to participate in bi-weekly meetings to provide project update presentations to share their research findings. These opportunities provide valuable feedback on their progress and allow them to receive direction for further research and collaboration. How have the results been disseminated to communities of interest?The key results have been published in prestigious journals such as Plant Phenomics, IEEE Robotics and Automation Letters and presented at AI-Ag workshops at NeurIPS 2021 and AAAI 2022. All the papers are listed in products. PI Sarkar and Co-PIs have delivered multiple invited talks that included results from this project. In order to help the nascent Cyber-Ag research community grow and thrive, the COALESCE team has organized international workshops such as the Third International Workshop on Machine Learning for Cyber-Agricultural Systems (MLCAS) and the AI for Agriculture and Food Systems (AIAFS) workshop under the umbrella of THE AAAI 2022 conference. What do you plan to do during the next reporting period to accomplish the goals?Debuggin the Field Problem: Robotic insect monitoring and mitigation that includes the following objectives - Learn areas frequented by pests, Detect and distinguish harmful pests and Spray pests with pesticide locally on plants with robots. Visual Servoing: Our main goal will be to utilize cameras on the tip of the robot and on the base of the robot to navigate through obstacles. Networking for Field - CPS We expect to perform field experiments to answer the key questions raised through last year's analysis. Under-Canopy Cover-Crop Planting We expect to continue to work with industry to advance under-canopy cover-crop robot planting in the 2022 season.
Impacts
What was accomplished under these goals?
Formulation of the Debuggin the FieldProblem On average 20% of crops are lost due to pests. Farmers are unable to systematically identify, localize, and mitigate pests in a timely and cost-effective manner. A fleet of mobile ground robots, equipped with dexterous sensors and manipulators, are uniquely equipped to manage insect infestations under the canopy and increase plant yield. Currently, soft robotic arms lack precision and robustness for applications. A soft arm controller, based on a non-linear autoregressive network with exogenous inputs (NARX) architecture, is being designed and tested to solve this problem. Precision within tree cm has currently been shown in simulation and tested on the physical system. Improvements to improve this precision and add orientation tracking will soon be available. Chowdhary, Krishnan, and Arti Singh will work together on this problem. Visual Servoing for Agricultural Robots For soft continuum arms, visual servoing is a popular control strategy that relies on visual feedback to close the control loop. However, robust visual servoing is challenging as it requires reliable feature extraction from the image as well as accurate control models and sensors to perceive the shape of the arm, both of which can be hard to implement in a soft robot. We circumventthese challenges by presenting a deep neural network-based method to perform smooth and robust 3D positioning tasks on a soft arm by visual servoing using a camera mounted at the distal end of the arm. A convolutional neural network is trained to predict the actuations required to achieve the desired pose in a structured environment. Integrated and modular approaches for estimating the actuations from the image are proposed and are experimentally compared. A proportional control law is implemented to reduce the error between the desired and current image as seen by the camera. The model together with the proportional feedback control makes the described approach robust to several variations such as new targets, lighting, loads, and diminution of the soft arm. Furthermore, the model lends itself to be transferred to a new environment with minimal effort. Krishnan and Chowdhary are working together on this problem Control of Soft Continum Arms Soft continuum arms (SCAs) that are controlled by visual servoing (VS) present trade-offs between the camera range and tracking accuracy. Cameras placed at a distance (eye-to-hand) can observe a larger workspace area and the SCA tip, while a camera at the end effector (eye-in-hand) can more accurately survey the target. In this project, we present a hybrid eye-to-hand and eye-in-hand VS scheme to track a desired object in the SCA's worksapce. When the target is not in the field-of-view of the tip camera, hand-to-eye VS is implemented using a wide field-of-view camera on the soft robot's base, to servo the soft robot's tip to a feasible region where the target is expected to be seen by the tip camera. This region is estimated by solving an optimization problem that finds the best region to place the SCA assuming a constant curvature model for the SCA. When the target is seen by the tip camera, the system switches to a hand-in-eye controller that keeps the target in the desired image position of the tip camera. Experimental results on the popular BR^2SCA demonstrates the effectiveness of the hybrid VS scheme under practical settings that include external disturbances. Krishnan and Chowdhary are working together on this problem Networking for Field CPS Emerging autonomous farm applications have diverse requirements e.g. teleoperation and virtual walkthrough have low latency requirement, whereas data collection for precision agriculture requires high throughput. Unfortunately, the IoT network in autonomous farms has limited connectivity options and power constraints. Every task or data transfer utilizes limited resources and depletes power. It is useful to have a centralized traffic controller or scheduling system that schedules tasks such that it satisfies requirements of tasks while managing limited resources. We will start off by studying different applications and understanding workload requirements of these applications. Then we understand different connectivity options that are optimal for different applications. Given these connectivity options, a centralized traffic controller can schedule jobs from different applications over different options, as per the application requirements and power constraints. Mittal, Arti Singh and Chowdhary are working together on this problem Foundational Advances in Planning for High Degree of Freedom Robotic Manipulators Hauser and students continued to advance the foundations of planning and reasoning AI for control of high degree of freedom manipulator arms. These advances are designed to enable arms to plan more complex trajectories faster and more efficiently. The problems are inspired by difficult challenges in manipulation afforded by agricultural robotic CPS. Hauser is working on this problem Under-Canopy Cover-Crop Planting We developed algorithms and software for under-canopy cover-crop planting robots. We showed through numerical simulations that a team of fivesuch robots can cover an eightyacre field planting cover-crops in under fivehours when operating fully autonomously. We evaluated our algorithms for autonomy on robots created by EarthSense Inc., and demonstrated the feasibility of vision and LiDAR based autonomy for field operation. This has inspired further development at EarthSense for bringing this technology to farmers as a lower-cost option towards the farms of the future. These activities were conducted at the Illinois Autonomous Farm, where we also demonstrated the feasibility of autonomous control of under-canopy robots using vision sensors, by extending our work from Narenthiran et al. 2020 RSS (CropFollow). Chowdhary is working on this problem Feasibility of Mechanical Weeding In our latest paper published in the prestigious IEEE Transactions of Robotics, we proposed AgBots 3.0, an algorithmic framework that leverages prediction of weed emergence patterns to optimize robot paths so that fields can be covered optimally with the least number of robots. Our results demontrate through extensive numerical simulation that mechanical weeding robots can feasibly keep fields weed-free, and highlight ways in which cost can be reduced by minimizing the number of robots through smart planning. This work was continuation from Chowdhary and Adam Davis' earlier NSF-CPS project funded through NIFA.
Publications
- Type:
Journal Articles
Status:
Published
Year Published:
2022
Citation:
W. McAllister, J. Whitman, J. Varghese, A. Davis and G. Chowdhary. 2022. Agbots 3.0: Adaptive Weed Growth Prediction for Mechanical Weeding Agbots. In: IEEE Transactions on Robotics, vol. 38, no. 1, pp. 556-568, Feb. 2022, doi: 10.1109/TRO.2021.3083204.
- Type:
Journal Articles
Status:
Published
Year Published:
2022
Citation:
S. Kamtikar, S. Marri, B. Walt, N.K. Uppalapati, G. Krishnan and G. Chowdhary. 2022. Visual Servoing for Pose Control of Soft Continuum Arm in a Structured Environment. In: IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 5504-5511, April 2022, doi: 10.1109/LRA.2022.3155821.
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2022
Citation:
S. Kamtikar, S. Marri, B. Walt, N. K. Uppalapati, G. Krishnan and G. Chowdhary. 2022. Visual Servoing for Pose Control of Soft Continuum Arm in a Structured Environment. In: RoboSoft Conference, and jointly in IEEE Robotics and Automation Letters. Edinburgh, Scotland, April 2022.
|