Progress 12/15/18 to 12/14/22
Outputs Target Audience: Researchers: This includes graduate students, undergraduate students, faculty, and postdoctoral associates in the performing team. It also includes other researchers who are not a part of this project but in the community, with whom we communicated through papers and presentations.These researchers were from diverse fields of agriculture, computer science, and engineering. Farmers: We interacted with a number of farmers to discuss with them technologies and potentials of mechanical weeding. Press: We interacted with several journalist to help them communicate the potential of mechanical weeding with small robots Stakeholders: We communicated with government and industry stakeholders about the potential of mechanical weeding agbots. This includes startup companies, more established companies, program managers in goverment funding organizations, and politicians. Students: We incorporated material from this course into various undergraduate and graduate courses. We also invited and communicated with K-12 students who were interested in learning more about agriculture and the role of technology in agriculture. Changes/Problems:
Nothing Reported
What opportunities for training and professional development has the project provided?1. Graduate students were mentored by the PIs; they often received instruction and advice from multiple PIs on the project. 2. Undergraduate students were mentored by PIs and graduate students. 3. Postdoc on the project received mentoring and also advised graduate students. Naveen Uppalpati, who graduated recently, began working first as a postdoc, and now as a research scientist, partially supported by this project. 4. Material for the ABE 426 Principles of Mobile Robotics course was updated based on some results from this work. ABE 424 was cross listed with ECE 426, now being taught across multiple departments. Chowdhary included material from this work in his CS 498 Mobile Robotics for Computer Scientists. Gazzola utilized learning from this course in his course on continuum systems modeling with coserat rods. 5. Graduate students and postdocs were provided with opportunities to present technical talks and write technical papers, and they were also provided with regular feedback. How have the results been disseminated to communities of interest?1. We published papers in top venues. 2. The PIs gave talks around the country and the world, including PI Chowdhary's talks at King Abdullah University of Science and Technology (KAUST), and several other talks. 3. We published videos on youtube and other venues. 4. News articles were published by others based on our work, for which we provided quotes and materials 5. The software Elastica was made open source and published on this website:https://www.cosseratrods. 6. Our work was covered by popular press. What do you plan to do during the next reporting period to accomplish the goals?
Nothing Reported
Impacts What was accomplished under these goals?
1. We created a hybrid hard-soft robotic arm: Our prototype arm was published in RSS 2020, and is the first of its kind to include hard arm and extensible soft components for agriculturl applications. From last year it was installed on a TerraSentia 2018 robot with newly designed arm coverings. The arm includes a traditional hard component as well as a soft extender. The goal of this hybrid arm is to mitigate the issues of high elasticity of soft arms. This approach combines the benefit of both soft and rigid link manipulators. This has led to a more advanced and field deployment ready arm that we intend to use this year in berry fields. 2. We made major advances in using visual servoing for approaching berries using cameras on the robot. In our earlier work, the position of the arm and the berry was provided to the robot control system through magnetic sensors. This was a good place to start, but a limiting assumption in real-world fields. This year we broke through this challenge by developing visually guided systems for the control of the robot arm. The system utilizes camera on the tip of the robot and a neural network to determine the pose difference between a desired image and the current image seen by the robot. This work is described in our R-AL + RoboSoft 2022 paper (Kamtikar et al.). The work is currently limited to structured settings, but we are working on extending it to field settings. 3. We created an architecture and workflow that utilizes two cameras, one on the robot body, and one at the tip of the arm (as discussed in #2) to control the robot end-effector pose. This will enable the robot to drive the end effector towards the berry by first seeing it from the camera on the base of the robot, and then approaching it using visual servoing. This architecture will be trained with real data using part-affinity fields to determine optimal grasping pose for the end effector. 4. We created models of the soft arm using Gazzola's methods and demonstrated that the predictions of the models match the actual arm movement. This is accepted for publication in ICRA 2022.
Publications
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2022
Citation:
S. Kim, H. Chang, C. Shih, N. Uppalapati, U. Halder, G. Krishnan, P. Mehta, and M. Gazzola. 2022. A physics-informed, vision-based method to reconstruct all deformation modes in slender bodies. IEEE International Conference on Robotics and Automation (ICRA), 48104817.
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2022
Citation:
T. Wang, U. Halder, E. Gribkova, M. Gazzola, and P. Mehta. 2022. Control-oriented modeling of bend propa- gation in an octopus arm. American Control Conference (ACC), 13591366.
- Type:
Journal Articles
Status:
Accepted
Year Published:
2023
Citation:
H. Chang, U. Halder, C. Shih, N. Naughton, M. Gazzola, and P. Mehta. 2023. Energy shaping control of a muscular octopus arm moving in three dimensions, Proceedings of the Royal Society A (Accepted).
|
Progress 12/15/20 to 12/14/21
Outputs Target Audience:The following categories of target audiences were reached: Researchers: This includes graduate students, undergraduate students, faculty, and postdoctoral associates in the performing team. It also includes other researchers who are not a part of this project but in the community, with whom we communicated through papers and presentations.These researchers were from diverse fields of agriculture, computer science, and engineering. Farmers: We interacted with a number of farmers to discuss with them technologies and potentials of mechanical weeding. Press: We interacted with several journalist to help them communicate the potential of mechanical weeding with small robots Stakeholders: We communicated with government and industry stakeholders about the potential of mechanical weeding agbots. This includes startup companies, more established companies, program managers in goverment funding organizations, and politicians. Students: We incorporated material from this course into various undergraduate and graduate courses. We also invited and communicated with K-12 students who were interested in learning more about agriculture and the role of technology in agriculture. Changes/Problems:
Nothing Reported
What opportunities for training and professional development has the project provided?1. Graduate students were mentored by the PIs; they often received instruction and advice from multiple PIs on the project. 2. Undergraduate students were mentored by PIs and graduate students. 3. Postdoc on the project received mentoring and also advised graduate students. Naveen Uppalpati, who graduated recently, began working first as a postdoc, and now as a research scientist, partially supported by this project. 4. Material for the ABE 426 Principles of Mobile Robotics course was updated based on some results from this work. ABE 424 was cross listed with ECE 426, now being taught across multiple departments. Chowdhary included material from this work in his CS 498 Mobile Robotics for Computer Scientists. Gazzola utilized learning from this course in his course on continuum systems modeling with coserat rods. 5. Graduate students and postdocs were provided with opportunities to present technical talks and write technical papers, and they were also provided with regular feedback. How have the results been disseminated to communities of interest?1. We published papers in top venues. 2. The PIs gave talks around the country and the world, including PI Chowdhary's talks at King Abdullah University of Science and Technology (KAUST), and several other talks. 3. We published videos on youtube and other venues. 4. News articles were published by others based on our work, for which we provided quotes and materials 5. The software Elastica was made open source and published on this website:https://www.cosseratrods.org/software/elastica/. This software makes it easy for users to model soft and continuum robots and train control policies for them using popular reinforcenent learning Python implmentations. This open source software is available as PyElastica on the above website for the use of the community. What do you plan to do during the next reporting period to accomplish the goals?We will continue our technical work as follows: 1. We will further develop the visual servoing system in the presence of obstacles, andintegrate the visual feedback system on our robots, so that the robot can automatically detect berries. 2. We will begin evaluation of the "full stack" system, tying together the perception, planning, and control loops to build a complete prototype.
Impacts What was accomplished under these goals?
1. We matured our hybrid arm robots: Our prototype arm from last year was installed on a TerraSentia 2018 robot with newly designed arm coverings. The arm includes a traditional hard component as well as a soft extender. The goal of this hybrid arm is to mitigate the issues of high elasticity of soft arms. This approach combines the benefit of both soft and rigid link manipulators. This has led to a more advanced and field deployment ready arm that we intend to use this year in berry fields. 2. We made major advances in using visual servoing for approaching berries using cameras on the robot. In our earlier work, the position of the arm and the berry was provided to the robot control system through magnetic sensors. This was a good place to start, but a limiting assumption in real-world fields. This year we broke through this challenge by developing visually guided systems for the control of the robot arm. The system utilizes camera on the tip of the robot and a neural network to determine the pose difference between a desired image and the current image seen by the robot. This work is described in our R-AL + RoboSoft 2022 paper (Kamtikar et al.). The work is currently limited to structured settings, but we are working on extending it to field settings. 3. We created an architecture and workflow that utilizes two cameras, one on the robot body, and one at the tip of the arm (as discussed in #2) to control the robot end-effector pose. This will enable the robot to drive the end effector towards the berry by first seeing it from the camera on the base of the robot, and then approaching it using visual servoing. This architecture will be trained with real data using part-affinity fields to determine optimal grasping pose for the end effector. 4. We created models of the soft arm using Gazzola's methods and demonstrated that the predictions of the models match the actual arm movement. This is accepted for publication inICRA 2022. 5. We planted one hundred acres of cover-crop with under-canopy autonomous robots. These robots benefitted from the autonomy work considered in this project.
Publications
- Type:
Journal Articles
Status:
Published
Year Published:
2022
Citation:
Kamtikar, Shivani Kiran, Samhita Marri, Benjamin Thomas Walt, Naveen Kumar Uppalapati, Girish Krishnan and Girish Chowdhary. 2022. Visual Servoing for Pose Control of Soft Continuum Arm in a Structured Environment.�IEEE Robotics and Automation Letters (RA-L) and RoboSoft.
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2021
Citation:
Havens Aaron and Chowdhary G. 2021. Forced Variational Integrator Networks for Prediction and Control of Mechanical Systems, Learning for Decision and Control (L4DC), ETH Zurich, Switzerland, June 2021.
- Type:
Conference Papers and Presentations
Status:
Awaiting Publication
Year Published:
2022
Citation:
Kim, Seung Hyun, Heng-Sheng Chang, Chia-Hsien Shih, Naveen Kumar Uppalapati, Udit Halder, Girish Krishnan, Prashant G. Mehta and Mattia Gazzola. 2022. A Physics-Informed, Vision-Based Method to Reconstruct All Deformation Modes in Slender Bodies.�arXiv preprint arXiv:2109.08372�(To Appear in ICRA 2022).
|
Progress 12/15/19 to 12/14/20
Outputs Target Audience:The following categories of target audiences were reached: Researchers: This includes graduate students, undergraduatestudents, faculty, and postdoctoral associates in the performing team. It also includes other researchers who are not a part of this project but in the community, with whom we communicated through papers and presentations.These researchers were from diverse fields of agriculture, computer sciecne, and engineering. Farmers: We interacted with a number of farmers to discuss with them technologies and potentials of mechanical weeding. Press: We interacted with several journalist to help them communicate the potential of mechanical weeding with small robots Stakeholders: We communicated with government and industry stakeholders about the potential of mechanical weeding agbots. This includes startup companies, more established companies, program managers in goverment funding organizations, and politicians. Students: We incorporated material from this course into various undergraduate and graduate courses. We also invited and communicated with K-12 students who were interested in learning more about agriculture and the role of technology in agriculture. Changes/Problems:
Nothing Reported
What opportunities for training and professional development has the project provided? 1. Graduate students were mentored by the PIs; they often recieved instruction and advice from multiple PIs on the project. 2. Undergraduate students were mentored by PIs and graduate students. 3. Postdoc on the project recieved mentoring and also advised graduate students. 4. Material for the ABE 426Principles of Mobile Robotics course was updated based on some results from this work. ABE 424 was cross listed with ECE 426, now being taught across multiple department students 5. Graduate students and postdocs were provided with opportunities to present technical talks and write technical papers, and they were also provided with regular feedback. How have the results been disseminated to communities of interest? 1. We published papers in top venues. 2. The PIs gave talks around the country and the world, including PI Chowdhary's Robotics Institute seminar at CMU, MIT LIDS talk, and several talks in agricultural institutes around the world, including at the University of Agricultural Sciences Dharwad. 3. We published videos on youtube and other venues. 4. News articles were published by others based on our work, for which we provided quotes and materials What do you plan to do during the next reporting period to accomplish the goals?We will continue our technical work as follows: 1. We will integrate the visual feedback system on our robots, so that the robot can automatically detect berries. 2. We will include a visual servoing component on the robot, so that the robot can do final movement updates in view of the berries. 3. We will begin evaluation of the "full stack" system, tying together the perception, planning, and control loops to build a complete prototype. 4. This work will be augmented with improved simulation models which will drive design synthesis for future soft arms.
Impacts What was accomplished under these goals?
We achieved the following critical progress points: 1. We developed two new robots with prototypehybrid arms: Our prototype arm from last year was installed on multiple robots. Thearmincludes a traditional hard component as well as a soft extender. The goal of this hybrid arm is to mitigate the issues of high elasticity of soft arms. This approach combines the benefit of both soft and rigid link manipulators. 2. We developed and validated newreinforcement learning-based algorithm for our hybrid hard-soft arm and used it to pick berries. 3. We created a prototype algorithm for detecting berries using machine learning for machine vision. This algorithm uses RGB-D cameras to find the berry using mask RCNN (or bounding box based method) and uses the mask to also remove issues associated with incident light on the sensor. This algorithm is now being improved to work in real-time on our robots. 4. We created models of the soft arm using Gazzola's methods and are using it to train reinforcement learning controllers. 5. The autonomy of the TerraSentia robot was validated in significant field trials, including over 50 KM with LIDAR based autonomy, and over 25 KM with machine vision based autonomy, all with Level 1 autonomy (hands off). 6. We introduced a framework for Levels of Autonomy for agricultural robots.
Publications
- Type:
Journal Articles
Status:
Accepted
Year Published:
2021
Citation:
Whitman, J.E., Maske, H., Kingravi, H.A. and Chowdhary, G. 2021. Evolving Gaussian Processes and Kernel Observers for Learning and Control in Spatiotemporally Varying Domains: With Applications in Agriculture, Weather Monitoring, and Fluid Dynamics. IEEE Control Systems Magazine, 41(1), 30-69.
- Type:
Journal Articles
Status:
Accepted
Year Published:
2021
Citation:
Noel Naughton, Jiarui Sun, Arman Tekinalp, Tejaswin Parthasarathy, Girish Chowdhary and Mattia Gazzola. 2021. Elastica: A Compliant Mechanics Environment for Soft Robotic Control. Robotics and Automation Letters.
- Type:
Conference Papers and Presentations
Status:
Accepted
Year Published:
2021
Citation:
Wang, Halder, Chang, Gazzola and Mehta. 2021. Optimal Control of a Soft CyberOctopus Arm. American Control Conference.
- Type:
Conference Papers and Presentations
Status:
Accepted
Year Published:
2020
Citation:
Chang, Halder, Shih, Tekinalp, Parthasarathy, Gribkova, Chowdhary, Gillette, Gazzola and Mehta. 2020. Energy Shaping Control of a Cyberoctopus Soft Arm. IEEE Conference on Decision and Control (CDC), 2020.
- Type:
Conference Papers and Presentations
Status:
Accepted
Year Published:
2020
Citation:
Satheeshbabu, S., Uppalapati, N.K., Fu, T. and Krishnan, G. 2020. Continuous Control of a Soft Continuum Arm Using Deep Reinforcement Learning. In 2020 3rd IEEE International Conference on Soft Robotics (RoboSoft) (pp. 497503).
- Type:
Journal Articles
Status:
Accepted
Year Published:
2020
Citation:
Uppalapati, N.K. and Krishnan, G. 2020. VaLeNS: Design of a Novel Variable Length Nested Soft Arm. IEEE Robotics and Automation Letters, 5(2), 11351142.
- Type:
Journal Articles
Status:
Accepted
Year Published:
2021
Citation:
Uppalapati, N.K. and Krishnan, G. 2021. Design and Modeling of Soft Continuum Manipulators Using Parallel Asymmetric Combination of Fiber-Reinforced Elastomers. Journal of Mechanisms and Robotics, 13(1). Https://doi.org/10.1115/1.4048223.
- Type:
Conference Papers and Presentations
Status:
Accepted
Year Published:
2020
Citation:
Uppalapati, N.K., Walt, B., Havens, A., Mahdian, A., Chowdhary, G. and Krishnan, G. 2020. A Berry Picking Robot With A Hybrid Soft-Rigid Arm: Design and Task Space Control. Proceedings of Robotics: Science and Systems, Corvalis, Oregon, USA.
|
Progress 12/15/18 to 12/14/19
Outputs Target Audience:Students, both at the univeristy including those who have participated in the projet, as well as K-12 students. Changes/Problems:
Nothing Reported
What opportunities for training and professional development has the project provided?1. Graduate students were mentored by the PIs;they often recieved instruction and advice from multiple PIs on the project. 2. Undergraduate students were mentored by PIs and graduate students. 3. Postdoc on the project recieved mentoring and also advised graduate students. 4. Material for the ABE 424 Principles of Mobile Robotics course was updated based on some results from this work 5. Graduate students and postdocs were provided with opportunities to present technical talks and write technical paprers, and they were also provided with regular feedback. How have the results been disseminated to communities of interest?1. We published papers in top venues. 2. The PIs gave talks around the country and the world. Including PI Chowdhary's Robotics Institute seminar at CMU, MIT LIDS talk, and several talks in agricultural institutes around the world, including at the University of Agricultural Sciences Dharwad. 3. We published videos on youtube and other venues. 4. News articles were published by others based on our work, for which we provided quotes and materials (OneZero and Google News results). What do you plan to do during the next reporting period to accomplish the goals?We will continue our technical work on the following fronts: 1. We will create and evaluate autonomous control algorithms for the hybrid control arms. We are now installing the arm on the TerraSentia 2018 robot and will be testing this in lab and the field. 2. We have identified a new magnetic sensor that can provide good local position tracking accuracy;we are purchasing this and will continue to evaluate. 3. We will integrate Gazzolla's simulation with the hybrid soft arm control design pipeline. 4. We expect to test the hybrid arm on the robot in berry fields this summer.
Impacts What was accomplished under these goals?
We achieved the following critical progress points: 1. We developed a new prototype hybrid arm: An arm that includes a traditional hard component as well as a soft extender. The goal of this hybrid arm is to mitigate the issues of high elasticity of soft arms. The mechanism is designed such that the soft extender with the manipulator can be brought very close to the target berry. It can then extend the soft arm if necessary. This approach combines the benefit of both soft and rigid link manipulators. 2. We developed a reinforcement learning based algorithm for our soft arm and evaluated it in simulation and on a lab mounted prototype. 3. We developed a simulation of soft arms for evaluating the effectiveness of reinforcement learning algorithms. 4. We evaluated the navigability and autonomy of the TerraSentia 2019 system in various polycultures and berry plantations.
Publications
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2019
Citation:
Satheeshbabu, S., Uppalapati, N.K., Chowdhary, G. and Krishnan, G. 2019. Open loop position control of soft continuum arm using deep reinforcement learning. In: 2019 International Conference on Robotics and Automation (ICRA) (pp. 5133-5139). IEEE.
- Type:
Journal Articles
Status:
Published
Year Published:
2019
Citation:
Chowdhary, Girish, et al. 2019. Soft robotics as an enabling technology for agroforestry practice and research. Sustainability 11.23 (2019): 6751.
- Type:
Journal Articles
Status:
Published
Year Published:
2019
Citation:
X. Zhang, F. Chan, T. Parthasarathy and M. Gazzola. 2019. Modeling and simulation of complex dynamic musculoskeletal architectures. Nature Communications, 10(1):112, 2019.
- Type:
Journal Articles
Status:
Accepted
Year Published:
2020
Citation:
N.K. Uppalapati and G. Krishnan. 2020. Design and modeling of soft continuum manipulators using parallel asymmetric combination of fiber reinforced elastomers. Accepted for publication in the ASME Journal of Mechanisms and Robotics.
- Type:
Conference Papers and Presentations
Status:
Under Review
Year Published:
2020
Citation:
S. Satheeshbabu, N.K. Uppalapati, T. Fu and G. Krishnan. 2020. Continuous control of a soft continuum arm using deep reinforcement learning. Submitted to the RoboSoft 2020 conference to be held in New Haven, CT.
- Type:
Journal Articles
Status:
Accepted
Year Published:
2020
Citation:
N.K. Uppalapati and G. Krishnan. 2020. VaLeNS: Design of a novel variable length nested soft arm. Accepted for publication in Robotics and Automation Letters.
- Type:
Conference Papers and Presentations
Status:
Under Review
Year Published:
2020
Citation:
Anay Pattanaik, Shuijing Liu, Zhenyi Tang, Gautham Bommannan and Girish Chowdhary. 2020. Robust deep reinforcement learning through state corruption based adversarial attacks. Submitted to ICRA.
|
|