Progress 09/01/23 to 08/31/24
Outputs
Target Audience:Our target audience for Year 3remains the same as in previous years of the project. The primary audience targeted was the project's Industry Advisory Board (IAB) that consists of industry partners including InnovaSea, BlueOcean Mariculture, OpenBlue, Ocean Era, E-Wave Technologies, and NEC Corporation. These are companies that are well-known and work intensively in the field of automated aquaculture, and we have continued engaging with them through regular IAB meetings. The second audience targeted is composed of both the wider aquaculture community and the robotics community. Our technologies are being developed for the aquaculture industry, so we will continue working to identify relevant aquaculture-related companies to join our IAB, along with exploring more opportunities to advance automated aquaculture technology. For the robotics community, aquaculture is a relatively untouched area. Making the community aware of the importance of automated aquaculture is one of our missions, so the robotics community remains included in our targeted audience. Changes/Problems:
Nothing Reported
What opportunities for training and professional development has the project provided?This project provided opportunities for the training of three PhD students during Year 3(Ivana Collado-Gonzalez, Justin Sitler, and Guoqing Zhang). Ivana and Justin were fully supported as Research Assistants by the project during the past year, and Guoqing was supported in part by the project, and also served as intellectual contributorto the project and its goals.The project has provided opportunities for all three students to have hands-on exposure to laboratory and field testing of underwater robots.The project also provided opportunities for the training of one master's student, two undergraduate students, and one high schoolstudent, who have served as part-time research contributors and have been presented with opportunities to learn about robotics research and to participate in our laboratory/field experiments and data-gathering. Specifically, during the summer of 2024 we supervised the two undergraduate students and high school student, whose tasks were inspired by the autonomous underwater robot navigation system in this project. The high school student worked on designing a feasible electrical and mechanical system for the integration of a stereo camera onto our current underwater robot platform, which could be used for close range sensing for manipulation tasks. One undergraduate student worked on the electrical and mechanical design, fabrication and assembly of a new ROV platform which will be used for testing and integrating new sensors and the robotic arms. The other undergraduate student worked on researching monocular camera underwater perception possibilities that could help the robot during navigation and manipulation tasks if a stereo camera is not available.During their summer experience, the students assisted with field experiments to collect relevant sonar and camera data using our AROV platform. These tests were conducted at the U.S. Merchant Marine Academy in Kings Point, NY. How have the results been disseminated to communities of interest?Our results have been disseminated to the communities of interest through the publication of two papers in refereed conference proceedings, and the presentation of those papers at the ICRA 2024 and UR 2024 robotics conferences. Additionally, two submitted journal articles were accepted for publication in the ASME Journal of Mechanisms and Robotics, and the IEEE Journal of Oceanic Engineering. One illustrative research videowas also published on YoutTube. The dissemination of these productswill hopefully contribute to the adoption of the algorithms and engineering designs developed in the course of this project. In addition, our results have been disseminated to the U.S. robotics research community through our team's participation in the 2024National Robotics Initiative P.I. Meeting. What do you plan to do during the next reporting period to accomplish the goals?Goals related to AROV Perception and Navigation Tasks. In the next reporting period, we intend to achieve the following: Writing and publishing our results on the integration of a monocular camera and an imaging sonar, and its performance in 3D reconstruction of underwater structures in turbid water environments. Integration of color camera information with our robot's stereo pair of imaging sonars, instead of only utilizing a single sonar,taking full advantage of our AROV's full stereo sonar 3D reconstruction setup that has been previously developed. Finishing the construction of a second AROV platform (capable of carrying a manipulator arm)while continuing our multi-modal perception integration work. Goals related to AROV Manipulation Tasks. In the next reporting period, we aim to achieve the following: Design, integration, and testing of a serial manipulator for dead fish removal. Experimental validation of perching configurations in a controlled laboratory environment (towing tank). Performance analysis of station keeping and power consumption while cleaning a fish pen fence, using (1) only vehicle thrusters, and (2) manipulators to hold the robot steady.
Impacts
What was accomplished under these goals?
Accomplishments under AROV Perception and Navigation Tasks: AROV design improvements: The Stevens team continued the construction of a new underwater AROV platform that will be used to combine the full stack of available sensing and localization capabilities with one or more robot arms. The team began electrical and mechanical designfor the integration of a stereo camera that would allow high-performance close range perception, important for manipulation tasks. We have made progress on Perception and Navigation Task (i), in the area of large-scale mapping using imaging sonars in stereo: The AROV's perception capabilities were enhanced by incorporating information (previously being discarded) from the AROV's on-board monocular color camera. A monocular camera, although highly sensitive to turbid water conditions and changes in ambient lighting, can aid in close range perception when sonars fail. Monocular cameras can be especially advantageous at detecting smaller objects at close range, which is essential for arm manipulation tasks. An algorithm was developed for merging monocular camera observations with information from one imaging sonar in order to perform 3D mapping and reconstruction. The depth information provided by the sonar is merged with the elevation information obtained by the monocular camera. This fusion of information allows us to expand the field of viewof our previous 3D stereo sonar approach when operating atclose range to obstacles. Laboratory and field tests were performed to validate the proposed approach. Experimental comparisons show an increase in 3D model recall (completeness) compared to other underwater 3D reconstruction techniques. Furthermore, our approach remains robust when operating in turbid water,while most other approaches fail. We have also made progress on Perception and Navigation Task (iii), in the area of AROV motion planning and transitions between fish pens: A manuscript was written and published describing a computationally efficient planning under uncertainty framework suitable for large-scale, feature-sparse environments such as fish farms. The strategy leverages SLAM graph and occupancy map data obtained from a prior exploration phase to create a virtual map, describing the uncertainty of each map cell using a multivariate Gaussian distribution. The virtual map is then used as a cost map in the planning phase, and performing belief propagation at each step is avoided. A receding horizon planning strategy is implemented, managing a goal-reaching and uncertainty-reduction tradeoff. Simulation experiments in a realistic underwater environment validate this approach. Experimental comparisons against a full belief propagation approach and a standard shortest-distance approach are conducted. Accomplishments under AROV Manipulation Tasks: We have made further progress on Manipulation Task (ii) Dead Fish Removal: Bimanual teleoperation simulation was achieved using Gazebo and the UUV Simulator package, with two haptic devices to control each AROV manipulator independently. Finally, the team has made progress on Manipulation Task (iii) Fish Pen Fence Cleaning: A kinematic stiffness model was derived for a perching underwater vehicle manipulator system(UVMS). This is necessary to predict displacements of the perched UVMS in the presence of disturbances, such as those generated by a cleaning tool or ocean currents. The stiffness model was derived using parallel robot theory, and validated in a physics-based simulator. Our results showed that the stiffness model was reliable for predicting steady-state displacement of the perched UVMS, and was able to do so with very low computation time. An optimization algorithm was derived using this stiffness model to plan optimal perching configurations in regard to rejecting disturbances. This algorithm is generalized for use with any arbitrary perching structure, and a model of the fish pen fence was used for testing the algorithm. We were able to successfully plan optimal perching configurations using the algorithm under a variety of conditions. Our results showed that the optimal configuration is highly dependent on the direction of the applied force, as well as the geometry of the perching structure.
Publications
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2024
Citation:
I. Collado-Gonzalez, J. McConnell, J. Wang, P. Szenher and B. Englot, "Real-Time Planning Under Uncertainty for AUVs Using Virtual Maps," Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Yokohama, Japan, May 2024, pp. 8334-8340.
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2024
Citation:
J. Sitler, S. Sowrirajan, B. Englot and L. Wang, "A Bimanual Teleoperation Framework for Light Duty Underwater Vehicle-Manipulator Systems," Proceedings of the 21st International Conference on Ubiquitous Robots (UR), New York, NY, USA, June 2024, pp. 683-690.
- Type:
Peer Reviewed Journal Articles
Status:
Accepted
Year Published:
2024
Citation:
J. Sitler and L. Wang, "Design, Kinematics, and Deployment of a Continuum Underwater Vehicle-Manipulator System," ASME Journal of Mechanisms and Robotics, Accepted, To Appear.
- Type:
Peer Reviewed Journal Articles
Status:
Accepted
Year Published:
2024
Citation:
J. McConnell, I. Collado-Gonzalez, P. Szenher and B. Englot, "Large-Scale Dense 3D Mapping Using Submaps Derived From Orthogonal Imaging Sonars," IEEE Journal of Oceanic Engineering, Accepted, To Appear.
|
Progress 09/01/22 to 08/31/23
Outputs
Target Audience:Our target audience for Year 2 remains the same as for Year 1.The primary audience targetedwas the project's Industry Advisory Board (IAB) that consists of industry partners including InnovaSea, BlueOcean Mariculture, OpenBlue, Ocean Era, E-Wave Technologies, and NEC Corporation. These are companies that are well-known and workintensively in the field of automated aquaculture, and we have continued engaging with them through regular IAB meetings. The second audience targeted is composed of boththe wider aquaculture community and the robotics community. Our technologies are being developed for the aquaculture industry, so we will continue working to identify relevant aquaculture-related companies to join our IAB, along with exploringmore opportunities to advance automated aquaculture technology. For the robotics community, aquaculture is a relativelyuntouched area. Making the community aware of the importance of automated aquaculture is one of our missions, so the robotics community remains included in our targeted audience. Changes/Problems:
Nothing Reported
What opportunities for training and professional development has the project provided?This project provided opportunities for the training of four PhD students during Year 2 (Ivana Collado-Gonzalez, John McConnell, Justin Sitler, and Guoqing Zhang), some of whom have been supported directly by the project, and others who have served as intellectual contributors to the project and its goals. The project has provided opportunities for all four students to have hands-on exposure to laboratory and field testing of underwater robots. The project also provided opportunities for the training of five undergraduate students and one master's student, who have served as part-time research contributors and have been presented with opportunities to learn about robotics research and to participate in our laboratory/field experiments and data-gathering. How have the results been disseminated to communities of interest?Our results have been disseminated to the communities of interest through the publication of one jourrnal article and the submission of two others, as well as the public release of design details, software tools, and illustrative research videos. This will hopefully contribute to the adoption of the algorithms and engineering designs developed in the course of this project. In addition, our results have been disseminated to the U.S. robotics research community through our team's participation in the 2023National Robotics Initiative P.I. Meeting, as well as through our biannual meetings with our team's Industry Advisory Board. What do you plan to do during the next reporting period to accomplish the goals?Goals related to AROV Perception and Navigation Tasks.In the next reporting period, we intend to achieve the following: Mechanical, electronic, and system integration of a stereo camera into the underwater AROV platform. Testing of stereo camera RGB imaging and depth recovery capabilities for enhanced perception in underwater environments. Integration of a new stereo camera that will provide RGB imagery and depth recovery for precise short range sensing during manipulation tasks. Fusionn of short-range stereo camera depth recovery with sonar sensing capabilities for improving 3D reconstruction and localization. Goals related to AROV Manipulation Tasks. In the next reporting period, we aim to achieve the following: Validation of our perching stiffness model via rigorous simulatedand physical experiments. Testing and integration of a serial manipulator with our AROV for perching applications relevant to fish pen cleaning. Bimanual teleoperation simulation using Gazebo and the UUV Simulator package, with two haptic devices to control each manipulator independently. Implementing a macro/micro framework for human-in-the-loop path planning (macro-level control) and manipulator trajectory control (micro-level control). This is intended to bridge the gap between perception/navigation and manipulation.
Impacts
What was accomplished under these goals?
Accomplishments under AROV Perception and Navigation Tasks: AROV design improvements: The Stevens team began the construction of a new underwater AROV platform that will be used to combine the full stack of available sensing and localization capabilities with one or more robot arms. We have made progress on Perception and Navigation Task (i), in the area of large-scale mapping using stereo imaging sonar: Stevens' algorithms for 3D reconstruction with a stereo pair of imaging sonars were successfully adapted to large-scale mapping scenarios using a novel submapping approach, and their compatibility with the AROV's simultaneous localization and mapping (SLAM) process was experimentally evaluated across several large-scale datasets.A manuscript was submitted to report these findings and experimental results (Product 3). We have also made progress on Perception and Navigation Task (iii), in the area of AROV motion planning and transitions between fish pens: The AROV's path following capabilities were enhanced by incorporating a pure pursuit guidance scheme. The underwater platform maneuverability was likewise improved by adding a velocity PID controller. Both the vehicle guidance and low level controller allow tuning for distinct environments and applications. In this case, both additions were tested using a realistic fish farm environment simulation. An algorithmic pipeline for underwater robots that uses exploration as a pre-processing step for planning under uncertainty to specified goals was created. The planning under uncertainty framework developed is computationally efficient and suitable for large-scale, feature-sparse environments. This strategy leverages SLAM graph and occupancy map data obtained from a prior exploration phase to create a virtual map, describing the uncertainty of each map cell using a multivariate Gaussian. The virtual map is then used as a costmap in the planning phase, and performing computationally costly online belief propagation is avoided. A receding horizon planning strategy was implemented, managing a goal-reaching and uncertainty-reduction tradeoff, while also performing trajectory tracking and incorporating perceptual feedback. We employed three stages: goal identification, utility computation, and action selection. Action selection and path planning are treated as tightly coupled problems. We consider two different action types, one type for lowering uncertainty (place-revisiting), and another for reaching the mission goal (shortest-distance). Place-revisiting actions look for poses that can lower uncertainty by revisiting locations the robot has previously observed, triggering SLAM loop closures. Shortest-distance actions represent locations that lay on the shortest feasible path to the goal. The utility of each action is evaluated using the uncertainty values in the virtual map that are encountered by the desired path to the candidate action. Simulation experiments in a realistic underwater environment were used to validate this approach and its application to underwater navigation with sonar. Experimental comparisons against a full belief propagation approach and a standard shortest-distance approach were conducted. Our results showed a decrease in uncertainty and pose error compared to a standard shortest-distance approach. Furthermore, our approach is much faster than full belief propagation, while still maintaining low uncertainty and pose error. Accomplishments under AROV Manipulation Tasks: We have made progress on Manipulation Task (i), Continuum manipulators: modular mechanism designs, low-cost sensors & state estimation: An underactuated waterproof gripper prototype was built and tested in shallow water. The servomotor is waterproofed via epoxy and an o-ring to seal the motor shaft. The gripper body is mostly 3D printed with PLA, which has low water absorption, and uses stainless steel fasteners to secure and connect components. Stainless steel extension springs give the underactuated fingers compliance to assist with grasping. We have also made progress on Manipulation Task (ii) Dead Fish Removal: Further improvements were made to the autonomous redundancy resolution algorithm which plans the AROV's trajectory. The primary task of the robot is to grasp an object at a known location. The secondary goals optimized by the redundancy resolution algorithm were updated to: (1) avoid joint limits in the manipulator; (2) prioritize manipulator motion when close to the goal and vehicle motion when far from the goal; (3) maintain an upright pose of the vehicle; (4) maintain a dexterity-maximizing manipulator pose when possible; (5) orient the robot's camera towards the object to improve visibility. Experimental testing of the robot involved autonomous execution of the redundancy resolution algorithm, and a manuscript was submitted to report these findings and experimental results (Product 2). Finally, the team has made progress on Manipulation Task (iii) Fish Pen Fence Cleaning: A low-cost prototype of a serial manipulator was produced. This manipulator has higher stiffness than the continuum manipulator, which may prove a valuable alternative design. The manipulator was also 3D printed using PLA. Perching analysis, applicable to scenarios where the AROV perches on a fish pen fence to perform cleaning operations, requires a stiffness model to predict robot deflection in the presence of external forces and disturbances. The kinematic stiffness model was derived using techniques borrowed from parallel robot analysis. Using this model, multiple perching configurations were tested to find the "strongest" perch over a variety of loading conditions.
Publications
- Type:
Journal Articles
Status:
Published
Year Published:
2022
Citation:
J. McConnell, I. Collado-Gonzalez, and B. Englot, "Perception for Underwater Robots," Current Robotics Reports, vol. 3, pp. 177186, December 2022.
- Type:
Journal Articles
Status:
Submitted
Year Published:
2023
Citation:
J. Sitler and L. Wang, "Design, Kinematics, and Deployment of a Continuum Underwater Vehicle-Manipulator System," Submitted to IEEE/ASME Transactions on Mechatronics, Under review.
- Type:
Journal Articles
Status:
Submitted
Year Published:
2023
Citation:
J. McConnell, I. Collado-Gonzalez, P. Szenher, and B. Englot, "Large-scale Dense 3D Mapping using Submaps derived from Orthogonal Imaging Sonars," Submitted to the IEEE Journal of Oceanic Engineering, Under review.
|
Progress 09/01/21 to 08/31/22
Outputs
Target Audience:Our target audience is the same as notedby our Lead P.I., Prof. Furukawa, at the University of Virginia: The primary audience targeted in the first year was the project's Industry Advisory Board (IAB) that consists of industry partners including InnovaSea, BlueOcean Mariculture, OpenBlue, Ocean Era, E-Wave Technologies, and NEC Corporation. These are the companies that are most known and working most intensively in the field of automated aquaculture. Therefore, it is most important that these companies remain as the IAB members. For this reason, we began to host biannual meetings with them. The second audience targeted in the first year was the other aquaculture community and the robotics community. Our technologies are being developed for the aquaculture industry, so we should have the IAB open to the other aquaculture-related companies and explore more opportunities to advance automated aquaculture technology. For the robotics community, aquaculture is relatively an untouched area. Making the community aware of the importance of automated aquaculture is one of our missions, so the robotics community is included in our targeted audience. Changes/Problems:
Nothing Reported
What opportunities for training and professional development has the project provided?This project has provided opportunities for the training of five PhD students (Ivana Collado-Gonzalez, John McConnell, Justin Sitler, Guoqing Zhang, and Qianwen Zhao), some of whom have been supported directly by the project, and others who have served as intellectual contributors to the project and its goals. The project has provided opportunities for all five students to have hands-on exposure to laboratory and field testing of underwater robots. The project also provided opportunities for the training of five undergraduate students and one master's student, who have served as part-time research contributors and have been presented with opportunities to learn about robotics research and to participate in our laboratory/field experiments and data-gathering. How have the results been disseminated to communities of interest?Our results have been disseminated to the communities of interest through the publication of one jourrnal article, the submission of a second journal article and a conference paper, as well asthe public release of design details, software tools, and relevant data on the web (via four separate websites detailing different aspects of the project). This will hopefully contribute to the adoption of the algorithms and engineering designs developed in the course of this project. In addition, our results have been disseminated to the U.S. robotics research community through our team's participation in the 2022 National Robotics Initiative P.I. Meeting, as well as through our biannual meetings with our team's Industry Advisory Board. What do you plan to do during the next reporting period to accomplish the goals?Goals related to AROV Perception and Navigation Tasks: In the next reporting period, for Perception and Navigation Task (i), we intend to complete our study of large-scale underwater mapping using stereo imaging sonar, and recommend a finalized system architecture and parameterization that offers reliable and accurate performance, to be documented in a journal article that is current in preparation. For Perception and Navigation Task (iii), we plan to complete the testing and refinement of autonomous path following capabilities, so that our AROV's custom-designed local planning module can be finalized. We also intend to complete and publish the details of the AROV's belief space planner for managing localization uncertainty while transitioning between fish pens. ForPerception and Navigation Task (ii), the team will make a determination of the size and shape of fiducial markers needed for successful localization and navigation along the external boundary of a fish pen. Goals related to AROV Manipulation Tasks: In the next reporting period, we aim to complete Manipulation Task (ii) Dead Fish Removal. The key lies in the success of field experiments for free-floating manipulation, and we believe that the proposed redundancy resolution algorithm is promising for achieving it. Another necessary technical task is integration of the gripper to the continuum manipulator. In addition, we plan to investigate the perching and climbing gait for the Manipulation Task (iii) Fish Pen Fence Cleaning. We plan to start by building the experimental apparatus that includes a mock fish pen and by testing the perching capability.
Impacts
What was accomplished under these goals?
Accomplishmentsunder AROV Perception and Navigation Tasks: AROV design improvements: The underwater robot design has been updated to contain a fiber optic gyroscope for achieving precise state estimation. The robot design has also been modified to be wider and shorter, ensuring more stable performance in the water. The new design has been tested successfully in tank and marina environments. We have made progress on Perception and Navigation Task (i), in the area oflarge-scale mapping using stereo imaging sonar: Stevens' algorithms for 3D reconstruction with a stereo pair of imaging sonars are being adapted to large-scale mapping scenarios, and their compatibility with the AROV's simultaneous localization and mapping (SLAM) process is being tested and evaluated. Successful results have been obtained in several large-scale underwater mapping scenarios. The algorithms have also been re-written in C++ for optimal performance and usage at high update rates. We have also made progress on Perception and Navigation Task (iii), in the area ofAROV motion planning and transitions between fish pens: A realistic simulation of the underwater fish pen environment was created for testing underwater perception and planning algorithms. New path following capabilities were developed for the underwater robot navigation stack and tested in simulation. Prior maps have been leveraged to aid belief space planning algorithms for exploration and planning tasks. AccomplishmentsunderAROV Manipulation Tasks: We have made progress on Manipulation Task(i),Continuum manipulators: modular mechanism designs, low-cost sensors & state estimation: For the underwater continuum manipulator hardware, we have completed several prototypes, and the design details have been published in the ASME Journal of Mechanisms and Robotics. The design and fabrication are detailed in this publication, as well as waterproofing, buoyancy, stiffness, and teleoperational control experiments to demonstrate the prototype's usability. An improved design was designed, constructed, and tested. This newer design includes a smaller actuation unit for ease of integration with an ROV and two continuum segments to allow for more dexterous manipulation. In preparing for future work on developing contact detection and force control capabilities for manipulation tasks, we have demonstrated an IMU-based force estimation method with the use of low-cost inertial sensors. The investigation and demonstration was carried out on a similarlightweight continuum manipulator design, which allowed the main continuum manipulator prototype to be used for water tank testing. We submitted a manuscript to report our findings, which is under review by theASME Journal of Mechanisms and Robotics. We carried out theoretical investigations on the state estimation problem of continuum manipulators. Specifically, we have investigated and formalized a model-based observer method for continuum robot state estimation (shape estimation). The formulation is documented in a manuscript that has been submitted to the American Control Conference and is under review. We have also made progress on Manipulation Task (ii) Dead Fish Removal: The continuum arm was mechanically and electronically integrated with the ROV platform. Control of the arm is carried out by an on-board computer, which can be remotely accessed via the same tether as the ROV. Teleoperational control was demonstrated in tandem with the ROV. Toward the autonomous free-floating manipulation task of picking up an object at a known location, we have formalized a redundancy resolution method.The redundancy resolution algorithm was created to control the entire 8 DoF system (4 DoF from the ROV, 4 DoF from the manipulator). This allows the robot to efficiently and autonomously plan its path for a given task while optimizing for certain secondary goals such as minimizing ROV motion for close-range tasks. In addition, autonomous control of the ROV and manipulator was tested. An acoustic short baseline (SBL) positioning system was used to locate the ROV for testing purposes. Position feedback was used to autonomously control the ROV and manipulator according to the path generated by the redundancy resolution algorithm. A manuscript is being written to report the results in the future. We have also completeda waterproof gripper design to be integrated into the modular underwater continuum manipulator. This design utilizes a standard servo motor that was waterproofed. A lead screw attached to the servo allows for strong and precise actuation. Testing and integration with the continuum arm have not been carried out yet.
Publications
- Type:
Journal Articles
Status:
Published
Year Published:
2022
Citation:
Sitler, J. L., and Wang, L. "A Modular Open-Source Continuum Manipulator for Underwater Remotely Operated Vehicles." ASME. J. Mechanisms Robotics.
- Type:
Journal Articles
Status:
Under Review
Year Published:
2022
Citation:
Zhang, G., Zhao, Q., and Wang, L. A Lightweight Modular Continuum Manipulator with IMU-based Force Estimation. Submitted to ASME. J. Mechanisms Robotics.
- Type:
Conference Papers and Presentations
Status:
Under Review
Year Published:
2022
Citation:
Zhang, G. and Wang, L. Shape Estimation of Continuum Robots via Modal Parameterization and Dual Extended Kalman Filter. Submitted to 2023 American Control Conference (ACC).
|
|