Progress 03/01/19 to 10/31/24
Outputs
Target Audience:The Fish Farming Industry: Throughout this project, we have collaborated extensively with commercial fish farms, including Logan Hollow Fish Farm (now Otter Slough State Hatchery since 2023) in Illinois, Flowers Fish Farm in Mississippi, and Aqua Blue Cichlids Fish Farm in Palm Bay, Florida. These partnerships have enabled us to reach a broader base of fish farmers, providing them with access to advanced research and technology. Such collaborations are crucial for the practical application and dissemination of our research findings. The Academic Community: Our collaboration with Southern Illinois University (SIU) has been particularly fruitful, leveraging their expertise in aquaculture. This partnership not only enhances our research capabilities but also offers SIU students exposure to advanced aquaculture technologies. The proximity of SIU to Logan Hollow Fish Farm facilitates hands-on learning experiences and fosters a collaborative environment for innovation in aquaculture practices. Our team has been active in sharing our research findings with the broader academic community. We have published four journal papers and presented fourteen conference papers. Additionally, we have been awarded one U.S. patent and have one pending patent. We were also invited to present at a special NIFA session titled "A Primer of USDA NIFA Programs, Priorities, and Projects Supporting U.S. Aquaculture" during Aquaculture America 2023. Furthermore, we presented at the Networking and Information Technology Research and Development (NITRD) Program's Intelligent Robotics & Autonomous Systems (IRAS) IWG meeting on November 4th, 2024, with a presentation titled "Pioneering the Future of Aquaculture in the U.S. through AI and Robotics: The Journey from HAUCS to IREPA." These opportunities have allowed us to showcase our technological advancements and highlight the impact of our work. Technology Commercialization: We are actively engaging with several companies through the Florida Atlantic University (FAU) technology transfer office to explore the commercialization potential of our work. These efforts aim to translate our research into practical solutions that can benefit the aquaculture industry. By collaborating with industry partners, we strive to bring innovative technologies to market, enhancing the efficiency and sustainability of fish farming practices. Changes/Problems:COVID-19 (C19) and the secondary impact of the pandemic posed significant challenges.These included adhering to social distancing requirements in lab environments and field testing efforts. Our original plan was to start field testing at Logan Hollow fish farm in the summer of 2020. However, due to C19 restrictions, this did not happen. Unfortunately, Logan Hollow fish farm went out of business soon after. In the summer of 2021, we partnered with Flowers Fish farm and moved our equipment there. We set up the necessary infrastructure and conducted limited tests in late summer 2021, hoping to continue in the summer of 2022. However, Flowers Fish farm also went out of business in the spring of 2022. Eventually, we were able to use the Touch of Nature research aquaculture farm at Southern Illinois University for our testing until Logan Hollow became operational again as the Otter Slough State Hatchery. Another complicating factor was the time and resources required to develop a National Defense Authorization Act (NDAA) compliant waterproof drone. New legislation in Florida prevents state entities, including universities, from using drones with critical electronic components made in China. Additionally, the costs of both electronic and mechanical components needed for prototype development increased significantly. Long lead times for some components also posed challenges. Due to these issues, we requested a no-cost extension, which allowed us to achieve the project goals. What opportunities for training and professional development has the project provided?The HAUCS project offered significant opportunities for students and researchers at various academic levels. Over twenty undergraduate students from diverse fields such as mechanical engineering, electrical engineering, and aquaculture participated, gaining hands-on experience in mechanical design, software development, system testing, and field operations. These students also made valuable contributions to journal and conference papers. Notably, an SIU undergraduate aquaculture major, who interned at HBOI during the summer, played a crucial role in the successful 40-day system tests at Otter Slough State Hatchery in 2024. Seven graduate students with varied research interests contributed to platform development, sensor development, and AI algorithm development within the HAUCS project. They had the opportunity to present their work at international conferences, with one student winning the best student paper award at SPIE Ocean Sensing and Monitoring XVI in April 2024. This achievement sparked interest in the HAUCS concept at the Naval Air Warfare Center Aircraft Division (NAWCAD), leading to FAU receiving the prestigious NAWCAD Air Talent program award to sponsor a team of undergraduates for the "Environmental Sensing Drone Launched from an Unmanned Surface Vessel (USV)" project. Additionally, three postdoctoral researchers brought their expertise in path planning, fluid dynamics, AI algorithms, and robotic systems to the HAUCS project. This involvement allowed them to advance their research interests while addressing real-world problems. How have the results been disseminated to communities of interest?The Fish Farming Industry and Policy Government Bodies: We conducted numerous one-on-one training sessions with farmers to educate them on advanced technologies such as drones and machine learning. These efforts have significantly enhanced the acceptance and understanding of these technologies, which are poised to drive the future of fish farming. The project also provided opportunities for students, such as those majoring in Aquaculture at SIU, who may become the future workforce in the industry. On November 4th, 2024, we presented at the Networking and Information Technology Research and Development (NITRD) Program's Intelligent Robotics & Autonomous Systems (IRAS) IWG meeting. Our presentation, titled "Pioneering the Future of Aquaculture in the U.S. through AI and Robotics: The Journey from HAUCS to IREPA," highlighted our advancements. The Academic Community: As a result of this project, we published four peer-reviewed journal papers and presented over ten conference papers. Notably, a conference paper titled "Waterproof UAVs for Sensing in Coastal Zones and Estuaries," presented by student Will Fairman at SPIE Defense + Commercial Sensing 2024, won the best student paper award. We have been awarded a US patent (Ouyang B.; Wills P. S.; Hallstrom J.; Su T.-C.; Riche M.; Reiff P.; Tang Y. "Hybrid aerial/underwater robotics system for scalable and adaptable maintenance of aquaculture fish farms," US Patent 11,150,658, 10/19/2021) and have one pending patent. Funding Opportunities: This project has inspired several new proposals: USDA-NIFA, through NFS Cyber Physical Systems: "Intelligent Resource Efficient Pond Aquaculture (IREPA): Cyber Physical System to Improve Fish Farm Productivity in the U.S.," $1.2M, PI: Ouyang, Co-PIs: Wills, Tang, Su, 12/1/2023 - 11/30/2026. Naval Warfare Center Aircraft Division (NAWCAD), Physically-constrained CycleGAN Training Data Generator (PCGAN-TAG) to Support Deep Learning-based Underwater LiDAR Enhancement, $200k, PI: Ouyang, 3/1/2025 - 2/28/2027. Naval Warfare Center Aircraft Division (NAWCAD) Air Talent program, "Environmental Sensing Drone Launched from an Unmanned Surface Vessel (USV)," $15k, PI: Ouyang, 6/1/2024 - 5/31/2025. General public: We produced two YouTube videos on the HAUCS project "Journey of HAUCS": (5min): https://youtu.be/TsAHDJLZQkQ (1min): https://youtu.be/PLpPiapTXPo What do you plan to do during the next reporting period to accomplish the goals?
Nothing Reported
Impacts
What was accomplished under these goals?
The project team, comprising researchers from diverse backgrounds in electronic engineering, mechanical engineering, machine learning, and aquaculture, proudly announces the successful development of the Hybrid Aerial/Underwater Robotic System (HAUCS) framework--a mission we set out to achieve at the start of the project. HAUCS aims to revolutionize pond fish farm operations by implementing an Internet of Things (IoT) ecosystem. The primary focus of the HAUCS project is to enhance water quality monitoring. A central concept in HAUCS is the adoption of cost-effective mobile robotic sensing platforms to monitor key environmental metrics, addressing the deficiencies in current pond water quality monitoring. These mobile platforms enable HAUCS to adapt seamlessly to farms of varying scales. A critical aspect of the HAUCS mobile robotic platform is its platform-neutral deployable sensing module. This module is an edge device with a topside and payload subsystem interconnected via a winch. The two subsystems communicate wirelessly via a short-range link (e.g., Bluetooth), while the edge (topside) connects to the farm fog server via a long-range link such as LoRA. In addition to the mobile sensing platform, another important component of HAUCS is a cloud-based digital twin. The digital twin integrates sensor data and platform information, allowing farm managers to visualize the entire farm's status in real-time. A data-driven machine learning prediction model within the digital twin provides the ability to foresee potential issues on the farm, giving managers sufficient time to react (e.g., when dissolved oxygen levels become critically low). Compared to other state-of-the-art pond water quality monitoring solutions, HAUCS offers the following advantages: Improved scalability: Capable of collaborative monitoring and decision-making on farms of varying scales. More accurate reporting of pond conditions: Capable of sampling multiple locations in the pond to monitor spatial and temporal variations. Mitigated biofouling: Avoids maintaining sensors in bio-productive water. Broad coverage: Novel sensing patterns can cover different areas of a large pond. From the project's inception, close collaboration with farms was essential. The system development effort encompassed four iterative phases: Design → Development → Lab Test → Farm Validation. This approach allowed innovative ideas to be vetted by farmers and aquaculture experts, whose inputs guided the development path forward. We collaborated with three farms: Otter Slough State Hatchery (formerly Logan Hollow Fish Farm) in Illinois, Flowers Fish Farm in Mississippi, and Aqua Blue Cichlids in Florida. Throughout this process, we received enthusiastic support from the pond farms and experienced firsthand the problem we aimed to solve--mitigating the labor-intensive and time-consuming water quality monitoring tasks on fish farms. For instance, Flowers Fish Farm had to halt operations due to a labor shortage when technicians left for a new car factory in the area. With support from Mr. Peter Reiff, manager of Otter Slough State Hatchery, we validated the sensors, platform, communication infrastructure, and overall system. A core design consideration was the principle of "platform neutrality." This was motivated by the need for farms to continue using existing platforms such as trucks or all-terrain vehicles in daily operations. Our sensor design can seamlessly adapt to these platforms and integrate them into the digital twin. This also benefited sensor design, as we could test the sensors on trucks at the farm before incorporating them onto the drone. Through this process, we significantly reduced the sensor's Size, Weight, and Power (SWaP). Similarly, our drone designs improved in reliability, endurance, and performance. Initially, we evaluated commercially available drones, which generally used Chinese-made flight controllers and other critical electronic components. In 2022, the State of Florida passed legislation requiring all drones to comply with the National Defense Acquisition Act (NDAA), which prohibits critical electronic components made in China. The team viewed this as an opportunity rather than an obstacle. As a result, my lab--Systems and Imaging Labs (SAIL)--became the first lab at FAU to develop NDAA-compliant drones. This allowed us greater access to different drone controls and configuration parameters, enabling us to develop multiple drones. Notably, a novel waterproof coaxial drone offered great potential in flight time and functionalities. It is also worth noting that one crucial enabler in the design work is the adoption ofadditive manufacturing in both the platform development and sensor design. Another desirable outcome was that with NDAA-compliant electronics and software, we could access sensor data and drone status and upload the data to the cloud in real-time. This led to the implementation of a cloud-based digital twin, which farm managers and technicians at Otter Slough State Hatchery greatly appreciated. As mentioned earlier, validating the development on farms was a crucial aspect of our effort. While earlier tests focused on components (e.g., sensors, platforms, web portals), in the summer of 2024, we successfully conducted a full test of the ecosystem for an extended period of 40 days at Otter Slough State Hatchery, in collaboration with SIU. The system operated in a real operational environment and endured various challenging weather conditions. In summary, we are grateful for the support from USDA-NIFA, particularly Dr. Steve Thomson, for their unwavering support of our project. It was truly rewarding to see the concept we envisioned become operational. This project also laid the foundation for developing a full cyber-physical system (CPS) for pond aquaculture farms.
Publications
- Type:
Peer Reviewed Journal Articles
Status:
Published
Year Published:
2024
Citation:
Kunapinun, A., Fairman, W., Wills, P. S., Hanisak, D., & Ouyang, B. (2024). Advancing Seaweed Cultivation: Integrating
Physics Constraint in Machine Learning for Enhanced Biomass Forecasting in IMTA Systems. Remote Sensing, 16(23),
4418. https://doi.org/10.3390/rs16234418
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2024
Citation:
William Fairman, Paul S. Wills, Luis Vila, Bing Ouyang, "Waterproof UAVs for sensing in coastal zones and estuaries,"
Proc. SPIE 13061, Ocean Sensing and Monitoring XVI, 1306105 (6 June 2024);
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2024
Citation:
Alisa Kunapinun, William Fairman, Paul S. Wills, Dennis Hanisak, Shagundeep Singh, Bing Ouyang, "Advancing seaweed
cultivation analytics: integrating polynomial regression models with IoT for optimized harvesting in an integrated
multitrophic aquaculture (IMTA) system," Proc. SPIE 13036, Big Data VI: Learning, Analytics, and Applications, 130360B
(10 June 2024);
- Type:
Conference Papers and Presentations
Status:
Accepted
Year Published:
2025
Citation:
Bing Ouyang, Paul Wills, Jason Hallstrom, Yufei Tang, Tsung-Chow Su and James Garvey, "Innovating U.S. Aquaculture
with AI and Robotics: The Journey from HAUCS to IREPA," AQ2025, March 7, 2025.
- Type:
Conference Papers and Presentations
Status:
Accepted
Year Published:
2025
Citation:
Alisa Kunapinun, William Fairman, Paul Wills, Bing Ouyang, "Integrating Bi-LSTM and physics constraint models for improved biomass and water quality prediction in the aquaculture systems," SPIE DCS25, April, 2025
|
Progress 03/01/23 to 02/29/24
Outputs
Target Audience:During this period of performance, our targeted audiences include: • The fish farming industry: We are excited to renew the collaboration with Logan Hollow Fish Farm which is now being established as a research facility by the state of Illinois. Therefore this collaboration will enable us to reach a broader fish farmer base. In addition, we also continue our collaboration with our research partners Aqua Blue Cichlids fish farm at Palm Bay, Florida. • The academic community: We continue our collaboration with Southern Illinois University. This allows us to leverage their expertise in aquaculture. Furthermore, since SIU and Logan Hollow are located in the same area, this provides the opportunity for SIU students to be exposed to advanced technology for the aquaculture farms. We presented two conference papers at Aquaculture America 2023 and a conference paper at SPIE Defense + Commercial Sensing 2023. In addition, we were invited to present at a special NIFA session in Aquaculture Americal 2023 to showcase technological advancement in Aquaculture: "A primer of USDA NIFA programs, priorities, and projects supporting U.S. aquaculture". We have two conference papers accepted for SPIE Defense + Commercial Sensing 2024. Technology commercialization: We continue to engage with several companies through the FAU technology transfer office about the commercialization potential of our work. Changes/Problems:
Nothing Reported
What opportunities for training and professional development has the project provided?The HAUCS project provided substantial opportunities for undergraduate, graduate students, and postdoctoral researchers. Last year, two undergraduate interns (2 minority students and one female student) participated in the HAUCS project. They gained experience in various aspects of the projects - mechanical design, software development, and system testing. Two graduate students with diverse research interests contribute to the HAUCS project in platform development, and AI algorithm development. One graduate student had the opportunity to present at international technical conferences. Postdoctoral researcher Dr. Alisa Kunapinun joined the team in May 2023 and was able to contribute her expertise in AI algorithms and robotic system development to the HAUCS project. operations. How have the results been disseminated to communities of interest? We are excited to renew the collaboration with Logan Hollow Fish Farm which is now being established as a research facility by the state of Illinois. Therefore this collaboration will enable us to reach a broader fish farmer base. In addition, we also continue our collaboration with our research partners Aqua Blue Cichlids fish farm at Palm Bay, Florida. We continue our collaboration with Southern Illinois University. This allows us to leverage their expertise in aquaculture. Furthermore, since SIU and Logan Hollow are located in the same area, this provides the opportunity for SIU students to be exposed to advanced technology for aquaculture farms. We presented two conference papers at Aquaculture America 2023 and a conference paper at SPIE Defense + Commercial Sensing 2023. In addition, we were invited to present at a special NIFA session in Aquaculture Americal 2023 to showcase technological advancement in Aquaculture: "A primer of USDA NIFA programs, priorities, and projects supporting U.S. aquaculture". We have two conference papers accepted for SPIE Defense + Commercial Sensing 2024. What do you plan to do during the next reporting period to accomplish the goals?The primary goal is to conduct extensive field tests at Logan Hollow Fish Farm in the summer of 2024. This operation will allow us to validate the robotic system design in the real field operational environment. The field tests will also provide data to support the AI algorithm development. To facilitate the field test, we have converted commercially available waterproof drones to be NDAA-complaint. We also will evaluate our coaxial drone during the field tests. We have been collaborating closely with SIU. The HBOI team will instruct the SIU students about the robotic system developed at FAU. SIU students will run the robotic system during the day-to-day operations on Logan Hollow during the tests.
Impacts
What was accomplished under these goals?
Last year, we continued to make good progress in multiple aspects of the project. We re-establish collaborations with Logan Hollow Fish Farm, which has become a research facility for the state of Illinois. One of our strong beliefs from the start of the project is that we need to have close engagement with the fish farmers. Therefore we are very excited about this new opportunity. As part of the effort, we redesigned the sensor payload to support truck-mounted operation on Logan Hollow. The new design significantly reduced the power consumption, improved the reliability of the communication link and extended the communication range. The sensor design was validated on Logan Hollow in the summer of 2023. We developed three different types of NDAA-compliant drones. In addition to the conventional quadcopter and a vertical-take-off-landing (VTOL) plane, an innovative coaxial drone that is more compact and easier to waterproof has been developed by employing a novel print-in-place 3D-print technique. The web app supporting sensor data visualization and analysis has been developed and successfully tested on Logan Hollow.
Publications
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2023
Citation:
E. Weber, B. Ouyang, P. Wills, "Evaluating Environmental Variables that Influence Pond Dissolved Oxygen to Inform Prediction Model Development," Aquaculture America 2023.
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2023
Citation:
W. Fairman, P. S. Wills, J. E. Garvey, A. Davis, A. Karim, and B. Ouyang, "Development and Field Tests of Hybrid Aerial Underwater Robotic System (Haucs) Payload for Autonomous Data Collection," Aquaculture America 2023.
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2023
Citation:
W. Fairman, P. S. Wills, D. Hanisak, A. Karim, S. Singh, and B. Ouyang, "Deployment of the Pseudorandom Encoded light for Evaluating Biomass (PEEB) sensor in an Integrated Multi-Trophic Aquaculture (IMTA) system," Proc. SPIE 12522, Big Data V: Learning, Analytics, and Applications.
|
Progress 03/01/22 to 02/28/23
Outputs
Target Audience:During this period of performance, our targeted audiences include: The fish farming industry: While, unfortunately, Flowers Fish Farm went out of business due to a labor shortage, we continue to work with Ms. Kelly Flowers and provide information to their future buyers about continued collaboration on the farm. We sustained our collaboration with our research partners Aqua Blue Cichlids fish farm at Palm Bay, Florida. While the ponds of Aqua Blue Cichlids are smaller than those in a commercial fish farm, this facility allows us to evaluate our sensor payload design initially. The academic community: We established a collaboration with Southern Illinois University. This allows us to use their research aquaculture farm - Touch of Nature (ToN), as our field experiment site. We publish two journal papers: Den Ouden, C., Wills, P. S., Lopes, L., Sanderson, J. and Ouyang, B. (2022). Evolution of the Hybrid Aerial Underwater Robotic System (HAUCS) for Aquaculture: Sensor Payload and Extension Development. Vehicles. 4. 390-408.). Two of the co-authors are undergraduate students (Lopes and Sanderson). Davis, A., Wills, P.S., Garvey, J.E., Fairman, W., Karim, M.A., Ouyang, B. (2023) Developing and Field Testing Path Planning for Robotic Aquaculture Water Quality Monitoring.Appl. Sci. 13, 2805. We presented a conference paper at SPIE Defense + Commercial Sensing 2022; We have two conference papers accepted for Aquaculture America 2023 and one conference paper accepted for SPIE Defense + Commercial Sensing 2023. Technology commercialization: We continue to engage with several companies through the FAU technology transfer office about the commercialization potential of our work. Filed the full patent application: Ouyang, Wills, Fairman, Den Ouden, Lopes, Platform-Independent Mobile Environmental Sensing System, PCT/US22/53371, Dec. 19, 2022. General public: We produced two YouTube videos on the HAUCS project "Journey of HAUCS": (5min): https://youtu.be/TsAHDJLZQkQ (1min): https://youtu.be/PLpPiapTXPo Changes/Problems:In addition to the disruption due to COVID-19 to the laboratory work, the main obstacle we faced was the difficulty in achieving one of our main project objectives - testing the HAUCS framework for an extended period on a farm during the farm operational season (May - October). We initially planned this activity for the summer of 2022. However, we received the unfortunate news that Flowers Fish Farm will need to close due to a labor shortage. We established a collaboration with SIU to conduct tests at their ToN research farm. Another issue we must mitigate is that Florida has new legislation prohibiting the state entity from using drones with critical electronic components made in China. We are developing an NDAA complaint drone with funding from the HBOI foundation. But the drone won't be ready till the end of 2023. Therefore, we are applying for a no-cost extension and plan to conduct the final experimental study on a production farm in the summer of 2024. What opportunities for training and professional development has the project provided?The HAUCS project provided substantial opportunities for undergraduate, graduate students, and postdoctoral researchers. Last year, four undergraduate interns (2 minority students and one female student) participated in the HAUCS project. They gained experience in various aspects of the projects - mechanical design, software development, material science, and system testing. Undergraduate interns Lucas Lopes and Josh Sanderson are co-authors of a journal paper. Four graduate students with diverse research interests contribute to the HAUCS project in platform development, path planning algorithms, and field operations. Two graduate students are the lead authors of the two journal papers published last year. Two graduate students also had the opportunity to present at international technical conferences. Dr. Srijita Mukherjee, a postdoctoral researcher at HBOI/FAU, contributes her expertise in path planning in developing the operation and coordination of the HAUCS platforms during the sensing operations. How have the results been disseminated to communities of interest?During this period of performance, our targeted audiences include: The fish farming industry: While, unfortunately, Flowers Fish Farm went out of business due to a labor shortage, we continue to work with Ms. Kelly Flowers and provide information to their future buyers about continued collaboration on the farm. We sustained our collaboration with our research partners Aqua Blue Cichlids fish farm at Palm Bay, Florida. While the ponds of Aqua Blue Cichlids are smaller than those in a commercial fish farm, this facility allows us to evaluate our sensor payload design initially. The academic community: We established a collaboration with Southern Illinois University. This allows us to use their research aquaculture farm - Touch of Nature (ToN), as our field experiment site. We publish two journal papers: Den Ouden, C., Wills, P. S., Lopes, L., Sanderson, J. and Ouyang, B. (2022). Evolution of the Hybrid Aerial Underwater Robotic System (HAUCS) for Aquaculture: Sensor Payload and Extension Development. Vehicles. 4. 390-408.). Two of the co-authors are undergraduate students (Lopes and Sanderson). Davis, A., Wills, P.S., Garvey, J.E., Fairman, W., Karim, M.A., Ouyang, B. (2023) Developing and Field Testing Path Planning for Robotic Aquaculture Water Quality Monitoring.Appl. Sci.13, 2805. We presented a conference paper at SPIE Defense + Commercial Sensing 2022; We have two conference papers accepted for Aquaculture America 2023 and one conference paper accepted for SPIE Defense + Commercial Sensing 2023? General public: We produced two YouTube videos on the HAUCS project "Journey of HAUCS": (5min):https://youtu.be/TsAHDJLZQkQ (1min): https://youtu.be/PLpPiapTXPo What do you plan to do during the next reporting period to accomplish the goals?One main objective is to operate the drone-based data collection system for an extended period during the fish farm operational season (May - October). We initially planned this activity for the summer of 2022. While this plan was disrupted due to the unfortunate shutdown of Flowers Fishfarm, during the summer of 2022, we rebuilt the infrastructure on the SIU ToN research farm. However, because a new Florida legislation prohibits any state entity from using drones with critical electronic components made in China, HBOI is developing an NDAA complaint waterproof drone with HBOI foundation funding. However, such a drone will be ready at the end of 2023. We are applying for a no-cost extension and plan to conduct the final experimental study on a production farm in the summer of 2024. On the technology development front, there are several objectives we hope to achieve: Further improve the electronic and mechanical design of the payload. A new PCB board will be developed for both the topside and payload modules to reduce power consumption and improve the reliability of the communication link. Integrating payload with HBOI NDAA-complaint waterproof drone. The payload will be integrated with the NDAA complaint-waterproof drone currently under development with (HBOI foundation funding) and tested at the HBOI campus. Improve the DO prediction model and path planning algorithm. The team will work with SIU to conduct extensive tests at their ToN research farm. This will enable the team to validate the overall HAUCS framework: automated collecting data using multiple sensing platforms fed into the prediction model for real-time farm condition prediction. Analyzing the field data. The field data and experimental results will be analyzed at HBOI in preparation for the final report. During the field test, we will also test the automated drone charging.
Impacts
What was accomplished under these goals?
We made significant progress in multiple aspects of the project. One of our strong beliefs from the start of the project is that we need to have close engagement with the fish farmers. This motivated us to establish collaborations with Logan Hollow Fish Farm, Flowers Fish Farm, and Aqua Blue Cichlids. Unfortunately, after Logan Hollow Fish Farm stopped operation in 2020 due to Covid, Flowers Fish Farm went out of business in 2022 due to a labor shortage. On the other hand, this further demonstrated the importance of projects such as HAUCS to ensure the sustainability of aquaculture in the US. We have continued to assist Ms. Kelly Flowers, owner of the Flowers Fish Farm, with the future buyers of the farm to seek continued collaboration. We have continued our partnership with Blue Aqua Cichlids. While the ponds on this farm are smaller than those on a commercial fish farm, it allows us to conduct initial system testing since its location is very close to HBOI. Another critical progress is that we further improved the design of the platform-independent mobile sensing system. The durability and repeatability of the winch mechanical design were significantly improved. The electronic designs, particularly the payload module, were also improved to ensure easier replacement/repair in the field. While we, unfortunately, could not continue our experiments at Flowers Fish Farm, we established collaboration with Southern Illinois University (SIU). This allowed us to continue the experimental work at SIU's Touch of Nature (ToN) research farm. In this process, we received tremendous support from Dr. Jasme Garvey, Director of the Center for Fisheries, Aquaculture, And Aquatic Sciences at SIU. During the early summer of 2022, we re-established our infrastructure at SIU ToN farm. In August 2022, we tested drone-based data acquisition at ToN-SIU. The tests achieved three objectives: Validate the enhanced payload design (Fairman et al., AA23); In particular, demonstrate the ability to measure DO vertical distribution in the pond - a first-of-the-kind inDO monitoring practices. Developed and validated the drone path planning algorithm (published in Davis et al.. al. 2023).
Publications
- Type:
Journal Articles
Status:
Published
Year Published:
2022
Citation:
Den Ouden, C.J.; Wills, P.S.; Lopes, L.; Sanderson, J.; Ouyang, B. Evolution of the Hybrid Aerial Underwater Robotic System (HAUCS) for Aquaculture: Sensor Payload and Extension Development. Vehicles 2022, 4, 390-408. https://doi.org/10.3390/vehicles4020023
- Type:
Journal Articles
Status:
Published
Year Published:
2023
Citation:
Davis, A.; Wills, P.S.; Garvey, J.E.; Fairman, W.; Karim, M.A.; Ouyang, B. Developing and Field Testing Path Planning for Robotic Aquaculture Water Quality Monitoring. Appl. Sci. 2023, 13, 2805. https://doi.org/10.3390/app13052805
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2022
Citation:
Fairman, W., Wills, P.S., Hanisak, D., Ouyang, B. "Pseudorandom encoded-light for evaluating biomass (PEEB): a robust COTS macroalgal biomass sensor for the integrated multi-trophic aquaculture (IMTA) system," Proc. SPIE 12097, Big Data IV: Learning, Analytics, and Applications, 120970J (31 May 2022)
- Type:
Conference Papers and Presentations
Status:
Accepted
Year Published:
2023
Citation:
Fairman, W., Wills, P. S., Davis, A., Garvey, J., Karim, M. A.,�Ouyang, B., "Hybrid Aerial Underwater RobotiC System (HAUCS): Automated Water Quality Monitoring with UAVs," Aquaculture America 2023.
- Type:
Conference Papers and Presentations
Status:
Accepted
Year Published:
2023
Citation:
Ouyang, B., Wills, P. S., Hallstrom, J. O., Su, T-C, "Journey of HAUCS:
Robotic Systems for Scalable, Adaptable Maintenance of Pond Aquaculture Fish Farms," Aquaculture America 2023.
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2022
Citation:
Davis, A., Mukherjee, S., Wills, P. S., Ouyang, B., "Path planning algorithms for robotic aquaculture monitoring," Proc. SPIE 12097, Big Data IV: Learning, Analytics, and Applications, 120970K (31 May 2022) https://doi.org/10.1117/12.2618783
|
Progress 03/01/21 to 02/28/22
Outputs
Target Audience:? During this period of performance, our targeted audiences include: • The fish farming industry: We continued collaborating with our research partners, Flowers Fish Farm at Dext Missouri and Aqua Blue Cichlids fish farm at Palm Bay, Florida. There are two main goals in these collaborations. The first goal is to field test the instruments developed in the project. The second and equally important goal is to educate the fish farms about advanced technologies. During the process, we have received tremendous support from the farm owners, Ms. Kelly Flowers and Mr. Baron Kalmeyer. In particular, Ms. Kelly asked a local student working on her farm to participate in our experiments, which helped our research and gave the student first-hand experience with the robotic technology. • The academic community: We are very excited to publish our manuscript in the prestigious IEEE Internet of Things Journal (JCR'20 Impact Factor: 9.9936). We also submitted an invited paper to MDPI Vehicles. We presented a conference paper at SPIE Defense + Commercial Sensing 2021 (virtual presentation); We have two additional conference papers accepted for SPIE Defense + Commercial Sensing 2022. The academic community: We are engaging with several companies about the commercialization potential of our work. We are very excited that our patent on the HAUCS framework has been granted (US Patent 11,150,658); We just filed another patent application for our platform-neutral mobile sensing system. (US Patent application 63/311,937). • General public: We published on Facebook our field experiments at Flowers Fish Farm: https://www.facebook.com/watch/?v=1047807419353752 Changes/Problems:COVID-19 continued to bring some interruptions, in particular to laboratory work. Another challenge was the supply chain issue with electronic components. Nevertheless, we have learned how to work around these difficulties and recover from these setbacks. However, one challenge just emerged that would alter our plan. One focus is to demonstrate the drone-based data collection system on the collaboration farm during the fishing season (May - October). We initially planned this activity for the summer of 2022. However, we received the unfortunate news that Flowers Fish Farm will need to close due to a labor shortage. One good news is that Illinois plans to take over Logan Hollow farm as a research facility. Therefore, we will aim to do simulated test runs on the Logan Hollow Farm ponds in the summer of 2022. We will apply for a no-cost extension and plan to conduct the final experimental study on a production farm in the summer of 2023. What opportunities for training and professional development has the project provided?The HAUCS project provided substantial opportunities for undergraduate, graduate students, and postdoctoral researchers. Four undergraduate interns participated in the HAUCS project. They gained experience in various aspects of the projects - mechanical design, software development, aquaculture instruments deployment, and testing. In addition, there were two summer interns (one NSF REU intern and one HBOI intern) who worked on the project during the summer. Undergraduate interns Lucas Lopes and Josh Sanderson are co-authors in a journal paper under review. Three graduate students with diverse research interests contribute to the HAUCS project in platform development, path planning algorithms, and field operations. In total, nine undergraduate and graduate students contribute to the HAUCS project. This includes three minority students and one female student. Dr. Srijita Mukherjee, a postdoctoral researcher at HBOI/FAU, contributes her expertise in path planning in developing the operation and coordination of the HAUCS platforms during the sensing operations. How have the results been disseminated to communities of interest?During this period of performance, our targeted audiences include: • The fish farming industry: We continued collaborating with our research partners, Flowers Fish Farm at Dext Missouri and Aqua Blue Cichlids fish farm at Palm Bay, Florida. There are two main goals in these collaborations. The first goal is to field test the instruments developed in the project. The second and equally important goal is to educate the fish farms about advanced technologies. During the process, we have received tremendous support from the farm owners, Ms. Kelly Flowers and Mr. Baron Kalmeyer. In particular, Ms. Kelly asked a local student working on her farm to participate in our experiments, which helped our research and gave the student first-hand experience with the robotic technology. • The academic community: We are very excited to publish our manuscript in the prestigious IEEE Internet of Things Journal (JCR'20 Impact Factor: 9.9936). We also submitted an invited paper to MDPI Vehicles. We presented a conference paper at SPIE Defense + Commercial Sensing 2021 (virtual presentation); We have two additional conference papers accepted for SPIE Defense + Commercial Sensing 2022. The academic community: We are engaging with several companies about the commercialization potential of our work. We are very excited that our patent on the HAUCS framework has been granted (US Patent 11,150,658); We just filed another patent application for our platform-neutral mobile sensing system. (US Patent application 63/311,937). • General public: We published on Facebook our field experiments at Flowers Fish Farm: https://www.facebook.com/watch/?v=1047807419353752 What do you plan to do during the next reporting period to accomplish the goals?The focuses of our year-three work will focus on the following areas: • One main focus is to demonstrate the drone-based data collection system in the fishing season (May - October). We initially planned this activity for the summer of 2022. However, we received the unfortunate news that Flowers Fish Farm will need to close due to a labor shortage. One good news is that Illinois is planning to take over Logan Hollow farm as a research facility. Therefore, we will aim to do simulated test runs on the Logan Hollow Farm ponds in the summer of 2022. We will apply for a no-cost extension and plan to conduct the final experimental study on a production farm in the summer of 2023. • On the technology development front, there are several objectives we hope to achieve: Refining the sensing system design based on the issues identified during the tests in Oct. 2021. These include reducing the size, weight, and power consumption (i.e., SWaP), improving the control software. We also aim to test the integration of the Kirigami-based robotic extension with the drone. In the prediction model development, one main while continuing to improve the robustness of the LSTM model against the potential missing data values and improve accuracy. One important aspect is to include more daytime measurements such as pond color in the model. Another objective is to fully develop the fish farm phone app and integrate it with the prediction model. On the path planning side, we aim to implement the hybrid path planning algorithm on the actual drones to be used in the farm operation and test both at HBOI test ponds and Logan Hollow. We will further develop the chess-board charging solution, in particular its reliability. We also will evaluate other more efficient ways to operate/power the drone, such as a tethered drone.
Impacts
What was accomplished under these goals?
Despite the impact of COVID-19, our project was very productive last year. We have made tremendous progress in all aspects of the project. One continued success in this performance period is the continued engagement with the fish farm industry. We continued our close collaboration with our collaborative farms. We strongly believe that for our project to contribute to the fish farm industry, the participation of the farmers in the full development cycle is essential. In April, we migrated our infrastructure from Logan Hollow Fish Farm to Flowers Fish Farm. We collected data on the farm during the operational period between July and August. More importantly, this data collection process offered us the opportunity to educate the farmers on advanced technologies. Ms. Kelly Flowers even hired a student from the local university -- Southeast Missouri State University, to participate and support our research. This will be a model that we intend to explore more in the future. With the support from the farms, We conducted two successful data collection tests, one on Flowers Fish Farm between July and August of 2021 in conjunction with their operational DO and temperature data monitoring. One on Aqua Blue Cichlids from June 2021 to August 2021 on Aqua Blue Cichlids consisted of dissolved oxygen, water temperature, and weather data at a higher sampling rate (sample/15-minute). Another important milestone was developing a platform-independt mobile sensing system and demonstrating this system on Flowers Fishfarm using a waterproof drone - Splashdrone 3. The system was set up at our research collaborator - Flowers Fish Farm, in October 2021 for the initial field test of the drone-based sensing operation. With the consent of the farm owner MS Kelly and manager, Mr. Frampton, the HBOI team (Dr. Paul Wills) piloted the drone during the tests. Since the primary goal of the experiment was to validate the functionalities of the sensing system and identify any potential issues, the tests were conducted during day time when the farmworkers. During the tests, Dr. Wills acted as the pilot to command the drone takeoff, landing and flying the drone to the desired location for sensing operation. While the field tests validated the basic concept, one issue we identified during the tests was that since the payload was connected to the drone body through a string, it is more susceptible to ambient conditions. For example, strong side wind could induce a pendulum effect of the payload, which impacts the drone's flight stability. We aim to develop a deployable robotic extension to address this issue that can provide better controllability. To facilitate the integration of the machine learning model with the farm, we are also developing a mobile app to support pond aquaculture operations. This can be an information center of fishpond status and farm environmental conditions. To our knowledge, this is the first such app developed oriented toward aquaculture farm operations. This app will extract data from the cloud (AWS), which can then be integrated with a prediction model for pond condition forecast. Additional information about the farm equipment/status, such as the aerator usage, etc., can also be displayed in this app. For infrastructure development, we are developing an auto charging solution prototype. The prototype design employs a chessboard-style contact charger. During the charging, the drone lands on the pad. The charging controller determines which color pads are connected to the battery, and the controller routes +/- wires of the charger to the pads connected to the battery.
Publications
- Type:
Journal Articles
Status:
Published
Year Published:
2021
Citation:
Ouyang B., Wills S. P., Tang Y., Hallstrom J. O., Su T.-C., Namuduri K., Mukherjee S., Ignacio Rodriguez-Labra J., Li Y. and Den Ouden C. J., "Initial Development of the Hybrid Aerial Underwater Robotic System (HAUCS): Internet of Things (IoT) for Aquaculture Farms," in IEEE Internet of Things Journal, vol. 8, no. 18, pp. 14013-14027, 15 Sept.15, 2021, doi: 10.1109/JIOT.2021.3068571.
- Type:
Journal Articles
Status:
Under Review
Year Published:
2022
Citation:
Den Ouden C. J., Wills S. P., Lopes L., Sanderson J. and Ouyang B., "Evolution of the Hybrid Aerial Underwater Robotic System (HAUCS) for Aquaculture: Sensor Payload and Extension Development," Submitted to MDIP Vehicles.
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2021
Citation:
Guifang Tang, Yanjun Li, Paul S. Wills, Dennis Hanisak, Bing Ouyang, "Development of a macroalgal biomass sensor for an integrated multi-trophic aquaculture (IMTA) system,", Proc. SPIE 11730 Proceedings Volume 11730, Big Data III: Learning, Analytics, and Applications, 2021.
|
Progress 03/01/20 to 02/28/21
Outputs
Target Audience:During this period of performance, our targeted audiences include: • The fish farming industry: We continued the effort to engage with the fish farms. While our original collaborative partner farm - Logan Hollow farm, unfortunately, went out of business due to COVID-19 related losses, its owner Mr. Pete Reiff agreed to continue to be an unpaid consultant for this project. Pete has since helped us to find another partner farm - Flowers Fish Farm at Dexter, Mo. We have since transferred all the infrastructure installed at Logan Hollow to Flowers Fish Farm. We also engaged with another fish farm at Palm Bay, Florida - Aqua Blue Cichlids. With the enthusiastic support from the Owner Mr. Baron Kalmeyer, we acquired a high quality dataset from the ponds at Aqua Blue farm. • The academic community: We are very excited that our manuscript submitted to the prestigious IEEE Internet of Things Journal (JCR'20 Impact Factor: 9.9936) has been accepted. We presented conference papers at SPIE Defense + Commercial Sensing 2020 and 2020 IEEE 92nd Vehicular Technology Conference: VTC2020-Fall (both were virtual presentations); We have two additional conference papers accepted for SPIE Defense + Commercial Sensing 2021. • General public: Co-PI Wills discussed the HAUCS project in his Ocean Science Lecture talk on Jan. 13, 2021, titled "Fish Culture Research at HBOI the Last 10+ Years and the Future." (https://da-dk.facebook.com/HarborBranch/videos/fish-culture-research-at-hboi-the-last-10-years-and-the-future/490364625281833/). The article titled "Revolutionizing the World's Inefficiencies in Fish Farming" featuring the HAUCS concept published initially in "Owl Research & Innovation" was linked in USDA-NIFA eBulletin: Beet of April 2020 (https://content.govdelivery.com/accounts/USDANIFA/bulletins/283a04b). Changes/Problems:COVID-19 undoubtfully impacted the execution of our project. One most significant loss is that our collaboration partner Logan Hollow Fish Farm went out of business due to COVID-19. This impacted the data collection and sensor field-testing we originally planned to conduct on Logan Hollow in year-two. In the Spring and Summer of 2020, the lab work was significantly limited while the faculty and students learn to adjust to the social distancing requirement. We have since adapted protocols to allow lab work to be conducted more efficiently. COVID-19 also impacted hirings. The postdoc researcher Ms. Srijita Mukherjee will join HBOI SAIL in May 2021 since her graduation was delayed due to COVID-19. Another graduate student who originally planned to start at FAU in Fall 2020 will start in Fall 2021. Having said that, we have learned how to work around these difficulties and are recovering from these setbacks. We will double our effort in year-three to make up for all the lost time. What opportunities for training and professional development has the project provided?The HAUCS project provided substantial opportunities for undergraduate, graduate students, and postdoctoral researchers. A six-member undergraduate senior design team at FAU School of Engineering is designing a detachable payload under the faculty members' guidance on the HAUCS project. There are currently two other undergraduate interns contributing to the algorithm, software, and electronic development in the HAUCS project. In particular, undergraduate student Mr. Jose Ignacio Rodriguez-Labra from Univ. of West Michigan contributed to developing the LSTM dissolved oxygen prediction model. Jose also is a co-author in a journal paper accepted by the prestigious IEEE Journal of Internet of Things. Three graduate students with diverse research interests contribute to the HAUCS project in platform development, path planning algorithms, and field operations. In total, Eleven undergraduate and graduate students are contributing to the HAUCS project. This includes four minority students and three female students. Dr. Yanjun Li, a postdoctoral researcher at HBOI/FAU, continued to contribute his mechanical engineering experience and knowledge in the HAUCS robotic platform's mechanical design. How have the results been disseminated to communities of interest?• The fish farming industry: We continued the effort to engage with the fish farms. While our original collaborative partner farm - Logan Hollow farm, unfortunately, went out of business due to COVID-19 related losses, its owner Mr. Pete Reiff agreed to continue to be an unpaid consultant for this project. Pete has since helped us to find another partner farm - Flowers Fish Farm at Dexter, Mo. We have since transferred all the infrastructure installed at Logan Hollow to Flowers Fish Farm. We also engaged with another fish farm at Palm Bay, Florida - Aqua Blue Cichlids. With the enthusiastic support from the Owner Mr. Baron Kalmeyer, we acquired a high quality dataset from the ponds at Aqua Blue farm. • The academic community: We are very excited that our manuscript submitted to the prestigious IEEE Internet of Things Journal (JCR'20 Impact Factor: 9.9936) has been accepted. We presented conference papers at SPIE Defense + Commercial Sensing 2020 and 2020 IEEE 92nd Vehicular Technology Conference: VTC2020-Fall (both were virtual presentations); We have two additional conference papers accepted for SPIE Defense + Commercial Sensing 2021. • General public: Co-PI Wills discussed the HAUCS project in his Ocean Science Lecture talk on Jan. 13, 2021, titled "Fish Culture Research at HBOI the Last 10+ Years and the Future." (https://da-dk.facebook.com/HarborBranch/videos/fish-culture-research-at-hboi-the-last-10-years-and-the-future/490364625281833/). The article titled "Revolutionizing the World's Inefficiencies in Fish Farming" featuring the HAUCS concept published initially in "Owl Research & Innovation" was linked in USDA-NIFA eBulletin: Beet of April 2020 (https://content.govdelivery.com/accounts/USDANIFA/bulletins/283a04b). What do you plan to do during the next reporting period to accomplish the goals?• One main focus is to incorporate the automated data acquisition into the whole fishing season (May - October), initially planned for year-two. We have moved the infrastructure hardware from Logan Hollow to Flowers farm. We plan to travel to the farm in the Spring to set up the infrastructure. An HBOI graduate student will stay on the farm to support sensor testing and data collection during the summer. This will allow us to quickly address any issues and deficiencies of the HBOI sensor package and use the data to support prediction model development. • Another crucial area that we aim to make progress is to field-test our coaxial drone-based baseline HAUCS robotic platform and sensor. We plan to test the integration of the platform and the extended payload at Flowers Fish Farm this summer during this fishing season. We will field-test the initial detachable payload prototype in Fall 2021. • Path planning algorithm development will be another critical area we hope to make additional progress. While Ms. Srijita Mukherjee's graduation was delayed to May 2021 due to COVID-19, she has incorporated the path planning design in her Ph.D. dissertation research. This allows us to make up some loss of time in the path planning development. Her progress has been reflected in both the IEEE IoT journal paper and the upcoming SPIE CDS conference paper. After she joins the HBOI SAIL lab in June 2021, we are confident that Ms. Mukherjee will contribute even more to this vital aspect of the HAUCS framework. • In the prediction model design, the main focus will continue to improve the prediction reliability, particularly to extend the reliable prediction window. In this regard, we will attempt to validate the hybrid sensing mode - "seeding" the model with high temporal resolution data and collecting data at sparser frequency during the normal operation on Flowers Fish Farm during the summer. • For automatic charging, one main focus will be implementing distributed charging and integrated discharging to leverage the phone battery wireless charging technology. In addition, we will continue tracking alternative power sources include the solid-state battery. We also will continue this collaboration with Boston Engineering Co. to explore Aluminum powder-based drone energy sources.
Impacts
What was accomplished under these goals?
A significant success in this period of performance is the continued engagement with the fish farm industry. It is our strongest belief that for our project to contribute to the fish farm industry, the participation of the farmers in the full development cycle is essential. Due to the impact of COVID-19, Logan Hollow Fish Farm, unfortunately, went out of business. However, deeply impressed by our achievements in year one, Mr. Pete Reiff, the owner of Logan Hollow Fish Farm, agreed to continue our collaborations. We established collaboration with another farm with his help - Flowers Fish Farm at Dexter, Mo. We have since transferred all the infrastructure installed at Logan Hollow to Flowers Fish Farm to conduct system testing and farm data acquisition in summer 2021. We also engaged with a fish farm at Palm Bay, Florida - Aqua Blue Cichlids. With the enthusiastic support from the Owner Mr. Baron Kalmeyer, we acquired pond data from the ponds at Aqua Blue Cichlids, which consisted of dissolved oxygen, water temperature, and weather data at a higher sampling rate (sample/15-minute). Improving the machine-learning-based dissolved oxygen prediction model is another focus area. In this regard, the high-quality dataset from Aqua Blue is instrumental in simulating various sensing modes and tuning the hyperparameters of the Long Short-Term Memory (LSTM) based prediction model to improve the prediction accuracy. One important discovery is that the accuracy of a longer prediction window can be achieved by feeding imputed raw data acquired at a lower resolution to a prediction model trained at higher temporal resolution. For example, we achieved 95% prediction accuracy for 8-hr-lookahead against a network trained on sample/15-minute training data in our simulation. One possible operational scenario will be a hybrid sensor operation. In such an operation, stationary sensors may be placed in a pond for a short period to "seed" the model. Then mobile sensing platform can be employed in regular operation at a reduced sampling frequency (i.e., sample/one-hour or less) to reduce the power consumption. In HAUCS sensing platform development, we continued to develop the coaxial-rotor-copter drone-based design. The coaxial-rotor-copter has been adopted in the Mars Helicopter design for NASA's Mars 2020 Perseverance Rover mission. To support our development effort, we developed our own laboratory test environment that includes a motor thrust measurement station, a six-degree-of-freedom test flight stand, and a first-of-the-kind simulation tool that integrates the RealFlight flight simulator and the control simulation panel in the Ardupilot Mission Planner. This environment allows us to evaluate different design choices in the lab non-destructively. We have since successfully conducted test flights of our initial design and are currently working on refining the design to improve its stability and controllability. Another important aspect is the sensing payload that will enable the HAUCS platform to acquire underwater sensor data while staying in-air. We are currently evaluating two alternative designs. We integrated a payload extension design with the drone and tested it in the lab environment. This design will be able to measure the near-surface D.O. level in the pond. In addition, a datable payload is under development by a senior design team at FAU School of Engineering that can measure the D.O. and temperature vertical profile in the pond.A baseline path planning algorithm based on a hybrid control system has been developed to control and regulate the HAUCS framework. For drone automated charging, the team is investigating the concept of distributed-charging and integrated discharging. This may enable us to leverage the existing wireless charging technology developed for cellphone batteries.
Publications
- Type:
Journal Articles
Status:
Accepted
Year Published:
2021
Citation:
Ouyang B., Wills S. P., Tang Y., Hallstrom J. O., Su T.-C., Namuduri K., Mukherjee S., Ignacio Rodriguez-Labra J., Li Y. and Den Ouden C. J., Initial Development of the Hybrid Aerial Underwater Robotic System (HAUCS): Internet of Things (IoT) for Aquaculture Farms, IEEE Journal of Internet of Things, (in press).
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2020
Citation:
Ouyang B., Wills P., Li Y. and Den Ouden C., "Hybrid Aerial Underwater Robotic System (HAUCS): the Initial Instruments Development and Deployment," 2020 IEEE 92nd Vehicular Technology Conference (VTC2020-Fall), Victoria, BC, Canada, 2020, pp. 1-6, doi: 10.1109/VTC2020-Fall49728.2020.9348740.
- Type:
Conference Papers and Presentations
Status:
Accepted
Year Published:
2021
Citation:
Mukherjee S., Namuduri K., Wills P. S., Ouyang B., "Multi-Agent Systems (MAS) related data analytics in the Hybrid Aerial Underwater Robotic System (HAUCS)," SPIE CDS Big Data III: Learning, Analytics, and Applications, 2021.
- Type:
Conference Papers and Presentations
Status:
Accepted
Year Published:
2021
Citation:
Weber E., Michieletto L., Wills P. S., Ouyang B., "Environment data acquisition to support a machine-learning based prediction model for pond aquaculture farms," SPIE CDS Big Data III: Learning, Analytics, and Applications, 2021.
|
Progress 03/01/19 to 02/29/20
Outputs
Target Audience:During this period of performance, our targeted audiences include: The fish farming industry: Our initial deployment (a sensor package co-mounted on a farm truck to streaming data through LoRa network) at Logan Hollow Fish Farm was a huge success. It gave the farms first-hand evidence of how robotic technology can help them. As a result, the farm requested HBOI to install a second unit on another farm vehicle. The acceptance of the farmers is critical for us to be successful in deploy the fully automated solution in the future. The academic community: We have submitted a manuscript to the IEEE Internet of Things Journal, special issue on Internet of Food: Emerging Trend and Challenges. The paper is being invited for revision. We will be presenting at the annual NRI PI conference in Feb. 2020. Our conference paper has been accepted for SPIE Defense + Commercial Sensing (April 2020); We have submitted another abstract to the Marine Technology Society TechSurge conference (April 2020); General public: An article titled "Revolutionizing the World's Inefficiencies in Fish Farming" featuring the HAUCS concept is published in FAU Division of Research "Owl Research & Innovation" (https://issuu.com/fau4/docs/owl_research_and_innovation_spring_). HBOI summer intern Casey Den Ouden presented her work at the 2019 HBOI Summer Intern Presentation to an audience consisting of HBOI supporters and local Vero Beach residents. Changes/Problems:Our original plan was to have a postdoctoral researcher onboard in year-one to investigate the HAUCS platform path planning algorithms. Unfortunately, partly due to the delay in FAU receiving the funding (that was caused by a USDA database error), partly due to the competitive nature of the robotic technology industry, a candidate who initially accepted the postdoctoral researcher position at FAU decided to pursue an industrial position instead. As such, we did not have a postdoctoral researcher to work on path planning in year-one. Through our collaboration with Dr. Kamesh Namuduri at the University of North Texas, whose lab has focused on drone path planning, we were able to identify a new candidate Ms. Srijita Mukherjee, who is a Ph.D. student of Dr. Namuduri. Ms. Mukherjee will join FAU in August as a postdoctoral researcher to focus on path planning algorithm development. In fact, Ms. Srijita Mukherjee visited HBOI for one-month in the summer of 2019 to gain a better understanding of the HAUCS project. It is worth pointing out that since this module is relatively independent of other HAUCS modules, while it is unfortunate there is a delay in the research in this area, this delay should not have a severe impact on the progress of the HAUCS project. What opportunities for training and professional development has the project provided?During the summer of 2019, two NSF REU students participated in the HAUCS project. Ms. Kayleigh Taylor, a sophomore from Univ. of Maryland, gained experience in developing the LoRa communication network. Mr. Jose Ignacio Rodriguez-Labra from Univ. of West Michigan contributed to the development of the LSTM dissolved oxygen prediction model. During the summer of 2019, one HBOI summer intern, Ms. Casey Den Ouden, contributed to the HAUCS robotic platform development effort. From Fall 2019, Ms. Casey Den Ouden and Mr. Ethan Weber are pursuing their graduate study at Florida Atlantic University. Ms. Den Ouden is one of the recipients of the FAU President Fellowship award. Ms. Den Ouden will continue to investigate HAUCS robotic platform development. Mr. Weber will work closely with Logan Hollow Farm managers on the deployment of the HAUCS framework on the farm. Dr. Yanjun Li, who is a postdoctoral researcher at HBOI/FAU, was able to contribute his experience and knowledge in mechanical engineering in the mechanical design of the HAUCS robotic platform. During the summer of 2019, Ms. Srijita Mukherjee, a Ph.D. student from the University of North Texas, visited HBOI/FAU for one-month to investigate the path planning design for the HAUCS framework. Dr. Yanjun Li, Mr. Jose Ignacio Rodriguez-Labra, and Ms. Casey Den Ouden all contributed to a manuscript submitted to the IEEE Internet of Things Journal that is currently invited for revision. How have the results been disseminated to communities of interest?The fish farming industry: Our initial deployment (a sensor package co-mounted on a farm truck to streaming data through LoRa network) in Sept. 2019 at Logan Hollow Fish Farm was a great success. It gave the farms first-hand evidence of how robotic technology can help them. As a result, the farm requested HBOI to install a second unit on another farm vehicle. The acceptance of the farmers is critical for us to be successful in deploy the fully automated solution in the future. The academic community: We have submitted a manuscript to the IEEE Internet of Things Journal, special issue on Internet of Food: Emerging Trend and Challenges. The paper is being invited for revision. We will be presenting at the annual NRI PI conference in Feb. 2020. Our conference paper has been accepted for SPIE Defense + Commercial Sensing (April 2020); We have submitted another abstract to the Marine Technology Society TechSurge conference (April 2020); General public: An article titled "Revolutionizing the World's Inefficiencies in Fish Farming" featuring the HAUCS concept is published in FAU Division of Research "Owl Research & Innovation" (https://issuu.com/fau4/docs/owl_research_and_innovation_spring_). HBOI summer intern Casey Den Ouden presented her work at the 2019 HBOI Summer Intern Presentation to an audience consisting of HBOI supporters and local Vero Beach residents. What do you plan to do during the next reporting period to accomplish the goals?Built upon our success in the year-one effort, the focuses of our year-two work will focus on the following areas: We will further expand the collaboration with Logan Hollow Farm. We will incorporate the automated data acquisition into the whole fishing season (May - October). One HBOI graduate student (Ethan Weber) will work alongside the farm managers for an extended time during the summer. This will allow us to quickly address any issues and deficiencies of the HBOI sensor package. It should also be noted that this truck-mounted automated sensor data collection is a novel design that we are planning to apply for a new patent application. Another important area that we aim to make great progress is to field-test our coaxial drone-based baseline HAUCS robotic platform and sensor. The platform and payload extension are currently undergoing a test in the HBOI Systems and Imaging Lab (SAIL). The integrated system will then be tested in the SAIL lab and at HBOI aquaculture ponds. We will aim to have the HAUCS robotic platform field-tested at Logan Hollow Farm. We will make a great effort to do such a test during this fishing season. We aim to have an amphibious prototype design this year. One approach we are very keen to explore is the sail-powered on-water movement. Path planning algorithm development will be another important area we hope to make additional progress. Last year due to the unfortunate delay of funds released to FAU, the original postdoctoral researcher, we hoped to investigate this topic, decided to accept an industrial position instead. Fortunately, Ms. Srijita Mukherjee, a Ph.D. student at Dr. Namuduri's lab at the University of North Texas, will join the HBOI SAIL lab after her graduation in August. Dr. Namuduri's lab has been funded by multiple federal agencies to develop various drone path planning algorithms. Therefore, we hope Ms. Mukherjee will contribute to this important aspect of the HAUCS framework. Building upon our success in prediction model development in year-one, one important task for this module is to develop an effective user interface so that the farm managers and operators can receive near-real-time inputs from the prediction model. This again ties into the "collaborative" aspect of the HAUCS project. We will also attempt alternative implementations to explore further the spatial correlations among different ponds. The main goal will be to extend the reliable prediction window from 4-hours to 8-hours. For automated charging, one technology we will focus on during year-two study will be wireless battery charging that has experienced tremendous progress due to the demand in the mobile phone industry. We will also explore alternative power sources include the solid-state battery. We are also collaborating with Boston Engineering Co. to explore Aluminum powder-based drone energy sources. We will continue this collaboration in year-two.
Impacts
What was accomplished under these goals?
One major success in this period of performance is that truck-mounted automated dissolved oxygen (D.O.) acquisition systems were developed and deployed at our research partner - Logan Hollow Fish Farm. Using LoRa (LongRange) technology, the system can stream data from all the ponds on the farm. Between September 2019 and October 2019, the farm used this unit during their daily operation and successfully collected one-month-worth of data from all 70 ponds on the farm. After the first system was installed on the farm in September 2019 at the suggestion from the HBOI team, Logan Hollow farm requested HBOI to develop a second unit. This unit has been installed on a different vehicle at Logan Hollow Farm in Feburay 2020. We realized from earlier on that HAUCS will be a collaborative system, and it would be critical to have the acceptance and collaboration from the fish farm managers and operators. The success of our initial deployment effort, especially the enthusiastic support we received from the farm is therefore very exciting and encouraging. Using the data from the farm (both the handwritten historical data collected by the farm and the data from the HBOI automated sensors), a machine-learning-based dissolved oxygen prediction model is developed. In the model implementation, Long Short-Term Memory (LSTM) based recurrent neural network (RNN) is adopted. LSTM can provide better preservation of long-range dependencies in time series prediction. However, one challenge with LSTM is that it does not work well with incomplete data (i.e., gaps in the time series data). On the other hand, incomplete data or missing data can be expected to frequently occur in the field due to interferences from various sources. To address this issue, a novel solution is adopted to train a feedforward neural network (FNN) to learn the patterns of the ponds and to predict the missing values. The model has achieved excellent preliminary results: (an average success rate of 92% for all ponds and a minimum success rate of 78% for 4-hour-look-ahead prediction). Another area that the project team focused on is to evaluate different drones as the foundation to develop the amphibious HAUCS platform. Various designs were evaluated concerning the endurance and wind resistance - two critical features for the HAUCS platform. In addition to the classic quadcopter, we investigated two alternative drones - Ornithopter (flapping-wing drones) and Vertical Take-Off and Land (VTOL) drones that can transition between copter mode and plane mode. However, currently, most ornithopter designs aim at microform factors (i.e., ~10 cm wingspans) with the minimum payload. For the VTOL platforms, the endurance in the plane mode is substantially longer than the copters, even though transitioning from the copter mode to the plane mode consumes additional power. In recent months, a different platform - coaxial-rotor-copter drone has drawn our attention. Coaxial-rotor-copter has been adopted in the Mars Helicopter design for NASA's 2020 Mars Rover mission. In 2015, Ascent Aerosystems commercialized a similar design - the Sprite drone. Ascent has since moved into the area of industrial and military-grade drones with their rugged Spirit drone. The Sprite-like coaxial-rotor-copter design has some attractive features. The drone body is very compact and can be easily made water-tight. It is easy to integrate different types of payloads with the main body. Ascent Aerosystems spec claims the ability to operation wind speed up to 40mph and a flight time of >50minutes with dual batteries and minimum payload. For this reason, we have decided to adopt the coaxial-rotor drone as the foundation of the HAUCS robotic platform. Currently, we have developed a coaxial-rotor drone prototype and are conducting laboratory testing of this prototype. Another important aspect is the payload extension that will enable the HAUCS platform to acquire underwater sensor data while staying in-air. A prototype of the payload extension has also been developed and is being tested in the lab environment. We also investigate other aspects of the HAUCS framework, including automated platform charging/maintenance. This topic is closely related to the development of the platform, platform power source. At the current stage, we are focusing on keeping track of different charging solutions that have been proposed. Outdoor drone charging stations such as the Skysense Outdoor Charging Pad, the Edronic Drone Charging Station are available, albeit at high prices (>$15K). However, a new product from Heisha Co. - Heisha C200 with a price of $1,000 seems very promising. Another technology we are tracking is wireless battery charging that is advancing rapidly due to the high demand in the phone industry.
Publications
- Type:
Journal Articles
Status:
Under Review
Year Published:
2020
Citation:
Initial Development of the Hybrid Aerial Underwater Robotic System (HAUCS): Internet of Things (IoT) for Aquaculture Farms
- Type:
Conference Papers and Presentations
Status:
Accepted
Year Published:
2020
Citation:
Machine learning in aquaculture: an LSTM based dissolved oxygen
prediction model for the aquaculture fish farm
- Type:
Conference Papers and Presentations
Status:
Submitted
Year Published:
2020
Citation:
Hybrid Aerial Underwater Robotic System (HAUCS): Internet of Things (IoT) for Aquaculture Farms
|
|