Progress 06/15/21 to 06/14/22
Outputs
Target Audience:Due to covid restrictions in 2021, extension and outreach efforts were mostly limited to virtual event presentations. PI Shakoor presented and discussed the FieldDock project at the Ohio State Graduate Research Symposium hosted by the Horticulture and Crop Science Graduate Student Association (April, 2022). The project was also presented at the CROPs conference in Huntsville, Alabama in June, 2022. Both of these invited presentations were aimed towards project outreach and wereattended by a diversity of scientific trainees (graduate students and postdoctoral associates). In May, 2022, PI Shakoor hosted 3 young women high school students from St. Joseph's Academy in Saint Louis, MO for their 2022 Senior Service Projects. These students volunteered in the Shakoor lab for 80 hours each and gained quite a bit of hands-on experience in plant phenotyping, engineering and research. One young minority woman from the service project stayed on for a paid summer internship and is currently working in the lab on this project and related research. PI Shakoor is also currentlyhosting a summer REU female undergraduate student intern for 2022. Changes/Problems:The project was and continues to be significantly challenged by the global supply chain disruptions. Several critical components of the FieldDock hardware system were unavailable and continue to be very slow to procure. We have started sourcing alternative parts that we can use for the initial prototype version of the FieldDock, however these systems will not have optimal design-to-manufacture features. As part of our TTP goals, we are using design-to-manufacture principles to ensure that the FieldDock prototype built by the end of the project will be optimized for commercial development. Our engineers have been in contact with component manufacturers who expect these supply issues to ease somewhat in late 2022 but there will be shortages off and on into 2023 and 2024 for some components. What opportunities for training and professional development has the project provided?Through participation in several conferences and seminars in the field, this project has provided training and professional development for four postdoctoral researchers, one computational scientist, one research technician and several summer interns. How have the results been disseminated to communities of interest?Due to covid restrictions in 2021, extension and outreach efforts were mostly limited to virtual event presentations. PI Shakoor presented and discussed the FieldDock project at the Ohio State Graduate Research Symposium hosted by the Horticulture and Crop Science Graduate Student Association (April, 2022). The project was also presented at the CROPs conference in Huntsville, Alabama in June, 2022. Both of these invited presentations were aimed towards project outreach and were attended by a diversity of scientific trainees (graduate students and postdoctoral associates). In May, 2022, PI Shakoor hosted 3 young women high school students from St. Joseph's Academy in Saint Louis for their 2022 Senior Service Projects. These students volunteered in the Shakoor lab for 80 hours each and gained quite a bit of hands-on experience in plant phenotyping, engineering and research. One young minority woman from the service project stayed on for a paid summer internship and is currently working in the lab on this project and related research. PI Shakoor is also currently hosting a summer REU female undergraduate student intern for 2022.Results of this project have also been disseminated via presentations at conferences and webinars, including the Salk Institute Suberin Club Seminar Series (Mar, 2022) and CROPS Conference (June, 2022). What do you plan to do during the next reporting period to accomplish the goals?In year three of the project, we will aim to make significant progress in the integration of the FieldDock's edge computing device for GWAS/Modeling. The FieldDock prototype and minimally viable product (Version 1) will be nearly complete as well. We will continue FieldDock system optimization for autonomous UAV development and inter-platform data analysis, and the first autonomous FieldDock flight is planned for September, 2022. We will establish the data transfer and feature extraction pipeline to process raw data and generate derived data products, and we will generate calibrated and mosaicked data processing methods for the drone collected imagery.
Impacts
What was accomplished under these goals?
At the end of year 2, the project is well on its way to successfully developing a prototype 'smart farm' system, or FieldDock. The project is on track to conduct its first autonomous flight in a crop research field in September, 2022. If successful, the FieldDock project will validate the possibility of remote autonomous UAV platforms capable of capturing and processing large amounts of data daily/weekly facilitated entirely by solar power. This weekly capture of unbiased information will give tremendous insight into how crops respond to environmental conditions in real time throughout their life cycle. Researchers and crop breeders will have direct and daily access to remote field and crop conditions that could accelerate and improve global crop breeding strategies.The development of FieldDock's capabilities, namely UAV technical specifications and power needs, solar charge and power requirements, onboard edge computational hardware/software and wireless communications have been progressing steadily. UAV Garage Structure: Design/development of the UAV garage has also begun and the initial prototype will be made of 80/20 aluminum. This will allow the UAV garage to be easily adjusted without having to do a complete redesign in the event of a UAV model/size change later in development. The garage will open as a "drawer" with a scissor lift to elevate the UAV platform up above the UAV garage roof line to ensure the maximum amount of UAV landing surface area, while maximizing the efficiency and simplicity of all moving parts. Minimizing power needs for these moving parts is essential due to the system's solar power requirements.Development and integration of the UAV garage components such as the Edge computer, Drone induction charger, Hanger controls, Edge computer battery box, Drone induction charger battery box, and the Hanger controls battery box are ongoing and nearly ready for a full field test. Power distribution and management testing: Charge controller components crucial to operations are still on backorder until a later date. Delivery is expected July 2022.Lithium ion battery size/type and 100W solar panel selection have been identified and a custom mount has been developed and integrated to the roof of the UAV garage structure. Custom PCB's for solar charge control and overall system power distribution have been designed and are scheduled for delivery in July 2022. UAV communication integration with FieldDock software/hardware and testing: Telemetry collection software is complete. The FieldDock team is able to receive and record autopilot and mission telemetry via WiFi during test flights. UAV Payload Tests: Static drone tests are in process. These include testing payload power supply (+5V for mission computer, +12V for Micasense Altum camera), using a Wibotic charger to charge the drone battery, and optimizing the location of the payload CG (Center of gravity) relative to the drone CG. Sensor integration and testing: Due to supply chain shortages and time restrictions (weather conditions conducive to flight tests), the FieldDock engineers are focused on autonomous flight requirements in order to meet our project deadline for the first autonomous flight. Further integration of other sensors will happen in year 3 once other system components are brought online. Progress on the wireless sensor network is yet to be determined. Autonomous avionics development: Hardware in the loop (HITL) testing with the Aegis UAV has been ongoing and will continue up to and beyond the in-field autonomous flights when they begin in September 2022.Software development continues on the Mission computer and the MissionControl software, as well as the integration of system components such as the Nvidia Nano edge computer that will retrieve images from the Micasense multispectral camera, and the System Controller that will manage five separate daemons, BackendComms, RS485Comms, FlightOps, Camera, and SysLocalComms, and all communications with the FieldDock Backend. The primary focus here recently has been on MissionControl and a standalone telemetry app in order to prepare for a real flight test, completely under computer control by September 2022. Work also continues on BackendComms and the FieldDock Backend as we continue to add functionality to both ends to transfer software updates and download log files. Graphic User Interface: An updated, optimized version of the Graphic User Interface (GUI) for FieldDock has been completed and shared with the team. Overall design responsiveness has been optimized for multiple breakpoints on multiple platforms (smartphone, tablet, desktop, etc.). Additional items on multiple pages were built into the design that will act as a placeholder for future features. Completion of these features will largely depend on time and resource availability beyond the scope of the project priorities. Further, flight configuration parameters via the GUI are in progress. The configuration parameter settings are data that do not change very often, but may need to be configured for specific installations. Certain read-write parameters can be changed by authorized users, and these parameters are grouped into System Configuration, Measurement Configuration, Pre-Flight Operations Sequence, and Post-Flight Operations Sequence settings.
Publications
- Type:
Journal Articles
Status:
Published
Year Published:
2021
Citation:
Sagan, Vasit, et al. "Data-driven artificial intelligence for calibration of hyperspectral big data." IEEE Transactions on Geoscience and Remote Sensing 60 (2021): 1-20.
|
Progress 06/15/20 to 06/14/21
Outputs
Target Audience:Due to covid restrictions in 2020-2021, extension and outreach efforts were limited to virtual event presentations. In February 2021, the FieldDock project was presented at a Science in St. Louis Public Seminar series hosted by the Academy of Science Saint Louis, St. Louis County Libraries and Zonta Club of St. Louis's Girls Can STEM Initiative. The presentation was recorded for YouTube and shared with the Saint Louis Science Center and their YES Teens youth group. There were a total of 85 recorded participants, with representation from Saint Louis community residents and primary participation from middle and high school students. PI Shakoor also presented and discussed the FieldDock project at two student career panels including the NSF REU Graduate Student Career Panel (July, 2021) and BioBash: Exploring Scientific Career Paths (May, 2021), both hosted by the Donald Danforth Plant Science Center. The REU presentation was made to a group of approximately 30 undergraduate students pursuing a diversity of STEM careers. The BioBash event was attended by scientific trainees (graduate students, technicians and postdoctoral associates) at the Danforth Center and local agtech and biotech companies. Changes/Problems:?The project was and continues to be significantly challenged by the global supply chain disruptions. Several critical components of the FieldDock hardware system were unavailable and continue to be very difficult, if not impossible to source. For example, one of the critical processors (STM32L072RZT6 ARM32) and transceiver (THVD1450DGKR) of the main environmental sensor stack continue to be unavailable at the time of this report. We have started sourcing alternative parts that we can use for the initial prototype version of the FieldDock, however these systems will not have optimal design-to-manufacture features. As part of our TTP goals, we are using design-to-manufacture principles to ensure that the FieldDock prototype built by the end of the project will be optimized for commercial development. Our engineers have been in contact with component manufacturers who expect these supply issues to ease somewhat in late 2022 but there will be shortages off and on into 2023 and 2024 for some components. What opportunities for training and professional development has the project provided?Through participation in several conferences and seminars in the field, this project has provided training and professional development for three postdoctoral researchers and one research technician. How have the results been disseminated to communities of interest?As described in an earliersection, due to covid restrictions in 2020-2021, extension and outreach efforts were limited to virtual event presentations. In February 2021, the FieldDock project was presented at a Science in St. Louis Public Seminar series hosted by the Academy of Science Saint Louis, St. Louis County Libraries and Zonta Club of St. Louis's Girls Can STEM Initiative. The presentation was recorded for YouTube and shared with the Saint Louis Science Center and their YES Teens youth group. There were a total of 85 recorded participants, with representation from Saint Louis community residents and primary participation from middle and high school students. PI Shakoor also presented and discussed the FieldDock project at two student career panels including the NSF REU Graduate Student Career Panel (July, 2021) and BioBash: Exploring Scientific Career Paths (May, 2021), both hosted by the Donald Danforth Plant Science Center. The REU presentation was made to a group of approximately 30 undergraduate students pursuing a diversity of STEM careers. The BioBash event was attended by scientific trainees (graduate students, technicians and postdoctoral associates) at the Danforth Center and local agtech and biotech companies. What do you plan to do during the next reporting period to accomplish the goals?In year two of the project, we will engage in the discovery phase of minimal computing resources needed for the GWAS/modeling scripts and aim to make significant progress in the integration of the FieldDock's edge computing device for GWAS/Modeling. The FieldDock prototype and minimally viable product (Version 1) will be nearly complete as well. We will continue FieldDock system optimization for autonomous UAV development and inter-platform data analysis, and the first autonomous FieldDock flight is planned for September, 2022. We will establish the data transfer and feature extraction pipeline to process raw data and generate derived data products, and we will generate calibrated and mosaicked data processing methods for the drone collected imagery. Project Impact:Successful development of remote autonomous UAV platforms pose many challenges. The FieldDock system requires that enough solar power be captured and stored in order to regularly facilitate and power a fully charged UAV battery capable of ~20 minutes of flight with a reasonable sensor payload, wireless and wired communication of various system operations, wireless transfer of large datasets and imaging files, edge processing of large imaging files, image processing and downsampling of large image files, environmental sensor measurements at a high temporal resolution, operations of all electro-mechanical components, and cellular connectivity to the cloud. There are also many challenges associated with remote autonomous flight that the FieldDock project is addressing such as navigation, plot identification, and UAV landing/taking off safely aided by computer vision techniques, just to name a few. If successful, the FieldDock project will validate the possibility of remote autonomous UAV platforms capable of capturing and processing large amounts of data daily/weekly facilitated entirely by solar power. Remote infrastructure requirements for plant phenotyping equipment and the need for manual human-collected measurements will no longer be necessary. This daily capture of unbiased information will give tremendous insight into how crops respond to environmental conditions in real time throughout their life cycle. Researchers and crop breeders will have direct and daily access to remote field and crop conditions that could accelerate and improve global crop breeding strategies.
Impacts
What was accomplished under these goals?
At the end of year 1, the project is well on its way to successfully developing a prototype 'smart farm' system, or FieldDock. The project is on track to conduct its first autonomous flight in a crop research field in September, 2022. The FieldDock's onboard sensor suite, UAV technical specifications and power needs, solar charge and battery capacity requirements, onboard edge computational hardware and gateway hardware have been identified and are currently being sourced and/or developed. Sourcing of FieldDock onboard sensors: FieldDock mechanical and electrical infrastructure has been largely designed and will be developed to be modular. Modularity will allow the easy swapping of system component replacements as well as accommodate the integration of existing and/or any new 3rd party sensors for the foreseeable future. Universal connectors for the FieldDock system have been developed allowing custom sensor enclosures, 3rd party sensors and other ancillary system components to connect anywhere on the system bus. Temperature, humidity and air pressure sensors have been selected and will be housed in a small, custom waterproof enclosure that will allow for optimum ambient air measurements. Multiple instances of this "sensor module" will be connected to the FieldDock system in order to validate sensor accuracy including long term sensor drift as well as demonstrating reliability and resiliency to measurements crucial to flight command. This sensor module will also be used for the wireless sensor network, taking further advantage of the developed electronic architecture.Wind, rain, PAR and soil sensors have also been identified and acquired for integration. Design/development of aluminum hardware for FieldDock shell: The FieldDock physical mechanical structure will be made of aluminum and powder coated for long term weather and corrosion resistance. All enclosures are being designed to ensure no ingress will occur of water or other natural elements that could compromise internal electronics.The pole structure that all environmental sensors, battery boxes and the system controller/gateway mounts to has been fully designed and 5 prototype units have been manufactured. These units will be used to test form and fit of mechanical manufacturing specifications, water ingress in indoor/outdoor experiments, and the form and fit of the companion electrical components. Design/development of the UAV garage has also begun and the initial prototype will be made of 80/20 aluminum. This will allow the UAV garage to be easily adjusted without having to do a complete redesign in the event of a UAV model/size change later in development. Likely, the garage will open as a "drawer" with a scissor lift to elevate the UAV platform up above the UAV garage roof line to ensure minimum water ingress while maximizing the efficiency and simplicity of all moving parts. Minimizing power needs for these moving parts is essential due to the system's solar power requirements. Design/development of system electronics: Many iterations of various FieldDock electrical system PCB's have been developed and tested such as sensor module boards, cellular modem daughter boards (for compatibility with the system controller), battery pack and solar charge controller board, sensor connector board (junction board) and signal conditioning adapter modules for sensor plug-and-play functionality. Power distribution and management testing: charge controller components crucial to operations are still on backorder until a later date. Small quantities of these components acquired before the global shortage are being used to continue development and prototyping. However, the final FieldDock system power distribution system will require additional numbers of these components. Delivery is expected Q2 2023. Lithium ion battery size/type and solar panel selection have been identified and acquired for development. Custom PCB's for solar charge control and overall system power distribution have been designed and multiple prototype iterations have been acquired and tested. UAV communication integration with FieldDock software/hardware and testing: The FieldDock edge computer system will extract images from the MicaSense Altum camera via the camera's HTTP interface over WiFi. From here, the edge computer will perform the data reduction, image mosaicking, field plot auto-segmentation and vegetative index calculations. The UAV mission computer (Pixhawk) will receive its mission file from the system controller via WiFi as well through the Mavlink serial interface. It will control the phases of flight and transmit mission telemetry while in flight during the testing phase only. Sourcing of UAV hardware: Aegis (formally Emergent RC) Intense Eye V2 Quadcopter became the choice for our preliminary drone testing/prototyping due to its compatibility with WiBotic's wireless battery charging technology. While we test our SITL (software in the loop) and eventually HITL (hardware in the loop) with this UAV, we will outfit it with temporary 3D printed components that have the same size, shape, and weight of the actual components we intend to integrate for safety purposes. Identification of payload sensors: The UAV payload with a MicaSense multispectral camera has been identified and sourced based on weight, physical size, power consumption, data resolution, project research requirements and its ability to integrate with the custom avionics box that will likely house a Pixhawk. A downwelling radiation sensor, WiFi/cellular antenna, wireless charging antenna and battery will also be mounted to the UAV. Sensor integration and testing: To date, our ambient air sensors (temperature, humidity, air pressure) have been integrated and undergone periodic testing. Further integration of other sensors will happen at a later date once other system components are brought online. Initial testing of the MicaSense Altum has been carried out during preliminary UAV flights conducted by Co-PI Sagan's group at Saint Louis University. Autonomous avionics development: Development of autonomous flight control software is underway in addition to the FieldDock electrical, power and computational system to ensure firmware and systems communications are consistent, well-understood and validated throughout the development phase. SITL and HITL testing with the Aegis UAV have begun and will continue throughout year 2 of the FieldDock project. Graphic User Interface: A preliminary working version of the Graphic User Interface (GUI) for FieldDock has been completed. The GUI is a web application demonstrating up-to-date responsive design principles so that users can view the GUI on a variety of device platforms (desktop, tablet, smartphone, etc.). The GUI will be hosted within the backend (AWS) and use the Phoenix web framework and Elixir coding language. With this initial prototype, software engineers can begin best practices for connecting all FieldDock collected data in the cloud database to the GUI for user interaction. As new FieldDock derived data products are successfully sent and stored in the cloud, they will subsequently be connected to features within the GUI for user viewing and downloading. Version 2 of the GUI will begin development in year 2 based on feedback from software engineers and is expected to be completed in Q4 2021.
Publications
- Type:
Journal Articles
Status:
Published
Year Published:
2021
Citation:
Maimaitijiang, M., et al. "A fully automated and fast approach for canopy cover estimation using super high-resolution remote sensing imagery." ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences 3 (2021): 219-226.
|