NRI: INT: COLLAB: Collaborative Autonomy and Safety for Teamed Human-Unmanned Aircraft Systems in Fast Evolving Wildfire Environment

NRI: INT: COLLAB: COLLABORATIVE AUTONOMY AND SAFETY FOR TEAMED HUMAN-UNMANNED AIRCRAFT SYSTEMS IN FAST EVOLVING WILDFIRE ENVIRONMENT

Sponsoring Institution

National Institute of Food and Agriculture

Project Status

COMPLETE

Funding Source

AFRI COMPETITIVE GRANT

Reporting Frequency

Annual

Accession No.

1017046

Grant No.

2019-67021-28993

Cumulative Award Amt.

$328,675.00

Proposal No.

2018-06102

Multistate No.

(N/A)

Project Start Date

Dec 1, 2018

Project End Date

Nov 30, 2023

Grant Year

2019

Program Code

[A7301]- National Robotics Initiative

Recipient Organization
UNIVERSITY OF MISSOURI
(N/A)
COLUMBIA,MO 65211

Performing Department
Mechnical and Aerospace Engineering

Non Technical Summary
Accurate predictions of wildfire spread are critical for effective wildfire management to support decision makings of fire managers and to ensure safety of firefighters. However, the lack of real time wildfire and wind data, both of which change in space and time, makes it difficult to achieve operational wildfire spread prediction. Unmanned Aircraft System (UAS) is emerging in many civilian applications and shows great potential in wildfire management. This project aims to develop and evaluate a collaborative human-UAS wildfire spread prediction and situational-awareness system for wildfire management. The UASs will work side-by-side with fire managers and ground firefighters to perform collaborative tasks. This new paradigm brings new research challenges from multiple aspects. First and foremost, the UASs must achieve sufficient autonomy in their mission so that they can autonomously collect the most useful information in dynamic wildfire environments. Besides wildfire sensing, the UASs also need to pay close attention to firefighters' safety by monitoring their vicinity. The second challenge is associated with effective teaming and collaboration between humans (fire managers and firefighters) and UASs. In particular, there is a need for humans to interact with and direct UASs' autonomy based on their domain knowledge and expert opinions for more effective wildfire management. To address these challenges, this project will includes four tasks: (1) fire sensing and wind estimation using a team of UASs to enable data-driven wildfire spread prediction, (2) UAS coordination and path planning algorithms governing UAS autonomy to sense dynamic wildfires while monitoring firefighters' safety risk, (3) teamed human-UAS collaboration, including human-directed autonomy and a human-UAS interaction interface to support human awareness of UAS operation, and (4) evaluation of the proposed research by flying a team of UASs over prescribed fires.This project has the potential to transform wildfire management by enabling operational wildfire spread prediction and situation awareness for firefighters through teamed human-UASs collaboration. Using UASs to sense fire characteristics and wind parameters will fill the critical gap of real time data collection and data assimilation for operational wildfire spread prediction. The multi-UAS autonomy algorithms allow UASs to effectively collect the most useful information about dynamic wildfires and to monitor the safety of firefighters and other people on the ground. The approach of human-directed autonomy supports humans in-the-loop to optionally direct UAS teams to certain locations and tasks for effective human-UAS collaboration. Besides wildfire management, this research will also benefit other emergency response applications in which humans and autonomous robots increasingly work together. The PIs will develop new and unique education and outreach programs, including a Wildfire-UAS Field Trip program and an annual outreach workshop series to provide interdisciplinary training to undergraduate/graduate students and to outreach to broader communities and the general public.

Animal Health Component

10%

Research Effort Categories

Basic

70%

Applied

10%

Developmental

20%

Classification

Knowledge Area (KA)	Subject of Investigation (SOI)	Field of Science (FOS)	Percent
807	7210	2020	100%

Knowledge Area
807 - Disaster Preparedness, Mitigation, Response, and Recovery;

Subject Of Investigation
7210 - Remote sensing equipment and technology;

Field Of Science
2020 - Engineering;

Keywords

wildfire, environmental monitoring

computer science, aerospace engineering, wildfire management, unmanned aircraft (uas)

autonomy, safety, teamed human-uas collaboration, sensing, modeling, simulation, control

Goals / Objectives
The main goal of this project is to develop innovative approaches that can transform wildfire management by enabling operational wildfire spread prediction and situation awareness for firefighters using a team of UASs. The UASs will work side-by-side with fire managers and ground firefighters to perform collaborative tasks. They can autonomously collect the most useful fire, wind, and ground crew information in fast evolving wildfire environments. At the same time, the multi-UAS system can support humans' interaction and direction with UASs' autonomy. This ability of influencing UASs' autonomy by incorporating human inputs would greatly improve humans' flexibility and willingness to work with UASs, which is essential for achieving ubiquitous collaborative robots. Finally, the collected fire and wind data from UASs will be used to support operational and accurate wildfire spread predictions.Detailed research objectives include:1) Develop fire sensing and wind estimation methods using a team of UASs and advanced data assimilation to assimilate UAS data into a dynamic model to enable data-driven wildfire simulation for operational wildfire spread predictions.2) Develop UAS coordination and path planning algorithms governing UASs' autonomy, including a) Information-driven global task allocation to coordinate a team of UASs to collect the most useful information in dynamic wildfire environments, and b) Human safety-aware local path planning to guide UASs to sense dynamic wildfires while in the meantime monitoring firefighters according to their safety risks.3) Develop innovative approaches to support teamed human-UASs collaboration, including a novel human-directed autonomy approach that allows fire manager or firefighter to direct UASs' autonomy based on their domain knowledge and expert opinions, and a human-UAS interaction interface to support human awareness of UASs' operation.4) Comprehensive evaluation of the proposed research, including evaluation by flying a team of UASs over real prescribed fires on lands managed by Kansas Biological Survey (KBS).This project also has several education and extension objectives. The PIs will develop new and unique education programs, including a Wildfire-UAS Field Trip program, an annual outreach workshop series, and new courses and learning materials to provide interdisciplinary training to both undergraduate and graduate students. Furthermore, to disseminate results to the broader community and the general public, the PIs will establish an open data repository hosting wildfire and UAS data and share with the research community, develop a web-based wildfire simulation portal open to the public, and organize club events to educate the public's view on UASs.

Project Methods
The methods of the project include:Using multiple UASs to cooperatively collect real time wind and wildfire data, which can significantly improve the sensing coverage, accuracy, efficiency, and robustness compared to a single sensing platform.Using innovative data assimilation methodology to assimilate the collected wildfire and wind data into wildfire simulation to achieve accurate fire spread predictions. It also allows quantification of firefighter safety based on predicted fire spreads.Developing information-driven global task allocation to coordinate a team of UASs to collect the most useful information in dynamic wildfire environments.Developing human safety-aware local path planning to guide UASs to sense dynamic wildfires while in the meantime monitoring firefighters' safety risks.Developing a Confidence-based Information Fusion and Incorporation (CIFI) method that allows human inputs to be systematically fused into UASs' autonomy algorithms.Efforts include:Developing a Wildfire-UAS Field Trip ProgramDeveloping new courses and learning materials at each institution to train both undergraduate and graduate students. These include "Data-driven Modeling and Simulation" at GSU, "Cooperative Control of Unmanned Aerial Vehicles" and "Distributed Estimation and Data Fusion" at Mizzou, and "Aerospace Instrumentation" at KU.Developing short course materials covering interdisciplinary topics such as real time sensing and modeling, and distributed algorithms in multi-UASs. Special efforts are given to attract underrepresented students.Developing an Annual Outreach Workshop series to reach out to scientific communities, fire management users, and the general public.Evaluation plan includes:Types of evaluation plans:Flight tests with simulated wildfiresThe flight tests will use real UASs on open tallgrass prairie fields. To evaluate algorithms that need information from wildfire spread, we will use the DEVS-FIRE model to generate simulated wildfire spread and integrate the simulation with real UAS flight test (a form of hardware-in-the-loop simulation). Since there is no real wildfire involved, realistic assumptions will be made on camera footprints, sampling rates, and measurement noise levels to simulate the UAS sensing of wildfires. To evaluate the algorithms involving firefighter safety, virtual firefighters will be added into the simulations. The simulated fires and virtual firefighters will be visualized on a map corresponding to the grass field shown on a tablet. The GUI allows users to provide inputs, e.g., selecting a virtual firefighter and increasing its safety risk, to evaluate the human-directed autonomy approach.Field tests over real prescribed firesKansas Biological Survey (KBS) has committed to supporting our UAS field tests over real prescribed fires conducted on the KU Field Station. KBS conducts regular prescribed fires every spring for vegetation management, with smaller fires (0.5-10 ha) occurring on relatively level ground and homogenous fuel load, and a medium fire (~ 100 ha) in hilly areas with heterogeneous fuel loads. KBS will notify us the timing and spatial extent of the prescribed fires and provide us estimation of fuel level. It will also share the GIS data and weather data related to the prescribed fires, which are needed to run wildfire simulations, and will share the real fire and wind measurement data for validation purpose. We will take full advantage of the prescribed fires at KBS and plan to carry out field tests in each year to evaluate the developed research.Evaluation MetricsSpatial and temporal resolution/accuracy of fire and wind sensing.Accuracy of fire spread prediction.Coverage frequency --- quantifies UASs' frequency of visiting different locations of the wildfire areaUAS-target location error --- difference between UASs' locations and the fire line or firefighters' locations.Response time --- quantifies the time it takes for the UASs to exhibit changed behavior in response to humans' inputs.

Progress 12/01/18 to 11/30/23

Outputs
Target Audience:Students working on the project, faculty collaborators, and researchers in related fields. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided? At GSU, 2 PhD students, 10 MS students, and 1 undergraduate student were supported and trained while working on this project. They learned knowledge in wildfire simulation and UAS path planning. At MU, 3 PhD students and 1 undergraduate student were supported and trained to build the UAS, learn the path planning algorithms, and do the simulations. 2 PhD students graduated in the summer of 2020 and 2022, respectively, under the support of this project. At KU, 4 PhD students, 1 MS student, and 4 undergraduate students were supported and trained to design, build, and flight test KHawk UAS. They learned knowledge in fire and wind sensing as well as human-in-the-loop path planning for UAS fire monitoring. 3 Ph.D. student graduated and 1 MS student graduated under the support of this project. How have the results been disseminated to communities of interest?The research results have been disseminated in three ways: 1) Publications; 2) Conference presentations; 3) conference posters. 4) workshop, 5) Media reports. Publications (reported in Products/Publications) Conference Presentations X. Hu, J. Bent, J. Sun, Wildfire Monitoring with Uneven Importance Using Multiple Unmanned Aircraft Systems, Proc. 2019 International Conference on Unmanned Aircraft Systems, 2019 H. Le, X. Hu, Extended Model Space Specification for Mobile Agent-based Systems to Support Automated Discovery of Simulation Models, Proc. 2020 Winter Simulation Conference, 2020 X. Hu, M. Ge, Modeling and Simulating Prescribed Fire Ignition Techniques, Proc. 2021 Annual Modeling and Simulation Conference, 2021 S M T. Islam, X. Hu, Real-Time On-board Path Planning for UAS-based Wildfire Monitoring, Proc. 2021 International Conference on Unmanned Aircraft Systems, 2021 Hu, X."Automated Model Discovery for Steering Behavior Simulation" 2022 Annual Modeling and Simulation Conference(2022). Hu, X.."Data Assimilation For Simulation-Based Real-Time Prediction/Analysis."2022 Annual Modeling and Simulation Conference(2022). Hu, X."Wildland Fire Simulation and Data Assimilation based on the DEVS-FIRE Model." DEVS Virtual Workshop (2022). Hu, X., In-situ and remotely-sensed data integration for wildfire management session, The 2022 ESIP (Earth Science Information Partner) January Meeting, "Wildland Fire Simulation and Data Assimilation using UAS data." (2022). Hu, X., Fick, W., Liu, Z., Chao, H., Xin, M. "Smart and Safe Prescribed Burning for Rangeland and Farmland Communities."2022 S&CC Principal Investigators' Meeting(2022). S M T. Islam, X. Hu, A Decentralized Importance-Based Multi-UAV Path Planning Algorithm for Wildfire Monitoring, Proc. 2023 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2023 M. Yan, X. Hu, Towards A Map-Based Web Application for Prescribed Fire Simulation, Proc. IEEE SoutheastCon 2023, April 2023 X. Hu, A Tutorial on Bayesian Sequential Data Assimilation for Dynamic Data Driven Simulation, Proc. 2023 Annual Modeling and Simulation Conference (ANNSIM 2023), 2023 X. Hu, Spatiotemporal Simulation, Data Assimilation, and Digital Twin for Wildland Fire Management, invited speaker, Georgia State University Scientific Computing Day, November 3, 2023 X. Hu, Dynamic Data Driven Simulation - Real-Time Data for Dynamic System Analysis and Prediction, invited talk, Modeling and Analysis Innovation Center, The MITRE Corporation, April 25, 2023 P. Shobeiry, M. Xin, X. Hu, H. Chao, UAV Path Planning for Wildfire Tracking Using Partially Observable Markov Decision Process, AIAA SciTech Forum, Virtual, Jan. 11-15, 2021. H. Ding, M. Xin, Extended Event-Triggered Consensus Strategies for Linear Multi-Agent Systems Based on Condensation Graph, The 60th IEEE Conference on Decision and Control, Austin, TX., Dec. 13-15, 2021. Jia, Q., Xin, M., Hu, X., Chao, H. Learning based Wildfire Tracking with Unmanned Aerial Vehicles. 2022 American Control Conference, Atlanta, GA, June 8-10, 2022. H. Flanagan, H. Chao, and S.G. Hagerott, Model Based Roll Controller Tuning and Frequency Domain Analysis for a Flying-Wing UAS, International Conference on Unmanned Aircraft Systems, 2019. J. Matt, H. Flanagan, and H. Chao, Evaluation and Analysis of ArduPilot Automatic Tuning Algorithm for the Roll Tracking Controller of a Small UAS, AIAA SCITech Atmospheric Flight Mechanics Conference, 2020. H. Flanagan, H. Chao, and Y.Q. Chen, Lateral Fractional Order Controller Design and Tuning for a Flying-Wing UAS, International Conference on Unmanned Aircraft Systems, 2020. J. Matt, S.G. Hagerott, B.C. Svoboda, H. Chao, and H. Flanagan, Frequency Domain System Identification of a Small Flying-Wing UAS, AIAA SCITech Atmospheric Flight Mechanics Conference, 2022 P. Tian, H. Chao, and H. Wu. UAS-based wind estimation using sinusoidal gust model. In AIAA Scitech 2019 Forum (p. 1597), 2019. P. Tian and H. Chao. Model aided estimation of angle of attack, sideslip angle, and 3D wind without flow angle measurements. In 2018 AIAA Guidance, Navigation, and Control Conference(p. 1844), 2018. H. Chao, J. Mat, H. Flanagan, P. Tian, and S. Gowravaram, Atmospheric Sensing of Wildland Fire Plumes Using KHawk UASs. In 100th American Meteorological Society Annual Meeting. AMS, 2020. S. Gowravaram, H. Chao, H. Flanagan, P. Tian, J. Goyer, M. Xin, and X. Hu, Wildland Fire Monitoring and Mapping Using Orthorectified Near-Infrared and Thermal UAS Imagery, 101th American Meteorological Society Annual Meeting, Special Symposium on Meteorological Observations and Instrumentation, 2021. H. Flanagan, P. Tian, and H. Chao, Wind and Turbulence Estimation during Wildland Fire using KHawk Fixed-Wing UAS, 101th American Meteorological Society Annual Meeting, Special Symposium on Meteorological Observations and Instrumentation, 2021. Conference Posters H. Chao, J. Mat, H. Flanagan, P. Tian, and S. Gowravaram, Atmospheric Sensing of Wildland Fire Plumes Using KHawk UASs. In 100th American Meteorological Society Annual Meeting. AMS, 2020. Workshop • X. Hu, UAS Data Enabled Operational Fire Spread Simulation, workshop presentation, 2021 UAS Integration for Fire Operation Workshop, virtual workshop, November 17, 2021 • H. Chao, Fire Metrics Measurement Using sUAS, 2021 UAS Integration for Fire Operation Workshop, virtual workshop, November 17, 2021. • Haiyang Chao, Remote and In-situ Sensing of Wildland Fires Using UAS, March 10, 2021 NASA UAS Workshop Media Reports 1. "GSU professor developing drones to help fight wildfires", WSB-TV Channel 2 News, March 25, 2019.https://www.wsbtv.com/news/local/atlanta/gsu-professor-developing-drones-to-help-fight-wildfires/934043900/ 2. "Drones Provide Eye-in-the-Sky to Help Fight Fires," U.S. Department of Agriculture Blog, Aug 15, 2019 https://www.usda.gov/media/blog/2019/07/09/drones-provide-eye-sky-help-fight-fires 3. "The High-Tech Future of Firefighting", Georgia State University Research Magazine, Spring 2019 Issue, pp. 6-7, 2019. https://news.gsu.edu/research-magazine/spring2019/the-high-tech-future-of-firefighting 4. "Researcher Leads Project To Explore Use Of Drones To Fight Wildfires", Georgia State University News, January 23, 2019 1. https://news.gsu.edu/2019/01/23/georgia-state-grant-to-study-using-drones-to-fight-wildfires/ 5. "Project Looks Into How Drones Can Predict Spread of Wildfire", Associated Press, January 20, 2019 https://apnews.com/3546526d7ce34881a92df0688268e3a4 6. "Aerospace engineering faculty, students create drone to detect wildfires", The University Daily Kansan, Feb. 6, 2019. http://www.kansan.com/news/aerospace-engineering-faculty-students-create-drone-to-detect-wildfires/article_f866c2dc-28bb- 11e9-8e46-6ffa9a9dbb58.html 7. "MU professor developing integrated drones to predict spread of wildfires", Missourian, Jan. 12, 2019. https://www.columbiamissourian.com/news/higher_education/mu-professor-developing-integrated-drones-to-predict-spreadof- wildfires/article_e520bf78-1387-11e9-a487- f70294d3109e.html?utm_source=Email%20Newsletters&utm_campaign=Daily%20Edition&utm_medium=Email&utm_content =headline 8. Drones Take Their Place on the Cutting Edge of Wildfire Fighting (https://www.flyingmag.com/drones-wildfire-fighting/) , FlyingMag.com, October 7, 2021 9. "China Has a Drone Army to Fight Off Wildfires. So Why Doesn't America?", Daily Beast, Oct. 18, 2022 https://www.cnas.org/press/in-the-news/china-has-a-drone-army-to-fight-off-wildfires-so-why-doesnt-america 10. Understanding fire behavior, 2021, KU Field Station. https://biosurvey.ku.edu/field-station-2021 11. Workshop provides training for prescribed burns, Joe Piezuch EKPBA Vice President May 24, 2023. https://www.republic-online.com/agriculture/workshop-provides-training-for-prescribed-burns/article_3a576a48-f919-11ed-ac6c-5f9e56acffa6.html What do you plan to do during the next reporting period to accomplish the goals? Nothing Reported

Impacts
What was accomplished under these goals? We report our accomplishments for each goal separately, as below. 1. Fire sensing, wind estimation, and data assimilation to enable data driven wildfire simulation. 1.1 Designed, built, flight tested two low-cost UAS for fire sensing applications, called KHawk-Thermal fixed-wing UAS and KHawk CarbonQuad multirotor UAS. Both UAS are equipped with a thermal camera. 1.2 Achieved system identification of small fixed-wing UAS using frequency domain methods and performed model-based innerloop controller design 1.3 Developed a UAS based fire perimeter sensing system, which is guided by a human operator. Multiple fire RGB/NIR maps have been generated at preset time intervals after each flight test. 1.4 Developed a new method for measurement of fire line location and rate of spread (ROS) based on sequential UAS thermal maps. The method is validated using the October 2019 Welda KS prescribed fire data (IJRS 2022). 1.5 Proposed a new method for spatiotemporal representation of grass fire evolution using time labeled UAS NIR orthomosaics generated from aerial images with limited footprints. (JSTARS 2022) 1.6 Performed a systematic comparison between thermal and NIR aerial imagery for grass fire detection using the generated GRAFFITI data set (MS thesis). 1.7 Provide a survey of the state-of-the-art methods for wind sensing and estimation using small fixed-wing UAS (JAIS2021). 1.8 Developed a 9-state-Extended Kalman Filter which uses UAS GPS/inertial/pressure measurements for mean wind estimation at the flame height (SciTech). 1.9 Generated preliminary results for the prevailing wind and turbulence estimation using UAS flight data collected when flying through fire-generated plumes (AMS). 1.10 Developed the capability of prescribed fire ignition modeling for symmetrically modeling different ignition techniques of prescribed fires, and applied it to simulating the prescribed fire event of Welda, Kansas in October 2019. Simulation results were compared with real data collected from UAS. A journal paper has been published based on this work. 1.11 Developed a prototype map-based prescribed fire simulation system. The system allows users to define ignition procedures and simulate the fire growth using a map-based tool. 1.12 Developed a data assimilation algorithm for assimilating real-time and noisy fire location data extracted from UAS' monitoring into the fire spread simulation. 2. UAS path planning 2.1. Developed and analyzed a real-time on-board path planning algorithm for UAS-based wildfire monitoring with uneven importance. With this path planning algorithm, the UAS more frequently visits the regions that have more active spreading behaviors. 2.2. Extended the previous algorithm to support multi-UAS path planning for fire front monitoring. 2.3. Added the capability of displaying fire perimeter location data in the Mission Planner map user interface. This capability allows a UAS pilot to see the real time fire shape on the map while planning a UAS' flying path in real time. 2.4. Developed several path planning algorithms to guide multiple UASto collect fire information more efficiently, including: 1) Designed a path planning algorithm based on partially observable Markov decision process (POMDP), for a simulated wildfire in which multiple UAS can track multiple fire spots autonomously to maximize the sensing coverage; 2) Designed a path planning algorithm for a group of UASto track multiple spreading wildfire zones. Due to limited observable information, the fire evolution is hard to model. A regression neural network is online trained with real-time UAS observation data and applied for fire front prediction. A Q-learning-based path planning algorithm was designed to track the fire fronts effectively and collect the most useful information, which is in turn utilized to better retrain the fire front prediction model. Various practical factors are taken into account by cost function designs such as moving target tracking, field of view of UAS, spreading speed of fire zones, collision/obstacle avoidance, and maximum information collection. 3. Evaluation 3.1 Flight test of KHawk fixed-wing UAS, Hexcopter,and CarbonQuad for system test, wind estimation, and fire sensing. 3.2 Multiple flight sessions have been performed over 11 prescribed fire burns in Kansas for the test of the developed fire and wind sensing system. 3.3 Collected prescribed grass fire data using low-cost thermal camera,NIR camera and wind sensing payload. The collected data is used for validation of the proposed fire and wind sensing algorithms.

Publications

Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: X. Hu, A Tutorial on Bayesian Sequential Data Assimilation for Dynamic Data Driven Simulation, Proc. 2023 Annual Modeling and Simulation Conference (ANNSIM 2023), 2023
Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: S M T. Islam, X. Hu, A Decentralized Importance-Based Multi-UAV Path Planning Algorithm for Wildfire Monitoring, Proc. 2023 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2023
Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: M. Yan, X. Hu, Towards A Map-Based Web Application for Prescribed Fire Simulation, Proc. IEEE SoutheastCon 2023, April 2023
Type: Journal Articles Status: Accepted Year Published: 2023 Citation: X. Hu, M. Ge, S. Gowravaram, H. Chao, M. Xin, Prescribed Fire Simulation with Dynamic Ignitions Using Data from UAS-based Sensing, Journal of Simulation, DOI: 10.1080/17477778.2023.2217335, 2023
Type: Journal Articles Status: Published Year Published: 2023 Citation: S. Gowravaram, Chao, H., Lin, Z., Parsons, S., Zhao, T., Xin, M., Hu, X., Tian, P., Flanagan, H., Wang, G. Prescribed Grass Fire Mapping and Rate of Spread Measurement Using NIR Images from a Small Fixed-Wing UAS. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 16, pp.3519-3530, 2023.
Type: Conference Papers and Presentations Status: Published Year Published: 2022 Citation: X. Hu. Data Assimilation For Simulation-Based Real-Time Prediction/Analysis. Proc. 2022 Annual Modeling and Simulation Conference (ANNSIM 2022).
Type: Journal Articles Status: Published Year Published: 2019 Citation: X. Hu, P. Wu, A Data Assimilation Framework for Discrete Event Simulations, ACM Transactions on Modeling and Computer Simulation (TOMACS), 29(3), Article No. 17, July 2019
Type: Conference Papers and Presentations Status: Published Year Published: 2019 Citation: X. Hu, J. Bent, J. Sun, Wildfire Monitoring with Uneven Importance Using Multiple Unmanned Aircraft Systems, Proc. 2019 International Conference on Unmanned Aircraft Systems (ICUAS'19), 2019
Type: Conference Papers and Presentations Status: Published Year Published: 2021 Citation: S M T. Islam, X. Hu, Real-Time On-board Path Planning for UAS-based Wildfire Monitoring, Proc. 2021 International Conference on Unmanned Aircraft Systems (ICUAS), 2021
Type: Conference Papers and Presentations Status: Published Year Published: 2021 Citation: X. Hu, M. Ge, Modeling and Simulating Prescribed Fire Ignition Techniques, Proc. 2021 Annual Modeling and Simulation Conference (ANNSIM'21), 2021
Type: Journal Articles Status: Published Year Published: 2023 Citation: Sun, T., Xin, M. Inverse Covariance Intersection-based Distributed Estimation and Application in Wireless Sensor Network. IEEE Transactions on Industrial Informatics, 19(10), 10079-10090, 2023.
Type: Conference Papers and Presentations Status: Published Year Published: 2022 Citation: S M T. Islam, X. Hu, Towards Decentralized Importance-based Multi-UAS Path Planning for Wildfire Monitoring, Proc. 2022 17th Annual System of Systems Engineering Conference (SOSE), pp. 67-72, doi:10.1109/SOSE55472.2022.9812651, 2022
Type: Books Status: Published Year Published: 2023 Citation: Xiaolin Hu, Dynamic Data-Driven Simulation Real-time Data for Dynamic System Analysis and Prediction, World Scientific, April 2023
Type: Conference Papers and Presentations Status: Published Year Published: 2019 Citation: H. Flanagan, H. Chao, and S.G. Hagerott, Model Based Roll Controller Tuning and Frequency Domain Analysis for a Flying-Wing UAS, International Conference on Unmanned Aircraft Systems, 2019.
Type: Conference Papers and Presentations Status: Published Year Published: 2021 Citation: H. Flanagan, P. Tian, and H. Chao, Wind and Turbulence Estimation during Wildland Fire using KHawk Fixed-Wing UAS, 101th American Meteorological Society Annual Meeting, Special Symposium on Meteorological Observations and Instrumentation, 2021.
Type: Conference Papers and Presentations Status: Published Year Published: 2022 Citation: J. Matt, S.G. Hagerott, B.C. Svoboda, H. Chao, and H. Flanagan, Frequency Domain System Identification of a Small Flying-Wing UAS, AIAA SciTech Forum, Atmospheric Flight Mechanics Conference, 2022
Type: Conference Papers and Presentations Status: Published Year Published: 2019 Citation: P. Tian, H. Chao, and H. Wu. UAS-based wind estimation using sinusoidal gust model. In AIAA SciTech 2019 Forum (p. 1597), 2019.
Type: Conference Papers and Presentations Status: Published Year Published: 2020 Citation: H. Chao, J. Mat, H. Flanagan, P. Tian, and S. Gowravaram, Atmospheric Sensing of Wildland Fire Plumes Using KHawk UASs. In 100th American Meteorological Society Annual Meeting. AMS, 2020.

Progress 12/01/21 to 11/30/22

Outputs
Target Audience:Students working in the project, and faculty collaborators Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided? At GSU, 2 PhD students and 5 MS students were supported and trained while working on this project. They learned knowledge in wildfire simulation and UAS path planning. At MU, 2 PhD students were supported and trained to build the UAS, design the path planning algorithms, and do the simulations. 1 Ph.D student graduated in the summer of 2022 under the support of this project. At KU, 2 PhD students and 2 undergraduate students were supported and trained to develop new methods for fire metrics measurement and to design, build, and flight test KHawk UAS. They learned knowledge in UAS based fire and wind sensing as well as human-in-the-loop path planning for UAS fire monitoring. 1 Ph.D. student graduated in the spring of 2022 and 1 M.S. student graduated in the summer of 2022 under the support of this project. How have the results been disseminated to communities of interest?The research results have been disseminated in three ways: 1) Publications, 2) Presentations; and 3) Media. Publications Hu, X. (Accepted/Forthcoming/In Press) (2023). Dynamic Data-Driven Simulation - Real-time Data for Dynamic System Analysis and Prediction. World Scientific. Islam, T., Hu, X. (Second Revision & Resubmit) Real-time Path Planning for Dynamical Wildfire Monitoring with Uneven Importance. Applied Intelligence. Hu, X., Ge, M., Gowravaram, S., Chao, H., Xin, M. (Submitted) Prescribed Fire Simulation with Dynamic Ignitions using Data from UAS-based Sensing. Journal of Simulation. Hu, X. (Published) (2022). Data Assimilation For Simulation-Based Real-Time Prediction/Analysis. Proc. 2022 Annual Modeling and Simulation Conference (ANNSIM 2022). Islam, S. M. T., Hu, X. (Published) (2022). Towards Decentralized Importance-based Multi-UAS Path Planning for Wildfire Monitoring. Proc. 2022 17th Annual System of Systems Engineering Conference (SOSE), Proc. 2022 17th Annual System of Systems Engineering Conference (SOSE). M. Yan, X. Hu (Accepted) (2023). Towards A Map-Based Web Application for Prescribed Fire Simulation. Proc. IEEE SoutheastCon 2023. Jia, Q., Xin, M., Hu, X., Chao, H. (Published) (2022). Learning based Wildfire Tracking with Unmanned Aerial Vehicles. Proc. 2022 American Control Conference, Proc. 2022 American Control Conference. Gowravaram, S., Chao, H., Lin, Z., Parsons, S., Zhao, T., Xin, M., Hu, X., Tian, P., Flanagan, H., Wang, G. (Revised & Resubmitted) Prescribed Grass Fire Mapping and Rate of Spread Measurement Using NIR Images from a Small Fixed-Wing UAS. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. Gowravaram, S., Chao, H., Zhao, T., Parsons, S., Hu, X., Xin, M., Flanagan, H., Tian, P. (Published) (2022). Prescribed Grass Fire Evolution Mapping and Rate of Spread Measurement Using Orthorectified Thermal Imagery from a Fixed-Wing UAS. International Journal of Remote Sensing, 43(7), 2357-2376. Justin Matt, Steven G. Hagerott, Benjamin C. Svoboda, Haiyang Chao, and Harold Flanagan, Frequency Domain System Identification of a Small Flying-Wing UAS, AIAA SCITech Atmospheric Flight Mechanics Conference, 2022 Sun, T., Xin, M. (Accepted/In Press) (2023). Inverse Covariance Intersection-based Distributed Estimation and Application in Wireless Sensor Network. IEEE Transactions on Industrial Informatics. Presentations Hu, X., 2022 Annual Modeling and Simulation Conference (ANNSIM'22), "Automated Model Discovery for Steering Behavior Simulation." (2022). Hu, X., 2022 Annual Modeling and Simulation Conference (ANNSIM'22), "Data Assimilation For Simulation-Based Real-Time Prediction/Analysis." (2022). Hu, X., DEVS Virtual Workshop, "Wildland Fire Simulation and Data Assimilation based on the DEVS-FIRE Model." (2022). Hu, X., In-situ and remotely-sensed data integration for wildfire management session, The 2022 ESIP (Earth Science Information Partner) January Meeting, "Wildland Fire Simulation and Data Assimilation using UAS data." (2022). Hu, X., Fick, W., Liu, Z., Chao, H., Xin, M., 2022 S&CC Principal Investigators' Meeting, "Smart and Safe Prescribed Burning for Rangeland and Farmland Communities." (2022). Chao, H., Tactical Fire Remote Sensing Advisory Committee Spring 2022 Meeting, "UAS based Grass Fire Metrics Measurement", (2022). Chao, H., Jefferson County FireFighter Meeting, "Toward UAS based wildfire measurement and prediction", (2022). Matt, J., AIAA SCITech Atmospheric Flight Mechanics Conference,"Frequency Domain System Identification of a Small Flying-Wing UAS," (2022). Gowravaram, S., 102nd American Meteorological Society Annual Meeting, Special Symposium on Meteorological Observations and Instrumentation, "Fire monitoring and metric measurements using low-cost UAS and Satellite Multispectral Data," (2022). Svoboda, B., University of Kansas Undergraduate Research Meeting, "Design and Flight Testing of a Quadcopter UAS for Wildland Fire Monitoring," (2022). Media • "China Has a Drone Army to Fight Off Wildfires. So Why Doesn't America?", Daily Beast, Oct. 18, 2022 https://www.cnas.org/press/in-the-news/china-has-a-drone-army-to-fight-off-wildfires-so-why-doesnt-america What do you plan to do during the next reporting period to accomplish the goals? Extend the onboard importance-based path planning algorithm to multiple UAS and carry out comprehensive analysis and testing. Improve the accuracy and robustness of the data assimilation algorithm to support construction of the dynamically-changing fire perimeter based on noisy fire location data extracted from UAS monitoring. Test the algorithm using real UAS data collected from a previous prescribed fire event. Enhance the functions of the map-based prescribed fire simulation system. Organize our second UAS integration for fire operation workshop at the University of Kansas to foster collaborations between research and UAS application in wildland fire management. Develop a new method for real time fire front sensing and visualization based on low-cost thermal UAS imagery. Develop new systems and methods to guide multirotor UAS to detect and follow grassfire boundaries autonomously. Collection of a high quality UAS fire data set with cameras from multiple bands with high-fidelity intercamera calibration. Flight validation of the proposed autonomous fire following system and methods.

Impacts
What was accomplished under these goals? 1. Fire sensing, wind estimation, and data assimilation to enable data driven wildfire simulation. Developed a prototype map-based prescribed fire simulation system. The system allows users to define ignition procedures and simulate the fire growth using a map-based tool. Developed a data assimilation algorithm for assimilating real-time and noisy fire location data extracted from UAS monitoring into the fire spread simulation, and obtained preliminary results. Proposed a new method for spatiotemporal representation of grass fire evolution using time labeled UAS NIR orthomosaics generated from aerial images with limited footprints.Performed a systematic comparison between thermal and NIR aerial imagery for grass fire detection using the generated GRAFFITI data set. Performed frequency domain system identification of a flying-wing UAS to support later UAS-based wind and turbulence estimation during wildland fires. 2. UAS path planning Analyzed the real-time on-board path planning algorithm for UAS-based wildfire monitoring with uneven importance. Obtained preliminary results of extending the algorithm that works for single UAS to multiple UAS. Added the capability of displaying fire perimeter location data in the Mission Planner map user interface. The goal is to allow a UAS pilot to see the real time fire shape on the map while planning a UAS' flying path in real time. Designed a path planning algorithm for a group of UASs to track multiple spreading wildfire zones. A real-time machine learning-based model was developed to predict fire front from the real-time observation data. A Q-learning-based path planning algorithm was designed to track the fire fronts effectively and collect the most useful information, which is in turn utilized to better retrain the fire front prediction model. 3. Evaluation Collected prescribed grass fire data using low-cost thermal camera and NIR camera Generated a new grass fire aerial image dataset (GRAFFITI) and developed novel methods for near-infrared (NIR) imagery-based fire front identification and fire depth estimation using small unmanned aircraft systems.

Publications

Type: Journal Articles Status: Published Year Published: 2022 Citation: Gowravaram, S., Chao, H., Zhao, T., Parsons, S., Hu, X., Xin, M., Flanagan, H., Tian, P. (2022). Prescribed Grass Fire Evolution Mapping and Rate of Spread Measurement Using Orthorectified Thermal Imagery from a Fixed-Wing UAS. International Journal of Remote Sensing, 43(7), 2357-2376.
Type: Journal Articles Status: Accepted Year Published: 2022 Citation: Sun, T., Xin, M. (Accepted, 2022). Inverse Covariance Intersection-based Distributed Estimation and Application in Wireless Sensor Network. IEEE Transactions on Industrial Informatics.
Type: Journal Articles Status: Under Review Year Published: 2022 Citation: Gowravaram, S., Chao, H., Lin, Z., Parsons, S., Zhao, T., Xin, M., Hu, X., Tian, P., Flanagan, H., Wang, G. (Revised & Resubmitted). Prescribed Grass Fire Mapping and Rate of Spread Measurement Using NIR Images from a Small Fixed-Wing UAS. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.
Type: Journal Articles Status: Under Review Year Published: 2022 Citation: Hu, X., Ge, M., Gowravaram, S., Chao, H., Xin, M. (Submitted/Under Review) Prescribed Fire Simulation with Dynamic Ignitions using Data from UAS-based Sensing. Journal of Simulation.
Type: Conference Papers and Presentations Status: Published Year Published: 2022 Citation: Jia, Q., Xin, M., Hu, X., Chao, H. (Published) (2022). Learning based Wildfire Tracking with Unmanned Aerial Vehicles. Proc. 2022 American Control Conference.

Progress 12/01/20 to 11/30/21

Outputs
Target Audience: Nothing Reported Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided? At GSU, 2 PhD students and 3 MS students were supported and trained while working on this project. They learned knowledge in wildfire simulation and UAS path planning. At MU, 2 PhD students were supported and trained to build the UAS, design the path planning algorithms, and do the simulations. At KU, 2 PhD students and 2 undergraduate students were supported and trained to develop new methods for fire metrics measurement and to design, build, and flight test KHawk UAS. They learned knowledge in UAS based fire and wind sensing as well as human-in-the-loop path planning for UAS fire monitoring. 1 Ph.D. student graduated in the summer of 2021 under the support of this project. How have the results been disseminated to communities of interest?The research results have been disseminated in four ways: 1) Publications, 2) Presentations; 3) Workshop; 4) Media. Publications: S. Gowravarm, H. Chao, T. Zhao, S. Parsons, X. Hu, M. Xin, H. Flanagan, and P. Tian, "Prescribed Grass Fire Evolution Mapping and Rate of Spread Measurement Using Orthorectified Thermal Imagery from a Fixed-Wing UAS," International Journal of Remote Sensing, Accepted, 2022. P. Tian, H. Chao, M. Rhudy, J. Gross, and H. Wu, "Wind Sensing and Estimation Using Small Fixed-Wing UAVs: A Survey," AIAA Journal of Aerospace Information Systems (JAIS), 18, no. 3, pp 132-143, 2021. J. Matt, H. Flanagan, and H. Chao, "Evaluation and Analysis of ArduPilot Automatic Tuning Algorithm for the Roll Tracking Controller of a Small UAS," AIAA SciTech Atmospheric Flight Mechanics Conference, 2021. X. Hu, M. Ge, S. Gowravaram, H. Chao, M. Xin, "Simulating Prescribed Fire with Multiple Ignitions using Data from UAS-based Sensing," SIMULATION, submitted, 2021 SMT. Islam, X. Hu, "Real-Time On-board Path Planning for UAS-based Wildfire Monitoring," Proc. 2021 International Conference on Unmanned Aircraft Systems (ICUAS), 2021 X. Hu, M. Ge, "Modeling and Simulating Prescribed Fire Ignition Techniques," Proc. 2021 Annual Modeling and Simulation Conference (ANNSIM'21), 2021 P. Shobeiry, M. Xin, X. Hu, and H. Chao, "UAV Path Planning for Wildfire Tracking Using Partially Observable Markov Decision Process," AIAA SciTech Forum,Guidance Navigation and ControlConference,2021. P. Shobeiry and M. Xin, "An Optimal Control Approach for Consensus of General LTI Multi-Agent Systems," ASME Journal of Dynamic Systems, Measurement, and Control, 143(9), 091002, 2021. H. Ding and M. Xin, "Extended Event-Triggered Consensus Strategies for Linear Multi-Agent Systems Based on Condensation Graph," The 60th IEEE Conference on Decision and Control, Austin, TX., Dec. 13-15, 2021. Presentations S. Gowravaram, H. Chao, H. Flanagan, P. Tian, J. Goyer, M. Xin, and X. Hu, "Wildland Fire Monitoring and Mapping Using Orthorectified Near-Infrared and Thermal UAS Imagery," 101th American Meteorological Society Annual Meeting, Special Symposium on Meteorological Observations and Instrumentation, 2021 H. Flanagan, P. Tian, and H. Chao, "Wind and Turbulence Estimation during Wildland Fire using KHawk Fixed-Wing UAS," 101th American Meteorological Society Annual Meeting, Special Symposium on Meteorological Observations and Instrumentation, 2021 X. Hu, M. Ge, "Modeling and Simulating Prescribed Fire Ignition Techniques," Proc. 2021 Annual Modeling and Simulation Conference (ANNSIM'21), 2021 S M T. Islam, X. Hu, "Real-Time On-board Path Planning for UAS-based Wildfire Monitoring," Proc. 2021 International Conference on Unmanned Aircraft Systems (ICUAS), 2021 P. Shobeiry, M. Xin, X. Hu, and H. Chao, "UAV Path Planning for Wildfire Tracking Using Partially Observable Markov Decision Process,"AIAA SciTech Forum,Guidance Navigation and ControlConference,2021. H. Ding and M. Xin, "Extended Event-Triggered Consensus Strategies for Linear Multi-Agent Systems Based on Condensation Graph," The 60th IEEE Conference on Decision and Control, Austin, TX., Dec. 13-15, 2021. Workshop X. Hu, "UAS Data Enabled Operational Fire Spread Simulation," Workshop presentation, 2021 UAS Integration for Fire Operation Workshop, virtual workshop, November 17, 2021 H. Chao, "Fire Metrics Measurement Using sUAS," 2021 UAS Integration for Fire Operation Workshop, virtual workshop, November 17, 2021. Haiyang Chao, "Remote and In-situ Sensing of Wildland Fires Using UAS," March 10, 2021 NASA UAS Workshop Media Drones Take Their Place on the Cutting Edge of Wildfire Fighting (https://www.flyingmag.com/drones-wildfire-fighting/) , FlyingMag.com, October 7, 2021 What do you plan to do during the next reporting period to accomplish the goals? Collection of a high quality UAS fire data set with another reference in fire front location measurement. Develop a new method for real time fire front sensing and visualization based on low-cost thermal UAS imagery. Develop new methods and system to support autonomous fire boundary tracking using Quadcopter UAS. Develop a method to construct the dynamically-changing fire perimeter in real time based on noisy fire location data extracted from UAS monitoring. Develop fully automated fire monitoring using a single UAS. The UAS should be able to collect data, pass data to a ground station for processing or carry out on-board processing, and plan its flying path in a fully automated way. Develop data assimilation method that works with UAS-detected fire location data. Develop a prototype web-based prescribed fire ignition and fire spread simulation system allowing users to define prescribed ignition plans and simulate prescribed burning events. We have already started some initial work on this task. Develop an AI and vision-based tracking algorithm to support autonomous path planning for fire front tracking using a single UAS and/or multiple UASs.

Impacts
What was accomplished under these goals? 1. Fire sensing, wind estimation, and data assimilation to enable data driven wildfire simulation. Developed a new method for recapturing the grass fire evolution using time labelled orthomosaics collected by a smallfixed-wing UAS at low altitudes. Developed a new method for UAS thermal imagery based grass fire metrics measurement including fire front location andfire rate of spread. Developed a new and low-cost method for UAS NIR imagery based grass fire metrics measurement including fire frontlocation and fire rate of spread. Started to develop the capability of real-time fire monitoring using low-cost thermal data from KHawk Quadcopter UAS. Started to develop a new UAS model based algorithm for estimation of vertical wind velocity during fire smoke encounters. Developed the capability of prescribed fire ignition modeling for symmetrically modeling different ignition techniques ofprescribed fires, and applied it to simulating a real prescribed fire event. Started to develop a data assimilation method for assimilating real-time and noisy fire location data extracted from UASmonitoring into the fire spread simulation. 2. UAS path planning Developed and analyzed a real-time on-board path planning algorithm for UAS-based wildfire monitoring with unevenimportance. With this path planning algorithm, the UAS more frequently visits the regions that have more active spreadingbehaviors. Started to add the capability of displaying real time fire perimeter location data in the Mission Planner map user interface.This capability allows a UAS pilot to see the real time fire shape on the map while planning a UAS' flying path in real time. Designed a path planning algorithm for a group of UASs to track multiple spreading wildfire zones. Due to limitedobservableinformation, the fire evolution is hard to model. A regression neural network is online trained with real-time UASobservation data and applied for fire front prediction. To track fire fronts effectively, a UAS path planning algorithm isproposed by Q-learning. Various practical factors are taken into account by cost function designs such as moving target tracking, field of view of UASs, spreading speed of fire zones, collision/obstacle avoidance, and maximum information collection. 3. Evaluation Flight test of KHawk fixed-wing UAS and Quadcopter for system test and real-time fire sensing. Worked on the data collected from the 2019 prescribed fire data for validation of the proposed methods on grass fire metrics measurement and fire evolution reconstruction.

Publications

Type: Journal Articles Status: Published Year Published: 2021 Citation: P. Shobeiry and M. Xin, "An Optimal Control Approach for Consensus of General LTI Multi-Agent Systems," ASME Journal of Dynamic Systems, Measurement, and Control, 143(9), 091002, 2021.
Type: Conference Papers and Presentations Status: Published Year Published: 2021 Citation: P. Shobeiry, M. Xin, X. Hu, and H. Chao, "UAV Path Planning for Wildfire Tracking Using Partially Observable Markov Decision Process," AIAA SciTech Forum, January 11-21, 2021, Virtual Event.
Type: Conference Papers and Presentations Status: Published Year Published: 2021 Citation: H. Ding and M. Xin, "Extended Event-Triggered Consensus Strategies for Linear Multi-Agent Systems Based on Condensation Graph," The 60th IEEE Conference on Decision and Control, Austin, TX., Dec. 13-15, 2021.
Type: Journal Articles Status: Accepted Year Published: 2022 Citation: S. Gowravarm, H. Chao, T. Zhao, S. Parsons, X. Hu, M. Xin, H. Flanagan, and P. Tian, "Prescribed Grass Fire Evolution Mapping and Rate of Spread Measurement Using Orthorectified Thermal Imagery from a Fixed-Wing UAS," International Journal of Remote Sensing, Accepted, February 2022.
Type: Journal Articles Status: Submitted Year Published: 2022 Citation: X. Hu, M. Ge, S. Gowravaram, H. Chao, M. Xin, "Simulating Prescribed Fire with Multiple Ignitions using Data from UAS-based Sensing," SIMULATION, under review, submitted in 2021.

Progress 12/01/19 to 11/30/20

Outputs
Target Audience: Nothing Reported Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided?Education and Training: At GSU, 2 PhD students and 1 MS student were supported and trained while working on this project. They learned knowledge in wildfire simulation and UAS path planning. At MU, 2 PhD students were supported and trained to build the UAS, learn the path planning algorithms, and do the simulations. 1 PhD student graduated in the summer 2020 under the support of this project. At KU, 2 PhD students and 1 undergraduate student were supported and trained to design, build, and flight test KHawk UAS. They learned knowledge in fire and wind sensing as well as human-in-the-loop path planning for UAS fire monitoring. 1 Ph.D. student graduated in the summer of 2020 under the support of this project. Workshop and Conferences: Due to the COVID-19 pandemic, the planned workshop in the second year was postponed. Conferences were attended virtually. How have the results been disseminated to communities of interest?The research results have been disseminated in three ways: 1) Publications, 2) Conference presentations; 3) conference posters. Publications B. Jia, and M. Xin, "Data-Driven Enhanced Nonlinear Gaussian Filter," IEEE Transactions on Circuits and Systems II, 67(6), p. 1144 - 1148, 2020. P. Shobeiry, M. Xin, X. Hu, and H. Chao, "UAV Path Planning for Wildfire Tracking Using Partially Observable Markov Decision Process," AIAA SciTech Forum, Guidance, Navigation, and Control Conference, accepted to appear, 2020. P. Tian, H. Chao, M. Rhudy, J. Gross, and H. Wu, "Wind Sensing and Estimation Using Small Fixed-Wing UAVs: A Survey", AIAA Journal of Aerospace Information Systems (JAIS), accepted to appear, 2020. J. Matt, H. Flanagan, and H. Chao, "Evaluation and Analysis of ArduPilot Automatic Tuning Algorithm for the Roll Tracking Controller of a Small UAS", AIAA SciTech Forum, Atmospheric Flight Mechanics Conference, accepted to appear, 2020. H. Flanagan, H. Chao, and Y.Q. Chen, "Lateral Fractional Order Controller Design and Tuning for a Flying-Wing UAS", International Conference on Unmanned Aircraft Systems, 2020. X. Hu, M. Ge, S. Gowravaram, H. Chao, M. Xin, "Prescribed Fire Simulation using Data from UAS-based Sensing", International Journal of Wildland Fire, submitted, 2020. H. Le, X. Hu, "Extended Model Space Specification for Mobile Agent-based Systems to Support Automated Discovery of Simulation Models", Proc. 2020 Winter Simulation Conference, 2020. Conference presentations H. Flanagan, H. Chao, and Y.Q. Chen, "Lateral Fractional Order Controller Design and Tuning for a Flying-Wing UAS", International Conference on Unmanned Aircraft Systems, 2020. H. Le, X. Hu, "Extended Model Space Specification for Mobile Agent-based Systems to Support Automated Discovery of Simulation Models", Proc. 2020 Winter Simulation Conference, 2020. Conference Posters H. Chao, J. Mat, H. Flanagan, P. Tian, and S. Gowravaram, "Atmospheric Sensing of Wildland Fire Plumes Using KHawk UASs". In 100th American Meteorological Society Annual Meeting. AMS, 2020. What do you plan to do during the next reporting period to accomplish the goals? Upgrade the current UAS data communication subsystem to enable real time fire situation awareness during the fire. Analyze the data of NIR and thermal cameras for fire detection effectiveness under fire and smoke conditions. Enable autonomous fire boundary tracking using fixed-wing UAS and Hexcopter UAS. Develop fully automated fire monitoring using a single UAS. The UAS should be able to collect data, pass data to a ground station for processing or carry out on-board processing, and plan its flying path in a fully automated way. Develop data assimilation method that works with UAS-detected fire location data. Develop an AI and vision-based tracking algorithm to support autonomous path planning for fire front tracking using a single UAS and/or multiple UASs. Organize the outreach workshop in Year 3 of the project to foster collaborations between research and UAS application in wildfire management.

Impacts
What was accomplished under these goals? 1. Fire sensing, wind estimation, and data assimilation to enable data driven wildfire simulation. 1.1 Developed a new method for measurement of fire line location and rate of spread (ROS) based on sequential UAS thermal maps. The method is validated using the October 2019 Welda KS prescribed fire data. 1.2 Developed data processing method for mean wind estimation at flame height, using UAS GPS/inertial/pressure measurements. 1.3 Simulated the prescribed fire event of Welda, Kansas in October 2019 using data collected from UASs and compared the simulation results with real observation data. 2. UAS path planning 2.1. The proposed path planning algorithm, partially observable Markov decision process (POMDP), has been tested for a simulated wildfire in which multiple UAVs can track multiple fire spots autonomously to maximize the sensing coverage. 2.2. Started to develop a new real time onboard path planning algorithm that allows a UAS to dynamically adjust its flying direction without relying on information from a ground station. The new algorithm uses real time data collected by the UAS to direct the UAS to monitor the perimeter of a wildfire with special attentions to the most active regions of the fire. 3. Evaluation 3.1 Flight test of KHawk fixed-wing UAS and Hexcopter for system test and wind estimation. 3.2Worked on the data collected from the last prescribed fire for validation of the proposed method.

Publications

Type: Journal Articles Status: Published Year Published: 2020 Citation: B. Jia and M. Xin, Data-Driven Enhanced Nonlinear Gaussian Filter, IEEE Transactions on Circuits and Systems II, 67(6), p. 1144 - 1148, 2020.
Type: Conference Papers and Presentations Status: Accepted Year Published: 2021 Citation: P. Shobeiry, M. Xin, X. Hu, and H. Chao, UAV Path Planning for Wildfire Tracking Using Partially Observable Markov Decision Process, AIAA SciTech Forum, Guidance, Navigation, and Control Conference, Accepted, 2020.
Type: Journal Articles Status: Accepted Year Published: 2021 Citation: P. Tian, H. Chao, M. Rhudy, J. Gross, and H. Wu, "Wind Sensing and Estimation Using Small Fixed-Wing UAVs: A Survey", AIAA Journal of Aerospace Information Systems (JAIS), accepted to appear, 2020.
Type: Conference Papers and Presentations Status: Accepted Year Published: 2021 Citation: J. Matt, H. Flanagan, and H. Chao, "Evaluation and Analysis of ArduPilot Automatic Tuning Algorithm for the Roll Tracking Controller of a Small UAS", AIAA SciTech Forum, Atmospheric Flight Mechanics Conference, Accepted, 2020.
Type: Conference Papers and Presentations Status: Published Year Published: 2020 Citation: H. Flanagan, H. Chao, and Y.Q. Chen, "Lateral Fractional Order Controller Design and Tuning for a Flying-Wing UAS", International Conference on Unmanned Aircraft Systems, 2020.
Type: Journal Articles Status: Submitted Year Published: 2021 Citation: X. Hu, M. Ge, S. Gowravaram, H. Chao, M. Xin, "Prescribed fire simulation using data from UAS-based sensing", International Journal of Wildland Fire, submitted, 2020
Type: Conference Papers and Presentations Status: Published Year Published: 2020 Citation: H. Le, X. Hu, "Extended Model Space Specification for Mobile Agent-based Systems to Support Automated Discovery of Simulation Models", Proc. 2020 Winter Simulation Conference, 2020

Progress 12/01/18 to 11/30/19

Outputs
Target Audience: Nothing Reported Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided?At GSU, 2 PhD students, 1 MS student, and 1 undergraduate student were supported and trained while working on this project. They learned knowledge in wildfire simulation and UAS path planning. At MU, 2 PhD students and 1 undergraduate student were supported and trained to build the UAS, learn the path planning algorithms, and do the simulations. At KU, 3 PhD students and 1 undergraduate student were supported and trained to design, build, and flight test KHawk 55-Themal UAS. They learned knowledge in fire and wind sensing as well as human-in-the-loop path planning for UAS fire monitoring. How have the results been disseminated to communities of interest?The research results have been disseminated in three ways: 1) Publications; 2) Conference presentations; 3) Media reports. Publications X. Hu, P. Wu, "A Data Assimilation Framework for Discrete Event Simulations", ACM Transactions on Modeling and Computer Simulation (TOMACS), 29(3), Article No. 17, July 2019 B. Jia, and M. Xin, "Data-Driven Enhanced Nonlinear Gaussian Filter," IEEE Transactions on Circuits and Systems II, Accepted in 2019 and in press 2020. X. Hu, J. Bent, J. Sun, "Wildfire Monitoring with Uneven Importance Using Multiple Unmanned Aircraft Systems", Proc. 2019 International Conference on Unmanned Aircraft Systems (ICUAS'19), 2019 H. Flanagan, H. Chao, and S.G. Hagerott, "Model Based Roll Controller Tuning and Frequency Domain Analysis for a Flying-Wing UAS", International Conference on Unmanned Aircraft Systems, 2019. Conference presentations H. Flanagan, H. Chao, and S.G. Hagerott, "Model Based Roll Controller Tuning and Frequency Domain Analysis for a Flying-Wing UAS", International Conference on Unmanned Aircraft Systems, 2019. X. Hu, J. Bent, J. Sun, "Wildfire Monitoring with Uneven Importance Using Multiple Unmanned Aircraft Systems", Proc. 2019 International Conference on Unmanned Aircraft Systems (ICUAS'19), 2019 Media Reports 1. "GSU professor developing drones to help fight wildfires", WSB-TV Channel 2 News, March 25, 2019. https://www.wsbtv.com/news/local/atlanta/gsu-professor-developing-drones-to-help-fight-wildfires/934043900/ 2. "Drones Provide Eye-in-the-Sky to Help Fight Fires," U.S. Department of Agriculture Blog, Aug 15, 2019 1. https://www.usda.gov/media/blog/2019/07/09/drones-provide-eye-sky-help-fight-fires 3. "The High-Tech Future of Firefighting", Georgia State University Research Magazine, Spring 2019 Issue, pp. 6-7, 2019 1. https://news.gsu.edu/research-magazine/spring2019/the-high-tech-future-of-firefighting 4. "Researcher Leads Project To Explore Use Of Drones To Fight Wildfires", Georgia State University News, January 23, 2019 1. https://news.gsu.edu/2019/01/23/georgia-state-grant-to-study-using-drones-to-fight-wildfires/ 5. "Project Looks Into How Drones Can Predict Spread of Wildfire", Associated Press, January 20, 2019 1. https://apnews.com/3546526d7ce34881a92df0688268e3a4 6. "Aerospace engineering faculty, students create drone to detect wildfires", The University Daily Kansan, Feb. 6, 2019. http://www.kansan.com/news/aerospace-engineering-faculty-students-create-drone-to-detect-wildfires/article_f866c2dc-28bb-11e9-8e46-6ffa9a9dbb58.html 7. "MU professor developing integrated drones to predict spread of wildfires", Missourian, Jan. 12, 2019. https://www.columbiamissourian.com/news/higher_education/mu-professor-developing-integrated-drones-to-predict-spread-of-wildfires/article_e520bf78-1387-11e9-a487-f70294d3109e.html?utm_source=Email%20Newsletters&utm_campaign=Daily%20Edition&utm_medium=Email&utm_content=headline What do you plan to do during the next reporting period to accomplish the goals?During the next reporting period, we plan to do the following tasks: Support real time fire situation awareness by developing methods to process image data (RGB, Thermal, NIR) and location data in real time. Enable two-way communication and data transfer between flying UASs and the base station (a laptop) so that UASs can be controlled through a path planning algorithm instead of being controlled manually. More comprehensive simulation study of a selected prescribed fire using data collected from UAS. This would prepare the simulation model for data assimilation using UAS data. Develop data assimilation method that works with UAS-detected fire location data. Develop a robust path planning algorithm to support autonomous path planning for fire front tracking using a single UAS and/or multiple UASs. Organize the outreach workshop in Year 2 of the project to foster collaborations between research and UAS application in wildfire management.

Impacts
What was accomplished under these goals? The deadly and catastrophic California wildfires in 2017 and the recent Australia wildfire signify the urgent need of advanced technologies and tools for operational wildfire spread prediction and situation awareness for firefighters and people in and near wildfire areas. Accurate prediction of wildfire spread for active burning wildfires is critical for effective and safe wildfire management. However, the lack of real time wildfire and wind data, both of which change in space and time, makes it difficult to achieve operational wildfire spread prediction. This project will develop innovative research that can transform wildfire management by enabling operational wildfire spread prediction and situation awareness for firefighters using a team of unmanned aircraft systems (UASs). The interdisciplinary nature and prior experiences of the research team provide great potential for producing fruitful outcomes that will benefit this important societal issue of wildfire management. The team's collaboration with KBS on field tests over prescribed fires will enable validation and transfer of the resulting knowledge to real wildfire management.The results of this research will also have great impact on other civilian and defense emergency response applications where humans and UASs increasingly work together.Under the four major goals of this project, the following has been accomplished in this report period. 1. Fire sensing, wind estimation, and data assimilation to enable data driven wildfire simulation. 1.1 Developed initial UAS based fire line sensing system, which is guided by a human operator. Multiple fire RGB/NIR maps have been generated at preset time intervals after each flight test. 1.2 Developed a 9-state-Extended Kalman Filter which uses UAS GPS/inertial/pressure measurements for mean wind estimation at the flame height. 1.3 Carried out simulations for the prescribed fire of Welda, Kansas in October 2019 using data collected from UASs and obtained preliminary results. 1.4 Published a journal paper that summarized the Particle Filter-based Data Assimilation method for discrete event systems. The data assimilation method will be used in this project to assimilate UAS data into the wildfire spread simulation model. 2. UAS path planning 2.1. Developed a multi-UAS path planning algorithm for fire front monitoring. The new algorithm takes into consideration the uneven spreading rates of different segments of a fire perimeter when dividing the fire monitoring task. 2.2. The proposed path planning algorithm, partially observable Markov decision process (POMDP), has been tested for some simulation scenarios in which multiple UAVs can track multiple moving targets. 3. Human-UAS collaboration 3.1. Directed an undergraduate student to develop a proof-of-concept mobile App demonstrating a human-UAS interaction interface. 4. Evaluation 4.1 Two flight sessions have been performed over two prescribed fire burning in Kansas for the test of the developed fire and wind sensing system. Fire maps and mean wind can be estimated offline.

Publications

Type: Journal Articles Status: Awaiting Publication Year Published: 2020 Citation: Jia, B., and Xin, M., Data-Driven Enhanced Nonlinear Gaussian Filter, IEEE Transactions on Circuits and Systems II, Accepted in 2019 and in Press, 2020.