Progress 08/01/18 to 03/31/20
Outputs
Target Audience: We have spoken about our mortality mapping results to multiple CA state Agencies including CALFIRE (forest and wildfire), CA Natural Resources Agency, and CalTrans (transportation). Each of these state agencies has responsibility over land or right-of-way management, and having access to regular tree mortality monitoring is of high interest. We had conversations with the US Forest Service Region 5 scientists, forest supervisors, and leadership staff. We formed collaborations with top university scientists in California who work on forest mortality, including Brandon Collins (UC Berkeley) and John Battles (UC Berkeley), and have provided them with sample data. We held outreach discussions with conservation organizations like The Nature Conservancy and Sierra Nevada Conservancy who either directly manage land or provide funding to land managers to improve forest stewardship. We were invited to present our California forest mortality mapping work to the annual meeting of the California Forest Pest Council (November 13-14, 2019). We presented the results of our mortality mapping and methods to around 50 attendees, including the lead Region 5 USFS Aerial Detection Survey specialist. We hosted an intern during summer 2019. This intern, a masters student in the Environmental Observation and Informatics program at the University of Wisconsin-Madison, worked on improving our model training data collection procedure while learning more about classifying airborne and satellite imagery to detect ecological patterns. Changes/Problems:There were no major delays or disruptions in our research plan. However, we decided to extend our research timeline under a no-cost extension to allow continued model experimentation to maximize the accuracy of our deep learning algorithms to distinguish tree mortality from living trees. What opportunities for training and professional development has the project provided?We provided training to two field assistants during the project. They accompanied the PI and Co-I on a week-long field campaign to data on individual tree mortality. We trained them in using GPS-enabled tablets to geolocate individual trees on satellite imagery, assess their status, and identify their species. We also hosted an intern at Salo Sciences during May-August 2019. The intern was a masters student from University of Wisconsin-Madison, in the Environmental Observation and Informatics program. The intern worked on improving our methods to produce training data for model training, while learning to process large volumes of airborne and satellite imagery using the Google Earth Engine platform in order to detect ecological patterns. Both the PI and Co-I were invited to and attended the CA Forest Pest Council annual meeting. This professional development opportunity was significant for the company since it was our first public presentation of our results to a core audience for tree mortality mapping data. How have the results been disseminated to communities of interest?We have disseminated our results to multiple state agencies in California, USFS scientists and leadership staff, local and national NGOs, the CA Forest Pest Council, and many university scientists. Our dissemination mainly takes the form of individual outreach to key staff at each organization through in-person or video meetings where we share a formal presentation of our methods and results. We have shared sample data with scientists from University of California Berkeley who are experts in forest mortality. What do you plan to do during the next reporting period to accomplish the goals?
Nothing Reported
Impacts
What was accomplished under these goals?
Western US forest ecosystems are experiencing severe stress from drought, heat, fires, and pest outbreaks. Tree mortality is widespread across the region and is expected to increase as these stresses shift in geographic range and intensity. Higher fire frequency and intensity, resulting from increasing rates of tree mortality, are expected to increase economic burdens on a wide range of institutions, including natural resource managers, firefighters, state and federal agencies, and homeowners. Yet, these stakeholders currently rely on coarse resolution maps to quantify and monitor regional tree mortality. In this project, Salo Sciences built and tested a system for forest mortality monitoring at previously inaccessible spatial and temporal scales by combining daily, high resolution satellite imagery with deep learning algorithms and extensive field observations. Our comprehensive (or "wall-to-wall") maps of mortality at the individual tree scale can be used to improve the coordination of forest management activities through precisely identifying areas facing the greatest fire hazard, optimizing timber salvage strategies, and forecasting future risks. Our final model performed well, with a 69% accuracy score in distinguishing live trees from dead trees. This is comparable to the accuracy of USFS aerial detection surveys, which boast >=70% accuracy for stand-level mortality, though we classified at the tree scale, not at the stand scale. True accuracy of our system is very likely higher due to the time difference between our validation field work (2018) and the satellite imagery date (2016). We see some clear opportunities to improve our approach, such as including imagery from multiple years, and better tuning our input training datasets, which we plan to implement as we refine our product development pipeline. While there is room for improvement, we believe this work demonstrates the feasibility of satellite-based dead tree detection, which could become a central element of any forest management planning and monitoring system. We've performed preliminary testing to apply this model to the whole state of California (salo.ai/demos/mortality), and we're now writing a manuscript for a peer-reviewed journal to share this work. We received two significant opportunities due to the outcomes and outreach associated with this project. First, we were invited to join a consortium of companies, universities, and non-profits that submitted and won a competitive proposal for a large California Energy Commission grant to develop a next-generation wildfire risk model that accounts for extreme tree mortality. Second, the software capacity developed during Phase I allowed us to demonstrate the application of deep learning to derive new ecological insights, such as mapping tree height. This ultimately led to our successful grant application to the Gordon and Betty Moore Foundation to build the California Forest Observatory--a real-time forest fuels and wildfire hazard mapping system launching late summer 2020. Objective 1: Develop a stratified geographic sampling scheme to sample imagery from the varied ecosystems and land use types across California California is home to a diverse array of forested ecosystems, each with gradients of forest structure, elevation, and species groups. To generalize across systems, pattern recognition algorithms should be trained using imagery spanning these gradients. We built software that randomly samples image tiles in equal proportions across a range of gradients as defined by the user. The user defines the sampling scheme by providing a categorical geospatial dataset--like a land use map, ecoregion map or a soil map--and it samples image tiles evenly across each class to ensure equal representation. These tiles are used as inputs into our deep learning model. Objective 2: Collect a robust set of field observations based on the stratified geographic sampling scheme to train a deep learning model To gather field data, we completed a data collection campaign in October 2018. The PI and Co-I, along with two field assistants, mapped the locations of individual dead tree stems using a GPS system linked to satellite imagery on handheld tablets, identifying individuals to the species level when possible. In total, we mapped the locations of 4,820 trees over 175 acres, spanning an elevation gradient from 4,500 ft to 7,800 feet. While our original objective was to collect field data for algorithm training, we soon discovered that even a few thousand tree locations would not be sufficient to train the complex deep learning models we were using. Instead, we used these field observations as independent model validation data. To generate data for model training, we developed a semi-supervised, clustering-based method to classify dead trees in high resolution imagery. Our goal was to generate a map of bare ground, live trees, and dead trees. For our basemap imagery we used high resolution airborne multispectral data from the USDA's National Agriculture Imagery Program (NAIP). We used a clustering algorithm to identify areas that may contain tree mortality, and a custom tree detection model to filter out only those areas that actually contain trees (rather than misidentified areas of bare ground). Based on photo interpretation, we viewed this as a conservative estimate of tree mortality--the tree detection model would occasionally omit dead trees and instead classify them as ground. Objective 3: Standardize image pre-processing steps, such as scaling RGB values, calculating spectral indices (such as normalized ratios between bands), and performing noise removal In early testing we found we didn't need to develop several pre-processing tasks. Following a series of model cross-validation experiments, we found no statistical difference in model performance between the transformed and non-transformed datasets. Instead, we focused on developing a standardized image gridding system and a high-throughput image loading / data normalization pipeline to facilitate rapid image processing in a cloud computing environment. The gridding system reprojects and aligns data from multiple image sources into a standardized grid system, which can be sampled using the software described in Objective 1. This system outputs consistent and spatially indexed imagery, ensuring perfect overlap between response and covariate datasets. These grids are designed for consistent indexing and alignment to feed paired image tiles into the dataloader. This reduced memory overhead to allow much larger training datasets to be processed. Objective 4: Optimize deep learning architecture and model hyperparameters We used a grid search approach to compare model architecture and hyperparameter settings. We evaluated model performance using cross-validation on the training data. For our [bare ground, live tree, dead tree] U-Net model, we computed balanced accuracy scores of 0.70, weighted precision scores of 0.80, weighted recall scores of 0.65, and weighted F1 scores of 0.69. Using the 4,820 dead tree crowns as independent validation data, we computed accuracy scores of 0.69, precision scores of 0.60, and recall scores of 0.45. We suspect these scores are lower than they might be, however, due to the difference between the image collection date (Sep. 2016) and the field data collection date (Oct. 2018). Nearly 45 million trees died during that two year span, mostly in areas previously impacted by severe mortality. The low recall scores are likely driven by the trees that died during this period but were not dead at the time of image acquisition, flagging them as false negatives.
Publications
|
Progress 08/01/18 to 07/31/19
Outputs
Target Audience:We made consistent outreach efforts to a range of potential end-users throughout the reporting period. State agencies in California we have spoken to include CALFIRE (forest and wildfire), CA Natural Resources Agency, and CalTrans (transportation). Each of these state agencies has responsibility over land or right-of-way management, and having access to regular tree mortality monitoring is of high interest. We also had conversations with the US Forest Service Region 5 scientists, forest supervisors, and leadership. We held outreach discussions with conservation organizations like The Nature Conservancy and Sierra Nevada Conservancy whom either directly manage land or provide funding to land managers to improve forest stewardship. While not in the reporting period, but occurring in just two weeks (November 13-14, 2019), we were invited to present our California forest mortality mapping work to the annual meeting of the California Forest Pest Council. Changes/Problems:There were no major delays or disruptions in our research plan. However, we decided to extend our research timeline (see Accomplishments) to allow continued model experimentation to maximize the accuracy of our deep learning algorithms to distinguish tree mortality from living trees. Instead of the original project end date of March 31, 2019, we are targeting a project end date of January 31, 2020. What opportunities for training and professional development has the project provided?By engaging with stakeholders in the academic, non-profit, and government agency communities and demonstrating our mortality mapping capacity, we received two significant opportunities. First, we were invited to join a consortium of companies, universities, and non-profits to submit an (ultimately successful) competitive proposal for a large California Energy Commission grant to develop a next-generation wildfire risk model that accounts for extreme tree mortality. Second, the software capacity we gained during our Phase I project allowed us to demonstrate the application of deep learning to derive new ecological insights, such as mapping tree height. This ultimately led to our successful grant application to the Gordon and Betty Moore Foundation to build the California Forest Observatory--a real-time forest fuels and wildfire hazard mapping system. How have the results been disseminated to communities of interest?
Nothing Reported
What do you plan to do during the next reporting period to accomplish the goals?
Nothing Reported
Impacts
What was accomplished under these goals?
Western US forest ecosystems are experiencing severe stress from drought, heat, fires, and pest outbreaks. Forest dieback, driven by high rates of tree mortality, is widespread across the and is expected to increase as these stresses shift in geographic range and intensity. Higher fire frequency and intensity, resulting from increasing rates of tree mortality, are expected to increase economic burdens on a wide range of institutions, including natural resource managers, firefighters, state and federal agencies, and homeowners. In this project, Salo Sciences built and tested a system for forest mortality monitoring at previously inaccessible scales by combining daily, high resolution satellite imagery with deep learning algorithms and extensive field observations. Our comprehensive (or "wall-to-wall") maps of mortality at the individual tree scale can be used to improve the coordination of forest management activities through precisely identifying areas facing the greatest fire hazard, optimizing timber salvage strategies, and forecasting future risks. Objective 1. Development of a stratified geographic sampling scheme, designed to sample imagery from the varied ecosystems and land use types across California, to capture a wide range of image variability for training a deep learning model. California has a diverse set of forest ecosystems with varying gradients of ecological, phenological, and morphological properties. In order for our AI algorithm to accurately map tree mortality anywhere in the state, it must be able to distinguish dead trees from the variations in these properties. To achieve this generalizability, the algorithm must be trained with satellite imagery from across these gradients. We built sampling software that randomly samples image tiles in equal proportions across a range of environmental patterns, as defined by the user. The software can aggregate tiles from multiple input sources for cases where we use more than a single source of satellite imagery or other datasets. These image tiles are then used as inputs into our mortality mapping AI algorithm. Objective 2. Collection of a robust set of field observations based on the stratified geographic sampling scheme to train a deep learning model. Field observations are central to validating the algorithm outputs. Field-verified data of tree mortality will be used to assess the accuracy of the final tree mortality maps produced by the algorithm. To gather enough field data, we completed a large field data collection campaign in October 2018. The PI and Co-I, along with 2 field assistants, mapped the location of individual dead tree stems using a GPS system linked to satellite imagery on handheld tablets. We identified each tree species when possible, and collected data on the approximate time since death. In total, we mapped more than 5,000 trees over 175 acres across multiple environmental gradients. While our original objective was to collect field data for algorithm training, these data are more appropriate for algorithm output validation. Objective 3. Standardization of image pre-processing steps, such as scaling RGB values, calculating indices (such as normalized ratios between bands), and performing noise removal. Preliminary research found we did not need to develop some preprocessing tasks, like atmospheric correction, since the corrections implemented by the data provider, Planet, were of high quality. Additionally, testing has shown that our algorithms are robust to the inter-scene and inter-sensor variation in Planet data. This reduces the need for noise removal and whole scene normalization, eliminating a huge pre-processing burden. Instead we focused on developing a standardized gridding system and a high-throughput image loading / data normalization pipeline to facilitate rapid image processing in a cloud computing environment. The gridding system reprojects and aligns data from multiple image sources into a standardized grid system. This system outputs consistent and spatially indexed imagery, ensuring perfect overlap between datasets. The image grids are designed to seamlessly feed image tiles into the dataloader. The dataloader in turn feeds images into the algorithm given a list of tile paths. By reading in the images real-time, we reduce memory overhead to allow much larger training datasets to be processed. The dataloader also performs per-tile image normalization on-the-fly, reducing the number of pre-processing steps and allowing flexibility in how and when we perform normalization. Objective 4. Optimization of the deep learning algorithm, including selection of algorithm architecture(s), image input size, and learning rate strategy. This objective forms the core of our work under Phase I. We have identified the appropriate family of convolutional neural network (CNN) architectures to test and developed the software to test different iterations of model structure, inputs, and model hyperparameters. As part of this testing phase, we experimented with varying image size and resolutions, combinations of image sensor data, loss functions, and model depths. Our results showed that we could achieve accuracies ranging from 70-85% in distinguishing tree mortality. However, our insights gained from the experimentation lead us to believe we can achieve much higher accuracy (approaching 95%). We received a no-cost extension of our Phase I project and continue these experiments to maximize the accuracy of our algorithm.
Publications
|