Progress 07/15/24 to 07/14/25
Outputs
Target Audience:The target audiences reached during this reporting period include the project's Advisory Board, academics (university/college faculty and researchers), graduate students in environmental economics, scientists, and government agencies including USDA, along with their staff. Changes/Problems:
Nothing Reported
What opportunities for training and professional development has the project provided?The project is providing training for a mid-career research scientist at Clark University, along with providing skillset development opportunities at ICF for early and mid-career staff. How have the results been disseminated to communities of interest?Initial results of LLFs for valuation meta-analysis and BT have been presented at multiple conferences and invited seminars in both the US and internationally. Presentations to date include: Johnston, R.J. and K. Moeltner. 2024. Advancing the Frontier of Data Synthesis for Environmental Benefit Transfer: From Globally Linear Meta-Regression to Local Linear Forests. Environmental Economics Seminar, Yale School of the Environment, New Haven, CT, November 20. Johnston, R.J. and K. Moeltner. 2025. Random Forests for Benefit Transfer. 30th Annual Meeting of the European Association of Environmental and Resource Economists, Bergen, Norway, June 16-19. Johnston, R.J. and K. Moeltner. 2025. Advancing the Frontier of Data Synthesis for Environmental Benefit Transfer: From Meta-Regression to Local Linear Forests. Association of Environmental and Resource Economists Summer Conference, Santa Ana Pueblo, NM, May 28 - 30. Johnston, R.J. and K. Moeltner. 2025. Advancing the Frontier of Data Synthesis for Environmental Benefit Transfer: From Globally Linear Meta-Regression to Local Linear Forests. Society for Benefit-Cost Analysis 2025 Annual Conference, Washington, DC, March 13-14. What do you plan to do during the next reporting period to accomplish the goals?Work during the next reporting period will include tasks necessary to complete Objectives 1 - 5. We intend to complete the final steps required for Objective 1, finalizing all theory and procedures necessary to guide model implementation, metadata development and spatial-salience survey design. We will continue to develop, pretest and implement the novel, nationwide, push-to-web spatial salience survey of randomly selected households across the continental US (Objective 2). Using the integrated survey, geospatial and socioeconomic data, we will design and implement a ML spatial salience classification model (SCM) to predict, for every CT nationwide, the degree to which any HUC 8 watershed in the conterminous US is considered to be relatively salient (or is prioritized) by CT residents for water quality improvements (Objective 3). The next task will retrospectively update the PDs extant metadata on per household WTP for water quality improvements to incorporate information on the extent to which the improvements that were valued by each underlying, primary study observation in the metadata occurred in salient or non-salient watersheds, for the originally sampled households, as predicted by the SCM (Objective 4). This new set of spatial salience variables, calculated for each metadata source observation, will support an augmented specification of the meta-analysis via LLF. This extended model will predict per household benefits as a function of the degree to which spatially heterogeneous water quality improvements in the considered policy scenario occur in home, salient or non-salient areas for households living in each CT within the BT market area, in addition to effects of the wide array of additional MRM variables (Objective 5).
Impacts
What was accomplished under these goals?
An in-person project kick-off meeting was held in Reston, VA on February 27-28, with all senior personnel in attendance. Among other meeting objectives, a comprehensive plan of work was developed for project tasks over the three-year project timeframe. Based on workplans developed during this meeting, tasks during the first project year emphasized tasks required to complete Objectives 1, 2, 3 and 4. We have now largely completed Objective 1--developing the foundational theory and analytical methods required to integrate novel methods for valuation meta-analysis (applied to per household willingness to pay (WTP) for water quality improvements) with spatial salience classifications derived using machine learning (ML) algorithms and household survey data. The integrated approach will support spatially explicit benefit transfers (BTs) of water quality benefits from heterogeneous resource conservation over large spatial scales. The first component of this work was to finalize the theory and corresponding variable definitions required to update the research team's existing valuation metadata on water quality values with spatial salience information. As detailed in the original proposal, we will update the metadata with variables that quantify the "salience" of affected waters, defined as the extent to which the water quality improvements considered within each primary study (within the metadata) occur within "home," "salient" or "ordinary" bodies of water, for the originally sampled households. We hypothesize that the salience of these affected waters will influence households' WTP for water quality improvements. We anticipate that incorporation of this information within valuation meta-analysis will allow more accurate WTP prediction and thus more accurate BT. As an initial step towards these proposed models, formal definitions of the proposed spatial salience variables were required, allowing these variables to be calculated for metadata observations. To summarize, each metadata observation includes a per household WTP estimate for a water quality scenario that reflects improvements to waters within a set of hydrologic unit code (HUC) 8 watersheds. We denote these improved watersheds as the "affected watershed area" or AWA. Each WTP estimate was produced using primary study data drawn from an original stated preference survey that sampled a population within a given sampled area (or SA). For each of these primary-study observations, we first identify all Census tracts (CTs) in the SA and HUC 8 watersheds contained in its SA, as well as all watersheds included in its AWA. We also determine the share of the sampled population from the original study for whom a given HUC 8 watershed (covered by the study's valuation scenario) is a "home HUC" (where they live). Using nationwide household survey data to be collected under Objective 2 (see below), we will then predict whether each HUC 8 watershed covered by each observation's AWA is predicted to be "salient" (or important) for corresponding CT residents, when considering water quality improvements. This salience information will be used to determine what proportion of improved miles or area are home, salient, and ordinary (non-salient and non-home) for the observation's SA population. Grounded in the formal definitions and procedures designed during the first project year, these novel spatial salience variables will be derived via implementation of a spatial salience questionnaire (Objective 2) integrated with a novel application of ML methods for spatial salience classification and prediction (Objective 3). We will then update the underlying metadata with the newly derived spatial-salience variables (Objective 4), allowing estimation of an extended meta-analysis that incorporates this new information (Objective 5). The meta-analysis will thus account for the extent to which spatially heterogeneous water quality improvements due to conservation occur to water bodies that are salient or non-salient to households. We also made progress towards completion of Objective 2, developing initial plans for the large-sample spatial salience survey that will elicit data on the degree to which US households value, or view as "salient," potential water quality improvements in different HUC 8 watersheds nationwide. This survey design extends methods developed by the PDs for another ongoing USDA AFRI project, Partnership: Next Generation Choice Experiment Architecture for Spatially Explicit Agricultural Conservation and Ecosystem Service Valuation (#2022-67023-36735), which develops questionnaire architecture to elicit spatial salience information statewide, with an application to Virginia. The current project will adapt this architecture for larger-scale, nationwide US applications. Within this new online questionnaire, respondents will be asked, in a guided fashion and via map interactions, to identify HUCs of special importance across the US mainland for water body uses and water quality improvements, e.g., due to recreational preferences, family ties, or for other reasons. The interface will present each respondent with an interactive map of the contiguous US, overlaid with HUC 8 boundaries of AWA (showing water bodies, key places, etc.) that allow survey respondents to pan and zoom to any desired location, as well as search for features using a natural language search tool, and then indicate the salience of the watershed containing these features and explain the reasons for salience. To support predictions of the extent to which watersheds are salient to respondents (Objective 3), the survey data will be linked to supporting information from publicly available, nationwide geospatial and socioeconomic data layers. Examples of these secondary watershed and population data summarized during the first year included watershed and individual land use area, length of streams, area of waterbodies, number of unique native fish species, area of public land, average number of cloudy days per year, average number of clear days per year, average depth of precipitation per year, area of ecoregions, number of drinking water wells and intakes, public health statistics, average air quality, length of impaired stream segments, and presence of a coastal beach. Progress has also been made towards Objective 5, which will specify and implement the meta-analysis of per household WTP for water quality improvements. We have designed the underlying econometric model and code that will be used to implement the proposed modeling and subsequent BT predictions of changes in water quality value from expected improvements. The approach is grounded in meta-analysis implemented via Local Linear Forests (LLFs), a novel approach to data-synthetic BT that allows the metadata to be processed in a manner that enables more efficient, accurate and straightforward BT value predictions, compared to current best-practice meta-regression models (MRMs). As applied to valuation meta-analysis, LLFs are essentially a hybrid approach that combines elements of Random Forests (RFs) and locally weighted meta-regression models (LW-MRM). Initial evaluations suggest that this new approach substantially improves BT accuracy without sacrificing theoretic properties, while reducing econometric and computational difficulties relative to leading alternatives. For example, we find that forest-based models substantially improve the within-sample accuracy of welfare predictions and tighten confidence intervals of predicted benefits for out-of-sample transfers. Simultaneously, these models avoid the implementation challenges of complex, regression-based approaches such as LW-MRM.
Publications
|