Source: ARIZONA STATE UNIVERSITY submitted to NRP
USING FIELD LEVEL SOIL QUALITY DATA FOR CROP INSURANCE: A BIG DATA SIMULATION AND CREDIBILITY APPROACH TO IMPROVE CROP INSURANCE PRICING AND AGRICULTURAL LAND SUSTAINABILITY PRACTICE
Sponsoring Institution
National Institute of Food and Agriculture
Project Status
COMPLETE
Funding Source
Reporting Frequency
Annual
Accession No.
1025086
Grant No.
2021-67023-33818
Cumulative Award Amt.
$498,935.00
Proposal No.
2020-06355
Multistate No.
(N/A)
Project Start Date
Jan 1, 2021
Project End Date
Jun 30, 2024
Grant Year
2021
Program Code
[A1641]- Agriculture Economics and Rural Communities: Markets and Trade
Recipient Organization
ARIZONA STATE UNIVERSITY
660 S MILL AVE STE 312
TEMPE,AZ 85281-3670
Performing Department
WPC: Agribusiness, Morrison Sc
Non Technical Summary
This project proposes an improved crop insurance premium pricing method that uses soil information and big weather data to increase premium pricing accuracy. Crop insurance may be improved by an improved liability setting (i.e. setting the insurable yield) procedure in which the expected farm yield for the upcoming growing season is adjusted for field-level soil effects. Also, crop insurance may be improved by providing premium discounts or surcharges to incentivize sustainable production practices and disincentivize crop expansion on marginal lands.The extent of crop insurance premium pricing accuracy improvement that can be gained by using soil and weather information will be investigated. The proposed premium pricing method uses premium rates simulated at the farm that account for soil effects and the RMA premium rate that is derived from the farm's yield history and county loss history. A credibility approach is used to optimally weight the two derived premium rates, to obtain a more accurate premium rate.The proposed big data approach requires field-level crop yield data, large weather data sets, and soil information that is measured on a 30m by 30m grid for the continental USA. Most of this data is publicly available. This method would not require a large program design change to implement and may be a cost-effective way to benefit from field level soil information.
Animal Health Component
100%
Research Effort Categories
Basic
(N/A)
Applied
100%
Developmental
(N/A)
Classification

Knowledge Area (KA)Subject of Investigation (SOI)Field of Science (FOS)Percent
60160303010100%
Goals / Objectives
The federal crop insurance program (FCIP) is the most extensive direct support program for US agricultural producers, providing approximately $100 billion in crop insurance coverage. The program is structured as a private-public partnership, in which the USDA's Risk Management Agency (RMA) prices, regulates and administers the program and private insurers deliver the insurance. Crop insurance helps producers manage price and yield variability caused by market shocks, droughts, floods, disease, pest infestation, and other perils. The FCIP began in 1938 and has grown in size and popularity since the 1990s (Glauber, 2013). In general, it has surpassed traditional farm support programs in federal outlays. Crop insurance may become more important in the future as uncertainty regarding future weather events and crop price variability may increase. Crop insurance faces the challenge of adverse selection and moral hazard, and improving policy design may reduce these risks. A large body of research has examined and critiqued the FCIP program, and many of these suggestions have been adopted in design improvements (Woodard and Verteramo-Chiu, 2017; Rejesus et al., 2015; Coble et al., 2013; Finger, 2013; Woodard et al., 2011; Skees and Reed, 1986). Future efforts to improve FCIP efficiency are essential as mispriced and poorly designed crop insurance may lead to negative externalities (Adhikari et al., 2012).Over the past several years, the general premium rate-making structure of the APH and revenue policies have stayed relatively the same. Reference premium rates are developed for the county, and farm premium rates are adjusted according to a rating schedule based on the farm's actual production history. The pricing structure adjusts premium rates relative to expected farm yields for the crop type and risk classification. Due to crop rotation, the producer's expected yield and yield risk may change year to year based on the soil quality of the fields where the crops are grown or if the producer puts more land into production. For example, a producer may grow soybeans on high-quality land one year and grow soybeans on low-quality land the next year. The FCIP does not account for the change in risk of increasing the soybeans on lower quality land the second year. The difference in soil quality may also affect the expected yield, which is used for setting the yield guarantee (liability). Similarly, if a producer puts marginal land into production, the crop insurance premium may not reflect the increased yield risk. Due to these soil quality effects, there may exist an inefficiency in liability determination and rate-setting, leading to premium mispricing.Mispriced crop insurance can affect the producer's decision making and management practices and may lead to adverse environmental outcomes such as acreage expansion on marginal lands. If producers grew only one crop, it would be relatively simple to adjust premium rates relative to a soil quality score to provide a disincentive for expansion on marginal lands, and soil quality's effect on expected yield would be present in historical yields. However, because crops are grown in rotation, it is more challenging to make this adjustment as the soil quality score must be determined each year based on the field units that the crop is grown.Following the 2008 Farm Bill, the RMA started collecting field-level crop yield data geographically matched to field locations. With big data technologies from data-science and geographic information systems, there is an opportunity to quantify the effect of soil on crop yield risk. Agriculture is well suited to take advantage of large geolocated data sets and applied computing technologies, and the FCIP may benefit from improved efficiency (Coble et al., 2018). Woodard and Verteramo-Chiu (2017) and Li et al. (2016), investigated the crop insurance premium pricing efficiency impacts of using soil information. They found that including soil information significantly improved crop insurance premium pricing through improved liability setting. Following these results, they suggested that the RMA's premium pricing approach be redesigned to price premiums at the field level using soil information. They expect that soil information may improve crop insurance more in areas with higher soil heterogeneity, such as the western states. Completely redesigning the premium rate making approach to incorporate soil may be a significant undertaking that may lose key actuarial information from past loss experience. However, much of the pricing accuracy improvements may be gained by weighting a simulated premium price at the farm level with the price derived from the current premium rate-making methodology and may not require significant program changes.The goal of the research is to develop an improved premium pricing method for the federal crop insurance program (FCIP) that considers the effect of soil quality. To this end, the study will examine a big data simulation and credibility-based crop insurance pricing method that uses producers' actual production history and field-level soil quality for the upcoming growing season. The extent of possible mispricing will be examined, such as the percentage of farms affected by mispricing, and potential factors associated with mispricings such as geographic location or other factors like weather.The existing crop insurance rate-setting method uses actual farm-level yields but does not use field-level soil information. Field level soil information may be used to improve crop insurance by increasing the accuracy of liability setting (i.e., setting the insurable yield) by using expected yield estimates for the upcoming growing season that are adjusted for field-level soil effects. Field level soil information may also be used to provide premium discounts or surcharges to incentivize sustainable agricultural practices and disincentivize crop expansion on marginal lands.Since 2009, specific field-level boundary information has been collected by the RMA and may be useful for improving crop insurance. The proposed big data approach uses simulated premium rates that incorporate field-level soil quality and are simulated using the estimated weather (e.g., temperature and precipitation) at the farm location since 1895. This simulated premium price is then weighted with the premium price derived from the current rate-making method using a modified credibility approach. The proposed premium rate making method may improve on existing approaches by providing additional information by using field-level soil information for premium pricing.The rationale behind the proposed project is to improve the premium rate, making the accuracy of the FCIP. Mispriced crop insurance can affect a producer's decision making and management practices. It may also lead to producers insuring their high-risk crops and not insuring their low-risk crops (adverse selection). These effects reduce the efficiency of the FCIP, which affects taxpayers and producers.
Project Methods
Credibility theory is used in many insurance applications. The most significant accuracy credibility theory applies a statistical model approach to solve the credibility weighting problem. The problem arises in insurance if an insureds risk experience is consistently better (worse) than the underlying risk pool. They may want their risk-adjusted to reflect this lower risk level. The question for insurers is, "How credible is the insureds risk experience?" This is an essential question for insurers as accurate premium pricing is critical to both customer acquisition and loss reserving (Klugman et al., 2012). To solve this problem, the premium rate derived from the risk classification, called the manual rate, must be appropriately weighted with the premium derived from the insured's loss experience. Several methods are used to estimate the weight parameters, including both frequentist and Bayesian estimation methods (Pai et al., 2015). The Bühlmann credibility model, developed by Bühlmann (1967) approximated by a linear formula of the past data, and the parameters are selected to minimize a squared error loss function. Conditional on the underwriting parameters, the past losses are assumed independent and identically distributed and have the same mean and variance.Methods: Proposed Simulation and Credibility Premium Pricing ApproachThe current crop insurance premium rate-making procedure adjusts for farm-level yields. However, it does not adjust for the next growing season's change in soil quality. Farm crop yields are simulated to account for this effect. Crop yields are simulated at the given farm, holding soil quality corresponding to next year's growing season fixed, and a farm premium rate is derived. This premium rate is then weighted with the original premium rate quoted by the RMA for a given coverage level. The method follows a 5-step procedure.Step 1: Estimate an aggregate level crop yield model that accounts for the effects of soil qualityFirst, estimate a crop yield model on the pooled farm-level yield data set that models the effects of weather and change in soil quality. The weather variables may include temperature, precipitation, drought indices, and their respective quadratic terms. The National Climate Data Center provides gridded monthly weather values are extracted at the geo-located field unit since 1895. Alternatively, daily values can be extracted since 1980. The weather values correspond to a gridded weather product's grid cell and are interpolated to best approximate the weather at that location from a network of weather stations. The effect of soil quality is approximated using a farm soil score, based on the gSSURGO database 30m resolution soil map linked to the corresponding VALU1 table soil scores. The soil scores are extracted at the field unit boundaries and aggregated to the common land unit (CLU). The extracted values of the weather and soil score variables are then weighted to the farm-level based on field unit size.Step 2: Downscale the model from Step 1 and simulate crop yields using weather estimated at the farm since 1895, holding the effects of soil quality constantThe model estimated in Step 1 is then downscaled to the farm level and used to simulate yields holding the soil score for the upcoming growing season constant while varying the weather values. The soil score is based on the field unit mix expected in the coming year. The yields are simulated, holding soil quality constant. Simultaneously, the values of weather are varied based on the history of estimated weather at the farm since 1895 or 1980, depending on the weather data used. This simulation is run 500 times to account for variation introduced from the error term. The frequency distribution derived from the simulation represents the empirical yield distribution that may have occurred at the farm holding the field mix constant (i.e., keeping the soil quality constant).Step 3: From the simulated crop yields derive the actuarially fair premium rateFrom the simulated crop yields, the pure premium rate is derived, which is the expected lossat a given coverage level λ, absent of any cost loading. Following from Goodwin and Mahul(2004), the expected insured loss is the probability of a loss multiplied by the expected loss. More formally, the expected insured loss is determined using a maximum likelihood procedure to fit the parameters of an assumed distribution. Several distributions have been proposed by researchers, including the gamma, beta, and normal (Goodwin and Mahul, 2004; Sherrick et al., 2004; Ramirez et al., 2003; Just and Weninger, 1999). Alternatively, non-parametric methods can be used to estimate yield density (Ozaki et al., 2008; Ker and Goodwin, 2000). This process results in the pure premium rate that does not include the loadings for administration costs and other expenses. The value of the pure premium rate is sensitive to the assumed distribution, and a sensitivity analysis is necessary to evaluate the effects of the specification assumption. The pure premium rate plus the loading rate gives the simulated premium rate.Step 4: Weight the premium rate from Step 3 with the RMA premium rate quoted for the farm, using a modified credibility approachSimilar to the credibility problem outlined above, the optimal weighting of the unsubsidized simulated premium and the RMA premium rate is needed that best reflects the true hypothesized mean. Recall from the Buhlmann model as discussed above as approximated by a linear specification of the past data, and weighted based on the process variance and the mean-variance. For our problem, instead of using the data to linearly approximate andthen derive the pure premium rate, we can use the RMA premium rate as the true hypothesized premium rate, and then approximate it using a linear combination of the simulated premium and the RMA premium rate from the previous year.Step 5: Evaluate the accuracy of the proposed method by an out of sample testing procedure that uses the next years RMA quoted premium rate as the true premium rateAn out of sample evaluation is then conducted to determine the efficiency gains that the proposed credibility premium method would have resulted in if it was used in place of the current method. The crop yield data is split into training and testing subsets, and the weight parameter is fitted using the training data set.

Progress 01/01/21 to 06/30/24

Outputs
Target Audience:The target audiences for this project research are agricultural producers, rural residents, farm and rural appraisers and agribusiness, governmental organizations, and the agricultural/rural lending sector. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided?The project has allowed the postdoctoral fellow to continue expanding their expertise in agricultural insurance. The master's student from MSU has also benefited from being exposed to and trained in R programming, cloud data technologies such as Google Cloud Platform, and risk and insurance. A postdoctoral fellow with training work experience found a job. The Master's student graduated from SU and found a career of his choice. How have the results been disseminated to communities of interest?In the first year, preliminary results of one of the academic article outputs were presented at the NC-1177 regional meeting. Project results will be presented in a track session at the AAEA Applied Risk Analysis conference in 2022. 1. AAEA presentation: "More Than Mean Effects: Modeling the Effect of Soil Quality on Higher-Order Moments of Crop Yields" Authors: Rylan Moore, Ardian Harri, Keith Coble, Xiaofei Li, Mitchell Roznik. 2.Roznik, R., and Ashok K. Mishra. "Estimating the Effect of Temporal Changes in Soil Related Crop Productivity Index on Crop Yield Risk." In review, American Journal of Agricultural Economics. 3.Roznik, M., and Ashok K. Mishra. "Developing a Soil Quality-Adjusted Crop Insurance Policy Using Field-Level Data." Selected Presentation at the 2022 AAEA Annual Meeting, Anaheim, CA, July 31-Aug. 2, 2022. What do you plan to do during the next reporting period to accomplish the goals? Nothing Reported

Impacts
What was accomplished under these goals? In the first year of the project the team have focused on developing the data required for analysis, presenting preliminary analysis, and writing initial drafts of manuscripts. There were six outputs in 2021. 1) A U.S. weather and satellite image data series was created and warehoused in a cloud database. The data was prepared so that it could be easily accessed by collaborators and their graduate students. The raw data series was hundreds of terabytes and required distributed cloud computing. 2) Preliminary results of a related research paper were presented at the NC-1177 regional meeting. 3) A track session for AAEA Applied Risk Analysis was proposed for the 2022 annual meeting entitled Investigating the Relationship Between Crop Risk and Soil Quality. Several papers from the project will be presented at the annual meeting. 4) A MSU master's student has been prepared and assigned to the project. 5) The MSU master student, with the help of the ASU postdoctoral fellow, has begun the literature review and initial exploratory analysis using the weather and satellite data. 6) Monthly meetings with the ASU and MSU teams have been conducted to address roadblocks and assess progress. 7)A graduate student was also funded for 24 months. The two co-PIs and the graduate student worked on a research project that investigated the significance of soil quality in modeling crop yield distributions along with its implications for the premium ratemaking methodology employed by the Risk Management Agency (RMA) for the area crop insurance programs.

Publications


    Progress 01/01/23 to 12/31/23

    Outputs
    Target Audience: Nothing Reported Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided?The project has provided me with time to participate in academic research opportunities with researchers in other universities. It has provided me with gaining newer knowledge in economic theory development and estimation technique by working with graduate students. How have the results been disseminated to communities of interest?Results from the studies within the project have been disseminated through a variety of outlets. Together with my graduate students and fellow researchers we have presented research findings at professional meetings. What do you plan to do during the next reporting period to accomplish the goals?Finish the project int he next six months.

    Impacts
    What was accomplished under these goals? The results of these projects have been several outlets. 1. Published one thesis. 2. Published one preview reviewed jorunal article and submbmited one article for publication consideration. 3. Presented two papers at at the 2022 AAEA Annual Meeting, Anaheim, CA, July 31-Aug. 2, 2022. At Mississippi State Univeristy, two co-PIs were funded, each for 2.5 months. A graduate student was also funded for 24 months. The two co-PIs and the graduate student worked on a research project that investigated the significance of soil quality in modeling crop yield distributions along with its implications for the premium ratemaking methodology employed by the Risk Management Agency (RMA) for the area crop insurance programs. At Arizona State University (ASU) one facutly was sunded by 0.2 months and one post-doctoral fellow.

    Publications

    • Type: Conference Papers and Presentations Status: Other Year Published: 2022 Citation: Roznik, M., and Ashok K. Mishra. Developing a Soil Quality-Adjusted Crop Insurance Policy Using Field-Level Data. Selected Presentation at the 2022 AAEA Annual Meeting, Anaheim, CA, July 31-Aug. 2, 2022.
    • Type: Conference Papers and Presentations Status: Other Year Published: 2022 Citation: Moore, R. More Than Mean Effects: Modeling the Effect of Soil Quality on Higher-Order Moments of Crop Yields. Selected Presentation at the 2022 AAEA Annual Meeting, Anaheim, CA, July 31-Aug. 2, 2022.
    • Type: Journal Articles Status: Submitted Year Published: 2023 Citation: Roznik, M., and Ashok K. Mishra. Estimating the Effect of Temporal Changes in Soil Related Crop Productivity Index on Crop Yield Risk.
    • Type: Theses/Dissertations Status: Submitted Year Published: 2023 Citation: Moore, Rylan, "Utilizing soil quality data for premium rate making in the federal crop insurance program" (2023). Theses and Dissertations. 5946. https://scholarsjunction.msstate.edu/td/5946
    • Type: Journal Articles Status: Published Year Published: 2023 Citation: Roznik, M., Ashok K. Mishra, M. Boyd. Using a Machine Learning Approach and Big Data to Augment WASDE Forecasts: Empirical Evidence from US Corn Yield. Journal of Forecasting, Vol. 42(6), 2023: 1370-1384.


    Progress 01/01/22 to 12/31/22

    Outputs
    Target Audience:The target audience for this research project in the second year are policymakers and academics. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided?The project has provided the postdoctoral fellow the time to continue to expand their expertise in agricultural insurance. The master's student from MSU has also benefited by being exposed and trained in R programming, cloud data technologies such as Google Cloud Platform, and risk and insurance. How have the results been disseminated to communities of interest?In the second year, preliminary results of two of the academic article outputs were presented at the AAEA annual meeting. A track session at the AAEA Applied Risk Analysis conference in 2022 was hosted to show project results and related work. What do you plan to do during the next reporting period to accomplish the goals?The next steps of the project are to continue to improve the econometric modeling and complete and submit manuscripts to academic journals. Once the manuscripts are drafted, selected papers will be presented at the 2023 AAEA annual meeting for feedback. Then the manuscripts will then be submitted to academic journals for publication.

    Impacts
    What was accomplished under these goals? In the second year of the project, the team focused on adjusting and improving the analysis, collecting supplementary data sources, and writing drafts of the manuscripts. There were five outputs in 2022. 1) A farm-level yield data set with growing degree days, precipitation, and soil attribute measured at the pixel level. 2) A county-level yield, weather, and soil data set. 3) A track session at the AAEA annual meeting entitled "Investigating the Relationship Between Crop Risk and Soil Quality" was held, and two presentations showed preliminary results attributed to the project. 4) An MSU master's student is preparing a master's thesis using data prepared by the ASU postdoctoral fellow. 5) Monthly meetings with ASU and MSU teams have been conducted to assess progress and address roadblocks. 6) Presented paper at the SCC-76 annual meetings.

    Publications


      Progress 01/01/21 to 03/16/22

      Outputs
      Target Audience:The target audiences for this project research are agricultural producers, rural residents, farm and rural appraisers and agribusiness, governmental organizations, and the agricultural/rural lending sector. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided?The project has provided the postdoctoral fellow time to continue to expand their expertise in agricultural insurance. The master student from MSU has also benefited by being exposed and trained on R programming, cloud data technologies such as Google Cloud Platform, and risk and insurance. How have the results been disseminated to communities of interest?In the first-year, preliminary results of one of the academic article outputs were presented at the NC-1177 regional meeting. A track session at the AAEA Applied Risk Analysis conference in 2022 has been scheduled for presenting project results. What do you plan to do during the next reporting period to accomplish the goals?The next steps of the project are to complete the actuarial modeling and finish drafting manuscripts. Once the manuscripts are drafted, selected papers will be presented at the AAEA Applied Risk Analysis conference and others will be presented and shared to other outlets. The manuscripts will then be submitted to academic journals for publication.

      Impacts
      What was accomplished under these goals? In the first year of the project the team have focused on developing the data required for analysis, presenting preliminary analysis, and writing initial drafts of manuscripts. There were six outputs in 2021. 1) A U.S. weather and satellite image data series was created and warehoused in a cloud database. The data was prepared so that it could be easily accessed by collaborators and their graduate students. The raw data series was hundreds of terabytes and required distributed cloud computing. 2) Preliminary results of a related research paper were presented at the NC-1177 regional meeting. 3) A track session for AAEA Applied Risk Analysis was proposed for the 2022 annual meeting entitled: Investigating the Relationship Between Crop Risk and Soil Quality. Several papers from the project will be presented at the annual meeting. 4) A MSU masters student has been prepared and assigned to the project. 5) The MSU master student with the help of the ASU postdoctoral fellow have begun the literature review and initial exploratory analysis using the weather and satellite data. 6) Monthly meetings with the ASU and MSU teams have been conducted to address roadblocks and assess progress.

      Publications


        Progress 01/01/21 to 12/31/21

        Outputs
        Target Audience:The target audiences for this project are researchers (academics) agricultural producers, RMA, governmental organizations, risk management companies, policymakers and the agricultural/rural lending sector. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided?The project has provided the postdoctoral fellow time to continue to expand their expertise in agricultural insurance. The master student from MSU has also benefited by being exposed and trained on R programming, cloud data technologies such as Google Cloud Platform, and risk and insurance. How have the results been disseminated to communities of interest?In the first-year, preliminary results of one of the academic article outputs were presented at the NC-1177 regional meeting. A track session at the AAEA Applied Risk Analysis conference in 2022 has been scheduled for presenting project results. What do you plan to do during the next reporting period to accomplish the goals?The next steps of the project are to complete the actuarial modeling and finish drafting manuscripts. Once the manuscripts are drafted, selected papers will be presented at the AAEA Applied Risk Analysis conference and others will be presented and shared to other outlets. The manuscripts will then be submitted to academic journals for publication.

        Impacts
        What was accomplished under these goals? In the first year of the project the team have focused on developing the data required for analysis, presenting preliminary analysis, and writing initial drafts of manuscripts. There were six outputs in 2021. 1) A U.S. weather and satellite image data series was created and warehoused in a cloud database. The data was prepared so that it could be easily accessed by collaborators and their graduate students. The raw data series was hundreds of terabytes and required distributed cloud computing. 2) Preliminary results of a related research paper were presented at the NC-1177 regional meeting. 3) A track session for AAEA Applied Risk Analysis was proposed for the 2022 annual meeting entitled: Investigating the Relationship Between Crop Risk and Soil Quality. Several papers from the project will be presented at the annual meeting. 4) A MSU masters student has been prepared and assigned to the project. 5) The MSU master student with the help of the ASU postdoctoral fellow have begun the literature review and initial exploratory analysis using the weather and satellite data. 6) Monthly meetings with the ASU and MSU teams have been conducted to address roadblocks and assess progress.

        Publications