Source: OKLAHOMA STATE UNIVERSITY submitted to
DSFAS: A GLOBALLY AVAILABLE, LOCALLY TAILORED CROP EVAPOTRANSPIRATION FORECAST TOOL
Sponsoring Institution
National Institute of Food and Agriculture
Project Status
ACTIVE
Funding Source
AFRI COMPETITIVE GRANT
Reporting Frequency
Annual
Accession No.
1030694
Grant No.
2023-67022-40019
Cumulative Award Amt.
$295,661.00
Proposal No.
2022-11582
Multistate No.
(N/A)
Project Start Date
Aug 15, 2023
Project End Date
Aug 14, 2025
Grant Year
2023
Program Code
[A1541]- Food and Agriculture Cyberinformatics and Tools
Project Director
Sadler, J.
Recipient Organization
OKLAHOMA STATE UNIVERSITY
(N/A)
STILLWATER,OK 74078
Performing Department
(N/A)
Non Technical Summary
Evapotranspiration (ET) information is critical for climate-smart irrigation practices, but local crop ET forecasts are currently unavailable for most farmers. The overall objective of this project is to develop and deliver a proof-of-concept framework for globally available short-term crop ET forecasts. These forecasts will be tailored to local conditions using information from farmer-provided smartphone images. The project will build from the free Canopeo app already used worldwide. The new proof-of-concept tool will use Canopeo images and metadata in conjunction with globally available weather forecasts and novel deep-learning models. Weather forecasts will be used to forecast reference ET, which will be adjusted to estimate the ET of the farmer's particular crop (e.g., via a crop coefficient) using an established method informed by the percent canopy cover measured by Canopeo. To further tailor ET estimates to the soil moisture conditions (i.e., estimate the water stress coefficient) is more difficult because there is no establishedscalable method for estimating local soil moisture in irrigated crops. To fill this gap, a novel deep-learning method will be developed. This method will estimate the water stress coefficient based on recent weather and crop (and soil) conditions as reflected in the mobile phone images. The forecasts will be delivered to beta testers and their feedback will be used to refine the crop ET forecast system. Beta testers will come from the 30k+ Canopeo users worldwide and from Oklahoma irrigators engaged through the Oklahoma State University Extension network.
Animal Health Component
100%
Research Effort Categories
Basic
(N/A)
Applied
100%
Developmental
(N/A)
Classification

Knowledge Area (KA)Subject of Investigation (SOI)Field of Science (FOS)Percent
11172102050100%
Knowledge Area
111 - Conservation and Efficient Use of Water;

Subject Of Investigation
7210 - Remote sensing equipment and technology;

Field Of Science
2050 - Hydrology;
Goals / Objectives
The overall goalof this project is todevelopanddelivera proof-of-concept model for globally available crop ET forecasts based on smartphone images and tailored to local soil moisture conditions using deep learning (DL).To accomplish our overall goal, we will focus on three objectives:Extend existing methods to forecast site-specific crop ET for well-watered conditions anywhere globally. Develop a deep learning model to estimate crop water stress based on recent weather and images provided by the farmer.Deliver the proof-of-concept model and collect feedback.
Project Methods
Reference ET is the estimated ET that would occur from a reference crop (grass or alfalfa) in ideal growing conditions (well-watered and relative humidity above 45%). Reference ET is adjusted based on weather conditions including relative humidity, wind speed, and crop height. The crop coefficient is the ratio between the crop ET and the reference ET and further adjusts the reference ET to reflect the crop-specific water usage at the crop's given growth stage and level of canopy development. Finally, the ET estimate is adjusted by the water stress coefficient. When the crop is water stressed, the crop water stress coefficient drops below 1, and crop ET is reduced.Objective 1:Extend existing methods to forecast site-specific crop ET for well-watered conditions anywhere globally.EffortsWe plan to accomplish the objective for this aim in the following steps. Each of these steps will be prototyped locally first offline then they will be transitioned to a cloud environment for use in beta testing.Develop a data pipeline for retrieving globally available weather forecasts to forecast reference ET.We will use NOAA's global forecasting system (GFS) to obtain weather forecasts needed for forecasting reference ET. GFS forecasts include temperature, wind speed, relative humidity, and cloud cover which are needed to compute reference ET.Implement the A&P method to estimate crop coefficient.To estimate the crop coefficient, we will use the proven and versatile A&P method which models crop coefficient based on percent canopy cover and the crop height. Both percent canopy cover and crop height will be supplied by participating users through the Canopeo app.Combine the reference ET and crop coefficient to produce a site-specific crop ET forecast for well-watered conditions.We will develop and deliver, via automated e-mails, 7-day forecasts as an alpha version of the Crop ET Forecasts. The forecast time window can be adjusted from 1 to 16 days based on user feedback.EvaluationThe efforts of Objective 1 will be evaluated based on the ability of the tool to produce a reference ET forecast and tailor that forecast based on the crop coefficient. We will also evaluate this based on feedback from ET forecast users.Objective 2:Develop a deep learning model to estimate crop water stress based onrecent weather and images provided by the farmer.EffortsIn this objective, we will iteratively develop novel deep-learning frameworks to estimate the soil moisture conditions from recent weather and images provided by farmers in theCanopeomobile app. We propose the followingtwo-foldcontributions in this proposal.Deep Learning Model to Quantify Crop Water Stress in Images: We propose an end-to-end deep learning framework that provides an initial estimate of crop water stress directly from raw images. The images from the Canopeo app often include crop canopy, crop residue, and soil. Thus, the prediction model involves two sub-tasks: (i) Identify soil and crop canopy in raw images, and (ii) Estimate the water stress from these image segments. We propose a multi-scale object detection-based deep learning model [22, 23, 24] calledMap-Soilto identify different object types like plant, soil, residue, shadows, and other unknown objects. Importantly, the proposed model will utilizeResNet, a pre-trained object detection model, as a backbone architecture and it will also include Graph Neural Networks (GNNs) to capture robust global representations of images from their pixels in addition to their local features. OurMap-Soilmodel will identify bounding polygons around image objects like soil and leaves which will be further used for the water stress prediction task. For that task, we will build on the gradient boosting decision tree (GBDT), which was able to distinguish three levels of water stress in maize (full irrigation, deficit irrigation, and drought stress). By first segmenting the soil and crop portions of the images, we expect to be able to provide a more detailed categorization and quantification of the crop water stress. Team members' existing datasets with thousands of soil moisture observations and collocated Canopeo images will be used to train and evaluate the proposed model. We will evaluate our model predictions with machine learning metrics like accuracy, precision, recall, F1, and mean absolute error.Knowledge-Infused Hindcast Model:The existing work, use only the image details to estimate water stress. To advance the state of the science, we propose to refine those estimates via a Knowledge-Infused modelto hindcast soil moisture. This model will, for example, consider temporal information collected from external sources like recent (14-day) local weather patterns, soil characteristics, and potentially past photographs (if available) to hindcast soil moisture. Random Forests, XGBoost regression, and Long Short-term Memory approaches have all shown good potential for soil moisture hindcasts and will be explored here.EvaluationThe efforts of Objective 2 will be evaluated usingthe error in the model that is used to predict soil moisture from crop photographs.Objective 3:Deliver the proof-of-concept model and collect feedbackEffortsWe plan to accomplish the objective for this through the following steps.Deliver forecasts to interested Canopeo users.Starting with its first iteration (Objective 1), we will offer the crop ET forecasts to the existing Canopeo user base. We will ask for their feedback about the usefulness of the forecasts and the forecast delivery. Since the users providing this feedback could be from anywhere on the globe, this feedback will be collected in a scalable way (e.g., an online survey).Deliver to Oklahoma Irrigators via OSU Extension. We will educate and deliver the model to irrigators in Oklahoma through the OSU Extension Master Irrigator class. The content will be delivered by Dr. Sharma and a graduate student. At these meetings, the process of using the tool will be explained and feedback will be gathered.Deliver to rural Oklahoma producers.An undergraduate participant in OSU's Rural Scholar's program will spend 10 weeks during the summer in rural Oklahoma communities working with irrigators and training them to use the crop ET forecast tool. This student will also collect feedback about the forecasts from the producers. OSU Extension educators in each county will help this Scholar connect with local irrigators.EvaluationThe efforts of Objective 3 will be evaluated based on the number of users (both global and from Oklahoma) that use the crop ET forecast tool. We will also evaluate the efforts based on the number of users we are able to collect feedback.

Progress 08/15/23 to 08/14/24

Outputs
Target Audience:The target audience that we reached during this reporting period were Oklahoma agricultural producers producers. These producers were mainly in the southwest portion of the state of Oklahoma - a region that is frequently in drought conditions. Changes/Problems:A major challenge is some technical difficulties that we are currently experiencing with the Canopeo App. One of the assumptions of this project was that the current Canopeo app functionaliy would continue to be functional and that the app would be available for download on both Android and iOS devices. We however, did not take into account sufficiently the ever present cost of maintaining any software including this app, and the Canopeo app, as a academic-based product, does not currenly have a consistent funding source. The Canopeo app is currently not available for download on the Google Play Store. This means that Android users that do not currently have the Canopeo app cannot currently download and install the app. The app is, however, available on the Apple App Store, so iPhone users can download the app. This could be a considerable barrier because it means that, currently, unless the producer has an iPhone or they already have the app installed, the producers cannot use the new functionaliy and provide feedback. Given these challenges, we may have to make some adjustments in the budget to try to resolve these issues, or we may have to take a different approach to getting feedback. We still are on track to finish the additional functionality that we originally proposed, however, the issues with the Canopeo app may reduce our ability to collect feedback from producers. What opportunities for training and professional development has the project provided?This project has been the primary source of funding for 2 graduate students: Mamata Pandey (a Biosystems Engineering graduate student) and Mohammed Rakib (a Computer Science graduate student). Both of these students have been supported by this project to attend both regional and international research conferences. Mamata presented her work at the Oklahoma Governor's Water conference and presented at the American Society of Agricultural and Biosystems Engineers (ASABE) Annual International Meeting (AIM) in Anaheim, CA. Mohammed presented at the Data Science and Advanced Analytics conference in San Diego, CA. Jeff Sadler (PD) was able to attend the program PD Meeting that was combined with the Precision Agriculture and Agricultural Engineering programs in Manhattan, KS. This was a good opportunity for Dr. Sadler to make connections with other PD's throughout the US. In addition to these meetings, a major professional development opportunity was Mamata's participation in the Oklahoma St. University's Rural Scholar program. The program included a graduate level class on qualitative research and community development. In this class, Mamata was required to design and develop a survey instrument for data collection and get IRB approval. The main part of the program, however, was staying in the rural community for 10 weeks over the summer. This was a unique professional development opportuinty in which Mamata was able to build relationships in the community, perform service, and learn a little about life in rural America, in addition, of course, to performing her research which was related to this project in which she made connections with agricultural producers who are willing to provide us feedback on the ET forecast tool that we are developing. How have the results been disseminated to communities of interest?Most of the results are still in progress and have not been disseminated yet. We have published a paper on the deep learning models for predicting VWC which is available online. What do you plan to do during the next reporting period to accomplish the goals?Goal 1: Extend existing methods to forecast site-specific crop ET for well-watered conditions anywhere globally. The plan for the next reporting period is to finish the implementation of the crop coefficient part of the workflow. Goal 2: Develop a deep learning model to estimate crop water stress based on recent weather and images provided by the farmer. The plan for the next reporting period is to explore the use of satellite data for improving soil moisture (VWC) predictions. Goal 3: Deliver the proof-of-concept model and collect feedback. The plan for the next reporting period is to present to agricultural producers about our tool at the Master Irrigator program in the spring. We also plan to identify beta users from the Master Irrigator progam and follow up with the producers already identified through the Rural Scholar program. We will orient these users on the Canopeo mobile app and the ET forecasts functionality and collect their feedback.

Impacts
What was accomplished under these goals? Goal 1:Extend existing methods to forecast site-specific crop ET for well-watered conditions anywhere globally. We have made significant progress in this goal. To accomplish this, we have extended thefunctionality of the existingCanopeo mobile app. With our work in this reporting period the functionality has been added so thatbeta users receive forecasts of the reference ET when they upload photos of their crop in the Canopeo app. The forecast is based on the users geolocation that is collected from the phone's GPS and recorded (with the users permission) via the Canopeo app. Since this functionality only required geolocation, it can be applied anywhere in the world. The reference ET, which is based on weather data, is fetched from the Open-Meteoweb service. Currently only thereference ET(estimated ET of a reference crop (i.e., alfalfa) under well watered conditions) is delivered to the user.We are still working on scaling that value to the crop using the crop coefficient. This functionalityhas been implemented in Amazon Web Services. In this way, physical compute resources did not have to be purchased and we only pay for what we use. Up to this point, our usage has been small enough that it has been free. This work is currently being described in a draft manuscript. Goal 2: Develop a deep learning model to estimate crop water stress based on recent weather and images provided by the farmer. We have made signficant progress in this goal. We have developed machine learning and deep learning models that estimate volumetric water content (VWC) in the soil based on coincident top-down crop photos. The data are from automated observing stations where weather data, soil moisture data (through a neutron probe), and the crop photos are taken at regular intervals. We have tried two approaches to accomplish our goal. In Approach 1 we used two steps: in step 1, the model identified the soil patches; in step 2 the model used the pixels identified to be part of soil patches as input to predict the VWC. In Approach 2 we used the entire photo as input to predict VWC. We also explored using just meteorological data, just the photo, or a combination of the two to predict VWC. The best performing model was to use Approach 2 (all of the photo) and just the photo data and not the meteorological data. This approach produced a mean absolute percent error of 3.8%. The model that we used for this was a fine-tuned version of the MobileNetV2 convolutional neural network (CNN). Goal 3: Deliver the proof-of-concept model and collect feedback. The main progress in this area is connections with agricultural producers that we have established in two main ways. First, one of the graduate students, Mamata Pandey, on the project was an Oklahoma St. University Rural Scholar over the past summer. As a Rural Scholar, she lived in a Rural Community for 10 weeks and in that time she developed relationships with members of the community including agricultural producers. Mamata's project was to conduct interviews to learn the water concerns of the agricultural producers and to find potential beta users and feedback providers for the ET forecast tool that we are developing. She was able to interview 24 agricultural producers in the summer and several of them were interested in trying out the ET forecast tool and providing feedback. Second, Dr. Sumit Sharma (co-pi) has established an advisory committee of agricultural producers that can provide us feedback on the direction of the project.

Publications

  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Rakib, M., Mohammed, A. A., Diggins, C., Sharma, S., Sadler, J. M., Ochsner, T., & Bagavathi, A. (2024). MIS-ME: A Multi-modal Framework for Soil Moisture Estimation. Data Science and Advanced Analytics. arXiv preprint arXiv:2408.00963. (In Press) IEEE International Conference on Data Science and Advanced Analytics.
  • Type: Conference Papers and Presentations Status: Accepted Year Published: 2024 Citation: Automating Evapotranspiration Forecasts Through the Canopeo App and Serverless Cloud Computing. Congress of the International Environmental Modelling and Software Society (iEMSs). June 2024. East Lansing, MI, USA.
  • Type: Conference Papers and Presentations Status: Accepted Year Published: 2024 Citation: Opportunities of Artificial Intelligence in Water Resources Extension. American Society of Biosystems and Agricultural Engineers (ASABE) Annual International Meeting (AIM). July 2024. Anaheim, CA, USA.
  • Type: Conference Papers and Presentations Status: Accepted Year Published: 2024 Citation: Challenges and Potential Solutions for Forecasting Reference Crop ET Globally?. American Society of Biosystems and Agricultural Engineers (ASABE) Annual International Meeting (AIM). July 2024. Anaheim, CA, USA.