Source: KANSAS STATE UNIV submitted to NRP
DEVELOP CANOPY SENSING AND COMPUTATIONAL SYSTEMS FOR REAL-TIME CONTROL AND FEEDBACK OF IRRIGATION TECHNOLOGY
Sponsoring Institution
National Institute of Food and Agriculture
Project Status
COMPLETE
Funding Source
Reporting Frequency
Annual
Accession No.
1011679
Grant No.
2017-67007-26153
Cumulative Award Amt.
$499,968.00
Proposal No.
2016-11496
Multistate No.
(N/A)
Project Start Date
Feb 1, 2017
Project End Date
Jan 31, 2022
Grant Year
2017
Program Code
[A7302]- Cyber-Physical Systems
Recipient Organization
KANSAS STATE UNIV
(N/A)
MANHATTAN,KS 66506
Performing Department
Biological & Ag Engineering
Non Technical Summary
Irrigated farms in U.S uses approximately 71 million acre-ft of water for crop productions from the diminishing natural water resources with Ogallala aquifer being an example for Great Plains region. Significant ongoing research in these states provides better tools and management methods to irrigators. Although most research has focused on single point approaches, limited research has determined spatial characteristics of water use. The goal of proposed study is to assess canopy temperature using sensing suite aboard small unmanned aircraft vehicle (sUAV) on dense spatial and temporal scale for precision irrigation strategies. Our hypothesis is that a small sUAV sensor suite containing thermal infrared (TIR) and visible-near infrared (VIS-NIR) cameras can record aerial imagery to create high resolution spatial canopy temperature maps. sUAV sensor suite (TIR/VIS-NIR camera, and high accuracy image localization and orientation sensing and data acquisition system) will measure canopy temperature during four critical growth stages of corn crop. Computational image analysis toolbox will implement image segmentations and mosaicking. Spatial canopy temperature along with meteorological parameters measured using ground reference system will generate spatial crop water stress maps (CWSM). Plant water stress indicators such as stomatal conductance and leaf water potential will be measured and correlated to CWSM. Expected outcome of this project will be a sUAS sensor suite to create canopy temperature maps and safe and accurate implementation protocols. Understanding of this technology will provide high resolution spatial assessment of crop water stress, allowing producers to make water stress based variable rate irrigation decisions for sustainable crop production.
Animal Health Component
30%
Research Effort Categories
Basic
70%
Applied
30%
Developmental
(N/A)
Classification

Knowledge Area (KA)Subject of Investigation (SOI)Field of Science (FOS)Percent
4047210202060%
1110210310020%
2032499102020%
Goals / Objectives
The overall research goal is to automate crop water stress monitoring for efficient use of available irrigation water and enhance crop water productivity. To achieve the overall goal, three specific project objectives would be:Integrate and test sensors required to measure crop water stress under field conditions; develop a complimentary sensor suite for positioning and mapping; hardware for data acquisition; and aerial image analysis and mosaicking package (AIAMP) to develop spatial-temporal canopy temperature and water stress maps for irrigation scheduling.Evaluate and validate sUAV with a sensor suite to measure crop water stress during critical growth stages for corn by measuring spatial canopy temperature; develop correlations between spatial canopy temperature maps and crop water stress indicators by conducting ground-based measurements. Develop and evaluate protocols for safe and effective use of sUAV at low altitudes for use by producers, service providers, and other stakeholders using sUAV sensing systems for precision irrigation applications.
Project Methods
The proposed project will develop a sensor package for precise image positioning, an orientation and mapping of aerial images captured using uncooled thermal infrared (TIR) sensor and visible-near infrared (VIS-NIR) camera aboard small Unmanned Aerial Vehicle (sUAV) to accurately assess crop canopy temperature pertinent to water stress; develop ground reference system for automated camera calibration; develop rapid image processing toolbox to quantify high spatial and temporal plant water stress status, and generate plant stress based irrigation scheduling. Increased canopy temperature indicates stomata closure and decreased transpiration rate in the presence of water stress (Gonzáles-Dugo et al., 2005; Taghvaeian et al., 2013). Spatial canopy temperature (SCT) and normalized difference vegetative index (NDVI) maps will be generated using aerial images from TIR and VIS-NIR cameras. Meteorological parameters, such as air temperature, relative humidity, solar radiation, and wind speed, will be measured using a ground-based weather station. Metrological parameters and SCT maps can be used to compute plant water stress index. Ground validation will be conducted during this project by measuring leaf temperatures and air temperatures at selected locations. In addition, plant water stress indicators, specifically stomatal conductance (gs) and leaf water potential (ΨL), will be measured at similar locations to find correlation between canopy temperature variation and crop water stress (Limpus & Henderson, 2009). Preliminary results from research in the Department of Biological and Agricultural Engineering at Kansas State University demonstrated that lightweight uncooled thermal infrared (UTIR) and VIS-NIR (visible-near infrared) cameras, a total payload of <500 grams, can obtain high resolution thermal and visible imagery. Imaging systems, like the ones used during preliminary studies will be integrated onto sUAV capable of flying at low (<120 m) altitudes. The proposed sensing system will establish protocols to capture aerial imagery of plant canopy; communication and sensing systems (between ground and sUAV) for image positioning, orientation and thermometric calibration; computational and image analysis toolbox to compute water stress; control plan of irrigation system; and aerial imaging after irrigation will provide crop stress feedback to make robust decision support tools, thus directly addressing the CPS program research area encompassing Science and Technology of Cyber-Physical Systems.

Progress 02/01/17 to 01/31/22

Outputs
Target Audience:The target audience was specifically farmers, industry professionals, service providers, remote sensing professionals, manufacturers, graduate students, high school students and teachers, and faculty. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided? Developing and designing new board, and stencils; and integration of components. Testing new boards for proper functioning of hardware and software. The entire process of programming radios and testing multiple radios simultaneously accessing base station for RTK corrections, provided knowledge of GPS systems. The positioning and mapping sensor suite is of great significance for ground and aerial unmanned aerial vehicles. Safe setup of unmanned aerial systems, programming missions and safe integrating cameras for remote sensing. Field data collection provided knowledge and skills on accurate design of experiments, setting protocols for standardized crop parameters collection, utilizing sensors, and techniques for data storage for analysis Students also got training on use of modeling software to develop geo-orthomosaics, spatial analysis using GIS and conducting statistical analysis. How have the results been disseminated to communities of interest?Results were disseminated through participation in departmental events, international conferences and one-to-one interaction with stake holders. What do you plan to do during the next reporting period to accomplish the goals? Nothing Reported

Impacts
What was accomplished under these goals? During 2020, progress and accomplishments were achieved for goals 1 and 2. The sensor suite for positioning and mapping utilizing OEM GNSS receiver and nine-axis gyro was fully integrated with a custom board designed to record real-time data and also provide the necessary output to thermal infrared camera for geotagging. The final product was tested aboard a small unmanned aerial vehicle for data integrity and ease of integration for image geotagging. The board design was improved to make it a four-layer to include all the necessary hardware for functionality of the board and also to support all output pins. All functionalities were completed along with data availability for thermal cameras and a serial port for data compatibility to other system for applications like planting and spraying. Additionally, tests were also conducted to quantify the impact of the focal length of thermal infrared cameras and flying altitude on image quality and the ability to extract accurate canopy temperatures for rapid analysis on the corn crop. Tests conducted for camera focal length and flying altitude study was analyzed and the manuscript was submitted for review. Additionally, corn crop was planted on 4.5-acre plot. The plot was divided into 3 sub-plots to apply irrigation at 50%, 75% and 100% required levels. Thermal and color infrared images were captured during three flying sessions conducted during different growth stages. In addition, ground truth data on camera sensor response was collected from reference panels with known temperatures for image calibration. Environmental data like solar radiation, sky temperature, wind speed, humidity, and air temperature were collected to ascertain crop water stress. Crop-based water stress metrics like stem water potential were captured using a leaf porometer and pressure bomb. Additionally, stationary systems comprising infrared radiometer, solar radiation, humidity, and air temperature were installed at seven locations to collect season-long data. This data was utilized to develop non-water stress baseline curves to quantify crop water stress. The data collected was analyzed to develop strategies for canopy temperature extractions for corn crop; quantify correlations between canopy temperature and soil moisture; and canopy temperature. The manuscript substantiating canopy temperature-based crop water stress quantification was completed and submitted for review.

Publications

  • Type: Conference Papers and Presentations Status: Accepted Year Published: 2021 Citation: Gadhwal, M., A. Sharda, H. S. Sangha, D. Merwe. 2021. Impact of Camera Lens Angle and sUAS Flying Altitude on Spatial Corn Canopy Temperature Evaluation 2101187. ASABE-AIM, July 12-16th, 2021.
  • Type: Journal Articles Status: Submitted Year Published: 2022 Citation: Gadhwal, M., A. Sharda, H. S. Sangha, D. Merwe. 2022. Impact of Camera Lens Angle and sUAS Flying Altitude on Spatial Corn Canopy Temperature. Submitted to Computers and Electronics in Agriculture
  • Type: Journal Articles Status: Submitted Year Published: 2022 Citation: Gadhwal, M., A. Sharda, and D. Merwe. 2022. Crop water stress quantification from satellite, manned aircraft and small unmanned aerial imagery to develop water stress management zones
  • Type: Other Status: Published Year Published: 2022 Citation: Sangha, H., M. Gardhwal, A. Sharda. 2021. Thermal infrared Imaging System Usage for Crop Health Assessments. MF3561. Kansas State University Agricultural Experiment Station and Cooperative Extension Service.
  • Type: Journal Articles Status: Submitted Year Published: 2022 Citation: Gadhwal, M., and A. Sharda. 2022. Crop water stress quantification using thermal infrared imaging for irrigation water management. Submitted to Computers and Electronics in Agriculture


Progress 02/01/20 to 01/31/21

Outputs
Target Audience:The target audience was specifically farmers, industry professionals, service providers, remote sensing professionals, manufacturers, graduate students, high school students and teachers, and faculty. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided? Developing and designing new board, and stencils; and integration of components. Testing new boards for proper functioning of hardware and software. The entire process of programming radios and testing multiple radios simultaneously accessing base station for RTK corrections, provided knowledge of GPS systems. The positioning and mapping sensor suite is of great significance for ground and aerial unmanned aerial vehicles. Safe setup of unmanned aerial systems, programming missions and safe integrating cameras for remote sensing. Field data collection provided knowledge and skills on accurate design of experiments, setting protocols for standardized crop parameters collection, utilizing sensors, and techniques for data storage for analysis Students also got training on use of modeling software to develop geo-orthomosaics, spatial analysis using GIS and conducting statistical analysis. How have the results been disseminated to communities of interest?Results were disseminated through participation in departmental events, international conferences and one-to-one interaction with stakeholders. What do you plan to do during the next reporting period to accomplish the goals?In the Year-5, the research plan includes The graduate student is planning to take the qualifier exams, work on manuscripts and also conduct field experiments to collect Year-3 of field data. The data will include aerial imaging using thermal, color infrared, and color cameras. In addition, data including actual plant canopy temperature; environmental data including sky temperature, wind speed, air temperature; soil profile moisture; stem water potential, and crop yield will be collected. Safe flying protocols to collect standardized data will be developed for researchers to collect quality remote sensing data.

Impacts
What was accomplished under these goals? During 2020, progress and accomplishments were achieved for goals 1 and 2. The sensor suite for positioning and mapping utilizing OEM GNSS receiver and nine-axis gyro was fully integrated with a custom board designed to record real-time data and also provide the necessary output to thermal infrared camera for geotagging. The final product was tested aboard a small unmanned aerial vehicle for data integrity and ease of integration for image geotagging. The board design was improved to make it a four-layer to include all the necessary hardware for functionality of the board and also to support all output pins. This work is in progress and the goal is to complete this task in 2021, because of the challenges of COVID on parts and accessibility support staff. Additionally, tests were also conducted to quantify the impact of the focal length of thermal infrared cameras and flying altitude on image quality and the ability to extract accurate canopy temperatures for rapid analysis on the corn crop. Tests conducted for camera focal length and flying altitude study was analyzed and manuscript writing is in progress. Additionally, corn crop was planted on 4.5-acre plot. The plot was divided into 3 sub-plots to apply irrigation at 50%, 75% and 100% required levels. Thermal and color infrared images were captured during three flying sessions conducted during different growth stages. In addition, ground truth data on camera sensor response was collected from reference panels with known temperatures for image calibration. Environmental data like solar radiation, sky temperature, wind speed, humidity, and air temperature were collected to ascertain crop water stress. Crop-based water stress metrics like stem water potential were captured using a leaf porometer and pressure bomb. Additionally, stationary systems comprising infrared radiometer, solar radiation, humidity and air temperature were installed at seven locations to collect season-long data. This data was utilized to develop non-water stress baseline curves to quantify crop water stress. The data collected was analyzed to develop strategies for canopy temperature extractions for corn crop; quantify correlations between canopy temperature and soil moisture; and canopy temperature. The manuscript substantiating canopy temperature-based crop water stress quantification is in progress.

Publications

  • Type: Journal Articles Status: Published Year Published: 2020 Citation: 1. Hatton, N.*, A. Sharda, D. Merwe, and W. Schapaugh. 2020. Remote thermal infrared imaging for rapid screening of sudden death syndrome in soybean. Volume 178, 105738. Computers and Electronics in Agriculture. https://doi.org/10.1016/j.compag.2020.105738
  • Type: Journal Articles Status: Published Year Published: 2020 Citation: 2. Sangha, H. S.*, A. Sharda, L. Koch, G. Wang, and P. Prabhakar. 2020. Impact of camera focal length and sUAS flying altitude on spatial crop canopy temperature evaluation. Computers and Electronics in Agriculture. Vol. 172. https://doi.org/10.1016/j.compag.2020.105344


Progress 02/01/19 to 01/31/20

Outputs
Target Audience:The target audience was specifically farmers, industry professionals, service providers, remote sensing professionals, manufacturers, graduate students, high school students and teachers, and faculty. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided? Developing and designing new board, and stencils; and integration of components. Testing new boards for proper functioning of hardware and software. The entire process of programming radios and testing multiple radios simultaneously accessing base station for RTK corrections, provided knowledge of GPS systems. The positioning and mapping sensor suite is of great significance for ground and aerial unmanned aerial vehicles. Safe setup of unmanned aerial systems, programming missions and safe integrating cameras for remote sensing. Field data collection provided knowledge and skills on the accurate design of experiments, setting protocols for standardized crop parameters collection, utilizing sensors, and techniques for data storage for analysis Students also got training on use of modeling software to develop geo-orthomosaics, spatial analysis using GIS and conducting statistical analysis. How have the results been disseminated to communities of interest?Results were disseminated through participation in departmental events, international conferences, and one-to-one interaction with stakeholders. What do you plan to do during the next reporting period to accomplish the goals?In the Year-4, the research plan includes 1. Field experiments will be conducted for Year-4. The data will include aerial imaging using thermal, color infrared, and color cameras. In addition, data including actual plant canopy temperature; environmental data including sky temperature, wind speed, air temperature; soil profile moisture; stem water potential and crop yield will be collected. 2. Safe flying protocols to collect standardized data will be developed for researchers to collect quality remote sensing data.

Impacts
What was accomplished under these goals? During this year, progress and accomplishments were achieved for goals 1 and 2. The sensor suite for positioning and mapping utilizing OEM GNSS receiver and nine-axis gyro was fully integrated with a custom board designed to record real-time data and also provide the necessary output to thermal infrared camera for geotagging. The final product was tested aboard a small unmanned aerial vehicle for data integrity and ease of integration for image geotagging. The testing of the system indicated that software and hardware modification are needed to provide GPS data output Additionally, tests were also conducted to quantify the impact of the focal length of thermal infrared cameras and flying altitude on image quality and the ability to extract accurate canopy temperatures for rapid analysis on the corn crop. Tests were also conducted using broadband color infrared cameras to test a new vegetative index to quantify disease and water stress. Data collected was analyzed and a manuscript covering color infrared research has been submitted for consideration to computers and electronics in Agriculture and data is being analyzed for camera focal length and flying altitude study. This study will help producers and remote sensing users to select cameras for corn and results would provide direction on how to analyze data accurately and rapidly. Additionally corn crop was planted on 4.5 acre plot. The plot was divided into 3 sub-plots to apply irrigation at 50%, 75% and 100% required levels. Thermal and color infrared images were captured during three flying sessions conducted during different growth stages. In addition, ground truth data on camera sensor response was collected from reference panels with known temperatures for image calibration. Environmental data like solar radiation, sky temperature, wind speed, humidity, and air temperature were collected to ascertain crop water stress. Crop based water stress metrics like stem water potential were captured using a leaf porometer and pressure bomb. The data collected was analyzed to develop strategies for canopy temperature extractions for corn crop; quantify correlations between canopy temperature and soil moisture; and canopy temperature

Publications

  • Type: Journal Articles Status: Submitted Year Published: 2019 Citation: Sangha, H. S.*, A. Sharda, L. Koch, G. Wang, and P. Prabhakar. 2020. Impact of camera focal length and sUAS flying altitude on spatial crop canopy temperature evaluation. Computers and Electronics in Agriculture.
  • Type: Conference Papers and Presentations Status: Accepted Year Published: 2019 Citation: 12. Sangha, H., A. Sharda, D. van der Merwe, and W. Schapaugh Jr. 2019. Use of thermal imagery to identify stress in crops grown in various environments and soil types. Paper No. 1901809. In proceeding of the Annual International Meeting American Society of Agricultural and Biological Engineers. Boston, MA, U.S., July 7th to July 10th, 2019.
  • Type: Journal Articles Status: Published Year Published: 2019 Citation: Hatton, N.*, E. Menkeb, A. Sharda, D. Merwe and W. Schapaugh. 2019. Assessment of sudden death syndrome in soybean through multispectral broadband remote sensing aboard small unmanned aerial systems. Computers and Electronics in Agriculture. https://doi.org/10.1016/j.compag.2019.105094
  • Type: Journal Articles Status: Submitted Year Published: 2019 Citation: Hatton, N.*, A. Sharda, D. Merwe, and W. Schapaugh. 2019. Remote thermal infrared imaging for rapid screening of sudden death syndrome in soybean. Computers and Electronics in Agriculture.
  • Type: Conference Papers and Presentations Status: Accepted Year Published: 2019 Citation: 11. Sharda, A. 2019. Thermal sensor and Mission Planning Considerations for Quality Data Collection and Analytics. AUVSI Xponential  All things Unmanned. Chicago, IL. Aril 30th  May 2nd, 2019.


Progress 02/01/18 to 01/31/19

Outputs
Target Audience:The target audience was specifically farmers, industry professionals, manufacturers, graduate students, high spchool students and teachers, and faculty. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided? Developing and designing new board, and stencils; and integration of components. Testing new boards for proper functioning of hardware and software. The entire process of programming radios and testing multiple radios simultaneously accessing base station for RTK corrections, provided knowledge of GPS systems. The positioning and mapping sensor suite is of great significance for ground and aerial unmanned aerial vehicles. Safe setup of unmanned aerial systems, programming missions and safe integrating cameras for remote sensing. Field data collection provided knowledge and skills on accurate design of experiments, setting protocols for standardized crop parameters collection, utilizing sensors, and techniques for data storage for analysis Students also got training on use of modeling software to develop geo-orthomosaics, spatial analysis using GIS and conducting statistical analysis. How have the results been disseminated to communities of interest?Results were disseminated through particpation in departmental events, international conferences and one-to-one interaction with stake holders. What do you plan to do during the next reporting period to accomplish the goals?For year-3 1. Field experiments will be conducted for Year-3. The data will include aerial imaging using thermal, color infrared, and color cameras. In addition, data like, actual plant canopy temperature; environmental data including sky temperature, wind speed, air temperature; soil profile moisture; stem water potential and crop yield will be collected. 2. Safe flying protocols to collect standardized data will be developed for researchers to collect quality remote sensing data.

Impacts
What was accomplished under these goals? During this year, progress and accomplishments were achieved for goal 1 and 2. The sensor suite for positioning and mapping utilizing OEM GNSS receiver, and nine axis gyro was fully integrated with a custom board to designed to record real-time data and also provide necessary output to thermal infrared camera for geo-tagging. The final product was tested aboard a small unmanned aerial vehicle for data integrity and ease of integration for image geo-tagging. Additionally tests were also conducted to quantify impact of focal length of thermal infrared cameras and flying altitude on image quality and ability to extract accurate canopy temperatures for rapid analysis on soybean crop. Tests were also conducted using broadband color infrared cameras to test a new vegetative index to quantify disease and water stress. Data collected was analyzed and a manuscript covering color infrared research was submitted for consideration to computers and electronics in Agriculture and data is being analyzed for camera focal length and flying altitude study. Additionally corn crop was planted on 4.5 acre plot. The plot was divided into 3 sub-plots to apply irrigation at 505, 75% and 100% required levels. Thermal, and color infrared images were captured during three flying sessions conducted during different growth stages. In addition, ground truth data on camera sensor response was collected from reference panels with known temperature for image calibration. Environmental data like solar radiation, sky temperature, wind speed, humidity, and air temperature was collected to ascertain crop water stress. Crop based water stress metrics like stem water potential was captured using leaf porometer and pressure bomb. The data collected was analyzed to develop strategies for canopy temperature extractions for corn crop; quantify correlations between canopy temperature and soil moisture; and canopy temperature derived water stress and soil moisture.

Publications

  • Type: Conference Papers and Presentations Status: Submitted Year Published: 2018 Citation: Hatton, N. M., A. Sharda, D. van der Merwe, and W. Schapaugh Jr. 2018. Remote Thermal infrared imaging for rapid screening of sudden death syndrome in soybean. Paper No. 1800881. In proceeding of the Annual International Meeting American Society of Agricultural and Biological Engineers. Detroit, MI, U.S., July 26 to August 1, 2018
  • Type: Journal Articles Status: Submitted Year Published: 2018 Citation: Hatton, N.,* E. Menke, A. Sharda, W. Schapaugh, and D. van der Merwe. 2018. Assessment of Sudden Death Syndrome in Soybean Through Broadband Color Infrared Remote Sensing. Computers and Electronics in Agriculture. Submitted
  • Type: Conference Papers and Presentations Status: Submitted Year Published: 2018 Citation: Sangha, H. S., A. Sharda, L. Koch, G. Wang, and P. Prabhakar. 2018. Impact of camera lens angle and sUAS flying altitude on spatial crop canopy temperature evaluation. Paper No. 1800840. In proceeding of the Annual International Meeting American Society of Agricultural and Biological Engineers. Detroit, Michigan, U.S., July 29th to August 1st, 2018.


Progress 02/01/17 to 01/31/18

Outputs
Target Audience:Target audience was graduate studnents, faculty, industry professionals and faculty. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided?During integration of dual frequency GNSS reciever, great deal of skill and knowledge was attained to understand the reciever design, functionality and programing for standalone operation. One to one meetings with systems engineers with TOPCON Precision Ag. provided a high level understanding of board design and skill to integrate on a custom design board which is being planned for Year 2 of the study. Similarly, radio were procured from MicroHard Inc., to acquire treal-time RTK correction from a ground based base station. One-to-one time spent with engineers provided understanding on radio communication and programming for different topologies. Programming individual radio and integrating with the GNSS reciever provided an understsnding of radio communication strategies and hardware development needs How have the results been disseminated to communities of interest? Nothing Reported What do you plan to do during the next reporting period to accomplish the goals?For year-2 1. The plan is to develop to develop and fully integrated all intented components of sensor suite for positioning and mapping by desiging and developing a custom board. A 9-axis IMU will be integrated to determine real-time image orientation. A custom software will be developed to record real-time location and orientation data for each image and syncronize with image trigger. Further the board will have the functionality of outputting GPS data through a UART port for thermal infrared camera, and a trigger functionality to trigger any color infrared camera at a defined distance. 2. Color infrared cameras of different focal lengths when flying at various altitudes will be tested to quantify their impact on image quality and ability to accurately extract canopy temperatures.

Impacts
What was accomplished under these goals? From the perspective of major goals, most accomplishment was achieved for goal 1, which was development of sensor suite for positioning and mapping. This required major OEM product acquisition, working with the engineering teams to understand OEM integration on the developmental board first for initial integration and tests. Additionally thermal infrared cameras were purchased with different focal length to understand the relative impact of camera selection and flying altitude for rapid and accurate extraction of canopy temperatures.

Publications