Source: WASHINGTON STATE UNIVERSITY submitted to NRP
CPS: SMALL: LOCALIZED, GEOSPATIAL SENSING OF CANOPY AND FRUIT MICROCLIMATE FOR REAL-TIME MANAGEMENT OF SUNBURN IN APPLE
Sponsoring Institution
National Institute of Food and Agriculture
Project Status
COMPLETE
Funding Source
AFRI COMPETITIVE GRANT
Reporting Frequency
Annual
Accession No.
1018573
Grant No.
2019-67021-29226
Cumulative Award Amt.
$449,742.00
Proposal No.
2018-09051
Multistate No.
(N/A)
Project Start Date
Mar 1, 2019
Project End Date
Feb 28, 2023
Grant Year
2019
Program Code
[A7302]- Cyber-Physical Systems
Recipient Organization
WASHINGTON STATE UNIVERSITY
240 FRENCH ADMINISTRATION BLDG
PULLMAN,WA 99164-0001
Performing Department
Biological Systems Engineering
Non Technical Summary
Washington State is the primary producer of fresh market apples, sweet cherries, pears and other high value crops in the US. Despite impressive statistics, growers continue to face enormous challenges to deliver high-quality, fresh-market produce which is also critical part of the US consumer diet. It is due to crop loss caused by sunburn (apple), fruit splitting (cherry), frost damage, and insect infestation. Growers can use several active methods to mitigate crop loss, however, they lack monitoring systems that link 'real-time agricultural data analytics' on crop physiology and 'micro-climate' at the orchard block level with 'Intelligent Control' technologies. This knowledge gap reduces yields and produce makertability causing economic loss to growers and decreases consumers aceess to nutrientional fruit produce at affordable cost. Our project, lead by New Investigator PI-Khot, will help in overcoming this hurdle by developing an internet of things compatible precision monitoring and control system that is able to determine the physiology of high-value tree fruit crops in the orchard environment and trigger the protection system actuation for preventing crop losses. This is the long-term goal of the project. Developed technology will be tested to monitor and manage sunburn in premium apple cultivars. Sunburn is detrimental to apple production where excess heat and sunlight, becoming a frequent phenomenon in the region, causes 6 to 30% annual fruit loss. Successful research, development, promotion, and commercialization of the technology can then be translated to other crops (cherry, pear, grape, blueberry) to manage pertinent abiotic stressors.
Animal Health Component
40%
Research Effort Categories
Basic
30%
Applied
40%
Developmental
30%
Classification

Knowledge Area (KA)Subject of Investigation (SOI)Field of Science (FOS)Percent
40274101020100%
Knowledge Area
402 - Engineering Systems and Equipment;

Subject Of Investigation
7410 - General technology;

Field Of Science
1020 - Physiology;
Goals / Objectives
The long-term goal of this project is to develop a monitoring and control system that is able to precisely determine the physiology of high-value tree fruit crops in the orchard environment. The overall research objective of the project is to harnesses concept capabilities from the localized Internet of Things (IoT) and implement them to monitor and manage apple sunburn. Once established, the technology can then be applied to monitor and manage other key abiotic stressors (e.g. sweet cheery splitting, frost damage, etc.) in the production of other high-value tree fruit crops. To accomplish this overall objective, our team will pursue three (3) specific aims:Aim #1: Develop an IoT-enabled remote sensing network for monitoring the micro-climate and physiology of tree fruit to provide localized and spatial data inputs including air temperature, and sunlight as well as infrared and RBG images of fruit;Aim #2: Develop and test methods/interfaces that provide real-time, on-board data processing and modeling of data via an embedded computer as well as a means for reliable transmission, and data storage;Aim #3: Evaluate the applicability of the on-board sensing and control technology to actuate an overheard evaporative cooling system for apple sunburn management.
Project Methods
Aim 1:Develop an IoT-enabled remote sensing network for monitoring the micro-climate and physiology of tree fruit to provide localized and spatial data inputs including air temperature, and sunlight as well as infrared and RBG images of fruit.The plan is to create a network of interconnected, computer-based-sensing units and deploy them in the field to obtain real-time temperatures (spatial imaging), sunlight, RGB images of the canopy and fruit, and micro-climate data directly at the field block level. This step will be followed by performing data mining for need-based, real-time management of apple sunburn. The system includes the i) sensing nodes, ii) data and decision support hub, iii) real-time evaporative cooling control/actuation hardware, and iv) cloud based online data storage, backup, analysis and reporting infrastructure.A sensing node will consist of a single board computer (Raspberry Pi 3 or similar)configured as a server, a thermal sensing module, an RGB camera module, and amicro-climate monitoring module.Each node will be installed in the field such that thermal (raw data with tiff format) and RGB (JPG format) images will cover the apple tree canopy (including fruits). Images will be processed on-board (using OpenCV) immediately after they are acquired (i.e. real-time image processing), and stored on board. The results of on-board image processing and compression will then be transferred for further processing and storage. Micro-climate data can be stored in a large repository in DAT/TXT format for the final decision-making process to proceed. One of the nodes (also configured as a server) will be equipped with an external hard drive. This will play the role of the "data and decision support hub" where data from all the individual nodes can be stored.In Year-1, we will develop and field test the prototype of the above concept. Year-2 and -3 efforts will be towards implementing this technology for apple canopy and FST monitoring for effective sunburn management in two modelapple cultivars.Aim 2: Develop and test methods/interfaces that provide real-time, on-board data processing and modeling of data via an embedded computer as well as a means for reliable transmission, and data storage;The algorithm for estimating FST based on thermal-RGB imagery has been implemented in Matlab® by our group (Chandel et al., 2018). This algorithm will be adopted on-board the single board computer using openCV.Post segmentation, a few morphological operators will be applied on the masked image for precise separation of the target fruit andextracted will bethe mean (and maximum) FST will be extracted from segmented fruit. The FST estimated from each node will be transferred to a 'Decision Support System' for its fusion with weather based FST data for the final decision on whether or not to trigger the cooling system.In each growing season (Years 2, 3), we will collect data on air temperature and canopy fruit surface temperature, with resolution in 1-5 minutes, in experimental apple blocks using the integrated monitoring system from Aim 1. We also propose to measure emissivity and albedo in this study for 'Gala and Fuji' cultivars. We want to collect measurements at different growth stages and re-estimate the parameters to strengthen the model already developed in our prior efforts (Li et.al., 2014). Based on previous experience to perform the FST measurements of the fruit surface temperatures we propose to use a Lux meter to measure albedo and a hand-held infrared thermometer and hand-held thermocouple to measure emissivity. Open field meteorological data will be obtained from the AgWeatherNet web portal at www.weather.wsu.edu. In-field micro-climate data will also be used in similar FST model development. To improve and validate both models, a database will be created using values for the two different cultivars and sites. Once the database is complete, identification of new parameters will be compared to existing algorithms. Testing of the model will be achieved using the data collected in the field and lab combined with historical temperature records under different environmental conditions from AgWeatherNet.Aim 3: Evaluate the applicability of the on-board sensing and control technology to actuate an overheard evaporative cooling system for apple sunburn managementA solid set canopy delivery system (SSCDS) that is currently under evaluation by our group for orchard spraying (Sinha et al., 2017) will be modified to apply a mist of water within the canopies for evaporative cooling.SSCDS will enable a timely, quick and precise application for efficient and economical cooling of the canopy micro-climate, with expected >10 fold savings in water due to misting of water, using micro-emitters at precise times compared to traditional nozzles that generate coarse size droplets over-head the canopy.In year 2, a decision support system will be integrated with SSCDS to make this system versatile. SSCDS will be setup and tested to control four adjacent sub-plots (30 ft × 4 lines as a test plot × 3 replicate plots × 2 apple cultivars). The 'control unit' will consist of a microcontroller (Arduino), radio frequency module (XBee Pro) and relay switches for controlling different solenoid valves connected to SSCDS-linked evaporative cooling system.The control system proposed for SSCDS will receive the input of critical FST data from the decision support system. Evaporative cooling decisions will be made based on input obtained from the monitoring system that have integrated FST data and climate-based FST estimates. It will communicate remotely with the 'control unit' responsible for solenoid valve-based actuation of SSCDS linked evaporative cooling system; to cool the canopy and fruit surface. Preliminary tests will be performed on automated SSCDS to check the functionality of the developed system. Several system parameters such as fluid flow rate, pressure drop in the system, and variability in the spray volume will be accessed along the spray line in various field topographic conditions (evident in WA tree fruit orchards). Moreover, solenoid valve sequencing will also be monitored and correction measures will be implemented based on system performance.In Year 3, we will field test an optimized and automated operation of an evaporative cooling system in 1 acre blocks of two sunburn susceptible cultivars, 'Gala and Fuji' apples. An optimized system prototype will be deployed in a cooperating grower's apple orchard, near Prosser, WA, with 300 ft × 4 lines as a test plot × 3 replicate plots × 2 apple cultivars. The flowmeters will be installed in irrigation lines to estimate/assess the water savings/day/week/ season and savings in pumping energy. Cyclic irrigation (cooling) will be conducted based on the critical FST data from the real-time sensing system. During harvest, we will collect per tree yield data of randomly selected trees within a treatment block, and quantify the fruit loss per tree related to SSCDS and conventional (sprinkler) evaporative cooling method.Knowledge from experienced growers and other experts on our WSU Center for Precision and Automated Agricultural System advisory board will be used to aid in determining cut-off temperatures for sprinkler (or SSCDS) operation as a function of weather conditions and crop physiology. The sensor network will be deployed, in consultation with growers, to provide real-time data on temperature variation, which will be both an input to the decision making with the intelligent controller and an output tracking the impact of mitigation technologies.

Progress 03/01/19 to 02/28/23

Outputs
Target Audience:Tree fruit growers, crop consultants, agricultural engineering and associatedresearch community working in atechnology space Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided?2020 Graduate student and couple PIs attended ASABE-AIM 2019. This platform provides professional networking opportunities. Graduate student also participated in 'Flask Talks' at 115th Annual Meeting of Washington State Tree Fruit Association and Next Generation Grower Network workshops, platforms critical to develop student's presentation and research communication skills. 2021 Graduate students presented pertinent outcomes at regional 'Next Generation Network workshop'; & at international venues such as 'Training on Precision Agriculture organized by Centre for Advanced Agricultural Science and Technology for Climate Smart Agriculture and Water Management, India and at '2020 IEEE International Workshop on Metrology for Agriculture and Forestry', Italy. The latter was awarded as 'Best paper presented by the young researcher'. Trough these opportunities, student was exposed to grower and international research community, enabling networking and professional development opportunities. 2022 Graduate students presented pertinent outcomes at regional annual event '117th Northwest Hort Expo'. Graduate students also provided opportunities to share pertinent results in WSU Tree Fruit Extension team organized field days (offered in English and Spanish), helping them develop extension education skills. They also shared the research to community college organized event (in Summer 2021) and to the international delegation as part of CA-WA-DU collaboration (Fall 2022). Through these opportunities, student was exposed to grower and international research community, enabling networking and professional development opportunities. 2022-23 Graduate students presented pertinent outcomes at the regional annual event '118th Northwest Hort Expo'. Graduate students also provided opportunities to share pertinent results in WSU Tree Fruit Extension team organized field days (offered in English and Spanish) for grower education, with NIFA funded AgAID team members for additional collaboration explorations, with national and international researchers visiting WSU, and with several local high-school and community college students. Students also presented research findings at 2022 ASABE-AIM and '2022 IEEE International Workshop on Metrology for Agriculture and Forestry'. Through these opportunities, students were exposed to grower and international research community, enabling networking and professional development opportunities. How have the results been disseminated to communities of interest?2020 Research outcomes were published in peer-reviewed journals (Information Processing in Agriculture, Sensors) and were presented at ASABE-AIM 2019. Furthermore, project members participated in stakeholder annual meetings (e..g. Annual Meeting of Washington State Tree Fruit Association; WSU CPAAS Ag Tech Day 2019; Next Generation Grower Network workshop 2020; Columbia Basin Tree Fruit Club meeting 2019) to disseminate the project activity specific outcomes. 2021 Research outcomes were published in peer-reviewed journals (Computers and Electronics in Agriculture, Sensors, IEEE Explore) and were presented at ASABE-AIM 2020 and IEEE international workshop on Metrology for Agriculture and Forestry. Furthermore, project members participated in stakeholder meetings (e.g., Next Generation Grower Network workshop 2020; WSU Digital Agriculture Summit) and international workshops to disseminate the project activity specific outcomes.? 2022 The technology and related results were disseminated to communities of interest through 1) field days, 2) Talks at 117th Northwest Hort Expo, Yakima WA 3) invited talks at XII International Symposium on Integrating Canopy, Rootstock and Environmental Physiology in Orchard Systems, Wenatchee, WA, USA & at Annual Meeting of American Society of Horticultural Sciences, Denver, CO, USA; 4) invited session talk at Specialty Crop Engineering: Advanced Technologies for Specialty Crop Production, ASABE Annual International Meeting, July 12, 2021. (Virtual) 2022-23 The technology and related results were disseminated to communities of interest through 1) field days and grower meetings, 2) Talks at 118th Northwest Hort Expo, Wenatchee, WA, and 3) at ASABE Annual International Meeting held at Houston, TX during July 17-20, 2022. What do you plan to do during the next reporting period to accomplish the goals? Nothing Reported

Impacts
What was accomplished under these goals? Pertaining to this project, variants of crop physiology sensing (CPS) unit prototypes were developed with incremental refinements for season long monitoring of the apple fruit surface temperature (FST) for sunburn monitoring and management. The finalized crop physiology sensing (CPS 3.0) unit utilizes thermal-RGB and in-field microclimate data to estimate real-time apple FST at 5-min interval. The acquired data was processed on-board the computing unit to evaluate imagery and weather data derived apple FST and was shared with the remote host user in real-time. A 4G LTE cellular router was used as a networking device to enable IoT capability of CPS 3.0. The router was embedded with a cellular subscriber identification module (SIM) to create a local WiFi grid so that internet connectivity can be provided to all the sensing nodes. The web connected sensor nodes were remotely accessed and controlled through the host computer and edge-processed data was sent to the cloud using remote access and Cloud Backup TeamViewer services. Each of the sensing nodes were powered by a localized 30 W ploy crystalline solar panel. Regarding the imagery based FST estimation, as the edge compute on CPS unit, our team continued to improve the fruit segmentation algorithm. Fruit segmentation and FST estimation algorithm needed enhancement for cultivar and color independent system operations. Therefore, we developed, and field evaluated mask region-convolution neural network (R-CNN) aided fruit segmentation model and edge compute compatible FST estimation algorithm. Season long field data was collected in 2021 using eight CPS units (in cv. WA38 [Cosmic crisp] and cv. Honeycrisp). This data was used to develop and validate the mask R-CNN based fruit segmentation model. The optimized model had 91.4% segmentation accuracy. The improved model aided CPS units were then deployed, and field evaluated in 2022 season. Overall, mask R-CNN-aided algorithm reliably segmented fruits, with dice similarity coefficient of 0.89, and estimated FST with <0.5? error compared to ground truth FST data. The FST estimation time for each instance by CPS unit was 37s, about 22%-time improvement over the existing algorithm. The CPS unit has been field validated for three years and in 2022, we piloted automated actuation of fogging system to mitigate sunburn. We also evaluated the efficacy of different sunburn mitigation techniques. The CPS enabled us to contrast collected imagery and weather based FST data with crop quality data at harvest and postharvest. Pertinent scientific information-based knowledge was shared with grower stakeholders and is being written as peer-reviewed publications. As a continuum, a localized crop physiology sensing based approach is being translated to monitor apple crop water stress throughout the season and to manage heat stress in other perennial specialty crops (e.g., Grapes). We will continue to report the results and findings to the scientific community through peer reviewed publications. The technology and outcomes will be shared in upcoming grower meetings and also written as extension articles.

Publications

  • Type: Journal Articles Status: Submitted Year Published: 2023 Citation: Amogi, B. R., Ranjan, R., Khot, L. R., 2023. Mask R-CNN aided fruit surface temperature monitoring algorithm with edge compute enabled internet of things system prototyped for automated apple heat stress management, Information Processing in Agriculture. (Submitted and Under Review; Manuscript Number: IPA-D-23-00050)
  • Type: Journal Articles Status: Submitted Year Published: 2023 Citation: Ranjan, R., Amogi, B. R., Chandel, A. K., Khot, L. R., Sallato, B. V., Peters, R. T., 2022. Efficacy evaluation of apple sunburn mitigation techniques using crop physiology sensing system, Computers and Electronics in Agriculture. (Submitted and Under Review; Manuscript Number: COMPAG-D-22-02386)
  • Type: Conference Papers and Presentations Status: Published Year Published: 2022 Citation: Amogi, B.R., Ranjan, R., Khot, L.R., 2022. Reliable image processing algorithm for sunburn management in green apples, in: 2022 IEEE Workshop on Metrology for Agriculture and Forestry (MetroAgriFor). pp. 186190. https://doi.org/10.1109/MetroAgriFor55389.2022.9964902
  • Type: Conference Papers and Presentations Status: Published Year Published: 2022 Citation: Amogi B. R., Ranjan R., Khot L.R., 2022. Robust fruit segmentation-based algorithm for a resource-constrained edge device to effectively monitor heat stress in apple orchards. Paper No. 2201168, ASABE 2022 Annual International Meeting, Houston, TX, July 1720, 2022 (Oral Presentation).
  • Type: Other Status: Published Year Published: 2023 Citation: Khot, L. R., Amogi, B. R., Sallato, B. V., 2023. Cooling off with clouds. Irrigation Today magazine, Irrigation Association, Fairfax, VA, April 20, 2023. https://irrigationtoday.org/features/cooling-off-with-clouds/
  • Type: Other Status: Published Year Published: 2022 Citation: (Invited) Talk on Crop monitoring & Management Technologies for Modern Orchard System, IoT4Ag: The Internet of Things for Precision Agriculture an NSF Engineering Research Center. August 8, 2022. Time: 30 min.
  • Type: Websites Status: Published Year Published: 2022 Citation: Heat stress one of the hot topics at Smart Orchard field day, by Jonelle Mejica, Good Fruit Grower, July 28, 2022. https://www.goodfruit.com/heat-stress-one-of-the-hot-topics-at-smart-orchard-field-day/
  • Type: Theses/Dissertations Status: Under Review Year Published: 2023 Citation: Basavaraj Amogi. July 2023. PhD Thesis. Edge-intelligence enabled infield sensing system for heat stress mitigation in apple orchards.


Progress 03/01/21 to 02/28/22

Outputs
Target Audience:Tree fruit growers, crop consultants, agricultural engineering research community Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided?Graduate students presented pertinent outcomes at regional annual event '117th Northwest Hort Expo'. Graduate students also provided opportunities to share pertinent results in WSU Tree Fruit Extension team organized field days (offered in English and Spanish), helping them develop extension education skills. They also shared the research to community college organized event (in Summer 2021) and to the international delegation as part of CA-WA-DU collaboration (Fall 2022). Through these opportunities, student was exposed to grower and international research community, enabling networking and professional development opportunities. How have the results been disseminated to communities of interest?The technology and related results were disseminated to communites of interest through 1) field days, 2) Talks at117thNorthwest Hort Expo, Yakima WA 3)invited talks atXII International Symposium on Integrating Canopy, Rootstock and Environmental Physiology in Orchard Systems, Wenatchee, WA, USA & atAnnual Meeting of American Society of Horticultural Sciences, Denver, CO, USA; 4) invited session talk atSpecialty Crop Engineering: Advanced Technologies for Specialty Crop Production, ASABE Annual International Meeting, July 12, 2021. (Virtual) What do you plan to do during the next reporting period to accomplish the goals?The project objective for the next year would to further enhancement of the versatility of CPS 3.0. The crucial in-field fruit quality estimation capability will be added in the upgraded unit (CPS 3.0) along with the existing sunburn monitoring features. The RGB imagery data collected by the unit will be utilized to monitor fruit color development throughout the season. Such parameters are crucial in terms of fruit quality and maturity assessment. Also, CPS 3.0 compatible decision support system (DSS) will be developed that can be integrated with a evaporative cooling system for water and energy efficient sunburn management. From past few seasons and literature, we have preliminary identified threshold temperatures for sunburn browning and necrosis damage in WA 38 and Honeycrisp. We will continue to refine these thresholds as they are critical for accurate sunburn management related decision making. During the 2022 production season, we also plan to continue research on the micro-emitters/foggers based cooling strategies for sunburn management. Season long field experiment will be conducted with automated system in Honeycrisp and Cosmic Crisp apple orchards. The sunburn management efficacy of FST supervised convective cooling system will be compared the conventional overhead evaporative cooling in terms of water saving, energy use and fruit quality. To evaluate the water uses (gallon per day), flow sensors will be deployed in the conventional and convective cooling system. Fruit quality assessment will be performed at the end of the production season to evaluate the suitability of the integrated system for sunburn management, using prior developed methods.

Impacts
What was accomplished under these goals? Our team continued development and modification of a crop physiology sensing (CPS 3.0) unit to monitor the in-field apple fruit surface temperature (FST) for apple sunburn monitoring and management. The acquired data was processed on-board the computing unit to evaluate imagery and weather data derived apple FST and was shared with the remote host user in real-time. A 4G LTE cellular router was used as a networking device to enable IoT capability of CPS 3.0. The router was embedded with a cellular subscriber identification module (SIM) to create a local WiFi grid so that internet connectivity can be provided to all the sensing nodes. The web connected sensor nodes were remotely accessed and controlled through the host computer and edge-processed data was sent to the cloud using remote access and Cloud Backup TeamViewer services. Each of the sensing node was powered by a localized 30 W ploy crystalline solar panel. The developed CPS3.0 unit was validated for season long sunburn monitoring in 2021 field season in two sites (cv. WA38 M.9 rootstock, WSU Roza farm and cv. Honeycrisp on M.106 rootstock, Hancock Farmland Services) near Prosser, WA. In the WA38 research block three treatments namely fogging, shadenet and untreated control were setup through randomization. Fogging type evaporative cooling system utilizes 2-way fogging type emitters while shadenet had 12% screening factor. An untreated control treatment had no heat mitigation strategy. Each treatment has three replicates with 10 trees in each replicate. Treatments were installed on July 8, 2021. At the Honeycrisp site, three replicates of five treatments namely netting, fogging, conventional evaporative cooling, fogging and netting combined (termed as 'fognet' hereafter) were established along with no treatment control. Foggers used were same as WA 38 block while the netting had around 15% screening factor. Conventional evaporative cooling treatment was set up using the existing overhead sprinklers used by growers at 20 min ON/OFF cycles. Each replicate of four treatments and untreated control had 10 trees. Treatments were established on July 15, 2021. At both the sites, CPS 3.0 unit was installed in each treatment and enabled to monitor mean measured FST (FSTi), maximum FST (FSTi-max), mean FST of the 10%, 15% and 20% hottest part of the fruit surface (i.e., FST10, FST15, and FST20, respectively), and FSTw (weather-model-predicted FST). Ground truth FST, fruit size and albedo data were collected to validate the imagery based FST and FSTw. To evaluate the impact of sunburn management strategy on the fruit, tree vigor, fruit quality and storability, 6 to 10 trees from each replicate were strip harvested. The harvested apples were evaluated for % sunburn incidence immediately after harvest in the field. Significant findings: CPS 3.0 nodes were rugged and able to distinguish the heat stress mitigation treatment effects throughout the season. Weather-model based FST estimates were somewhat ambiguous confirming need to further refine the approach to: i) quantify localized weather parameters appropriately, and ii) energy balance approach with fruit physiology (size, color [albedo], % shading) data.

Publications

  • Type: Journal Articles Status: Published Year Published: 2022 Citation: Ranjan, R., R. Sinha, L.R. Khot, and M. Whiting. 2022. Thermal-RGB imagery and in-field weather sensing derived sweet cherry wetness prediction model. Scientia Horticulturae, 294, 110782 https://doi.org/10.1016/j.scienta.2021.110782
  • Type: Other Status: Published Year Published: 2021 Citation: Ranjan, R., B. R. Amogi, L. R. Khot, and R. Troy Peters. 2021. Real-time in-orchard apple heat stress monitoring system. Washington State University  Fruit Matters, July 2021.
  • Type: Other Status: Published Year Published: 2022 Citation: Talk at 2022 WSU Weather School. Feb 4, 2022. (Attendees: 157; Virtual). The event will be sponsored by WSTFA, WA Wine, WSGS Grape Society, WA Blueberry, WA Potato Commissions
  • Type: Conference Papers and Presentations Status: Published Year Published: 2021 Citation: Talk at Climate, Weather and Washington Agriculture session at Annual Tilth Conference, Lynnwood, WA. November 19, 2021. (Attendees: ~30 in-person and ~60 virtual small growers, farmers, public organization representatives).
  • Type: Other Status: Published Year Published: 2021 Citation: (Invited) Talk on Technologies for Automated Apple Heat Stress Monitoring and Mitigation, 117th Washington State Tree Fruit Association Annual Meeting & NW Hort Expo, Yakima, WA. December 7, 2021. Time: 20 min, Participants: ~300.
  • Type: Other Status: Published Year Published: 2021 Citation: Talk on WSU AgWeatherNet 2021 Updates, WSGS Annual Meeting, Grandview, WA. November 19, 2021. Time: 15 min. Participants: ~50.
  • Type: Other Status: Awaiting Publication Year Published: 2021 Citation: (Invited) Session talk Crop stress monitoring technologies in Digital Ag Era, Environmental Molecular Sciences Laboratory (EMSL) Integration Webinar on Environmental Sensors, PNNL, Richland, WA. October 4-7, 2021. Time: 30 min, Participants: ~75.
  • Type: Theses/Dissertations Status: Published Year Published: 2021 Citation: Dr. Rakesh Ranjan. May 2021. PhD Thesis. Title: Sensing integrated automated solid set canopy delivery system for crop loss management in deciduous fruits and grapevines. Washington State University.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2021 Citation: Amogi*, B. R., R. Ranjan, L. R. Khot, T. R. Peters, B. Sallato, and C. Torres. 2021. Crop physiology sensing system for abiotic stress management in specialty fruit crops. Research News Flash, 117th Annual Meeting & NW Hort. Expo., Yakima, WA. December 6-8, 2021. Time: 5 min. Participants: ~250.


Progress 03/01/20 to 02/28/21

Outputs
Target Audience:Tree fruit growers, crop consultants, agricultural engineering research community Changes/Problems:COVID-19 presented challanges related to field deployment and extensive testing of CPS2.0 in cooperating grower field sites. We plan to conduct pertinent technology deployment and data collection in 2021 and potentially 2022 field season. What opportunities for training and professional development has the project provided?Graduate students presented pertinent outcomes at regional 'Next Generation Network workshop'; & at international venues such as 'Training on Precision Agriculture organized by Centre for Advanced Agricultural Science and Technology for Climate Smart Agriculture and Water Management, India and at '2020 IEEE International Workshop on Metrology for Agriculture and Forestry', Italy. The latter was awarded as 'Best paper presented by the young researcher'. Trough these opportunities, student was exposed to grower and international research community, enabling networking and professional development opportunities. How have the results been disseminated to communities of interest?Resarch outcomes were published in peer-reviewed journals (Computers and Electronics in Agriculture, Sensors, IEEE Explore) and were presented at ASABE-AIM 2020 and IEEE international workshop on Metrology for Agriculture and Forestry. Furthermore, project members participated in stakeholder meetings (e..g. Next Generation Grower Network workshop 2020; WSU Digital Agriculture Summit) and international workshops to disseminate the project activity specific outcomes.? What do you plan to do during the next reporting period to accomplish the goals?The project objective for the next year would to further enhancement of the versatility of CPS2.0. Crucial in-field fruit quality estimation capability will be added in the upgraded unit (CPS3.0) along with the existing sunburn monitoring features. The RGB imagery data collected by the unit will be utilized to monitor the fruit growth (i.e., size and shape) and color development in the fruit throughout the season. Such parameters are crucial in terms of fruit quality and maturity assessment. Furthermore, CPS3.0 compatible decision support system (DSS) will be developed that can be integrated with a convective cooling system for water and energy efficient sunburn management. In year 2020, we conducted field studies to evaluate the threshold temperature for sunburn browning and necrosis damage in Cosmic Crisp, a newly developed cultivar by Washington State University. Such FST information is very critical for accurate sunburn management related decision making. Moreover, critical FST for other apple cultivars grown in Washington State will be adopted form previous reported studies. The cultivar specific FST information will be the embedded in the DSS data base for FST supervised convective cooling to manage sunburn. During the 2021 production season, we are also planning to research the micro-emitters/foggers to creating a mist within the canopy to facilitate cooling. FST supervised low-volume convective cooling can further assist in reducing the water use. Season long field experiment will be conducted in commercial Honeycrisp and Cosmic Crisp apple orchards. The sunburn management efficacy of FST supervised convective cooling system will be compared the conventional overhead evaporative cooling in terms of water saving, energy use and fruit quality. To evaluate the water uses (gallon per day), flow sensors will be deployed in the conventional and convective cooling system. Fruit quality assessment will be performed at the end of the production season to evaluate the suitability of the integrated system for sunburn management, using methods developed for 2020 season.

Impacts
What was accomplished under these goals? Our team developed a crop physiology sensing (CPS 1.0) unit to monitor the in-field apple fruit surface temperature (FST) for apple sunburn monitoring and management during the 2019 production season. In year 2, we upgraded CPS (2.0) unit with an internet of things (IoT) capability to facilitate real-time sensor control, data visualization, sharing and cloud computing potential. This was desirable to enhance the versatility of the developed sunburn monitoring unit for precise management decisions. Also, CPS 1.0 unit consists an expensive thermal-RGB imager (~$1250/unit), resulting in higher cost of the monitoring system. In CPS2.0, we have replaced existing imagery sensor with an inexpensive unit. Overall, CPS 2.0 unit consist of a low-cost thermal module (model: Lepton 3.5, FLIR Systems, Inc., Wilsonville, OR, USA) and a Red-Green-Blue (RGB) sensor (model: Pi Camera V2, sensing module: Sony IMX219, Raspberry Pi foundation, Cambridge, UK). The thermal module was integrated on a breakout board (model: Lepton V2, FLIR Systems, Inc., Wilsonville, OR, USA) to capture a radiometric thermal image with raw temperature information embedded to each pixel. Also, the unit consist of an all-in-one weather sensor (model: ATMOS 41, METER Group, Inc., Pullman, WA, USA) to capture in-field weather data. The thermal-RGB and weather sensing units were integrated with IoT enabled edge computing unit (model: Raspberry Pi 4B, Raspberry Pi foundation, Cambridge, UK). The acquired data was processed on-board the computing unit to evaluate imagery and weather data derived apple FST and was shared with the remote host user in real-time. A cellular router (model: Diolink, network: 4G LTE) was used as a networking device to enable IoT capability of CPS2.0. The router was embedded with a cellular subscriber identification module (SIM) to create a local WiFi grid so that internet connectivity can be provided to all the nodes. The web connected sensor nodes were remotely accessed and controled through the host computer and edge-processed data was sent to the cloud using remote access and Cloud Backup TeamViewer services (TeamViewer AG, Göppingen, Germany). The developed CPS2.0 unit was validated for season long sunburn monitoring in year 2020. The sunburn mitigation efficacy of commonly adapted management practices was examined using this monitoring tool. An apple research block (cv. Cosmic Crisp, rootstock: G41 and M9, architecture: bi-axis) was longitudinally divided in three equal parts and treated with evaporative cooling (EC), control and shade netting (SN), respectively. One CPS unit was deployed in each treatment throughout the season to monitor sunburn susceptibility. On maturity, total 32 replicate trees (4 treatment × 2 rootstock/treatment × 4 replicate trees/rootstock) were harvesting by zones (top, mid and bottom) and stored for fruit quality analysis. The external and internal fruit quality was analyzed in the lab to evaluate the physical parameters, sunburn defects, soluble solid content (% ?Brix), fruit firmness (%), and titratable acidity (g L-1). The season long FST data collected from CPS2.0 indicated that mean and maximum FST in EC and SN treated block was significantly lower than control block. The fruit quality data is being analyzed and result will be presented in the next report. Overall, upgraded system reduced the cost of the CPS by ~65% and effectively monitored the sunburn throughout the season.

Publications

  • Type: Journal Articles Status: Published Year Published: 2020 Citation: Ranjan, R., Khot, L. R., Peters, R. T., Salazar-Gutierrez, M. R., & Shi, G. (2020). In-field crop physiology sensing aided real-time apple fruit surface temperature monitoring for sunburn prediction. Computers and Electronics in Agriculture, 175, 105558.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2020 Citation: Ranjan, R., Sinha, R., Khot, L. R., Peters, R. T., & Salazar-Gutierrez, M. R. (2020, November). Internet of Things enabled crop physiology sensing system for abiotic crop stress management in apple and sweet cherry. In 2020 IEEE International Workshop on Metrology for Agriculture and Forestry (MetroAgriFor) (pp. 273-277). IEEE.
  • Type: Journal Articles Status: Published Year Published: 2020 Citation: Wang, B., Ranjan, R., Khot, L. R., & Peters, R. T. (2020). Smartphone application-enabled apple fruit surface temperature monitoring tool for in-field and real-time sunburn susceptibility prediction. Sensors, 20(3), 608.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2020 Citation: Ranjan, R*., Sinha, R., Khot, L. R., Peters, R. T., Salazar-Gutierrezb, M.R. (Nov 4-6, 2020). Internet of Things enabled crop physiology sensing system for abiotic crop stress management in apple and sweet cherry. Presented in 2020 IEEE international workshop on Metrology for Agriculture and Forestry. November 46, 2020, pp. 273-277 (Awarded as Best Paper Presented by a Young Researcher).
  • Type: Conference Papers and Presentations Status: Published Year Published: 2020 Citation: Ranjan, R*., Wang, B., Khot, L.R., Peters. T. R. (Oct 1-5, 2020). AppSense1.0: A Smartphone Application for Real-time Apple Sunburn Monitoring. Presented at 2020 WSU Digital Agriculture Summit (Virtual lightening talk)
  • Type: Conference Papers and Presentations Status: Published Year Published: 2020 Citation: Ranjan, R*., Khot, L.R., Peters. T. R., Salazar-Gutierrez, M.R., Sallato, B. (Oct 1-5, 2020). Internet of Things enabled crop physiology sensing unit for apple sunburn management. Abstract published at 2020 WSU Digital Agriculture Summit (Abstract#07)
  • Type: Conference Papers and Presentations Status: Published Year Published: 2020 Citation: Ranjan, R*., Khot, L.R., Peters. T. R., Salazar-Gutierrez, M.R. (July 13-15, 2020). Field evaluation of visible-infrared and microclimate sensing aided crop physiology sensing system for apple sunburn management. Paper Presented at Annual International Meeting of American Society of Agricultural and Biological Engineers (Virtual oral presentation)
  • Type: Conference Papers and Presentations Status: Published Year Published: 2020 Citation: Ranjan, R*., Wang, B., Khot, L.R., and Peters. T. R. AppSense1.0: A Smartphone Application for Real-time Apple Fruit Surface Temperature Monitoring. Panel discussion and technology demonstration at Next Generation Network workshop. Wenatchee, WA, Feb 23, 2020. Time: 20 minutes, Participants: ~40.
  • Type: Other Status: Published Year Published: 2020 Citation: Ranjan, R*., Khot, L.R., Peters. T. R., Salazar-Gutierrez, M.R. Training session on Internet-of-Things enabled in-field sensing technologies for abiotic crop stress management, International Training on Precision Agriculture organized by Centre for Advanced Agricultural Science and Technology for Climate Smart Agriculture and Water Management (CAAST - CSAWM), Rahuri, India. September 28, 2020. Time: 60 mins. Participants ~1000


Progress 03/01/19 to 02/29/20

Outputs
Target Audience:Tree fruit growers, crop consultants, agricultural engineering research community Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided?Graduate student and couple PIs attended ASABE-AIM 2019. This platform provides professional networking opportunities. Graduate student also participated in 'Flask Talks' at 115th Annual Meeting of Washington State Tree Fruit Association and Next Generation Grower Network workshops, platforms critical to develop student's presentation and research communication skills. How have the results been disseminated to communities of interest?Resarch outcomes were published in peer-reviewed journals (Information Processing in Agriculture, Sensors) and were presented at ASABE-AIM 2019. Furthermore, project members participated in stakeholder annual meetings (e..g.Annual Meeting of Washington State Tree Fruit Association;WSU CPAAS Ag Tech Day 2019; Next Generation Grower Network workshop 2020;Columbia Basin Tree Fruit Club meeting 2019) to disseminate the project activity specific outcomes. What do you plan to do during the next reporting period to accomplish the goals?This year, project activities would be focused on the integration of developed CPS unit with a custom build evaporative cooling control unit for FST supervised evaporative cooling. A decision support system (DSS) is being developed for the control unit that wirelessly receives the real-time FST input from the CPS unit and take the decision of sprinkler actuation based on FST threshold. The sunburn critical threshold FST varies significantly with the apple cultivars due to variation in the degree of its susceptibility for sunburn. Therefore, critical temperature values for different apple cultivars produce by the growers in the Washington State shall be incorporated in the DSS for appropriate decision making on the cooling of the canopy. The control system comprises of a microcontroller, a radio frequency (RF) trans-receiver module, and a relay module. Moreover, a coordinator configured RF module is coupled with the CPS unit to establish communication between control system and CPS unit. The CPS unit is programmed to transmit the latest estimated FST value at an interval of 5 minute that is received by the router RF module of the control unit. The DSS compares the received FST value with the threshold FST set for the tested cultivar and actuates the relay switch connected with the solenoid valve when FST exceeds the critical limit. The actuation of solenoid valve would turn on the overhead evaporative cooling system to reduce the heat stress in the canopy and fruit. Team is planning to replace conventional high volume overhead cooling nozzle with the micro-emitters, that creates a mist/fog within the canopy to cool-down the microclimate. Such micro-emitters could further assist in reducing the water and energy requirement during sunburn management. Season long field experiment will be conducted in commercial Honeycrisp and Cosmic Crisp apple orchard. Fruit quality assessment will be performed at the end of the production season to evaluate the suitability of the integrated system for sunburn management. Firstly, the experts will conduct a visual inspection of the fruit sunburn and rating will be provided to the sampled fruits. Furthermore, various fruit quality parameters (color, size, weight, soluble solid content [SSC], fruit firmness and titratable acidity [TA]) will be assessed in the lab for the collected fruit samples and result would be contrasted against the control treatment fruits. Moreover, a flow sensor deployed in the sunburn management system to monitor the daily water use (gallon per day). Similarly, water use evaluation will be conducted for conventional evaporative cooling system. Finally, Water use and energy efficacy of the developed system will be evaluated, and results will be contrasted against the control treatment.

Impacts
What was accomplished under these goals? Pertaining to this project, a crop physiology sensing (CPS) unit prototype has been developed to monitor the in-field apple fruit surface temperature (FST) for apple sunburn monitoring and management. The CPS unit utilizes thermal-RGB and in-field microclimate data to estimate real-time apple FST. The developed sunburn management tool consists of two major components namely, a) sensing unit, b) data processing and sharing unit. The sensing unit consists of a visible-infrared imaging sensor and an all-in-one miniature weather sensor which acquires thermal-RGB imagery and in-field microclimate data for FST estimation. The real-time analysis of the acquired data is performed by the data processing unit and the results are shared with the host user computing device for appropriate decision-making. The CPS unit has been programmed to autonomously acquire the thermal-RGB and microclimate data at 5 min intervals, process the data for real-time FST estimation and wirelessly share the data with user host computing device. Additionally, a smartphone application-enabled handheld FST monitoring tool has been developed. This tool integrates a smartphone connected thermal-RGB imaging sensor and a custom developed application ('AppSense 1.0') for apple fruit sunburn estimation. This tool is configured to acquire and analyze imagery data on-board the smartphone. The tool also utilizes geolocation-specific weather data from an open field weather station to estimate FST. In 2019 production season, two automated CPS units were deployed in the commercial orchards of two high-value apple cultivars grown in Washington State (cv 'Honeycrisp' and 'Cosmic Crisp'). For each cultivar, field data was collected for three consecutive days between 12-5 pm at 5-minute intervals. As a reference measurement, a contact type thermal probe of accuracy ±0.4 °C was also utilized for actual apple FST (FSTa) measurements. The developed CPS unit performed reliably in the field condition to monitor the FST during the day. The imagery-based mean FST for both cultivars was found consistent with the actual FST. Moreover, the CPS unit was able to estimate the FST of highly stress region (sunlit zone) on the fruit to monitor localize sunburn. The weather model-based FST was found highly sensitive to sunlight exposure, fruit reflectance, and emissivity. Moreover, the smartphone application-enabled handheld sunburn monitoring tool was also validated in the same production season for 'Gala', 'Fuji', 'Red delicious' and 'Honeycrisp'. The developed tool successfully acquired thermal-RGB images and weather data and was able to perform real-time data analysis on-board the smartphone to estimate apple FSTs. The tool estimated imagery based FST was consistent with ground truth measurements. A significant difference was recorded between weather and thermal-RGB imagery predicted FSTs, possibly due to the remote locations of the open field weather stations. Overall, the developed sunburn-monitoring unit demonstrated great potential for real-time FST monitoring in the apples.

Publications

  • Type: Other Status: Published Year Published: 2019 Citation: Ranjan, R*. and Khot, L.R. Demonstration and presentation of Fruit surface temperature monitoring system for sunburn management in apple. Columbia Basin Tree Fruit Club meeting, Prosser, WA, July 23, 2019. Time: 10 minutes, Participants: ~30.
  • Type: Journal Articles Status: Published Year Published: 2019 Citation: Shi, G., Ranjan, R., Khot, L.R., 2019. Robust image processing algorithm for computational resource limited smart apple sunburn sensing system. Information Processing in Agriculture. doi: https://doi.org/10.1016/j.inpa.2019.09.007
  • Type: Journal Articles Status: Published Year Published: 2020 Citation: Wang, B., Ranjan, R., Khot, L.R., Peters, R.T., 2020. Smartphone application-enabled apple fruit surface temperature monitoring tool for in-field and real-time sunburn susceptibility prediction. Sensors. 20, 608.
  • Type: Journal Articles Status: Under Review Year Published: 2020 Citation: Ranjan, R., Khot, L.R., Peters, R.T., Salazar-Gutierrez, M.R., Shi, G., 2020. In-field crop physiology sensing aided real-time apple fruit surface temperature monitoring for sunburn prediction. Computers and Electronics in Agriculture. COMPAG_2020_790 (Submitted)
  • Type: Conference Papers and Presentations Status: Published Year Published: 2019 Citation: Wang, B*., Ranjan, R., Khot, L.R. (December 9-11, 2019). A Smartphone Application for Real-time Apple Fruit Surface Temperature Monitoring. Paper Presented at 115th Annual Meeting of Washington State Tree Fruit Association, Wenatchee, WA, USA
  • Type: Conference Papers and Presentations Status: Published Year Published: 2019 Citation: Ranjan, R*., Shi, G., Khot, L.R., Peters. T. R., Salazar-Gutierrez, M.R. (August 22, 2019). Internet of Things enabled Crop Physiology Sensing System for Apple Sunburn Management. Poster presentation and demonstration at WSU Ag Tech day 2019, Prosser, WA, USA
  • Type: Conference Papers and Presentations Status: Published Year Published: 2019 Citation: Wang, B*., Ranjan, R., Khot, L.R. (August 22, 2019). Mobile Application for Real-time Apple Fruit Surface Temperature Monitoring. Poster presentation and demonstration at WSU Ag Tech day 2019, Prosser, WA, USA
  • Type: Conference Papers and Presentations Status: Published Year Published: 2019 Citation: Ranjan, R*., Shi, G., Khot, L.R. (July 7-10, 2019). Internet of Things based System for Crop Loss Management: Prototyping a VisibleInfrared and Microclimate Sensing Derived Apple Sunburn Monitoring System. Paper Presented at Annual International Meeting of American Society of Agricultural and Biological Engineers, Boston, Massachusetts, USA
  • Type: Other Status: Published Year Published: 2020 Citation: Ranjan, R*., Wang, B., Khot, L.R., and Peters. T. R. AppSense1.0: A Smartphone Application for Real-time Apple Fruit Surface Temperature Monitoring. Panel discussion and technology demonstration at Next Generation Network workshop. Wenatchee, WA, Feb 23, 2020. Time: 20 minutes, Participants: ~40.