Source: UNIVERSITY OF CALIFORNIA, MERCED submitted to
COLLABORATIVE RESAERCH: NRI: INT: MOBILE ROBOTIC LAB FOR IN-SITU SAMPLING AND MEASUREMENT
Sponsoring Institution
National Institute of Food and Agriculture
Project Status
NEW
Funding Source
Reporting Frequency
Annual
Accession No.
1024609
Grant No.
2021-67022-33452
Project No.
CALW-2020-08993
Proposal No.
2020-08993
Multistate No.
(N/A)
Program Code
A7301
Project Start Date
Nov 1, 2020
Project End Date
Oct 31, 2024
Grant Year
2021
Project Director
Carpin, S.
Recipient Organization
UNIVERSITY OF CALIFORNIA, MERCED
PO BOX 2039
MERCED,CA 95343
Performing Department
School of Engineering
Non Technical Summary
Increasing population, decreasing arable land, climate change, and a declining skilled workforce pose unprecedented challenges to our ability to satisfy the growing demand for food on a global scale. Accurate assessments of multiple spatiotemporal conditions, such as evapotranspiration, are instrumental to fine tune the amount of water used in farming. Today, precise water use measurement requires the use of a pressurized chamber, an instrument that is cumbersome to operate and greatly limits the number of measurements that can be made given the need for human collection of plant specimens in the field. Consequently, critical parameters for large orchards are obtained by interpolating very sparse sample sets, thus failing to capture the inherent variability requiring precise adjustment of agricultural inputs. In this project, for the first time, we will develop a mobile robotic lab that is capable of autonomously selecting regions to sample, physically collect leaves and immediately perform on-board analysis to measure leaf water potential, improving the accuracy, precision and efficiency used in present-day pressure chamber technology. The mobile lab system features both aerial vehicles as well as ground robots, and during the project we will design a novel robotized pressure chamber enabling the measurement of leaf water potential at scale. The system will be tested and validated in the field in four different agronomic testbeds in California on four different perennial crops in collaboration with commercial partners.This project will develop the scientific and technological foundations to create a new mobile robotics lab to perform sampling and analysis of water leaf potential at spatiotemporal scale not achievable with current technologies. The PIs will work on hardware/software co-design of novel actuators to autonomously acquire and analyze specimens in the field. We will tackle fundamental questions about coordination of heterogeneous robotic systems operating under resource constraints, and perception algorithms to extract how to best perform leaf water potential measurements observing human behaviors. Finally, collected data will be used to quantitatively confirm or disprove our hypothesis that current sampling practices fail to capture the existing heterogeneity in leaf water potential.The proposed system has broad applicability andcan be used to improve agriculture practices in a wide variety of crops throughout the US with the potential of great savings in the use of water and agrichemicals. Both participating universities are Hispanic Serving Institutions (HSIs), and students from all backgrounds will be involved. Findings will be presented at leading conferences and papers will be freely made available. Hardware designs and code will be open source and data collected during the project will also be made freely available to the scientific community. Results will be disseminated to the broader public through the University of California TV, and existing outreach initiatives at both institutions will be leveraged to engage K-12 students, as well as industry stakeholders.
Animal Health Component
0%
Research Effort Categories
Basic
70%
Applied
(N/A)
Developmental
30%
Classification

Knowledge Area (KA)Subject of Investigation (SOI)Field of Science (FOS)Percent
1021131202033%
4027410202034%
4047310202033%
Goals / Objectives
The overarching goal of this project is to develop and deploy heterogeneous teams of autonomous robots (specifically, aerial and ground robots) to enable frequent and dense sampling in the field. The motivating hypothesis is that an increase in sampling density and frequency can indicate noticeable spatiotemporal variability in water potential that would remain otherwise undetected because of insufficient sampling resolution. In pursue of this goal, the project tackles four key objectives, described below.Objective 1 --Robotized Pressure Chamber Development: We will develop all the components to sample individual leaves and assess their water potential, autonomously. The system will consist of a ground mobile base, a manipulator, a pneumatic control board, and an integrated leaf acquisition and pressure chamber device. Components will be optimized through co-design of hardware, sensing, and control.Objective 2 -- Visual Sensing for Accurate Determination of Leaf Water Potential: We will rely on visual sensing to determine the leaf water potential. To achieve so, we will establish new image quality enhancement algorithms using limited training data and develop new algorithms for imitation learning form human experts to help address the long-standing challenges of bubbling and presence on non-xylem water when measuring leaf water potential.Objective 3 -- Multi-robot Coordination and Planning: We will study how to effectively coordinate multiple aerial and ground vehicles to perform targeted sampling in areas of interest while being cognizant of the inherent operational constraints due to the limited energy supply provided by the batteries. Coordination tasks will be cast as optimization problems related to the orienteering problem.Objective 4 -- Evaluation: We will test our developed system in the field at separate locations in northern and southern California where at least four different specialty crops (grapes, almonds, citrus, and avocados) are grown. The validation task serves two purposes. First, it will provide feedback for the iterative evolution of the design and implementation of the hardware/software system we will develop. Second, collected data will be used to test our working hypothesis that current sampling practices fail to capture spatiotemporal variability in leaf water potential.This project iscollaborativebetween UC Merced and UC Riverside
Project Methods
Efforts: Two types of experiments will be performed during the four year project. The first set of experiments aims at perfecting the accuracy of the robotized pressure chamber we will develop. To this end, measurements obtained with the robotized pressure chamber will be cross-validated with leaf water potential measurements obtained with a manually operated portable pressure chamber. These initial experiments will ensure adequate data accuracy before we perform the data analysis process described in the next subsection. We anticipate these experiments to take place in the first and second year of the project. Once the robotized pressure chamber has been perfected, data collection experiments will involve the entire system. We note that UAVs and ground robots do not need to operate at the same time, but it is instead foreseeable that imagery collected by the UAV will be processed off-line. This is an acceptable approach because the underlying physical phenomena are slow varying. Ground robots will then collect data from both the pressure chamber and the soil probe and store them as entries with spatio-temporal references for the subsequent data analysis.Evaluation:The co-robot system we will develop will be tested in the field to prove or disprove the hypothesis that current interpolation approaches based on few measurements per tens of acres fail to capture significant variability in leaf water potential. The value of the hypothesis is that more accurate estimates are essential for tuning inputs and implementing precision agriculture practices.All outdoor robot testing will adhere to any applicable regulations, as for example in the case of aerial robot testing which is regulated by UC and FAA policies (and in which case we will work with the UC Center of Excellence on Unmanned Aircraft System Safety hosted at UC Merced). All robot testing and equipment use will adhere to Standard Operating Procedures (SOPs) developed by the PIs and approved by both institutions' Environmental and Health Safety (EHS) departments.In addition to simulation models and lab tests, the proposed system will be deployed and evaluated in various testbeds where different specialty crops are grown. Specifically, we have identified four testbeds to evaluate system performance. These testbeds include (1) an experimental vineyard managed by one of our industrial partners located in Firebaugh (Fresno county, CA), and (2) almond orchard located in Merced County. In Riverside County, in consultation with local commercial partners, tests will be performed in the Agricultural Operations (Ag Ops) facilities managed by UC Riverside where (3) citrus and (4) avocados are grown. These testbeds are geographically distributed and used to grow different crops, thus allowing to operate the proposed solution under heterogeneous conditions.

Progress 11/01/21 to 10/31/22

Outputs
Target Audience:This project relies on a strong collaboration with the Almond Board of California (www.almonds.com). During this second year we have engaged with scientists and practioners from the board to keep our research aligned with the interests of the growers they represent. In particular, on May 13, 2022 we have visited an almond orchard located near Merced (CA) and together with the owner we actively took part in a complete set of pressure chamber measurements throughout the orchard. This process has allowed us to refine our approach for the selection of the sampling locations. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided?This grant is fully supporting one PhD student (Azin Shamshirgaran) who started in January 2020. Various undergraduate students have also been involved as lab assistants in support of this project. How have the results been disseminated to communities of interest?During this period, we have published three papers at international robotics conference (see section "Products" for details.) What do you plan to do during the next reporting period to accomplish the goals?During the next year, we will integrate the device developed at UC Riverside with the robots currently present at UC Merced, and commence field validation (objective 4). While this was anticipated to be done in year two, we are postponing this goal to year three to take advantage of a new manipulator (TM-900) we have secured at no additional cost to this project. With our commercial partner we will continue to engage almond growers to get domain feedback about our proposed solution. In year three we anticipate testing our sampling algorithms in the field.

Impacts
What was accomplished under these goals? This project is being jointly developed by UC Merced and UC Riverside, and consequently the four objectives are split between the two institutions. The UC Merced team is mostly in charge of objective 3 (planning) and objective 4 (evaluation). As the partner institution is still working towards finalizing the design and implementation of the robotized pressure chamber, objective 4 has not been pursued yet, and we have mostly been concerned with the planning aspect. With regard to the planning problem tackled at UC Merced, in year two we have studied the information path planning problem (IPP) for a single robot in a stochastic environment with static obstacles subject to a preassigned constraint on the distance it can travel. Given a set of candidate sampling locations, i.e., the location of the sentinel trees where measurements should be made, the objective is to determine a path for the robot that allows to visit as many sampling locations as possible to accurately reconstruct an unknown underlying scalar field while not exceeding the assigned travel budget. Owing to the fact that Gaussian Processes (aka kriging) are one of the leading approaches for modeling physical phenomena such as soil moisture, our algorithm embraces this representation and builds upon it. More specifically, our algorithm balances exploration and exploitation to determine a sequence of locations ensuring that a preassigned final site is reached before the budget is consumed. Using mutual information as a reward criterion, as well as a generative model to predict consumed energy, the algorithm iteratively determines where to sample next, and when to end the mission. Our findings are validated in simulation in various scenarios and lead to a better reconstruction with less failures when compared with other methods. In our formulation, "better reconstruction" means a reconstruction with smaller error measured in terms of mean squared error (MSE).

Publications

  • Type: Conference Papers and Presentations Status: Published Year Published: 2022 Citation: " A. Shamshirgaran, S. Carpin. "Reconstructing a Spatial Field with an Autonomous Robot Under a Budget Constraint". Proceedings of the 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems, 8963-8970
  • Type: Conference Papers and Presentations Status: Published Year Published: 2022 Citation: " T. Thayer, S. Carpin. "Solving Stochastic Orienteering Problems with Chance Constraints Using Monte Carlo Tree Search". Proceedings of the 2022 IEEE International Conference on Automation Science and Engineering, 1170-1177
  • Type: Conference Papers and Presentations Status: Published Year Published: 2022 Citation: S. Carpin. "Scheduling problems for robotics in precision agriculture". Proceedings of the 2022 IEEE International Symposium on Circuits and Systems", 1357-1361


Progress 11/01/20 to 10/31/21

Outputs
Target Audience:This project relies on a strong collaboration with the Almond Board of California (www.almonds.com). During this first year we have engaged with scientists and practioners from the board to keep our research aligned with the interests of the growers they represent. More precisely, on May 25, 2021 we have visited an almond orchard located near Modesto (CA) and spent a day with board members performing measurements of leaf water potential using a pressure chamber. This has informed the design of the mechanism being developed at UC Riverside. We have also regularly met with board members to update them about our progress and collect their feedback about our iterative design process. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided? Nothing Reported How have the results been disseminated to communities of interest?During this persiod, we have published two papers at a top international robotics conference (IROS -- see section "Products") and we have submitted two mor to another top international robotics conference (ICRA -- see section "Products"). What do you plan to do during the next reporting period to accomplish the goals?During the next (second) year, we will integrate the device developed at UC Riverside with the robots currently present at UC Merced, and commence field validation (objective 4). With our commercial partner (Almond Board of California), we are scheduling meetings with almong growers to get domain expertise about how they select the sentinel trees to monitor during the growing season to determine water potential. Our objective for the next year are therefore: - integrate the pressure chamber with the mobile robot; - integrate our planning algorithms with inputs provided by almond growers regarding "interesting" regions to sample; - commence field validation.

Impacts
What was accomplished under these goals? This project is being jointly developed by UC Merced and UC Riverside, and consequently the four objectives are split between the two institutions. The UC Merced team is mostly in charge of objective 3 (planning) and objective 4 (evaluation). As the partner institution is still working towards finalizing the design and implementation of the robotized pressure chamber, objective 4 has not been pursued during the first year, and we have mostly been concerned with the planning aspect. Water potential measurement is currently implemented using very sparse sampling, and the underlying objective of this project is to develop a robotic system that can scale up the efficiency of this process. One of the complicating factors in this process is that measurements must be taken during a restricted temporal window -- typically between 11am and 3pm. Additionally, robots have limited battery life and therefore it is necessary to solve an optimization problem where the input is a set of locations where data should be acquired (sentinel trees) as well as a temporal deadline and an energy budget. The output is a schedule (or route) that determines the subset of sentinel trees that should be visited and the ordering. A further complicating aspect is that the time necessary to collect a sample is not deterministic, but is rather influenced by numerous factors (e.g., it may take quite some time for the robot to be able to collect a leaf). In the four papers produced during this period (two published and two submitted), we have explored different techniques to solve this optimization problem. The two papers appeared at IROS refine a previous planning approach that cast this problem as an instance of the orienteering problem and solve them using a method based on constrained Markov Decision Process. The emphasis of these two works have been in decreasing the computational costs and make the methods more robust to unexpected events occurring during execution (e.g., large deviations from the expected costs). In addition, we have developed two new approaches to solve the aforementioned optimization problem. The first relies on reinforcement learning and tackles the problem of learning from simulated experience how to select the next location to visit. The second instead relies on Monte-Carlo tree search, and tackles the problem of dynamic replanning when low probability events occur (e.g., abnormally high times to complete an operation). Both these new methods have been described in two papers submitted to the 2022 ICRA conference. @font-face { panose-1:2 4 5 3 5 4 6 3 2 4; mso-font-charset:0; mso-generic- mso-font-pitch:variable; mso-font-signature:3 0 0 0 1 0;}@font-face { panose-1:2 15 5 2 2 2 4 3 2 4; mso-font-charset:0; mso-generic- mso-font-pitch:variable; mso-font-signature:-536859905 -1073732485 9 0 511 0;}p.MsoNormal, li.MsoNormal, div.MsoNormal {mso-style-unhide:no; mso-style-qformat:yes; mso-style-parent:""; margin:0in; mso-pagination:widow-orphan; ; mso-ascii- mso-ascii-theme-font:minor-latin; mso-fareast- mso-fareast-theme-font:minor-latin; mso-hansi- mso-hansi-theme-font:minor-latin; mso-bidi- mso-bidi-theme-font:minor-bidi;}p {mso-style-noshow:yes; mso-style-priority:99; mso-margin-top-alt:auto; margin-right:0in; mso-margin-bottom-alt:auto; margin-left:0in; mso-pagination:widow-orphan; ; mso-fareast-}.MsoChpDefault {mso-style-type:export-only; mso-default-props:yes; mso-ascii- mso-ascii-theme-font:minor-latin; mso-fareast- mso-fareast-theme-font:minor-latin; mso-hansi- mso-hansi-theme-font:minor-latin; mso-bidi- mso-bidi-theme-font:minor-bidi;}div.WordSection1 {page:WordSection1;}

Publications

  • Type: Conference Papers and Presentations Status: Published Year Published: 2021 Citation: T. Thayer, S. Carpin. "A Resolution Adaptive Algorithm for the Stochastic Orienteering Problem with Chance Constraints" Proceedings of the 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems, 6388-6395
  • Type: Conference Papers and Presentations Status: Published Year Published: 2021 Citation: T. Thayer, S. Carpin. "A Fast Algorithm for Stochastic Orienteering with Chance Constraints." Proceedings of the 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems, 7398-7945
  • Type: Conference Papers and Presentations Status: Submitted Year Published: 2021 Citation: A. Shamishirgaran, S. Carpin. "Spatial field learning with an autonomous robot". Proceedings of the 2022 IEEE International Conference on Robotics and Automation.
  • Type: Conference Papers and Presentations Status: Submitted Year Published: 2021 Citation: T. Thayer, S. Carpin. "Solving Stochastic Orienteering Problems with Chance Constraints Using Monte Carlo Tree Search". Proceedings of the 2022 IEEE International Conference on Robotics and Automation.