CE Indust/Mnfctr Engineering
Non Technical Summary
The goal of this project is to use proprioception, localized sensing, and observed forces to develop robust, autonomous fruit picking methods. Fresh market tree fruit growers still rely on a large seasonal labor force for harvesting operations. Despite extensive research over the past thirty years, robotic harvesters are not yet commercially available. Prior work has considered manipulation a robot position control problem, disregarding the need for sensor input after physical contact with the fruit. However, when picking fruit such as apples and pears, professional pickers use active perception, incorporating both visual and tactile input about fruit orientation, stem location, and the fruit's immediate surroundings. We propose to embrace this physical contact by incorporating a rich set of in-hand sensors in an extended manipulation feedback loop with the goal of providing fine control over how the fruit is separated from the tree. To overcome the constraints of data collection in the field, we will develop a learning framework for compartmentalizing the tasks and design an instrumented proxy to serve as a training environment.While our primary focus in this project is fresh market apple and pear harvesting, we believe that this framework will be useful for numerous other agricultural applications that involve physical manipulation. For example, harvesting methods used for greenhouse sweet peppers and tomatoes are highly dependent on knowledge of peduncle orientation. However, automating production has been difficult due to similar challenges with occlusions and determining crop orientation. Another potential area of application for this learning framework is plant phenotyping, using soft tactile sensors, in addition to other sensor types, to measure a plant's physical properties.
Animal Health Component
Research Effort Categories
Goals / Objectives
The overarching goal of this project is to use proprioception, localized sensing, and observed forces to develop robust, autonomous fruit picking methods. The proposed project has the following set of objectives:1. Create an extended manipulation feedback loop incorporating a rich set of in-hand sensors with the objective of providing fine control over how the fruit is separated from the tree.2. To overcome the constraints of data collection in the field, we will develop a learning framework for compartmentalizing the tasks and design an instrumented proxy to serve as a training environment.3. Validate closed loop controllers during limited field trials in a commercial apple orchard.
To accomplish our project objectives, we propose (i) Using a rich set of in-hand sensors; (ii) Creating a small set of metrics that directly capture our physical goals; (iii) Setting up a physical training environment that mimics the real world but allows us to both know the correct values for the metrics and add structured "noise"; (iv) Breaking up the learning into a set of manageable components. We will use three evaluation criteria: (i) Improvement in our ability to estimate environmental data as measured by both more accurate values and reduced standard deviations; (ii) Ability to perform the end-to-end task as quantitatively measured by our evaluation metrics on the test apparatus; (iii) Ability to perform the end-to-end task in the real world orchard environment, as measured by reduction in observed collisions (from video), overall accuracy (number of fruits removed), and stem retention (counts).