Source: UNIVERSITY OF CALIFORNIA, DAVIS submitted to NRP
MECHANICAL MASS-HARVESTING OF TREE FRUITS USING INNOVATIVE MULTI-LEVEL FRUIT CAPTURE SYSTEMS
Sponsoring Institution
National Institute of Food and Agriculture
Project Status
COMPLETE
Funding Source
Reporting Frequency
Annual
Accession No.
1023838
Grant No.
2020-67021-32428
Cumulative Award Amt.
$400,000.00
Proposal No.
2019-06456
Multistate No.
(N/A)
Project Start Date
Aug 15, 2020
Project End Date
Aug 14, 2024
Grant Year
2020
Program Code
[A1521]- Agricultural Engineering
Recipient Organization
UNIVERSITY OF CALIFORNIA, DAVIS
410 MRAK HALL
DAVIS,CA 95616-8671
Performing Department
Ag:Biological & Ag Engineering
Non Technical Summary
Mechanical harvesting is crucial for the fruit industry, especially given the rising cost of farm labor and the labor shortages. However, mechanical harvesting for tree fruits is notoriously difficult because existing methods such as trunk or canopy shaking result in unacceptable fruit damage. The development of "fruiting wall" type tree architectures, where almost all fruits are visible and easily accessible has enabled robotic harvesting to emerge as a potential mechanization solution, for apples. However, not all fruit trees are or can be trained as "fruiting walls". Three-dimensional tree architectures like open-center, central-leader and tall spindle are extremely widespread, and research has shown that their large and complex structures severely limit fruit visibility and reachability. This limitation reduces the applicability of computer vision-guided robots. Yet, our growers urgently need practical, cost-effective mechanical harvesting solutions.This research aims to advance shake-catch mechanical harvesting approaches for fruit trees by developing innovative fruit capture and retrieval technology. Our approach utilizes a large number of adjacent telescopic booms that carry soft, actuated inflatable cylinders. After insertion into the canopy's fruit-bearing volume, the booms of a single layer form a soft 'catching surface' for fruits. Multiple boom layers, at different heights, will be inserted, and will collectively establish a Multi-level Fruit Catching and Retrieval (MFCR) system for fruits detached by trunk shaking. Instead of expensive vision-guided robot arms, a large number of low-cost, simple actuators (booms and tubes) are used to capture and retrieve fruits. Key findings will be shared with growers, industry, academia and the general public using well-established channels available to our team.
Animal Health Component
25%
Research Effort Categories
Basic
25%
Applied
25%
Developmental
50%
Classification

Knowledge Area (KA)Subject of Investigation (SOI)Field of Science (FOS)Percent
4021115202050%
4021114202050%
Knowledge Area
402 - Engineering Systems and Equipment;

Subject Of Investigation
1114 - Peach; 1115 - Pear;

Field Of Science
2020 - Engineering;
Goals / Objectives
The overall goal of this work is to facilitate the development of commercial, cost-effective harvesting machines for large trees with three-dimensional canopies, based on the shake-catch approach. To do so, this project will overcome the main problem of shake-catch, i.e., catching fruits as they fall through the canopy - during shaking - before they hit other branches or fruits.Our proposed concept utilizes concave telescopic booms, that carry soft, actuated inflatable cylindrical tubes on top of them. Booms will be inserted from both sides of the tree, thus establishing a Multi-level Fruit Catching and Retrieval (MFCR) system that spans the entire volume of the fruit-bearing canopy.Objective #1: Model-based design of MFCRThe overall goal of this objective is to generate 3D architectural models of cling peach and pear trees that are representative of the range of architectures that would be observed in typical production systems, and use them to guide the design of the MFCR system, and evaluate its performance in simulation.Objective #2: Fabrication and control of MFCRUnder this objective, a physical embodiment of the MFCR will be designed and fabricated, and a control system will be developed to properly sequence the motorized actuation of its elements.Objective #3: Evaluation of MFCR performanceIn year 1, variants of booms and tubes will be fabricated and tested, and therefore, in-lab fruit catching experiments will be performed, mainly to assess qualitatively the operation of the prototypes. In years 2 and 3, quarter-tree scale MFCRs will be evaluated in orchards, with primarily open-center clingstone peach and pear trees.
Project Methods
Activity 1A. Development of digital tree modelsthis activity will focus on determining the appropriate ranges of model parameters that represent architectures typically found in cling peach and pear systems. In order to determine these parameters, we will use laser-scanning (LiDAR) data to measure the tree structure and fruit distribution of actual trees, which provides millions of (x,y,z) Cartesian coordinates that produce detailed mappings of plant geometry in 3D.Activity 1B. Simulation of canopy penetration and fruit retrievalTo evaluate the extent of boom penetration, we will calculate intersections between virtual branches, fruit, and harvester booms/tubes and generate 'interference metrics' that can be used to guide the design. For laser-scanned trees, the normalized return intensity recorded by the instrument for each point will be used as a surrogate measure of branch thickness based on a calibration procedure, as smaller branches tend to give returns at lower intensity.Activity 1C. Optimization of MFCR structureThere is a very large number of parameters related to geometry and structure that must be selected carefully. We will select reasonable ranges and constraints for these parameters and use stochastic optimization methods to optimize them. Inside the inner loop of the optimizer, for a particular instance of an MFCR design (set of parameters), a large number of Monte-Carlo shake-catch simulations (from Activity 1B) will be executed, that sample the modeled pear and cling peach tree structures and corresponding fruit distributions, and the appropriate initial fruit accelerations. Mean values of fruit collection and retrieval efficiencies will be used in the optimization's objective function.Objective #2: Fabrication and control of MFCRActivity 2A. Fabrication of MFCR unitWe will start with small-scale prototypes and lab experiments to optimize subsystems such as the inflatable tubes, telescopic booms, prior to building a large, quarter-tree machine. The crucial elements of the MFCR are the retractable support booms and the inflatable tubes carried by those booms. The booms will consist of nested segments which will extend to approximately double their contracted length. These segments need to be lightweight, impact resistant, and sufficiently rigid so as not to sag too much when extended. There must also be some degree of flexibility so the boom may deflect slightly to the side if it encounters an obstacle such as a large tree branch while extending.Activity 2B. Actuation and control of MFCR systemThe MFCR will be computer controlled to properly sequence the motorized actuation of the reels, opening and closing of the air valves, and monitoring the air pressure within the tubes. An industrial Programmable Logic Controller (PLC) may be used, but cheaper microcontrollers will be tried first for prototyping. Each motor will have a relative encoder and appropriate contact switch to sense the boom's fully retracted position and current extension. Pressure sensors will also monitor the inflatable tube pressure. Supervisory control software will monitor the actuation commands and sensed states of the system to detect anomalies that may trigger interruption of the operation (e.g., boom getting stuck).Objective #3: Evaluation of MFCR performanceActivity 3A. Evaluation of fruit collection capabilityStatistics of fruit impact force will be generated (mean, maximum, 95th percentile of impact force) and used to tune MFCR operating parameters (e.g., air pressure). Also, key harvester performance metrics will be assessed. These include FCE, FRE, fruit damage (% of fruits caught at 'Downgrade' level), selectivity of maturity of fruit removed (size, firmness and brix ranges of fruit detached), and tree harvest rate (trees/minute) from machine and experiment logs.Activity 3B. Postharvest assessment of fruit damageAverage numbers of quality traits will be calculated from individual fruit data for comparison between hand and machine harvested fruits and between fruit groups from different MFCR layers (using two-sample t-test).Objective #4: Outreach and DisseminationActivity 4A. Outreach and DisseminationThe primary goals of outreach and dissemination activities will be to 1) elicit feedback from potential end-users in order to improve usability and ultimately probability of adoption, and 2) widely disseminate results of the project to the industry and scientific communities. Industry Outreach: Project findings and related information will be shared directly to growers during annual grower research meetings, seasonal horticulture field meetings, grower conventions, and field demos. Scientific outreach: The project is an integrated, collaborative effort among engineering and horticultural scientists. Progress and results will be shared with colleagues across these disciplines via professional society publications and presentations. Information will also be shared via institutional publications and a project website to gain maximum visibility for the research.

Progress 08/15/20 to 08/14/24

Outputs
Target Audience:One Ph.D. student andtwo M.Sc. students were mentored in this project. Also,one project scientist, one post-doctoral researcher, and one full-time development engineer were engaged in research and development. Targeted audiences were reached via workshops, seminars, presentations to Commodity Boards, and conference presentations. The audiences included growers and allied industry, entrepreneurs, researchers, faculties, students, extension agents, visiting scholars, guest lecturers, and the general public. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided?Ph.D. student (Kaiming Fu) • Completed the required coursework for Ph.D. degree in ECE and passed the exam to advance to candidacy. • Earned a double-major master's degree in statistics (Data Science track). • Became proficient in use of Helios 3D plant modeling framework. • Was trained to use terrestrial LiDAR scanner and process data. • Attended weekly research meetings with project PIs. ?• Attended departmental research seminars. • Attended the ASABE 2022 Annual International Meeting and presented current research outcomes. • Attended International Forum for Agricultural Robotics (FIRA USA) conference, 2022 • Completed Coursera online courses: GPU Programming Specialization (Introduction to Concurrent Programming with GPUs; Introduction to Parallel Programming with CUDA; CUDA at Scale for the Enterprise; CUDA Advanced Libraries). • Prepared and publishedresearch paper and is currently finishing his dissertation draft. M.Sc. student (Isidro Pantoja-Gomez) Finished coursework Attended lab meetings Worked on fabricating the MFCR system. Learned how to use an "artificial fruit," a commercial instrumented sphere that records impact forces as it collides with objects. Designed "fruit" drop experiments on the MFCR, collected data, and analyzed results. Learned how to use the ANSYS CAD simulation software. Wrote a MSc thesis document, and graduated. How have the results been disseminated to communities of interest?The PIs gave may talks and presentations to growers, researchers and industry. "Harvest-support technology development at UC Davis", Stavros Vougioukas, invited speaker. UCANR Small Fruit and Vegetable Meeting, Aug. 18, 2020. "Human-robot collaboration for fruit harvesting". Stavros Vougioukas, panelist. CITRIS-CPAR, Open Problems for Robots inFood Supply Chain, Aug. 24, 2020. "Co-Robotic harvest-aid platforms can increase harvesting speed". Stavros Vougioukas, invited speaker. Washington State Tree Fruit Extension and Oregon State University Extension Service, Oct. 15, 2020. "Human and ag-robot collaboration for fruit harvesting", Stavros Vougioukas, invited speaker. Ag Expo 2021: The Future of Farming. Un. of Florida, May 10-11. 2021. "Computer-Aided Design and Management of Agricultural Production Systems", Brian Bailey, invited speaker. Seed Central event on Digital and AI Technologies for Agriculture. December 9, 2021. "Overview of the Helios 3D Plant Modeling Framework", Brian Bailey, invited speaker. Crops in silico Symposium. May 13, 2022 "Overview of the Helios 3D Plant Modeling Framework", Brian Bailey, invited speaker. Iowa State University AI Institute for Resilient Agriculture Seminar Series. May 20, 2022 "Collaborative-Robotic Harvest-aid platforms can increase harvesting speed." Webinar during the 2021 World Ag Expo. Aug. 19, 2021. "Human-robot Collaboration for Fruit Harvesting." CITRIS Research Webinar. Sept. 15, 2021. "Agricultural Robotics and labor." Digital Transformation Workshop, UC Davis Graduate School of Management. Nov. 10, 2021. "Robotic harvesters and harvest-aids: Challenges and opportunities." International Conference on Digital Technologies for Sustainable Crop Production (DIGICROP 2022), Bonn, Germany, March 28, 2022. "Robotic harvesting and precision yield mapping." UC ANR Precision Ag Workgroup Meeting. Mar. 9, 2022. "Agricultural robotics, robotic-aided harvesting, and automation." Webinar organized by the Phenomics and Plant Robotics Center (P2RC) at the University of Georgia. Mar. 17, 2022. "Worker and Robot Collaboration for Fruit Harvesting." Project meeting of UC MRPI Labor & Automation in California Agriculture: Equity, Productivity, & Sustainability, UC Merced, May 20, 2022. "Theory and Research in Mechanized & Robotic Harvesting." Emerging Technologies Workshop, Mechanical &Robotic Session. UC Davis, May 24, 2022. "Integration of machine learning and 3D plant biophysical models for acceleration of agricultural system design, management, and crop improvement", Brian Bailey, invited speaker. Alphabet/google technical seminar series. October 12, 2022 "Mechanized Harvesting, Autonomous Machines & The Future of Labor." UCANR Pomology Cross-Commodity Discussion Meeting. Online. September 16. 2022. "Harvest-Assist Collaborative Robots." FIRA USA, Fresno, CA. October 18, 2022. "Agricultural Robots: Technology, Applications, and Adoption." Sustainable Ag Expo. San Luis Obispo. November 14, 2022. "Robotic and Robot - Assisted Fruit Harvesting." Conference on Farm Labor: Demand, Supply, and Markets. UC Davis. March 16, 2023. "Efficiency and Fairness Metrics for Scheduling Harvest-assist Robots." IEEE International Conference on Robotics and Automation (ICRA2023) "TIG-IV: From Farm To Fork" workshop (online). May 27, 2023. "The present and future of the use of autonomous equipment and robotic harvesters in field-based fruit production." HortForum Webinar. International Society for Horticultural Science April 27, 2023. "Robotic Harvest-Aids & worker Well-Being." Western Center for Agricultural Health and Safety Workshop on Emerging Technology in Agriculture: Keeping Health & Safety at the Forefront. UC Davis, May 11, 2023. "Canning Peach Mechanization Research Fund Meeting." UC Davis, May 7, 2024. "Advances in automation technologies and effect on farm workers."California Plant and Soil Conference.February 6,2024. "Labor in 2023 and the Turn Towards Automation." IFTA Annual Conference, Yakima, Washigton. February 27, 2024. What do you plan to do during the next reporting period to accomplish the goals? Nothing Reported

Impacts
What was accomplished under these goals? Objective #1: Model-based design of MFCR Asimulator was completed to create a computer-aided MFCR design tool and illustrate its capabilities. The simulations represent interactions between a proposed harvester design, tree architecture, and fruit to quantifyhow the harvester design affects the quantity and quality of harvested fruit. The tool allows for the determination of the optimal harvester design based on a defined set of variable design parameters and a specified objective function. In this study, the number of harvester layers was determined that results in the optimal number of "marketable" fruits harvested. When applied to a pear orchard, it was determined that the optimal number of layers was 4. Next, the simulator was extended toincorporate more design parameters. This has required making the simulations more computationally efficient and integrating an efficient optimization algorithm. To improve efficiency to facilitate thousands of candidate design evaluations as part of the optimization, a new simulator was developed for calculating intersections between the harvester geometry, tree, and fruit based on a single pre-computation that can be efficiently queried during the optimization process. Based on the simulator, we used a genetic algorithm to optimize a multi-level fruit-catching system for marketable fruits, setting a benchmark for subsequent validation of the adjustable MFCR system. By leveraging voxel-based GPU computing for accelerated interference analysis, we explored a broader range of harvester parameters. Our optimization focusedon three critical design parameters: the number of layers, the heights of these layers, and the number of arms per layer, all of which significantly influence the yield of marketable fruit. We then compare this optimized design to an adjustable system that modifies layer heights based on visible fruits detected by cameras in simulations. Utilizing synthetic image data, we trained a fruit detector with YOLOv5 to accurately map visible fruit distribution.We subsequently optimized layer heights using Gurobi to minimize fruit dropping distances, a crucial factor in reducing fruit damage. The performance comparison between the fixed and adjustable MFCR designs, conducted across various tree size datasets, showsed that the adjustable MFCR design offers notable adaptability for unseen trees in the test dataset by adjusting layer heights according to visible fruit distribution, especially when the training dataset is limited. Objective #2: Fabrication and control of MFCR We constructed 2-meter long booms made of linked aluminum blocks. The unique linking design allows the booms to be folded or coiled yet remain rigid when extended horizontally. Each block was machined to accommodate loops of hydraulic hose extending outwards from the sides of the block. Hydraulic hose was selected because it is flexible enough to deflect around large branches and tough enough to withstand repeated scraping against branches. Closely spaced loops of hose provide a compliant catching surface to minimize damage to the fruit. Considerable testing of various loop designs was conducted. As there is a small amount of play in each joint of the linked blocks, the accumulated play resulted in about a 15-centimeter droop in a two-meter-long boom. Various types of spacers between blocks were tested to offset the play. It was found that thin rubber edge protectors affixed to the outer edges of the blocks reduced the droop to less than 3 centimeters.Methods of stowage and deployment of the booms were explored with an emphasis on reliability, robustness, and cost. We arrangedthe booms in vertical chutes with a 90- degree bend at the bottom. In such a scheme, most of the blocks are stacked vertically in a chute, with the end block extending horizontally from the bottom of the chute. Wheels with "fingers" protruding radially were mounted at the bottom of the chute, and those fingers will engage the hose loops next to the side walls of each block. Rotating the wheels with an electric motor drives the blockchain forward or backward, causing it to extend or retract. Objective #3: Evaluation of MFCR performance Many variations of hose loops are possible and were tested by varying parameters such as the length of the loop, the spacing between loops, the angle of the loops relative to horizontal, and whether the far end of the loop was free or supported. Three methods were used to evaluate the performance of the MFCR. The first method was a simulation using ANSYS Explicit Dynamics. A computer model of the booms was constructed and simulated fruit was dropped onto these booms from various heights and in different locations along the catching surface. The ANSYS program then calculated the G forces experienced by the fruit when impacting the catching surface. Past literature on the subject of fruit bruising specifies a "bruising threshold" of about 25 G's. The simulations indicated that the fruit would experience less than 25 G's when landing on our catching surface. This was confirmed by our second method which made use of an Impact Recording Device or IRD. The IRD is a ball about the size and weight of a peach, and it houses a triaxial recording accelerometer. When dropped on our catching surface, the IRD registered about 19 G's on average. The third method involved taking our catching surface to a peach orchard. The booms were inserted into a tree canopy, and a large number of peaches were snipped at the stem and allowed to drop freely onto the catching surface. Also, a large number of peaches were picked by hand to provide a control sample typical of traditional peach harvesting. All peaches were collected, packed in padding, and transported to the UC Davis Postharvest Technology Center. At the Center each peach was examined for bruising and damage by trained evaluators. The results showed less bruising on the peaches caught on the catching surface than those which were picked by hand.

Publications

  • Type: Peer Reviewed Journal Articles Status: Published Year Published: 2024 Citation: Fu, K., Vougioukas, S. G., Bailey, B. (2024). Computer-aided design and optimization of a shake-catch fruit catching and retrieval soft fruit harvester. Computers and Electronics in Agriculture. Computers and Electronics in Agriculture, 225, 109334.


Progress 08/15/22 to 08/14/23

Outputs
Target Audience:One Ph.D. student, two M.Sc. students, one full-time development engineer, and one project scientist were engaged in research and development. The Ph.D. student was co-advised and mentored by co-PI Bailey and PI Vougioukas. The MSc students were advised and mentored by PI Vougioukas, primarily through laboratory instruction. Also, the PI and Co-PI gave several presentations on the modeling and design of mechanical harvesting technologies. Targeted audiences were reached via workshops, seminars, and conference presentations. The audiences included growers, entrepreneurs, researchers, faculties, students, extension agents, and the general public. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided?PhD student (Kaiming Fu) • Attended weekly research meetings with project PIs. Presented at 3 lab meetings. • Attended departmental research seminars. Attended International Forum for Agricultural Robotics (FIRA USA) conference, Oct. 18-20, 2022 M.Sc. student (Isidro Pantoja-Gomez) Isidro finished the required coursework and worked on fabricating the small-scale MFCR system. He also learned how to use the ANSYS CAD simulation software to model fruit drop experiments on the MFCR. He also performed real drop experiments with an "artificial fruit" in the lab and field and performed real fruit (cling peaches) drop experiments in a commercial orchard. He is currently analyzing the results and writing his MSc thesis. How have the results been disseminated to communities of interest?Co-PI Bailey gave the following presentations: "Integration of machine learning and 3D plant biophysical models for acceleration of agricultural system design, management, and crop improvement", Brian Bailey, invited speaker. Alphabet/google technical seminar series. October 12, 2022 PI Vougioukas gave the following presentations: "Mechanized Harvesting, Autonomous Machines & The Future of Labor." UCANR Pomology Cross-Commodity Discussion Meeting. Online. September 16. 2022. "Harvest-Assist Collaborative Robots." FIRA USA, Fresno, CA. October 18, 2022. "Agricultural Robots: Technology, Applications, and Adoption." Sustainable Ag Expo. San Luis Obispo. November 14, 2022. "Robotic and Robot - Assisted Fruit Harvesting." Conference on Farm Labor: Demand, Supply, and Markets. UC Davis. March 16, 2023. "Efficiency and Fairness Metrics for Scheduling Harvest-assist Robots." IEEE International Conference on Robotics and Automation (ICRA2023) "TIG-IV: From Farm To Fork" workshop (online). May 27, 2023. "The present and future of the use of autonomous equipment and robotic harvesters in field-based fruit production." Hort Forum Webinar. International Society for Horticultural Science April 27, 2023. "Robotic Harvest-Aids & worker Well-Being." Western Center for Agricultural Health and Safety Workshop on Emerging Technology in Agriculture: Keeping Health & Safety at the Forefront. UC Davis, May 11, 2023. What do you plan to do during the next reporting period to accomplish the goals?For Objective 1, our goal is to extend the simulator and optimizer to enable the MFCR to adjust the distances between layers based on an estimate of the fruit distribution of each tree. For Objective 2, we plan to build a system that retracts and extends the arms from and into the canopy, respectively. For Objective 3, we plan to perform more artificial and real fruit drop experiments on a larger version of the MFCR in the lab and an orchard.

Impacts
What was accomplished under these goals? Objective #1: Model-based design of MFCR An initial study was completed to create a computer-aided MFCR design tool and illustrate its capabilities. The simulations simultaneously represent interactions between a proposed harvester design, tree architecture, and fruit in order to quantify how the harvester design affects the quantity and quality of harvested fruit. To our knowledge, this is the first time such a system has been created that can simultaneously represent these three aspects within a model-based harvester design. The tool allows for the determination of the optimal harvester design based on a defined set of variable design parameters and a specified objective function. In this study, the number of harvester layers was determined that results in the optimal number of "marketable" fruits harvested. When applied to a pear orchard, it was determined that the optimal number of layers was 4. This work has been submitted for publication and is currently under review. The next stage of model development and application is to incorporate more design parameters. This has required making the simulations more computationally efficient and integrating an efficient optimization algorithm. To improve efficiency to facilitate thousands of candidate design evaluations as part of the optimization, a new scheme was developed for calculating intersections between the harvester geometry, tree, and fruit based on a single pre-computation that can be efficiently queried during the optimization process. We have integrated and tested several candidate optimization algorithms, including the "NOMAD" algorithm and genetic algorithm. Objective #2: Fabrication and control of MFCR This year's efforts have focused on constructing and testing multiple full-size booms that form the catching surface for the fruit. Each boom is two meters long and is made of 27 linked aluminum blocks. The unique linking design allows the booms to be folded or coiled yet remain rigid when extended horizontally. Each block is machined to accommodate loops of hydraulic hose extending outwards from the sides of the block. Hydraulic hose was selected because it is flexible enough to deflect around large branches and tough enough to withstand repeated scraping against branches. Closely spaced loops of hose provide a compliant catching surface to minimize damage to the fruit. Considerable testing of various loop designs was conducted and is described in detail under Objective #3. As there is a small amount of play in each joint of the linked blocks, the accumulated play resulted in about a 15-centimeter droop in a two-meter-long boom. Various types of spacers between blocks were tested to offset the play. It was found that thin rubber edge protectors affixed to the outer edges of the blocks reduced the droop to less than 3 centimeters. Methods of stowage and deployment of the booms were explored with an emphasis on reliability, robustness, and cost. Layers of 2-meter-long booms cannot be driven through an orchard while horizontally extended, as the spacing between the tree canopies is often less than two meters. Attempts were made to wrap the booms around wheels of various diameters and then extend them by rotating the wheel. This proved to be very challenging with small-diameter wheels due to sliding between multiple layers of blocks on the wheel. The blocks did not stay neatly stacked, and the hose loops hindered the use of restraining rims. This method worked better with large-diameter wheels, but unfortunately, the necessary wheel diameter was greater than the desired distance between the catching layers. This means wheels in one layer would interfere with the wheels in the layers above and below that layer. Now, we are focusing on arranging the booms in vertical chutes with a 90-degree bend at the bottom. In such a scheme, most of the blocks are stacked vertically in a chute, with the end block extending horizontally from the bottom of the chute. Wheels with "fingers" protruding radially are mounted at the bottom of the chute, and those fingers will engage the hose loops next to the side walls of each block. Rotating the wheels drives the blockchain forward or backward, causing it to extend or retract. The weight of the vertical blockchain provides a gravity assist in extending the boom. A cable attached to the back end of the blockchain may be needed to help pull the blockchain back up the chute during retraction. Objective #3: Evaluation of MFCR performance Many variations of hose loops are possible and were tested by varying parameters such as the length of the loop, the spacing between loops, the angle of the loops relative to horizontal, and whether the far end of the loop was free or supported. We also tested covering the loops with sleeves of rip-stop nylon to provide a catching surface with minimal openings. Unfortunately, covering the loops with fabric created an undesirable trampoline effect. It was found that loops having a length of about 40 centimeters and supported at the far end provided an adequate catching surface. The loops could be angled slightly upwards by compressing them inside the blocks with short pieces of aluminum tubing. By angling the loops the fruit was less likely to fall off the end of the loop and more likely to gently roll down towards the boom. Three methods were used to evaluate the performance of the MFCR. The first method was a simulation using ANSYS Explicit Dynamics. A computer model of the booms was constructed and simulated fruit was dropped onto these booms from various heights and in different locations along the catching surface. The ANSYS program then calculated the G forces experienced by the fruit when impacting the catching surface. Past literature on the subject of fruit bruising specifies a "bruising threshold" of about 25 G's. The simulations indicated that the fruit would experience less than 25 G's when landing on our catching surface. This was confirmed by our second method which made use of an Impact Recording Device or IRD. The IRD is a ball about the size and weight of a peach, and it houses a triaxial recording accelerometer. When dropped on our catching surface, the IRD registered about 19 G's on average. The third method involved taking our catching surface to a peach orchard. The booms were inserted into a tree canopy, and a large number of peaches were snipped at the stem and allowed to drop freely onto the catching surface. Also, a large number of peaches were picked by hand to provide a control sample typical of traditional peach harvesting. All peaches were collected, packed in padding, and transported to the UC Davis Postharvest Technology Center. At the Center each peach was examined for bruising and damage by trained evaluators. The results showed less bruising on the peaches caught on the catching surface than those which were picked by hand.

Publications

  • Type: Journal Articles Status: Under Review Year Published: 2023 Citation: Fu, K., Vougioukas, S.G., Bailey, B. (2023) Computer-aided design and optimization of a multi-level fruit-catching system for soft fruit harvesting. Computers and Electronics in Agriculture.
  • Type: Journal Articles Status: Published Year Published: 2023 Citation: Arikapudi R., Vougioukas, S.G. (2023). Robotic Tree-fruit Harvesting with Arrays of Cartesian Arms: A Study of Fruit Pick Cycle Times. Computers and Electronics in Agriculture (211), 108023.


Progress 08/15/21 to 08/14/22

Outputs
Target Audience:One Ph.D. student, one M.Sc. student, and one full-time development engineer were engaged in research and development. The Ph.D. student was co-advised and mentored by co-PI Bailey and PI Vougioukas. The MSc student was advised and mentored by PI Vougioukas, primarily through laboratory instruction. Also, the PI and Co-PI gave several presentations on the modeling and design of mechanical harvesting technologies. Targeted audiences were reached via workshops, seminars, and conference presentations. The audiences included growers and allied industry, entrepreneurs, researchers, faculties, students, extension agents, and the general public. Changes/Problems:As noted in our previous report, the COVID-19 pandemic brought the design and lab testing process almost to a halt because we could not be physically present to fabricate and test alternative designs. Hence, a lot of that work had to be done during the second year of the project. Our approach has not changed, but as a result, we anticipate that a no-cost extension will be necessary for the project since its end date is 8/14/2023, i.e., the end of next summer. What opportunities for training and professional development has the project provided?Ph.D. student (Kaiming Fu) • Has completed the required coursework for Ph.D. degree in ECE and passed the exam to advance to candidacy. • Has earned a double-major master's degree in statistics (Data Science track). • Attended weekly research meetings with project PIs. • Attended the ASABE 2022 Annual International Meeting and presented current research outcomes. • Attended departmental research seminars. • Completed Coursera online courses: GPU Programming Specialization (Introduction to Concurrent Programming with GPUs; Introduction to Parallel Programming with CUDA; CUDA at Scale for the Enterprise; CUDA Advanced Libraries). M.Sc. student (Isidro Pantoja-Gomez) Isidro is finishing the required coursework and has been working on fabricating the small-scale MFCR system. He also learned how to use an "artificial fruit," a commercial instrumented sphere that records impact forces as it collides with objects; he designed "fruit" drop experiments on the MFCR, collected data, and analyzed results. The results are currently being used to optimize the design of the MFCR. How have the results been disseminated to communities of interest?Co-PI Bailey gave the following presentations: "Computer-Aided Design and Management of Agricultural Production Systems", Brian Bailey, invited speaker. Seed Central event on Digital and AI Technologies for Agriculture. December 9, 2021. "Overview of the Helios 3D Plant Modeling Framework", Brian Bailey, invited speaker. Crops in silico Symposium. May 13, 2022 "Overview of the Helios 3D Plant Modeling Framework", Brian Bailey, invited speaker. Iowa State University AI Institute for Resilient Agriculture Seminar Series. May 20, 2022 PI Vougioukas gave the following presentations: "Collaborative-Robotic Harvest-aid platforms can increase harvesting speed." Webinar during the 2021 World Ag Expo. Aug. 19, 2021. "Human-robot Collaboration for Fruit Harvesting." CITRIS Research Webinar. Sept. 15, 2021. "Agricultural Robotics and labor." Digital Transformation Workshop, UC Davis Graduate School of Management. Nov. 10, 2021. "Robotic harvesters and harvest-aids: Challenges and opportunities." International Conference on Digital Technologies for Sustainable Crop Production (DIGICROP 2022), Bonn, Germany, March 28, 2022. "Robotic harvesting and precision yield mapping." UC ANR Precision Ag Workgroup Meeting. Mar. 9, 2022. "Agricultural robotics, robotic-aided harvesting, and automation." Webinar organized by the Phenomics and Plant Robotics Center (P2RC) at the University of Georgia. Mar. 17, 2022. "Worker and Robot Collaboration for Fruit Harvesting." Project meeting of UC MRPI Labor & Automation in California Agriculture: Equity, Productivity, & Sustainability, UC Merced, May 20, 2022. "Theory and Research in Mechanized & Robotic Harvesting." Emerging Technologies Workshop, Mechanical &Robotic Session. UC Davis, May 24, 2022. Ph.D. student Kaiming Fu gave the following presentations: "Computer-aided design and optimization of a shake-catch soft fruit harvester", Kaiming Fu, presenter. ASABE Annual Meeting, Houston, TX. July 19, 2022 What do you plan to do during the next reporting period to accomplish the goals? Nothing Reported

Impacts
What was accomplished under these goals? Objective #1: Model-based design of MFCR We continued the development of a harvester computer-aided design (CAD) prototype framework that can simultaneously represent interactions between a proposed harvester design, tree architecture, and fruit. To our knowledge, this is the first time such a system has been created that can simultaneously represent these three aspects within a model-based harvester design. Brief details of new component developments are listed below: Processed LiDAR dataset of 175 peach and pear trees collected in Year 1 for incorporation into the design analysis. A hierarchical bounding box algorithm was implemented to considerably accelerate simulations, which enabled the utilization of a large tree architecture dataset collected in Year 1. Improved model representation of branch reactive force due to harvester insertion. Improved model representation of interactions between fruit and branches when falling. A design optimization test case was updated based on the above improvements listed above. A manuscript draft has been improved and updated based on the above additions and results. Objective #2: Fabrication and control of MFCR A substantial challenge for the MFCR is the spacing between rows of trees precludes using one-piece booms long enough to reach the center of the trees. An array of long booms cannot be driven through an orchard without the booms striking the tree canopies. Thus, a major focus this past year has been exploring a variety of compact extendable booms. No less than six different methods of creating extendable booms were examined this past year. Some ideas were ultimately rejected as too complex, too heavy, or too expensive, while for other ideas, prototypes were fabricated and tested. Among the concepts considered were the Stored Tubular Extended Member (STEM), which NASA uses to extend solar arrays; nesting extrusions of various shapes; an inflatable boom with orthogonal inflated catching arms; ganging of heavy-duty drawer slides; linked blocks that are similar to a chain but are self-supporting in the horizontal direction. The STEM boom was deemed too complex and expensive, and it lacked the necessary lateral strength. Nesting off-the-shelf extrusions proved to be too bulky, and custom-made extrusions were too costly for prototyping. An inflatable boom was constructed and tested but lacked the necessary strength to support a load of fruit. The drawer slides concept was also built, but attaching a suitable catching surface was unsuccessful. The linked blocks concept, which we call the "block-chain," is the leading contender for the boom design at this point, with several prototypes built and tested in the lab. The block chain is a unique design that can be stowed vertically for easier passage between tree canopies but then is self-supporting when extended horizontally, even to a length of two meters. Lengths of durable but flexible tubing extending from the sides of each block to form a catching surface that can squeeze past obstructions while providing a cushioned landing for the fruit. All the block-chains in each layer can be extended simultaneously with a common actuator. A mechanism is being developed to allow the extension of all other blockchains, even if one or more are obstructed. Objective #3: Evaluation of MFCR performance To facilitate the effectiveness of the catching surface in minimizing damage to the fruit, an Impact Recording Device (IRD) was procured. The IRD is about the size and weight of a large peach, and it records the G-forces when it strikes an object. With the IRD, drop tests onto various surface configurations have been done repeatedly to generate a robust data set. Studies in the literature indicate that fruit is generally not bruised so long as the impact forces do not exceed 25 G's. Our tests show impacts in the 15G range when the IRD is dropped from a height of 12 inches onto a catching surface made of loops of flexible tubing. We've also tested drops onto booms covered in various types of foam rubber padding. However, it appears that loops of tubing laying on the top surface of an aluminum block-chain boom provides adequate cushioning since some of the energy is absorbed by the flexing of the boom itself. The spacing of the booms, the length of the tubing loops, and the angle at which the loops extend from the sides of the block-chain are in the process of being optimized.

Publications

  • Type: Conference Papers and Presentations Status: Published Year Published: 2022 Citation: Fu, K., Vougioukas, S.G., Bailey, B. (2022). Computer-aided design and optimization of a shake-catch soft fruit harvester. ASABE Annual International Meeting. Houston, Texas. Presentation # 2200344.
  • Type: Journal Articles Status: Under Review Year Published: 2022 Citation: Arikapudi R., Vougioukas, S.G. (2022). Robotic Tree-fruit Harvesting with Arrays of Cartesian Arms: A Study of Fruit Pick Cycle Times.Computers and Electronics in Agriculture.


Progress 08/15/20 to 08/14/21

Outputs
Target Audience:One graduate (Ph.D.) student and one full-time development engineer were engaged in research and development. The student was mentored in the first year of the project. Also, the PI and Co-PI gave several presentations related to mechanical harvesting technologies. The audiences included growers and allied industry, entrepreneurs, researchers, faculties, students, extension agents, visiting scholars, guest lecturers, and the general public. Changes/Problems:Our approach has not changed, but the COVID-19 pandemic slowed down the design and testing process because we could not be physically present in the lab to fabricate and test alternative designs. What opportunities for training and professional development has the project provided?PhD student (Kaiming Fu): Coursera online-courses with certificates:Introduction to Data Science in Python; The Arduino Platform and C Programming; Machine Learning. Weekly research meetings with project PIs Has completed required coursework for degree, and will be soon taking the exam to advance to candidacy. Attended departmental research seminars. Prepared draft manuscript describing research conducted on this project, and has prepared a draft research proposal. Was trained to use terrestrial LiDAR scanner and process data. Became proficient in use of Helios 3D plant modeling framework. Development engineer (Dennis Sadowski) Participated in two meetings of the commodities boards where he presented our work on shake/catch, but also listened to all the other presentations at those meetings and increased his knowledge of orchard management, and of pests and diseases affecting fruit trees. He attended every on-campus seminar that was even remotely related to fruit harvesting or shake/catch. He did a lot of individual studies, particularly in the area of inflatable tubes and eversion. Also spent time studying telescoping structures and mechanisms. How have the results been disseminated to communities of interest?PI-Vougioukas gave the following presentations. - "Harvest-support technology development at UC Davis", Stavros Vougioukas, invited speaker. UCANR Small Fruit and Vegetable Meeting, Aug. 18, 2020. - "Human-robot collaboration for fruit harvesting". Stavros Vougioukas, panelist. CITRIS-CPAR, Open Problems for Robots in Food Supply Chain, Aug. 24, 2020. -"Co-Robotic harvest-aid platforms can increase harvesting speed". Stavros Vougioukas, invited speaker. Washington State Tree Fruit Extension and Oregon State University Extension Service, Oct. 15, 2020. - "Human and ag-robot collaboration for fruit harvesting", Stavros Vougioukas, invited speaker. Ag Expo 2021: The Future of Farming. Un. of Florida, May 10-11. 2021. Also, we have developed a manuscript detailing the CAD-based harvester design framework, which demonstrates an example harvester design optimization. This manuscript is in the internal revision stage, and will be submitted for peer review shortly. What do you plan to do during the next reporting period to accomplish the goals? Nothing Reported

Impacts
What was accomplished under these goals? Objective #1: Model-based design of MFCR We have successfully developed a harvester computer-aided design (CAD) prototype framework that is able to simultaneously represent interactions between a proposed harvester design, tree architecture, and fruit. To our knowledge, this is the first time such a system has been created that can simultaneously represent these three aspects within model-based harvester design. Brief details of novel component developments are listed below: The 3D solid model of the proposed harvester design is created in a similar manner as traditional CAD tools, which represents geometry using a mesh of planar elements. We implemented this geometry creation in our software framework such that parameters of the design (e.g., size, number of arms, location of harvester layers) can be modified on-the-fly without requiring an additional step in the workflow to create the geometry in 3rd party software. We developed an approach to represent the 3D tree architecture within the design framework, both directly based on field-measured data and based on a procedural tree generator that can generate an infinite number of hypothetical plant architectures. For the former approach, we collected LiDAR scanning data for roughly 175 peach and pear trees in order to generate the 3D models. The end result is the ability to determine intersections between the proposed harvester design and various plant architectures. We developed a model for fruit falling and impact with other branches and the harvester to quantify frequency and severity of fruit bruising. This ultimately allowed us to predict the fraction of marketable fruit harvested for simulated harvester designs and tree architectures. We performed an illustrative example harvester design optimization based on a multi-level fruit catching and retrieval (MFCR) system in which the optimal number of harvester catching layers was determined in order to maximize the marketable fruit collection efficiency. This illustrated the viability of the system for CAD-based harvester design, and will serve as the basis for continued work through the project. Objective #2: Fabrication and control of MFCR Many ideas were considered over the year. Some were rejected outright as impractical, some reached the stage of conceptual drawings only, and others led to the design and construction of simple prototypes and test apparatus. We mainly experimented with booms that were self-extending through the use of elastic bands and could be retracted by pulling on a cord and booms that extended and retracted using cords. Ideas are constantly refined based on the results of lab and field tests and we are gradually closing in on a concept that is practical and feasible.

Publications