Source: N Y AGRICULTURAL EXPT STATION submitted to
PRECISION CROP LOAD MANAGEMENT FOR APPLES
Sponsoring Institution
National Institute of Food and Agriculture
Project Status
EXTENDED
Funding Source
OTHER GRANTS
Reporting Frequency
Annual
Accession No.
1023632
Grant No.
2020-51181-32197
Project No.
NYG-632521
Proposal No.
2020-02633
Multistate No.
(N/A)
Program Code
SCRI
Project Start Date
Sep 1, 2020
Project End Date
Aug 31, 2025
Grant Year
2020
Project Director
Robinson, T. L.
Recipient Organization
N Y AGRICULTURAL EXPT STATION
(N/A)
GENEVA,NY 14456
Performing Department
Horticulture
Non Technical Summary
Controlling the final fruit number on an apple tree is one of the most economically critical management practices in apple growing. Optimizing fruit numbers within a narrow, economically optimum range is currently imprecisely done by pruning, chemical thinning and remedial hand thinning which is very expensive. Our previous work has shown that precisely controlling crop load has large economic benefits.We have previously developed ideas and tactics to precisely control crop load by calculating the optimum fruit number per tree, manually counting buds, flowers and fruits and by using various computer models we have developed (carbon balance model, fruit growth rate model and the pollen tube growth model) to help growers achieve the optimum number of fruits per tree; however, the process is tedious and time consuming.Through this project we will further develop precision crop load management tools consisting of computer models, machine vision, robotics and decision support tools to which will allow apple growers to accurately calculate a target fruit number for each tree and then quickly count flower buds and later fruitlets using machine vision and geo-referenced maps to guide the severity of pruning and later guide bloom and post-bloom chemical thinning, and lastly to guide human workers when hand thinning to maximize crop value.This project directly addresses SCRI priority area number 3 "to improve production efficiency, handling and processing, productivity, and profitability over the long term"using a systems approach of plant physiology, crop management, computer vision, robotics, economics, sociology and extension.
Animal Health Component
0%
Research Effort Categories
Basic
5%
Applied
60%
Developmental
35%
Classification

Knowledge Area (KA)Subject of Investigation (SOI)Field of Science (FOS)Percent
2051110102040%
2051110202050%
2051110303010%
Knowledge Area
205 - Plant Management Systems;

Subject Of Investigation
1110 - Apple;

Field Of Science
2020 - Engineering; 1020 - Physiology; 3030 - Information and communication;
Goals / Objectives
It is the long-term goal of this project to provide applegrowers with easy to use rules, on-line computer models, computer vision, and automated vehicles to adjust fruit number to an optimum target to maximize crop value and profitability.Project Objectives:1.Develop and disseminate user-friendly computer-based models and comprehensive crop load management strategies to achieve optimal crop load and maximize crop value.2.Develop, demonstrate, and deploy machine vision and robotic tools to accurately count reproductive structures during dormancy, bud break, bloom, and at the fruitlet stage and geo-reference the information for each tree in the orchard.3.Develop autonomous vehicles and end effectors designed for crop load management that can precisely measure and adjust crop load during dormancy, bud break, bloom, and at the fruitlet stage.4.Evaluate the economic and sociological impacts of adopting precision crop load management for sustainable apple production.5.Initiate rapid and effective outreach efforts to increase the adoption of precision crop load management for sustainable apple production.
Project Methods
1) We will perform replicated crop management field trials with the five most important varieties over four seasons in both University apple research orchards and grower orchards in NY, MI, WA, VA, PA, NC and MA. The field research data will be evaluated both horticulturally and economically to define the optimum target number of apples at each of the locations and with important high-value apple varieties. The field trials willdetermine the economic optimum crop load for apple regions. We will also conduct field analysesof imaging for bud quality and removal by pruning or thinning. Field studies will image,mapand predictearly fruitlet drop to precisely manage thinning. Other field studies will study the fruit set mechanism in 'WA38'. In Washington we will study the. idea of managing crop load.by a bee exclusion system for Fuji and WA38.2) We will perform replicated field research and laboratory studies to improve the pollen tube growth model, the carbohydrate model and the fruit growth rate model and extend this information to apple growers. We willdevelop a universal pollen tube growth model. We will. refine the leaf area development portionof the carbohydrate balance model3) We will conduct engineering, fabrication, testing, and then further refinement in iterative fashion of computer vision systems for counting reproductive structures of a variety of apple trees in commercially relevant planting systems and at various times of the season. We will also develop a cloud-based information system for data processing, storage, and communication to then guide human worker actions. This will include algorithm development and then asensing platform. for fruitlet and branch detection. We will then develop a data management system to process. the information from. the sensors. We will also develop anA\augmented HMI (human machine interface) for human workers. We. will then conduct field trials of prototype computer vision systems in grower orchards in the eastern and western USA.4) Wewill conduct engineering, fabrication, testing and further refinement in iterative fashion of autonomous vehicles for collecting geo-referenced information on flower and fruit number on each apple tree in an orchard with varying apple tree shapes and at various times of the season.5) Wewill conduct engineering, fabrication, testing and further refinement in iterative fashion of end effectors to prune or thin flowers or fruitlets of apple trees in a precise manner with varying apple tree shapes and at various times of the season.6) We will conduct economic studies of the impact of precisely controlling crop load and the economic impact of adopting manual methods or computer vision and robotic methods of precisely controlling crop load. This will include. the development of partial crop budget tools to assess economic impacts of adopting precision crop load management. We will studyapple grower behavior to identify effective ways to increase adoption of precision load management strategies. We will study apple grower stress related to load management strategies.7) We willconduct a robust outreach effort to help apple growers nationwide understand the economic impact and need for precisely controlling crop load and then help them adopt either manual or computer vision/robotic methods of precisely controlling crop load. An important part of our outreach effort will be to form and manage an Advisory Group (AG) of apple orchard owners, consultants, and relevant orchard personnel from each major apple growing region (Northwest, Midwest, and East). We will communicate annually via meetings to a national audience of fruit growers, extension educators and nurserymen via the annual conference of the IFTA held in February each year. The extension team will lead grower field days in their respective apple growing regions that will demonstrate the results of this Project to a broader audience. The extension team will conduct or coordinate applied research/demonstration trials in coordination with Project research PI's. The extension team will develop Extension durable products/deliverables as the project gets going and produces results. Technology transfer of Project outputs/deliverables will be a high priority. Products/deliverables may include but not be limited to publications, website, video products, social media, webinars, spreadsheets/apps. Information derived from project results will published in fruit grower magazines such as the Fruit Quarterly magazine which is sent to all NY and MI apple growers and is available on-line to all growers and trade publications including the GoodFruit Grower, the Fruit Grower News and the American Fruit Grower. The entire project team will publish our results in scientific journals to extend information to the worldwide scientific community. Each co-project director plans 3 scientific publications from this project.?

Progress 09/01/22 to 08/31/23

Outputs
Target Audience:All commercial apple growers in the USA, agricultural scientists, agricultural engineers, agricultural technology companies worldwide are the primary audience. Changes/Problems:Frost/freeze damage in 2023 impacted in horticultural research activities in NC, as one of two cultivars used ('Gala') in coordinated experiments had significant damage. What opportunities for training and professional development has the project provided?Undergraduate students MI Austin Chase, John Loch, Alex Deahl, and Reilly Ford. Students learned skills in experimental design, measuring plant organ growth and phenotyping buds, fruitlets, and leaves. Students were trained to operate lab instrumentation and record, collate, and enter data NC Caroline Pace and Elizabeth Pace, NC State University. Aided in the execution of a series of field trials, including an independent trial integrating computer vison to inform hand-thinning decisions. NY Thomas Acri, Cornell an undergraduate majoring in Plant Science assisted in tree sampling and data collection. Jules Hart- Cornell University Assisted post-doctoral scientist with field trials, sample prep, and microscopy Graduate Students. MI Laura Hillmann (Ph.D. candidate). Laura has made significant progress in all areas of the project. She has been trained in Dr. Tom Sharkey's lab (MSU) on sample preparation, extraction techniques and measurement of nonstructural carbohydrates in fruit tissues. Presentations are provided under 'Products' NY Mario Maranda Sazo, a Ph.D. student was working on the whole tree destructive sampling project NC James Larson, NC State University. PhD Student. Led and coordinated the 2nd year of a research project to evaluate a portable Vis/NIR spectrometer to predict fruit abscission after a chemical thinner application and compared this with other methods to predict fruit abscission. Mr. Larson also supported related coordinated research evaluating a separate novel model to predict fruit abscission potential (Einhorn) and efforts to optimize bud density and crop load of Honeycrisp. He presented original research from this project at two scientific conferences and three extension meetings. PA Rashmi Sahu: Oral presentation at 2022 ASABE International Annual Meeting. Poster presentation at 2022 NABEC Meeting. Oral presentation at 2022 Penn State FREC grower field day Wenan Yuan: Oral presentation at 65th International Fruit Tree Association Annual Conference Technical MI Denise Ruwersma and Gail Byler contributed to all crop load experiments to include initial selection of trees, measurements of trunks and bud numbers, pruning and hand thinning to target bud and crop levels, leaf area and fruit growth measures, harvests and fruit quality analyses Michael Brown- Assisted post-doctoral scientist with field trial execution VA Kenneth Savia: Field Research Specialist at the Alson H Smith Jr. Agriculture Research and Extension Center, Winchester, VA. Kenny was trained to collect and tabulate field data for fruit set (%), and fruit quality parameters. NY Spenser Howden conducted several plant growth regulator trials and collected data for all of the trials. Kaspar Kuehn, a technician, managed the trees and assisted in tree destructive sampling WA Zachary Chapman is an Agricultural Research Technologist 2 hired in March 2021 on this project. Jill Dinius is the temporary research field assistant helping across different projects, and from November 2020 to the present, she was actively involved in the field data collection for these experiments. NC Chris Clavet. Tatiana Zuber, Kayla Martineau; NC State University The NCSU technical support team conducted the following: experiment set-up, applied treatments, and collected, organized, and analyzed data associated with Objective 1. Clavet presented research associated with this project at a scientific meeting. Postdoctoral and Professional MI Mokhles Elsysy. has directed and supported all undergraduate activities related to Objective 1. Dr. Elsysy will statistically analyze data and aid in the development of the prediction model. NY Luis Gonzalez conduted analyzed and published all the experiments done at Geneva NY and Brent Arnoldussen led the development of the new universal pollen tube growth model. VA Md Tabibul Islam: Postdoctoral Research Associate at the Alson H Smith Jr. Agriculture Research and Extension Center, Winchester, VA. Tabibul helped design the field studies, flag trees, and analyze data. WAJebu Md Mia started a post-doc position on November 15, 2021, and he is still part of the project. Sara Serra is a research associate professor actively involved in all the objectives coordinating technical staff and collecting and analyzing data. Ryan Sheick is a research assistant actively involved in data collection for all the objectives. How have the results been disseminated to communities of interest?T. Einhorn presented data to stakeholders in several meetings during 2022. Dr. Elsysy and Ms. Hillmann each presented results at academic symposia Presentations(academic) Northeast Plant Growth Regulator Working Group, American Society of Horticultural Science, Presentations (Extension) New York fruit team webinar, Lake Ontario fruit region preharvest tour Extension Contacts 2022 Weekly breakfast meetings. Weekly in-person meetings with virtual zoom option. April through June. Seasonal guidance and updates given on thinning including PACMAN project. Approx. 100 weekly participants, 12 meetings. 2022 Weekly seasonal email updates. Updates sent weekly April through June. Guidance provided on precision thinning, models, and other tools included PACMAN project. Approx. 350 email subscribers, 20 updates. In PA,Eight presentations related to this project were delivered to stakeholders (members of the apple industry) in the southeast US and four presentations at scientific conferences. The results of the project were disseminated to grower communities, including International Fruit Tree Association 65th Annual Conference and Tours (presentation and demonstration); Penn State FREC grower field day (presentations); and 2022 Mid-Atlantic Fruit and Vegetable Convention (presentations and posters). PIs and graduate students attended the 2022 ASABE International Annual Meeting and 2022 Northeast Agricultural and Biological Engineering Conference (NABEC) meeting to present the research results from this project to the academic society. One journal paper was published from the results of this project. In NY, we started a collaborative effort between the SCRI project and interested Ag-tech companies from around the world. CCE-LOF carried out one pruning severity study on Honeycrisp at Orchard Dale Fruit Farm, and validated two cell phone camera technologies to count and measure fruitlet and trunk diameters. We also worked with several companies who are developing rovers or drones to count fruitlets and fruit (Moog from Buffalo, NY; Vivid from Ontario, Canada; Green Atlas from Australia and Outfield from England). CCE-LOF conducted a very successful fruit summer tour in Orleans County where digital technologies from six companies were featured and the Cornell Data Initiative was launched. There were conversations with AgerPix from Spain and Munckof/Aurea imaging from the Netherlands for applied research in 2023. In the fall, we conducted Farm Vision yield estimations of Fuji and Evercrisp at two farms in WNY. The NY group published and extension article "Scientists, extension educators, and Ag-tech innovators are working together to fine-tune and validate the adoption of digital technologies for precision crop load management" (T. Robinson, Y. Jiang, L. Gonzalez, M. Miranda Sazo, and C. Kahlke), pages 1-2, Vol. 22, Issue 14, Sept. 12, 2022. A collaborative effort between NY, MI, WA, and PA presented a summer webinar series on Orchard efficiency: Labor and Technology. The meetings showcased growers and other specialists leading the development of several AG-technologies for the U.S. apple industry. What do you plan to do during the next reporting period to accomplish the goals?NY We will continue the evaluation of commercially available computer vision systems to guide pruning, chemical thinning and hand thinning for more precision in crop load management. We will publish results in scientific journals in in grower publications. We will conduct a field day in the summer to present the concepts of PACMAN MI Complete all NSC analyses. Complete microscopic analyses of vascular tissues of pedicels. Complete histological analyses of ovaries. Complete modeling of year 2 fruit set predictions from four sites. Make modifications to improve all protocols and implement/execute 2022 experiments We will continue to work with and add to the existing datasets. Additionally, we will be conducting field trials to validate the model with pollen tube data and actual thinning trials in 2023. VA We will repeat the identical blossom studies conducted on 'Gala', and 'Red Delicious' in 2022, with the following changes: a) Examine different rates of LSS and oil; b) investigate other potential thinning materials, e.g. Regalia and potassium bicarbonate. NC Coordinated physiological trials related to Objective 1 will be continued. Efforts will be expanded to evaluate a universal pollen tube growth model. Research with a Vis/NIR spectrometer to predict fruit abscission will be continued and expanded to include other commercially relevant cultivars and thinning timings. A commercially available computer vision system will continue to be evaluated to inform hand-thinning decisions. Local/regional extension presentations are planned for the next reporting period along with reporting to scientific audiences. WA The goal of the next reporting period is to finish all the fruit analysis left and work on data collected in 2022. Analyzing these data will probably help improve protocols for next year PAIn the next year, we will work on 1) continue to improve the machine vision system for flower buds and green fruit detection with higher accuracy; 2) develop a flower bud thinning mechanism and then conduct field tests to investigate the potential of developing a robotic bud extinction system for early crop load adjustment; 3) integrate the machine vision system and green fruit thinning mechanism to develop an automated robotic green fruit thinning system. ?

Impacts
What was accomplished under these goals? Objective 1 In 2023 we conducted four multi-location (NY, MI, NC, WA) uniform experiments on (a) flower bud density, (b) crop load, (c) spur quality and (d) fruitlet abscission. Regression analysis showed different slopes and intercepts at each location indicating that at a given crop load, larger fruits are produced in some climates than in other climates. These results formed the basis of an economic analysis of crop load and crop value that was conducted by the economics team under objective 4. The project on fruit abscission led by Einhorn (MI) developed a user-friendly alternative to the fruit growth rate model which generated accurate predictions of final fruit set, comparable to those of the current model for the second consecutive year in four US apple regions including grower collaborator sites. This development should significantly improve stakeholder adoption of the model. We conducted aquantitative assessment of variation in bud size from whole canopies of Gala and Honeycrisp trees demonstrated a 2 to 3 fold range in bud size in the orchard. Measurements to support relationships between ovary size of king and lateral flowers, vegetative tissues and final fruit size are in progress. Our proposed user-friendly fruit growth rate model generated accurate predictions, In NC, Kon led a project conducted by cooperators in NY, MI and NC in which we assessed a Vis/NIR spectrometer to predict fruitlet abscission potential. In WA, studies on the fruit set mechanism in 'WA38' were conducted by Musacchi. These studies support the theory of natural shedding of this variety is driven by high competition between fruitlets. In NY, Peck successfully developed a universal Pollen Tube Growth Model (PTGM) and conducted evaluations of the accuracy of this universal model.Machine learning techniques were applied to this dataset to create new PTGMs for each of the original cultivars ['Cripps Pink' ('Pink Lady®'), 'Fuji', 'Gala', 'Golden Delicious', 'Granny Smith', 'Honeycrisp', and 'Red Delicious']. Additionally, the dataset was used to create two machine-learning-delivered "universal models" that can be used for any cultivar and continually expand as more data becomes available. Lastly, by adding field data, we have developed a rapid protocol to develop robust cultivar-specific PTGMs for new cultivars. In VA, Sherif conducted field trials on mature trees of 'Red Delicious', and 'Gala' to 1) evaluate how different rates of lime sulfur (calcium polysulfide) and how the addition of mineral oil (JMS stylet oil) affect the efficacy of apple blossom thinning sprays and, consequently, fruit set and crop load; and 2) evaluate the suitability and efficacy of potential blossom thinners. Concerning suitability, crop safety was assessed to establish if the chemical flower thinning agents could cause unintended damage to the fruit and/or foliage. In addition, the efficacy of prospective blossom-thinning agents was examined to determine the materials' effectiveness in lowering fruit set. All the blossom thinning treatments were applied according to the outputs of the pollen tube growth model (PTGM). In NY, Cheng worked to improve the MaluSim apple thinning carbohydrate model by collectingtree growth and dry matter data on 'Honeycrisp' trees on four rootstocks during dormancy and at fruit harvest over five years as well as at six developmental stages in the fifth year to run the MaluSim model to determine the fit of the model for 'Honeycrisp' Objective 2 The PSU group developed an on-tree measurement system of apple fruit size using a stereo vision system and neural network models including Faster R-CNN and Mask R-CNN. The system had a mean absolute error ranging from 1.1 to 4.2mm. A fruitlet growth tracking algorithm was developed and able to correctly match 74% of all detected fruits during the mature fruit growth stage in September and October. A flower bud mapping algorithm was developed to map classifier detection results into dense growth stage maps utilizing RGB image geoinformation. YOLOv4 classifiers for six apple flower bud growth stages in various network sizes were trained based on 5040 RGB images, and the best model achieved a 71.57% mAP for a test dataset consisted of 360 images. Using YOLOv4 as a representation of state-of-the-art object detectors, the sensitivity of YOLOv4's for apple blossom detection was quantified against artificial image distortions including white noise, motion blur, hue shift, saturation change, and intensity change.The importance of various training dataset attributes was examined based on model classification accuracies of flower detection, including dataset size, label quality, negative sample presence, image sequence, and image distortion levels. A method for in-field tracking of fruit diameters using a stereo vision system was developed and tested.A deep learning-based flower bud detection at tree level was developed to identify flower buds at silver tip, green tip, and tight cluster. The results showed around 70% overall mAP, while if combine the silver tip and green tip, the precision of bud detection can increase to over 90%. In NY, Jiang focused on tree data collection using a high-resolution laser scanner. He found the optimal configuration for particular apple orchards and a general protocol on how to optimize the data acquisition in different orchards. They also developed a QGIS plugin that can show simulated tree information in a geographical information system. Objective 3 The PSU group have developed branch cutting mechanisms to remove branches from apple trees. A three degree-of freedom (DoF) branch cutting mechanism was developed and tested in an apple orchard. The results indicated that the mechanism could prune branches up to 1 inch at different orientations. A fruit pulling removal device was developed. No significant difference was found on the required fruit removal forces for the fruits with diameter ranging from 10 -30 mm. Objective 4 In NY, Gomez's team conducted an economic evaluation of the horticultural data produced under objective 1 to determine in the four climates (NY, NC, MI and WA) the optimum flower bud load and optimum fruit load to maximize crop economic value. They found that the optimum crop load for Gala was ~9 fruits/cm2 TCA and for Honeycrisp ~8 fruits/cm2 TCA. They also found that the WA climate gave larger fruit size with Gala than the NY, MI or NC climate but that with Honeycrisp the NY climate gave similar large fruit size as the WA climate while the NC climate gave smaller fruit size. The economics team also began developing partial fruit production budgets to be used in the future analysis (2023 and 2024) of the cost effectiveness of the technology developed in this project for precisely controlling crop load. Objective 5 In 2023 we met with our Advisory Committee (AC) consisting of 21 grower, nursery, and industry representatives in March in where project PI's presented their research objectives and plan of work followed by an open discussion where committee members gave advice and direction to the project leaders The PACMAN.extension.org, website from this project is an easily remembered acronym for Precision Apple Crop load MANagement. The website was used to highlight our activities and results outside of the scientific publication audience. The website target audience is grower, industry, University and Extension practitioners. Several field research & demonstration trials were established featuring precision crop load management protocols in NY, MI and VA. In addition, field days and grower/industry/research visits were conducted by project personnel in NY, MI, VA and WA featuring precision crop load management technologies.?

Publications

  • Type: Journal Articles Status: Published Year Published: 2023 Citation: Hussain, M., He, L., Schupp, J., Lyons, D. and Heinemann, P., 2023. Green fruit segmentation and orientation estimation for robotic green fruit thinning of apples. Computers and Electronics in Agriculture, 207, p.107734.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Sahu, R. and He, L., 2023. Real-Time Bud Detection Using Yolov4 for Automatic Apple Flower Bud Thinning. In 2023 ASABE Annual International Meeting, paper no. 2300423. American Society of Agricultural and Biological Engineers. St. Joseph, Michigan.
  • Type: Journal Articles Status: Published Year Published: 2023 Citation: Larson, J.E. and T.M. Kon. 2023. Apple fruitlet abscission prediction. I. Development and evaluation of reflectance spectroscopy models. HortScience. 58:1085-1092. DOI: https://doi.org/10.21273/HORTSCI17244-23
  • Type: Journal Articles Status: Published Year Published: 2023 Citation: Larson, J.E., P. Perkins-Veazie, and T.M. Kon. 2023. Apple fruitlet abscission prediction. II. Characteristics of fruitlets predicted to persist or abscise by reflectance spectroscopy models. HortScience. 58:1095-1103. DOI: https://doi.org/10.21273/HORTSCI17245-23
  • Type: Journal Articles Status: Published Year Published: 2023 Citation: Gonzalez, L., A. Wallis, J. Clements, M. Miranda Sazo, C. Kahlke, T.M. Kon, and T.L. Robinson. 2023. Evaluation of computer vision systems and applications to estimate trunk cross-sectional area, flower cluster number, thinning efficacy and yield of apple. Horticulturae. 9(8): 880. DOI: https://doi.org/10.3390/horticulturae9080880
  • Type: Journal Articles Status: Published Year Published: 2023 Citation: Larson, J.E., P. Perkins-Veazie, G. Ma, and T.M. Kon. 2023. Quantification and prediction with near infrared spectroscopy of carbohydrates throughout apple fruit development. Horticulturae. 9(2):279. DOI: https://doi.org/10.3390/horticulturae9020279
  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Robinson, T.L., L. Gonzalez, Y. Ziang, G. Peck, B. Arnoldson, M. Gomez, M. Guerra, M.M. Sazo, C. Kahlke, T. Einhorn, A. Wallis, S. Musacchi, S. Serra, K. Lewis, T. Schmidt, P. Heinemann, L. He, T. Kon, S. Sherif, J. Clements, and C. Layer. 2023 Studies in precision crop load management of apple. Acta Hort. 1366:219-225. DOI: 10.17660/ActaHortic.2023.1366.25
  • Type: Journal Articles Status: Published Year Published: 2023 Citation: Wallis, A., J. Clements, M.M. Sazo, C. Kahlke, K. Lewis, T. Kon, L. Gonzalez, Y. Jiang, and T. Robinson. 2023. Digital technologies for precision apple crop load management (PACMAN) part I: Experiences with tools for predicting fruit set based on the fruit growth rate model. Fruit Quarterly 31(1):8-13.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Miranda Sazo, M., Robinson, T.L, and L. Cheng. 2023. Effects of Four Rootstocks on Dry Matter Accumulation and Partitioning of Honeycrisp Trees. Annual Conference of the American Society for Horticultural Sciences. Orlando, Florida. August 4, 2023. https://ashs.confex.com/ashs/2023/meetingapp.cgi/Session/12291
  • Type: Journal Articles Status: Published Year Published: 2023 Citation: Jiang, Y., Wallis, A., Clements, J., Miranda Sazo, M., Kahlke, C., Lewis, K., Basedow, M., and T.L. Robinson. 2023. Digital technologies for precision apple crop load management (PACMAN) Part II: An overview of digital technologies currently available for PACMAN. Fruit Quarterly. Vol. 31(2): 19-23.
  • Type: Journal Articles Status: Published Year Published: 2023 Citation: Gonzalez L., Francescatto, P., Carra, B., Robinson, T. (2023). Metamitron efficacy at different rates, timing and weather factors for apple fruitlet thinning in New York State. Horticulturae 9(11):1179 /10.3390/horticulturae9111179
  • Type: Journal Articles Status: Published Year Published: 2023 Citation: Robinson, T.L., L. Gonzalez, Y. Jiang, M. Miranda Sazo, C. Kahlke, 2024. Evaluating the variability in fruit buds and fruitlets along a row using digital technology. Fruit Quarterly 31(4): 23-28.
  • Type: Journal Articles Status: Published Year Published: 2023 Citation: Sapkota, S., Salem, M., Jahed, K.R., Artlip, T.S., Sherif, S.M. (2023). From endodormancy to ecodormancy: the transcriptional landscape of apple floral buds. Frontiers in Plant Science, 13. doi.org/10.3389/fpls.2023.1194244
  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Gonzalez, L., P. Francescatto, B. Carra, E. Casagrande and T.L. Robinson. 2023. Influence of various weather factors on the efficacy of Metamitron for apple fruitlet thinning in New York State. Abstracts of the ASHS Annual Meeting in Orlando Florida. Aug. 2, 2023 (60 people) Miranda Sazo, M., T.L. Robinson and L. Cheng. 2023. Effects of Four Rootstocks on Dry Matter Accumulation and Partitioning of 'Honeycrisp' Trees. Abstracts of the ASHS Annual Meeting in Orlando Florida. Aug. 3, 2023 (50 people) Serra S., Md Jebu Mia and Musacchi S. Effect of Pollinators Exclusion System on WA 38 Fruit Set In Washington State. Poster ASHS Annual Conference of the American-Society-for-Horticultural-Science (ASHS) (Orlando, FL) on August 4, 2023.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Larson, J., P. Perkins-Veazie, and T. Kon. 2022. Fruitlet chlorophyll content dictates prediction of fruitlet abscission following a chemical thinner application. 98th Cumberland-Shenandoah Fruit Workers Conference, Winchester, VA Kon, T., J. Larson, A. Vogel, and C. Clavet. 2022. Informing apple hand-thinning with computer vision. 98th Cumberland-Shenandoah Fruit Workers Conference, Winchester, VA
  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Gonzalez L., Miranda M., and Robinson, T.L. 2023. Programs of fruit thinning in different cultivar in 2021. NEPGR 2023 Mar 8, 2023 (35 people) Gonzalez L., Miranda M., and Robinson, T.L. 2023. Bloom Thinning Gala and Honeycrisp. NEPGR 2023 Mar 8, 2023 (35 people) Kon, T. and C. Clavet. 2023. Effects and interactions of Accede as part of a multi-step thinning program. Northeast Plant Growth Regulator Working Group, Wilkes Barre, PA Kon, T. and J. Larson. 2023. Update on predicting fruitlet abscission with a NIR/Vis spectrometer. Northeast Plant Growth Regulator Working Group, Wilkes Barre, PA


Progress 09/01/21 to 08/31/22

Outputs
Target Audience:All commercial apple growers in the USA, agricultural scientists, agricultural engineers, agricultural technology companies worldwide are the primary audience. Changes/Problems:The severe outbreak of fire blight at Cornell Orchards in 2022 disrupted some of our work on shoot leaf area development in relation to shoot growth for improving the Malusim thinning model What opportunities for training and professional development has the project provided?Undergraduate Students In. MI, six undergraduate students worked in Einhorn's lab on this projects. They learned skills in experimental design, measuring plant organ growth and phenotyping buds, fruitlets, and leaves. Students were trained to operate lab instrumentation and record, collate, and enter data. Graduate Students In NY, Mario Maranda Sazo, (Ph.D. candidate) worked with Cheng on the whole tree destructive sampling project. In NY, Tian Qiu, (first year Ph.D. student in Electrical and Computer Engineering) worked with Jiang starting in Jan 2021 and has focused on the use of full-perspective, high resolution 3D point cloud for apple tree characterization. He developed an initial data processing pipeline and investigated several key algorithms for tree segmentation, trunk detection, and trunk diameter measurement in the past year. In NY, Hehong Li, (first year Master of Engineering student in Electrical and Computer Engineering) worked with Jian since September 2021 and has explored the use of an augmented reality (AR) device for improved human machine interaction. She developed an application on the device for real-time data display. In MI, Laura Hillmann (Ph.D. candidate) worked with Einhorn in all areas of the project. She has been trained in Dr. Tom Sharkey's lab (MSU) on sample preparation, extraction techniques and measurement of nonstructural carbohydrates in fruit tissues. Presentations are provided under the section entitled 'Other Products'. In NC, James Larson, (PhD Candidate) worked with Kon on a research project to evaluate a portable Vis/NIR spectrometer to predict fruit abscission after a chemical thinner application and compared this with other methods to predict fruit abscission. He presented research on predicting fruit carbohydrate content with near infrared spectroscopy at the annual conference of the American Society of Horticultural Science and at an extension meeting in NC. In VA, Rashmi Sahu is the graduate student working on this project since August 2021. Graduate students Azlan Zahid,Magni Hussain, and Omeed Mirbod also have been contributing to the project with some of their work during 2021, but they were not paid by the project. Technical In NY, Spenser Howden is a research support specialist with Robinson who worked on the field experiments on this project. Kathy Campo a technician did data entry and fruit quality analysis. In NY, Kaspar Kuehn is a technician with Cheng who managed the trees and assisted in tree destructive sampling. Graham O'Donnell, a recent Cornell graduate who assisted in tree destructive sampling and data collection. In MI, Denise Ruwersma and Gail Byler are technicians with Einhorn who contributed to all crop load experiments to include initial selection of trees, measurements of trunks and bud numbers, pruning and hand thinning to target crop levels, leaf area and fruit growth measures (for both crop load experiments and fruit set prediction experiments), harvests and fruit quality analyses In NC, Chris Clavet is a research specialist; Tatiana Zuber is a research technician; Diana Vercillo is a research technician; and Cassie Detrio-Darby is a research technician who worked with Kon on this project. In WA, Zachary Chapman is an Agricultural Research Technologist worked with Musacchi on this project. Jill Dinius is a temporary research field assistant who helped in the field data collection for these experiments. Ryan Sheick is a Research assistant who was actively involved in data collection in all the objectives. In VA, Kenneth Savia is a field research specialist who worked with Sherif and collected and tabulated field data for fruit set, crop load and fruit cracking. Postdoctoral and Professional In NY, Luis Gonzalez is a post doc with Robinson and is responsible for this project. In MI, Mokhles Elsysy is a post doc with Einhorn and has directed and supported all undergraduate students who worked on this project, organized all lab activities related to harvest and evaluation of fruit quality and will statistically analyze data and aid in the development of the prediction model. In WA, Sara Serra is a research associate professor who works with Musacchi and is actively involved in all the objectives coordinating technical staff and collecting and analyzing data. In WA, Tory Schmidt is a Horticultural Associate working at the WTFRC who works with Musacchi and is actively involved in objectives 1A1 and 1A2, coordinating technical staff and collecting data. In VA, Md Tabibul Islam is a Postdoctoral Research Associate who works with Sherif and has assisted in designing the field experiments, flagging trees and analyzing data. Tabibul has received training on different laboratory analyses for fruit physical and chemical attributes (e.g. fruit weight, size, firmness, color, starch index, prix and ethylene content). How have the results been disseminated to communities of interest?The PACMAN project was introduced to the national fruit growers conference (IFTA) in February by Robinson and through scientific articles. In addition, our project website pacman.extension.org, had seven new posts in 2022, 13 Posts total. 1,565 all-time page views. In addition, individual project leaders in each state have communicated results of the project in state and regional meetings and webinars. We presented results from this project at the 2022 International Horticultural Congress (IHC) in Angers, France, August 15-19, 2022 "Advances in precision crop load management of apple - A USA national project". In NY, we started a collaborative effort between the SCRI project and interested Ag-tech companies from around the world. CCE-LOF carried out one pruning severity study on Honeycrisp at Orchard Dale Fruit Farm and validated two cell phone camera technologies to count and measure fruitlet and trunk diameters. We also worked with several companies who are developing rovers or drones to count fruitlets and fruit. CCE-LOF conducted a very successful fruit summer tour in Orleans County where digital technologies from six companies were featured and the Cornell Data Initiative was launched. A collaborative effort between NY, MI, WA, and PA presented a summer webinar series on Orchard efficiency: Labor and Technology. The meetings showcased growers and other specialists leading the development of several AG-technologies for the U.S. apple industry. In NY, Robinson presented concepts of precision crop load management at the winter fruit schools and also presented 12 webinars on crop load management in May and June. In MI, T. Einhorn presented data to stakeholders in several meetings during 2022. Dr. Elsysy and Ms. Hillmann each presented results at academic symposia Presentations(academic) Northeast Plant Growth Regulator Working Group, American Society of Horticultural Science, Presentations (Extension) New York fruit team webinar, Lake Ontario fruit region preharvest tour In NC, Kon made six presentations related to this project to stakeholders in NC and one scientific presentation at a professional conference. Stakeholders in NC, SC, and GA have been introduced to the project, provided feedback on project direction, and have had the opportunity to tour research plots via extension programming. Six presentations related to this project were delivered to stakeholders in the southeast US and one scientific presentation at a professional conference. In VA, Sherif posted results from this project to the Virginia Tech Tree Fruit Extension and Outreach Facebook Page. https://www.facebook.com/VtechPomology. This page is managed by Dr. S. Sherif and administered by Virginia Tech Extension Specialists and Virginia Cooperative Extension Agents to provide information about tree fruit production. 1147 people follow this page. Sherif, S, also posted results from this project to his blog. https://blogs.ext.vt.edu/tree-fruithorticulture/. In PA, eight presentations related to this project were delivered to stakeholders (members of the apple industry) in the southeast US and four presentations at scientific conferences. The results of the project were disseminated to grower communities, including International Fruit Tree Association 65th Annual Conference and Tours (presentation and demonstration); Penn State FREC grower field day (presentations); and 2022 Mid-Atlantic Fruit and Vegetable Convention (presentations and posters). PIs and graduate students attended the 2022 ASABE International Annual Meeting and 2022 Northeast Agricultural and Biological Engineering Conference (NABEC) meeting to present the research results from this project to the academic society. One journal paper was published from the results of this project. Field research & demonstration trials were established featuring precision crop load management protocols. In NY, MI and VA, two bud-load pruning trials were set up in Western New York, on Honeycrisp and Gala. Additional demonstration projects included: (1) an on-farm trial to fine tune the fruit growth rate model, (2) the use of ATS guided by the pollen tube growth model, (3) an evaluation of metamitron, 6-BA, and ACC, and (4) We collaborated with an industry vendors of precision crop load management tools to measure fruit growth rates versus the traditional Cornell hand method. In MI we initiated field validation trial of Farm Vision crop load management technologies.. In NY, three presentations on precision crop load management were mad Cornell LOF summer tour organized in Wayne County. The results of the AFS method of measuring fruit growth rates was presented. In MI, an annual summer field day/tour attended by 250 featured precision technologies under development for tree fruit production. In VA, we extended the findings of our research on precision crop load management to Virginia apple growers and stakeholders through fruit schools, orchard meetings, the Tree Fruit Horticulture Updates blog website (https://blogs.ext.vt.edu/tree-fruithorticulture) and the Virginia Tech Tree Fruit Extension and Outreach Facebook page (https://www.facebook.com/ssherif.VT). In WA, Project PI's and Consultant met in Summer 2021 and visited Project trial sites and commercial orchards to discuss commercial crop load practices. In VA, we conducted a state-wide survey to assess the impacts of our extension program on grower's knowledge of different horticulture-related topics, including apple blossom and fruit thinning. What do you plan to do during the next reporting period to accomplish the goals?We will repeat the multi-location (NY, MI, NC and WA) detailed physiological studies to evaluate the effect of climate on crop load management results. The economics team in NY will calculate the economic optimum crop load for each of the four locations in this study. In NY we will determine the degree to which the current MaluSim model fits the dry matter data obtained from our destructive sampling of Honeycrisp on four rootstocks, and then collect shoot growth, leaf area and photosynthesis data to improve the sub-models where lack of fit is observed. In NY, Peck continue developing a universal apple pollen tube growth rate model using advanced data analyses, such as machine/deep learning. In MI, Einhorn will continue the other physiological studies we have started and will complete all NSC analyses; complete microscopic analyses of vascular tissues of pedicels; complete histological analyses of ovaries, complete modeling of year 2 fruit set predictions from four sites. Make modifications to improve all protocols for 2023. In NC, Koon will continue and expand his research with a Vis/NIR spectrometer to predict fruit abscission with the evaluation of other commercially relevant cultivars and thinning timings to determine if this technology continues to show merit in. Local/regional extension presentations are planned for the next reporting period along with reporting to scientific audiences. In WA, Musacchi will continue studies on WA38 cultivar to optimize thinning and pollination. They will also evaluate systems and sensing / algorithm efforts for fruitlet detection of 3 commercial companies. They will also provide ground truthing efforts. In NY, Jiang will evaluate the several commercial companies vision systems to detect flower buds, fruitlets and fruits near harvest. In PA, the engineering team will continue the development of precision pruning with robotic end effectors. They will give extension presentations at grower meetings (such as Mid-Atlantic Fruit and Vegetable Convention) and to the IFTA conference in Hershey PA to disseminate the research results from the project. They also plan to train graduate student and possibly undergraduate students on development of scientific skills through daily research activities, publications, and presentations. In VA, Sherif will examine the efficacy and safety of materials and rates for chemicals blossom thinning on two apple cultivars, 'Honeycrisp' and 'Fuji'. The thinning experiment with GA4+7" will be repeated to validate the results of the 2022 season and optimize the rate and application timing of GA4+7 products to reduce the incidence of stem-end cracking in 'Gala' apples. The extension team will organize demonstrations, field days and grower meetings to present results from this project.

Impacts
What was accomplished under these goals? Impact of this project. The horticultural results from this studies were evaluated in an economic study to define the optimum crop load in economic terms. The optimum flower bud load can then be achieved through more precise pruning and fruit thinning We have also worked in 2022 to improve three mathematical models which growers use to help achieve the optimum crop load. This year's work has led to improvements in all three models. We also made progress on developing an alternative simpler model to the fruit growth rate model. Through this project we are developing computer vision systems which can count the number of flower buds or fruits on each tree. This year we have made significant progress and have a worked with 3 commercial companies to count fruitlets per tree. Major activities and accomplishments. Objective 1 In 2022 we conducted four multi-location (NY, MI, NC, WA) uniform experiments on (a) flower bud density, (b) crop load, (c) spur quality and (d) fruitlet abscission. Regression analysis showed different slopes and intercepts at each location indicating that at a given crop load, larger fruits are produced in some climates than in other climates. These results formed the basis of an economic analysis of crop load and crop value that was conducted by the economics team under objective 4. The project on fruit abscission led by Einhorn (MI) developed a user-friendly alternative to the fruit growth rate model which generated accurate predictions of final fruit set, comparable to those of the current model. In NC, Kon led a project conducted by cooperators in NY, MI and NC in which we assessed a Vis/NIR spectrometer to predict fruitlet abscission potential. In WA, studies on the fruit set mechanism in 'WA38' were conducted by Musacchi. These studies support the theory of natural shedding of this variety is driven by high competition between fruitlets. In NY, Peck successfully developed a universal Pollen Tube Growth Model (PTGM) and conducted evaluations of the accuracy of this universal model In VA, Sherif conducted laboratory and greenhouse experiments to develop pollen tube growth models (PTGM) for two club apple cultivars. The raw data used for generating these models will be incorporated in a bigger dataset for developing a universal PTGM valid for all the commercial apple cultivars. In NY, Cheng worked to improve the MaluSim apple thinning carbohydrate model by collecting tree growth and dry matter data on 'Honeycrisp' grown on four different rootstocks. Objective 2 The PSU group developed an on-tree measurement system of apple fruit size using a stereo vision system and neural network models including Faster R-CNN and Mask R-CNN. The system had a mean absolute error ranging from 1.1 to 4.2mm. A fruitlet growth tracking algorithm was developed and able to correctly match 74% of all detected fruits during the mature fruit growth stage in September and October. In NY, Jiang focused on tree data collection using a high-resolution laser scanner. He found the optimal configuration for particular apple orchards and a general protocol on how to optimize the data acquisition in different orchards. They also developed a QGIS plugin that can show simulated tree information in a geographical information system. Objective 3 The PSU group have developed branch cutting mechanisms to remove branches from apple trees. A three degree-of freedom (DoF) branch cutting mechanism was developed and tested in an apple orchard. The results indicated that the mechanism could prune branches up to 1 inch at different orientations. A fruit pulling removal device was developed. No significant difference was found on the required fruit removal forces for the fruits with diameter ranging from 10 -30 mm. Objective 4 In NY, Gomez's team conducted an economic evaluation of the horticultural data produced under objective 1 to determine in the four climates (NY, NC, MI and WA) the optimum flower bud load and optimum fruit load to maximize crop economic value. They found that the optimum crop load for Gala was ~9 fruits/cm2 TCA and for Honeycrisp ~8 fruits/cm2 TCA. They also found that the WA climate gave larger fruit size with Gala than the NY, MI or NC climate but that with Honeycrisp the NY climate gave similar large fruit size as the WA climate while the NC climate gave smaller fruit size. The economics team also began developing partial fruit production budgets to be used in the future analysis (2023 and 2024) of the cost effectiveness of the technology developed in this project for precisely controlling crop load. Objective 5 In 2022 we met with our Advisory Committee (AC) consisting of 21 grower, nursery, and industry representatives in August in Geneva NY where project PI's presented their research objectives and plan of work followed by an open discussion where committee members gave advice and direction to the project leaders The PACMAN.extension.org, website from this project is an easily remembered acronym for Precision Apple Crop load MANagement. The website was used to highlight our activities and results outside of the scientific publication audience. The website target audience is grower, industry, University and Extension practitioners. Several field research & demonstration trials were established featuring precision crop load management protocols in NY, MI and VA. In addition, field days and grower/industry/research visits were conducted by project personnel in NY, MI, VA and WA featuring precision crop load management technologies.?

Publications

  • Type: Conference Papers and Presentations Status: Published Year Published: 2022 Citation: Arnoldussen, B., G. Peck. Analytics, Modeling, and Machine Learning of a Decade of Pollen Tube Growth Data in Seven Apple Cultivars. HortScience 57(9):S146 (abstr.).
  • Type: Conference Papers and Presentations Status: Published Year Published: 2022 Citation: Sazo, M.M, Robinson, T.L., Cheng, L. 2022. Nutrient uptake and accumulation of Honeycrisp trees as affected by four rootstocks from establishment to maturity. Annual Conference of American Society for Horticultural Science. August 1, 2022. https://ashs.confex.com/ashs/2022/meetingapp.cgi/Paper/38117
  • Type: Journal Articles Status: Accepted Year Published: 2022 Citation: Larson, J. and T. Kon. 2022. Vis/NIR Spectroscopy is a Promising Tool to Predict Fruit Set and Chemical Thinner Response. Acta Horticulturae.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2022 Citation: Clavet, C.D. and T.M. Kon. 2022. Comparison of Accede"-based multi-step thinning programs on Red Delicious apple. HortScience (Abstr.) HortScience 57(9):S92.
  • Type: Journal Articles Status: Published Year Published: 2022 Citation: Hussain, M., He, L., Schupp, J. and Heinemann, P. (2022). Green Fruit Removal Dynamics for Development of Robotic Green Fruit Thinning End-Effector. Journal of the ASABE, 65(4), pp.779-788.
  • Type: Journal Articles Status: Published Year Published: 2022 Citation: Hillmann L., L. Gonzalez Nieto, T. Kon, S. Musacchi, T. Robinson, S. Serra, and T. Einhorn, 2022. A Modified Apple Fruit Set Prediction Model to Guide Repeat Thinner Applications. Fruit Quarterly. Vol. 30(2): 4-6.
  • Type: Journal Articles Status: Published Year Published: 2022 Citation: Serra S., Roeder S., Sheick R., and Musacchi S., 2022. Pistil Biology of WA 38 Apple and Effect of Pollen Source on Pollen Tube Growth and Fruit Set. Agronomy 2022, 12(1), 123; https://doi.org/10.3390/agronomy12010123
  • Type: Journal Articles Status: Published Year Published: 2022 Citation: Serra, S., Sheick, R., Roeder, S. and Musacchi, S. (2022). WA 38 abscission and fruit development in an open pollination scenario. Acta Hortic. 1346, 129-138. DOI: 10.17660/ActaHortic.2022.1346.17. https://doi.org/10.17660/ActaHortic.2022.1346.17
  • Type: Journal Articles Status: Published Year Published: 2022 Citation: Carra, B., Francescatto, P., Kovaleski, A.P., Sander, G.F., Pasa, M.S., Racsko, J. and Robinson, T.L. 2022. Early flower bud development and plant growth regulators to improve return bloom of pears. Acta Hortic. 1342, 351-358 DOI: 10.17660/ActaHortic.2022.1342.50 https://doi.org/10.17660/ActaHortic.2022.1342.50
  • Type: Journal Articles Status: Published Year Published: 2022 Citation: Francescatto, P., Carra, B., Fontanella Sander, G. and Robinson, T.L. 2022. Ethylene evolution of flowers of different apple cultivars varies in timing and intensity. Acta Hortic. 1342, 23-30 DOI: 10.17660/ActaHortic.2022.1342.4 https://doi.org/10.17660/ActaHortic.2022.1342.4
  • Type: Journal Articles Status: Published Year Published: 2022 Citation: Gonzalez, L., Francescatto, P., Lordan, J. and Robinson, T.L. (2022). Rate and timing of metamitron affect thinning efficacy of Gala apple trees under American northeast conditions. Acta Hortic. 1344, 55-64 DOI: 10.17660/ActaHortic.2022.1344.9 https://doi.org/10.17660/ActaHortic.2022.1344.9
  • Type: Conference Papers and Presentations Status: Published Year Published: 2022 Citation: Hillmann, L., and Einhorn, T. Comparison between relative fruit growth rate and fruit size distribution for abscission and final fruit set prediction of Gala apples. American Society for Horticultural Science Annual Conference, Chicago, IL, 2-August 2022.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2022 Citation: Hillmann, L., M. Elsysy, T., L. Gonzalez Nieto, T. Kon, S. Musacchi, T. Robinson, S. Serra and Einhorn, T.C. A comparison between relative fruit growth rate and fruit size distribution models to predict apple fruitlet abscission. 31st International Horticultural Congress, Angers, France, 14-20, August 2022.
  • Type: Journal Articles Status: Published Year Published: 2022 Citation: Robinson, T.L., M. Miranda Sazo, C. Kahlke, and M. Wiltberger. 2022. A Cornell Vision for Modern Processing Orchards for Improved Production and Profitability in NY State. Fruit Quarterly 30:(1):4-9.
  • Type: Journal Articles Status: Published Year Published: 2022 Citation: Robinson T.L., L. Gonzalez, Y. Jiang, M. Gomez, M. Guerra, Mario Miranda Sazo, C. Kahlke, T. Einhorn, A. Wallis, S. Musacchi, S. Serra, K. Lewis, T. Kon, J. Clements, and C. Layer. 2022. Studies in Precision Crop Load Management of Apple. Fruit Quarterly 30:(4):4-7.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2022 Citation: Robinson T.L., L. Gonzalez, L. Cheng, Y. Jiang, G. Peck, B. Arnoldson, M. Gomez, M. Guerra, Mario Miranda Sazo, C. Kahlke, T. Einhorn, A. Wallis, S. Musacchi, S. Serra, K. Lewis, T. Schmidt, P. Heinemann, L. He, T. Kon, S. Sherif, J. Clements, and C. Layer. 2023. Studies in precision crop load management of apple. ISHS International Horticulture Congress, France Aug. 17, 2022
  • Type: Conference Papers and Presentations Status: Published Year Published: 2022 Citation: Sahu, R., He, L. 2022. Apple Bud Detection for Precision Branch Pruning Using Deep Learning Neural Networks. Presented at NABEC 2022, July 31-August 3, 2022, Edgewood, MD. (Poster)
  • Type: Conference Papers and Presentations Status: Published Year Published: 2022 Citation: Sahu, R., He, L. 2022. Early-stage apple bud detection using convolution neural network. Presented at ASABE 2022 Annual International Meeting, July 18-20, 2022, Houston, TX. He, L., Heinemann, P. 2022. Robotic crop load management for apples. Presented at FIRA USA, October 18-20, 2022, Fresno, CA.


Progress 09/01/20 to 08/31/21

Outputs
Target Audience:All commercial apple growers in the USA, agricultural scientists, agricultural engineers, agricultural technology companies worldwide are the primary audience. Changes/Problems: Significant frost/freeze events in NC and MI damaged the apple crop While this resulted in loss and/or complications of some research associated with Objective 1, our team was still able to make progress on all sub-objectives of Objective 1. The protocols for the two multi-location trials (Obj 1A1 and 1A2) will be adjusted to make their execution more feasible. Sensing system integration and evaluation was delayed largely because of logistics challenges caused by the pandemic. The engineering team (including Cornell, Penn State, and MOOG) has discussed thoroughly and proposed solutions for it. Thus, we will conduct the system integration and field evaluation for apple orchards in New York in Year 2. At the same time, we started some development work (e.g., Obj 2h Augmented HMI) in Year 1 to ensure the overall success and progress of the project One PI from Penn State will be moving to another institution but will continue to serve as a PI on the project What opportunities for training and professional development has the project provided?Undergraduate Students In. MI, six undergraduate students (Austin Chase, John Loch, Lenz Blix, Megan Gawn, Alex Deahl and Wesley Banning) worked in Einhorn's lab on this projects.They learned skills in experimental design, measuring plant organ growth and phenotyping buds, fruitlets, and leaves. Students were trained to operate lab instrumentation and record, collate, and enter data. Graduate Students In NY, Mario Maranda Sazo, (Ph.D. candidate) worked with Cheng on the whole tree destructive sampling project. In NY, Tian Qiu, (first year Ph.D. student in Electrical and Computer Engineering) worked with Jiang starting in Jan 2021 and has focused on the use of full-perspective, high resolution 3D point cloud for apple tree characterization. He developed an initial data processing pipeline and investigated several key algorithms for tree segmentation, trunk detection, and trunk diameter measurement in the past year. In NY, Hehong Li, (first year Master of Engineering student in Electrical and Computer Engineering) worked with Jian since September 2021 and has explored the use of an augmented reality (AR) device for improved human machine interaction. She developed an application on the device for real-time data display. In MI, Laura Hillmann (Ph.D. candidate) worked with Einhorn in all areas of the project. She has been trained in Dr. Tom Sharkey's lab (MSU) on sample preparation, extraction techniques and measurement of nonstructural carbohydrates in fruit tissues. Presentations are provided under the section entitled 'Other Products'. In NC, James Larson, (PhD Candidate) worked with Kon on a research project to evaluate a portable Vis/NIR spectrometer to predict fruit abscission after a chemical thinner application and compared this with other methods to predict fruit abscission. He also worked on the project led by Einhorn to evaluate a novel model to predict fruit abscission and on the project to optimize bud density and crop load of Honeycrisp. He presented research on predicting fruit carbohydrate content with near infrared spectroscopy at the annual conference of the American Society of Horticultural Science and at an extension meeting in NC. In VA, Rashmi Sahu is the graduate student working on this project since August 2021. Graduate students Azlan Zahid, Magni Hussain, and Omeed Mirbod also have been contributing to the project with some of their work during 2021, but they were not paid by the project. Technical In NY, Spenser Howden is a research support specialist with Robinson who worked on the field experiments on this project.Kathy Campo a technician did data entry and fruit quality analysis. In NY, Kaspar Kuehn is a technician with Cheng who managed the trees and assisted in tree destructive sampling. Graham O'Donnell, a recent Cornell graduate who assisted in tree destructive sampling and data collection. In MI, Denise Ruwersma and Gail Byler are technicians with Einhorn who contributed to all crop load experiments to include initial selection of trees, measurements of trunks and bud numbers, pruning and hand thinning to target crop levels, leaf area and fruit growth measures (for both crop load experiments and fruit set prediction experiments), harvests and fruit quality analyses In NC, Chris Clavet is a research specialist; Tatiana Zuber is a research technician; Diana Vercillo is a research technician; and Cassie Detrio-Darby is a research technician who worked with Kon on this project. In WA, Zachary Chapman is an Agricultural Research Technologist worked with Musacchi on this project. Jill Dinius is a temporary research field assistant who helped in the field data collection for these experiments.Ryan Sheick is a Research assistant who was actively involved in data collection in all the objectives. In VA, Kenneth Savia is a field research specialist who worked with Sherif and collected and tabulated field data for fruit set, crop load and fruit cracking. Postdoctoral and Professional In NY, Luis Gonzalez is a post doc with Robinson and is responsible for this project. In MI, Mokhles Elsysy is a post doc with Einhorn and has directed and supported all undergraduate students who worked on this project, organized all lab activities related to harvest and evaluation of fruit quality and will statistically analyze data and aid in the development of the prediction model. In WA, Sara Serra is a research associate professor who works with Musacchi and is actively involved in all the objectives coordinating technical staff and collecting and analyzing data. In WA, Tory Schmidt is a Horticultural Associate working at the WTFRC who works with Musacchi and is actively involved in objectives 1A1 and 1A2, coordinating technical staff and collecting data. In VA, Md Tabibul Islam is aPostdoctoral Research Associate who works with Sherif and has assisted in designing the field experiments, flagging trees and analyzing data. Tabibul has received training on different laboratory analyses for fruit physical and chemical attributes (e.g. fruit weight, size, firmness, color, starch index, prix and ethylene content). How have the results been disseminated to communities of interest? Results of this project have been communicated to scientific and grower community through the project website (PACMAN), through a presentation to the national fruit growers conference (IFTA) in February by Robinson and through scientific articles. In addition individual project leaders in each state have communicated results of the project in state and regional meetings and webinars. In NY, Robinson presented concepts of precision crop load management at the winter fruit schools and also presented 12 webinars on crop load management in May and June. In NY, Jiang gave a digital seminar on Plant Morphological Phenotyping: From 2D to 3D on Oct 28, 2021. Audience included faculty members, professional/postdoctoral researchers, and students who are interested in digital agriculture. In MI, Einhorn, made an oral presentation in December 2021 to stakeholders at the Great Lakes Expo in Grand Rapids, MI on the fruit set prediction model which appears very promising after the first year of research and results have received considerable stakeholder interest. A user-friendly model (Excel data sheet template and instructions) will be provided to stakeholders to beta test for the 2022 thinning season. Einhorn also gave a presentation on apple thinning at the 2021 Spring virtual MSU Extension Fruit Education Meetings to Stakeholders. Einhorn was interviewed for a 2021 article by the Editor of the Good Fruit Grower journal describing our proposed research to stakeholders. L. Hillmann gave a presentation at the annual conference of the American Society for Horticultural Science to academic peers. In NC, Kon made six presentations related to this project to stakeholders in NC and one scientific presentation at a professional conference. Stakeholders in NC, SC, and GA have been introduced to the project, provided feedback on project direction, and have had the opportunity to tour research plots via extension programming. Six presentations related to this project were delivered to stakeholders in the southeast US and one scientific presentation at a professional conference. In VA, Sherifposted results from this project to the Virginia Tech Tree Fruit Extension and Outreach Facebook Page.https://www.facebook.com/VtechPomology.This page is managed by Dr. S. Sherif andadministered by Virginia Tech Extension Specialists and Virginia Cooperative Extension Agents to provide information about tree fruit production.1147 people follow this page.Sherif, S,also posted results from this project to his blog.https://blogs.ext.vt.edu/tree-fruit-horticulture/.This blog website is managed by Dr. S. Sherif and is primarily dedicated to discuss topics pertinent to tree fruit production in Virginia. There is now a total of 352 subscribers to this blog, the majority of them are tree fruit growers, extension agents and agricultural consultants. Several blogs had articles pertinent to apple crop load management. Field research & demonstration trials were established featuring precision crop load management protocols.In NY, MI and VA, two bud-load pruning trials were set up in Western New York, on Honeycrisp and Gala. Additional demonstration projects included: (1) an on-farm trial to fine tune the fruit growth rate model, (2) the use of ATS guided by the pollen tube growth model, (3) an evaluation ofmetamitron, 6-BA, and ACC, and (4)We collaborated with anindustry vendors of precision crop load management tools (Automated Fruit Scouting - AFS) to evaluate and improve anew AI-driven tool (AFS method) to measure fruit growth rates versus the traditional Cornell hand method. In MI we initiated field validation trial of Farm Vision crop load management technologies. Farm Vision is developing a smartphone-based app for detecting and measuring fruitlets. Two sites were established on commercial orchards to validate the technology Field days and grower/industry/research visits were conducted by project personnel featuring precision crop load management technologies.In NY, three presentations on precision crop load management were mad Cornell LOF summer tour organized in Wayne County.The results of the AFS method of measuring fruit growth rates was presented.In MI, an annual summer field day/tour attended by 250 featured precision technologies under development for tree fruit production.In VA,we extended the findings of our research on precision crop load management to Virginia apple growers and stakeholders through fruit schools, orchard meetings, the Tree Fruit Horticulture Updates blog website (https://blogs.ext.vt.edu/tree-fruit-horticulture) and the Virginia Tech Tree Fruit Extension and Outreach Facebook page (https://www.facebook.com/ssherif.VT). In WA, Project PI's and Consultant met in Summer 2021 and visited Project trial sites and commercial orchards to discuss commercial crop load practices.Mr. Rod Farrow, Project Producer-Consultant had meetings with project PI Musacchi and Co-PI Schmidt. Project advisory board member Garrett Grubbs, Chelan Fruit,at a Washington Fruit and Produce Company orchard, Dan Plath and project advisory board member Darrin Belton to discuss the commercial use of the AI-based automated fruit recognition and scouting AFS system.A half day was spent with Steve Mantel, owner of Innov8 Ag at the WTFRC-WSU-Innovat8 Ag "Smart Orchard". We focused on the Green Atlas Cartographer and its ability to count buds, map bloom, and calculate bloom density on the fly. We discussed in detail the value proposition of crop load management, including the variation of value proposition across annual activities (pruning, cluster thinning, bloom thinning, green fruit thinning), varieties and geographic locations. In VA,we conducted a state-wide survey to assess the impacts of our extension program on grower's knowledge of different horticulture-related topics, including apple blossom and fruit thinning. When asked about the impact of predictive models for fruit and blossom thinning, > 56% of respondents rated our extension program "beneficial" and "highly beneficial" for their knowledge of the benefits and utilization of the MaluSim model. In contrast, only 29% found that the PTGM model for blossom thinning was beneficial, and 27% rated it "not beneficial". Such a low rating for the PTGM was predicted to some extent given the potential risk of spring frost and the severe damage it can cause to apple blossoms, making the growers unwilling to start any early thinning treatment. What do you plan to do during the next reporting period to accomplish the goals? We will repeat the multi-location (NY, MI, NC and WA) detailed physiological studies to evaluate the effect of climate on crop load management results. The economics team in NY will calculate the economic optimum crop load for each of the four locations in this study. In NY we will determine the degree to which the current MaluSim model fits the dry matter data obtained from our destructive sampling of Honeycrisp on four rootstocks, and then collect shoot growth, leaf area and photosynthesis data to improve the sub-models where lack of fit is observed. In NY, Peck and Yu will supervise a newly hired post-doc in developing apple pollen tube growth rate models using advanced data analyses, such as machine/deep learning. The position is based in the Peck Lab at Cornell University. Dr. Yu Jiang will provide additional project guidance and supervision. In MI, Einhorn will continue the other physiological studies we have startedand will complete all NSC analyses; complete microscopic analyses of vascular tissues of pedicels; complete histological analyses of ovaries, complete modeling of year 1 fruit set predictions from four sites. Make modifications to improve all protocols for 2022. In NC, Koon will continue and expand his research with a Vis/NIR spectrometer to predict fruit abscission with the evaluation of other commercially relevant cultivars and thinning timings to determine if this technology continues to show merit in. Local/regional extension presentations are planned for the next reporting period along with reporting to scientific audiences. In WA, Musacchi will continue studies on WA38 cultivar to optimize thinning and pollination. They will also evaluate the MOOG Data Rover in 2 styles of orchard canopies (formally trained and free form). Moog corporation will provide training data for fruitlet detection for use in FY 2022 growing season. They will also train grower's to tag fruitlets using Microsoft Azure Machine Learning platform. They will support university partners in the implementation of the Data Rover sensing / algorithm efforts for fruitlet detection. They will also Support university partners in determiningaccuracy of fruitlet detection against ground truthing efforts.They will consult with university partners to determine the extensions of existing Data Rover sensor technology for a broader range of the orchard life cycle In NY, Jiang will evaluate the data rover on several orchard tree canopy shapes to determine its robustness to count buds, flowers and fruits in different styles of orchard canopies. In PA, the engineering team will acquire large amounts of images from early flower buds to green fruitlets using the MOOG imaging rover.They willcomplete the tagging of images taken by the MOOG rover, build the deep learning network models, and then test the algorithms for accuracy.They will.demonstrate the MOOG rover with imaging systems in Adams County, PA.They willgive extension presentations at grower meetings (such as Mid-Atlantic Fruit and Vegetable Convention) and to the IFTA conference in Hershey PA to disseminate the research results from the project. They also plan to train graduate student and possibly undergraduate students on development of scientific skills through daily research activities, publications, and presentations. In VA, Sherif will examine the efficacy and safety of materials and ratesfor chemicals blossom thinning on two apple cultivars, 'Honeycrisp' and 'Fuji'.The thinning experiment with GA4+7" will be repeated to validate the resultsof the 2021 season and optimize the rate and application timing of GA4+7 products to reduce the incidence of stem-end cracking in 'Gala' apples. The extension team will organize demonstrations, field days and grower meetings to present results from this project.

Impacts
What was accomplished under these goals? Impact of this project.Crop load management is the most economically important management practice in apple growing. Growers use pruning, chemical thinning and hand thinning to imprecisely manage crop load. This project conducted detailed physiological studies with several cultivars in four distinct apple growing regions of the US (West, Midwest, Northeast and Southeast) to precisely define the optimum flower bud load and then fruit load to maximize crop value. The horticultural results from these studies are being evaluated in an economic study to define the optimum crop load in economic terms. Once known, the optimum flower bud load can then be achieved through more precise pruning and fruit thinning We have also worked to improve three mathematical models we have previously developed (pollen tube growth model, apple carbohydrate model, and fruit growth rate model) which growers use to help achieve the optimum crop load. This year's work has led to improvements in all three models. We also made progress on developing an alternative simpler model to the fruit growth rate model. A limitation to grower adoption of strategies to precisely control crop load is the difficulty in manually counting the number of fruits on each tree.Through this project we are developing computer vision and autonomous vehicles which can count and georeference the number of flower buds or fruits on each tree. This year we have made significant progress and have a functioning prototype rover which can count the number of buds, flowers or fruitlets on each tree, upload the data to the cloud, process the date and communicate actionable information to orchard workers thought earphones. Major activities and accomplishments. Objective 1 In 2021 we conducted four multi-location (NY, MI, NC, WA) uniform experiments on (a) flower bud density, (b) crop load, (c) spur quality and (d) fruitlet abscission.At each of the four locations each of the four experiments was conducted using 'Gala' and 'Honeycrisp' apple cultivars. Regression analysis showed different slopes and intercepts at each location indicating that at a given crop load, larger fruits are produced in some climates than in other climates.These results will form the basis of an economic analysis of crop load and crop value being conducted by the economics team under objective 4. The project on fruit abscission led by Einhorn (MI) developed a user-friendly alternative to the fruit growth rate model which generated accurate predictions of final fruit set, comparable to those of the current model. In NC, Kon conducted an assessment of a portable Vis/NIR spectrometer to predict fruitlet abscission potential. They achieved greater than 90% accuracy in predicting whether a fruitlet persisted through the drop period. In WA, studies on the fruit set mechanism in 'WA38' were conducted by Musacchi.These studies support the theory of natural shedding of this variety is driven by high competition between fruitlets. In NY, Peck successfully updated the Pollen Tube Growth Model (PTGM) housed in the Network for Environment and Weather Applications (NEWA). The new interface worked really well for those who tried it this spring. The ability to store data is a game-changer for NEWA. In VA, Sherifconducted laboratory and greenhouse experiments to develop pollen tube growth models (PTGM) for two club apple cultivars. These models will be used directly by apple growers to time their blossom thinning sprays. Additionally, the raw data used for generating these models will be incorporated in a bigger dataset for developing a universal PTGM valid for all the commercial apple cultivars. In NY, Cheng worked to improve the MaluSim apple thinning carbohydrate model bycollecting tree growth and dry matter data on 'Honeycrisp' grown on four different rootstocks. At six key developmental stages throughout the growing season we destructively harvested the trees. Objective 2 Engineers at the MOOG corporation developed a system of computer vision to count the number of buds of flowers on a high density apple tree.Fruitlet data images from individual trees was collected throughout 2 season in New York (2019 and 2020).This data was used by MOOG engineers to develop an AI systems to learn to detect flower buds and fruitlets. The system was mounted on an electric ATV and named the "Data Rover".Efficiency of the Data Rover in counting buds at green tip was approximately 75% when correlated against ground truthing manual counts.Efforts to improve the accuracy have shown that significant cost would be associated with incremental improvements. How good is good enough is still in question? This question will be addressed with the advisory committee. A second step in developing a grower friendly system was to develop a way to process and then communicate information gathered by the rover to orchard workers.In 2020 engineers at MOOG developed a demo system of communicating individual tree data pulled from the cloud to field laborers when standing in front of a specific tree. The PSU group developed an on-tree measurement system of apple fruit size using a stereo vision system and neural network models including Faster R-CNN and Mask R-CNN. The system had a mean absolute error ranging from 1.1 to 4.2 mm. A fruitlet growth tracking algorithm was developed and able to correctly match 74% of all detected fruits during the mature fruit growth stage in September and October. In NY, Jiang focused on tree data collection using a high-resolution laser scanner. He found the optimal configuration for particular apple orchards and a general protocol on how to optimize the data acquisition in different orchards. They also developed a QGIS plugin that can show simulated tree information in a geographical information system. Objective 3 The PSU group have developed branch cutting mechanisms to remove branches from apple trees. A three degree-of-freedom (DoF) branch cutting mechanism was developed and tested in an apple orchard. The results indicated that the mechanism could prune branches up to 1 inch at different orientations. A fruit pulling removal device was developed. No significant difference was found on the required fruit removal forces for the fruits with diameter ranging from 10 -30 mm. Objective 4 In NY, Gomez's team began the economic evaluation of the horticultural data produced under objective 1 to determine in the four climates (NY, NC, MI and WA) the optimum flower bud load and optimum fruit load to maximize crop economic value.This effort will be completed for the data from year 1 by January 2022. The economics team also began developing partial fruit production budgets to be used in the future analysis (2023 and 2024) of the cost effectiveness of the technology developed in this project for precisely controlling crop load. Objective 5 In 2021 we organized an Advisory Committee (AC) consisting of 21 grower, nursery, and industry representatives. An initial meeting was held via Zoom on March 30, 2021 where project PI's presented their research objectives and plan of work followed by an open discussion where committee members gave advice and direction to the project leaders A website was created in the first quarter of 2021, PACMAN.extension.org, 'PACMAN' hopefully being an easily remembered acronym for Precision Apple Crop load MANagement. The website will be a platform to highlight our activities and results outside of the scientific publication audience. Therefore, the website target audience is grower, industry, University and Extension practitioners. Several field research & demonstration trials were established featuring precision crop load management protocols in NY, MI and VA. In addition,field days and grower/industry/research visits were conducted by project personnel in NY, MI, VA and WA featuring precision crop load management technologies.? ?

Publications

  • Type: Journal Articles Status: Published Year Published: 2021 Citation: Allen, W.A., T. Kon, and S.M. Sherif. 2021. Evaluation of blossom thinning spray timing strategies in apple. Horticulturae. 7(9):1-12. https:// doi.org/10.3390/horticulturae7090308
  • Type: Journal Articles Status: Published Year Published: 2021 Citation: Fazio G, and Robinson T. 2021. Designer rootstocks: the basis for precision management of apple orchards. Acta Hortic. 1314:275-286 DOI: 10.17660/ActaHortic.2021.1314.35.
  • Type: Journal Articles Status: Published Year Published: 2021 Citation: Robinson, T.L., P. Francescatto and J. Lordan. 2021. Advances in precision crop load management of apple. Acta Hortic. 1314:133-138 DOI: 10.17660/ActaHortic.2021.1314.18.
  • Type: Journal Articles Status: Published Year Published: 2021 Citation: Roeder S., Serra S. and Musacchi S. (2021). Pollination Period and Parentage Effect on Pollen Tube Growth in Apple Plants 10(8):1618. DOI: 10.3390/plants10081618
  • Type: Journal Articles Status: Published Year Published: 2021 Citation: Roeder S., Serra S., and Musacchi S. (2021). Novel Metrics to Characterize In Vitro Pollen Tube Growth Performance of Apple Cultivars. Plants 10(7):1460. DOI: 10.3390/plants10071460.
  • Type: Journal Articles Status: Published Year Published: 2021 Citation: Yang X, Chen L-S, Cheng L. Leaf Photosynthesis and Carbon Metabolism Adapt to Crop Load in ⿿Gala⿿ Apple Trees. Horticulturae. 2021; 7(3):47. https://doi.org/10.3390/horticulturae7030047
  • Type: Journal Articles Status: Published Year Published: 2021 Citation: Zahid, A., Mahmud, M.S., He, L., Heinemann, P., Choi, D., and Schupp, J. 2021. Technological advancements towards developing a robotic pruner for apple trees: A review. Computers and Electronics in Agriculture, 189, 106383.
  • Type: Book Chapters Status: Published Year Published: 2021 Citation: Larson, J.E., T.M. Kon, and A. Malladi. 2021. Apple fruitlet abscission mechanisms. Hort. Rev. 49:243-274. 10.1002/9781119851981.ch5
  • Type: Conference Papers and Presentations Status: Published Year Published: 2021 Citation: Larson, J.E., P. Perkins-Veazie, G. Ma, T.M. Kon, and M.L. Parker. 2021. Predicting apple carbohydrate content with near infrared spectroscopy. HortScience (Abstr.) HortScience 56(9):S94
  • Type: Conference Papers and Presentations Status: Published Year Published: 2021 Citation: Mirbod, O., Choi, D., Heinemann, P., He, L., and Schupp, J. 2021 In-Field Apple Size and Location Tracking Using Machine Vision to Assist Fruit Thinning and Harvest Decision-Making. 2021 ASABE Annual International Meeting, Paper No. 2100831. St. Joseph, Mich.: ASABE.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2021 Citation: Zahid, A., Mahmud, M.S. and He, L. 2021. Evaluation of Branch Cutting Torque Requirements Intended for Robotic Apple Tree Pruning. 2021 ASABE Annual International Meeting, Paper No. 2100262. St. Joseph, Mich.: ASABE.
  • Type: Journal Articles Status: Published Year Published: 2021 Citation: Robinson, T. and L. Gonzalez. 2021. Do bi-axis trees improve apple yield or fruit quality? Fruit Quarterly 29:(2):5-7.
  • Type: Journal Articles Status: Published Year Published: 2021 Citation: Gonzalez, L. and T.L. Robinson. 2021. Effect of different reflective ground covers on the coloring of apples at harvest. Fruit Quarterly 29:(4):10-15.