Progress 03/15/19 to 06/14/23
Outputs
Target Audience:Tree Fruit Growers, plicymakers, extension specialists, agricultural engineers Changes/Problems:
Nothing Reported
What opportunities for training and professional development has the project provided?Students and investigators obtained UAV licenses. Undergraduate students learned communications protocols between base-station computers and autonomous vehicles. A Ph.D. student and M.S. student graduated as a result of this project. A third Ph.D. student has completed 2/3 of the graduate program related to this project. How have the results been disseminated to communities of interest?Grower event - poster presentation Mid-Atlantic Fruit and Vegetable Growers Convention, Hershey, PA, February, 2023 Scientific meeting presentations American Society of Agricultural and Biological Engineers, Houston, TX, July, 2022 Northeast Agricultural and Biological Engineering Conference, Edgewood, MD, August, 2022 American Society of Agricultural and Biological Engineers, Omaha, NE, July, 2023 Northeast Agricultural and Biological Engineering Conference, Guelph, ON, Canada, August, 2023 What do you plan to do during the next reporting period to accomplish the goals?
Nothing Reported
Impacts
What was accomplished under these goals?
Objective 1 Objective completed prior to 2022-23. Objective 2 A UGV with auto-navigation capabilities was tested in the orchard in late Summer 2022. This utilized path planning to navigate the unit around the trees, poles, and other obstacles. A computational fluid dynamic model was developed to determine the effects of wind speed and wind direction on static and mobile heating of the orchard. Utilizing the concept of "percentage of protected canopy (PPC)", the simulation model was used to assess which heater scenarios would provide the maximum PPC. It was determined that mobile heating increased PPC compared to static heating, and angling the forced air heater towards into the wind was more effective than angling with the wind. Based on the results of the simulation scenarios, a mobile heater was mounted on the UGV to test an automatic control for setting the heater angle based on a wind direction sensor also mounted on the UGV. Objective 3 The ability of the network to provide and execute navigation paths for the UGV was examined. A UAV captured thermal images, and five simulated potential target "cold" locations for UGV destination in an actual research orchard block. Path planning algorithms were used to determine optimal paths, avoiding collisions with tree rows and other obstacles. Computation time of path planning algorithm execution was calculated. The UGV executed the navigation to the five target locations utilizing the path planning results. Comparing the proposed path with the actual GPS coordinates of the UGV path, the calculated deviation from the proposed path was 3.44 cm RMS error latitudinally and 8.28 cm longitudinally. These were considered to be very acceptable results. The mean computational time for creating the paths was 7.85 seconds, which was well within a practical operational time frame. The fully integrated, autonomous system communications were being developed when the project ended. This would provide the real-time capture of thermal images, transfer to a base station, execution of the path planning algorithm, and signal to the UGV to move to the destination for heating using the optimal path, without collisions and within acceptable navigation error.
Publications
- Type:
Journal Articles
Status:
Accepted
Year Published:
2023
Citation:
Hua, W., Heinemann, P.H., He, L. 2023. CFD simulation of porous canopy heat transfer in apple orchard-based frost protection. Journal of the ASABE. 66(4)
- Type:
Journal Articles
Status:
Published
Year Published:
2023
Citation:
Yuan, W., Hua, W., Heinemann, P.H., He, L. 2023. UAV photogrammetry-based apple orchard blossom density estimation and mapping. Horticulturae. 9(2). 266.
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2022
Citation:
Hua, W., He, L., Heinemann, P.H. 2022. Numerical modeling of fluid flow and heat transfer in a 2d orchard with COMSOL Multiphysics: a case study. Northeast Agricultural and Biological Engineering Conference. The American Society of Agricultural and Biological Engineers. 17 pp.
- Type:
Theses/Dissertations
Status:
Published
Year Published:
2022
Citation:
Canto, M.T. 2022. Prevention of frost damage to apple tree bud stages using an autonomous multi-vehicle system. M.S. Thesis. Department of Agricultural and Biological Engineering, The Pennsylvania State University, University Park, PA. 56 pp.
- Type:
Theses/Dissertations
Status:
Published
Year Published:
2022
Citation:
Yuan, W. 2022. Development of a UAV-based multi-dimensional mapping framework for precise and convenient frost management in apple orchard. Ph.D. Dissertation. Department of Agricultural and Biological Engineering, The Pennsylvania State University, University Park, PA. 110 pp.
|
Progress 03/15/21 to 03/14/22
Outputs
Target Audience:Tree Fruit Growers, plicymakers, extension specialists, agricultural engineers Changes/Problems:
Nothing Reported
What opportunities for training and professional development has the project provided?Training for students and investigators in drone piloting A PhD student graduated as a result of this project. The title of the dissertation is Development of a UAV-based Multi-dimensional Mapping Framework for Precise and Convenient Frost Management in Apple Orchard. How have the results been disseminated to communities of interest?Presentations made at the 2022 Mid-Atlantic Fruit and Vegetable Growers Convention, Hershey PA, February 2022, targeting fruit growers: Oral presentation and a poster on heater temperature tests. Oral presentation on UGV path planning concept. Oral presentation on frost temperature sensing and heat requirement mapping Presentations made at the International Fruit Tree Association 65th Annual Conference and Tours, Hershey, PA, and Fruit Research and Extension Center, Biglerville, PA, February, 2022, targeting fruit growers: Oral presentation on frost temperature sensing and heat requirement mapping Presentation and UAV demonstration on mapping apple flower stages for frost protection decisions. What do you plan to do during the next reporting period to accomplish the goals?Develop a path planning algorithm for multiple UGVs based on an optimal heating strategy. Final field testing with system integration will be performed.
Impacts
What was accomplished under these goals?
Objective 1 Using YOLOv4 as a representation of state-of-the-art object detectors, the sensitivity of YOLOv4's for apple blossom detection was quantified against artificial image distortions including white noise, motion blur, hue shift, saturation change, and intensity change. The importance of various training dataset attributes was examined based on model classification accuracies of flower detection, including dataset size, label quality, negative sample presence, image sequence, and image distortion levels. Objective 2 Two temperature field tests in an apple orchard equipped with two propane heaters were accomplished. A computational fluid dynamic model (CFD) for heat transfer in porous canopies was developed and validated. The influences of heater output, heating duration, and heating angles were simulated and analyzed. Developed a ground vehicle platform that can be operated autonomously using a RF-based transceiverand UDP protocol, built and tested a UAV that can capture and send spatiotemporal maps in real-time using a heuristic approach. Implemented a path planning algorithm to prioritize and map optimal paths to target locations in the field based on temperature deficit data from the UAV. Performed field and simulation tests to assess the UGV's navigation and path planning algorithm's capabilities when given imitation spatiotemporal map data. Objective 3 Developed a real-time wireless communication network between the vehicles and ground station using Robot OperatingSystem and MATLAB. Performed field testing for the transmission of thermal images and pose data from the UAV to the base station. The ability of the network to provide and execute navigation paths for the UGV was also examined.
Publications
- Type:
Journal Articles
Status:
Published
Year Published:
2022
Citation:
Yuan, W., Choi, D., Dimitrios, V., Heinemann, P., He, L. 2022. Sensitivity examination of yolov4 Regarding Test Image Distortion and Training Dataset Attribute for Apple Flower Bud Classification. International Journal of Remote Sensing. In press. DOI: 10.1080/01431161.2022.2085069
- Type:
Conference Papers and Presentations
Status:
Accepted
Year Published:
2022
Citation:
Hua, W., He, L., Heinemann, P.H. 2022. A heat transfer model for an apple orchard equipped with mobile heaters during frost. 2022 ASABE Annual International Meeting, Paper no. 2200191.
|
Progress 03/15/20 to 03/14/21
Outputs
Target Audience:Stakeholders including the tree fruit industry, policymakers, farmers, extension specialists and educators, and agricultural engineers are the target audience of the project. Changes/Problems:Ground vehicle for field testing has been not yet delivered due to the change in internal process of Penn State. Once it is delivered, the heater and other components can be integrated. What opportunities for training and professional development has the project provided?Three graduate students (two master's students and one Ph.D.) have been funded from this project. Two students will be graduating next year. How have the results been disseminated to communities of interest?Research conducted from this project has been published in a journal (Remote Sensing) and presented in conferences (NSF PI meeting, ASABE International Annual Meeting, and Mid-Atlantic Fruit and Vegetable Convention). What do you plan to do during the next reporting period to accomplish the goals?Objective 1: Integrate UAVs' communication with a ground vehicle so that the system can conduct field experiments for testing. Continue upgrading algorithms for machine vision flower stage evaluation and temperature sensing (thermal requirement analysis). Objective 2: Place heaters on ground robots and test in orchard during cold conditions. Continue to improve the CFD model for determining optimal heating equipment placement and cycles under differing scenarios. Objective 3: Complete the networking and hardware configuration for testing and analysis.
Impacts
What was accomplished under these goals?
Objective 1: Heating requirement assessment methodology was developed for frost protection in an apple orchard utilizing unmanned aerial vehicle (UAV)-based thermal and RGB cameras. YOLOv4 classifiers for six apple flower bud growth stages in various network sizes were trained based on 5040 RGB images, and the best model achieved a 71.57% mAP for a test dataset consisted of 360 images. A flower bud mapping algorithm was developed to map classifier detection results into dense growth stage maps utilizing RGB image geoinformation. Heating requirement maps were created using artificial flower bud critical temperatures to simulate orchard heating demands during frost events. Objective 2: Heaters were tested in the orchard to determine the distribution of heat. Temperature sensors were placed in a horizontal and vertical array within the orchard canopy. Two 150,000 BTU heater units were placed in different configurations and run at 10 minute intervals. It was determined that best heating was provided with heaters placed 90n degrees perpendicular to the row, i.e. directly facing the trees. Typically in a frost night, 2-5 degrees C increase in temperature of air surrounding the blossoms is sufficient to protect the blossoms from frost damage. In the 45 degree angle and 90 degree angle configurations, even these small heaters supplied sufficient heating. Development of a computational fluid dynamics (CFD) model of heating using the specifications of the tested heaters was initiated during this reporting period. The model will help predict the air temperature response to moving heaters under different scenarios. Considerable refinement of the model is needed and will continue. Bio-electrical survey for alternative heating method was completed. Objective 3: UGV & UAS hardware has been assembled with communication and required heating payload (Not complete, in process of assembling). Data for integrating ground and aerial vehicle system was collected and is being analyzed.
Publications
- Type:
Journal Articles
Status:
Published
Year Published:
2021
Citation:
Yuan W, Choi D. UAV-Based Heating Requirement Determination for Frost Management in Apple Orchard. Remote Sensing. 2021; 13(2):273. https://doi.org/10.3390/rs13020273
- Type:
Conference Papers and Presentations
Status:
Other
Year Published:
2021
Citation:
Choi D., He L., Heinemann P., Lyon, D., Crassweller R., Sommer J. 2021. Integrated Design of Sensing, Network, and Cooperative Control of Multi-Vehicle Systems for Preventing Frost and Freeze Damage to Flowers and Buds of Fruit Trees. 2021 NSF Cyber-Physical Systems Principal Investigators' Meeting
June 2-4, 2021. Award ID#: 1836974 (Oral Presentation)
- Type:
Other
Status:
Other
Year Published:
2021
Citation:
Choi D., He L., Heinemann P., Lyon, D., Crassweller R., Sommer J. 2021. Integrated Design of Sensing, Network, and Cooperative Control of Multi-Vehicle Systems for Preventing Frost and Freeze Damage to Flowers and Buds of Fruit Trees. 2021 NSF Cyber-Physical Systems Principal Investigators' Meeting
June 2-4, 2021. Award ID#: 1836974 (Poster)
- Type:
Conference Papers and Presentations
Status:
Other
Year Published:
2021
Citation:
Choi, D. (February 11, 2021). "Computer Vision & Drone Imaging in Apple Orchards," Mid-Atlantic Fruit and Vegetable Convention, State Horticultural Association of Pennsylvania, Virtual.
- Type:
Conference Papers and Presentations
Status:
Other
Year Published:
2020
Citation:
Yuan, W. (Presenter), & Choi, D. (July 13, 2020). "Flowering Stage Classification and Temperature Monitoring for Frost Damage Management in Apple Orchard Using UAV-based RGB and Thermal Cameras," 2020 American Society of Agricultural and Biological Engineers Annual International Meeting, Online.
|
Progress 03/15/19 to 03/14/20
Outputs
Target Audience:Stakeholders including the tree fruit industry, policymakers, farmers, extension specialists and educators, and agricultural engineers are the target audience of the project. Changes/Problems:
Nothing Reported
What opportunities for training and professional development has the project provided? Two graduate students in this project had the opportunity to get training and take FAA drone flight pilot licenses to operate drones. The project supports graduate students attendingconferences to present research results (canceled or changed to virtual due to COVID-19) How have the results been disseminated to communities of interest?A news article regarding the project has been published to community members. Title: NSF grant supports development of robotic frost protection in orchards. Penn State News. March 25, 2019. Link:https://news.psu.edu/story/565198/2019/03/25/research/nsf-grant-supports-development-robotic-frost-protection-orchards What do you plan to do during the next reporting period to accomplish the goals?Objective 1. Various deep learning algorithms will be tested and implemented for automatic flower stage identification. Flight patterns for the thermal camera (night flight) will be investigated. A critical temperature map will be created based on flower stages identified by the developed deep learning algorithm. Objective 2. More comprehensive heating tests with designed size heater will be conducted in the orchards during cold nights or mornings, especially when frost event occurs. An autonomous ground vehicle will be developed and tested with the integration of RTK GPS, IMU, and LiDAR sensors. The autonomous ground vehicle will be used to move the heater to the targeted locations in the orchard that the UAV system identifies. Objective 3. Protocols for communications between vehicles will be tested and implemented. Feasibility tests will be conducted for real-time data transfer between vehicles.
Impacts
What was accomplished under these goals?
Objective 1. Two UAVs (large and medium sizes)have been tested to acquire images of canopies at proximity. Also, ground vehicle-based imaging was tested. We finalized the larger size UAV-based system (DJI Matrice600 with 10 times optical zoom for an RGB camera) is superior for proximal image acquisition of flowers among tested platform. An RGB image database of various stages of flowers from bud to full bloom was captured. A thermal camera was installed on a large size UAV and calibration tests were conducted. In addition, we have also investigated exposing biomass to critical temperature events (both minimum and maximum) to observe physiological changes at a cellular scale. Objective 2. A simplified mathematic model was established to simulate the heat transfer in an open space. The heater size and fuel consumption over a certain period of time could be obtained from the model. An experiment system with a heater and 20 temperature sensors was tested in an apple orchard to investigate the heat transfer in the orchard environment. The temperature change over the defined area was measured, and the capacity of the heating area and sustained time were identified as well. An inertial navigation system (including an RTK GPS and an IMU) was tested in an orchard environment with a ground utility vehicle. The trajectory of the vehicle and the orientation of the vehicle heading were recorded and analyzed. Objective 3. We have been testing and researching the viability of integrating microcontrollers on small scale UAV platforms for communication and control in an integrated multiplatform system. We have begun creating the networking backbone for the multiplatform system.
Publications
- Type:
Conference Papers and Presentations
Status:
Other
Year Published:
2020
Citation:
Yuan, Wen & Choi, Daeun. (2020). Flowering stage classification and temperature monitoring for frost damage management in apple orchards using UAV-based RGB and thermal cameras. 2020 ASABE Annual International Meeting Paper.
|
|