Progress 03/05/14 to 12/31/18
Outputs
Target Audience:Through out this project, research results have been published in referred journals, and project outcomes have been shared with Florida Citrus Industry through various means including workshops, forums and extension events. Changes/Problems:
Nothing Reported
What opportunities for training and professional development has the project provided?During this reporting period, there was one graduate student working on this project. Student was involved in daily research activities, completed PhD dissertation and published his results. How have the results been disseminated to communities of interest?The work from this project has been presented at conferences and published in referred journals. What do you plan to do during the next reporting period to accomplish the goals?This is the final year of this project.
Impacts
What was accomplished under these goals?
Mass Harvester Development: The majority of our work has been directed toward the development of a new Over the ToP Citrus Harvester (OTPCH) for processed fruit, which has been specifically designed for high density semi-dwarf trees. This machine has unique features that will allow the machine to harvest trees as young as four years old, and adapt to the growing tree as it reaches maturity at approximately 8 ft tall and 8 ft wide. Proposed planting density for ACPHS trees of around 375 trees per acre yielding approximately 700 field boxes per acre may be harvested at speeds approaching 1.5 mph. Preliminary trials suggests that harvesting removal efficiencies of 94% or better can be achieved based on grapefruit harvesting trials and later confirmed on Valencia citrus. The primary research and development tasks 1) develop the OTP platform and power plant, 2) the shaking mechanism and mount design, 3) catch frame assembly, and 4) fruit conveyance approach. The University of Florida began working on an active catch frame design for full size trees in traditional groves in 2009, while the platform development was started in 2012. Field trials began in 2013 and continue as we seek to improve the machine performance. In 2015, we began design and fabrication of a second generation prototype which would improve ruggedness in normal grove conditions and enable the implementation of material handling systems. In addition, we are developing technologies for the fresh market harvest using robotics technologies. In 2015 to 2017 we designed, fabricated and tested a new prototype end effector for citrus harvesting. Plus worked on several machine vision technologies described below. In 2016 and 2017, we continued the fabrication of a second prototype which will have the new adaptive shaker head, load leveling and a catch frame to capture the fruit along with an internal conveyance system to transfer the harvested fruit to the trailing transport vehicles. In addition, in 2016-2017, we were developing a tree profiling technology that would use LEDDAR to measure tree canopy size and shape and could be used to control the harvesting head position to optimize fruit harvest efficiency and reduce tree limb damage. In 2017 and 2018, we completed fabrication of the new harvesting mechanism which was mounted on the new over the top platform. The completion of the second prototype continues even now, as the systems hydraulics and controls are designed and implemented on the machine platform. Once all fabrication and assembly tasks are completed, the new prototype will be tested in field conditions. Fruit Position Estimation. One of the first steps in automated fruit harvesting is determining the location of fruit on a tree. Computer vision technology can be leveraged to provide the position of the fruit relative to the robotic arms used to pick the fruit. This issue motivates the use of another common reconstruction technique known as structure from motion (SfM), where multiple views are generated from a single moving camera. In this setup, the amount of camera motion needed to accurately estimate depth can be dynamically tuned for the current features of interest. One caveat with this class of algorithms is that they expect persistent visibility of the feature of interest. This is unrealistic in the fruit harvesting problem, where foliage may temporarily obstruct visibility. In the work summarized here, we analyze the robustness of a class of SfM techniques to intermittent observability, and provide conditions to guarantee accurate position estimation of the fruit. In addition, we provide a method for estimating the position of the fruit when they are not visible to the camera. A naïve approach would be to hold the position estimate constant at the estimate value when last observed by the camera. However this does not account for fruit movement due to external disturbances such as wind gusts or branch movement due to other fruit being picked. In this work we develop a method for evolving the estimated fruit position over time, even when the fruit is not visible to the camera. In addition to robustifying the estimators, this relaxes constraints on the camera motion. Whereas before, the camera had to be far enough away to see all fruit, and therefore move in large enough arcs to induce sufficient parallax, now the camera can temporarily move closer to a small group of fruit, increasing the accuracy of the vision based estimation and reducing the required translation of the camera (features closer to the camera require less translation to induce sufficient parallax for depth estimation). In 2017 and 2018, we explored two other topics in machine vision related to robotic fruit harvesting. The first being the development of an adaptive visual servo control in the presence of fruit motion and secondly the effect of unknown time delays. First, one practical challenge in robotic harvesting is to grasp a moving fruit, where unsuccessful pick cycles may result in lower harvesting efficiency. The fruit motion can be due to exogenous disturbances such as wind gust, canopy unloading, and particularly, fruit detachment forces. With respect to the specific aim of developing model-based controllers to compensate for unknown fruit motion, a direct adaptive visual servo control law was developed that estimates the fruit motion and offers appropriate compensation. The fruit motion in the image plane and along the optical axis is modeled as a second-order spring-mass system. The unknown parameters of fruit motion are identified using an adaptive parameter update law, and a robust feedback term is included in the control law to account for modeling uncertainties. Lyapunov-based stability analysis guarantees uniformly ultimately bounded regulation of the robot to the fruit position. Secondly, image feedback has become a common means to provide feedback to autonomous systems (e.g., robotic manipulator). However, when used in a feedback loop, acquiring and then processing images introduces delays in both the state feedback and the control input. Since the image processing time depends on the complexity of the image/environment, the delays can be time-varying. Such effects can destabilize the closed-loop system. With respect to the specific aim of developing visual servo controllers in the presence of time delays, a robust control approach is introduced that enables an autonomous system to track a desired trajectory despite the time-varying input and state delays. Lyapunov-based analysis methods were used to prove the tracking error is uniformly ultimately bounded.
Publications
- Type:
Journal Articles
Status:
Published
Year Published:
2019
Citation:
You, K., Burks, T., and Schueller, J. 2019. Development of an Adaptable Vacuum Based Orange Picking End Effector. Agricultural Engineering International: the CIGR Ejournal., Accepted final revisions.
- Type:
Journal Articles
Status:
Published
Year Published:
2018
Citation:
Ni, Z, and T.F. Burks, 2018, 3D Dense Reconstruction of Plant or Tree Canopy based on Stereo Vision, The CIGR Ejournal. Manuscript 4358. Vol. 20. No.2, pp. 248-260.
- Type:
Journal Articles
Status:
Published
Year Published:
2019
Citation:
Gangadharan, S., Burks, T., and Schueller, J. 2019. A Comparison of Citrus Canopy Profile Generation Approaches using Ultrasonic and Leddar Sensors. Computers and Electronics in Agriculture, Vol. 156, pp. 71-83.
|
Progress 10/01/16 to 09/30/17
Outputs
Target Audience:
Nothing Reported
Changes/Problems:
Nothing Reported
What opportunities for training and professional development has the project provided?This project was supporting 1 post doctorate and 1 graduate student during this reporting year. How have the results been disseminated to communities of interest?The results from this work has been published in referred journals and has been presented at society conferences. What do you plan to do during the next reporting period to accomplish the goals?We plan to continue to work toward program objectives.
Impacts
What was accomplished under these goals?
Mass Harvester Development: The majority of our work has been directed toward the development of a new Over the ToP Citrus Harvester (OTPCH) for processed fruit, which has been specifically designed for high density semi-dwarf trees. This machine has unique features that will allow the machine to harvest trees as young as four years old, and adapt to the growing tree as it reaches maturity at approximately 8 ft tall and 8 ft wide. Proposed planting density for ACPHS trees of around 375 trees per acre yielding approximately 700 field boxes per acre may be harvested at speeds approaching 1.5 mph. Preliminary trials suggests that harvesting removal efficiencies of 94% or better can be achieved based on grapefruit harvesting trials and later confirmed on Valencia citrus. The primary research and development tasks 1) develop the OTP platform and power plant, 2) the shaking mechanism and mount design, 3) catch frame assembly, and 4) fruit conveyance approach. The University of Florida began working on an active catch frame design for full size trees in traditional groves in 2009, while the platform development was started in 2012. Field trials began in 2013 and continue as we seek to improve the machine performance. In 2015, we began design and fabrication of a second generation prototype which would improve ruggedness in normal grove conditions and enable the implementation of material handling systems. In addition, we are developing technologies for the fresh market harvest using robotics technologies. In 2015 to 2017 we designed, fabricated and tested a new prototype end effector for citrus harvesting. Plus worked on several machine vision technologies described below. In 2014 and 2015, we began running field trials in Valencia orange to evaluate if performance remained consistent with results we had seen in grapefruit. These trees were substantially larger than the previous semi-dwarf grapefruit trees we had harvested. Initially the harvesting results were substantially lower than in grapefruit, partly because the trees were larger but also because the trees were un-skirted. This resulted in a large percentage of fruit being located below the shaker line. We plan on improving performance in this area in the coming year by skirting the trees and insuring that the tree head height doesn't exceed the tunnel clearance of the harvester. In addition, we are working on a new adaptive shaker control for the harvester that we believe will improve harvesting efficiency while also reducing tree damage. This can be accomplished by dynamically measuring canopy size and adjusting the shaker to the profile of the tree. In 2016 and 2017, we continued the fabrication of a second prototype which will have the new adaptive shaker head, load leveling and a catch frame to capture the fruit along with an internal conveyance system to transfer the harvested fruit to the trailing transport vehicles. In addition, in 2016-2017, we were developing a tree profiling technology that would use LEDDAR to measure tree canopy size and shape and could be used to control the harvesting head position to optimize fruit harvest efficiency and reduce tree limb damage. Fruit Position Estimation. One of the first steps in automated fruit harvesting is determining the location of fruit on a tree. Computer vision technology can be leveraged to provide the position of the fruit relative to the robotic arms used to pick the fruit. This issue motivates the use of another common reconstruction technique known as structure from motion (SfM), where multiple views are generated from a single moving camera. In this setup, the amount of camera motion needed to accurately estimate depth can be dynamically tuned for the current features of interest. One caveat with this class of algorithms is that they expect persistent visibility of the feature of interest. This is unrealistic in the fruit harvesting problem, where foliage may temporarily obstruct visibility. In the work summarized here, we analyze the robustness of a class of SfM techniques to intermittent observability, and provide conditions to guarantee accurate position estimation of the fruit. In addition, we provide a method for estimating the position of the fruit when they are not visible to the camera. A naïve approach would be to hold the position estimate constant at the estimate value when last observed by the camera. However this does not account for fruit movement due to external disturbances such as wind gusts or branch movement due to other fruit being picked. In this work we develop a method for evolving the estimated fruit position over time, even when the fruit is not visible to the camera. In addition to robustifying the estimators, this relaxes constraints on the camera motion. Whereas before, the camera had to be far enough away to see all fruit, and therefore move in large enough arcs to induce sufficient parallax, now the camera can temporarily move closer to a small group of fruit, increasing the accuracy of the vision based estimation and reducing the required translation of the camera (features closer to the camera require less translation to induce sufficient parallax for depth estimation). In 2016 and 2017, work was undertaken to implement a new approach using multi-perspective cameras, with the specific aim of improving fruit mapping for efficient robot path planning and servo control operations. A new robust fruit localization approach was developed. Fruit localization is one of the building blocks in many robotic agricultural operations (e.g., yield mapping and robotic harvesting) that determines 3D Euclidean positions of the fruits using one or several sensors. Therefore, it is crucial to guarantee the performance of the localization methods in the presence of fruit detection errors and unknown fruit motion (e.g., due to wind gust), so that the desired efficiency of the subsequent systems can be achieved. For instance, inaccurate localization may severely affect fruit picking efficiency in robotic harvesting. The developed estimation-based localization approach provided estimates of the fruit positions in the presence of fruit detection errors and unknown fruit motion, and it is based on a new sensing procedure that uses multiple (>2) inexpensive monocular cameras. A nonlinear estimator called particle filter is developed to estimate the unknown position of the fruits using image measurements obtained from multiple cameras. Since the accuracy of localization is affected by errors in fruit detection, the developed sensor model includes non-Gaussian fruit detection errors along with image noise. Fruit motion can significantly reduce harvesting efficiency due to errors in locating moving fruits. Therefore, in contrast to existing methods, the dynamics of fruit motion are derived and included in the localization framework to obtain time-varying position estimates of the moving fruits. A detailed theoretical foundation was laid for the new estimation-based fruit localization approach, and it was validated through extensive Monte Carlo simulations.
Publications
- Type:
Journal Articles
Status:
Published
Year Published:
2017
Citation:
Mehta, S.S., C. Ton, S. Asundi, T. F. Burks, 2017, "Multiple Camera Fruit Localization using a Particle Filter," Computers and Electronics in Agriculture, vol. 142, pp. 139-154.
|
Progress 10/01/15 to 09/30/16
Outputs
Target Audience:We continue to work with a selected group of Florida Citrus Growers who have demonstrated interest in the future of high density citrus production. These growers have provided groves for testing and evaluation of equipment systems as we have been developing. Citrus owners and grove managers continue to be interested in the work we are doing. Changes/Problems:
Nothing Reported
What opportunities for training and professional development has the project provided?There have been mulitple graduate students who have worked on this project in various capacities. This project has enabled them to study and do research in the areas of agricultural automation, robotics and machine vision. How have the results been disseminated to communities of interest?Conference papers, proceedings articles and journal articles have been produced as a result of this research. What do you plan to do during the next reporting period to accomplish the goals?We will continue to develop our second prototype over the top citrus harvester during the next reporting cycle, and will continue to work on novel machine vision approaches for robotic harvesting of citrus.
Impacts
What was accomplished under these goals?
The majority of our work has been directed toward the development of a new Over the ToP Citrus Harvester (OTPCH) for processed fruit, which has been specifically designed for high density semi-dwarf trees. This machine has unique features that will allow the machine to harvest trees as young as four years old, and adapt to the growing tree as it reaches maturity at approximately 8 ft tall and 8 ft wide. In addition, we are also developing technologies for the fresh market harvest using robotics technologies. We are currently designing and testing a new prototype end effector for citrus harvesting. Plus working on several machine vision technologies described below. Mass Harvester Development. In 2014 and 2015, we began running field trials in Valencia orange to evaluate if performance remained consistent with results we had seen in grapefruit. These trees were substantially larger than the previous semi-dwarf grapefruit trees we had harvested. Initially the harvesting results were substantially lower than in grapefruit, partly because the trees were larger but also because the trees were un-skirted. This resulted in a large percentage of fruit being located below the shaker line. We plan on improving performance in this area in the coming year by skirting the trees and insuring that the tree head height doesn't exceed the tunnel clearance of the harvester. In 2016, we continue the fabrication of a second prototype which will have the new adaptive shaker head, load leveling and a catch frame to capture the fruit along with an internal conveyance system to transfer the harvested fruit to the trailing transport vehicles. Fruit Position Estimation. One of the first steps in automated fruit harvesting is determining the location of fruit on a tree. Computer vision technology can be leveraged to provide the position of the fruit relative to the robotic arms used to pick the fruit. This issue motivates the use of another common reconstruction technique known as structure from motion (SfM), where multiple views are generated from a single moving camera. In this setup, the amount of camera motion needed to accurately estimate depth can be dynamically tuned for the current features of interest. One caveat with this class of algorithms is that they expect persistent visibility of the feature of interest. This is unrealistic in the fruit harvesting problem, where foliage may temporarily obstruct visibility. In the work summarized here, we analyze the robustness of a class of SfM techniques to intermittent observability, and provide conditions to guarantee accurate position estimation of the fruit. In addition, we provide a method for estimating the position of the fruit when they are not visible to the camera. A naïve approach would be to hold the position estimate constant at the estimate value when last observed by the camera. However this does not account for fruit movement due to external disturbances such as wind gusts or branch movement due to other fruit being picked. In this work we develop a method for evolving the estimated fruit position over time, even when the fruit is not visible to the camera. In addition to robustifying the estimators, this relaxes constraints on the camera motion. Whereas before, the camera had to be far enough away to see all fruit, and therefore move in large enough arcs to induce sufficient parallax, now the camera can temporarily move closer to a small group of fruit, increasing the accuracy of the vision based estimation and reducing the required translation of the camera (features closer to the camera require less translation to induce sufficient parallax for depth estimation). Simulations of a moving camera observing a moving object were performed to verify these results. An observer was used to estimate object position during the periods in which the object was in view, and simulation parameters were set as identical to those used in previous work in order to compare our results to the case when the object is persistently in view. A switching signal with randomly generated dwell times was used to signify duration spent with and without the object in view. Efforts to date has focused on developing the mathematical analysis to show the robustness of a class of SfM algorithms to intermittent measurements, provided a set of dwell time conditions are met and a motion model of the moving fruit is known. Future work will investigate the effectiveness of different motion models for estimating fruit velocity, and will explore adaptive methods for learning the model online. In another study, motivated by an effort to study layered vision systems for robotic harvesting, we investigated the problem of fruit localization using multiple cameras in the fruit detection layer of the vision system. A pseudo stereo-vision approach is presented where fruit matching is accomplished by loosely holding the epipolar constraint to reduce computation time. In the presence of noise, heuristics are presented to identify the greatest subset of cameras with matched fruits. Subsequently, the fruit depth is obtained by minimizing the summation of the image reprojection error in cameras with matching fruit. Monte Carlo simulations are performed to establish localization efficiency of the proposed approach under varying design parameters and image noise. In a related work, another localization approach was studied using a new sensing procedure that uses multiple (≥ 2) inexpensive monocular cameras. A nonlinear estimator called particle filter is developed to estimate the unknown position of the fruits using image measurements obtained from multiple cameras. The particle filter is partitioned into clusters to independently localize individual fruits, while the behavior of the clusters is manipulated at global level to maintain a single filter structure. Since the accuracy of localization is affected by errors in fruit detection, the presented sensor model includes non-Gaussian fruit detection errors along with image noise. Fruit motion (e.g., due to wind gust) can significantly reduce harvesting efficiency due to errors in locating moving fruits. In contrast to existing methods, the dynamics of fruit motion are derived and included in the localization framework to obtain time-varying position estimates of the moving fruits. A detailed theoretical foundation is provided for the new estimation-based fruit localization approach, and it is validated through extensive Monte Carlo simulations. The performance of the estimator is evaluated by varying the design parameters, measurement noise, number of fruits, amount of overlap in clustered fruit scenarios, and fruit velocity. Correlation of these parameters with the performance of the estimator is derived, and guidelines are presented for selecting the design parameters and predicting performance bounds under given operating conditions.
Publications
- Type:
Journal Articles
Status:
Published
Year Published:
2016
Citation:
Ni(g), Z. T.F. Burks, and W.S. Lee. 2016, 3D Reconstruction of Plant/Tree Canopy using Monocular and Binocular Vision. Journal of Imaging. 2016, 2(4), 28; doi: 10.3390/jimaging2040028.
- Type:
Journal Articles
Status:
Accepted
Year Published:
2016
Citation:
Mehta, S.S., W. MacKunis, T. F. Burks, 2016, Robust Visual Servo Control in the Presence of Fruit Motion for Robotic Citrus Harvesting, Computers and Electronics in Agriculture, vol. 123, pp. 362-375.
|
Progress 10/01/14 to 09/30/15
Outputs
Target Audience:Were are development customized machniery for the Florida Citrus Industry, which also may have application to the emerging Florida Olive industry. Changes/Problems:
Nothing Reported
What opportunities for training and professional development has the project provided?This project has employeed two graduate students and one post doctorate student who are working on tasks associated with this project. How have the results been disseminated to communities of interest?Yes they have been presented at national and International conferences and are being published in referred journals. What do you plan to do during the next reporting period to accomplish the goals?We will be primarily focusing on building the commercial prototype of the over the top mass harvester, and continuing to improve our vision-based detection, mapping and servo control approaches.
Impacts
What was accomplished under these goals?
Mass Harvester Development.In 2014 and 2015, we began running field trials in Valencia orange to evaluate if performance remained consistent with results we had seen in grapefruit. These trees were substantially larger than the previous semi-dwarf grapefruit trees we had harvested. Initially the harvesting results were substantially lower than in grapefruit, partly because the trees were larger but also because the trees were un-skirted. This resulted in a large percentage of fruit being located below the shaker line. We plan on improving performance in this area in the coming year by skirting the trees and insuring that the tree head height doesn't exceed the tunnel clearance of the harvester. In addition, we are working on a new adaptive shaker control for the harvester that we believe will improve harvesting efficiency while also reducing tree damage. This can be accomplished by dynamically measuring canopy size and adjusting the shaker to the profile of the tree. Next year's results should prove this out. In addition, we are also developing technologies for the fresh market harvest using robotics technologies. We are currently designing and testing a new prototype end effector for citrus harvesting. Plus working on several machine vision technologies described below.
Publications
- Type:
Conference Papers and Presentations
Status:
Accepted
Year Published:
2015
Citation:
Burks, T.F. 2015. Development of Agricultural Robots for PA Tasks in Orchard: Opportunities and Challenges. BARD workshop No. WS-102-2015: Innovations in Agricultural Robotics for Precision Agriculture, Held Concurrently with European Conference on Precision Agruculture, July 12-14, 2015, Tel Aviv, IS. (Invited Keynote Speaker)
- Type:
Journal Articles
Status:
Submitted
Year Published:
2015
Citation:
Mehta, S.S., W. MacKunis, T. F. Burks, 2015, Robust Visual Servo Control in the Presence of Fruit Motion for Robotic Citrus Harvesting, Computers and Electronics in Agriculture, Submitted
- Type:
Journal Articles
Status:
Submitted
Year Published:
2015
Citation:
Ni, Z. and T.F. Burks, 2015, 3D Dense Reconstruction of Plant or Tree Canopy based on Stereo Vision, International Journal of Agricultural and Biological Engineering, Submitted.
- Type:
Journal Articles
Status:
Submitted
Year Published:
2015
Citation:
Ni, Z. T.F. Burks, and W.S. Lee. 2015, 3D Reconstruction of Plant/Tree Canopy using Monocular and Binocular Vision. Journal of Imaging: Special Issue "Image Processing in Agriculture and Forestry", Submitted.
- Type:
Conference Papers and Presentations
Status:
Accepted
Year Published:
2015
Citation:
Burks, T.F., K. Morgan, F.M. Roka. 2015. Evaluation of Impacts of Mechanically Harvesting High-density Semi-dwarf Citrus on Tree Health and Yield, Florida State Horticultural Society Annual Meeting. Paper No. C-01.
- Type:
Conference Papers and Presentations
Status:
Accepted
Year Published:
2015
Citation:
Ni(g), Z., Burks, T., 2015, 3D Reconstruction of Citrus Tree Canopy using Multiple View Stereo Vision. 2015 ASABE Annual International Meeting, New Orleans, LA USA, July 26-July 29, 2015
|
Progress 03/05/14 to 09/30/14
Outputs
Target Audience: The target audience of this CRIS project is the Fresh and Processed citrus producers and related industries that will benefit from the development of new machinery systems that can assist in the improved economic health of the citrus Industry. Machinery technology for the production and harvesting of citrus grown in high density is an emerging area of citrus. Due to changes in the grove architecture new machinery is needed to produce and harvest. In this study we are focusing on two related but different production systesm, one for fresh and the other for processed fruit. Our primary focus is on developing a mass harvesting approach for processed and a selective robotic solution for fresh. There will be strategic technology development required in the fresh harvesting arena, while the processed fruit harvester will look more like harvesters for other commodities, such as olive. However tree size and grove architecture will demand new approaches versus even the closest comparable crop, such as olive. Ultimately it is hoped that these successful harvesting and production systems will be adopted by growers and used to help the regain an competitive edge in the global market place. Changes/Problems:
Nothing Reported
What opportunities for training and professional development has the project provided? Through the funding provided, Post doctorates and graduate students have furthered their research and academic careers as well as have opportunities to attend conferences to present research results. How have the results been disseminated to communities of interest? Results have been presented at conferences and published in conference proceedings and refereed journals. What do you plan to do during the next reporting period to accomplish the goals? Each objective will continue to see further scientific development, simulation and laboratory testing. Once all experiments have been completed the results will be published.
Impacts
What was accomplished under these goals?
The majority of our work has been directed toward the development of a new Over the ToP Citrus Harvester (OTPCH) for processed fruit, which has been specifically designed for high density semi-dwarf trees. This machine has unique features that will allow the machine to harvest trees as young as four years old, and adapt to the growing tree as it reaches maturity at approximately 8 ft tall and 8 ft wide. Proposed planting density for ACPHS trees of around 375 trees per acre yielding approximately 700 field boxes per acre may be harvested at speeds approaching 1.5 mph. Preliminary trials suggests that harvesting removal efficiencies of 94% or better can be achieved based on grapefruit harvesting trials and later confirmed on Valencia citrus. The primary research and development tasks 1) develop the OTP platform and power plant, 2) the shaking mechanism and mount design, 3) catch frame assembly, and 4) fruit conveyance approach. The University of Florida began working on an active catch frame design in 2009, while the platform development was started in 2012. Field trials began in 2013 and continue as we seek to improve the machine performance. A new platform design is currently being developed to improve ruggedness in normal grove conditions and enable the implementation of material handling systems. In addition, we are also developing technologies for the fresh market harvest using robotics technologies. We are currently designing and testing a new prototype end effector for citrus harvesting. Plus working on several machine vision technologies described below. 3D Reconstruction of Tree Canopy. Three dimensional (3D) reconstruction of tree canopy is the first important step in order to measure canopy geometry, such as, height, width, volume, and leaf cover area. In this research, binocular stereo vision was used to recover the 3D information of the canopy. Multiple images from multiple views around the target tree were taken. Structure-from-motion (SfM) method was employed to recover the camera matrix for each image, and the corresponding 3D coordinates of the feature points used to recover the camera matrix were also calculated. Through this method, a sparse projective reconstruction of the target was realized. After that, ball pivoting algorithm was used to do surface modeling to realize dense reconstruction. At last, this reconstruction was transformed to metric reconstruction through ground truth points, which were from camera calibration of binocular stereo cameras. Four experiments were taken, one for already known geometric box, and the other three are a croton plant with big leaves and salient features, a jalapeno pepper plant with median leaves, and a lemon tree with small leaves. A whole-view reconstruction of the target was realized. The comparison of the reconstructed box's size with the real box's size showed that this 3D reconstruction is in metric reconstruction. Integrated Estimation and Path Planning. One of the challenges in improving harvest efficiency and reducing cycle time is to determine a dynamically feasible task-space trajectory, free of obstacles and with minimum traverse time, to reach the estimated fruit position. Due to kinematics and nonlinear dynamics of the robot, the minimum-time geometric path of the robot end-effector is not necessarily the shortest path. Approaches considering a near-minimum-time path corresponding to the geodesic trajectories in the robot's inertia space may not be obstacle-free. We developed an integrated estimation and path planning approach to obtain a dynamically feasible, obstacle-free trajectory for the robot end-effector to reach the estimated fruit position. A cooperative multi-layered vision system is assumed to identify fruits to be harvested using standard image processing methods. Using the line-of-bearing measurements provided by the vision system, the Euclidean position of the fruits can be identified using a nonlinear estimator. An incremental sampling-based kinodynamic motion planning algorithm, Rapidly-exploring Random Tree (RRT), is used for the robot path planning. The integrated estimation and planning framework considers biased growth of the RRT towards a fruit using the posterior distribution of the fruit position obtained from an estimator. A path that minimizes an objective function comprised of path length and entropy of the distribution is optimal with respect to cycle time and harvest efficiency. Smoothing techniques can be applied to the optimal path to obtain a desired trajectory (as shown in Figure 1). The computed torque controller with a robustifying feedback term developed is used to track the desired trajectory. Numerical simulations were carried out to validate the proposed estimation, path planning, and control approach for robotic citrus harvesting. Fruit Position Estimation. One of the first steps in automated fruit harvesting is determining the location of fruit on a tree. Computer vision technology can be leveraged to provide the position of the fruit relative to the robotic arms used to pick the fruit. This issue motivates the use of another common reconstruction technique known as structure from motion (SfM), where multiple views are generated from a single moving camera. In this setup, the amount of camera motion needed to accurately estimate depth can be dynamically tuned for the current features of interest. One caveat with this class of algorithms is that they expect persistent visibility of the feature of interest. This is unrealistic in the fruit harvesting problem, where foliage may temporarily obstruct visibility. In the work summarized here, we analyze the robustness of a class of SfM techniques to intermittent observability, and provide conditions to guarantee accurate position estimation of the fruit. In addition, we provide a method for estimating the position of the fruit when they are not visible to the camera. An observer was used to estimate object position during the periods in which the object was in view, and simulation parameters were set as identical to those used in previous work in order to compare our results to the case when the object is persistently in view. A switching signal with randomly generated dwell times was used to signify duration spent with and without the object in view. Efforts to date has focused on developing the mathematical analysis to show the robustness of a class of SfM algorithms to intermittent measurements, provided a set of dwell time conditions are met and a motion model of the moving fruit is known. Future work will investigate the effectiveness of different motion models for estimating fruit velocity, and will explore adaptive methods for learning the model online.
Publications
- Type:
Journal Articles
Status:
Published
Year Published:
2014
Citation:
Mehta, S.S., T. F. Burks, 2014. Vision-based control of robotic manipulator for citrus harvesting. Computers and
Electronics in Agriculture, Vol. 102, (2014), pp. 146-148.
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2014
Citation:
Mehta, S.S., T. F. Burks, 2014. "Path Planning and Robust Control for Robotic Citrus Harvesting", Robotics and
Associated High-Technologies and Equipment for Agriculture and Forestry (RHEA), Madrid, Spain, 2014, pp. 467-476.
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2014
Citation:
Ni(g), Z., Burks, T., Lee, W.S., 2014, 3D Reconstruction of Small Plant From Multiple Views. 2014 Montreal, QC,
Canada, July 13-July 16, 2014, Paper No. 141893190.
- Type:
Conference Papers and Presentations
Status:
Awaiting Publication
Year Published:
2014
Citation:
Burks, T.F., N. Aldisory(g, W.S. Castle, F.M. Roka. 2014. Development of Harvesting Equipment for Higher Density Citrus Grove Architectures, Florida State Horticultural Society Annual Meeting. Paper No. C-16.
|
Progress 01/01/12 to 09/30/12
Outputs
Target Audience: Florida citrus growers and the agricultural engineering community, as well as the fruit and vegetable production growers. Changes/Problems: The primary problem we have encountered is with regard to canopy excitation. It is a challenge to uniformly excite the canopy, without damaging the tree at major points of contact between the shaker fingers and the primary canopy scaffolds. We are continuing to improve this portion of the machine performance. What opportunities for training and professional development has the project provided? There are currently five graduate students (4 PhD and 1 MS)working on various aspects of this project. These students are taking classes and participating in varioius aspects fo systems development. This shouldprovide them with significant practical training that will come in handy in future careers as engineers. How have the results been disseminated to communities of interest? Refereed journal articles and conference presentations and articles. What do you plan to do during the next reporting period to accomplish the goals? We have been making revisions to the prototype with anticipation of improving harvesting efficiency performance. The current improvements have included mostly a redesign on the shaker fingers to improve the impact on the tree canopy in hopes of improving harvesting efficiency. We also plan to increase engine power in the spring and add some form of suspension to improve vehicle performance. This improvements will be coupled with further field tests in a hope to improve harvester performance.
Impacts
What was accomplished under these goals?
We have built a prototype harverster for over the top citrus production and harvesting in high density citrus groves. This machine has been tested in an experimental high density grapefruit grove to evaluate the harvesters effectiveness in exciting the tree canopy to dislodge the fruit for harvesting. Field trials were conducted where removal efficiency was evaluated and also in trials where canopy accelerations were monitored to determine the uniformity of tree canopy excitation. It was observed that the central portion of the tree was not being effectively harvested and this was confirmed by low acceleration data values. These experimental results have lead to modification in the shaker design which will be tested during the upcoming harvesting season. We have also made improvements in the hydualic power delivery systems to improve power delived to the wheels for forward propulsion.
Publications
- Type:
Journal Articles
Status:
Accepted
Year Published:
20013
Citation:
Niphadkar(g), N., Burks, T. F., Qin, J., and Ritenour, M. A. 2013. Estimation of Citrus Canker Lesion Detection Size Limit using Hyperspectral Reflectance Imaging. International Journal of Agricultural & Biological Engineers Vol. 6 No. 3 (2013)41-51.
|
|