Progress 09/01/14 to 08/31/18
Outputs
Target Audience:The outcomes of this project were communicated with researchers and industry representatives from around the world through publication and presentation in nine local, national, and international professional conferences. Project outcomes, including two full-scale research prototypes of the developed bin-managing robots, were also displayed and demonstrated to tree fruit growers, agricultural equipment manufacturers, technical service providers that were present at seven Washington State University (WSU) Center for Precision and Automated Agricultural Systems (CPAAS) Tech-Expo and Field Day events, and local/regional agricultural technology trade shows. In addition, the latest research prototype, along with poster presentations, have been displayed and introduced to local political leaders and (WSU) university leadership, 20+ US Land Grant University agriculture college Deans/Associate Deans and 20+ US industry leaders, as well as researchers from USDA multi-state projects, and from different part of the world in four special events. Project accomplishments were also released to local news outlets, which in turn disseminated the information to the general public. Changes/Problems:
Nothing Reported
What opportunities for training and professional development has the project provided?Two MS students and two PhD students were actively involved in this project, one each graduated (a M.S. student from Oregon State University and one Ph.D. student from Washington State University) in the past year three years. The PIs and students interacted frequently to provide guidance to the students, both in person and over video conferences. Students and scholars carried out day-to-day research activities including data collection and analysis. Students were also supervised and actively participated in research paper writing, attended international conference to present their research findings, and involved in technology demonstration to the industry. How have the results been disseminated to communities of interest?The bin managing robot research prototype has been tested in commercial orchards in the state of Washington in 2016 and 2017 harvest seasons, which allowed many growers and orchard workers to observe the use of the robotic machinery, as well as the efficiency improvement of using the machinery, in actual working sites. The prototype and field test results have also been displayed and demonstrated at the 2014-2017 WSU CPAAS Tech Expo/Field Day events in Prosser WA annually), 2016 PNW Precision Farming Trade Shows and 2017 Washington Wine Grade Conference both in Kennewick WA which attracted local growers from both Oregon and Washington, equipment manufacturers, technology and service providers, researchers, and media. The research findings were presented via posters and/or talks in five additional venues. The project team has published three journal articles on this work to three peer-reviewed professional journals, with an additional one being submitted for consideration for publication. In addition, the team has presented multiple papers in various national and international professional conferences and presented poster presentations at local tree fruit industry annual conferences. We have also released our research accomplishments to local news outlets, which then disseminated the information to the general public. What do you plan to do during the next reporting period to accomplish the goals?
Nothing Reported
Impacts
What was accomplished under these goals?
A self-propelled autonomous in-orchard bin managing robot research prototype has been developed, fabricated, and tested in both laboratory and orchard environments. This prototype consists of a passive mechanical suspension system, a four-wheel independent steering (4WIS) system, a 4WIS coordinate control system, an autonomy control system, and a computer control interface. A GPS-based navigation system and human/bin detection system was developed and integrated into this research prototype. The navigation control system, supported by an intelligent steering strategy selection algorithm, could switch among one of the four predesigned steering modes (i.e., Ackermann steering, active-front-and-rear steering, crab steering, and spinning) to steer the prototype accurately following the desired path under different situations, such as entering an aisle, steering back to centerline, and maneuvering the bin. A multi-robot simulation with bin-managing robots and human pickers has been developed to model bin management under real-world conditions. The multi-robot coordination methods developed in this work reduce the time to collect bins in a real-world orchard simulation by up to 30%. It is expected that the completion of these activities will lead to a successful development of an intelligent in-orchard bin-managing system. When commercialized, the technology can reduce the labor use in bin managing during harvest season while improve the bin-managing efficiency. Major activities completed / experiments conducted A self-propelled bin-managing robot research platform has been designed and fabricated, with situation awareness system and a GPS-based navigation control system (supported by an automated steering controller) integrated on the platform. This platform has been tested and used to conduct the field study of robotic operations for in-orchard bin management during harvest. The platform has successfully run autonomously for over 40 kilometers. Simulation developed of typical orchard conditions in the real world. Multiple bin-managing robots were situated in the orchard along with human pickers. Previously collected picking rates for human pickers have been integrated into a multi-robot orchard simulation. Qualitative observations of human pickers have been integrated into the simulator. The humans move around the orchard realistically while interacting with the bin-managing robots. A study on effective Long-term Autonomy was conducted by the OSU group. The bin-managing robot will need to operate in the orchard autonomously for long periods of time. This requires a robust system for navigating through the environment and avoiding obstacles and humans in the orchard. The team has collaboratively developed a perception system to detect and avoid humans in the orchard. As the ultimate goal of this project was to autonomously control multiple robotic vehicles managing bins in the orchard environment, we have developed a scalable multi-robot coordination method applicable to the orchard environment. The OSU group has developed and tested (both in simulation and on hardware in the orchard assisted by the WSU group) auction-based methods to improve the efficiency of multiple bin-managing robots operating in the orchard. As the bin-managing robots will need to interact with humans working in the orchard and avoid them while they are picking the fruit, the OSU group has developed and validated detection algorithms to identify humans and efficient planning methods for avoiding human workers with the robot during harvest. A real-time stochastic optimization technique has been developed to maximize the speed of the robot and minimize the effect of disturbances during planed execution. Perception modalities for detecting humans, bins, and tree-rows in the orchard have been evaluated. Laser-based detection uses the DBSCAN algorithm to cluster data points, fits an ellipse to each cluster, and detects humans and tree rows based on a threshold on ellipse size. A similar algorithm using line detection was developed to detect bins. Algorithms were tested on the bin dog platform in the orchard environment. Adaptive replanning and bin priority reasoning were integrated into the multi-robot coordination algorithms. These improvements were tested in realistic simulations. Developed and tested the ROS (Robot Operating System) code to allow the low-level components to communicate, serving as the "plumbing" for the robot's various subsystems. Implemented and validated localization algorithms for both GPS-enabled and GPS-denied settings on both computer simulation and field tests using physical robotic bin-dog research platform. Both tests included the use of different steering modes, showing that reinforcement learning could be autonomously used to switch between four possible steering modes with near-optimal results. Key outcomes or other accomplishments realized Designed, implemented, and tested autonomous laser-based control system for a robot in GPS-denied environments with rough ground terrain. The path tracking performance and spatial requirement of an operation are closely related with longitudinal speed, control gain, and steering mode. Those factors need to be carefully selected depending on the requirement of the operation. Proposed, developed, and validated a scalable coordination between autonomous robots, which was implemented for fruit bin-carriers to increase the effectiveness of fruit harvesting, specifically to assist human workers to transport fruit bins within the orchard. To achieve a more optimal system result, robots use an auction-based approach to coordinate their decisions to pick up bins in the orchard. The effectiveness of the approach was verified in simulation and tested in the field using a physical robot. Determined that human pickers could be successfully integrated into a multi-robot simulation environment. The bin-managing robots move at sufficient speed to supply the human pickers with bins at a rate matching how quickly they were filled. Human, bin, and tree row detection algorithms using a low-cost laser scanner developed and tested in the orchard environment. Perception modalities for detecting humans, bins, and tree-rows in the orchard have been evaluated. Laser-based detection uses the DBSCAN algorithm to cluster data points, fits an ellipse to each cluster, and detects humans and tree rows based on a threshold on ellipse size. A similar algorithm that uses line detection was developed to detect bins. Algorithms were tested on the bin dog platform in the orchard environment. Developed and validated a real-time optimization algorithm to improve the time and energy efficiency of mobile robots operating in environments with uncertainty. Adaptive replanning and reasoning about which bins are prioritized over others was integrated into the algorithm. The algorithm was tested in simulation and showed improvements over the state of the art. Field trials were conducted to test the multi-robot coordination algorithms with one real bin dog platform and several simulated bin dog platforms. Risk-Aware Graph Search software was made publicly available (https://github.com/osurdml/RAGS)to provide safe and efficient trajectory optimization for mobile robotic vehicles.
Publications
- Type:
Journal Articles
Status:
Published
Year Published:
2018
Citation:
Ye, Y., L. He, Z. Wang, D. Jones, G. Hollinger, M. Taylor, and Q. Zhang, (2018). Orchard maneuvering strategy for a robotic bin-handling machine bin-dog a self-propelled platform for bin management in orchards. Biosystems Engineering, 169: 85-103.
- Type:
Journal Articles
Status:
Published
Year Published:
2017
Citation:
Ye, Y., Z. Wang, D. Jones, L. He, M. Taylor, G. Hollinger, and Q. Zhang, (2017). Bin-dog: a robotic platform for bin management in orchards. Robotics, 6(2): Article 12 (17pp).
- Type:
Journal Articles
Status:
Published
Year Published:
2017
Citation:
Jones, D., and G. Hollinger, (2017). Planning energy-efficient trajectories in strong disturbances. Robotics and Automation Letters, 2(4): 2080-2087.
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2016
Citation:
Jones, D., and G. Hollinger, (2016). Real-time stochastic optimization for energy-efficient trajectories. In Proc. of the Robotics: Science and Systems Workshop on Robot-Environment Interaction for Perception and Manipulation (RSS), June 19, Ann Arbor, MI.
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2016
Citation:
Ye, Y. L. He, and Q. Zhang, (2016). Steering control strategies for a four-wheel-independent-steering bin managing robot. In IFAC-PaperOnLine, 49(16): 39-44 (The 5th IFAC Conference on Sensing, Control and Automation for Agriculture), August 14-17, Seattle, WA.
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2016
Citation:
Ye, Y., L. He, and Q. Zhang, (2016). A robotic platform bin-dog for bin management in orchard environment. Presented at American Society of Agricultural and Biological Engineers (ASABE) Annual International Meeting, ASABE Paper No. 162462088, July 17-20, Orlando, FL.
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2016
Citation:
Ye, Y., L. He, and Q. Zhang, (2016). Steering strategy selection of a robotic platform in orchard environment. In Proc. of the 13th International Conference on Precision Agriculture(ICPA). July 31-August 3, St. Louis, MO.
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2015
Citation:
Zhang, Y., Y. Ye, Z. Wang, M. Taylor, G. Hollinger, and Q. Zhang, (2015). Intelligent in-orchard bin-managing system for tree fruit production. in Proc. IEEE International Conference on Robotics and Automation Workshop on Robotics in Agriculture (ICRA), Seattle, WA, May 2015.
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2014
Citation:
Ye, Y., L. Yu, and Q. Zhang, (2014). Wheel slip control on an intelligent bin-dog system on rough terrain. The XVIIIth CIGR World Congress, September 16-19, Beijing, China.
- Type:
Theses/Dissertations
Status:
Published
Year Published:
2016
Citation:
Ye, Y., (2016). A Maneuverability Study on a Wheeled Bin Management Robot in Tree Fruit Orchard Environments. Ph.D. Dissertation, April 2016, Washington State University.
- Type:
Theses/Dissertations
Status:
Published
Year Published:
2015
Citation:
Zhang, Y. (2015). Multi-Robot Coordination: Applications in Orchard Bin Management and Informative Path Planning. M.S. Thesis, September 2015, Oregon State University.
- Type:
Journal Articles
Status:
Submitted
Year Published:
2018
Citation:
Jones, D., Z. Wang, Y. Ye, L. He, G. Hollinger, M. Taylor, and Q. Zhang, (2018). Multi-robot coordination for autonomous in-orchard bin-managing robots. Robotics and Automation Letters
- Type:
Theses/Dissertations
Status:
Awaiting Publication
Year Published:
2018
Citation:
Jones, D. (2018). Planning Energy Efficient Trajectories in Strong Disturbances. Ph.D. Dissertation, June 2020, Oregon State University.
|
Progress 09/01/16 to 08/31/17
Outputs
Target Audience:The outcomes of this project were communicated with researchers and industry representatives from around the world through publication and presentation in six local, national, and international meetings and conferences. Project outcomes were also demonstrated and discussed with local news outlets, which disseminated the information to general public. Changes/Problems:
Nothing Reported
What opportunities for training and professional development has the project provided?Two MS students and two PhD students were actively involved in this project, one each graduated (a M.S. student from Oregon State University and one Ph.D. student from Washington State University) in the past year three years. The PIs and students interacted frequently to provide guidance to the students, both in person and over video conferences. Students and scholars carried out day-to-day research activities including data collection and analysis. Students were also supervised and actively participated in research paper writing, attended international conference to present their research findings, and involved in technology demonstration to the industry. How have the results been disseminated to communities of interest?The bin managing robot research prototype has been tested in commercial orchards in the state of Washington in 2016 harvest season. The prototype and field test results have also been displayed and demonstrated at the 2017 WSU CPAAS Tech Expo in Prosser WA in July 2017, which attracted local growers from both Oregon and Washington, equipment manufacturers, technology and service providers, researchers, and the medias. The research findings were presented via posters and/or talk in five additional venues. The project team has published one journal article on this work to the Robotics journal, an open access journal with wide dissemination. Two manuscripts have been submitted to journals for consideration of publication, and a couple more under writing. What do you plan to do during the next reporting period to accomplish the goals? Final integration between the different mechanical and computational components will be completed and tested. Field tests will be conducted to test the multi-robot coordination algorithms with one actual bin dog platform and several simulated bin dog platforms. More field tests will be continued to validate all functions of the bin managing robot system in real commercial orchards.
Impacts
What was accomplished under these goals?
A self-propelled bin managing robot research prototype has been developed, fabricated and tested in both laboratory and orchard environments. This prototype consists of a passive mechanical suspension system, a four-wheel independent steering (4WIS) system, a 4WIS coordinate control system and a computer control interface. A GPS-based navigation system and human/bin detection system was developed and integrated to this research prototype. This navigation control system, supported by an intelligent steering strategy selection algorithm, could switch among one of the four predesigned steering modes, including Ackermann steering, active-front-and-rear steering, crab steering and spinning, to steer the prototype accurately following the desired path under different situations, such as entering an aisle, steering back to centerline, and maneuvering the bin. A multi-robot simulation with bin-managing robots and human pickers has been developed to model bin management under real-world conditions. The multi-robot coordination methods developed in this work reduce the time to collect bins in a real-world orchard simulation by up to 30%. It is expected that the completion of these activities will lead to a successful development of an intelligent in-orchard bin-managing system. When commercialized, the technology can reduce the labor use in bin managing during harvest season while improve the bin-managing efficiency. Major activities completed / experiments conducted: A self-propelled bin managing robot research platform has been designed and fabricated, with situation awareness system and a GPS-based navigation control system (supported by an automated steering controller) being integrated on the platform. This platform has been tested and used to conduct the field study of robotic operations for in-orchard bin management during harvesting. Simulation developed of typical orchard conditions in the real world. Multiple bin-managing robots situated in the orchard along with human pickers. Previously collected picking rates for human pickers have been integrated into a multi-robot orchard simulation. Qualitative observations of human pickers have been integrated into the simulator. The humans move around the orchard realistically while interacting with the bin-managing robots. A real-time stochastic optimization technique has been developed to maximize the speed of the robot and minimize the effect of disturbances during path planning execution. Perception modalities for detecting humans, bins, and tree rows in the orchard have been evaluated. Laser-based detection utilizes the DBSCAN algorithm to cluster data points, fits an ellipse to each cluster, and detects humans and tree rows based on a threshold on ellipse size. A similar algorithm utilizes line detection was developed to detect bins. Algorithms tested on the bin dog platform in the orchard environment. Adaptive replanning and bin priority reasoning were integrated into the multi-robot coordination algorithms. These improvements were tested in realistic simulations. Data collected: Path tracking data were collected to study the influence of longitudinal speed and control gain on (i) path tracking accuracy when tracking curvy paths using both Ackermann and AFRS steering modes; and (ii) spatial requirement when completing tasks including merging and cornering using all four steering modes. Benchmarked performance data of different control modes were collected in laboratory and field conditions. Benchmarked autonomous navigation capabilities in field conditions were collected with both laser and differential GPS localization. Collected data of real-time disturbances that may affect robot trajectories for use with real-time stochastic optimization. Data collected on human and bin detection in the orchard environment to test the laser-based perception algorithms. Summary statistics and discussion of results: The research platform had a high path tracking accuracy performance at low longitudinal speed, with a root mean square error (RMSE) of 0.07 m and absolute mean lateral error (ABSE) of 0.06±0.03 m when tracking a Leminiscate curve at a longitudinal speed of 0.40 m·s-1 under Ackermann steering mode and 0.04 m and 0.03±0.02 under AFRS mode. Field tests proved the steering strategy selection algorithm could effectively generate appropriate steering strategy for the situation to guide the prototype following a shortest path to complete designated tasks including steering into an aisle from headland, steering back to centerline of an aisle, and loading a bin on an aisle in real orchard environment. Completed field tests for simplified bin management, results demonstrated the feasibility of autonomously controlling the robot in GPS-denied environments -- robot was able to successfully enter, exit, and traverse orchard rows without making contact with static obstacles. The real-time trajectory optimization was tested in simulation environments. The preliminary results showed approximately a 20% gain in energy efficiency using the developed methods. Adaptive replanning and priority reasoning improved the number of bins collected by the multi-robot optimization in the simulated orchard environment by approximately 10%. Key outcomes or other accomplishments realized: Designed, implemented, and tested autonomous laser-based control system for robot in GPS-denied environments with rough ground terrain. The path tracking performance and spatial requirement of an operation are closely related with longitudinal speed, control gain and steering mode. Those factors need to be carefully selected depending on the requirement of the operation. We proposed a scalable coordination between autonomous robots, which was implemented for fruit bin-carriers to increase the effectiveness of fruit harvesting, specifically to assist human workers to transport fruit bins within the orchard. To achieve a more optimal system result, robots use an auction-based approach to coordinate their decisions to pick up bins in the orchard. The effectiveness of the approach was verified in simulation. Determined that human pickers could be successfully integrated into a multi-robot simulation environment. The bin-managing robots move at sufficient speed to supply the human pickers with bins at a rate matching how quickly they were filled. Human, bin, and tree row detection algorithms using a low-cost laser scanner developed and tested in the orchard environment. Developed a real-time optimization algorithm to improve the time and energy efficiency of mobile robots operating in environments with uncertainty. Adaptive replanning and reasoning about which bins are prioritized over others integrated into algorithm. Algorithm tested in simulation and showed improvements over the state of the art.
Publications
- Type:
Journal Articles
Status:
Published
Year Published:
2017
Citation:
Ye, Y., Z. Wang, D. Jones, L. He, M. Taylor, G. Hollinger, and Q. Zhang, (2017). Bin-dog: A robotic platform for bin management in orchards, Robotics, vol. 6, no. 2, article 12.
- Type:
Journal Articles
Status:
Under Review
Year Published:
2017
Citation:
Jones, D. and G. Hollinger, (2017). Planning energy-efficient trajectories in strong disturbances, Robotics and Automation Letters.
- Type:
Journal Articles
Status:
Under Review
Year Published:
2017
Citation:
Ye, Y., L. He, Z. Wang, D. Jones, G. Hollinger, M. Taylor, and Q. Zhang, (2017). Orchard maneuvering strategy for a robotic bin-handling machine bin-dog a self-propelled platform for bin management in orchards . Biosystems Engineering.
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2016
Citation:
Jones, D. and G. Hollinger, (2016). Real-time stochastic optimization for energy-efficient trajectories . In Proc. of the Robotics: Science and Systems Workshop on Robot-Environment Interaction for Perception and Manipulation (RSS), June 19, Ann Arbor, MI.
|
Progress 09/01/15 to 08/31/16
Outputs
Target Audience:The outcomes of this project were communicated with researchers and industry representatives from around the world through publication and presentation in six local, national, and international meetings and conferences. Project outcomes were also demonstrated and discussed with local news outlets, which disseminated the information to general public. Changes/Problems:
Nothing Reported
What opportunities for training and professional development has the project provided?With two MS students and two PhD students actively involved in this project, one each graduated (a M.S. student from Oregon State University and one Ph.D. student from Washington State University) in the past year. The PIs and students interacted frequently to provide guidance to the students, both in person and over video conferences. Students and scholars carried out day-to-day research activities including data collection and analysis. Students were also supervised and actively participated in research paper writing, attended international conference to present their research findings, and involved in technology demonstration to the industry. How have the results been disseminated to communities of interest?The bin managing robot research prototype has been tested in commercial orchards in the state of Washington. The prototype and preliminary results have also been displayed and demonstrated at the 2016 Precision Farming Expo in Kennewick WA in January 2016, which attracted local growers, equipment manufacturers, technology and service providers, researchers, and medias. The team has presented their research outcomes at the Annual International Meetings of American Society of Agricultural and Biological Engineers (ASABE) in New Orleans, LA on July 27, 2015 and in Orlando, FL on July 18, 2016; at the Robotics: Science and Systems Workshop on Robot-Environment Interaction for Perception and Manipulation in Ann Arbor, MI on June 19, 2016; at the 13th International Conference on Precision Agriculture in St. Louis, MO on July 31, 2016; and the 5th IFAC Conference on Sensing, Control and Automation for Agriculture in Seattle on August 15, 2016. MS Student Yawei Zhang presented his MS thesis on September 16, 2015 to the Oregon State University research community. PhD Student Yunxiang Ye presented his PhD dissertation on April 22, 2016 to the Washington State University research community. Their thesis and dissertation are publicly available on the OSU or WSU publication dissemination website. Their works were presented via a poster and/or talk in five additional venues. The project team has submitted a journal article on this work that is currently under review by the Journal of Field Robotics, a top-tier venue for this type of research. What do you plan to do during the next reporting period to accomplish the goals? Additional situation awareness sensors will be integrated to the research platform. It will support the complete automated bin managing process, and the integrated system will be tested in commercial orchards. A sensor-fusion based navigation system, consisting of a GPS-based navigation system and laser-based navigation system, will be developed for the research prototype to enable reliable and accurate guidance of the prototype while traveling in tree rows, and then be tested in commercial orchard environments. The intelligent steering strategy selection algorithm will be integrated into the navigation system to dynamically generate an optimized steering strategy for the situation in real time. Energy efficient optimization algorithms will be implemented on the vehicle and tested in the orchard environment. Real-time optimization algorithms will be improved to properly trade off between observing the environment and dealing with modeled disturbances. Perception algorithms will be examined to detect humans in the orchard environment with sufficient fidelity to avoid collisions with them. The vehicle will predict the human's movement and avoid them appropriately. Final integration between the different mechanical and computational components will be completed and tested. Field tests will be continued to validate all functions of the bin managing robot system in real commercial orchards.
Impacts
What was accomplished under these goals?
A self-propelled bin managing robot research prototype has been developed, fabricated and tested in both laboratory and orchard environments. This prototype consists of a passive mechanical suspension system, a four-wheel independent steering (4WIS) system, a 4WIS coordinate control system and a computer control interface. A GPS-based navigation system was developed and integrated to this research prototype. This navigation control system, supported by an intelligent steering strategy selection algorithm, could switch among one of the four predesigned steering modes, including Ackermann steering, active-front-and-rear steering, crab steering and spinning, to steer the prototype accurately following the desired path under different situations, such as entering an aisle, steering back to centerline, and maneuvering the bin. A multi-robot simulation with bin-managing robots and human pickers has been developed to model bin management under real-world conditions. The multi-robot coordination methods developed in this work reduce the time to collect bins in a real-world orchard simulation by up to 30%. It is expected that the completion of these activities will lead to a successful development of an intelligent in-orchard bin-managing system. When commercialized, the technology can reduce the labor use in bin managing during harvest season while improving bin-managing efficiency. 1. Major activities completed / experiments conducted; A self-propelled bin managing robot research prototype has been developed and fabricated, with situation awareness sensors installed on the prototype. A GPS-based navigation control system, as well as an intelligent steering strategy selection algorithm, was developed, integrated, and successfully implemented on the research prototype. Field tests were conducted to study the influence of longitudinal speed and control gain (look-ahead distance) on (i) path tracking accuracy when tracking curvy paths; and (ii) spatial requirement when completing various tasks including merging and cornering using different steering modes. Simulation developed of typical orchard conditions in the real world including multiple bin-managing robots situated in the orchard along with human pickers. Previously collected picking rates for human pickers have been integrated into a multi-robot orchard simulation. Qualitative observations of human pickers have been integrated into the simulator. The humans move around the orchard realistically while interacting with the bin-managing robots. A real-time stochastic optimization technique has been developed to maximize the speed of the robot and minimize the effect of disturbances during path planning execution. Perception modalities for detecting humans in the orchard have been evaluated. Stereo vision and laser range scanners were being examined as potential solutions. 2. Data collected; Path tracking data were collected to study the influence of longitudinal speed and control gain on (i) path tracking accuracy when tracking curvy paths using both Ackermann and AFRS steering modes; and (ii) spatial requirement when completing tasks including merging and cornering using all four steering modes. Path tracking data were collected to evaluate the performance of the steering strategy selection algorithm in natural orchard environment. Path tracking data were also collected in the simplified bin management test to analyze the automatic orchard traversal performance of the research prototype. Benchmarked performance data of different control modes were collected in laboratory and field conditions. Benchmarked autonomous navigation capabilities in field conditions were collected with both laser and differential GPS localization. Collected data of real-time disturbances that may affect robot trajectories for use with real-time stochastic optimization. 3. Summary statistics and discussion of results The GPS-based navigation control system has been successfully integrated and implemented on the research prototype to track curvy paths and complete tasks using all four steering modes (Ackermann, AFRS, crab steering and spinning). The research prototype had a high path tracking accuracy performance at low longitudinal speed, with a root mean square error (RMSE) of 0.07 m and absolute mean lateral error (ABSE) of 0.06±0.03 m when tracking a Leminiscate curve at a longitudinal speed of 0.40 m·s-1 under Ackermann steering mode and 0.04 m and 0.03±0.02 under AFRS mode. For quickly merging to a desired path at a longitudinal speed of 0.60 m·s-1, it required having a 2.8 m operation space under crab steering mode, which was 12.5% smaller than Ackermann or AFRS steering modes. Performing cornering task at a longitudinal speed of 0.60 m·s-1 using spinning steering mode required a 3.0 m (width) and 1.2 m (length) space for spinning the prototype which was greatly reduced compared to Ackermann (5.3×10.0 m) and AFRS (4.6×6.5 m) steering modes. Field tests proved the steering strategy selection algorithm could effectively generate appropriate steering strategy for the situation to guide the prototype following a shortest path to complete designated tasks including steering into an aisle from headland, steering back to centerline of an aisle, and loading a bin on an aisle in real orchard environment. Completed the test for simplified bin management, results demonstrated the feasibility of autonomously controlling the robot in GPS-denied environments -- robot was able to successfully enter, exit, and traverse orchard rows without making contact with static obstacles. The real-time trajectory optimization was tested in simulation environments. The preliminary results showed approximately a 20% gain in energy efficiency using the developed methods. 4. Key outcomes or other accomplishments realized. To steer the robotic prototype with the four-wheel independent steering system, it is critical to guarantee the accuracies of angle and speed control of all wheels. Any inaccurate angle and speed control will considerably increase driving resistance and decrease system performance. The four steering modes greatly improve the efficiency and effectiveness of maneuvering the research platform in orchard environment. Those steering modes are beneficial to achieve different bin-managing objectives such as bin engagement, straight driving between tree rows and lane entering. Designed, implemented, and tested autonomous laser-based control system for robot in GPS-denied environments with rough ground terrain. The path tracking performance and spatial requirement of an operation are closely related with longitudinal speed, control gain and steering mode. Those factors need to be carefully selected depending on the requirement of the operation. We proposed a scalable coordination between autonomous robots, which was implemented for fruit bin-carriers to increase the effectiveness of fruit harvesting, specifically to assist human workers to transport fruit bins within the orchard. To achieve a more optimal system result, robots use an action based approach to coordinate their decisions to pick up bins in the orchard. The effectiveness of the approach was verified in simulation. Determined that human pickers could be successfully integrated into a multi-robot simulation environment. The bin-managing robots move at sufficient speed to supply the human pickers with bins at a rate matching how quickly they were filled. Developed a real-time optimization algorithm to improve the time and energy efficiency of mobile robots operating in environments with uncertainty. Algorithm tested in simulation and showed improvements over the state of the art.
Publications
- Type:
Theses/Dissertations
Status:
Published
Year Published:
2016
Citation:
Ye, Y., 2016. A maneuverability study on a wheeled bin management robot in tree fruit orchard environments. Ph.D. Dissertation, April 2016, Washington State University.
- Type:
Theses/Dissertations
Status:
Published
Year Published:
2015
Citation:
Zhang, Y., 2015. Multi-robot coordination: applications in orchard bin management and informative path planning. M.S. Thesis, September 2015, Oregon State University.
- Type:
Journal Articles
Status:
Under Review
Year Published:
2016
Citation:
Ye, Y., Z. Wang, Y. Zhang, D. Jones, L. He, G. Hollinger, M. Taylor, W. Smart, Q. Zhang, 2016. Bin-Dog: A Self-Propelled Platform for Bin Management in Orchards. Journal of Field Robotics,
- Type:
Conference Papers and Presentations
Status:
Accepted
Year Published:
2016
Citation:
Ye, Y. L. He, Q. Zhang. 2016. Steering control strategies for a four-wheel-independent-steering bin managing robot. Paper and presentation at the 5th IFAC Conference on Sensing, Control and Automation for Agriculture, August 14-17, Seattle, WA.
- Type:
Conference Papers and Presentations
Status:
Accepted
Year Published:
2016
Citation:
Ye, Y., L. He, Q. Zhang. 2016. Steering strategy selection of a robotic platform in orchard environment. Presentation at the 13th International Conference on Precision Agriculture, July 31-August 3, St. Louis, MO
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2016
Citation:
Ye, Y., L. He, Q. Zhang. 2016. A robotic platform bin-dog for bin management in orchard environment. Presentation at American Society of Agricultural and Biological Engineers (ASABE) Annual International Meeting, ASABE Paper No. 162462088, July 17-20, Orlando, FL.
- Type:
Conference Papers and Presentations
Status:
Other
Year Published:
2015
Citation:
Zhang, Y., Y. Ye, Z. Wang, M. Taylor, G. Hollinger, and Q. Zhang, 2015. Intelligent In-Orchard Bin-Managing System for Tree Fruit Production, Poster and Presentation at the 2015 National Robotics Initiative (NRI) PI Meeting, November 5-6, Washington, DC
- Type:
Journal Articles
Status:
Published
Year Published:
2016
Citation:
Jones, D. and G. Hollinger, 2016. Real-time stochastic optimization for energy-efficient trajectories. In Proc. of the Robotics: Science and Systems Workshop on Robot-Environment Interaction for Perception and Manipulation (RSS), June 19, Ann Arbor, MI.
- Type:
Conference Papers and Presentations
Status:
Other
Year Published:
2015
Citation:
Zhang, Y., Y. Ye, Z. Wang, M. Taylor, G. Hollinger, and Q. Zhang, 2015. Intelligent in-orchard bin-managing system for tree fruit production, Poster and Presentation at CPAAS Open House & Ag Tech Day, September 17, Prosser, WA.
|
Progress 09/01/14 to 08/31/15
Outputs
Target Audience:The outcome of this project was communicated with researchers and industry representatives from around the world through publication and presentation in local, national, and international meetings and conferences. Project outcomes were also demonstrated and discussed with local news outlets, which disseminated the information to general public. Changes/Problems:
Nothing Reported
What opportunities for training and professional development has the project provided?One MS student and two PhD students were actively involved in this project. The PIs and students interacted frequently to provide guidance to the students, both in person and over video conferences. Students and scholars carried out day-to-day research activities including data collection and analysis. Students were also supervised for research paper writing, presentation, and publications. How have the results been disseminated to communities of interest?The bin managing robot research platform has been tested in commercial orchards in the state of Washington. The platform and preliminary results have also been displayed and demonstrated at WSU-CPAAS Open House and Technology Expo, which attracted local growers, researchers, and medias. The team has presented this work in the CIGR Automation Technology for Off-road Equipment (ATOE) Conference in Beijing, China on September 18, 2014, the IEEE International Conference on Robotics and Automation (ICRA) Workshop on Robotics in Agriculture in Seattle, WA on May 30, 2015, and the Annual International Meeting of American Society of Agricultural and Biological Engineers (ASABE) in New Orleans, LA on July 27, 2015. MS Student Yawei Zhang also presented his MS thesis on September 16, 2015 to the Oregon State University research community. His thesis is publicly available on the OSU publication dissemination website. The work was presented via a poster and/or talk in two additional venues. What do you plan to do during the next reporting period to accomplish the goals?• Auto-navigation and steering control systems will be integrated to the research platform, and the integrated system will then be tested both in laboratory and field environments. • Baseline tests of bin managing performance under different steering modes in different working environments will be evaluated, and a high-level steering control algorithm will be developed for best fitting the working environments will be developed based on the obtained baseline data. • High-level robot navigation algorithms will be developed to navigate the bin-managing robots among human pickers in working sites, and both simulations and field tests will be conducted to confirm the functionality and performance. • Control algorithms for accurately positioning the robot, moving the robot, and handling apple bins in a GPS-denied environment will be validated and tuned on the physical robot. • Final integration between the different mechanical and computational components will be completed and tested. • Field tests will be continued to validate all functions of the bin managing robot system in real commercial orchards.
Impacts
What was accomplished under these goals?
A self-propelled bin managing robot research platform has been developed and fabricated. This platform consists of a passive mechanical suspension system, a four-wheel independent steering (4WIS) system, a 4WIS coordinate control system and a computer control interface. Laboratory and field tests have validated that the fabricated platform could perform all designed functionalities in actual orchard environments. A multi-robot simulation with bin-managing robots and human pickers has been developed to model bin management under real-world conditions. The multi-robot coordination methods developed in this work reduce the time to collect bins in a real-world orchard simulation by up to 30%. It is expected that the completion of these activities will lead to a successful development of an intelligent in-orchard bin-managing system. When commercialized, the technology can reduce the labor use in bin managing during harvest season while improve the bin-managing efficiency 1) Major activities completed / experiments conducted; • A self-propelled bin managing robot research platform has been developed and fabricated. • Different sensors have been installed on the platform and are currently being tested. • Different control algorithms have been designed and prototyped. • The basic functionalities of the platform, including passive mechanical suspension for ensuring consistent traction force on all wheels, four-wheel independent steering, and three-mode steering (Ackermann, crab, and spinning) coordinating control interface, have been validated in both laboratory and field environments. • The interface between the high-level robot operation and the low-level control has been designed and implemented, and more testing is ongoing. • Simulation developed of typical orchard conditions in the real world. Multiple bin-managing robots situated in the orchard along with human pickers. Pickers take fruit from trees and place in bins in realistic time confirmed by human trials. • Previously collected picking rates for human pickers were integrated into a multi-robot orchard simulation. The pickers take the apples off trees and place them in bins at realistic rates. • Qualitative observations of human pickers were integrated into the simulator. The humans move around the orchard realistically while interacting with the bin-managing robots. 2) Data collected; • Basic performance data of the fabricated robotic research platform has been collected from both laboratory and field tests, and all the basic functions of the platform have been validated. • Laser scanner data has been gathered in-orchard and analyzed for control algorithms. • A program interface has been developed for exchanging information between lower-level controller for actuators on the platform and higher-level controller for auto steering. • Multi-robot coordination simulation data collected comparing a novel auction-based algorithm to a greedy baseline. • Simulation data collected determining length of time for auction-based method and baseline method to fill 160 bins. • Collected simulation data of autonomous bin-managing robots interacting with human pickers during bin collection. Integrated previously collected data on human pickers speeds. 3) Summary statistics and discussion of results • Critical functionalities such as bin loading, driving over a bin, straight driving between tree rows and entering tree lane have been achieved. • Performance tests showed the platform could achieved desired functionalities in both laboratory and field environments. The platform could reach a maximum speed of 1.5 m·s-1 without load and 1.0 m·s-1 with full load. Each wheel could achieve a steering rate of 60°·s-1 with a maximum error of ±2°. The coordinate control system could successfully achieve different steering modes, including Ackerman steering, crab steering, and spinning, on the platform. • Field test showed the passive mechanical suspension could effectively even the load distribution on all four wheels and allow all four wheels to firmly and responsively engage with ground surface. • Developed auction-based method shown to increase the number of bins filled by approximately 50% for varying numbers of robots (from 2 to 20). • Total time to retrieve all 160 bins was reduced by up to 30% using auction-based method. • In the real world, 1 to 5 pickers usually work together to fill one bin. Since each picker can fill one bin in one hour, a bin can be filled in roughly 12 minutes if 5 pickers work together. A picker can usually fill 8 to 10 bins per day. • The above statistics were integrated into the multi-robot simulator and operated in tandem with the multi-robot coordination algorithm to achieve a 30% improvement in bin collection time. 4) Key outcomes or other accomplishments realized. • To steer the robotic platform with the four-wheel independent steering system, it is critical to guarantee the accuracies of angle and speed control of all wheels. Any inaccurate angle and speed control will considerably increase driving resistance and decrease system performance. • The three steering modes greatly improve the efficiency and effectiveness of maneuvering the research platform in orchard environment. Those steering modes are beneficial to achieve different bin-managing objectives such as bin engagement, straight driving between tree rows and lane entering. • Developed a scalable coordination architecture between autonomous robots, which is implemented for fruit bin-carriers to increase the effectiveness of fruit harvesting, specifically to assist human workers to transport fruit bins within the orchard. To achieve a more optimal system result, robots use an auction-based approach to coordinate their decisions to pick up bins in the orchard. The effectiveness of the approach was verified in simulation. • Determined that human pickers can be successfully integrated into a multi-robot simulation environment. The bin-managing robots move at sufficient speed to supply the human pickers with bins at a rate matching how quickly they are filled.
Publications
- Type:
Theses/Dissertations
Status:
Published
Year Published:
2015
Citation:
Zhang, Y., 2015. Multi-robot coordination: applications in orchard bin management and informative path planning. M.S. Thesis, September 2015, Oregon State University.
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2014
Citation:
Ye, Y., L. Yu, and Q. Zhang, 2014. Wheel-slip control on an intelligent bin-dog system in natural orchard environments. In Proc. of 6th Automation Technology for Off-road Equipment (ATOE) Conference, September 16-19, Beijing, China.
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2015
Citation:
Zhang, Y., Y. Ye, Z. Wang, M. Taylor, G. Hollinger, and Q. Zhang, 2015. Intelligent in-orchard bin-managing system for tree fruit production, in Proc. Int. Conf. on Robotics and Automation Workshop on Robotics in Agriculture (ICRA), May 26-30, Seattle, WA.
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2015
Citation:
Ye, Y., S. Ma, and Q. Zhang, 2015. Investigate the influence of steering modes on path tracking performance of a bin-dog system. Presentation at American Society of Agricultural and Biological Engineers (ASABE) Annual International Meeting, July 26-29; New Orleans, LA.
- Type:
Other
Status:
Published
Year Published:
2015
Citation:
Y. Zhang, Y. Ye, Z. Wang, M. Taylor, G. Hollinger, and Q. Zhang, 2015. Intelligent in-orchard bin-managing system for tree fruit production, Poster and Presentation at CPAAS Open House & Ag Tech Day, September 17, Prosser, WA.
- Type:
Other
Status:
Published
Year Published:
2015
Citation:
Y. Zhang, Y. Ye, Z. Wang, M. Taylor, G. Hollinger, and Q. Zhang, 2015. Intelligent In-Orchard Bin-Managing System for Tree Fruit Production, Poster and Presentation at the 2015 National Robotics Initiative (NRI) PI Meeting, November 5-6, Washington, DC.
|
|