Source: UNIVERSITY OF ILLINOIS submitted to
COOPERATIVE HUMAN-ROBOT NETWORKS FOR AGRICULTURAL APPLICATIONS
Sponsoring Institution
National Institute of Food and Agriculture
Project Status
TERMINATED
Funding Source
Reporting Frequency
Annual
Accession No.
1003617
Grant No.
2014-67021-22109
Project No.
ILLU-000-635
Proposal No.
2014-07531
Multistate No.
(N/A)
Program Code
A7301
Project Start Date
Aug 15, 2014
Project End Date
Aug 14, 2018
Grant Year
2014
Project Director
Dankowicz, H.
Recipient Organization
UNIVERSITY OF ILLINOIS
2001 S. Lincoln Ave.
URBANA,IL 61801
Performing Department
041544081
Non Technical Summary
Challenges to full deployment of autonomous robotic systems in agriculture include uncertainty associated with variable environmental conditions; unmodeled dynamics affecting the operation of mobile robotic platforms across highly uneven terrain; large variations in crop material properties; and the need for cooperative coordination in human-robotic interactions. Each of these challenges put significant demands on the versatility of hardware and software architectures, e.g., support for interchangeability of tools or platforms, as well as coordinated performance across a network of highly differentiated systems. This project targets the theoretical development and experimental validation of such control and planning architectures for cooperative networks of humans and robotic manipulators on mobile platforms in agricultural applications, using new insights from the field of adaptive control theory and scaled physical hardware realizations. Emphasis is placed on two key applications in precision agriculture: i) semi-automated seeding operations with dedicated, robotic refilling vehicles and ii) crop inspection and treatment operations in uncertain environments. Versatile control designs are sought that sustain guaranteed and coordinated performance even in the presence of significant delays in network communication and actuation channels, and that do not depend on detailed model knowledge of individual nodes in the network. The proposed architectures provide a technology innovation in support of robust human-robot interfaces, as well as interchangeability of tools, platforms, and crops with successful operation across highly variable terrain. Through collaboration with Deere & Co, the proposed research program is grounded in agricultural engineering practice, and lays the scientific foundation for downstream integration, application-specific design and development, and large-scale testing and deployment. Through integration with robotics and mechatronics education, it pools resources across multiple technological disciplines and provides an impetus to ongoing curricular engagement with industrial sponsors.
Animal Health Component
0%
Research Effort Categories
Basic
100%
Applied
0%
Developmental
0%
Classification

Knowledge Area (KA)Subject of Investigation (SOI)Field of Science (FOS)Percent
40253102020100%
Knowledge Area
402 - Engineering Systems and Equipment;

Subject Of Investigation
5310 - Machinery and equipment;

Field Of Science
2020 - Engineering;
Goals / Objectives
The major goal of this project is to build a theoretical and experimental framework for cooperative networks of human operators and robotic manipulators on mobile platforms in agricultural applications that provides guaranteed performance in rapidly changing environments, associated with highly variable terrain topography and soil conditions, and that supports interchangeability of tools, platforms, and crops.Specific research objectives include:1. Design and analysis of cooperative, adaptive strategies for control of task-space performance of networks of robotic manipulators on mobile platforms; specifically, the verification of guaranteed bounds on time-delay margins, the tuning of the architecture to ensure a predictable response that is indifferent to high-frequency noise from the environment, and the incorporation of algorithms that establish a working domain of safe operation.2. Design and analysis of motion-planning algorithms for semi-automated seeding operations with robotic refilling vehicles; including conceptual design, as well as computer-based modeling and simulation, of the corresponding robotic manipulators and refilling system.3. Design and testing of scaled experimental models of human-robotic networks on mobile platforms across uneven and variable terrain; specifically the construction and deployment of customized manipulators for soil sampling and onboard soil characterization, as well as for multi-robot, cooperative handling of plant material and inspection.
Project Methods
This project will use i) theoretical tools of applied mathematics, multibody systems, and nonlinear dynamics (Years 1-2), ii) computational tools of modeling and simulation (Years 1-2), and iii) engineering design, realization, and experimentation techniques (Years 2-3), for the development, deployment, and validation of adaptive control architectures for individual robotic manipulators on mobile platforms, as well as manipulators in networks that may include human operators. The implementation of such adaptive control architectures will be analyzed numerically by means of simulation data measuring sensitivity to time delay in communication and actuation channels, to uncertainty in operation conditions, and to significant unmodeled dynamics of the mobile platforms. They will be evaluated experimentally by means of sensor data measuring tracking performance and quantifying the degree of response predictability across different experimental platforms. Successful designs will support interoperability of tools, platforms, and crops for operation across highly variable terrain. In the context of semi-automated seeding operations, the project will use tools from robotic path and motion planning (Year 2) to propose and analyze real-time algorithms for guidance of robotic refilling vehicles that ensure continuous operation of seeding units. The implementation of such planning algorithms, as well as of possible engineering designs of the corresponding robotic manipulators, will be evaluated using multibody vehicle simulations for a set of characteristics benchmark operating conditions. The results will be interpreted in the context of flexibility, safety, and cost-effectiveness. Collaboration with the Robotics Group at the John Deere Technology Innovation Center will be used to provide up-front design constraints in target agricultural applications and to integrate application-specific feedback on control performance. Experiential learning opportunities for engineering undergraduates interested in agricultural applications will be evaluated through the performance on required course elements of capstone design or independent-study courses. The impact of a workshop on applications of robotics to precision agriculture will be evaluated based on participant surveys.

Progress 08/15/14 to 08/14/18

Outputs
Target Audience:The target audiences reached by the project effort during the reporting period included academic and industrial researchers in the discipline of robot control and agricultural technology, undergraduate mechanical engineering students enrolled at the University of Illinois at Urbana-Champaign, members of the Robotic Systems technical research group at John Deere, and localelementary and middle school children. These audiences have been reached throughjournal papers ininternational peer-reviewed journals;publication of conference papers and presentations at international conferences on robotic systems,engineering science, and agricultural technology;advising of an agricultural robotics capstone design project in collaboration with the John Deere Robotic Systems group, including presentations to faculty and undergraduate students in the Department of Mechanical Science and Engineering at the University of Illinois at Urbana-Champaign;reporting meetings with members of the John Deere Robotic Systems group; and outreach efforts. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided?The project has provided a large array of opportunities for training activities: A PhD student who was funded by the award was mentored in the application of adaptive control theory to robotic systems, as well as theauthoring of journal manuscripts,conference papers, and a doctoral dissertation, defended successfully in April 2015. This individual was subsequently funded by the award as apostdoctoral scholar, during which time hereceived further training in experimental robotics (with the assistance of the project collaborator Dan Block of the Department of Electrical and Computer Engineering at the University of Illinois at Urbana-Champaign),authoring of journal and conference manuscripts; presentation skills;mentoring and supervision of other researchers and students; applying for tenure-track faculty positions; andproposal writing and grant management. The postdoctoral scholar attended the National Robotics Initiative grantee meeting on one occasion and presented a poster at this meeting. This work pertains to specific research objective #1. An MS student who was not funded by the award was mentored in the use of control-based, equation-free computational tools for performing linear control system design optimization, and in the authoring of a master's thesis, completed in May 2017. This work pertains to specific research objective #1. A PhD student who was funded by the awardwas mentored in the application of adaptive control theory to robotic systems, the design of agricultural technologies, and in control-based, equation-free tools for nonlinear system characterization,as well asthe authoring ofjournal manuscripts,a conference paper, and a dissertation proposal, defended successfully in February 2018. Thisstudentattended the National Robotics Initiative grantee meeting on one occasion.This work pertains to specific research objectives #1 and #2. A PhD student who was funded by the award was mentored in techniques for optimized scheduling of seed-refilling vehicles during planting operations, as well as the authoring of a journal manuscript.This work pertains to specific research objective #2. An MS student who was not funded by the award, but whose in-lab and field experiments were funded by the award,was mentored in the experimental application of adaptive control techniques to an experimental system of networked robotics manipulators, as well as inthe authoring of a journal manuscript and a master's thesis, completed in May 2017.This work pertains to specific research objectives #1 and #3. A PhD student who was funded by the award was mentored in experimental robotics,kinematic design, and hardware implementation of adaptive control strategies. Three undergraduate researcherswere mentored in a variety of experimental aspects of the project, including the design and implementation of motion and control strategies for an autonomous rover vehicle. They participated in weekly meetings and received training in research dissemination, in responsible conduct of research, and in a variety of technical tools relevant to robotics research.This work pertains to specific research objectives #2 and #3. A team of five undergraduate students were mentored by the principal investigator during a one-semester capstone design project in agricultural robotics, supported by and in collaboration with the John Deere Robotic Systems group. The students were trained in methods of modeling, design, and analysis of an autonomous robotic device that could be used to interact with plant material, with emphasis on properties of the plant material, the environment within which the device would operate, and other constraints imposed by the agricultural application. The students were required to submit several written reports to the industrial sponsor, and to present their work in several oral presentations to other undergraduate mechanical engineering students. This work pertains to specific research objective #3. How have the results been disseminated to communities of interest?The results of the research have been disseminated to communities of interest through submission of archival journal papers for publication in international peer-reviewed robotics and control journals, a keynote presentation at an international symposium on nonlinear and delayed dynamics of mechatronic systems, a presentation at an international conference on agricultural robotics, and participation in the annual National Robotics Initiative grantee meeting. An outreach event at the Next Generation Primary and Middle School in Champaign, Illinois, coordinated by the PI and in collaboration with Profs. Wissa and Tawfick from the Department of Mechanical Science and Engineering at UIUC, included demonstrations of the rover-based manipulator control algorithms investigated in this project, as well as bio-inspired flying and crawling robots. The event provided opportunities to disseminate knowledge about robotics and applications to agriculture to a highly engaged audience. Additional dissemination to communities of interest include advising of several undergraduate research assistants in the Department of Mechanical Science and Engineering at the University of Illinois at Urbana-Champaign, and reporting meetings with Stewart Moorehead, manager of field robotics at John Deere. Invited seminars were given by the project posdoctoral scholar at South Dakota State University, Oklahoma State University, and Northern Illinois Universityon the topic "Robust Adaptive Control Framework for Agricultural Robotics,"at Southern Illinois University Edwardsville and University of Rhode Island on the topic "Time-Delay Margin Analysis in Nonlinear Adaptive Control Systems," and at University of Michigan-Dearborn and Missouri University of Science and Technologyon the topic "Stability and Robustness of Adaptive Controllers for Underactuated Robotics Networks." What do you plan to do during the next reporting period to accomplish the goals? Nothing Reported

Impacts
What was accomplished under these goals? Robots are often deployed in settings where conditions are predictable and detailed knowledge of the robotic mechanism and its physical environment is available. In such circumstances, well-established robot control strategies typically guarantee successful operation over extended periods of time. When conditions are variable and unpredictable, successful deployment requires new control paradigms that adapt to uncertainty. Robotic systems for agricultural use face these very challenges. They commonly involve mobile platforms that operate across variable terrain and may require coordination over poor communication networks. To address such challenges, this research project has developed new fundamental and applied knowledge pertaining to the control of single or networked robotic manipulators on moving platforms in field environments. This research project has resulted in rigorous mathematical theory and validation using experimental data that show the successful application of a class of adaptive control designs to robotic systems, without using any detailed knowledge of the robot mechanism or the environment. As a result of this effort, some key obstacles to the deployment of robotic manipulators in agricultural applications may be overcome. For example, the outcomes provide proof-of-concept evidence for the stable and predictable behavior of a manipulator end effector that could be used in drive-by crop inspection and treatment. Importantly, the analysis provides insight into how on-the-fly refilling of tractor-pulled seed tanks may be achieved using dedicated robotic vehicles, allowing for increased efficiency and profitability. The knowledge gained in this ongoing research effort forms a foundation for future design and deployment of such field-scale robotic platforms. Objective #1: Theoretical analysis of the L1 adaptive control paradigm was documented in a paper published in the International Journal of Robust and Nonlinear Control. The paper considered several idealized forms of coordination across a network of manipulators on mobile platforms in the presence of communication delays. The analysis showed that the control parameters could always be tuned to ensure transient performance bounds given bounded communication delays. It also provided insights into the influence of network topology on the parameter tuning necessary for successful performance. Theoretical analysis of the L1 adaptive control paradigm applied to a single manipulator in the presence of actuator delays showed that the time delay margin could be increased significantly through purposeful compensation in the control design. Results were published in the proceedings of the IUTAM Symposium on Nonlinear and Delayed Dynamics of Mechatronic Systems, and documented in a manuscript, currently in review by IET Control Theory & Applications. In a paper published in IEEE Transactions on Automatic Control, we documented inequality conditions that guaranteed positive lower bounds on the time-delay margin for an L1 adaptive controller applied to a system with unknown matched nonlinearities. In the case of systems with constant coefficients, we derived approximate methods for estimating the dependence of the time-delay margin on the control system bandwidth. Collectively, these results constituted a change in fundamental knowledge that contributed to a growing literature in nonlinear adaptive control. In addition, we investigated the characterization of time-delay margins for a class of unmodeled systems using an experimental hardware-in-the-loop paradigm. This constituted a first attempt to overcome the absence of explicit estimates of the time delay margin of the adaptive control paradigm. Specifically, in the case of linear, time-invariant systems, this effort showed that feedback control could be coupled with nonlinear root-finding schemes in support of design optimization for robustness to time delays and unmodeled dynamics. We explored the application of this technique to characterization of a nonlinear system to external excitation and found that integration of L1 adaptive control eliminated the need for careful tuning of the control architecture. Objective #2: In consultation with collaborators at John Deere, we investigated the design and operational constraints on a seed-tank refilling vehicle that would ensure uninterrupted seeding operation, and documented our ideas in a conference paper published in the proceedings of the 5th IFAC Conference on Sensing, Control and Automation for Agriculture. In addition, engineering drawings were drafted for a candidate docking mechanism that would support pressurized engagement between a seed tank and a manipulator-guided seed-conveying mechanism. In support of planned scaled experiments demonstrating the ability of a robotic manipulator on a moving platform to accomplish the desired transfer of seeds to a moving seed tank, we completed fabrication of a light-weight three-degree-of-freedom robotic manipulator and mounted this on a small-scale commercial rover that was purchased with project funds and used in laboratory tests of coordination strategies for pairs of manipulators. This piece of equipment is intended tobe used for further testing of these ideas, bothin the lab and in the field, following the conclusion of the funded project. In order to further quantify the economic opportunity associated with on-the-fly seed-tank refilling, we investigated optimization of task scheduling for a single refilling vehicle servicing multiple simultaneous planters. The results demonstrated significant reduction in down time compared with off-field refilling from stationary seed tenders. This work was documented in a paper intended for submission to Computers and Electronics in Agriculture. This constituted a change in knowledge of an original technology with unique opportunities for practical impact in large-scale farming operations. Objective #3: Extensive experimental work on single and networked five-degree-of-freedom manipulators was performed under a variety of laboratory and field conditions in order to validate theoretical predictions for the performance of the adaptive control algorithms, including in the presence of communication delays and initialization errors. The experiments also investigated the ability of the control formulation to ensure a desired end-effector trajectory given uncertain platform movements for possible application to drive-by crop inspection. To this end, various movable platforms were designed, constructed, and instrumented to enable real-time data logging and control actuation. This workconstituted a significant change in fundamental knowledge, providing systematic evidence of the applicability of a theoretical control framework under field conditions andwas documented in an M.Sc. thesis. A journal manuscript documenting the initial experimental validation on a single rover-based manipulator was submitted for consideration by the Journal of Field Robotics, but ultimately declined for publication with a request for more realistic field data. Instead, a new manuscript describing the revised hardware implementation and the new set of experimental data for both single and networked manipulators is being prepared for journal submission. Finally, we advised a capstone design project, sponsored by the John Deere Robotic Systems group, on the use of robotic rovers and appropriate end effectors for automated weeding.

Publications

  • Type: Journal Articles Status: Under Review Year Published: 2018 Citation: Nguyen, K.-D. and Dankowicz, H. 2018. Input-Delay Compensation in a Robust Adaptive Control Framework. IET Control Theory & Applications.


Progress 08/15/16 to 08/14/17

Outputs
Target Audience:The target audiences reached by the project effort during the reporting period include academic and industrial researchers in the disciplines of robot control and agricultural robotics as a result of journal publications and conference presentations as well as elementary and middle school children as a result of outreach efforts. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided?Four graduate students received hands-on training in the theoretical and experimental tools relevant to this project, and in the dissemination of research results, e.g., the authoring of journal and conference manuscripts, conference presentation, and public outreach. Motivated by the research questions pursued in this project, the PI advised to completion two M.Sc. theses, one of which was directly supported by the grant and cited elsewhere, and one illustrating the use of nonlinear root-finding schemes coupled with feedback control in design optimization of linear systems. In addition,one undergraduate student was advised on an independent study project investigating hardware-in-the-loop, feedback-based paradigms for characterization of system dynamics. How have the results been disseminated to communities of interest?The results of the research have been disseminated to communities of interest through submission of archival journal papers for publication in international peer-reviewed robotics and control journals, a keynote presentation at an international symposium on nonlinear and delayed dynamics of mechatronic systems, apresentation at an international conferenceon agricultural robotics, and participation in the annual National Robotics Initiative grantee meeting. An outreach eventat the Next Generation Primary and Middle school in Champaign, Illinois, coordinated by the PI and in collaboration with Profs.Wissa and Tawfick from the Department of Mechanical Science and Engineering at UIUC, includeddemonstrations of the rover-basedmanipulator control algorithms investigated in this project, as well as bio-inspired flying and crawling robots. The eventprovided opportunities to disseminate knowledge about robotics and applications to agriculture to a highly engaged audience. What do you plan to do during the next reporting period to accomplish the goals?Advise three Ph.D. studentsin robotics research, presentation skills, journal and conference dissemination of research. Monitor journal manuscript submitted to Automatica on a delay compensation strategy for the adaptive control paradigm and, as appropriate, respond to peer reviews and editorial decisions. Submit journal manuscriptdescribing the experimental validation of the theoretical control framework on a single robotic manipulator, as well as on a network of manipulators in laboratory and field conditions. Submit journal manuscript on optimized trajectory and rhythm control for interactions between a robotic manipulator and an underactuated dynamical system in the presence of external disturbances and measurement noise. Submit journal manuscript on the use of L1 adaptive control to achieve stable control-based continuation of rhythmic interactions between a robotic manipulator and an underactuated dynamical systemwithout extensive tuning of control gains. Complete investigation of example manipulation tasks with agricultural relevance that require coordinated and complementary movement of multiple robotic manipulators and propose appropriate control strategies. Conduct additional field experiments on scaled models of robotic manipulators on movable platforms.

Impacts
What was accomplished under these goals? Robots are often deployed in settings where conditions are predictable, and detailed knowledge of the robotic mechanism and its physical environment is available. In such cases, existing strategies for control design guarantee safe and successful operation over extended periods of time. When conditions are highly variable and unpredictable, and when detailed knowledge of the mechanism and its environment is unavailable, safe and successful deployment requires new control paradigms that adapt to uncertainty and are robust to the influence of unmodeled dynamics. Robotic systems for agricultural use face these very challenges, as they commonly involve mobile platforms that operate across variable terrain and may require coordination over slow or lossy communication networks. To address such challenges, this research project has developed new fundamental and applied knowledge pertaining to the control of single or networked robotic manipulators on moving platforms in field environments. This ongoing research has resulted in rigorous mathematical theory and validation using experimental data that show the successful application of a class of adaptive control designs to robotic systems, without using any detailed knowledge of the robot mechanism or the environment. Theoretical predictions and corroborating experimental evidence illustrate the viability of the control approach, for example, in the presence of delays in communication and actuation channels. The work has shown that the control architecture may be designed to compensate for known actuator time delays, significantly increasing the domain of successful operation. The work has proposed and analyzed experimental techniques for designing against time delays and unmodeled dynamics. As a result of this effort, some key obstacles to the deployment of robotic manipulators in agricultural applications may be overcome. For example, the outcomes of the theoretical and experimental work provide insight into how on-the-fly refilling of tractor-pulled seed tanks may be achieved using dedicated robotic vehicles, allowing for increased efficiency and profitability. Similarly, the analysis provides proof-of-concept evidence for the stable and predictable behavior of a manipulator end effector used in drive-by crop inspection and treatment. The knowledge gained in this ongoing research effort forms a foundation for future design and deployment of such field-scale robotic platforms. Objective #1: As observed in previous annual reports, theoretical analysis of the L1 adaptive control paradigm applied to a single manipulator in the presence of actuator delays showed that the time delay margin for safe operation could be increased significantly through purposeful compensation in the control design. This constituted a change in fundamental knowledge that contributed to a growing literature in nonlinear adaptive control. During the reporting period, a journal manuscript documenting this analysis was finalized and submitted for possible publication in Automatica. In addition, an invited review of the theoretical analysis of delay robustness in L1 adaptive control systems was presented at a keynote lecture at the IUTAM Symposium on Nonlinear and Delayed Dynamics of Mechatronic Systems, in Nanjing, China, and accepted for publication in Procedia IUTAM. Finally, a paper under review by IEEE Transactions of Automatic Control was further revised and accepted for publication. During the reporting period, the use of nonlinear root-finding schemes coupled with feedback control was further explored as a way to support design optimization for robustness to time delays and unmodeled dynamics in linear systems, as well as the realization of cooordinated interactions between a robotic manipulator and an underactuated, nonlinear dynamical system. This constituted a change in knowledge of methods and techniques use in robotic system design and operation. For example, preliminary results showed that the integration of L1 adaptive control eliminated the need for careful tuning of the control architecture, even in the presence of measurement and actuation errors. Objective #2: During the reporting period, a paper was presented at the 5th IFAC Conference on Sensing, Control and Automation for Agriculture, AGRICONTROL 2016 in Seattle, WA, describing an initial study of the design and operational constraints on a seed-tank refilling vehicle that would ensure uninterrupted seeding operation. Detailed engineering drawings were developed for a docking mechanism that would support pressurized engagement between a seed tank and a manipulator-guided seed-conveying mechanism. Computer-based modeling and simulations of the on-the-fly refilling concept were presented at the National Robotics Initiative grantee meeting in November 2016. This constituted a change in knowledge of an original techology with unique opportunities for practical impact in large-scale farming operations. Objective #3: Extensive experimental work on single and networked five-degree-of-freedom manipulators was performed under a variety of laboratory and field conditions in order to further validate theoretical predictions for the performance of the adaptive control algorithms, including in the presence of communication delays. In this experimental work, it was determined that an alternative piecewise-smooth formulation of the L1 adaptive control framework was better suited for hardware implementation than the one relied on in the experimental effort during the previous reporting period. With the help of this implementation, good agreement with theoretical predictions was achieved, and previous concerns about instabilities associated with discretization were overcome. This constituted a significant change in fundamental knowledge, providing systematic evidence of the applicability of a theoretical control framework under field conditions. This work was documented in an M.Sc. thesis. A revised journal manuscript documenting the initial experimental validation on a single rover-based manipulator was resubmitted for consideration by the Journal of Field Robotics, but ultimately declined for publication with a request for more realistic field data. Instead, as described below, a new manuscript describing the revised hardware implementation and the new set of experimental data for both single and networked manipulators is being prepared for submission.

Publications

  • Type: Journal Articles Status: Awaiting Publication Year Published: 2017 Citation: Nguyen, K.-D., Li, Y. and Dankowicz, H. 2017. Delay Robustness of an L1 Adaptive Controller for a Class of Systems with Unknown Matched Nonlinearities. IEEE Transactions on Automatic Control, 10.1109/TAC.2017.2703913.
  • Type: Journal Articles Status: Submitted Year Published: 2017 Citation: Nguyen, K.-D. and Dankowicz, H. 2017. Input-Delay Compensation in a Robust Adaptive Control Framework. Submitted to Automatica, 2017.
  • Type: Theses/Dissertations Status: Published Year Published: 2017 Citation: Rodriguez Reina, A. 2017. Experimental Validation of an L1 Controller on a Single Robotic Manipulator on a Moving Platform and a Rrobotic Cooperative Network. M.Sc. thesis in Mechanical Engineering, University of Illinois at Urbana-Champaign.
  • Type: Conference Papers and Presentations Status: Awaiting Publication Year Published: 2017 Citation: Nguyen, K.-D. AND Dankowicz, H. 2017. Delay Robustness and Compensation in L1 Adaptive Control. In: Procedia IUTAM, Proceedings of IUTAM Symposium on Nonlinear and Delayed Dynamics of Mechatronic Systems.


Progress 08/15/15 to 08/14/16

Outputs
Target Audience:The target audiences reached by the project effort during the reporting period include: academic and industrial researchers in the discipline of robot control, undergraduate mechanical engineering students enrolled at the University of Illinois at Urbana-Champaign, and technical staff at John Deere. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided?During the reporting period, three undergraduate research assistants were engaged in different experimental aspects ofthis project. Theyparticipated in weekly meetingsand received training in research dissemination, in responsible conduct of research, and in a variety of technical tools relevant to robotics research. A postdoctoral scholar received training in authoring of journal and conference manuscripts and in professional presentation, as well as in mentoring and supervision of other researchers. The postdoctoral scholar attended the National Robotics Initiative grantee meeting and presented a poster at this meeting. The postdoctoral scholar also received career mentoring, specifically in applying for tenure-track faculty positions, as well as training in proposal writing and grant management. Three graduate students received training in the theoretical and experimental tools relevant to this project, as well as in the authoring of journal and conference manuscripts and a graduate thesis. One of thegraduate studentsattended theNational Robotics Initiative grantee meeting. How have the results been disseminated to communities of interest? The results of the research have been disseminated to communities of interest through submission of archivaljournal papersfor publication in international peer-reviewed robotics and control journals, submission of conference papers to international conferences on robotic systems, nonlinear control, and agricultural mechanization, as well as participation in the annual National Robotics Initiative grantee meeting. Additional dissemination to communities of interest include advising of several undergraduate research assistants in the Department of Mechanical Science and Engineering at the University of Illinois at Urbana-Champaign, and reporting meetings with Stewart Moorehead, manager of field robotics at John Deere. Invited seminars on the topic "Robust Adaptive Control Framework for Agricultural Robotics" were givenat South Dakota State University, Oklahoma State University, and Northern Illinois University. Invited seminars on the topic "Time-Delay Margin Analysis in Nonlinear Adaptive Control Systems" were given atSouthern Illinois University Edwardsville and University of Rhode Island. Invited seminars on the topic "Stability and Robustness of Adaptive Controllers for Underactuated Robotics Networks" were given at University of Michigan-Dearborn and Missouri University of Science and Technology. Efforts were also made to organize a workshop on agricultural robotics at the ASME International Design Engineering Technical Conferences meeting in Charlotte, NC in August 2016, in collaboration with Vijay Kumar of the University of Pennsylvania and Gary McMurray of Georgia Institute of Technology. These were abandoned as a result of the close proximity in time to the Agricontrol 2016 meeting in Seattle, Washington, and limited availability of industry representatives. As part of this effort, invitations were sent to Autonomous Tractor Corporation, Blue River Technology, Energid, Harvest Automation, ClearPathRobotics, Autonomous Solutions, Vision Robotics, Jaybridge Robotics, AGCO, Rowbot, Agrobot, Harvest CROO Robotics, Dorhout R&D, MackRobotics, as well as Frank Tobe of the Robot Report. What do you plan to do during the next reporting period to accomplish the goals?During the next reporting period, we will: 1. Present a paper at the 5th IFAC Conference on Sensing, Control and Automation for Agriculture, AGRICONTROL 2016 in Seattle, WA. Participation at this event will include the principal investigator and two project graduate students. 2. Finalize and submit a journal manuscript, co-authored by the principal investigator and the postdoctoral scholar,to the International Journal on Robust and Nonlinear Control, describing the delay compensation strategy for the adaptive control paradigm. 3. Finalize and submit a revised journal manuscript to the Journal of Field Robotics, co-authored by the principal investigator, the postdoctoral scholar, and one project graduate student,describing the theoretical and experimental results on the single robotic manipulator. 4. Conduct experiments on a network of three identical, five-degree-of-freedomindustrial manipulators, with internet-based communication, to validate the predictions regarding stable performance in the presence of communication delays, and document this in a journal publication, co-authoredby the principal investigator, the postdoctoral scholar, and one project graduate student. 5. Advise an undergraduate student on an independent study project investigating hardware-in-the-loop, feedback-based paradigms for characteriziation of system dynamics. 6. Present an invited keynote presentation onDelay Robustness and Delay Compensation of a Nonlinear Control Frameworkat the IUTAM Symposium on Nonlinear and Delayed Dynamics of Mechatronic Systems, in Nanjing, China, and publish an invitedpaper in the symposium proceedings. 7. Advise three graduate research assistantsin robotics research, presentation skills, journal and conference dissemination of research. Advise one graduate research assistant toward the completion of a M.Sc. degree in Mechanical Engineering. 8. Develop detailed engineering design drawings of a candidate roboticmechanismfor implementing on-the-fly seed tank refilling and demonstrate its function using acomputer-based simulation model. 9. Propose and investigate example manipulation taskswith agricultural relevancethat require coordinatedand complementarymovement of multiple robotic manipulators. 10. Conduct additional field experiments on scaled models of robotic manipulators on movable platforms, including customized end effectors for handling of soil and plant material. 11. Revisitthe previously abandoned organization of a workshop on agricultural robotics and automation, aiming to identify a possible location and likely availability of academic and industry participants.

Impacts
What was accomplished under these goals? Robots are often deployed in settings whereconditions are predictable, and detailed knowledge of the robotic mechanism and its physical environment is available. In such cases, existing strategies for control design guarantee safe and successful operation over extended periods of time. When conditions are highly variable and unpredictable, and when detailed knowledge of the mechanism and its environment is unavailable, safe and successful deployment requires new control paradigms that adapt to uncertainty and are robust to the influence of unmodeled dynamics. Robotic systems for agricultural use face these very challenges, as they commonly involve mobile platforms that operate across variable terrainand may require coordination over slow or lossy communication networks. To address such challenges, this research project has developed new fundamental and applied knowledge pertaining to the control of single or networked robotic manipulators on moving platforms in field environments. This ongoing research has resulted in rigorous mathematical theory and validation using experimental data that show the successful application of a class of adaptive control designs to robotic systems, without using any detailed knowledge of the robot mechanism or the environment. Theoretical predictions and corroborating experimental evidence illustrate the viability of the control approach, for example, in the presence of delays in communication and actuation channels. The work has shown that the control architecture may be designed to compensate for known actuator time delays, significantly increasing the domain of successful operation. The work has proposed and analyzed experimental techniques for designing against time delays and unmodeled dynamics. As a result of this effort, some key obstacles to the deployment of robotic manipulators in agricultural applications may be overcome. For example, the outcomes of the theoretical and experimental work provide insight into how on-the-fly refilling of tractor-pulled seed tanks may be achieved using dedicated robotic vehicles, allowing for increased efficiency and profitability. Similarly, the analysis provides proof-of-concept evidence for the stable and predictable behavior of a manipulator end effector used in drive-by crop inspection and treatment. The knowledge gained in this ongoing research effort forms a foundation for future design and deployment of such field-scale robotic platforms. Objective #1: Theoretical analysis of the adaptive control paradigm applied to a network of manipulators in the presence of communication delays was completed and documented in a journal manuscript to appear in the International Journal of Robust and Nonlinear Control. The paper considers several idealized forms of coordination across the network, viz., synchronization along a known desired joint-space trajectory, convergence to an emergent consensus trajectory, and a leader-follower scheme, in which the desired joint-space trajectory is known only to a single node in the network. The analysis showed that the control parameters could always be tuned to ensure transient performance bounds (i.e., input-to-output-stability in the presence of unknown external disturbances and lack of detailed system models) given bounded communication delays. It also provided insights into the influence of network topology on the parameter tuning necessary for successful performance. Theoretical analysis of the adaptive control paradigm applied to a single manipulator in the presence of actuator delays showed that the time delay margin could be increased significantly through purposeful compensation in the control design. These results were documented in a journal manuscript, to be submitted for possible publication in the International Journal of Robust and Nonlinear Control. The analysis also showed that the theoretical predictions regarding stability in the absence of time delays were robust to initialization errors associated with uncertain measurements of the system state. Additionally, a control formulation was proposed to ensure that the end effector of a manipulator on a moving platform would follow a desired task-space trajectory, as a step toward the realization of drive-by inspection and treatment tasks in orchard or field crop environments. Together with experimental data described below, these results were documented in a journal manuscript submitted to the Journal of Field Robotics, currently undergoing a first stage of revision. Theoretical and computational tools were developed to demonstrate the characterization of time-delay margins for a class of unmodeled systems using an experimental hardware-in-the-loop paradigm. These tools constitute a first attempt to overcome the absence of explicit estimates of the time delay margin of the adaptive control paradigm. Specifically, in the case of linear, time-invariant systems, this effort showed that feedback control could be coupled with nonlinear root-finding schemes in support of design optimization for robustness to time delays and unmodeled dynamics. Objective #2: In continued consultation with our collaborators at John Deere, we investigated the design and operational constraints on a seed-tank refilling vehicle that would ensure uninterrupted seeding operation. Our analysis of the opportunities and challenges associated with such a robotic system were documented in a conference paper to be presented at the 5th IFAC Conference on Sensing, Control and Automation for Agriculture. In support of planned scaled experiments demonstrating the ability of a robotic manipulator on a moving platform to accomplish the desired transfer of seeds to a moving seed tank, we completed fabrication of a light-weight three-degree-of-freedom robotic manipulator for mounting on an autonomous or remote-controlled rover platform, and investigated motion control algorithms using optical sensors for allowing the rover to track a secondary vehicle in field conditions. Objective #3: Experiments were performed on a five-degree-of-freedom industrial manipulator under a variety of laboratory and field conditions in order to validate the predicted dependence of the transient performance bounds on various control parameters and on the initialization error, as well as to demonstrate the ability of the control formulation to ensure a desired end-effector trajectory given uncertain platform movements for possible application to drive-by crop inspection. To this end, various movable platforms were designed, constructed, and instrumented to enable real-time data logging and control actuation. Initial experimental results were documented in the above-referenced journal manuscript submitted to the Journal of Field Robotics. Laboratory experiments yielded observations consistent with the theoretical predictions of the mathematical analysis. Field experiments highlighted the practical challenges to the control algorithm under realistic conditions, for example, necessitating the introduction of passive suspension mechanisms between the platform and the manipulator. Communication protocols and control implementations were developed and tested for planned experiments with multiple manipulators communicating over an internet-based wifinetwork and moving according to either of the forms of coordination described under Objective #1. In particular, to investigate a leader-follower implementation wherein the movement of a human operatordetermines the desired trajectory, sensors were designed and fabricated for sensing the interactions between a single manipulator and the operator arm, in support also of experimentally validating an assistive strategy in which a single manipulator could switch between stabilizing the human wrist, or non-intrusively tracking the arm movement.

Publications

  • Type: Journal Articles Status: Awaiting Publication Year Published: 2016 Citation: Nguyen, K.-D. and Dankowicz, H. 2016. Cooperative Control of Networked Robots on a Dynamic Platform in the Presence of Communication Delays. International Journal of Robust and Nonlinear Control.
  • Type: Conference Papers and Presentations Status: Awaiting Publication Year Published: 2016 Citation: Li, Y., Nguyen, K.-D. and Dankowicz, H. 2016. A Robust Adaptive Controller for a Seed Refilling System on a Moving Platform. The 5th IFAC Conference on Sensing, Control and Automation for Agriculture, AGRICONTROL 2016.
  • Type: Journal Articles Status: Other Year Published: 2016 Citation: Nguyen, K.-D., Rodriguez Reina, A. and Dankowicz, H. 2016. End-Effector Control in Crop and Orchard Drive-By Inspections and Treatments. The Journal of Field Robotics. Awaiting Revision.


Progress 08/15/14 to 08/14/15

Outputs
Target Audience:The target audiences reached by the project effort during the reporting periodinclude: academic and industrial researchers in the discipline of robot control, undergraduate mechanical engineering students enrolled at the University of Illinois at Urbana-Champaign, and technical professionals in the Robotic Systems technical research group at John Deere. These audiences have been reached through: submission for publication of an archival journal paperin an international peer-reviewed controljournal, publication of conference papers and presentations at international conferences onrobotic systems and engineering science, advising of an agricultural robotics capstone design project in collaboration with the John Deere Robotic Systems group, including presentations to faculty and undergraduate students in the Department of Mechanical Science and Engineering at the University of Illinois at Urbana-Champaign, and kick-off and reporting meetings with members of the John Deere Robotic Systems group. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided?The project has providedthe following opportunities for training activities: A senior graduate student has been mentored in the authoring of several full-length journal manuscripts and conference papers, as well as a doctoral dissertation, defended successfully in April 2015. This work pertains to specific research objective #1. A junior graduate student has been mentored in the use of control-based, equation-freecomputational tools for performing linear control system design optimization, in the case of model uncertainty.This work pertains to specific research objective #1. A junior graduate student has been mentored in the authoring of one full-length journal manuscript and conference paper.This work pertains to specific research objective #1. An undergraduate researcher has been mentored in the design and implementation of motion and control strategies for an autonomous rover vehicle. This work pertains to specific research objective #2. A team of five undergraduate students were mentored by the principal investigator during a one-semester capstone design project in agricultural robotics, supported by and in collaboration with the John Deere Robotic Systems group. The students were trained in methods of modeling, design, and analysis of an autonomous robotic device that could be used to interact with plant material, with emphasis onproperties of the plant material, the environment within which the device would operate, and other constraints imposed by the agricultural application. The students were required to submit several written reports to the industrial sponsor, and to present their work in several oral presentations to other undergraduate mechanical engineering students.This work pertains to specific research objective #3. The project has included the following opportunities for professional development: A postdoctoral scholar has developed proposal-writing skills, aiming to generalize the algorithms developed in the current research project to other applications, including robotic surgery. A postdoctoral scholar has developed human resources skills through supervision and timesheet approval foran undergraduate researcher. A postdoctoral scholar has developed skills in experimental robotics, with the assistance of the project collaborator Dan Block of the Department of Electrical and Computer Engineering at the University of Illinois at Urbana-Champaign. A senior graduate student has developed grant management skills through representing the project at the annual NRI meeting. A senior graduate student has developed research dissemination skills through technical presentations at two professional meetings. How have the results been disseminated to communities of interest?The results of the research have been disseminated to the target audiences through submission and publication of journal and conference papers, as well as technical presentations at several conferences. Additional dissemination to communities of interest include through kick-off and reporting meetings with the John Deere Robotic Systems group. What do you plan to do during the next reporting period to accomplish the goals?In addition to the general outline of the research effort in the original application, the following activities are planned during the next reporting period to accomplish the major goals and specific research objectives: Submission of a journal paper to Robotics and Autonomous Systems on predictable performance of networked manipulators on a common mobile platform, independently of arbitrary communication delays, for tracking synchronization, consensus, and leader-follower tasks.This work pertains to specific research objective #1. Submission of a journal paper on guaranteed input time-delay margins forsystems with time- and state-dependent coefficient matrices, including experimental validation on industrial robot.This work pertains to specific research objectives #1 and #3. Development of a mathematical theory to support improvements to input time-delay margin through purposeful introduction of time delay in the control architecture,includingapplications to infinite-dimensional systems, for example, deformable mechanical structures.This work pertains to specific research objective #1. Implement leader-follower algorithm for multiple fixed-base manipulators connected over an internet-based communication network, including manipulators interacting with a human operator.This work pertains to specific research objectives #1 and #3. Design and implement scaled experimental test of semi-automated seeding operation, including formation control and seed transfer, using severalcommercial rover platforms and a custom-made four-degree-of-freedom manipulator.This work pertains to specific research objective #2. Establish proof of concept of control-supported optimization of system robustness, even in the absence of an accurate system model, and document this work in a MSc thesis.This work pertains to specific research objective #1. Provide training to two undergraduate research assistants, two graduate students, and one post-doctoral scholar. Organize aworkshop on agricultural robotics and automation in conjunction with the 2016 ASME International Design Engineering Technical Conferences and Computers & Information in Engineering meeting in Charlotte, NC in August, 2016.

Impacts
What was accomplished under these goals? Robots are commonly deployed in industrial settings where the operating conditions are predictable, and detailed knowledge of the robotic mechanism and its physical environment is available. In such circumstances, existing strategies for control design guarantee safe and successful operation over extended periods of time. When the operating conditions are highly variable and unpredictable, and where detailed knowledge of the mechanism and its environment is unavailable, safe and successful deployment requires new paradigms of control design that adapt to uncertainty and are robust to the influence of unmodeled dynamics. Robotic systems for agricultural use face these very challenges, as they commonly involve mobile platforms that operate across variable terrain, need to support interchangeability of tools and implements, and require coordination between multiple devices over slow or lossy communication networks. To enable the coordinated use of networks of robotic manipulators on mobile platforms in agricultural applications, this research has developed a mathematical theory that shows that a particular class of adaptive control designs can be successfully applied to such systems, even when a mathematical model of the system is unavailable, andin the presence of delays in communication and actuation channels, and uncertain disturbances from a variable environment. This research has also provided preliminary experimental validation of this theory by showing predictable and stable performance on an industrial robot, without using any knowledge of the robot design. The anticipated broader impact includes the coordinated field deployment of robotic rovers for plant inspection, weed removal, and semi-automated seeding and harvesting operations. The following was accomplished during the reportingperiod for each of the specific research objectives: 1. We successfully developed mathematical adaptive control strategies for control of joint-space performance of networks of robotics manipulators on mobile platforms. This included the formulation of rigorous proofs of stabilityin the presence of bounded disturbances, even in cases where the system models were unavailable. We successfully derived conditions that guaranteed positive lower bounds on the time-delay margin, as a measure of robustness to unmodeled dynamics, even with very large values of the adaptive estimation gains, required to compensate for high-frequency disturbances. In the case of systems with constant coefficients, we derived approximate methods for estimating the dependence of the time-delay margin on the control system bandwidth.We showed that when the adaptive control algorithms were applied to networks of manipulators,performance bounds could be guaranteed in the presence of arbitrary communication delays, providedthat the architecture was tuned to satisfy delay-independent stability conditions. We demonstrated that successful performance could be accomplished for a variety of network topologies and control objectives, including tracking synchronization, consensus, and leader-follower schemes. These results were verified in numerical simulations of multiple robotic manipulators on a moving platform with unmodeled and unactuated dynamics. 2. We purchased a used industrial robotic manipulator and implemented the adaptive control strategies on custom-made motor control hardware. Preliminary results validated the theoretical predictions for the existence of a positive lower bound on the time delay margin, even with very large values of the adaptive gain. Comparisons with simple proportional-derivative control demonstrated significant improvements in the ability to accommodate unmodeled nonlinearities with predictable linear response. 3. In collaboration with the John Deere Robotic Systems group, we developed a conceptual design of a robotic refilling vehicle and formulated preliminary constraints on geometry and operation. These included constraints on approach vectors and formation control. We drafted engineering schematics for a robotic manipulator and refilling system and purchased a commercial rover for use in a scaled experimental test. We designed and implemented motion control algorithms that use ultrasonic sensors for determining the relative positioning of the rover relative to a stationarysecond vehicle. 4. We advised a capstone design project, sponsored by the John Deere Robotic Systems group, on the use ofrobotic rovers for automated weeding. We worked with Dan Block in the Department of Electrical and Computer Engineering to implement motor control hardware for several used industrial robotic manipulators for future testing of cooperative leader-follower schemes over an internet-based communication network.

Publications

  • Type: Theses/Dissertations Status: Published Year Published: 2015 Citation: Nguyen, K.-D. Stability and Robustness of Adaptive Controllers for Underactuated Lagrangian Systems and Robotic Networks. PhD dissertation, University of Illinois at Urbana-Champaign, April 2015.
  • Type: Journal Articles Status: Submitted Year Published: 2015 Citation: Nguyen, K.-D., Li, Y. and Dankowicz, H. Delay Robustness of an Adaptive Controller for a Class of Systems with Unknown Nonlinearities. Submitted to Transactions of Automatic Control, 2015.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2014 Citation: Nguyen, K.-D. and Dankowicz, H. Synchronization and Consensus of a Robot Network on an Underactuated Dynamic Platform. Proceedings of 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2014), September 14-18, 2014, Chicago, IL, USA, pp. 117-122, doi: 10.1109/IROS.2014.6942549.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2014 Citation: Nguyen, K.-D. and Dankowicz, H. Switching Adaptive Control of a Bioassistive Exoskeleton. Society of Engineering Science 51st Annual Technical Meeting, October 1-3, 2014, Purdue University, West Lafayette, IN, USA.