Source: UNIVERSITY OF ARIZONA submitted to NRP
AUTOMATED MACHINE FOR SIMULTANEOUS THINNING, WEEDING AND SPOT SPRAYING LETTUCE
Sponsoring Institution
National Institute of Food and Agriculture
Project Status
COMPLETE
Funding Source
Reporting Frequency
Annual
Accession No.
1000315
Grant No.
(N/A)
Cumulative Award Amt.
(N/A)
Proposal No.
(N/A)
Multistate No.
(N/A)
Project Start Date
Oct 1, 2013
Project End Date
Sep 30, 2018
Grant Year
(N/A)
Program Code
[(N/A)]- (N/A)
Recipient Organization
UNIVERSITY OF ARIZONA
888 N EUCLID AVE
TUCSON,AZ 85719-4824
Performing Department
Agri & Biosystems Engineering
Non Technical Summary
Mechanization and advanced automated technologies are needed in lettuce production to reduce manual labor requirements and to lower input costs. Over 80% of commercial lettuce fields are hand thinned and hand weeded, practices which cost growers approximately $170/acre. Furthermore, many pesticides are broadcast applied to the entire field. This wastes product as compared to selectively spot spraying the target plant. The objective of this project is to addresses these issues by developing an automated machine that can be used for thinning, weeding and for selectively spot spraying pesticides on the crop plant but not the surrounding soil surface. The machine will principally comprise of a camera-based machine vision system for detecting lettuce plants and their location, a micro-controller for controlling machine operations and spray systems for thinning, weeding and spot spraying plants. Field trials will be conducted to evaluate machine performance as compared to conventional methods and to quantify savings in labor and pesticide use. Growers, industry personnel and researchers will be informed about the technologies developed through field demonstrations, presentations and peer-reviewed publications.
Animal Health Component
60%
Research Effort Categories
Basic
15%
Applied
60%
Developmental
25%
Classification

Knowledge Area (KA)Subject of Investigation (SOI)Field of Science (FOS)Percent
40214302020100%
Knowledge Area
402 - Engineering Systems and Equipment;

Subject Of Investigation
1430 - Greens and leafy vegetables;

Field Of Science
2020 - Engineering;
Goals / Objectives
The overall objective of this project is to: Develop and evaluate the performance of an automated machine for simultaneous thinning, inter-row and intra-row weeding and spot spraying lettuce. This main objective is broken down into the following sub-objectives: 1. Develop a machine vision system for detecting lettuce plants in situ 2. Fabricate a prototype automated thinning, weeding and spot spraying machine 3. Evaluate the prototype in field trials 4. Transfer the technology developed via field day demonstrations, grower/industry meetings, popular press media releases, professional meeting presentations and peer reviewed manuscripts.
Project Methods
The automated machine for thinning, inter-row and intra-row weeding, spot spraying lettuce will principally comprise of a camera-based machine vision system for detecting lettuce plants and their location, a micro-controller for controlling machine operations and spray systems for thinning, weeding and spot spraying plants. The machine vision system will be developed to detect lettuce and weed plants in situ from images captured by a red (R), green (G), blue (B) camera mounted on the machine. First, a segmentation algorithm will be developed to establish whether each pixel in the image represents a plant or non-plant part. Standard techniques such as using the excessive green index or conversion of RGB values to hue (H), saturation (S), intensity (I) Next, a plant classification algorithm will be developed to determine crop plant and weed objects in the image. The top, bottom, left and right edges of each plant detected will be established to provide location of each plant within the image. A prototype automated machine will be fabricated principally comprising a toolbar, an open bottomed box and a horizontally oriented frame mounted on a parallel linkage assembly. The box and linkage assembly will be mounted to the toolbar. The trailing end of the linkage assembly will be supported by a wheel with an encoder directly coupled to its axle. The wheel will be positioned such that it rides on the crop bed and therefore keeps the frame at a constant height above the soil surface. The frame will be used to support the machine vision system's camera and the spray boom. The boom will comprise 19 spray nozzle/solenoid valve assemblies -2 over each of the 6 six rows (one for applying herbicides for thinning and/or weeding, one for applying pesticides) and 7 for applying herbicides between the six seed rows. Fluorescent tube lights will be mounted to the frame to provide artificial lighting. The implement will be enclosed in the open-bottomed box so that lighting conditions are controlled and images of good and consistent quality can be captured. Ambient light will be minimized by attaching a thin skirt made from a pliable rubber about 1 mm thick to the bottom of the box. As the machine travels through the field, sequential images of the crop bed will be captured. Associated with each image will be an encoder pulse count of when the image was captured. The size of each pixel in terms of real world inches will be calibrated prior to operation. The machine vision system will process the images and determine crop leaf edge boundaries in the lateral and transverse directions in terms of encoder pulse counts and real world distances relative to the center of the camera. This information will be sent as an input to the micro-controller. The micro-controller will be programmed to monitor encoder pulses as the machine travels through the field and turn the appropriate spray nozzle(s) on and off when the beginning and end of crop leaf edges are encountered. User defined "safety zones", where spray is not to be delivered, will be incorporated into the micro-controller program to reduce the likelihood of crop plant injury when herbicidal materials are sprayed. The performance of the automated machine will be evaluated in three trials. Each of these will be repeated over two growing seasons as multiple data sets are needed in order to draw valid and meaningful conclusions. The first will be as an automated weeding machine. The study will be conducted at the University of Arizona's Yuma Agricultural Center (YAC), Yuma, AZ with romaine lettuce. The automated machine will utilize a variety of chemicals known to kill weeds including herbicides, acids and fertilizers. The control treatment will be traditional hand weeding. Experimental design will be a randomized complete block design with 4 replications. Plot lengths will be 50 feet long by two beds wide. Weeding performance will be assessed in terms of percentage of weeds killed, final crop stand and weeding time. Because the spray based automated weeding system induces virtually no soil disturbance, crop yield data will also be recorded and analyzed to determine if there are significant differences in crop yield as compared to conventional cultivation methods. These data in conjunction with costs for labor and machinery will be used to determine gross revenues, production costs and net farm profits for each treatment. Studies will also be conducted to evaluate the performance of the machine as an automated thinner/spot sprayer. In the replicated trials at YAC with head lettuce, the automated machine will be operated so that it simultaneously 1) thins lettuce seedlings to the desired final plant spacing and 2) sprays insecticides on the "saved" crop plant. The control treatment plots will be 1) hand thinned and 2) broadcast sprayed. Sulfuric acid (10% v/v) will be used to thin lettuce in the machine thinned plots. Experimental design will be a randomized complete block design with 4 replications. Plot lengths will be 50 feet long by two beds wide. Time required for thinning and spraying will be measured for each treatment. Pest counts of the target species and feeding damage will be recorded using protocols developed by Dr. John Palumbo, U of A Entomologist. These data will be analyzed to determine insecticide knockdown efficiency and level of residual control for each treatment. Quantity of insecticides used will also be recorded to determine the level of pesticide savings. Crop yield data will also be recorded and analyzed to determine if there are significant differences between treatments. These data in conjunction with cost estimates for labor, pesticides and machinery will be used to determine gross revenues, production costs and net farm profits for each treatment. In the third set of trials, the machine will be configured so that it simultaneously sprays 1) an herbicide on intra-row weeds and 2) insecticides on "saved" crop plant. The control treatment plots will be 1) hand weeded and 2) broadcast sprayed. Experimental design will be a randomized complete block design with 4 replications. Plot lengths will be 50 feet long by two beds wide. Insecticide efficacy data will be collected and analyzed as stated for the automated thinner/spot sprayer trial. Weed densities will be measured before and after weeding by hand and machine to determine levels of weed control. Weeding time and yield data will also be collected to determine weeding labor costs and provide yield comparisons. These data in conjunction with pesticide use and machinery costs will be used to determine gross revenues, production costs and net farm profits for each treatment. Trials will be conducted in head lettuce at YAC. It is expected that the automated thinning, weeding and spot spraying machine will be developed in the first two years of the project. During years three and four, performance evaluation trials will be conducted. During this time, an outreach effort will be made highlighting the technology developed. Field day demonstrations will be held at YAC, articles will be written for popular press media releases and talks will be given at grower and industry meetings. In year five of the project, these efforts will continue and the results of the two year experiment will be presented at a professional meeting and at least one manuscript will be submitted for peer reviewed publication. These documentation and outreach efforts will be measured by recording attendance at meetings and obtaining feedback from surveys distributed.

Progress 10/01/13 to 09/30/18

Outputs
Target Audience:Growers, researchers, ag industry personnel, students, general public. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided? Nothing Reported How have the results been disseminated to communities of interest?Over the course of this project, an extensive outreach effort was made. One invited and four volunteered presentations were given at grower, industry and professional meetings (approximately 380 individuals). Four invited lectures/field demonstrations were given to high school and college students (approximately 84 individuals). The results were also incorporated into a 3-week module on robotic weeding and thinning for ASM 490, Case Studies in Production Agriculture, University of Arizona course. Presentations and demonstrations of the machine in operation (4) and tours (2) of the experimental sites were given to growers, researchers and equipment manufacturers (approximately 117 individuals) and to the general public (2) (approximately 1,500 individuals). Technologies developed and knowledge gained from this project were presented in a co-authored journal article on automated weeding technologies (Fennimore et al, 2016) and a technical paper written and presented at the 2014 California Weed Science Society Meeting (Siemens, 2014). During the course of this project, adoption of automated thinning technology has become wide spread. Combined, at least 50 units are operating commercially and, according to a 2018 survey, utilized by over 50% of California and Arizona growers. In the last couple of years, automated thinning machine manufacturers have begun offering machines equipped with additional spray assemblies for weeding and spot spraying. Several growers are utilizing these technologies. Although it is impossible to ascertain the effect this project has had, it is reasonable to conclude that increased adoption rates and improvements in the technological design of these machines resulted from the knowledge gained through the research and outreach efforts of this project. What do you plan to do during the next reporting period to accomplish the goals? Nothing Reported

Impacts
What was accomplished under these goals? The overall goal of developing and evaluating the performance of an automated machine for simultaneous thinning, inter-row weeding and spot spraying lettuce was achieved. This was accomplished through successful completion of the project's four sub-objectives. A brief description of the accomplishments for each follows. Sub-objective 1: Develop a machine vision system for detecting lettuce plants in situ. A machine vision system for detecting lettuce seedlings in situ was developed. The device principally comprises a camera for capturing images, a computer for detecting lettuce seedlings in the images and a micro-controller for controlling machine operations. A trigger activated red (R), green (G), blue (B) camera captures images as the machine travels through the field. To detect lettuce seedlings, a standard image processing approach is used. First, a segmentation algorithm establishes whether pixels in the image represents a plant or non-plant part. Then histogram plots of pixels determined to be plant parts are made and used to determine the top, bottom, left and right edges of each plant. The device is capable of imaging up to 3 plant rows simultaneously. Although not formally tested, it is estimated that the system is able to reliably detect about 98% of the lettuce plants in non-weedy fields and about 90% in weedy conditions at speeds of 0.5 mph. Sub-objective 2: Fabricate a prototype automated thinning, weeding and spot spraying machine. A prototype 3-row automated thinning, weeding and spot spraying machine was fabricated. The device principally comprised a toolbar, an open bottomed box and a horizontally oriented frame mounted on a parallel linkage assembly. The box and linkage assembly were mounted to the toolbar which attaches to a tractor. The trailing end of the linkage assembly was supported by a wheel with an encoder directly coupled to its axle. The wheel was positioned such that it rides on the crop bed and therefore keeps the frame at a constant height above the soil surface. The frame supports the machine vision system's camera and the spray boom. The boom comprised 10 spray nozzle/solenoid valve assemblies -2 over each of the 3 rows (one for applying herbicides for thinning/weeding, one for applying pesticides) and 4 for applying herbicides between the six seed rows. Fluorescent tube lights were mounted to the frame to provide artificial lighting. The implement is enclosed in the open-bottomed box so that lighting conditions are controlled and images of good and consistent quality are captured. Ambient light was minimized by attaching a thin skirt made from a pliable rubber about 1 mm thick to the bottom of the box. The device utilizes the machine vision and control systems developed in sub-objective 1 to determine lettuce seedling location and to activate solenoid valves connected to spray nozzles to deliver herbicides and pesticides at the proper time. Sub-objective 3: Evaluate the prototype in field trials. The performance of the automated machine was evaluated in four trials over two years. Two were as an automated thinning/weeding machine and two were as a lettuce thinning/spot spraying machine. The studies were conducted at the University of Arizona's Yuma Agricultural Center, Yuma, AZ with romaine (thinning/weeding) and iceberg (thinning/weeding) lettuce. Treatment effects in the thinning/weeding trials were significantly different and thus analyzed separately. In the first trial, weed pressure was very low. Automated weeding provided similar weed control levels to conventional methods. Labor requirements for conventional weeding were 2.8 hr/acre and substantially higher than automated weeding at 0.4 hr/acre. Plant population and plant growth were not affected when measured 14 days after treatment. Overall costs were comparable. In the second trial, weed density was very high. Use of the machine provided poorer weed control as compared to conventional methods. Labor requirements were reduced by over 47.5 hr/acre. Plant population and growth was not suppressed and costs were reduced by $500/acre. In summary, the results show the potential for automated weeding to reduce labor requirements without negatively affecting plant health. They also indicated the potential for substantial cost savings when weed pressure is high. Spot spraying the saved crop plant during lettuce thinning was not an effective method for controlling thrips. Operating the machine as a combination thinning and band spraying machine however, provided pest control levels that were as good as conventional broadcast methods. If the machine were used in this manner - as a thinning and pesticide applicator, growers would reduce application rates by about 75% and save approximately $78/acre in material, application and thinning costs. Sub-objective 4: Transfer the technology developed via field day demonstrations, grower/industry meetings, popular press media releases, professional meeting presentations and peer reviewed manuscripts. Accomplishments for this objective are reported in the Dissemination of Results section of this report.

Publications


    Progress 10/01/16 to 09/30/17

    Outputs
    Target Audience:Growers, ag industry personnel, researchers. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided? Nothing Reported How have the results been disseminated to communities of interest?The results were disseminated to stakeholders via the following outreach means. Presentations: Siemens, M.C. 2017. Automated Machines for Controlling In-Row Weeds, 2017 Southwest Ag Summit, Yuma, Ariz., February 23. 30 minutes. Attendance - 23. (Volunteered) Siemens, M.C. 2017. Precision In-Row Spraying System, 2017 What's Growing in Yuma, Yuma, Ariz., February 24-25. 2017. 16 hours. Attendance - 750. (Volunteered) Siemens, M.C. 2016. Engineering concerns for designing mechanical vegetable weeding systems. 2016 Salinas Valley Weed School, Salinas, Calif., November 2. 30 minutes. Attendance - 75. (Invited) Field Days/Demonstrations: An Automated In-Row Cultivator, 2017 Southwest Ag Summit, Yuma, Ariz., February 22. 2017. 4 hours. Attendance - 275. (Volunteered) Precision Spray Based In-Row Weeding System, 2017 Southwest Ag Summit, Yuma, Ariz., February 22, 2017. 4 hours. Attendance - 275. (Volunteered) Agricultural Engineering Technologies, University of Arizona Engineering Camp - Session I, Yuma, Ariz., July 14, 2017. 1 hour. Attendance - 15. (Volunteered) Agricultural Engineering Technologies, University of Arizona Engineering Camp - Session II, Yuma, Ariz., July 28, 2017. 1 hour. Attendance - 18. (Volunteered) Popular Press: Herzog, Blake. 2017. Ag summit displays new, future farm gear. Yuma Daily Sun, February 23, p. 1, 5. Yuma, Ariz.: The Sun. Johnson, B. 2016. Researchers work to improve mechanized weeding. Ag Alert, November 30, p. 7,9. Sacramento, Calif.: California Farm Bureau Federation. What do you plan to do during the next reporting period to accomplish the goals?Conduct second year field studies to evaluate the performance of the prototype automated thinning, weeding and spot spraying machine. Analyze data and report results.

    Impacts
    What was accomplished under these goals? First year trials were conducted to evaluate the performance of the prototype automated thinning, weeding and spot spraying machine. Data were analyzed. Results showed that automated weeding did not affect plant population or and plant growth when measured 14 days after treatment. This result is encouraging in that it indicates that automated in-row weeding with herbicides can be performed without long term injury to plants. Labor requirements for conventional weeding were 2.8 hr/acre and substantially higher than automated weeding at 0.4 hr/acre. Spot spraying the saved crop plant during lettuce thinning was not as effective at controlling thrips 11 days after treatment as compared to conventional broadcast and band spraying. As compared to band spraying, spot spraying represents more than an 80% reduction in applied insectides. Further study is needed to confirm these preliminary results.

    Publications


      Progress 10/01/15 to 09/30/16

      Outputs
      Target Audience:Growers, ag industry personnel, researchers. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided? Nothing Reported How have the results been disseminated to communities of interest?The results were disseminatedto stakeholders via the following outreach means. Siemens, M.C. 2016. Automated Technologies for In-row Weeding, 2016 Southwest Ag Summit, Yuma, Ariz., February 25. 30 minutes. Attendance - 35. (Invited presentation) Siemens, M.C. 2016. Automated Technologies for In-Row Weed Control. 2016 Pre-Season Vegetable Workshop, Yuma, Ariz., August 25. 30 minutes. Attendance - 96. (Volunteered presentation) Siemens, M.C. 2016. Automated Technologies for In-row Weeding, 2016 Lettuce Days, Yuma, Ariz., February 26-27. 15 hours. Attendance - 800. (Volunteered presentation) Andrade, K.G. 2015. Planting Seed of the Future: Yuma Ag Center Making Inroads with Agricultural Technology. Yuma Daily Sun, August 30, p. 1, 3. Yuma, Ariz.: The Sun. (Popular press) What do you plan to do during the next reporting period to accomplish the goals?Conduct field studies to evaluate the performance of the prototype automated thinning, weeding and spot spraying machine. Initiate data analysis.

      Impacts
      What was accomplished under these goals? The microcontroller software for an automated machine for thinning, weeding and spot spraying was updated so the machine can be used on up to 3 plant rows. Preliminary tests showed the system was able to reliably detect about 98% of the lettuce plants in non-weedy fields and about 90% in weedy conditions at speeds of 0.5 mph. We also completed fabrication of the prototype machine.

      Publications

      • Type: Journal Articles Status: Published Year Published: 2016 Citation: Fennimore, S.F., Slaughter, D.C., Siemens, M.C., Leon, R.G. & Saber, M.N. (2016). Technology for Automation of Weed Control in Specialty Crops. Weed Tech.30(4), 823-837.


      Progress 10/01/14 to 09/30/15

      Outputs
      Target Audience:Growers, ag industry personnel, researchers. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided? Nothing Reported How have the results been disseminated to communities of interest?We have continued to work with Aztera LLC, Tucson, AZ who is interested in commercializing the machine. Numerous correspondences via e-mail and phone were made. We have also visited with numerous growers and ag industry personnel to get their feedback and input on the technologies being developed in terms of practicality and improvements over existing systems. What do you plan to do during the next reporting period to accomplish the goals?During the next reporting period, we plan to finish programming the microcontroller so that the unit is capable of working with 3 plant rows and debug the upgraded system. We also plan to complete fabrication of the automated thinning, weeding and spot spraying machine.

      Impacts
      What was accomplished under these goals? The microcontroller for an automated machine for thinning, weeding and spot spraying was programmed and debugged. The system is able to reliably detect about 95% of lettuce plants in non-weedy fields and about 85% in weedy conditions. The system is able to operate with 2 plant rows on a single bed. Programming to make the unit capable of working with 3 plant rows was initiated. Computer-aided design work on hardware components for the machine was completed and fabrication was initiated.

      Publications


        Progress 10/01/13 to 09/30/14

        Outputs
        Target Audience: Growers, researchers and agricultural industry personnel. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided? Nothing Reported How have the results been disseminated to communities of interest? We have worked very closely with Aztera LLC, Tucson, AZ who is interested in commercializing the machine. Numerous site visits and correspondences via e-mail and phone have been made. We have also visited with numerous growers and ag industry personnel to get their feedback and input on the technologies being developed. What do you plan to do during the next reporting period to accomplish the goals? During the next reporting period, we plan to finish programming the microcontroller and debug the system. We also plan to design and fabricate the automated thinning, weeding and spot spraying machine.

        Impacts
        What was accomplished under these goals? A camera-based machine vision system for an automated machine for thinning, inter-row and intra-row weeding, spot spraying lettuce was developed. The system comprises red (R), green (G), blue (B) cameras mounted on bed following device and a computer for processing images. As the machine travels through the field, sequential images of the crop bed are recorded. Associated with each image is an encoder pulse count of when the image was captured. Lettuce plants and their location are detected in each image using the following process. First, a segmentation algorithm determines whether each pixel in the image represents a plant or non-plant part. This is done by converting RGB values to hue (H), saturation (S), intensity (I) color space and setting threshold values. A histogram based plant classification algorithm is then used to determine the crop plant "objects" in the image. Object size is also used to help differentiate between crop and weed plants. When crop plants are found, crop leaf edge boundary locations in the lateral and transverse directions are determined in terms real world distances relative to the center of the camera. This information can then be sentto a micro-controller to turn appropriate spray nozzle(s) on and off as the machine travels through the field. Microcontroller programming work has been initiated and is in the debugging stage.

        Publications