Source: BIRD'S EYE ROBOTICS, INC. submitted to
DEVELOPMENT OF COMPUTER VISION AND GRAPPLE MECHANISM TO IMPROVE POULTRY WELFARE ACROSS LIFE CYCLE THROUGH AUTONOMOUS ROBOTIC SOLUTIONS.
Sponsoring Institution
National Institute of Food and Agriculture
Project Status
COMPLETE
Funding Source
Reporting Frequency
Annual
Accession No.
1028476
Grant No.
2022-33530-37149
Cumulative Award Amt.
$175,000.00
Proposal No.
2022-00884
Multistate No.
(N/A)
Project Start Date
Jul 1, 2022
Project End Date
Feb 28, 2023
Grant Year
2022
Program Code
[8.3]- Animal Production & Protection
Project Director
Niewohner, S.
Recipient Organization
BIRD'S EYE ROBOTICS, INC.
19994 COUNTY ROAD 19
HERMAN,NE 680295128
Performing Department
(N/A)
Non Technical Summary
To meet rising global demand for protein the poultry industry has increased the size and scale of rural broiler farms across the country. Operating and maintaining poultry operations require significant labor inputs by either the producer themselves or hired labor. Standardized barn layouts are conducive to autonomous technologies, yet the poultry industry has lagged behind the 'precision agriculture' wave seen in other production verticals. This project seeks to develop computer vision technology and mechanical grapple to enable autonomous dead-bird retrieval capabilities given unique poultry barn considerations. The four technical objectives of this research are: develop forward-facing computer vision design and software within populated barns, develop and refine mortality recognition algorithms throughout entire bird life cycle, develop image recognition to distinguish between 'cull' (i.e. birds that may be injured or immobilized) and dead birds, and identify and prototype optimal grapple and retrieval hardware mechanism. This research will development and design a fully autonomous mortality recovery system that is economical in industry standard broiler barns. The potential commercial application of this proposed research will enable a highly trained robotic solution that can notice the most obscure in-barn abnormality even faster than the top trained poultry veterinary specialist on-site while decreasing mortality removal labor costs. This proposal aligns with the topic area Animal Production and Protection - Topic Area 8.3 research priorities by aiming to improve production efficiency and animal health and well-being. This proposal aligns with the USDA Strategic Goal 2: Maximize the Ability of American Producers to Proposer by Feeding the World
Animal Health Component
50%
Research Effort Categories
Basic
25%
Applied
50%
Developmental
25%
Classification

Knowledge Area (KA)Subject of Investigation (SOI)Field of Science (FOS)Percent
30732992020100%
Knowledge Area
307 - Animal Management Systems;

Subject Of Investigation
3299 - Poultry, general/other;

Field Of Science
2020 - Engineering;
Goals / Objectives
There is a critical need for automated mortality removal systems to reduce labor requirements and provide consistent/satisfactory removal performance to improve grower productivity and bird husbandry. This proposal aligns with the USDA Strategic Goal 2: Maximize the Ability of American Producers to Proposer by Feeding the World.This project aims to develop the FOX Robot (Fowl Observation and Extraction), an autonomous poultry robot that performs dead bird retrieval in an economic manner for poultry farmers. During Phase I, Birds Eye Robotics' R&D efforts will focus on selecting and optimizing key computer vision factors to improve autonomous navigation and dead bird retrieval in populated barns, namely, stereo-camera angles, frame rates, database-enabled algorithms and the associated grapple mechanism.The technical goal of this phase one Small Business Innovation Research Grant is to develop computer vision technology and mechanical grapple to enable autonomous dead-bird retrieval capabilities given unique poultry barn considerations. This goal is divided into four objectives with technical questions in italic:1) Develop forward-facing computer vision design and software within populated barns2) Develop and refine mortality recognition algorithms throughout entire bird life cycle3) Develop image recognition to distinguish between 'cull' (i.e. birds that may be injured or immobilized) and dead birds4) Identify and prototype optimal grapple and retrieval hardware mechanism
Project Methods
Objective 1: Develop forward-facing computer vision design and software within populated barnsThis technical objective will research how higher frame rates impact the processing capacity of the FOX Unit as a whole (with specific regards to the VSLAM navigation system). The Birds Eye Robotics team hypothesizes that initial 'off-the-shelf' image recognition software may be moderately effective for obstacle avoidance but may need significant sensitivity adjustments in settings in which more mature birds are crowding in front of the unit. The team will test varying focal points on the unit where the stereo camera is placed to ensure feeder line recognition for navigation while maintaining accurate obstacle avoidance. During the experimentation and execution of technical objectives ones and two our team will deploying multiple (likely four to five) FOX Unit prototypes to expedite the data collection process and learning opportunities within the barn.Objective 2: Develop and refine mortality recognition algorithms throughout entire bird life cycleThis technical objective will refine the operating speed at which the FOX unit navigates the broiler barns. An operating speed that is too slow will not properly stimulate the birds to cause the needed movement to reveal present bird mortality on the floor for the camera vision. An operating speed that is too fast will cause disruption and unneeded stress amongst the birds. This experiment will be conducted with varying speed intervals down a section of feeder lines with the team observing bird reaction and monitoring camera vision recognition capabilities.Objective 3: Develop image recognition to distinguish between 'cull' and dead birdsFor two months, this objective overlaps with technical objective 4: Identify and prototype optimal grapple and retrieval hardware mechanism. As the grapple system engages a cull or downer bird the improved image recognition must be able to recognize any sign of movement or life. To conduct this objective the team will manually observe and label photos to build the initial image library.

Progress 07/01/22 to 02/28/23

Outputs
Target Audience:The United States has the world's largest broiler industry, with over 9 billion chickens produced each year. The poultry industry is largely vertically integrated, resulting in major production consolidation to ~30 companies that contract with ~ 30k farms that produce 95% of the broiler chickens in the U.S. Operating and maintaining poultry operations require significant labor inputs by either the producer themselves or hired labor. Growers traditionally are raising broilers on contract with one of the major poultry companies, and one of the key variables that determines profitability is labor costs. For operations with 30,000 or more birds, labor costs accounted for 30% of the total costs of farming operations. As producers seek to operate multiple barns to reach economies of scale, the need for hired labor grows. Assuming a $16 hourly labor rate, this results in ~$15k in labor cost per barn each year for walking the barns to check on bird health, stimulate the flock and till bedding. Birds Eye Robotics will focus on American operations that contract with large vertically integrated corporations such as Tyson Foods, Pilgrims Pride, Sanderson Farms, and Perdue Foods. These companies have made focused efforts to ensure their growers have access to the best feeder and water lines to improve animal welfare and in some cases have subsidized the cost of their installations and upgrades. There are approximately 100,000 commercial broiler barns in the United States typically operated by family farms of four to eight barns. In 2019, more than half of all farmworkers in the U.S. were immigrants, and in recent years, new immigrant arrival has fallen 75%. The resulting labor shortage has driven up labor costs for many poultry producers across the country. Furthermore, poultry production labor is considered a 3D job (Dirty, Demanding, and Dangerous), as indicated by 100 deaths and 300k injuries in the poultry industry each year (1.5x the U.S. worker average) (National Center for Farmworker Health, 2014). The nature of this work naturally yields high turnover rates - 58% quit within 90 days, and only 14% stay on longer than 180 days. It is estimated that absenteeism and high turnover add an additional 4% to production costs, which can threaten profitability in an already labor intensive, thin-profit margin industry. Changes/Problems:The major unforeseen problem encountered throughout the first four months of this grant period was the widespread avian influenza, or HPAI, that placed restricted access to testing facilities on the research and development team. The major testing partner for phase I was Lincoln Premium Poultry, a company owned by Costco Wholesale, which manages hundreds of barns in eastern Nebraska and parts of Iowa. Nebraska State Veterinarian Roger Dudley said the region has increased its safety practices and heightened observational testing amid the nationwide outbreak. The avian influenza of 2022 was one of the worst outbreaks in industry history, triggering the deaths of some 52.7 million animals. Out of an abundance of caution for the birds, Birds Eye Robotics could not enter the testing barns for several weeks of this grant. Specifically, this delay in barn access pushed back objective three. To combat this challenge the team constructed a miniature test barn and purchased industry standard feeder and drinker lines to continue training guidance algorithms. Additionally, the team reprioritized research and development resources to begin work on objective four sooner than initially planned. In concluding months of the phase I Birds Eye Robotics regained access to testing barns in partnership with Lincoln Premium Poultry and successfully completed all four technical objectives. What opportunities for training and professional development has the project provided?As part of this proposal one student was budgeted to assist senior personnel inexpanding the image library for particular bird age-ranges and refine/train the recognition algorithms to reach acceptable alpha/beta error levels. This individual also supported in-barn testing and experimental design.Finally, throughout the entirety of this project many poultry veterainarians anmd barn technicians were able to meet with the Birds Eye Robotics team and were given considerations about the future of precision livestock technology. How have the results been disseminated to communities of interest?The initial phase of this project 'Development of computer vision and grapple mechanism to improve poultry welfare across life cycle through autonomous robotic solutions' concluded Feb. 2023. In summary, the research team spent over 400+ live hours in the barn developing a proof-of-concept unit with specific deliverables related to the computer vision design, mortality recognition, cull bird recognition, and a retrieval mechanism. The conclusion of phase I included a "Industry Showcase" in which the Caretaker was privately demoed for some of the large poultry operations to gain key points of feedback. Additionally, the company is now fundraising a large equity round to commercialize its technology. What do you plan to do during the next reporting period to accomplish the goals? Nothing Reported

Impacts
What was accomplished under these goals? The Birds Eye Robotics team deployed several robots both in the onsite test barn as well as live barns in partnership with Lincoln Premium Poultry. As part of this objective the robots have navigated the barns while the team experimented with different camera selection, location and viewing angles. Senior personnel developed image recognition to distinguish between cull and deadbirds.R&D started with over 13,000 cropped chicken images. The center of the images where good views of whole birds were present was rare because the spinner encourages the birds to make way for the robot. Most of the birds were at the edges of the image and only partially visible. R&D filtered out the partially visible birds which removed approx. 9,000 images from the data set leaving us with 4,000 whole chicken images to train on. R&D further filtered the images by their Laplacian variance to remove the blurry, dark, low contrast images reducing training set to the 600 highest quality images suitable for machine learning. R&D alsobuilt a robust data and training pipeline. Images of interest were be snapped and tagged in the barn by the onsite field engineer. They were then uploaded in real time to the training pipeline where further labeling was added or corrected beforetraining. The resulting model was then remotely uploaded to the robot once the team was confident in its execution. As R&D deployed more prototype units this system enabled more efficient identification and resolution of edge cases. Finally R&D developed anewly designed FRAC (Finger Rotating ACumutator) style retrieval system. The initial grapple required ground to be perfectly level with finely tuned height of grapple to avoid mishandling a carcass. A more flexible resilient system was required. The FRAC engineered prototype has proven to be more resilient to bird position, size, and uneven terrain. The Birds Eye Robotics team continued to do extensive testing in different moisture contents and became more confident in this new design structure in a proof-of-concept environment.

Publications