Source: FORGEBEE, LLC submitted to
AN AUTOMATED SYSTEM FOR HONEY BEE HUSBANDRY THAT ENABLE HIGH-THROUGHPUT BIOLOGICAL ASSAYS
Sponsoring Institution
National Institute of Food and Agriculture
Project Status
ACTIVE
Funding Source
Reporting Frequency
Annual
Accession No.
1030621
Grant No.
2023-33530-39847
Cumulative Award Amt.
$174,982.00
Proposal No.
2023-00963
Multistate No.
(N/A)
Project Start Date
Jul 1, 2023
Project End Date
Feb 28, 2025
Grant Year
2023
Program Code
[8.13]- Plant Production and Protection-Engineering
Project Director
Hamilton, A. R.
Recipient Organization
FORGEBEE, LLC
805 S ELM BLVD
CHAMPAIGN,IL 61820
Performing Department
(N/A)
Non Technical Summary
Honey bees are crucial for agriculture and serve as a model system for insect pollinator ecotoxicology. At the heart of a honey bee colony's health and productivity is a set of complex interactions among the workers and the queen, determining egg production and the health of the queen's offspring. The bee health crisis demands that researchers be able to study these interactions and how they are influenced by nutrition, pathogens, parasites, and pesticides. However, progress in these areas has been stymied by the expensive, time-consuming, and seasonal nature of traditional apiculture and experimental techniques. Similar limitations also impact the screening of pesticides and other bioactive compounds on honey bee development, creating a severe bottleneck for agrochemical businesses seeking to create pollinator safe pesticides.We have developed the Queen Monitoring Cage (QMC), a published and patent-pending system for maintaining and monitoring queen honey bees in the lab. This system allows for dozens or even hundreds of queens (each with a small retinue of 50-200 workers) to be housed in a single incubator or similar enclosure and easily observed for health and egg laying behavior. Eggs can be assayed via a removable plate system, permitting researchers to quantify the impact of environmental factors on queen egg laying, a process that would normally require the use of dozens of field colonies. QMCs also allow for eggs to be harvested year-round from a highly controlled environment for downstream applications, such as studies on honey bee development or larval toxicology screening. The system thus has promise to facilitate basic and applied research in the academic, government, and industrial sectors.In Phase 1 of this SBIR grant, we will 1) create a machine vision algorithm for the automated identification and assessment of eggs in egg laying plates, 2) adapt the QMC system to existing techniques for the automated tracking of behaviors (including the exchange of food with the queen), and 3) create a semi-autonomous QMC thereby drastically decreasing maintenance time. These advances will drastically increase the throughput and scalability of the QMC system while enabling researchers to track mortality and egg laying linked to the flow of nutrients, pathogens, and other environmental factors through the worker bees and to the queen. We will optimize and rigorously test each component of this system to ensure it is robust, reliable, and highly effective.Once commercialized, the QMC system will provide end-users with a series of cost-effective and high throughput solutions to increase the scale of queen and larval research by orders of magnitude while enabling entirely novel experimental paradigms. The availability of this technology will therefore have dramatic ramifications for the long-term health of honey bees and other insect pollinators. By facilitating research into how nutrition, pathogens, parasites, pesticides and other factors impact queen honey bee health and fecundity, and larval survival and development, this system will help address priorities Strategic Goals 2 and 4 of the USDA's 2022-2026 Strategic Plan as well as goals in all of the five subject areas of the 2022 USDA Annual Strategic Pollinator Priorities and Goals Report.
Animal Health Component
10%
Research Effort Categories
Basic
10%
Applied
10%
Developmental
80%
Classification

Knowledge Area (KA)Subject of Investigation (SOI)Field of Science (FOS)Percent
30730101130100%
Knowledge Area
307 - Animal Management Systems;

Subject Of Investigation
3010 - Honey bees;

Field Of Science
1130 - Entomology and acarology;
Goals / Objectives
Upon completion of this grant, ForgeBee will have demonstrated: 1) the feasibility of automating honey bee egg detection, 2) queen-worker behavioral tracking in the queen monitoring cage system (QMC), and 3) semi-automated egg-laying plate removal from the QMC.Create a machine vision honey bee egg detector.Optimize the egg detector's parameters and quantify its performance using the QMC egg-laying plates.Achieve sensitivity, specificity, and positive and negative predictive values > 0.95Adapt a convolutional neural network (CNNs) to detect and quantify worker-queen food transfer.Determine whether the CNN behavioral detector be used with QMCs and Raspberry Pi® powered cameras.The retrained CNN will be able to predict food exchanges (including which bees are donating or receiving food) with a high degree of certainty that is comparable to previously published metrics (sensitivity > 0.8, specificity > 0.9, positive predictive value: > 0.9, negative predictive value > 0.9)Determine whether the QMC adversely affect queen-worker food exchange patterns.The observed interaction patterns of bees within the QMCs (number of interactions, interaction partners, etc.) should not deviate significantly from previously published data sets.Create a semi-automated QMC system by automating ELP removal to reduce maintenance time and facilitate egg collections.Create an affordable robotic system robust to high temperature/humidity operating conditions.The semi-automated QMC should have a failure rate <0.1% when operated continuously at 35C, 50% relative humidity.Automation of egg-laying plate removal should not adversely affect bee survival or queen egg-laying relative to manually operated QMCs.Queen egg-laying rate in semi-automated and manually operated QMCs should not be significantly different.
Project Methods
Create a machine vision honey bee egg detector.Honey bee eggs have a stereotyped color and shape. We will develop a honey bee egg detection algorithm that relies on a series of adaptive thresholds and filters for color, contrast, size, shape, and position to determine whether an egg is present within the boundaries of each cell.We will test the performance of this detector using no fewer than 200 egg-laying plates (each with up to 284 eggs) harvested from queen monitoring cages (QMCs).We will examine detector output for true/false positives and negatives.Evaluation: We will optimize the parameters until we achieve sensitivity, specificity, and positive and negative predictive values > 0.95Adapt a convolutional neural network (CNNs) to detect and quantify worker-queen food transfer.We will enable automated observation of the QMCs by designing a detachable camera mount that can house a Raspberry Pi® HQ camera module.We will determine whether an existing CNN behavioral detector be used with QMCs and Raspberry Pi® powered cameras.Image acquisition will be performed using a single queen and 100 worker bees, each tagged with unique barcodes, placed within QMCs and monitored for one week using the Raspberry Pi® HQ camera10 QMCs will be monitored each at the following sites (40 total): 1) University of Illinois at Urbana-Champaign, 2) the Southern Horticultural Research Lab, 3) the Invasive Species and Pollinator Health Unit in Davis, California, and 4) Honey Bee Breeding, Genetics and Physiology Lab)We will use collated data from these experiments (n=40 QMC units total, n=500 images total) to compare the CNN output to manual annotation, testing for true/false positive and negative behavioral predictions.If the CNN does not perform satisfactorily, the team will use transfer learning in TensorFlow to retrain it with 1,000 curated images from the above dataset.Evaluation: The (re)trained CNN will be able to predict food exchanges (including which bees are donating or receiving food) with a high degree of certainty that is comparable to previously published metrics (sensitivity > 0.8, specificity > 0.9, positive predictive value: > 0.9, negative predictive value > 0.9)We will determine whether the QMC adversely affects food exchange patterns by comparing CNN output to previously published data.We will directly compare the characteristics of individual bees in the group (number of interactions, interaction partners and interaction duration) using generalized linear mixed models with negative binomial distributions (as we have done previously).We will also examine emergent network properties, such as using epidemiological models to track speed of food exchangesEvaluation: The observed interaction patterns of bees within the QMCs should not deviate significantly from previously published data sets.As part of our collaborative agreements, we will also use these statistical techniques to analyze the effects on work-queen food exchange and egg-laying behavior of: 1) plant phytochemicals (see letter of collaboration from the Southern Horticultural Research Lab), 2) insect growth regulators (see letter of collaboration from Invasive Species and Pollinator Health Research Unit) and 3) queen genotype (see letter of collaboration from the Honey Bee Breeding, Genetics and Physiology Lab).Create a semi-automated QMC system by automating ELP removal to reduce maintenance time and facilitate egg collections.We will use CAD and FDM/SLA 3D printing to create a semi-automated QMC by employing an inexpensive (ca. $100) linear actuator controlled by a Raspberry Pi® (Figure 4).The team will use a previously designed and constructed printed circuit board to power the semi-automated QMC. The linear actuator will raise and lower the ELP based on commands from the Raspberry Pi® unit.For initial testing, each QMC will be operated by an independent linear actuator. Later we will expand to allow for the operation of up to 10 QMC units with one Raspberry Pi® and one linear actuator.Semi-automated QMCs (N = 5) will be continuously operated at 32.5C, 50% relative humidity (standard operating conditions for the QMC) for a period of one month to determine real-world failure rates and the mean time between failures.Failure mode and effects analysis will be used to analyze the data and maximize reliability.If the heat and humidity have a negative effect on the inexpensive electronics, we will test conformal coating of exposed parts, and the use of moisture resistant linear actuators.Evaluation: The semi-automated QMC should have a failure rate <0.1% under these conditions.We will optimize the speed of egg-laying removal to prevent worker mortality and queen disruption.Evaluation: We will directly compare worker mortality (using Cox Proportional Hazards tests), and queen egg laying (using generalized estimating equations) in semi-automated QMC to existing data from manually operated QMCs.

Progress 07/01/23 to 06/30/24

Outputs
Target Audience:Over the last year, ForgeBee has actively engaged with stakeholders in industry, the USDA, and at academic institutes. Industry stakeholders (predominately agrichemical companies, but also some bee-related startups) have exprssed strong in the technology we are developing as part of this grant. Indeed, we have entered into a collaborative research agreement with a major agrichemical company to develop similar techniques (that are compliant with OECD-245 guidelines) for testing the impact of pesticides on honey bee mortality. We have also continued to liase with USDA labs over the last year. We established MTRAs with ARS labs at the Pollinator Health Laboratory in Davis, California and the Thad Cochran Southern Horticultural Research Laboratory in Poplarville, Mississippi to facilitate the dissemination of the technology we are developing. The latter group recently submitted an article for publication that uses the Queen Monitoring Cage to determine the influence of a harmful chemical in Yellow Jessamine nectar on queen egg-laying. We have also completed a pilot study to test the feasibility of behavioral tracking in the Queen Monitoring Cage as part of our collaboration with the Pollinator Health Laboratory. Several labs (including at Arizona State University, Mississippi State University, and the Volcani Institute of Israel) have adopted the use of the Queen Monitoring Cage over the last year, and are using it to track changes in queen fecundity in response to insecticides, herbicides and disease. Changes/Problems:We have not been able to find a camera system compatible with the Raspberry Pi control system that meets the criteria of the project in terms of affordability, ease of use, and (most importantly) optical clarity under the physical limitations of the observation system (i.e. imaging of very close, fast moving objects under infrared lighting). This has led to significant delays in the completion of the grant's primary goals, and will require some reallocation of the existing budget. However, we have recently found a potential solution that involves abandoning the Raspberry Pi system for discrete camera modules that have the necessary characteristics. We believe that this solution will permit us to complete the deliverables as initially described. What opportunities for training and professional development has the project provided? Nothing Reported How have the results been disseminated to communities of interest? Nothing Reported What do you plan to do during the next reporting period to accomplish the goals?1. Given the satisfactory performance of the egg detector, we are now working to standardize the image capture procedure (using purpose-built scaffolds for the camera and egg laying plate), improve the usability of the egg detector, and incorporate size estimation. 2. We are developing and testing a camera system with a substantially better lens. Its performance under infrared lighting is far superior to the system we used previously, and will allow us to complete the aims specified under Objective 2. 3. We are proceeding to stress-test the egg retrieval system under conditions of high heat and humidity. If it performs satisfactorily, we will then determine the impact of automated retrieval on QMC performance (mortality and egg-laying rate).

Impacts
What was accomplished under these goals? 1. We have created a honey bee egg detector that uses a simple set of parameters to accurately predict the presence or absence of honey bee eggs. The performance of the egg detector is within the range specified by the objective. 2. We have designed and fabricated a specialized queen monitoring cage for behavioral tracking (tracking QMC). The cage is capable of housing 100 workers and a honey bee queen. It features a removable egg-laying plate to quantify queen fecundity, a built-in lighting system of infrared LEDs to provide both ambient and backlighting, interchangeable glass windows, and a camera scaffold to fix one or more cameras in place. Workers have an ca. 95% survival rate for 14 days within the tracking QMC. We conducted several pilot studies to assess the performance of the tracking QMC. This includes a collaboration with the Pollinator Health Laboratory in Davis California to assess the effect of insect growth regulators on honey bee queen fecundity. In brief, images are analyzed by two machine vision detectors. A 'barcode detector' uses a unique identifier glued to the back of each bee to identify the bees in the image, their position and their orientation with respect to one another. A 'trophallaxis detector' then uses this information to determine which bees are exchanging food with one another (otherwise known as trophallaxis). Unfortunately, analysis of the resulting imaging data revealed that the performance of the barcode detector is below the desired thresholds for sensitivity and negative predictive value (<0.6 in both instances). In brief, the image quality was insufficient for the barcode detector to identify the barcode as reliably as we require. We were able to determine that this deficit in image quality was due to the quality of the lens used on the camera, and its properties under infrared light (performance was adequate under visible light). When a barcode was detected, however, it was identified correctly >90% of the time, and false positives (the detection of a barcode that did not exist) were virtually nonexistent. Similarly, the tophallaxis detector functioned as we predicted and correctly identified >90% of the trophallaxis events in instances where the barcode was correctly detected with functionally negligible (<5%) false positives. This suggests that once we can solve the issues related to barcode detection, the remainder of the analysis pipeline is likely to perform adequately. 3. We have designed a robotic system for the retrieval of egg-laying plates consisting of a linear actuator, a Raspberry Pi control unit, and 3D printed attachments to connect the actuator to a series of QMCs.

Publications