Recipient Organization
FIELD DATA SERVICES LLC
454 PARMA DR
ESSEX,MT 599169730
Performing Department
(N/A)
Non Technical Summary
Producers need to measure pollinator population size, distribution and effectiveness in numerous remote locations. Ecosystems involving commercial bees, feral bees and native pollinators along with weather, climate and other environmental factors are complex. Pollination effectiveness within these complexities can be a key component of crop yields and profitability. Understanding this complexity requires new tools for producers that reveal which species are present, in what abundance, and which specific portions of cropland could benefit from modified pollinator management practices. Beyond the immediate needs of producers are the educators, students, citizen scientists and agency land managers who have all taken great interest in pollinator health and what they can do as individuals to enhance the habitats and systems that support pollinators. This Phase I SBIR project will determine the technical feasibility of using a newly available LiDAR on a chip (LoC) to detect insect pollinators and characterize pollination events in the field.Numerous teams are working hard to apply AI-powered machine vision technologies to bear on identifying pollinators and other wildlife. The problem with these technology-intensive devices is that they are expensive, require huge batteries and high bandwidth internet connectivity. The way to make AI-powered devices affordable and practical for field use is to use a micro-powered motion detector to turn on the AI power only after a moving target is detected. Commercial trail cameras use a passive infrared (PIR) sensor for this purpose. The problem with a PIR sensor is that it will trigger only if the moving object is of a different temperature than the background, like a large warm-blooded animal moving across a cooler background. In October 2020 STMicroelectronics introduced the first commercialized multi-zoned rangefinder which is essentially a small LiDAR on a tiny $9 computer chip. This sensor does not require a temperature differential to detect motion. Our team will be testing the ability of this new sensor to detect small moving objects under a variety of outdoor conditions as a way to produce a small, affordable, micro-powered device for detecting and monitoring pollination events in the field.
Animal Health Component
90%
Research Effort Categories
Basic
5%
Applied
90%
Developmental
5%
Goals / Objectives
This Phase I SBIR project will determine the technical feasibility of using a newly available LiDAR on a chip (LoC) to detect insect pollinators in the field. There are substantial commercial and ecological needs to detect, quantify and characterize the effectiveness of pollinators on crops and critical wild plants. Current time-consuming methods that rely on manual observations are being augmented by powerful new machine vision, machine learning and other AI powered technologies. As promising as these new image-based technologies are, they are limited by the cost, complexity and electrical power required to bring the power of AI-enhanced computer vision to remotely located agricultural lands.If a producer wants to respond to a pollination deficit in time to preserve adequate yields, the producer needs to know the current state of pollination in almost real time during critically short blooms. Ineffective commercial hives can be mitigated on the spot, and building a record of year-to-year effectiveness of commercial, feral and native pollinators can help formulate site-specific pollinator management plans that account for weather and local pollinator habitat conditions.Leveraging rapidly evolving computer vision technologies to meet these producer needs will require new low-cost and low-power ways to bring these computer technologies to all corners of remote croplands. Running power-hungry computer vision cameras full-time in the field will fail to meet the logistical constraints of real world food production. Commercial camera traps, aka trail cameras, are designed to acquire images of critical events without the need for huge batteries, solar panels and cloud connectivity. Camera traps achieve this by leaving the power-hungry image sensor turned off most of the time. Only one small micro-powered electrical component is left turned on and searching for movement. This small component is a passive infrared (PIR) sensor. Only after the PIR sensor detects motion is the power-hungry image sensor turned on to acquire images. After the images are acquired and stored the power-hungry image sensor and main processor are again turned off to save battery power.A similar approach will be taken by this project in order to increase the feasibility of using computer vision technologies for monitoring pollinators. Micro-powered PIR sensors that are used in trail cameras would solve the power supply issue except for the fact that PIR sensors are only triggered when the moving target presents a temperature differential with a static background. Camera traps work well on large warm-blooded animals moving against a cooler background but PIR sensors are not always effective for detecting flying insects. PIR sensors are also susceptible to false triggers from sunlight reflecting off of vegetation that is moving in the wind.Another way to detect a moving object is using the image sensor enhanced by onboard digital imaging processing techniques. Onboard image processing can acquire a series of baseline images to determine if a new object has moved into the field of view. This process is called background subtraction because when a new image is taken, the pixels are subtracted from the previously acquired baseline images, pixel by pixel, to identify clusters of pixels that are new and different from the baseline. This method of detecting moving objects does not require a temperature differential of the moving target, but it has two disadvantages: 1) Image background subtraction requires that the power-hungry image sensor be turned on almost continuously, and 2) background subtraction is susceptible to false triggers from background vegetation moving in the wind.This project will evaluate a new and inexpensive electrical component for detecting insect pollinators that might replace or augment PIRs and image background subtraction for achieving cost effective monitoring of remote pollination events. In late 2020 STMicroelectronics introduced the first commercialized multi-zone rangefinder on a chip. This micro-powered $9 chip produces an 8x8 grid of LiDAR distance measurements. This grid of accurate distance measurements is achieved by emitting an 8x8 grid of tiny invisible infrared laser pulses. The chip measures the time between emitting a pulse and the return of reflected pulses. From these measurements the chip accurately measures the distance (within a couple of millimeters) to the nearest object within each of the 64 zones in its field of view.This LoC sensor has a field of view of about 45 degrees, and it can measure distances of up to several meters. This field of view and measurement capability is well-suited for detecting medium to large insect pollinators. An LoC sensor uses slightly more battery power than a PIR sensor but it uses orders-of-magnitude less battery power than operating an image sensor full time for background subtraction and image object recognition.The enticing potential of leveraging LoCs for monitoring pollinators begs further investigation. However, being a new and undocumented application for LoCs our team must first conduct substantial feasibility testing. The goal of this Phase I project is to measure and determine the envelope of conditions for which LoCs can be used to solve the problem of cost-effective use of machine vision technologies for monitoring insect pollination of agricultural crops and critical wild plants.To determine the envelope of conditions under which LoCs will be effective. An automated test system will be built that will automatically move variously sized targets past through an LoC sensor's field of view. The automated test systems will compile extensive and precise tabulations of positive detections and false negative and false positives. These test data will be taken while modifying target reflectivity, speed, size and distance from the sensor. Background conditions will also be manipulated and tested including moving vegetation and over a range of realistic outdoor lighting conditions.The goal of these tests will be to narrow and carefully define the range of insect characteristics and conditions for which LoCs will be effective. From this information an optimized device can be designed and evaluated for commercial potential and Phase II project feasibility. The following objectives were developed to meet the project goals:1) Identify and mitigate crosstalk noise that might affect LoC performance. This objective involves determination of electronic noise impacts on LoC performance from other electronic subsystems in a smart camera system such as high-speed clocks, USB, GPS and main processor.2) Measure and mitigate unit to unit variation of LoC sensors. LoC sensors being a newly commercialized component it is prudent to look for unit-to-unit variation, and if found, mitigate it by screening components or other measures.3) Experimentally verify that insect detection accuracy with LoC or an LoC plus smart camera (LoC+Cam) system is as good or better than existing machine vision devices.4) Experimentally verify that honey bee sized insect pollinators can be reliably detected when within a specified range of size, flying speeds and distances.5) Experimentally verify that software analysis of LoC and LoC+Cam sensor data can distinguish between major insect taxa.6) Experimentally verify end-to-end system performance of battery, data acquisition and analysis appropriate for remote field deployments.7) Experimentally verify that software analysis of LoC and LoC+Cam sensor data can detect insect pollination events.8)Resources permitting, we can address two stretch goals: A) determine whether an array of LoCs is substantially better than a single LoC unit. And B) determine if LoC assisted focus of a high-end camera board can improve detection of species and behaviors.
Project Methods
Technical feasibility testing will be narrowly and intensely focused on characterizing the ability of a LiDAR on a chip (LoC) to detect and monitor insect pollination in the field. Toward this end, a computer controlled test system will be designed and constructed that will move a simulated insect across the LoC's field of view at various speeds and distances. The computer will compile numerous instances of detections, false negatives and false positives under the varying conditions. Conditions that will be varied will include the size, reflectivity and wing orientation of the simulated insects, the speed and distance that the target moves across the field of view, the lighting conditions, and the type and movement of the background. The compiled results will be analyzed using common Gauge R&R statistical methods to determine the reliability of LoCs at detecting insect pollinators under the various conditions. Results from the following test and measurement efforts will be evaluated for their impacts on product design, partner outreach and initial commercialization strategies:1)System level electrical crosstalk that might degrade LoC sensor performance.2)LoC unit to unit variation that might require mitigation to achieve reliable product performance.3)LoC detectability of three categories of pollinator sizes: honey bees, bumble bees and moths/butterflies.4)End to end performance of LoC and LoC plus smart camera systems for detecting pollination events in terms of field robustness, battery power consumption and detection efficacy.5)Comparison of data usefulness between devices that employ only LoCs, LoCs plus a PIR, and LoCs plus a smart camera.The goal of these experiments will be to de-risk a commercial launch in 2023-2024. De-risking commercialization requires optimizing the design and target applications armed with a detailed understanding of the technical capability of LoCs to detect insects. Risk associated with commercialization can be minimized by targeting the mechanical design of the device for optimized distances to targets, sun shading and orientation. Detection efficacy can be optimized by developing the firmware that will extract insect detections from background noise. Optimizing and understanding the product's design and constraints will inform our team choice of field applications with the highest chances of success. Determining the envelope of targets and conditions will alter our team's technical directions, target audiences and potential customers.The spreadsheets of statistically analyzed detections will provide a breakdown of size, speed, distance, lighting and backgrounds. From these data we will extract detection reliability at, for instance, 50, 75 and 90% over size, speed, distance and background. From there we can analyze detectability over three broad categories of targets - honeybees, bumble bees and moths/butterflies. Armed with the detectability of these three categories our team can overlay pollinator dependent crops that could support initial beta testing and outreach during early commercialization. Using publicly available data on the market sizes of pollinator dependent crops our team can narrow our focus toward the market segments that will be most conducive to a successful launch. The above discussion of target audiences will also be an important part of the analysis. Outreach efforts with the Xerces Society and the complimentary pollinator projects at Montana State, Penn State, Kansas State, and UC Santa Barbara and UC Davis will all provide valuable inputs into market launch strategies.Overall, the methods and analyses of the LoC technical performance will be straightforward - simple statistical analyses of pass/fail detections over conditions. The automated nature of data acquisition will lend itself to large sample sizes and robust statistical confidence.