Progress 09/01/16 to 02/28/21
Outputs
Target Audience:Work completed during this Phase II project served at least two target audiences. ISTI served the USDA as the primary stakeholder by completing the development, prototyping, and extensive testing of the dual-sensory species-specific recognition system for automated bait delivery. Our Phase II collaborative research with our counterparts at the National Wildlife Research Council (NWRC) APHIS laboratory led to the development, prototyping, and field testing of SenseHogTM Dual-Sensory Species Specific Recognition product that supports the USDA's efforts in humanely and effectively protecting agricultural interests and livestock safety in the United States from the ecological and economic damages caused by sus scrofa or feral swine infestations. Clearly, once the completed SenseHogTM products are available in the market, farm and land owners are the ultimate beneficiaries of the outcome of this research. This research has also served a small audience in the educational community. Specifically, we initiated four internship positions for four undergraduate mechanical engineering students, each taking on a different challenging design task as the hardware of the dual-sensory systems matured. Some of these tasks included: integration of the new generation bait-box with the existing audio-only system early in Phase II, outfitting of new generation bait boxes with latching solutions, and external enclosures, and generating and testing CAD models throughout hardware revisions. These internships further advanced the students' ability to work in an interdisciplinary team and at the same time understand and effectively communicate issues and decisions that would impact multiple engineering disciplines involved with the design. Finally, our public-domain annotated vocalization databases will be of great benefit to larger research and educational communities working on similar problems. Changes/Problems:During the spring of 2020 work on the SenseHog™ project was significantly slowed for a period of about 3 months with the loss in productivity beginning early in March 2020 and ending towards the end of May 2020. The primary reasons for this lapse in efforts was the mandated quarantine of Larimer County in response to the COVID-19 pandemic. Due to state-mandated social distancing policies, employees at Information Systems Technologies Inc. werehindered in their ability to carry out the following essential tasks: Meeting with USDA-APHIS employees to retrieve equipment with valuable data required to carry out analysis of the recent field deployment in Motley County, Texas Working at ISTI's office which contains essential data and servers Arranging a final field test to ensure performance of systems in presence of non-targeted animals namely Bears Meeting with USDA-APHIS employees to finalize the non-provisional patent for our developed technology And preparing the final report for this project. Due to these special circumstances, ISTI made a request for a 6-month No-Cost Extension (NCE) for the current award so that the work that remains to be completed could be carried out on a more realistic timeline. This request was approved on May 19th of 2020 and extended the project end date to February 28th of 2021. What opportunities for training and professional development has the project provided?Training and professional development opportunities in this project were provided through the participation of four undergraduate mechanical engineering interns from Colorado State University. During these internship appointments, the mechanical engineers were tasked with a number of challenging problems related to the robust mating of the new generation bait-box with the pre-existing SenseHog Prototype and hardware integration of the full dual-sensory prototype. The training activities included providing necessary guidance during all phases of design and the opportunity to lead the mechanical integration design by creating computer models of the system, running simulations on modeled systems, selecting parts, building parts lists, and overseeing prototype fabrication. These interns worked on a part-time basis on all assembly and testing activities during the entire course of the project. In particular, the first intern was appointed to solve the problem of outfitting the new generation bait-boxes with electromechanical latches before the bear resistance testing in Washington. In addition to modeling a partial assembly and running simulation to determine the optimal latch and striker bolt placement, this intern outfitted 5 bait-boxes with 4 latches and striker bolts each before our deployment in Cathlamet. The second of these interns continued refining the mating solution. This intern revised and greatly improved the partial assembly modeled by the previous intern and converted to a more common .prt file. This intern designed the final CAD that we work with and his attention to detail was apparent. This intern was also responsible for developing a cable-routing solution using conduit. Lastly, this intern undertook the selection, purchase, and installation of all the parts for mating the transceiver enclosure with the new generation bait-boxes. The third intern assisted with latch re-position that was required after ingress issues arose in Vernon, TX. This intern was responsible for researching and selecting trail cam enclosures for housing the sensors and Embedded Computing Platform of the intelligent engine. This intern was also responsible for assisting in the retro-fitting of the dual-sensory systems to the new trail-cam enclosures. The fourth and most recent intern assisted with permanently affixing the transceiver enclosure to the bait-box units after a unit was damaged during deployment in Jamestown TN. This intern also assisted in fine-tuning the placement of latching components so that the lid-mounted latches interface more smoothly with the striker bolts among many other improvements in preparation for the February and July deployments in Motley County, TX. The PI, Mr. Jack Hall, also attended several presentations and seminars on machine learning at Colorado State University as well as seminars held at NWRC on the topic of automated animal identification from trail cam footage. These seminars were influential in the early developments of the visual subsystem. How have the results been disseminated to communities of interest?Throughout the course of this project we have worked closely with our collaborators at NWRC APHIS to evaluate and guide the progress made during every phase of this project. This has included regular in-person meetings with the APHIS project staff on numerous occasions to discuss experiment design for deployments, timetables, current results, and technical issues that arise through our investigations. We have also begun preparing a journal paper to disseminate the results of this research to the relevant communities. This document will provide technical analysis of systems performance in addition to commentary from APHIS partners on the relevant ecosystems, applications, and best-practice for effective usage of our automated species-specific recognition system. All software and datasets collected as part of this research will be made available online via our website and a private BitBucket repository. What do you plan to do during the next reporting period to accomplish the goals?As this project comes to an end, ISTI plans to continue marketing efforts. Several partners that we have met through the course of this research have expressed interest in utilizing our technology with their baiting and trapping systems. After securing full protections for our system, ISTI will continue open-door discussions with these partners. Project Summary: Over the course of this project, ISTI undertook the proposed work of designing a novel species-specific recognition system from the ground up. Building from the Phase I audio-only prototype, a field-ready dual-sensory inference system and bait-box were refined and regularly tested in realistic environments, leading to a robust and reliable inference system for triggering traps and baiting devices for targeted animals (feral swine in this case). The developed system utilizes dual sensory modes to arrive at high confidence decisions in real-time using both audio and video stream inputs. The developed system balances energy use and decision confidence in a hierarchical inference approach, first crudely interrogating the scene with motion only, then interrogating with audio inferences, and finally interrogating with both audio and video once the audio subsystem has gained sufficient evidence from the scene. Through numerous experiments and realistic field deployments of the system, we have demonstrated that such a system can be implemented for relatively low cost on a single embedded computing platform, and can greatly reduce the man-hours required to manage an invading population of feral swine. Due to the stringent requirements for non-target exclusion, a major focus of the Phase II developments was on the field testing and evaluation of prototypes in realistic deployment environments. To this end, over the course of the system's revisions ISTI and NWRC-APHIS conducted 7 field deployments and many more in-lab experiments to validate the individual subsystems and their various compositions (e.g., Audio Only, Visual only, Audio-Only with bait-box, Dual-Sensory with bait-box). As of the most recent deployments, the dual-sensory systems have performed very well, providing the targeted species an acceptable access rate while completely rejecting competing non-targets species e.g., bears. Although our prototypes performed exceedingly well thus far, we believe that further testing is warranted before large-scale deployment in areas with high-risk non-targets (e.g., American Black Bears) can be carried out. The usage of region-specific training data is critical to the systems proper operation and, accordingly, when systems are deployed in a new environment, proper precautions should be taken to adapt or replace the inference models so as to match typical observations from the new regions. In order to reproduce a system which discriminates between targeted and non-targeted species with high accuracy, there are several critical requirements that must be satisfied: 1) Collection of large region specific databases for audio time series and visual frames. Collection devices should be co-located during data collection and should model the approximate relative geometry that the trained system will encounter in its intended deployment environment. 2) Adherence to proper deployment protocol, ensuring the relative placement of the trail-cam and brain-unit remains consistent with the placement seen in model training. 3) Adherence to proper pre-baiting protocol so as to increase effectiveness and provide validation of deployed inference models prior to full deployment with actual toxicant/contraceptives or arming another device. Lastly, in future efforts to continue refining this system, the next logical step would be to design a custom PCB shield for Raspberry Pi which has the required sensors, i.e. camera, microphone and PIR motion, embedded in the shield design. Following this PCB development, a custom trail-camera enclosure could be purchased, or 3D printed, to house the Raspberry Pi + SenseHogTM shield stack. In this way, the SenseHog shield and enclosure could be distributed, with software publicly available, so users could individually interface the species recognition system with their pre-existing traps. The same form factor would allow for easy adoption and integration with a number of pre-existing technologies from potential future partners.
Impacts
What was accomplished under these goals?
Goal 1: The first major task of goal 1 for this Phase II project was concerned with specific objective 2: mating of the new bait-box with a robust electromechanical latching system. Considerable research went into finding the optimal placement and ultimately machining brackets and blocks for the mounting of these latches. After issues surfaced during a deployment in Vernon TX, in February and March of 2018, we changed the latch orientations on the new generation bait box. The second major task of Phase II was concerned with specific objectives 1 and 3 i.e. finalizing the audio subsystemSenseHog™ Ear. After testing with the BeagleBone Wireless platform was performed in Vernon TX, preliminary results and concerns about platform support warranted investigating other embedded platforms. After considerable research and testing of a number of classifiers, to provide more efficient processing and broader system support, ISTI ultimately chose to convert to the Raspberry Pi 3 Model B+ embedded platform and implemented two Convolutional Neural Network (CNN) based classifiers one for audio and one for visual channels. The third major task, related to specific objectives 4 and 5, was the development of the companion visual subsystem referred to asSenseHog™ Eye. As mentioned above, after testing several classifiers in conjunction with optical flow pre-processing, a CNN-based visual classifier was ultimately chosen. Using the abundant training data from field deployments and other trail cam footage provided by our collaborators at APHIS, this visual classifier was trained and tested extensively both in the field and in lab testing. The fourth major task, related to specific objectives 6 and 7, was the coupling of visual and audio subsystem statistics in order to make high confidence unlock decisions. After thorough testing of the audio subsystem and preliminary testing of the visual subsystem, in the summer and Fall of 2018, we began developing a fusion mechanism that efficiently utilized the streamed visual and audio statistics to make the final unlock decision. In the 2021 final technical report (submitted separately to the program manager), we overview the major developments that pertained to Goal 1 over the course of this multi-year project and discuss how each one of the specific objectives were addressed in much greater detail. Goal 2: In order to meet the specific objectives of Goal 2, comprehensive testing has been conducted all along the course of the system's development since the start of the project. Throughout all of these experiments and field deployments, in serving specific objective 1, large audio and video databases were collected, containing a diverse range of vocalizations and visual evidence of target and non-target species considered in this application. Acquisition of such a database is a critical requirement for any technology that seeks to achieve confusion-free species recognition. These databases have proven to be extremely valuable for use in refining the accuracy of both the visual and audio subsystems. In serving specific objectives 2 and 3, extensive field and lab testing of the individual subsystems and the dual-sensory prototype was conducted. Beginning with Phase II audio-only prototype testing, after each of the revisions of the prototype, described in the previous section under Goal 1, testing was performed in order to evaluate the effectiveness of the design changes. Bear entry resistance Studies were conducted in Cathlamet/Naselle after mating of the latching mechanism with new generation bait-box. Extensive audio-only testing was conducted both in Vernon, TX and using penned pigs at APHIS facilities in Fort Collins. After the addition of the companion visual subsystem, extensive false alarm rejection tests were performed in Tennessee and Alabama. This testing led to more modifications and finally two extensive tests were conducted in February and July of 2020 in Texas.The results collected from each of these field testing opportunities allowed ISTI to further refine the system at every stage of development and was fundamental to the development of these technologies.In the 2021 Final Technical report (submitted separately to the program manager), we overview the major developments that pertained to Goal 2 over the course of this Phase II project. The field testing efforts related to Goal 2 are discussed in further detail in Section 6 of the Final Technical Report. In light of these successful deployments and extensive data collection efforts achieved to date, no objectives remain for this Goal. Goal 3: Early Phase II experiments and prototyping primarily served the 1st, 4th, and 5th specific objectives of Goal 3. In collaboration with NWRC-APHIS, our experiments and discussions led to several critical revisions to the prototype which greatly improved the system's effectiveness. During our collaborative experiments, ISTI had many opportunities to receive feedback from not only the field biologists working alongside us during our field deployments, but also the land-owners who agreed to let us test our prototype systems on their property. In serving specific objectives 2 and 3, in 2020 ISTIfiled a provisional patent for the technologies developed over the course of this research. Furthermore, in the most recent reporting period ISTI prepared documentation with NWRC-APHIS and their partners in order to file a non-provisional patent application with the US Patent and Trademark Office (USPTO) to secure protection for the developed technologies. This non-provisional patent was submitted on the 15th of April, 2021 with reference no. USPTO Filing 17/230,453 - Intelligent Dual Sensory Species-Specific Recognition Trigger System. It must be pointed out that, specific objectives 4 and 5 were only partially addressed during this effort. The main reason beingduring the first 4 years of this effort, we were mainly concerned with the development and extensive testing of a robust system with high-confidence decisions in orderto provide a high quality product. However, during early 2020, the time frame when the product and technology had become mature enough for marketing, hosting of such technical demonstrations to larger audiences and meeting with potential end-users became infeasible given state mandated travel and gathering restrictions due to COVID pandemic.Nevertheless, we shall work closely with our collaborators at NWRC-APHIS to team up with bait box manufacturing companies e.g., ACTA, JagerPro,and BoarBuster, to offer to the market a complete and affordable multi-sensory species specific animal recognition system.
Publications
|
Progress 09/01/19 to 08/31/20
Outputs
Target Audience:The work completed during this reporting period primarily served our target audience. ISTI served the USDA as the primary stakeholder by continuing development of the algorithms and procedures supporting the capability of automated species identification using acoustic recordings and live trap camera photos. The collaborative research with our teammates at the NWRC-APHIS laboratory will lead to products that support the USDA's efforts in humanely and effectively protecting agricultural interests in the United States from the ecological and economic damages caused by feral swine infestations.The work completed during this reporting period primarily served our target audience. ISTI served the USDA as the primary stakeholder by continuing development of the algorithms and procedures supporting the capability of automated species identification using acoustic recordings and live trap camera photos. The collaborative research with our teammates at the NWRC-APHIS laboratory will lead to products that support the USDA's efforts in humanely and effectively protecting agricultural interests in the United States from the ecological and economic damages caused by feral swine infestations. Changes/Problems:During the spring of 2020 work on the SenseHog™ project was significantly slowed for a period of about 3 months with the loss in productivity beginning early in March 2020 and ending towards the end of May 2020. The primary reasons for this lapse in efforts was the mandated quarantine of Larimer County in response to the COVID-19 pandemic. Due to state-mandated social distancing policies, employees at Information Systems Technologies Inc. were been hindered in their ability to carry out the following essential tasks: Meeting with USDA-APHIS employees to retrieve equipment with valuable data required to carry out analysis of the recent field deployment in Motley County, TexasHallH Working at ISTI's office which contains essential data and servers Arranging a final field test to ensure performance of systems in presence of non-targeted animals namely Bears Meeting with USDA-APHIS employees to finalize the non-provisional patent for our developed technology And preparing the final report for this project. Due to these special circumstances, ISTI made a request for a 6-month No-Cost Extension (NCE) for the current award so that the work that remains to be completed could be carried out on a more realistic timeline. This request was approved on May 19th of 2020 and extends the project end date to February 28th of 2021. What opportunities for training and professional development has the project provided?
Nothing Reported
How have the results been disseminated to communities of interest?ISTI has kept the collaborators at NWRC-APHIS abreast of the progress made during every phase of this project and continue to meet in person with the APHIS project staff regularly to present current results, and discuss technical issues being investigated. As part of this Phase II project, we intend to submit two journal papers to related journals in the field in order to disseminate the results of this research. The first paper is currently being written but will not be submitted for publication until the non-provisional patent application has been submitted. The second journal publication will be prepared once this patent application has been approved and will discuss the algorithms, and system architecture in much greater detail. What do you plan to do during the next reporting period to accomplish the goals?Moving forward with the project goals, in the next reporting period ISTI plans to accomplish all remaining tasks for ?nalizing the SenseHog™ products and intends to focus more concertedly on marketing e?orts with the hopes of having a legally-protected, state-of-the-art, species management device with an eager addressable market at the conclusion of this Phase II research e?ort. Namely, we intend to: (a) Finalize work on the setup application being designed for SenseHog™ system. (b) Finalize and submit the Non-Provisional patent application with deliverables prepared in collaboration with NWRC-APHIS. (c) Publication of first journal paper in collaboration with NWRC-APHIS which documents the system development and its preliminary performance in realistic field deployments. (d) Schedule a ?nal deployment experiment with NWRC -APHIS which demonstrates false-alarm resistance and detection sensitivity capabilities of the system. (e) Expand e?orts to reach targeted domestic market both directly through our website and through licensed industry partners. Direct sales will primarily serve customers needing smaller installations, and indirect sales with an industry partner network is also crucial for getting our foot in the door of large-scale projects or other vertical markets where our system can be sold as an added feature to enhance other technologies e.g., bait box or trapping device capabilities.
Impacts
What was accomplished under these goals?
Goal 1: Throughout this reporting period, Goal 1 has been served primarily via specific objectives 4, 6 and 7. More specifically, Objective 4 focused on the refinement and testing of the visual-based, subsystem, referred to as SenseHog™ Eye, and investigated methods for feature extraction and classification which yield optimal performance. Extensive lab testing and the results and data from field deployments in the previous reporting period revealed shortcomings of the SenseHog™ Eye prototype. In particular, issues arose with optical flow pre-processing of the visual subsystem as well as the operation of the subsystem's night-time mode. Several software and one major hardware change had to be implemented during this reporting period to address the night-mode capture and visual inference issues. Subsequent field testing of the dual sensory system demonstrated that the modifications successfully remedied the optical flow bounding and night-mode inference issues. Objective 6, which involves the integration of the audio and visual subsystems (hardware and software) with the new generation bait box, also saw significant development over this reporting period. In particular, improvements to motion sensitivity, audio fidelity, and night-time image fidelity were made. These modifications drastically improved both motion sensitivity and accuracy of the visual subsystem. Additionally, a simple mechanism for selecting pre-set threshold settings was implemented during this period. These pre-sets are biased to allow either fewer missed detections or fewer false alarms, with a third "knee-point" pre-set which balances this tradeoff. The latter is based upon the knee-point of the receiver operating characteristic (ROC) curve generated using prior datasets. These improvements directly impact the accuracy of the two inference subsystems and simplify deployment especially for non-experts who will be setting up the systems in the field. Finally, as part of Objective 7 ISTI has continued conducting comprehensive testing of the dual-sensory SenseHog™ prototype through field deployments conducted in collaboration with our collaborators at NWRC-APHIS. These tests have provided valuable feedback on the detection and false-alarm sensitivities of the prototype systems by exposing them to different types of animals and interference typical of the regions where the systems are intended to be deployed. In this reporting period two pig field tests were conducted. The first of these was a 5-day field deployment of two prototype systems in Motley County, TX during February and March of 2020. The second test, also conducted in Motley County, TX, was an extensive 10 day trial conducted over 12-days during July of 2020. These deployments provided a variety of realistic deployment site conditions along with an abundance of both target and non-target species interaction. Furthermore, these field deployments indirectly served Goal 3's specific Objective 3 by providing previously unconsidered scenarios which greatly improved deployment protocol for the systems. Goal 2: ISTI continues to work closely with NWRC-APHIS collaborators to conduct field studies which serve Objectives 1 -3 of this project goal. These experiments have enabled the collection of a large database of both acoustic samples and trail camera photos that include acoustic recordings and sightings of different feral swine species, bears, deer, foxes, birds, and other common sources of wildlife typical for a ranching ecosystem. Such a database is required to train a dual-sensory detector-classifier that remains robust to a wide range of interference sources and environments/operating conditions. Furthermore, the experiments have provided many hours of realistic testing to demonstrate the robustness and effectiveness of the system in a variety of environments. Since the last reporting period, two more field experiments that emulate a realistic deployment have been conducted in collaboration with NWRC-APHIS. These experiments were conducted during February and July of 2020, and both took place in Motley County, TX, a region where several major ranches struggle to control pestilent feral swine populations. Unfortunately, due to COVID-19 the bear filed test which was planned for early Spring 2020 could not be conducted. The NCE allows us to postpone this important test to Spring 2021. The recent field deployments in Texas provided an abundance of novel environments and testing conditions that the dual-sensory prototype had never been exposed to. Testing conducted in Motley County primarily served to test the missed detection resistance capabilities of the system as the most common species to visit the baited sites were Sus Scrofa. Additionally, data collected from this deployment has expanded the database of target and non-target (bear excluded) class trail-cam footage. During the most recent Texas deployment, two SenseHog™ systems were exposed to 10 different sites for 2 nights in each site. This field experiment not only provided feedback on the operation of the prototypes in a variety of environmental conditions, with regular exposure to the targeted species, but also expanded the visual and audio databases for further training and testing of the classification subsystems, if need be. Goal 3: To address the third major goal concerning the delivery of a rugged, low-cost, and low-power completed SenseHog™ system to APHIS, ISTI has accomplished several important milestones in both the development of the system's core technologies as well as the preparation of proper documentation for a non-provisional patent application and operational instructions. During this reporting period, this goal was primarily served via specific Objectives 1, 2, and 3. Specific Objectives 4 and 5 were not served primarily due to COVID-19 issues and concerns. Objective 1 focuses on the collaborative efforts of ISTI and NWRC-APHIS to design a product that is both effective at administering feed to only the intended species, and is relatively simple to use and robust enough for long-term use and deployments. During this reporting period, several changes to not only the deployment protocol, but the inference systems' parameters were made in order to simplify the deployment of systems. In particular, the inference system was modified to accommodate three threshold pre-sets which are configured to 1) resist missed detections, for deployment in areas with lower likelihood of bear presence; 2) resist false alarms, for deployment in areas with higher likelihood of bear presence; and 3) a knee-point threshold pre-set which balances the tradeoff between the false alarm rate and probability of correct detection/classification. In serving specific Objective 2, ISTI has documented operational instructions which simplify deployment and troubleshooting of the systems in the field. ISTI has continued development of a mobile application which further simplifies the system deployment process by automating several tasks required to ensure proper camera placement, audio levels, and storage capacity of the SenseHog™ Brain's embedded processor. Additionally, ISTI and NWRC-APHIS has filed a new joint provisional patent to protect the intellectual property. A non-provisional patent is being prepared for submission before December 2020 which outlines the implementation details for the acoustic and visual subsystems of the SenseHog™ system along with the form and function of the full SenseHog™ system. [continued in full document]
Publications
|
Progress 09/01/18 to 08/31/19
Outputs
Target Audience:The work completed during this reporting period served two target audiences. First, ISTI served the USDA as the primary stakeholder by continuing development of the algorithms supporting the capability of automated species identification using acoustic recordings and live trap camera photos. The collaborative research with our teammates at the NWRC-APHIS laboratory will lead to products that support the USDA's efforts in humanely and effectively protecting agricultural interests in the United States from the ecological and economic damages caused by feral swine infestations. The grant has also served a small audience in the educational community. Specifically, we initiated an internship position for a university mechanical engineering student to lead the mechanical design work pertaining to the revisions made to the feeder-box itself following the May 2019 field test in Jamestown. The student hired was a junior at Colorado State University in Fort Collins. While not a primary audience to be served by the final product of which this project is concerned, the internship program allowed this student to receive hands-on training related to their field. The internship further advanced this students ability to work in an interdisciplinary team while recognizing and effectively communicating issues and decisions that would impact multiple engineering disciplines involved with the design. This student was hired in the spring and worked through the summer of 2019 and will be assisting with future modifications of the system through Fall 2019. Changes/Problems:From the data collected and observations made during the May and August 2019 field tests, ISTI was able to identify three major issues that precluded ideal operation of the SenseHog™ dual-sensory prototypes. The first major issue that was revealed from testing in Jamestown related to the form factor of the bait-box and transceiver enclosure pair. After analyzing damage encountered, ISTI determined that the ability to remove transceiver and bait-box components from one another was a shortcoming of the design, requiring operators to re-attach the components with every deployment, inevitably connecting and disconnecting latch cable ends repeatedly. When separated, the delicate cable ends of the bait-box were exposed and this led to severing of cable ends when transporting the separated components. The second major issue was discovered during the Hardaway field deployment and related to issues with PIR motion sensitivity. ISTI has determined this problem is due to two factors, the distance of the PIR sensor from the trail-cam's embedded lens and high internal temperature of the brain units making change detection more difficult in higher temperature ambient environments. Third major issue was image-segmentation failures due to improper camera exposure and capture rate settings. This failure is attributed to an oversight in software design and a lack of testing of image segmentation in night-mode operation. Images captured during night-time operation in Hardaway demonstrated that the exposure settings selected during testing at ISTI's facilities resulted in photos that were far too exposed when the full exposure period was used. Additionally, since both the exposure period and capture rate were set to ?xed values by the visual subsystem, rapidly capturing consecutive images would sometimes truncate the exposure period resulting in a severely underexposed image. The combination of these two oversights resulted in inadequate images for optical ?ow processing which in turn produced ROIs that did not capture individual moving targets but rather repeatedly captured the entire scene. Failure of the image segmentation pre-processing (exposure/capture rate settings were not dynamic and were too high) resulted in missed detections in the visual subsystem. What opportunities for training and professional development has the project provided?Training and professional development opportunities were provided in this reporting period through the participation of an undergraduate mechanical engineering intern on the project. The training activities included providing necessary guidance during all phases of design and the opportunity to lead the mechanical integration design by creating computer models of the system, selecting parts, machining specialized parts, building parts lists, and overseeing all hardware developments and modifications pertaining to the bait-box itself. Additionally, this student was tasked with identifying mechanical failures after field deployments, mating the transceiver enclosure and bait-box components as a single unit, and general refinements to all mechanical components of the bait-boxes. How have the results been disseminated to communities of interest?ISTI has kept the collaborators at NWRC-APHIS abreast of the progress made during every phase of this project and have met in person with the APHIS project staff on numerous occasions to present current results, and discuss technical issues being investigated. As part of this Phase II project, we intend to submit two journal papers to related journals in the field in order to disseminate the results of this research. The first paper is currently being written but will not be submitted for publication until the non-provisional patent application has been submitted. The second journal publication will be prepared once modifications of the visual subsystem have been comprehensively tested and another field deployment validates the improved dual-sensory system. What do you plan to do during the next reporting period to accomplish the goals?Moving forward with the project goals, in the next reporting period ISTI plans to accomplish all remaining tasks for ?nalizing the SenseHog™ products and intends to focus more concertedly on marketing e?orts with the hopes of having a legally-protected, state-of-the-art, species management device with an eager addressable market at the conclusion of this Phase II research e?ort. Namely, we intend to: (a) Fully implement the visual subsystem improvements in order to ?x the issues in night-time capture mode that were revealed from ?eld testing in Hardaway, AL. (b) Carry out necessary robustness and functionality testing of the improved components and subsystems. (c) Finalize work on the setup application being designed for SenseHog™ system. (d) Completion and submission of Non-Provisional patent application after seeking legal counsel to ?nalize deliverables. (e) Schedule a ?nal deployment experiment with NWRC -APHIS which demonstrates false-alarm resistance and detection sensitivity capabilities of the system. (f) Expand e?orts to reach targeted domestic market both directly through our website and through licensed industry partners. Direct sales will primarily serve customers needing smaller installations, and indirect sales with an industry partner network is also crucial for getting our foot in the door of large-scale projects or other vertical markets where our system can be sold as an added feature to enhance other technologies e.g., bait box or trapping device capabilities. In order to thoroughly test the modi?cations described in Sections S.2.2 and S.4.2 (Proposed Fixes TN/AL), ISTI intends to schedule several more in-house experiments designed to thoroughly test the visual subsystem and motion detector under a wide range of lighting conditions and using moving targets at varying ranges and ambient temperatures to determine the functional limits of the systems. This testing will be carried out at a facility in Colorado that has penned feral swine over a one-week period after developments from Section S.4.2 and the application described in Section S.5 are completed. Additionally, ISTI plans to schedule one more ?eld test in collaboration with our partners at NWRC -APHIS in order to verify the functionality of the system after modi?cations. This test will likely be carried out at a facility with penned feral hogs in Kerrville, TX. After confirming that the aforementioned issues have been resolved through pen-trial testing, a final comprehensive test will be conducted in a realistic setting in collaboration with NWRC-APHIS.
Impacts
What was accomplished under these goals?
Goal 1: During this reporting period, this goal has been served primarily via specific objectives 1-7. Objective 1 focused on optimizing and fine-tuning the design of the acoustic-based feral swine recognition subsystem or SenseHog™ Ear. An algorithm developed last year using the Mel Frequency Cepstral Coefficient (MFCC) feature extraction and convolutional neural network (CNN) classifier was adopted. As part of Objectives 2-3, this algorithm was implemented on the hardware and fully tested in conjunction with the new bait box. Objective 4 focused on the development, implementation and testing of the companion visual-based, subsystem, referred to as SenseHog™ Eye, feral swine recognition subsystem and investigate methods for feature extraction and classification for its optimal performance. Building on the visual subsystem developments from previous reporting periods, the visual subsystem continues to be refined using feedback from lab testing and the results and data from several field deployments of the dual-sensory SenseHog™ prototype. Significant improvements to the visual classifier performance have been made as more data is made available to the system. But, perhaps more importantly, extensive testing of this subsystem has also revealed several issues with optical flow pre-processing of the visual system as well as the operation of the subsystem's night-time mode that are being addressed. Objective 5 is concerned with the development, implementation, and testing of a fusion mechanism to combine the decisions of the SenseHog™ Ear and Eye subsystems for confusion-free feral swine identification. In this reporting period, this mechanism of fusion has been fully developed and implemented on the prototype systems. The fusion process chosen is a voting process which collaboratively uses a running real-time queue of visual and audio events to determine if there is sufficient confidence in the target species class to warrant an unlock command for the bait box control system. This simple and fast fusion mechanism has been utilized in the two most recent field deployments of the prototype systems. Objective 6, which involves the integration of the audio and visual subsystems (hardware and software) with the new generation bait box, also saw significant development over this reporting period. In addition to packaging the visual and audio components into a widely used trail-camera enclosure, the new-generation bait-box was further modified to mate the transceiver enclosure to the bait-box. This was done to make sure the two components cannot be separated, hence reducing the risk of damage to the electromechanical latches and also simplifying the setup procedure of the system. Moreover, revisions were made to the two-component bait-box and transceiver to accommodate a GPS receiver for accurate time-keeping along with minor revisions of the "trail cam" style enclosure (see Figure 1) to address shortcomings discovered through field testing. These improvements simplify deployment and minimize risk of malfunction and system damage between deployments. Finally, as part of Objective 7 ISTI has begun conducting comprehensive testing of the dual-sensory SenseHog™ prototype through several field deployments conducted in collaboration with USDA-APHIS contacts at the National Wildlife Research Center in Fort Collins CO. These tests have provided valuable feedback on the detection and false-alarm sensitivities of the prototype systems by exposing them to different types of animals and interference typical of the regions where the systems are intended to be deployed. Goal 2: To train a dual-sensory detector-classifier that remains robust to a wide range of interference sources and environments/operating conditions, ISTI has closely worked with the NWRC - APHIS to conduct several field studies in order to collect a large database of both acoustic samples and trail camera photos that include acoustic recordings and sightings of different feral swine species, bears, deer, foxes, birds, and other common sources of wildlife typical for a ranching ecosystem. Since the last reporting period, three more experiments that emulate a realistic deployment have been conducted in collaboration with NWRC-APHIS during May 2019 in Colorado and Tennessee, and August 2019 in Alabama. Both the field deployment in Tennessee and Alabama provided novel environments and testing conditions that the dual-sensory prototype had never been exposed to. Testing conducted in Jamestown, TN primarily served to test the false-alarm resistance capabilities of the system as the most common species to visit the baited sites were American black bears. Additionally, data collected from this deployment has expanded the database of ``Bear" class trail-cam footage. During the recent Alabama deployment, the systems were exposed primarily to feral swine in order to evaluate the detection sensitivity of the dual-sensory prototypes. This experiment simultaneously provided feedback on the operation of the prototypes in a variety of environmental conditions, with regular exposure to the targeted species, and expanded the visual and audio databases for further training and testing of the classifier subsystems. Goal 3: To address the third major goal concerning the delivery of a rugged, low-cost, and low-power completed SenseHog™ system to APHIS, ISTI has accomplished several important milestones in both the development of the system's core technologies as well as the preparation of proper documentation for a non-provisional patent application and operational instructions. Since live field-testing of the prototype systems is so essential to the development of these technologies, ISTI has begun participating in longer-term product testing and collecting feedback from the users on how to effectively improve the system. These testing experiments have been organized in collaboration with NWRC -APHIS and have allowed the systems to be exposed to harsh ecosystems severely impacted by feral swine. Testing for a month in Tennessee provided invaluable feedback on the robustness and usability of the systems, when operated for multiple weeks by a layperson; and testing for a week in Alabama revealed several critical system flaws related to motion registry and image segmentation. These issues will be remedied during the NCE period. In addition to documentation of operational instructions which simplify deployment and troubleshooting, ISTI has begun development of a mobile application which further simplifies the system deployment process by automating several tasks required to ensure proper camera placement, audio levels, and storage capacity of the SenseHog Brain's embedded processor. Prior to full commercialization, ISTI intends to obtain patents for the documented intellectual property and implementation details that have accumulated for both the acoustic and visual subsystems of the SenseHog™ system. ?
Publications
|
Progress 09/01/17 to 08/31/18
Outputs
Target Audience:The work completed during this reporting period served two target audiences. First, ISTI served the USDA as the primary stakeholder by continuing development of the algorithms supporting the capability of automated species identification using acoustic recordings and live trap camera photos. The collaborative research with our USDA teammates at the NWRC-APHIS laboratory will lead to products that support the USDA's efforts in humanely and effectively protecting agricultural interests in the United States from the ecological and economic damages caused by feral swine infestations. The grant has also served a small audience in the educational community. Specifically, we initiated an internship position for a university mechanical engineering student to lead the mechanical design work pertaining to the revisions made to the feeder-box itself following the March 2018 field test in Vernon. The student hired was a junior at Colorado State University in Fort Collins. While not a primary audience to be served by the final product of which this project is concerned, the internship program allowed this student to receive hands-on training related to their field. The internship further advanced this students ability to work in an interdisciplinary team while recognizing and effectively communicating issues and decisions that would impact multiple engineering disciplines involved with the design. This student was hired in the spring and worked through the summer of 2018 and has since gone on to take another part-time engineering internship this Fall. Changes/Problems:From the data collected and observations made during the March 2018 field test, ISTI was able to identify two major issues that precluded ideal operation of the SenseHog™ Ear prototypes. First, in light of the issues of March 2nd, further investigation revealed that the BBBW real-time clock wake function is not fully supported in any currently available kernel hence resulting in unpredictable and unstable wireless utility especially when used in conjunction with RTC-wake functions. The second major issue was revealed on March 3rd where one of the boxes was found to be left ajar due to jamming of one of the latches (mechanical problem). In order to address the first major concern, ISTI made the decision to transition from the Beagle Bone Black Wireless (BBBW)platform to a more affordable and more broadly supported embedded application computer. More specifically, after careful consideration platform transition to the Raspberry Pi 3 Model B was made following the March test. This new platform is not only less expensive but also more widely supported particularly with wireless capabilities. During the time since the latest field test, the wireless transceiver was also altered to operate on less commonly occupied channels of the 802.11b specification. While the Raspberry Pi's entire wireless system will be re-tested more rigorously with the new platform, if instability persists, a conduit system will be implemented which requires a wired connection between intelligent part of the system and the feeder box. The second major concern brought up by this field test was the jamming of the cam mechanism in one of the SouthCo Electromechanical latches. It was determined that due to the orientation of the latch, and the nature of the latching mechanism, issues of latch jamming would be unavoidable and the decision was made to re-mount the latches in the same location but in a different configuration and orientation. To address this issue, three possible solutions were considered to alter the mounting approach. These solutions were modeled in CAD software and prototyped via 3D printer to determine the most effective one. After careful considerations, it was determined that the lid mounted latch with striker bolt on interior of the feeder-box indeed provides the best option as it required the least modification to the original design and appeared to provide the best alternative to the original latch placement with regards to avoiding unintended ingress to the well in which the cam lies. In order to implement this latching solution, custom mounting components had to be re-designed to achieve the desired latch configuration. The first of these parts was a bracket specifically designed to securely affix the SouthCo R4-EM-R73-162 electromechanical latch to the lid of one of the prototype bait boxes. This part was machined from 6061 Aluminum for corrosion resistance and featured two mounting holes for the latch itself, and a groove through which the striker bolt can travel. The second of these parts was an aluminum guard plate designed to shield the exposed opposite side of the SouthCo R4-EM-R73-162 latch. This guard plate also featured two holes for affixing the plate to the latch and a groove through which the strike bolt travels. This plate was also machined from aluminum and was intended to prevent closure of the cam by anything besides the striker bolt. What opportunities for training and professional development has the project provided?Training and professional development opportunities were provided in this reporting period through the participation of an undergraduate mechanical engineering intern on the project. The training activities included providing necessary guidance during all phases of design and the opportunity to lead the mechanical integration design by creating computer models of the system, selecting parts, machining specialized parts, building parts lists, and overseeing all hardware developments and modifications pertaining to the feeder-box itself. Additionally, this student was tasked with retrofitting a pre-existing trail camera enclosure to fit the required electronic components for the audio and visual-aware intelligent SenseHogTM engine. How have the results been disseminated to communities of interest?ISTI has kept the collaborators at NWRC-APHIS abreast of the progress made during every phase of this project and have met in person with the APHIS project staff on numerous occasions to present current results, and discuss technical issues being investigated. As part of this Phase II project, we intend to submit two journal papers to the related journals in the field in order to disseminate the results of this research. The first paper is currently being written but will not be submitted for publication until more statistically comprehensive performance results are achieved using the acoustic-only system. ISTI believes that with the insights gained from the March field test in Vernon, TX, the modified acoustic-only system will perform much more reliably. The second journal publication will be prepared once full integration of the visual subsystem with the audio subsystem and a comprehensive dual-sensory test can be performed. ISTI has also prepared a provisional patent disclosure jointly with our APHIS collaborators which is being processed by APHIS attorneys. What do you plan to do during the next reporting period to accomplish the goals?In the next reporting period, ISTI hopes to address several of the specific objectives outlined in Goals 1, 2 and 3. As for Goal 1 remaining task, ISTI has initiated the design of an optimal fusion mechanism for combining the decision-making of the audio and video subsystems. We shall investigate the sequence of the audio and video processing and decision-making processes to determine which combination will provide complete immunity to false alarms. Among the Goal 2 objectives, the demonstration of the system's performance in eliminating the risk of accidental baiting of non-targeted and indigenous species using thorough statistical analysis is a top priority. Shortcomings discovered during the field test in March 2018 led to a less than ideal comprehensive testing of the acoustic-only system and ultimately warranted the need for an additional acoustic-only study. ISTI hopes to concurrently test the video subsystem during the nextstudies.However, in these testing decisions made by the visual subsystem will not affect the lock control instead they are stored for post-processing and analysis of how closely decisions from the two processing channels agree with actual events. To address Goal 3 objectives, ISTI has already begun collaborating with NWRC-APHIS to begin the patent application process for a provisional patent on the SenseHog™ technology . It is expected that this will be filed before the end of this year. As ISTI nears completion of the SenseHogTM products, operational and maintenance requirements are being recorded with the intent of compiling a comprehensive document on its usage. This document will feature sections which explain the function and operation of all components of the system, troubleshooting, and instructions for typical deployment of the system in an affected area. This document will potentially contain a section on calibrating the device for lighting levels of the imaged scene but further testing of the prototype trail cam enclosure and camera sensor are needed to finalize this decision. Lastly, ISTI hopes to begin organizing and hosting demonstrations and technical events that provide opportunities for potential end-users, USDA stakeholders, and possible industry representatives to evaluate the system and provide feedback to enhance the overall performance and field ruggedness. ISTI hopes to reach audiences in the nearby agricultural communities affected by feral swine populations.
Impacts
What was accomplished under these goals?
Goal 1: During this reporting period, this particular goal has been served primarily via Tasks 1, 2 and 4 of the proposed Phase II research effort. Task 1 of Phase II focused on optimizing and fine-tuning the design of the acoustic-based feral swine recognition subsystem or SenseHog™ Ear. During this reporting period and through extensive experiments with actual field data, we discovered several issues with computational cost and implementation of the original sequential coefficient state tracking (SCST) algorithm-based acoustic classifier. These issues led us to investigate alternative acoustic classification algorithms. A new approach was then devised using the Mel Frequency Cepstral Coefficient (MFCC) feature extraction method. Frames of audio data were first converted to their MFCC feature vectors that are subsequently applied to a convolutional neural network (CNN) classifier trained and tested on the same acoustic database previously used for training and testing of the SCST classifier. The benchmarking of the results on an extended dataset revealed the fact that the new CNN-based approach not only has a significantly lower computational need but also provided a much better job of excluding non-target samples. Therefore decision was made to implement this algorithm on the hardware and conduct field testing in conjunction with the visual subsystem. Task 2 of the proposed efforts involved the development of a companion visual subsystem, SenseHog™ Eye, and its testing on a comprehensive imagery dataset. The inclusion of the visual subsystem ensures that the wildlife management devices are only activated for feral swine and not for other wild animals or livestock. The work in this reporting period involved developing a robust and computationally efficient algorithm for removing the background and isolating regions of interest (ROI) in which the animals are located. The method is based upon using the optical flow algorithm which gives one a robust means of capturing motion within the scene. Once the animal ROIs are isolated, they are applied to a CNN-based classifier trained on a portion of the previously collected imagery datasets from both the Washington and Texas field tests. The remainder of that dataset was then utilized to test the trained CNN-based classifier to decide if a feral swine is present in any frame. The ROC curve showed probability of correct classification of 98% for hogs with less than 2% false alarm. The integration of the two subsystems is expected to eliminate the incidents of false alarms. Task 4 focused on software and hardware integration and development. A major part of the efforts during this reporting period was put towards accomplishing Task 4 since a decision was made to transition to a new platform following the March 2018 field test conducted in Vernon, TX. Implementing the SCST algorithm on the Beagle Bone Black Wireless (BBBW) + LogiBone stack required about 70-90% CPU capacity while streaming audio data. While this was sufficient for an audio only implementation, preliminary testing of a video subsystem with the audio classifier determined the infeasibility of this combination. On the hardware side of the system, improvements in environmental robustness were made to the transceiver enclosure. A conduit routing system was implemented in two enclosures to allow for undisturbed routing of latch power and control cables from the transceiver enclosure. Furthermore, modifications were made to the orientation and mechanism for affixing the electromechanical latches to the feeder-box. This change was motivated by several issues discovered during the March 2018 field test of acoustic prototypes which revealed a scenario where the electromechanical latches can become jammed, leaving the feeder-box unsecured. Goal 2: In order to train and test our dual-sensory detector-classifier, ISTI in conjunction with the NWRC - APHIS conducted several field tests during July 2016 in Texas, April-May 2017 in Washington, September 2017 in Colorado, and also March 2018 in Vernon, Texas. These were done to collect a large database of both acoustic samples and trail camera photos that include acoustic recordings and sightings of different feral swine species, bears, deer, foxes, birds, and other common sources of wildlife typical for a ranching ecosystem. Using these data sets, a comprehensive test was conducted to compare the two audio classifiers, namely the SCST and CNN-based systems, being developed for the SenseHogTM Ear system. This test set features 59 minutes of streamed data containing the most difficult interfering vocalizations from the Washington audio data collection in addition to several difficult man-made and environmental sources such as noise from wind, cars, and motorcycles. This testing set, as with a similar previous testing set, featured bear vocalizations from 20 distinct visitations during the Washington field test. A comparison of the best performing SCST and CNN audio classification models revealed the superior performance of the CNN-based method with less computational requirements. Issues related with computational cost and implementation difficulties of the original SCST-based acoustic classifier are critical in light of the fact that the visual system also uses the same hardware resources. Therefore, this shift had to be done in order to use only one hardware platform that serves both the acoustic and visual subsystems. On the visual classification side, trail camera footage collected over the various field exercises was used to refine the parameters of the optical flow front end processor. After ROI extraction was determined to be relatively stable, these photo databases were partitioned into training and testing subsets and ISTI began investigating the proposed classifier i.e. a matched subspace classifier using an over-complete dictionary trained on moment-based features as well as several state-of-the-art CNN-based visual classifiers. Extensive testing utilizing these datasets revealed that the CNN-based classifiers seemed to make better use of the over-abundance of training samples. This superior performance has been repeatedly confirmed across all tested photo datasets consisting both targeted feral swine and non-target animals namely bears. Goal 3: In tandem with the optimizations that have been made to the visual and acoustic signal processing algorithms, ISTI has once again transitioned to a new wireless processing platform which provided more stable wireless performance and power cycling. In particular, ISTI successfully transitioned from using the two-component BBBW+ Logi-Bone with FPGA mezzanine card, which was selected for the previous revisions of the sensor interface and front-end DSP core, to the newer, cheaper, and more reliable Raspberry Pi 3 Model B embedded processor. This microprocessor card features the latest and most up-to-date Raspberry Pi "stretch" kernel. Transitioning to this newer platform allows for not only more reliable wireless communication but also better support of future revisions of the SenseHog™ subsystems while simultaneously allowing ISTI to leverage the most up-to-date open source libraries available for image processing tasks that will be handled by the Raspberry Pi with the addition of the companion visual subsystem. Both of the newly developed CNN-based acoustic and visual classifiers were designed and successfully imported to this new platform. It should be noted that this transition allowed ISTI to utilize libraries for training CNN architectures. ISTI has also begun investing time into retrofitting a commercial trail camera to enclose the critical sensory and computing components of the SenseHog™ Ear and Eye. All electronic components are now enclosed in a waterproof enclosure and a two pin XT-60 power connector port is located on the bottom of the device where 5V DC power may be input.
Publications
|
Progress 09/01/16 to 08/31/17
Outputs
Target Audience:The work completed during this reporting period served two target audiences. First, ISTI served the USDA as the primary stakeholder by continuing development of the algorithms supporting the capability of automated species identification using acoustic recordings. The collaborative research with our USDA teammates at the NWRC APHIS laboratory will lead to products that support the USDA's efforts in humanely and effectively protecting agricultural interests in the United States from the ecological and economic damages caused by feral swine infestations. The research grant has also served a small audience in the educational community. Specifically, we initiated two internship positions for two university mechanical engineering students, one in the spring of 2017 and one over the summer of 2017, to lead the mechanical design work pertaining to the integration of the Phase I prototype specifications into the latest bait delivery system enclosure provided by APHIS. The first of these hired students was a junior at Colorado State University in Fort Collins. The second of these students was also a junior at Colorado State University. While not a primary audience to be served by the final product of which this project is concerned, the internship program allowed the students to receive hands-on training related to their field and interact with NWRC APHIS laboratory staff. The internship further advanced these students' ability to work in an interdisciplinary team while recognizing and effectively communicating issues and decisions that would impact multiple engineering disciplines involved with the design. The first student hired in the spring has since gone on to find a full time position in a related field this summer. The second student remains a part of the research effort and is currently working towards full mechanical integration of the acoustic-sensing prototype. Changes/Problems:Phase II research efforts did not commence until January 16th, 2017 due to a delay in receiving the partial funding for this Phase II project. As of now, the indirect costs have yet to be approved and hence only partial funding has been provided. Modifications had to be incorporated related to the allocation of Research and Development funds in order to begin the Phase II research efforts. Additionally, the PI (Mr. Kumar Srinivasan) accepted a new position elsewhere and a new employee, Mr. Jack Hall, resumed his duties as a PI. What opportunities for training and professional development has the project provided?Training and professional development opportunities were provided through the participation of two undergraduate mechanical engineering interns so far in this Phase II project. The training activities included providing necessary guidance during all phases of design and the opportunity to lead the mechanical integration design by creating computer models of the system, selecting parts, building parts lists, and overseeing new prototype fabrication. The project continues to work with one of these interns on a part-time basis for assembly and testing during field tests of the current acoustic recognition enabled prototype. How have the results been disseminated to communities of interest?ISTI has kept the collaborators at NWRC APHIS abreast of the progress made during every phase of this project and have met in person with the APHIS project staff on numerous occasions to discuss timetables, current results, and technical issues being investigated. As part of this Phase II project, we intend to submit two journal papers to the related journals in the field in order to disseminate the results of this research. The first paper will be submitted once the development of the acoustic-based subsystem is completed and we have obtained statistically meaningful performance results; while the second paper will be prepared upon the completion of the image-based subsystem and its integration and testing with the companion acoustic-based subsystem. What do you plan to do during the next reporting period to accomplish the goals?The integrated acoustic-based system will be initially tested the first week of September, 2017 near Fort Collins against captive feral pigs under the care and supervision of APHIS personnel. This pen trial field testing will expose the completed acoustic and motion sensing beta prototype to feral pigs and verify that the probability of detection and correct classification are acceptable for a near-zero false alarm rate threshold. A minimal false alarm rate that is still acceptable will be determined in collaboration with NWRC APHIS and consider the risk in conjunction with the additional protections provided by the mechanical safety features of the bait system. The purpose of this pen trial is to provide an end-to-end testing of the acoustic subsystem to identify any remaining design issues by exposing it to challenging scenarios. After completion of this pen trial, raw data and classification decisions will be utilized to troubleshoot any remaining issues that arise from the test. ISTI will document the effectiveness of the sensor for intelligent species recognition and targeted bait delivery using video recordings of the box while it is exposed to the penned animals in situ. The test results of photo-video evidence will later be used to provide results for the next report. To help constrain the amount of data collected, ISTI and APHIS will deploy game scouting cameras for the video recorders. The scouting cameras offer the benefits of motion activated recording and IR illumination. The motion sensor will limit the video recording to only occur while animals are interacting with the bait unit and the IR illumination will allow data to be collected 24/7. After successful testing and troubleshooting of the completed acoustic subsystem via pen trials, two complete acoustic and motion sensing prototypes will be sent to Camp Bullis (or Kurville) in San Antonio Texas in order to expose the systems to a realistic environment in an area that struggles to control the populations of feral swine. The purpose of this trial is to address the first and second major goals mentioned in this report. Specifically, this experiment will provide comprehensive testing of SenseHog™ Ear by exposing it to different types of animals and interference typical of the regions where the system is intended to be deployed. This test is tentatively scheduled for October 2017. Lastly, work during the next reporting period will involve fine-tuning the optical flow algorithm for the reliable identification of ROIs in video imagery. Work will also involve the further investigation of a suitable set of invariant features that are capable of discriminating feral swine from other commonly encountered animals while at the same time meeting the resource-limited requirements of the chosen hardware for near real-time implementation. ISTI will summarize our progress and preliminary testing results in a second interim report to be submitted before the anniversary of the initiation of Phase II funded work (one year after January 2017). This report will include details on the current state of the acoustic and visual data processing algorithms, testing and performance evaluation results, updated hardware design methodology, assembly process, and how the fully integrated acoustic system performed in the field. Additionally, ISTI and APHIS will include recommended objectives for finalizing the Phase II research and commercialization efforts needed to bring the product into early-stage trials governed by the USDA and Food and Drug Administration (FDA).
Impacts
What was accomplished under these goals?
Goal 1: During this reporting period, the project goals have been served via the first two tasks of the proposed Phase II research effort. Task 1 of Phase II focused on optimizing and fine-tuning the design of the acoustic-based feral swine recognition subsystem. The system was first fine-tuned by incorporating an additional step in the detection and classification algorithm to safe-guard against falsely classifying interference sources which results in inadvertently targeting a non-target species. In addition to fine-tuning of the acoustic-based system, integration of the optimized SenseHog™ Ear with the new generation bait box was completed during this period. A pig pen trial has been scheduled for Early September 2017 with our project collaborators from APHIS, in Fort Collins, in order to address the third objective under Goal 1, to conduct an end-to-end testing of SenseHog™ Ear to identify any remaining design issues by exposing it to challenging scenarios e.g., animal vocalizations from species not used in the training process and those containing regional specificity in vocalizations. A comprehensive system-level test of the SenseHog™ Ear has been tentatively scheduled in conjunction with collaborators from APHIS in early October 2017 in Texas to determine the detection, classification, and false-alarm rejection performance of the improved acoustic system against feral swine and potentially many different sources of interference. This test will also reveal any mechanical and electrical hardware issues that could arise in any realistic field testing with wild animals. Preliminary testing of the companion visual subsystem, SenseHog™ Eye, also began during this reporting period. Visual data collected from both the Washington and Texas field tests were utilized in preliminary investigations using a robust classifier to ensure that the wildlife management devices are activated only for feral swine and not for other wild animals or livestock. This dual-sensory decision-making concept is a proven approach to meet the critical detection and false-alarm rejection requirements. More specifically, the work in this period involved developing a robust and computationally efficient algorithm for removing the background and isolating regions in which the moving targets (animals) are located within each frame. The method is based upon using the optical flow algorithm which gives one a robust means of capturing motion within the scene. Once the regions-of-interest (ROIs) are identified and isolated, illumination invariant, rotation and translation invariant, and scale invariant feature extraction methods can be utilized to extract a set of reduced dimensional features that capture unique characteristics of animal patterns from video frames. Information System Technologies Inc. (ISTI) continues to research the image pre-processing, background removal, and invariant transformations of the data which yield the most promising and consistent results across different data sets. Goal 2: To train a detector-classifier that remains robust to a wide range of interference sources and environmental and operating conditions, ISTI worked with the NWRC - APHIS to build a large library of acoustic samples that include acoustic recordings of different feral swine species, bears, and other common sources of interference. Experiments were conducted in collaboration with NWRC APHIS first during July of 2016 in Texas and again during April-May of 2017 in Washington. The first of these studies was designed to collect an extensive set of feral swine vocalizations and video data using our acoustic system. This test was conducted in July of 2016 over 27 days at 9 different sites at Camp Bullis just outside of San Antonio, Texas. The second of these studies was designed to collect a large database of bear vocalizations together with video using the current revision of feeder box design. This test was conducted in April and May of 2017 over 45 days at 5 different sites in a commercial timber forest between the towns of Naselle and Cathlamet, Washington. For this study, each of the newly designed feeder boxes were monitored by three trail cameras which captured both motion-triggered, high frame rate, near range photo series; as well as lower frame rate, mid and far range, photo series. This deployment served multiple purposes: (a) to prove the robustness and resilience of NWRC APHIS' latest feeder box design against entry from non-target species; (b) to test the ruggedness of the locking mechanism designed by ISTI to withstand high pressure from large bears; and (c) to provide an opportunity for realistic non-target species audio (and video) data collection using the sensors that will be integrated in the final SenseHog™ Ear system. Using these large data sets of animal vocalizations collected in a variety of temporal and spatial settings, ISTI continues to conduct statistical analyses of these data and their test results to define an optimal set of parameters for the classifier as well as the choices of thresholds for the signal and quiescent detectors. Furthermore, steps were taken to ensure that a balanced training set of the target and interference classes was used for training the system. This was implemented by generating an equal number and length of training subsets for all classes considered. This task will conclude an additional end-to-end testing designed to stress the system by presenting it with challenging data and those containing animal vocalizations from species not used in the training process. Potential risks arising from regional specificity in vocalizations will be made apparent with concrete test data collected. Using the abundance of interfering samples collected during the Washington field test, an experiment was recently designed to test the false alarm rejection (bears) capabilities of the latest algorithm improvements for both the acoustic detector and classifier. This test set features approximately 46 minutes of streamed data containing the most difficult interfering vocalizations from the Washington audio data collection. This subset of data included 20 bear visitation events and a multitude of bird vocalizations. This preliminary false alarm rejection test indicated perfect detection and false alarm rejection performance of the acoustic-based system on this data set. Goal 3: In tandem with the optimizations that have been made to the acoustic signal processing algorithms, ISTI has successfully transitioned the intelligent acoustic-based classifier designed in Phase I to a new wireless processing platform which allows for greater flexibility in interfacing theclassifier with the automated latching system installed on the new generation feeder box units. More specifically, ISTI successfully transitioned the Logi-Bone FPGA mezzanine card, which was selected for the previous revisions of the sensor interface and front-end DSP core, to the newer, more flexible, BeagleBone Black Wireless (BBBW) embedded processor. This microprocessor card features the latest and most up-to-date kernel. Transitioning to this newer kernel allows for better support of future revisions of the SenseHog™ subsystems while simultaneously allowing ISTI to leverage to most up-to-date open source libraries available for image processing tasks that will be handled by the BBBW with the addition of the companion visual subsystem. The Logi-Bone FPGA card provides the signal bus for collecting real-time observations from the digital microphone. The high speed DSP features of the FPGA provide resources for preliminary signal conditioning such as filtering and equalization. The FPGA also buffers the 24 kHz audio sample stream into frames for transfer to the BeagleBone Black Wireless which reads audio data in N-length sample frames from the FPGA and performs the detection and classification steps on the data frame vectors.
Publications
|
|