REMOTE DETECTION AND CLASSIFICATION OF IMPORTED FIRE ANT COLONIES FOR SITE-SPECIFIC MANAGEMENT TO ENHANCE NURSERY QUARANTINE PROGRAMS

• Sponsoring Institution: National Institute of Food and Agriculture
• Project Status: ACTIVE
• Funding Source: OTHER GRANTS
• Reporting Frequency: Annual
• Accession No.: 1030933
• Grant No.: 2023-70006-40607
• Cumulative Award Amt.: $179,330.00
• Proposal No.: 2023-02964
• Project Start Date: Sep 1, 2023
• Project End Date: Aug 31, 2026
• Grant Year: 2023
• Program Code: [ARDP]- Applied Research and Development Program

Recipient Organization
TENNESSEE STATE UNIVERSITY
3500 JOHN A. MERRITT BLVD
NASHVILLE,TN 37209

Performing Department
(N/A)

Non Technical Summary
Regulated pests like imported fire ants have quarantine certification treatments that allow the movement of nursery plants from infested to non-infested areas. These treatments are often expensive, impractical, time-consuming or utilize older pesticide chemistries with negative user effects and uncertain future availability. This project aims to reduce some of the green industry producer impacts associated with the management of imported fire ants in nursery settings to meet federal quarantine requirements. To achieve this goal, we will focus on two objectives: 1) develop a detection and classification system for imported fire ant colonies using an unmanned aerial vehicle and two camera sensors to create site-specific maps where thermal and color image acquisition, image preprocessing, image fusion, and an artificial intelligence model training and validation will be accomplished and 2) develop a strategic extension plan to disseminate the UAV-based colony management technological knowledge among nursery growers with an introductory survey, field demonstrations, workshops, and outreach publications. The foundation for a site-specific imported fire ant colony management technology will be constructed through this project. The proposed technology will have several impacts, including 1) allowing regulatory personnel to quickly gauge the effectiveness of various best management programs to reduce colony densities on nursery properties, 2) allowing nursery personnel or UAV consulting companies to rapidly locate and treat colonies on the property, and 3) addresses a USDA regulatory concern regarding human search consistency inadequacies in fire ant management programs.

Animal Health Component

50%

Research Effort Categories

Basic

(N/A)

Applied

50%

Developmental

50%

Classification

Knowledge Area (KA)	Subject of Investigation (SOI)	Field of Science (FOS)	Percent
216	2110	2020	100%

Knowledge Area
216 - Integrated Pest Management Systems;

Subject Of Investigation
2110 - Ornamental trees and shrubs;

Field Of Science
2020 - Engineering;

Keywords

artificial intelligence

imported fire ants

remote sensing

thermal imaging

uav

Goals / Objectives
Primary Goal: Introduce an improved IFA colony detection and classification technology to nursery growers and enhance nursery quarantine programs.Objective1: Develop an artificial intelligence (AI)-driven IFA colony detection and classification system for development of site-specific mapsA UAV-based IFA colony detection and classification system will be developed for site-specific management. The system will fly over the nurseries and identify and locate the IFA colonies based on thermal properties and classify their sizes to generate maps.Objective 2: Outreach and evaluation of grower/stakeholder interest for adoption.A comprehensive extension education plan regarding the UAV-based IFA colony detection and classification system will be developed. This extension plan includes introducing the IFA detection and classification system to growers, a survey to assess the familiarity of growers with UAV technologies, and technology and information transfer to nursery growers and practitioners.

Project Methods
Objective1: Develop an artificial-intelligence (AI)-driven IFA colony detection and classification system for site-specific mapTask #1: UAV system integration, flight planning, and image acquisitionA few considerations need to be made regarding UAV hardware suitable for system development. Since the surface temperature of IFA colony mounds aregenerally higher than the adjacent ground (especially in the daytime), we will use a thermal camera with a 20× optical zoom lens to capture thermal images. Thermal images are usually lower in resolution than RGB images. To enhance image resolution and quality, we will also capture RGB images using a color camera with a 20× optical zoom lens to perform a fusion between RGB and thermal images. A DJI Matrice 300 RTK drone will be used to carry the image capturing cameras. Two different flights will be required to acquire thermal and RGB images due to this system having only a single gimbal connector for cameras.Task #2: Image sorting, preprocessing, and processing pipelineThe stored data will include images with or without IFA colonies. To separate images with colonies from all stored images, we will perform image sorting. It is possible that the light intensity will not be uniform in some sections of the experimental nursery, which will cause over or under-amplification of the contrast on acquired images. We propose to use animage enhancement algorithmto improve the quality of the raw sorted images. To further improve the quality and resolution of the images, we will apply an image fusion /overlaying approach to RGB and thermal images.Other image preprocessing operations will be performed, including cropping, resizing, and rotating to make the fused images usable for developing deep learning models to detect and classify IFA colonies. We will augment and annotate images to increase the diversity of data available for training machine/deep learning models.we plan to develop and use an enhanced deep network model (DeepSeg) by fusing Transformer and Mask-RCNN architecture to detect and segment IFA colonies in a complex nursery condition.Task #3: System validationThe IFA colony detection and classification system will be validated in separate field nurseries (not used for previous image collection). We will use approximately ten nursery sites planted with single trunk trees to evaluate the robustness of the developed system. The UAV attached with the camera will fly over the experimental validation sites to capture RGB and thermal images. Data will be processed using developed algorithms described in Task #2. We will generate a site-specific map from each site that considers other factors like tree age, species type, canopy size, etc. To evaluate the performance, a field scout experienced with IFA colony detection will work on identifying and marking the IFA colonies using visual assessments. The scout will use an RTK-GPS system to map the location of each IFA colony according to their sizes (very large, large, medium, small, very small). We will compare the manually generated map with the system-generated site-specific map. A comparative analysis of the generated site-specific map using the mean absolute error (MSE), the percent standard error (%SE), and the root mean square error(RMSE) will be performed, which determines how well the system performed for IFA colony detection and classification.Objective2: Outreach and evaluation of grower/stakeholder interest for adoptionTask #1: Introduction of the UAV-based IFA colony detection and classification technology to nursery growersAn extension article introducing UAV-based IFA colony detection and classification system will be publicly available on the TenneseeState University Extension Publication website, as well as published in Tennessee Nursery and Landscape Association (TNLA) industry publication, the Tennessee Greentimes. Growers also will be informed via our extension nursery grower email list and newsletters of the TNLA, Middle Tennessee Nursery Association, and the Tennessee Department of Agriculture. Links to the TSU extension publication, Tennessee Greentimes article, and the newsletters will be shared with Tennessee nursery growers and nursery extension personnel in other southern states with IFA issues to help generate interest in this technology before the field day demonstration. The extension publication will provide basic knowledge, operating procedures, and guidance for spraying and drenching operations used for nursery IFA management. An invitation to growers and stakeholders to attend the field demonstration event also will be included.Task #2: Assess grower familiarity with UAV applicationsWe anticipate that a portion of growers in Tennessee are familiar with UAV technologies. To know the percentage of nursery growers with UAV experience and to assess the grower's interest in UAV-based technologies for site-specific IFA management, we will conduct a brief survey in major nursery crop producing counties in Tennessee (Warren, Coffee, Grundy, Franklin, and DeKalb). This survey will provide a basis for understanding grower familiarity with UAV technologies and identify the area of educational needs to be addressed during the demonstration and workshop.Task #3: Technology and information transfer to practitioners and post-research adoption surveyWe will organize a training session/field demonstration for practitioners (nursery producers, industry professionals and service providers) at TSU Otis Floyd Nursery Research Center (NRC), McMinnville.We will also conduct a workshop for practitioners to address the issues stated above in greater detail and will include a detailed analysis of the IFA colonies detection and classification research results. The workshop will also provide a simple toolkit for evaluating costs associated with investing in the system and the benefit of investment associated with site-specific management to enhance the nursery quarantine program.

Progress 09/01/24 to 08/31/25

Outputs
Target Audience:Our target audience included researchers and students from Tennessee State University (TSU), scientists from TSU Otis Floyd Nursery Research Center, TSU and University of Tennessee Extension specialists, nursery farm owners, and members of the Middle Tennessee Nursery Association, the Tennessee Department of Agriculture, and the Tennessee Nursery and Landscape Association. A TSU professor, one graduate student, and one technician developed a framework for detecting imported fire ant (IFA) mounds using UAS (unmanned aircraft system) images and an object detection deep learning method. Another TSU professor and graduate students in McMinnville conducted field experiments on treatments against the IFA. Changes/Problems:Due to equipment malfunctions and poor weather conditions in Middle Tennessee, only limited UAS data was collected over the nursery farms. During data processing, the training of deep learning models was limited by computing power. We will address these issues by collecting more data under different conditions and using high-performance computing facilities. What opportunities for training and professional development has the project provided?On November 21, 2024, our project members and Dr. Kaitlin Barrios, a nursery extension specialist at the Otis L. Floyd Nursery Research Center, hosted a workshop, titled "Drone Use in Nursery Production", in McMinnville, TN. The PI introduced UAS technology and its applications for nursery operations, and demonstrated UAS missions for researchers, students, and stakeholders. Invited industry speakers delivered the knowledge on drone features, UAS pilot licensing, and data collection. How have the results been disseminated to communities of interest?1. Community Workshops and Seminars: Organized workshops and seminars tailored for faculty and stakeholders, where PIs presented key findings and insights. 2. Conference Presentations: The PIs presented their research related to this project at the conference ASABE Annual International Meeting 2025 and 2025 Southern Branch ASA Annual Meeting. They shared the results of this project with researchers, scientists, and students from other institutions. What do you plan to do during the next reporting period to accomplish the goals?1. Data Collection: The PIs will design additional UAV flight missions under various conditions, such as data collection time, weather conditions, and camera angles, to test the optimized parameters for AI-driven IFA colony detection from UAS images. 2. Object Detection Algorithm: Larger training datasets for YOLO models will be generated from various datasets. The different hyperparameters of the YOLO models, such as the learning rate, batch size, optimizer, and momentum, will be tested. Additionally, the characteristics of the detected IFA mounds will be extracted for classification.

Impacts
What was accomplished under these goals? Objective 1 Activities - UAS data collection and preparation As preliminary data, we collected UAS images from RGB and thermal cameras at an altitude of 15 meters with 85% overlap on May 2, 2024. During the UAS mission, the RTK GPS mode was used for precise georeferencing with higher geo-accuracy (<2 cm). We flew drones around noon to observe the highly heated fire ant mounds over the animal area on TSU campus farm. An orthomosaic image of RGB and thermal data was generated using the Structure from Motion (SfM) algorithm. This image indicates natural color and surface temperature, respectively. - IFA Colony Detection using YOLO model Different combinations of bands were created between the RGB and thermal images. The reflectance difference between the mound nest and the surrounding soil and green pixels is higher for the red and green bands than for the blue band. We used three-band channel image combinations because the YOLO model accepts three-band images (RGB, RGT, and TTT). We used the LabelMe tool to generate training data for the YOLO model. The training dataset was split into training (80%), testing (10%), and validation (10%) subsets. We trained the YOLOv8 object detection model with different hyperparameters to test how training settings influence detection accuracy. To evaluate the performance of YOLOv8-based object detection models with various input combinations and training configurations, we employed different validation and testing metrics, including precision, recall, F1-score, mAP50, and mAP50-95. The results showed that the precision, recall, and mean average precision (mAP) were higher for RGB and RGT images than for the mono-temporal thermal image (TTT). There were more detections around bare soil in the RGT image than in the RGB image. There were visible false detections around some objects in the field, such as artificial structures, in the RGB image. Objective 2 Activities We developed a framework for UAS data collection and detecting IFA using UAS images. TSU hosted the workshop, "Drone Use in Nursery Production", on November 21, 2024 in McMinnville, TN, providing the knowledge of UAS (drones) to researcher, producers, and students.

Publications

• Type: Conference Papers and Presentations Status: Published Year Published: 2025 Citation: Anjin Chang, 2025, Unoccupied Aircraft System (UAS) Programs for High Throughput Phenotyping (HTP) at Tennessee State University (TSU), 2025 Southern Branch ASA Annual Meeting, February 2-4, Dallas, TX, USA.
• Type: Conference Papers and Presentations Status: Published Year Published: 2025 Citation: Anjin Chang, 2025, Imported Fire Ants (IFA) Mound Detection using UAS Images, 2025 ASABE Annual International Meeting, July 13-16, Toronto, Canada.

Progress 09/01/23 to 08/31/24

Outputs
Target Audience:We targeted several stakeholders which include industrial nursery producers, students and technicians at Tennessee State University (TSU), scientists at TSU Otis Floyd Nursery Research Center, USDA-APHIS , USDA-ARS National Arboretum (McMinnville, TN), and TSU and University of Tennessee extension, as well as the Middle Tennessee Nursery Association, Tennessee Department of Agriculture, and Tennessee Nursery and Landscape Association. One TSU professor, two graduate students, and two summer interns participated in field repellent testing and laboratory soil bioassays against imported fire ants, while another TSU professor collected UAS (Unoccupied Aircraft System), also known as droneimagery. Changes/Problems:During the first year of the project, we had two major issues, each of which caused unexpected delays, deviations in data collection, and significant impacts on the spending rate. The first major change was that a PI, Dr. Md. Sutan Mahmud, left his position at Tennessee State University (TSU) to join the University of Georgia. This change meant that the PI role had to be transferred to Dr. Anjin Chang in order to manage and complete this project successfully. Since the PI transition was completed in January 2024, equipment purchases and student/technician recruitment have been delayed. The UAS and thermal camera were purchased and tested for high quality data collection in May 2024. In addition, due to equipment and weather conditions, we were only able to collect a few UAS data sets over the fire ant mound fields. Despite these challenges, we still made important progress in detecting the fire ant mounds using UAS data. What opportunities for training and professional development has the project provided?Tennessee State University hosted the 2024 Summer Apprenticeship Program (SAP) in July 2024 (7/1-7/27, 2024). Dr. Anjin Changprovided several training programs on theUAV flight mission, data collection, and data processing so that they could deeply understand cutting-edge technology in agriculture. Also, the PI trained graduate students at TSU to introduce UAV applications in agriculture, such as crop monitoring, UAV-based high-throughput phenotyping (HTP), and image processing. The graduate students learned more professional skills and started to apply them to their research topics. How have the results been disseminated to communities of interest?1. Community Workshops and Seminars: Organized workshops and seminars tailored for faculty and stakeholders, where PIs presented key findings and insights. 2. Participation in TN State Fair: Dr. Anjin Chang exhibited UAV platforms and cameras and the results from this project in the 4H program at the TN State Fair (August 2024). 3. Conference Presentations: The PIs presented their research related to this project at the conference ASABE Annual International Meeting 2024 and Association of 1890 Research Directors 2024 Research Symposium. They shared the results of this project with researchers, scientists, and students from the other institutions. What do you plan to do during the next reporting period to accomplish the goals?1. Data collection: The PIs will design more UAV flight missions in different environments (locations, times, seasons, sensors, etc.) to build large datasets for AI-driven IFA colony detection and classification. 2. Application of AI algorithm: We will construct training datasets for the machine/deep learning algorithms, and then test various AI-based object detection algorithms such as YOLO, U-Net, and CNN to detect IFA mounds and to classify them according to their size. 3. We will introduce the UAV and IFA detection and classification through workshops and field days for stakeholders, growers, and researchers.

Impacts
What was accomplished under these goals? Objective 1 Activities - Data Collection We collected unmanned aerial vehicle (UAV) imagery using existing UAV platforms and cameras at three different locations: Otis Floyd Nursery Research Center in McMinnville, TN (March 21, 2024), Magness Nursery in McMinnville (May 1, 2024), and TSU main campus farm in Nashville, TN (May 2, 2024) to test the availability of UAV and sensors. The UAV flew at lower altitudes (<50m) to collect very high resolution data for fire ant mound detection. The RTK GPS mode was used for precision georeferencing with higher geo-accuracy (<2 cm). - Data Processing for the IFA Colony Detection and Classification RGB and thermal imagery data collected at 15m altitude with 85% overlap on May 2, 2024 were used to generate othomosaic images with radiometric calibration. As a preliminary study, we determined the threshold value (42 degrees Celsius) to extract high temperature pixels from the thermal orthomosaic image. To remove noise pixels, morphological filtering, closing and opening, was applied to the binary classified map, and then pixel clusters were segmented as the individual fire ant mounds. According to the shape and area of the segment, inappropriate segments were also removed. We were able to generate prescription maps showing the distribution and area of the fire ant mound. Objective 2 Activities We developed materials and training programs for students at Tennessee State University. The knowledge of UAV was delivered through the 2024 summer apprenticeship program (July 2024) at TSU and 4H programs at Tennessee State Fair (August 2024). We also provided the IFA biological control and treatment research to the stakeholders.

Publications

• Type: Conference Papers and Presentations Status: Published Year Published: 2024 Citation: Anjin Chang, 2024, Identification of Imported Fire Ants (IF A) Mound using UAS Imagery, ASABE Annual International Meeting 2024, July 28-31, Anaheim, CA, USA.
• Type: Conference Papers and Presentations Status: Published Year Published: 2024 Citation: Chang, A. 2024. Web-based Visualization and Data Sharing Tools for Unoccupied Aerial System (UAS) Images in Agriculture, ASABE Annual International Meeting 2024, 28-31 July. 2024. Anaheim, CA.
• Type: Conference Papers and Presentations Status: Published Year Published: 2024 Citation: Oliver, J., K. Addesso, R. Archer, N. Youssef, P. O'Neal, S. Valles, R. Weeks, L. Alexander, and M. Pandery. 2024. Geographic distribution and incidence of viral and microsporidian pathogens in Tennessee imported fire ant populations. Association of 1890 Research Directors 2024 Research Symposium. 6-9 Apr. 2024. Nashville, TN.