Source: UNIVERSITY OF FLORIDA submitted to NRP
NOVEL SMARTPHONE VISION TOOL TO IMPROVE SPIDER MITE MONITORING IN STRAWBERRY AND ALMOND
Sponsoring Institution
National Institute of Food and Agriculture
Project Status
COMPLETE
Funding Source
Reporting Frequency
Annual
Accession No.
1019615
Grant No.
2019-67021-29691
Cumulative Award Amt.
$491,544.00
Proposal No.
2018-07207
Multistate No.
(N/A)
Project Start Date
Jun 1, 2019
Project End Date
May 31, 2023
Grant Year
2019
Program Code
[A1521]- Agricultural Engineering
Recipient Organization
UNIVERSITY OF FLORIDA
G022 MCCARTY HALL
GAINESVILLE,FL 32611
Performing Department
(N/A)
Non Technical Summary
The objective of this proposal is to develop an easy-to-use monitoring tool for twospotted spider mites (TSSM) using a smartphone vision and deep learning, and use the system in TSSM management.Strawberry and almond are two important horticultural crops worldwide. California and Florida are the leading producing states in the U.S. TSSM, Tetranychus urticae Koch, is a major foliar pest of strawberries and almond, causing significant yield losses. Recently, TSSM resistance to commonly used acaricides were documented in both states, leading to reexamination of management practices. Miticide is often applied weekly but sometimes when TSSM presence is detected or sampling indicates nominal thresholds are reached.To improve TSSM monitoring, we are proposing to develop an easy-to-use method to count mites on strawberry and almond leaves, implementing vision techniques and deep learning (artificial intelligence) using smartphone. This tool will allow stakeholders and pest control advisors (PCAs) to count actual TSSM numbers more accurately with life stages differentiated, frequently, and extensively in fields.Results can be used to choose and rotate miticide with appropriate modes of action (e.g., ovicide if eggs numerous) and make applications only when TSSM presence is detected or populations increase. The proposed system could be applicable to many other crops.This project aligns directly with the priority of "engineered devices, technologies, and tools to improve agriculturally relevant plant systems" in the Agricultural Engineering Program.With better TSSM monitoring, science-based decisions to apply controls will lead to improved resistance management, greater feasibility of using miticide alternatives, and environmental and human-health benefits by reducing amounts of miticide used.
Animal Health Component
50%
Research Effort Categories
Basic
0%
Applied
50%
Developmental
50%
Classification

Knowledge Area (KA)Subject of Investigation (SOI)Field of Science (FOS)Percent
40411222020100%
Knowledge Area
404 - Instrumentation and Control Systems;

Subject Of Investigation
1122 - Strawberry;

Field Of Science
2020 - Engineering;
Goals / Objectives
The main goal of this project is to develop a monitoring tool for TSSM using a smartphone and use the system in crop spider mite management. Hypotheses are: (I) A smartphone-based monitoring tool can be used to accurately count the number of TSSM on strawberry and almond leaves, and (II) The developed system can be used to improve TSSM management in strawberry and almond, and can be expanded to other crops. The specific objectives are to: (1) Develop software capable of detecting and counting live egg and motile TSSM on strawberry and almond leaves using machine learning algorithms including deep learning, and compare the counts with those obtained using magnification in the laboratory and a hand lens in the field; (2) Optimize smartphone imaging hardware and protocols for accuracy and speed of obtaining TSSM counts, while maintaining user-friendly features; (3) Evaluate the applicability of smartphone imaging for obtaining TSSM counts to make control decisions; and (4) Analyze economic benefits for the commercial version of the monitoring tool.
Project Methods
Various computer vision methodologies and machine learning algorithms will be implemented using stationary images and videos, and the results will be assessed by comparing with manual counts by experienced personnel. In this project, different convolution neural networks (CNNs) with some other recently developed networks.Macro images will be analyzed to detect the adult TSSMs, larvae + nymphs, and eggs on the almond/strawberry leaves. Several image analysis techniques will be tested to determine the optimum algorithm.We will integrate the smartphone GPS, inertial sensor, and camera to generate hi-resolution macro images of almond and strawberry leaves, and to georeferenced the knowledge extracted from the images. Several sets of smartphone accessories will be designed, created, tested and evaluated for image acquisition under the field condition. Two user-friendly smartphone applications (iOS, and Android), 'MiteCount' will be developed by converting the machine learning and image processing algorithms developed in obj. 1.We will characterize the spatio-temporal within-field distribution of TSSM in Florida strawberry and California strawberry and almond research plots and grower fields using georeferenced results from imaging software/smartphone vision. And we will compare efficacy of control decisions in strawberry plots using imaging software versus traditional hand-lens TSSM counts (motiles and eggs).Partial budgeting analysis is a standard technique to assess economic impact of a change in a farm system (Kay, Edwards, & Duffy, 1994). It only focuses on the changes in costs and benefits that would result from the changes in the production system (Warmann, 1995; Wossink & Osmond, 2002); those production costs which are fixed across practices are not considered. Using the new tool is expected to reduce miticide applications, which will save labor, chemicals, equipment use, energy, materials, and other inputs associated with the applications that must be considered. These reduced costs will be calculated by using changed amounts of inputs used.

Progress 06/01/19 to 05/31/23

Outputs
Target Audience:Strawberry, almond, and other vegetable crop growers Pest control advisors (PCAs) Extension personnel Agricultural consultants Researchers exploring efficient pest management methods and tools Changes/Problems:We had some difficulties with the Co-PIs leaving their institutions for other jobs and thus stopped working on this project. We found their replacements but not all, which posed unexpected problems such as a lack of coordination of field tests. In the first year of this project, we realized that it was slow to take images of strawberry leaves using a macro lens and smartphone. Therefore, this project developed a six-camera imaging box and portable device for spider mite imaging and detection, which was more convenient and faster than a smartphone app. After testing the six-camera imaging box in California, we realized the imaging box could be further improved. Therefore, we decided to use the portable device that only uses one high-resolution camera (64 MP) for image acquisition, which simplified the design of the imaging box. The probable device was integrated with the smartphone app for rapid pest detection and providing a spatial distribution map of spider mites for site-specific pest management. What opportunities for training and professional development has the project provided?We have trained two research associates, five graduate students, and three OPS employees in identifying important arthropods in this system and verifying that test identifications made by the deep learning tool were accurate. This included identifying TSSM and predatory mites Neoseiulus californicus and Phytoseiulus persimilis. In addition, we have trained two undergraduate students and one master's student in gaining new skills in smartphone app development, 3D design software (Fusion 360), 3D printing, etc. How have the results been disseminated to communities of interest?Firstly, we presented our spider mite detection technologies at several international conferences, state conferences, and university symposiums. Secondly, we reached out to stakeholders by giving an oral presentation at the 41st Annual Strawberry Agritech Conference organized by the Florida Strawberry Growers Association and demonstrating the smartphone application and portable device for strawberry pest detection at California Strawberry Field Day. What do you plan to do during the next reporting period to accomplish the goals? Nothing Reported

Impacts
What was accomplished under these goals? Firstly, this project developed a smartphone app for the detection of spider mites and predatory mites in real time. The smartphone app can also generate spatial distribution maps of spider mites for site-specific management. Secondly, a six-camera imaging box and a portable device were developed for automated imaging and detection of spider mites. Thirdly, our smartphone app was integrated with the portable device. The smartphone app is used to control the portable device to collect data and detect pests, and then the pest-counting data is transferred to the smartphone app and saved in the database. Fourthly, we compared the pest-counting accuracy and speed of smartphone app, hand lens, and portable device. The result shows that both smartphone app and portable device could achieve better spider mite detection accuracy using less time compared with hand lens. Fifthly, We completed an economic analysis investigating the economic benefits of using the tool developed in this project. The economic analysis indicates that the smartphone app is more time and cost-efficient than the traditional scouting method in managing twospotted spider mite infestations, for both large and small strawberry production operations. We are still working on the economic analysis of the portable device. Finally, we have reached out to strawberry growers, PCAs, and other stakeholders by giving an oral presentation at the 41st Annual Strawberry Agritech Conference hosted by the Florida Strawberry Growers Association and having a demo of our technologies at California Strawberry Field Day. Our efforts align with the project goal of offering an easy-to-use method for stakeholders and pest control advisors (PCAs) to count and manage TSSM effectively. Our work represents a vital contribution to efficient spider mite management. This project will broadly impact agricultural practices, not only for strawberry and almond crops but potentially for other crops as well. The combination of advanced imaging technology and deep learning presented a promising avenue for science-based decision-making in pest control.

Publications

  • Type: Journal Articles Status: Published Year Published: 2023 Citation: Zhou, C., W.S. Lee, O.E. Liburd, I. Aygun, X. Zhou, A. Pourreza, J.K. Schueller, and Y. Ampatzidis. 2023. Detecting two-spotted spider mites and predatory mites in strawberry using deep learning. Smart Agricultural Technology 4: 100229.
  • Type: Journal Articles Status: Submitted Year Published: 2023 Citation: Zhou, C., Lee, W.S., Zhang, S., Pourreza, A., Liburd, O.E., Schueller, J.K., Ampatzidis, Y. A. Smartphone application for site-specific pest management based on deep learning and spatial interpolation. Submitted to Computers and Electronics in Agriculture
  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Zhou, C., Lee, W.S., Kratochvil, W., Schueller, J.K., Pourreza, A. (2023) A portable imaging device for twospotted spider mite detection in strawberry. ASABE annual meeting (Oral presentation).
  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Zhou, C., Lee. W.S., Zhang, S., Pourreza, A. (2023) A smartphone application for mapping the populations of two-spotted spider mites in strawberry. 2023 AI in Agriculture Conference: Innovation and Discovery to Equitably Meet Producer Needs and Perceptions, Orlando, Florida (Oral presentation).
  • Type: Conference Papers and Presentations Status: Published Year Published: 2022 Citation: Zhou C., Lee W.S., Pourreza A., Schueller, J.K., Liburd O.E., Ampatzidis Y., Ramirez, G. 2022. Strawberry pest detection using deep learning and automatic imaging system. Proceedings of the 15th International Conference on Precision Agriculture.
  • Type: Other Status: Published Year Published: 2022 Citation: Zhou C., Lee W.S., Pourreza A., Liburd O.E., Schueller, J.K., Ampatzidis Y. 2022. Comparing Deep Learning Methods for Strawberry Pest Detection. UF ABE Poster Symposium. (Poster)
  • Type: Conference Papers and Presentations Status: Published Year Published: 2022 Citation: Zhou C., Lee W.S., Pourreza A., Liburd O.E., Schueller, J.K., Ampatzidis Y. 2022. Strawberry Pest and Predatory Mite Detection Using Artificial Intelligence. Florida Section of American Society of Agricultural and Biological Engineers 2022 Annual Meeting. (Presentation)
  • Type: Conference Papers and Presentations Status: Published Year Published: 2021 Citation: Zhou, C., Lee, W.S., Liburd O, & Aygun, I. (2021). Smartphone-based tool for two-spotted spider mite detection in strawberry. American Society of Agricultural and Biological Engineers 2021 Annual Meeting. (Paper and presentation)
  • Type: Conference Papers and Presentations Status: Published Year Published: 2020 Citation: Zhou, C., Lee, W.S., Aygun, I., & Liburd O. (2020). Detecting Two-Spotted Spider Mite Tetranychus urticae Koch (Acari: Tetranychidae) and Mite Egg under Strawberry Leaf Using Deep Learning. Entomological Society of America 2020 Annual Meeting (Invited).
  • Type: Conference Papers and Presentations Status: Published Year Published: 2020 Citation: Zhou, C., & Lee, W.S. (2020). Detection of Two-Spotted Spider Mite in Strawberry Using Artificial Intelligence. UF ABE Virtual Poster Symposium Presentation.
  • Type: Other Status: Published Year Published: 2020 Citation: Zhou C., Lee W.S., Aygun I., 2020. YOLO for Two-spotted Spider Mite Monitoring in Strawberry. Pathways Towards the Next Generation of Agriculture and Natural Resources in Florida, University of Florida, Gainesville, January-16, 2020. (Poster presentation)
  • Type: Other Status: Published Year Published: 2022 Citation: Zhou C., Lee W.S., Pourreza A., Liburd O.E., Schueller, J.K., Ampatzidis Y. 2022. Strawberry Pest Detection Using Artificial Intelligence and Smartphone Image. Envisioning 2050 in the Southeast AI Driven Innovations in Agriculture. (Poster)


Progress 06/01/22 to 05/31/23

Outputs
Target Audience:The economic analysis and field testing of the smartphone app, six-camera imaging box, and portable device target strawberry, almond, and other vegetable crop growers, pest control advisors (PCAs), extension personnel, agricultural consultants, and researchers exploring efficient pest management methods and tools. Changes/Problems:After testing the six-camera imaging box in California, we realized the imaging box could be further improved. Therefore, we developed a new portable device that only use one high-resolution camera (64 MP) for image acquisition, which simplified the design of the imaging box. The newly developed portable device was lighter and cheaper than the six-camera imaging box. What opportunities for training and professional development has the project provided?We have trained two undergraduate students and one master's student at the University of Florida in this project. They gained new skills in smartphone app development, 3D design software (Fusion 360), 3D printing, etc. How have the results been disseminated to communities of interest?We presented our spider mite detection technologies at the 2023 ASABE annual meeting and the 2023 Artificial Intelligence in Agriculture Conference: Innovation and Discovery to Equitably Meet Producer Needs and Perceptions, in Orlando, Florida. In addition, we reached out to stakeholders by giving an oral presentation at the 41st Annual Strawberry Agritech Conference organized by the Florida Strawberry Growers Association on May 16, 2023, and demonstrating the smartphone application and portable device for strawberry pest detection at California Strawberry Field Day on August 3, 2023. What do you plan to do during the next reporting period to accomplish the goals?This is the last Annual Report.

Impacts
What was accomplished under these goals? A new image-transferring method was implemented in the six-camera imaging box, which could transfer images from each camera to the Raspberry Pi 4B within 10 seconds. In addition, a pre-trained deep learning model was integrated with the six-camera imaging box, which enabled it to process and analyze the images and provide the results of the spider mite detection and counting in real time. Then, the six-camera imaging box has been successfully tested in the field for both almond and strawberry crops in California. A new portable device using a high-resolution camera (64 MP) was developed and integrated with the smartphone application. Then, both smartphone app and portable device were tested in a strawberry field in Florida. The result shows that both tools could achieve better spider mite detection accuracy using less time. We have published one journal paper (Paper title: Detecting two-spotted spider mites and predatory mites in strawberry using deep learning). Another paper (Paper title: A smartphone application for site-specific pest management based on deep learning and spatial interpolation) is ready for submission. We are currently working on a paper about the portable device. We completed an economic analysis investigating the economic benefits of using the tool developed in this project. The draft of the analysis is titled "Economic Analysis of Smartphone App in Spider Mites Detection in Strawberry Production." The economic analysis indicates that the smartphone app is more time and cost-efficient than the traditional scouting method in managing twospotted spider mite infestations, for both large and small strawberry production operations. The savings are greater for small operations due to their higher sample counts per acre. Moreover, traditional scouting is found to be more susceptible to errors. A comparison between the traditional scouting method and the smartphone app in terms of their accuracy for counting TSSM motiles and eggs in strawberry fields shows that the smartphone app exhibits a 12% higher accuracy than the manual method. When it comes to counting TSSM motiles, the difference in Mean Absolute Percentage Errors between the smartphone app and the manual method is smaller, with the app being only 4% more accurate. These findings suggest that the smartphone app provides a more consistent and accurate count, particularly for TSSM eggs. We are still working on the economic analysis of the portable device. We have completed a draft manuscript on labor-saving technology adoption in agriculture, focusing on the specialty crop industry (including strawberries), which is under revision for submission to a peer-reviewed journal.

Publications

  • Type: Journal Articles Status: Published Year Published: 2023 Citation: Zhou, C., W.S. Lee, O.E. Liburd, I. Aygun, X. Zhou, A. Pourreza, J.K. Schueller, and Y. Ampatzidis. 2023. Detecting two-spotted spider mites and predatory mites in strawberry using deep learning. Smart Agricultural Technology 4: 100229.
  • Type: Journal Articles Status: Other Year Published: 2023 Citation: Zhou, C., Lee, W.S., Zhang, S., Pourreza, A., Liburd, O.E., Schueller, J.K., Ampatzidis, Y. A. Smartphone application for site-specific pest management based on deep learning and spatial interpolation. Plan to submit it to Computers and Electronics in Agriculture (Ready for submission).
  • Type: Conference Papers and Presentations Status: Other Year Published: 2023 Citation: Zhou, C., Lee, W.S., Kratochvil, W., Schueller, J.K., Pourreza, A. (2023) A portable imaging device for twospotted spider mite detection in strawberry. ASABE annual meeting (Oral presentation).
  • Type: Conference Papers and Presentations Status: Other Year Published: 2023 Citation: Zhou, C., Lee. W.S., Zhang, S., Pourreza, A. (2023) A smartphone application for mapping the populations of two-spotted spider mites in strawberry. 2023 AI in Agriculture Conference: Innovation and Discovery to Equitably Meet Producer Needs and Perceptions, Orlando, Florida (Oral presentation).


Progress 06/01/21 to 05/31/22

Outputs
Target Audience:Strawberry, almond, and other vegetable crop growers Pest control advisors (PCAs) Changes/Problems: We requested a one-year no-cost extension to complete all objectives in the proposal. The major reasons for applying extension for this project were: (1) Due to the COVID-19 shutdown in 2020, all research activities were discontinued since mid-March, including field image acquisition and testing since the strawberry field was not accessible. As a result, we could not collect enough data in the first year of this project, and the data acquisition was delayed to the next growing season. (2) We were planning to develop smartphone imaging hardware to facilitate leaf imaging in 2019; however, we decided to develop a standalone six-camera imaging device to collect leaf images automatically. Therefore, the second objective of this project was changed. Development of a standalone six-camera imaging box would be more challenging but will be more useful for image acquisition. In addition, one more growing season will be needed to collect thousands of images using the six-camera imaging box to train deep learning models. Dr. Surendra K Dara had a new job and left the University of California, Cooperative Extension. What opportunities for training and professional development has the project provided?We have trained three undergraduate students at the University of Florida in this project. They gained new skills in pest management, artificial intelligence, smartphone app development, and precision agriculture. How have the results been disseminated to communities of interest?We presented the TSSM detection results and smartphone app at multiple conferences and symposiums. What do you plan to do during the next reporting period to accomplish the goals? The six-camera imaging box will collect enough images from strawberry and almond fields in California, and the images will be labeled for deep learning model training and testing. An iOS app will be developed for TSSM detection. The six-camera imaging box will be integrated with the smartphone app. We will evaluate the applicability of the six-camera imaging box and smartphone app for TSSM counting in the field. We will analyze the economic benefits of the commercial version of the TSSM monitoring tool.

Impacts
What was accomplished under these goals? During the 2021-2022 strawberry growing season (October 2021 - April 2022) in Florida, bare-root strawberry plants consisting of cultivars "Sweet Charlie" was established in the Small Fruit and Vegetable IPM greenhouse at the University of Florida in Gainesville, Florida. Strawberry seedlings were transplanted into 20-cm diameter pots filled with organic potting soil and watered as needed. Plants were given 2-3 g of Osmocote fertilizer after one week to promote growth. A colony of two-spotted spider mites was started on Pinto bean (Phaseolus vulgaris L.) using mites harvested from a strawberry field site in Citra, FL. A 25 x 51 cm tray of young bean plants was kept in a fine mesh cage to prevent mites from escaping and infesting other plants. The cage was refreshed with a new tray of beans every three weeks, and the mites were given one week to infest the new tray before the old one was removed. After the colony was established, the strawberries were infested with spider mites and kept in fine mesh cages to prevent the mites from spreading elsewhere. Once the strawberries reached a high level of infestation, plants were harvested and delivered to Dr. Lee's lab upon request. Two batches with several infested strawberry plants were delivered to Dr. Lee's lab. In California, a TSSM colony was established for image acquisition. In Florida, three different smartphones were used to collect more images for deep learning model training. Currently, we have labeled 3013 smartphone images for model training. We used the customized single-camera imaging platform and six-camera imaging box to collect thousands of images for deep learning model training and testing. We compared the Faster RCNN and YOLOv4 models for TSSM detection and found that YOLOv4 could achieve higher detection accuracy. In addition, we also found that different smartphones will affect the mite detection accuracy. We are working on the manuscript, which will be submitted to a peer-reviewed journal soon. More functions were added to the Android app. The android app was tested on different smartphones. Based on the preliminary result, we found the model inference speed can meet the real-time application requirement. The second prototype of the six-camera imaging box was developed with some improvements in both hardware and software. The light source has been modified such that the whole sampling bed receives a uniform illumination for each sampling section. The bed has been modified such that a grid appears in every section of the bed for facilitating image processing. An external power converter has been added to this device; thus, the user does not have to use the battery during image collection and data transfer. To view the whole sampling process, a camera was added to the sensor to monitor data collection, and the video can be livelily viewed on the LCD during the imaging. We have been working on an article on the mechanization of the specialty crop industry. The article aims to analyze the need for mechanization and its adoption in the industry. Several factors point to the urgent need for research and development of mechanization and automation technologies in the industry, including labor shortages, escalating labor costs, and intense competition from countries with low labor costs, particularly Mexico. For example, Florida is increasing the minimum wage to $15 per hour by 2026, which represents a 75% increase compared to the $8.56 per hour. Because of labor shortages, the specialty crop industry is increasingly relying on foreign guest workers admitted under the H-2A visa program, which is costing more than domestic workers. The article will provide an overview of mechanization adoption in the specialty crop industry, including research and development of labor-saving technologies in the strawberry industry. In addition to the work on mechanization, our team is also building a theoretical framework for analyzing how innovation will affect specialty crop industry performance and its trajectories under different scenarios of technological progress. This framework will be particularly relevant for the strawberry industry as it has been facing increasing competition from Mexico.

Publications

  • Type: Conference Papers and Presentations Status: Published Year Published: 2021 Citation: Zhou, C., Lee, W.S., Liburd O, & Aygun, I. (2021). Smartphone-based tool for two-spotted spider mite detection in strawberry. American Society of Agricultural and Biological Engineers 2021 Annual Meeting. (Paper and presentation)
  • Type: Conference Papers and Presentations Status: Submitted Year Published: 2022 Citation: Zhou C., Lee W.S., Pourreza A., Schueller, J.K., Liburd O.E., Ampatzidis Y., Ramirez, G. 2022. Strawberry pest detection using deep learning and automatic imaging system. Proceedings of the 15th International Conference on Precision Agriculture. (Paper submitted)
  • Type: Other Status: Published Year Published: 2022 Citation: Zhou C., Lee W.S., Pourreza A., Liburd O.E., Schueller, J.K., Ampatzidis Y. 2022. Strawberry Pest Detection Using Artificial Intelligence and Smartphone Image. Envisioning 2050 in the Southeast AI Driven Innovations in Agriculture. (Poster)
  • Type: Other Status: Published Year Published: 2022 Citation: Zhou C., Lee W.S., Pourreza A., Liburd O.E., Schueller, J.K., Ampatzidis Y. 2022. Comparing Deep Learning Methods for Strawberry Pest Detection. UF ABE Poster Symposium. (Poster)
  • Type: Conference Papers and Presentations Status: Published Year Published: 2022 Citation: Zhou C., Lee W.S., Pourreza A., Liburd O.E., Schueller, J.K., Ampatzidis Y. 2022. Strawberry Pest and Predatory Mite Detection Using Artificial Intelligence. Florida Section of American Society of Agricultural and Biological Engineers 2022 Annual Meeting. (Presentation)


Progress 06/01/20 to 05/31/21

Outputs
Target Audience:Strawberry, almond, and other vegetable crop growers Pest control advisors (PCAs) Changes/Problems: Lewis mite has been a pest of strawberries in California for about 10 years. It started in the Oxnard area and is now in all strawberry production regions on the coast. It is very similar to the two-spotted spider mite, and the pattern of the markings on their dorsal sides differentiate these two species. An earlier study found that Phytoseiulus persimilis does not feed on Lewis mites. Because of this reason, growers need to be able to figure out what kind of mite(s) they have and use the biocontrol agents accordingly. Both species respond similarly to pesticides and other control options. Dr. Kris Tollerup left the University of California as of December 2020. Dr. David Haviland, UC ANR, took over the roles and responsibilities. Dr. Feng Wu has a new job and left the University of Florida in April 2021. Dr. Zhengfei Guan at the University of Florida will replace his roles and responsibilities. What opportunities for training and professional development has the project provided? Nothing Reported How have the results been disseminated to communities of interest?We present the TSSM detection results at the Entomological Society of America 2020 Annual Meeting. What do you plan to do during the next reporting period to accomplish the goals? Use the 6-camera imaging device to collect spider mite and predatory mite images of strawberry leaf and almond leaf, which will be used for deep learning model training. Keep developing the smartphone app for spider mite detection. Evaluate the applicability of the 6-camera imaging device and smartphone app in the field. Analyze the economic benefits of the spider mite monitoring tool. We will 1) record the time efficiency of scouts with the new tool to detect mites and 2) determine the cost-effectiveness of the new tool.

Impacts
What was accomplished under these goals? We established both the greenhouse and field population of spider mites in Florida. In early spring (strawberry season), bare-root strawberry plants consisting of cultivars "Florida Brilliance" and "Florida Sensation" were established at the University of Florida, Plant Science Research and Education Unit, Citra, FL, and in the Small Fruit and Vegetable IPM greenhouse at the University of Florida in Gainesville, Florida. In the greenhouse, strawberry seedlings were transplanted into 20-cm diam. pots filled with organic potting soil and watered as needed. Plants were given 2-3 g of Osmocote fertilizer after one week to promote growth. While the strawberry plants were growing to a useable size, a colony of two-spotted spider mites was started on Pinto bean (Phaseolus vulgaris L.) using mites harvested from a strawberry field site in Citra, FL. A 25 x 51 cm tray of young bean plants was kept in a fine mesh cage to prevent mites from escaping and infesting other plants. The cage was refreshed with a new tray of beans every 3 weeks, and the mites were given 1 week to infest the new tray before the old one was removed. After the colony was established, the strawberries were infested with spider mites and kept in fine mesh cages to prevent the mites from spreading elsewhere. Once the strawberries reached a high level of infestation, plants were harvested and delivered to Dr. Lee's lab in batches of 4 infested strawberry plants once every week for 7 weeks until the supply of strawberry plants was exhausted. In California, we established a Lewis mite colony for image acquisition. We used two different smartphones to collect images for deep learning model training. Currently, we have thousands of images taken by different smartphones, which can be used to demonstrate that different smartphone cameras may not affect spider mite detection performance in the next reporting period. Several deep learning models, including YOLOV3, YOLOV3 tiny, YOLOV4, and YOLOV4 tiny, were trained on spider mite detection using all labeled images. The current result shows that YOLOV4 tiny model could be effectively used in spider mite detection in our Android app. One Android app was developed, which can use a deep learning model (YOLOV4 tiny) to detect spider mites from images or videos. The smartphone's current location (GPS) can also be recorded. One single camera imaging device and one 6-camera imaging box were developed for spider mite image acquisition and being tested. Objective 4 is to compare the economic effects of using the new tool with those of using the conventional tool of identifying spider mites. We have 1) collected the cost information of PCA services provided to strawberry growers in Florida. Generally, a scout charged strawberry growers $4.25 per acre for each visit. Scouts visit fields weekly starting from the third week of October. 2) collected time efficiency of monitoring spider mites. In addition to mites, scouts also monitor disease, insects, weeds, and general symptoms of strawberries within the field. A scout can monitor 100 acres per hour. But almost 70% of scouting time is spent on examining fields for the presence of TSSMs. Scouts sample ~50-100 leaves per block, and it takes about 10 seconds for them to examine each leaf.

Publications

  • Type: Conference Papers and Presentations Status: Published Year Published: 2020 Citation: Zhou, C., Lee, W.S., Aygun, I., & Liburd O. (2020). Detecting Two-Spotted Spider Mite Tetranychus urticae Koch (Acari: Tetranychidae) and Mite Egg under Strawberry Leaf Using Deep Learning. Entomological Society of America 2020 Annual Meeting (Invited).
  • Type: Conference Papers and Presentations Status: Published Year Published: 2020 Citation: Zhou, C., & Lee, W.S. (2020). Detection of Two-Spotted Spider Mite in Strawberry Using Artificial Intelligence. UF ABE Virtual Poster Symposium Presentation.


Progress 06/01/19 to 05/31/20

Outputs
Target Audience:Strawberry, almond, and other vegetable crop growers Pest control advisors (PCAs) Changes/Problems:From November 2019 to February 2020, we collected strawberry leaf samples in the fields in Florida, and then the TSSM population dropped dramatically and it became hard to find strawberry leaf infested with TSSM or other pests. We were planning to maintain TSSM colony in a greenhouse in the February 2020. Our collaborator in California said they had TSSM in their field and they could send some spider mites to us for starting TSSM colony in the greenhouse. However, we found that our collaborator couldn't send their spider mites across state line without permission. Then, with the coronavirus outbreak in USA, our campus in FL was closed and no research activity could be conducted in the lab or greenhouse. Therefore, we collected less images than we expected. During next project period, more images will be collected. Using a multi-camera imaging system instead of using a single camera (smartphone camera). Reason: mites move constantly, and imaging should be conducted in minimum time. Frame rate in high-resolution (that is required for this application) is usually not very fast. Using an array of cameras and a global shutter will significantly increase imaging speed, so that mite movements will not cause problems in image processing. What opportunities for training and professional development has the project provided? Nothing Reported How have the results been disseminated to communities of interest? Nothing Reported What do you plan to do during the next reporting period to accomplish the goals?We will establish both a greenhouse and field population of mites starting from October 2020. When we used a smart-phone and macro lens to collect image, we found the image quality was very low (blurred and shaded areas). A multi-camera imaging system with uniform illumination will be developed using a Raspberry pi, which can help us collect high-quality image and further improve spider mite detection accuracy. More images will be collected using the multi-camera imaging system. Due to the COVID-19 situation, we couldn't collect enough images as we expected. Therefore, we need to spend more time in collecting images during the next reporting period. More deep learning methods will be tested and smart-phone application will be developed. We will 1) collect the cost information of PCA service provided to Florida strawberry growers; 2) record working hours of growers or scouts with the new tool to detect mites; 3) compare chemical uses recommended by the two tools; and 4) determine the cost effectiveness of the new tool.

Impacts
What was accomplished under these goals? In order to automatically count the number of spider mites on the strawberry leaf, a total of 703 strawberry leaf images were collected using a smart-phone and a macro lens, and the deep learning method was tested to detect the spider mite and mite eggs from the images. However, the detection accuracy for spider mite and mite eggs is below 0.6, which need to be further improved in the near future. Objective 4 is to compare the economic effects of using the new tool with those of using the conventional tool of managing spider mites. Generally, growers contracts with a private pest control advisor (PCA) to assist with pest management and decisions. PCAs use the conventional tool to detect mites. We have collected the cost information of PCA service provided to strawberry and almond growers in California. An estimated cost of $125 per acre is charged to strawberry growers in California. The PCA charge for almond growers depends on the orchard year and sometimes is reflected in the price of fertilizer and chemicals as part of the service contract between growers and PCAs. This cost will be compared to that of using the new tool to determine the cost effectiveness of the new tool.

Publications

  • Type: Conference Papers and Presentations Status: Accepted Year Published: 2020 Citation: Zhou C., Lee W.S., Aygun I., 2020. Detecting Two-Spotted Spider Mite and Mite Egg under Strawberry Leaf using Deep Learning. The Proceedings of the 15th International Conference on Precision Agriculture. Minneapolis, USA.
  • Type: Other Status: Published Year Published: 2020 Citation: Zhou C., Lee W.S., Aygun I., 2020. YOLO for Two-spotted Spider Mite Monitoring in Strawberry. Pathways Towards the Next Generation of Agriculture and Natural Resources in Florida, University of Florida, Gainesville, January-16, 2020. (Poster presentation)