Progress 06/01/19 to 05/31/23
Outputs
Target Audience:Organic apple farmers, apple processors (domestic and export), apple commodity associations, academics, policymakers, AgriTech companies, regulatory (FDA), and funding agencies ( USDA-NIFA). Changes/Problems:The major change made to the project included the addition of the acoustic impulse technique to improve the result obtainable. The idea of measuring attenuated sound through the depth of the apple and then applying changes caused by infestation to delineate healthy apples from infested was not initially considered. This addition turned out to be a great plus for the project because a shorter data collection time (60 - 80 milliseconds) was obtained compared to the 5 - 10 s we obtained in the vibro-acoustic method that was initially proposed. Also, the classification accuracy from the test set data was almost perfect ( ~98%) compared to the vibro-acoustic method. There was a slight budget change to repair and upgrade the hyperspectral imaging system. This was less than the 10% change that should have triggered budget re-approval. Notwithstanding, USDA permission was sought before this minor change to the budget was effected. What opportunities for training and professional development has the project provided? During the four-year duration of this project, the PIs were availed the opportunity of training two graduate students (one M.S. and one Ph.D.) and one post-doctoral scholar. All of them have left to take up positions within the public and private sectors in the U.S. Two undergraduate students were also trained on the project. Knowledge gained on the project also availed the lead PI the opportunity to host and organized a one-week workshop for two visiting scholars/professors from Nigeria on AI/Nondestructive technology applications in the agrifood systems. How have the results been disseminated to communities of interest?Findings from the project in the final year were disseminated through the following outlets: American Society of Agricultural and Biological Engineers (ASABE) that were held in Houston TX and Omaha NE July 17 - 21, 2022, and July 9 - 13, 2023, respectively. USDA-NIFA AI in Agriculture: Innovation and Discovery to Equitably Meet Producers' Needs and Perceptions" held in Orlando Florida on April 17 - 19, 2023. Publication in subscription-based and open-access journals: Foods, Sustainability, Agriculture, Journal of ASABE, and Biosystems Engineering. Overall, the findings from the project in four years were disseminated at different national and international scientific conferences and symposia. Two USDA-NIFA sponsored AI conferences in 2022 and 2023 Adedeji, A.A., Ekramirad, N., Al Khaled, Y.A., Donohue, K., and Villanueva, R. (2023). Sensor Data Fusion and Machine Learning Approach for Pest Infestation Detection in Apples. A poster presented at the SEC Conference with the theme: "USDA-NIFA AI in Agriculture: Innovation and Discovery to Equitably Meet Producers' Needs and Perceptions" held in Orlando Florida on April 17 - 19, 2023. Al Khaled, Y.A., Ekramirad, N., Parrish, C.A., Donohue, K., Villanueva, R., and Adedeji, A.A. (2022). Acoustic application for codling moth detection in apples: high versus low frequency sensing. Poster presented at the SEC conference, Envisioning 2050 in the Southeast: AI-driven Innovations in Agriculture. Held at Auburn University from March 9 - 11, 2022. Ekramirad, N., Al Khaled, Y.A., Donohue, K., Villanueva, R., and Adedeji, A.A. (2022). Quality prediction of codling moth infested apples under long-term storage using hyperspectral imaging and machine learning. Poster presented at the SEC conference, Envisioning 2050 in the Southeast: AI-driven Innovations in Agriculture. Held at Auburn University from March 9 - 11, 2022. At the 2020, 2021, and 2022 annual international meetings of the American Society of Agricultural and Biological Engineers (ASABE). Doyle, L.E., Loeb, J.R., Ekramirad, N., Santra, D., Adedeji, A.A. (2022). Non-destructive classification and quality evaluation of proso millet cultivars using NIR hyperspectral imaging with machine learning. A paper presented (oral) during the Annual International Meeting of the American Society of Agricultural and Biological (ASABE) held in Marriott Marquis Houston Texas from July 17 - 20, 2022. Paper #: 2200944 Ekramirad, N., Al Khaled, Y.A., Villanueva, R., Parrish, C.A., Donohue, K., and Adedeji, A.A. (2022). Development of a sensor fusion model based on acoustic and hyperspectral imaging features with machine learning to classify codling moth-infested apples. A paper presented (oral) during the Annual International Meeting of American Society of Agricultural and Biological (ASABE) held in Marriott Marquis Houston Texas from July 17 - 20, 2022. Paper #: 2200941 Ekramirad, N., Al Khaled, Y.A., Donohue, K., Villanueva, R., Parrish, C.A., and Adedeji, A.A. (2021). NIR Hyperspectral Imaging with machine learning to detect and classify codling moth infestation in apples. A paper in the proceeding and presented (poster) during On-Demand Simulated Live Session at 2021 Annual International Virtual Meeting of American Society of Agricultural and Biological (ASABE) held online from July 12 - 15, 2021. Paper #: 2100066. JC -2 Al Khaled, Y.A., Ekramirad, N., Donohue, K., Doyle, L., Villanueva, R., Parrish, C.A., and Adedeji, A.A. (2021). Effects of low-intensity heat stimulation on ultrasonic acoustic emission detection of codling moth larvae activities in apples (On paper: Vibro-acoustic emission and heat stimulation effect on the detection of codling moth larvae in apples). A paper in the proceeding and presented (poster) during On-Demand Simulated Live Session at 2021 Annual International Virtual Meeting of American Society of Agricultural and Biological (ASABE) held online from July 12 - 15, 2021. Paper #: 2100070 Ekramirad, N., Donohue, K., Villanueva, R., Parrish, C.A., and Adedeji, A. A. (2020). Low frequency signal patterns for codling moth larvae activity in apples. A paper in the proceeding and presented (poster) during On-Demand Q&A oral session at 2020 Annual International Virtual Meeting of American Society of Agricultural and Biological (ASABE) held online from July 13 - 15, 2020. Paper #: 2001028. Oral. At the Canadian Society of Bioengineering (CSBE) annual meeting in 2022. Adedeji, A.A., Ekramirad, N., Al Khaled, Y.A., Parrish, C.A., Donohue, K., and Villanueva, R. (2022). Acoustic and hyperspectral imaging sensing for nondestructive insect infestation detection in apples. A paper presented (oral) during the Annual International Meeting of the Canadian Society of Bioengineering (CSBE) held at Delta Hotel - PEI Convention Center, Charlottetown, Prince Edward Island, Canada from July 24 - 27, 2022. Six peer-reviewed journal articles and one book chapter were published in mostly high-impact factor peer-reviewed journals. Peer-Reviewed Journals Khaled, Y.A., Ekramirad, N., Donohue, K., Villanueva, R., and Adedeji, A.A. (2023). Non-destructive hyperspectral imaging and machine learning-based predictive models for physicochemical quality attributes of apples during storage as affected by codling moth infestation. Agriculture - Digital Agriculture 13(5), 1086. Ekramirad, N., Khaled, Y.A., Donohue, K., Villanueva, R., and Adedeji, A.A. (2023). Classification of codling moth-infested apples using sensor data fusion of acoustic and hyperspectral features coupled with machine learning. Agriculture - Agricultural Technology 13(4), 839. Khaled, Y.A., Ekramirad, N., Parrish, C.A., Eberhart, P.S., Doyle, L., Donohue, K.D., Villanueva, R., and Adedeji, A.A. (2022). Nondestructive detection of codling moth infestation in apples using acoustic impulse response signals. Biosystems Engineering 224, 68-79. Ekramirad, N., Khaled, Y.A., Doyle, L., Loeb, J., Donohue, K.D., Villanueva, R., and Adedeji, A.A. (2022). Nondestructive detection of codling moth infestation in apples using pixel-based NIR hyperspectral imaging with machine learning and feature selection. Foods 11(8), 1 - 16. Ekramirad, N., Al Khaled, Y.A., Donohue, K., Villanueva, R., Parrish, C.A., and Adedeji, A.A. (2021). Development of pattern recognition and classification models for the detection of vibro-acoustic emissions from codling moth infested apples. Postharvest Biology and Technology 181, 111633 Ekramirad, N., Adedeji, A.A, Al Khaled, Y.A., and Villanueva, R. Impact of storage on nondestructive detectability of codling moth infestation in apples. Journal of ASABE. Submitted on March 4, 2023. Book Chapter Adedeji, A.A., Ekramirad, N., Khaled, Y.A., and Parrish, C.A. (2022). Acoustic emission and near-infrared imaging for nondestructive apple quality detection and classification. In Nondestructive Quality Assessment Techniques for Fresh Fruits and vegetables (Eds. P.B. Pathare and M.S. Rahman). Springer Nature. ISBN-13: 9789811954214. Chapter 13. What do you plan to do during the next reporting period to accomplish the goals?
Nothing Reported
Impacts
What was accomplished under these goals?
Overall Project Accomplishment: This project aimed to develop nondestructive methods based on acoustic sensing and hyperspectral imaging (HSI) approach coupled with advanced machine learning (ML) to effectively sort apples for codling moth (CM) infestation, the most devastating pest of apples. Our target audiences include AgriTech companies, apple farmers, commodity associations, colleagues in the academia who are working on related ideas, regulatory agencies who set policies and guidance for the prevention of organisms of quarantine concerns, and USDA-NIFA as reference for future funding opportunities. The expectations were that we would develop algorithms that apply individual and combined sensor data for nondestructive detection and quality evaluation of CM-infested apples. Below is a summary of our accomplishments under each objective: Objective 1: Authentication of acoustic signals source from CM-infested apples: This objective was critical in guiding how sensors are placed on apples for better data collection and in signal data processing, especially regarding setting baseline acoustic feature levels for developing appropriate ML classifiers for delineating between the control and infested apples. This was accomplished by using low-frequency (0.4 - 8 Hz) piezoelectric sensors to determine the main larvae displacement activities in and around the apples during the CM's life cycle. These activities were determined to include chewing, crawling, and boring. The intensity of each activity was determined as a function of signal amplitude, pulse duration, and frequency. To match acoustic signals to larvae activities, we used a digital camera with an adjustable fixture for monitoring and recording larva activities simultaneously. Patterns of different larva activities were obtained by correlating the video recording of larva activities to the vibro-acoustic signals being collected. The results illustrate that the signals from infested samples are clearly different from the ones from non-infested samples. It was observed that the boring moment has the highest signal strength compared to the crawling and chewing. The CM larva boring into apple pulp generated the highest amplitude and pulse duration. While the larva crawling action on the apple surface produced lower amplitude and pulse duration than other categories. Also, the peak frequency of larva crawling pulses had a mean significantly higher than the other two signals. The peak frequency of chewing and boring into flesh moments was not significantly different and had a large overlap, meaning that the chewing frequency is the dominant frequency in CM-infested apples. These results suggest that CM boring activities are mostly responsible for the vibro-acoustic signals detected in infested apples. Objective 2: Improve the test performance of noninvasive detection of CM-infested apples using combined HSI and AE sensor data coupled with ML. This objective was divided into different studies that assessed two acoustic sensing approaches, the use of HSI to evaluate apples for CM infection, and the potential of fusing sensor data from both methods to improve outcomes. Objective 2 - vibro-acoustic CM detection and classification: Artificially infested apples were held in specially designed sample holders connected with two low-frequency vibro-acoustic sensors in a way that allowed contact with the apple samples. Acoustic data were extracted with 21 features and ML models were applied. Initial results with limited samples produced good results with an overall accuracy of 98.6% for test-set data. However, as the set grew this was not repeatable, the accuracy dropped to about 80% in cross-validation studies. There were 2 issues observed. One was the need for this sensor to have contact with the apple and the sensitivity to ambient vibrations in the room (noise). The overall sensitivity to activity inside the apple was variable depending on the apple's contact with the sensor, which greatly varied the signal-to-noise ratio from apple to apple. The 2nd issue was the need to have the worm in an active state. To increase the likelihood of an active CM while scanning, the scan time can be increased. However, this is not practical. Alternatively, we applied heat stimulation (30°C) and recorded an improvement to the results, 94%. The 1st issue may be overcome with non-contact sensors (i.e., a laser vibrometer) that are more sensitive to sound vibrations. In conclusion, we established vibro-acoustic with external stimulation improves the sensitivity of CM detection. Objective 2 - Acoustic Impulse for CM detection and classification: This approach entailed the application of piezoelectric sensors to collect attenuated signals made from an impulse generator. The experiment was performed on control and artificially infested apples from three different cultivars - Gala, Fuji, and Granny Smith. Signals were recorded with a contact sensor, and 21 signal features were extracted to characterize relevant properties of the response. The features were evaluated using 11 classifiers to determine their impact on classification accuracy. Performance was evaluated using various feature subsets as well as all 21 features combined. The overall classification rates were between 80% and 92% for Fuji apples, between 92% and 99% for Gala apples, and 63% and 97% for Granny Smith apples. A very short data collection time of 60-80 ms per sample was observed. This is a significant improvement over the vibro-acoustic method. Objective 2 - HSI application to detect and classify CM-infested apples: Short-wave infrared (SWIR) HSI sensor in the wavelength range of 900-1700 nm was applied to collect data for the detection of CM infestation at the pixel level for three apple cultivars - Fuji, Gala, and Granny smith. A standard procedure was implemented to segment the region of interest to extract the spectral features which were then fed into ML classification models. Also, the optimal wavelengths were selected to develop the multispectral imaging models for faster classification purposes. The results indicated that the ensemble model could reach an accuracy of up to 97.4 % in the classification of the infested and healthy samples. The models with the optimum features obtained a classification rate of 91.6 % from 22 selected wavelengths. Pixel-based HSI showed the effectiveness of the method for the nondestructive classification of CM. Objective 3 - Determine the impact of storage conditions on detectability and quality of apple: The effect of storage at 3 temperatures (0, 4, & 10°C and 89% relative humidity) for a period of 90-140 days was observed on the detectability of CM infection using HSI method. Apple samples were tested periodically for 20 weeks. SWIR HSI was implemented to build ML models for predicting the quality attributes of Gala apples during storage using partial least squares and support vector regressions (SVR). The results showed that the quality prediction models for apples during cold storage at three different temperatures were relatively accurate with correlation coefficients of prediction (Rp) up to 0.90, 0.93, 0.97, and 0.91 for SSC, firmness, pH and moisture content, respectively. Also, the results of the functional ANOVA test revealed that CM infestation significantly (P<0.05) affects the apple's NIR spectra. Both acoustic and HSI methods showed very high accuracy greater than 90% in classifying infested apples. Final year summary. Objective two - Sensor Data Fusion: The project's final year was dedicated to developing a sensor data fusion approach that fuses features from both the acoustic sensors and the HSI sensor data. The goal is to harness the strength of both sensors. Applying different classifiers, the combined data from AE and HSI for selected features gave an accuracy of 94%, which is an improvement for individual results for each sensor.
Publications
- Type:
Journal Articles
Status:
Published
Year Published:
2023
Citation:
Khaled, Y.A., Ekramirad, N., Donohue, K., Villanueva, R., and *Adedeji, A.A. (2023). Non-destructive hyperspectral imaging and machine learning-based predictive models for physicochemical quality attributes of apples during storage as affected by codling moth infestation. Agriculture Digital Agriculture 13(5),1086.
- Type:
Journal Articles
Status:
Published
Year Published:
2023
Citation:
Ekramirad, N., Khaled, Y.A., Donohue, K., Villanueva, R., and *Adedeji, A.A. (2023). Classification of codling moth-infested apples using sensor data fusion of acoustic and hyperspectral features coupled with machine learning. Agriculture - Agricultural Technology 13(4), 839.
- Type:
Journal Articles
Status:
Published
Year Published:
2022
Citation:
Khaled, Y.A., Ekramirad, N., Parrish, C.A., Eberhart, P.S., Doyle, L., Donohue, K.D., Villanueva, R., and *Adedeji, A.A. (2022). Nondestructive detection of codling moth infestation in apples using acoustic impulse response signals. Biosystems Engineering 224, 68-79.
- Type:
Journal Articles
Status:
Submitted
Year Published:
2023
Citation:
Ekramirad, N., Adedeji, A.A, Al Khaled, Y.A., and Villanueva, R. Impact of storage on nondestructive detectability of codling moth infestation in apples. Journal of ASABE
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2023
Citation:
Adedeji, A.A., Ekramirad, N., Al Khaled, Y.A., Donohue, K., and Villanueva, R. (2023). Sensor Data Fusion and Machine Learning Approach for Pest Infestation Detection in Apples. A poster presented at the SEC Conference with the theme: USDA-NIFA AI in Agriculture: Innovation and Discovery to Equitably Meet Producers Needs and Perceptions held in Orlando Florida on April 17 19, 2023.
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2022
Citation:
Adedeji, A.A., Ekramirad, N., Al Khaled, Y.A., Parrish, C.A., Donohue, K., and Villanueva, R. (2022). Acoustic and hyperspectral imaging sensing for nondestructive insect infestation detection in apples. A paper presented (oral) during the Annual International Meeting of the Canadian Society of Bioengineering (CSBE) held at Delta Hotel - PEI Convention Center, Charlottetown, Prince Edward Island, Canada from July 24 27, 2022.
|
Progress 06/01/21 to 05/31/22
Outputs
Target Audience:Apple (organic) farmers, apple processors (domestic and export), apple commodity associations, policymakers, and funding agencies. Changes/Problems: The first change includes the addition of the acoustic impulse (high frequency ) method in addition to the vibro-acoustic method proposed in the project initially. This allowed us to explore every aspect of acoustic sensing. This method showed a better classification accuracy and a significantly lower data collection time in milliseconds compared to the vibro-acoustic method even though it is more expensive. The matched-filter approach to the vibro-acoustic method is another important addition. This addition (matches filter for pre-separation of acoustic signals based on pre-determined signal patterns from video-matched signal recording) improved the classification result from this method. The delay caused by the pandemic slowed the project. Hence, the request for a no-cost extension. What opportunities for training and professional development has the project provided?1. Nader Ekramirad: Is a Ph.D. candidate student working on the project. The project has allowed him to be trained in hyperspectral imaging (HSI) and acoustic (AE) signal data analyses. He also acquired skills in advanced machine learning approaches and learned implementation on the python platform. He will graduate in June 2022. 2. Chadwick A. Parrish, a Master's degree graduate was hired and trained on the project to acquire skills in machine learning - convoluted artificial neural network, plsr, svm, etc.; acoustic signal system design, hyperspectral imaging, etc. 3. Lauren Doyle and Julia Loeb are undergraduate student research assistants who were hired to help with some of the lab work while also being trained on AE and HSI systems handling, data collection, and machine learning applications for analyzing HSI and AE data. How have the results been disseminated to communities of interest?Yes, the results of findings from the project were disseminated at different forums in the last year, namely: 1. American Society of Agricultural and Biological Engineers (ASABE) that was held virtually online from July 12 - 15, 2021. 2. USDA funded Southeast Conference on AI dubbed "Envisioning 2050 in the Southeast: AI-driven Innovations in Agriculture" that was held at Auburn University from March 9 - 11, 2022. 3. Canadian Society of Bioengineering (CSBE) holding in Charlestown Prince Edward Island, Canada from July 24 - 27, 2022. In the current year: 4. We have published proceeding papers in international conferences (4) and made both oral and poster presentations 5. We have published two papers in a high-impact factor journal (Foods and Postharvest Biology and Technology), and two others under review in elite journals (Computer and Electronics in Agriculture, and IEEE Transactions on Instrumentation and Measurement). There are two other peer-reviewed papers being prepared for publication and one book chapter was accepted for publication. What do you plan to do during the next reporting period to accomplish the goals?We intend to complete the quality prediction and sensor data fusion data analysis aspect of the project, and wrap up the project with suggestions for future work.
Impacts
What was accomplished under these goals?
Impact Statement: Codling moth (CM) is the most devastating pest of pome fruits like apples. This is a major problem, especially for organic apple farmers. The occurrence of infestation diminishes the value of apples significantly and it is costly for stakeholders, particularly exporters of US apples. The current method of detection is mostly manual and ineffective. In this project, the primary goal was to develop robust and efficient nondestructive methods based on hyperspectral imaging (HSI) and acoustic signal sensing methods coupled with advanced machine learning to detect and classify infested apples from healthy ones. Each has its limitation - HSI has a limit of penetration but can detect CM eggs on surfaces, while acoustic can detect internal morphological damages caused by CM but can not detect CM eggs on a fruit surface. Our goal is to determine the individual and fused strength of both methods. The end-use is to deploy these classification models into apple processing plant sorting systems or hand-held systems to help in better detection of codling moth infestation of apple and to replace the manual and random methods commonly used that are inefficient. We have three specific goals, namely: to determine the source of the acoustic signals for better sensing of acoustic signals, to improve the classification results accuracies and determine the impact of storage on the classification rate, and lastly, to fuse sensor data for improved result. The last objective is currently being addressed. The first two objectives were completed in part in the first two years of the project, except for the acoustic impulse and storage effect-detectability aspects. In the current reporting year, we focused on completing data collection on the storage and heat simulation aspect of acoustic sensing. We also completed data collection on the HSI method application including quality evaluation in order to develop nondestructive models for them. Further analysis of new and collected data were completed and several manuscripts are being prepared, submitted, or published. Several conference presentations were made at international and national conferences. Objective two - Acoustic Impulse for CM detection and classification: This approach entails the application of piezoelectric sensors to collect attenuated signals made from an impulse generator. The experiment was performed on control and artificially infested apples from three different cultivars. Signals were recorded with a contact sensor, and 21 signal features were extracted to characterize relevant properties of the response. The features were evaluated using 11 classifiers to determine their impact on classification accuracy. Performance was evaluated using various feature subsets as well as all the 21 features combined. The overall classification rates were between 80% and 92% for Fuji apples, between 92% and 99% for Gala apples, and 63% and 97% for Granny Smith apples. A very short data collection time of 60-80 ms per sample was observed. This is a significant improvement over the vibro-acoustic method where the least time for high accuracy result was 5 s. Objective two - HSI application to detect and classify CM infested apples: Short wave near-infrared (SWNIR) HSI sensor in the wavelength range of 900-1700 nm was applied to collect data for the detection CM infestation at the pixel level for three apple cultivars - Fuji, Gala and Grani. An effective procedure was implemented to segment the region of interest (ROI) to extract the spectral features which were then fed into different machine learning classification models. Additionally, the optimal wavelengths were selected to develop the multispectral imaging models for faster classification purposes. The results indicated that the ensemble model could reach an accuracy of up to 97.4 % in the classification of the infested and healthy samples. The models with the optimum features obtained a classification rate of 91.6 % from 22 selected wavelengths. Pixel-based HSI showed the effectiveness of the method for nondestructive classification of CM. Objective three - detectability of infection under extended storage time: The effect of storage at three temperatures (0°C, 4°C, and 10°C and constant relative humidity of 89%) for a period of 90 days was observed on the detectability of CM infection using HSI method. Apple samples were taken and tested weekly. SWNIR HSI was implemented to build machine learning models for predicting the quality attributes of Gala apples during storage using partial least squares (PLS) and support vector regressions (SVR). The results showed that the quality prediction models for apples during cold storage at three different temperatures (0 °C, 4 °C, and 10 °C) were relatively accurate with correlation coefficients of prediction (Rp) up to 0.90, 0.93, 0.97, and 0.91 for SSC, firmness, pH and moisture content, respectively. Also, the results of the FANOVA (functional ANOVA) test revealed that CM infestation significantly (P<0.05)affects the apple's NIR spectra. Both acoustic and hyperspectral imaging sensing methods showed very high accuracy greater than 90% in classifying infested apples. In the final stage of this project, we will combine (fuse) all and selected features from both methods to determine if improvement to the degree of accuracy will be observed. Our hypothesis is that sensor data fusion will improve the detection and accuracy results. This project has led to the training of two undergraduate research assistants, two graduate students (M.S. and Ph.D.), and one post-doctoral fellow. We have published three peer-reviewed journals and book chapters while four journal articles are at various stages of preparation. We have also published four conference papers and two presentations at international conferences. A full patent was also filed for a unique idea from the project.
Publications
- Type:
Journal Articles
Status:
Under Review
Year Published:
2022
Citation:
Parrish, C.A., Ekramirad, N., Khaled, Y.A., Eberhart, P.S., Donohue, K., Villanueva, R., and Adedeji, A.A. Effects of noise reference integration on deep learning models for codling moth infestation detection. IEEE Transactions on Instrumentation and Measurement.
- Type:
Journal Articles
Status:
Submitted
Year Published:
2022
Citation:
Khaled, Y.A., Ekramirad, N., Parrish, C.A., Eberhart, P.S., Doyle, L., Donohue, K.D., Villanueva, R., and Adedeji, A.A. Nondestructive detection of codling moth infestation in apples using acoustic impulse response signals. Journal of Computers and Electronics in Agriculture. Submitted May 2022
- Type:
Book Chapters
Status:
Accepted
Year Published:
2022
Citation:
Adedeji, A. A., Ekramirad, N., Al Khaled, Y.A., and Parrish, C.A. Vibro-acoustic emission and near-infrared imaging for nondestructive apple quality detection and classification. In Nondestructive Quality Assessment Techniques for Fresh Fruits and vegetables (Eds&..). Springer
- Type:
Theses/Dissertations
Status:
Under Review
Year Published:
2022
Citation:
Nader Ekramirad, Ph.D. Candidate. NONDESTRUCTIVE MULTIVARIATE CLASSIFICATION OF CODLING MOTH INFESTED APPLES USING MACHINE LEARNING AND SENSOR FUSION.
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2022
Citation:
Khaled, Y.A., Ekramirad, N., Parrish, C.A., Donohue, K., Villanueva, R., and Adedeji, A. A. (2022). Acoustic application for codling moth detection in apples: high versus low frequency sensing. Poster presented at the SEC conference, Envisioning 2050 in the Southeast: AI-driven Innovations in Agriculture. Held at Auburn University from March 9 11, 2022.
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2022
Citation:
Ekramirad, N., Al Khaled, Y.A., Donohue, K., Villanueva, R., and Adedeji, A. A. (2022). Quality prediction of colding moth infested apples under long-term storage using hyperspectral imaging and machine learning. Poster presented at the SEC conference, Envisioning 2050 in the Southeast: AI-driven Innovations in Agriculture. Held at Auburn University from March 9 11, 2022.
- Type:
Conference Papers and Presentations
Status:
Accepted
Year Published:
2022
Citation:
Ekramirad, N., Al Khaled, Y.A., Donohue, K., Villanueva, R., and Adedeji, A. A. (2022). Development of a sensor fusion model based on acoustic and hyperspectral imaging features with machine learning to classify codling moth-infested apples. A paper accepted to be presented at the Annual International Meeting of the American Society of Agricultural and Biological (ASABE) that will hold from July 17 20, 2022 in Houston TX, USA.
- Type:
Theses/Dissertations
Status:
Published
Year Published:
2021
Citation:
Chadwick A. Parrish. Vibro-acoustic Codling Moth Larvae Detection in Apples - Deep Learning Application
- Type:
Journal Articles
Status:
Published
Year Published:
2021
Citation:
Ekramirad, N., Al Khaled, Y.A., Donohue, K., Villanueva, R., Parrish, C.A., and *Adedeji, A. A. Development of pattern recognition and classification 1 models for the detection of vibro-acoustic emissions from codling moth infested apples. Postharvest Biology and Technology 181, 111633. DOI: https://doi.org/10.1016/j.postharvbio.2021.111633
- Type:
Journal Articles
Status:
Published
Year Published:
2022
Citation:
Ekramirad, N., Khaled, Y.A., Doyle, L., Loeb, J., Donohue, K.D., Villanueva, R., and Adedeji, A.A. (2022). Nondestructive detection of codling moth infestation in apples using pixel-based NIR hyperspectral imaging with machine learning and feature selection. Foods 11(8), 1 - 16.
|
Progress 06/01/20 to 05/31/21
Outputs
Target Audience:Apple farmers, apple processors (domestic and export), apple community associations, policymakers and funding agencies. Changes/Problems:The COVID-19 pandemic caused a significant delay and loss of resources to pay staff and students when nothing was going on in the lab. My department at the University of Kentucky suffered additional delay because we failed EPA lab inspection and our labs were shut down completely from October to December 2020. My team managed to continue to utilize our time as best as we could. We are hoping that USDA will provide additional resources (cost-extension) to support students and staff to extend their stay beyond the end of the project life scheduled for May 2022. In spite of all the delays, our group remained productive last year, and this is evidenced by the number of scholarly output in students milestone (qualifying exam and defense), conference papers, and peer-reviewed journal articles at various stages of publication. We hope we can get additional resources to help finish the project successfully. What opportunities for training and professional development has the project provided?1. Nader Ekramirad: Is the Ph.D. student working on the project. The project has availed him the opportunity to be trained on acoustic signal data analysis. He also acquired skills in advanced machine learning approaches and learned implementation on python platform. 2.Chadwick Parrish: worked on the project as an M.S. student and he was trained on the application of deep learning approach to analyze acoustic signals from low frequency sensors. 3. Dr. Al Khaled: the post-doctoral researcher on the project developed acoustic signal data analysis skills through mentoring by one of the PIs, and acquired skills in advanced machine learning approach and could implement same on Python platform. 4. Lauren Doyle is an undergraduate student who has been hired to help with some of the lab works but is also being trained in the handling of AE and HSI systems, data collection and machine learning applications for analyzing HSI and AE data. How have the results been disseminated to communities of interest?The results of this project have been disseminated in the following ways: 1. Through the University of Kentucky Communication department who have written several articles on the aim of the project and some of the outcomes we have seen so far. 2. We have published proceeding papers in international conference and made both oral and poster presentations 3. We have published one paper in a high-impact factor journal (Foods), and another one is under review in an elite journal (Postharvest Biology and Technology). There are three other peer-reviewed papers being prepared for publication and one book chapter. What do you plan to do during the next reporting period to accomplish the goals?In the final year of the project, we intend to complete the storage study and the analyses of the remaining data. With this, we would have completed all the objectives we set out with and even more.
Impacts
What was accomplished under these goals?
IMPACT STATEMENT: The primary goal of this project was to develop a robust and efficient nondestructive method based on hyperspectral imaging (HSI) and acoustic emission (AE) sensing methods coupled with advanced machine learning to analyze data for the detection and classification of infestation in apples. The end-use is to deploy these classification models into apple processing plant sorting systems or hand-held systems to help in better detection of codling moth infestation of apple and to replace the manual and random methods commonly used. We had three goals, to determine the source of the acoustic signals, improve the classification results and determine the impact of storage on classification rate. The first two objectives are almost completed, remaining data analysis for some of the storage data being collected due to terminate at the end of July 2021. The classification improvement observed is seen with a test set (validation) result that is greater than 96% in some of the techniques. The determination of the acoustic sound source also helped in understanding signal patterns needed to target in machine learning applications. We are currently completing studies on the effect of storage at three temperatures (0°C, 4°C, and 10°C) for a period of 90 days, where testing is done weekly. This study is scheduled to be completed at the end of July 2021. 1. Authentication and characterization of AE signal source from codling moth (CM) infested apples. The first objective was to authenticate the source of the sound being detected in the passive (natural sound emission) sensing system and use the result to improve the machine learning method with regards to the threshold for acoustic data analysis. We set an acoustic system to detect potential sources of emission from healthy and infested apples on the hypotheses that these signals were coming from both larvae activities and biochemical changes taking place during the normal process of growth/maturity of apples. Method: We accomplished this goal by using high-frequency (0.4 - 8 Hz) sensors (piezoelectric) to determine that three main larvae displacement activities were going on at different intermittent in apple during its life cycle, which includes chewing, crawling, and boring (Figure 1 below). The intensity of each activity was determined as a function of signal amplitude, pulse duration and frequency. In order to match acoustic signals to larvae activities, we installed a digital camera with an adjustable gooseneck fixture for monitoring and recording larval activities. Video recordings were made by the Debut video capturing program (NCH Software Co., USA), which can display the local computer time on video images on the screen. Vibro-acoustic Signal Patterns in Apples: The identification of the signal associated with larvae activity is important because it can be used to study how external stimuli like heat or sound impacts the activity rate. Vibro-acoustic signal patterns related to specific activities in CM-infested apples are shown in Figure 1 as the vibration waveform and associated spectrograms. Patterns of different larval activities, such as crawling, feeding (chewing), boring into fruit (feeding and moving inside flesh), were obtained by placing the CM larvae on the apples, at the same time, video recording of the larval behavior was captured to correlate the vibro-acoustic signals being collected. The results illustrated that the signals from infested samples are clearly different from the ones from non-infested samples as shown in Figs 1A-C and 1D, respectively. For example, feeding signals show a chewing rate of 1 to 2.3 times per second (Fig. 1B) while crawling signals have a faster rate of (3 - 4)/s with lower amplitudes (Fig. 1A). It was observed that the boring moment has the highest signal strength compared to the crawling and chewing. This can be attributed to the direct contact of the larva with the apple pulp while gliding over the juicy surface inside the bored tunnel. These results agree with findings from our previous studies (Ekramirad et al., 2020), which showed high-amplitude broadband impulses as a result of movement and feeding activity of different larvae inside agricultural products. Figure 1 (Fig. 3 in #): Time signals and spectrograms of different CM larva activities moments : (A) crawling, (B) chewing, (C) boring, and (D) control samples. Figure 2 provides a statistical summary for the amplitude, pulse duration, and peak frequency for the three signal types from larval activities (crawling, chewing and boring) in the apple. The CM larval boring into apple pulp (internal activity) generated the highest amplitude and pulse duration with the average values of 5.11 mV and 883 ms, respectively. On the other hand, the larvae crawling action on the apple surface produced significantly lower amplitude and pulse duration (mean values of 1.38 mV and 209 ms, respectively) than the two other categories. In addition, the peak frequency of larval crawling pulses had a mean value of 2.94±0.81 Hz, which was significantly higher than the other two signals. In fact, the peak frequency of chewing and boring into flesh moments was not significantly different and had a large overlap, meaning that the chewing frequency is the dominant frequency in the internal activities of CM larva. Figure 2 (Fig. 4 #): A comparison of signal amplitude, pulse duration, and frequency for different larval moments. These understandings of larvae activities acoustic pattern are critical in acoustic signal data processing, especially with regards to setting baseline acoustic features levels for developing appropriate machine learning classifiers for delineating between apples with larvae activities and those who are larvae free. 2. Improve the test performance of noninvasive detection of CM-infested apples using combined HSI and AE sensor data applied to machine learning methods, in comparison to our preliminary result. In accomplishing objective two, we applied two types of sensing methods, AE (low and high-frequency sensors) and HSI. The latter method HSI application is ongoing, while the last set of data are being collected on the AE. For data collection from the AE method, we applied both low and high-frequency sensors in passive (listening for natural sound emission) and impulse (measuring attenuated sound through apple samples) methods, we applied both machine and deep learning methods. The results from deep learning are not as impressive because of the number of data analyzed - the classification rate was between 57 - 93%. In the passive high frequency (60-150 kHz) sensing method, the classification results (<66% classification rate) were even worse than applying deep learning. However, the classification results obtained from the high-frequency impulse test set result gave classification in milliseconds that are generally greater than (90%). This is a major improvement compared to the preliminary result our group obtained before this project (Li et al., 2018). The preliminary results from the hyperspectral data in the near-infrared range are quite promising in terms of classification based on multi-pixel analytical approach that are well above 96% for the test set. The analysis of these result are not complete but from the result obtained so far, even before the sensor data fusion approach, that provides greater reliability than any of the result reported in our preliminary studies before the project (Ekramirad et al. 2017; Li et al, 2018 and Rady et al., 2017). References Ekramirad et al. (2017). https://doi.org/10.13031/trans.12184 #Ekramirad et al. (2021). Development of pattern recognition & classification models for the detection of vibro-acoustic emissions from codling moth-infested apples. Postharvest Biology and Technology. Submitted Li et al. (2018). https://doi.org/10.13031/trans.12548 Rady et al. (2017). https://doi.org/10.1016/j.postharvbio.2017.03.007
Publications
- Type:
Journal Articles
Status:
Published
Year Published:
2020
Citation:
Adedeji, A. A., Ekramirad, N., Rady, A., Hamidisepehr, A., Donohue, K., Villanueva, R., Parrish, C.A., and Li, M. (2020). Non-destructive technologies for detecting insect infestation in fruits and vegetables under postharvest conditions: A critical review. Foods 9(7), 927.
- Type:
Journal Articles
Status:
Under Review
Year Published:
2021
Citation:
Ekramirad, N., Al Khaled, Y.A., Donohue, K., Villanueva, R., Parrish, C.A., and *Adedeji, A. A. Development of pattern recognition and classification 1 models for the detection of vibro-acoustic emissions from codling moth infested apples. Postharvest Biology and Technology. Submitted February 10, 2021.
- Type:
Book Chapters
Status:
Other
Year Published:
2021
Citation:
Adedeji, A. A., Ekramirad, N., Al Khaled, Y.A., and Parrish, C.A. Vibro-acoustic emission and near-infra red imaging for nondestructive apple quality detection and classification. In Nondestructive Quality Assessment Techniques for Fresh Fruits and vegetables (Eds&..). Springer. Due June 30, 2021.
- Type:
Theses/Dissertations
Status:
Accepted
Year Published:
2021
Citation:
Chadwick Parrish. Vibro-acoustic Codling Moth Larvae Detection in Apples - Deep Learning Application
- Type:
Conference Papers and Presentations
Status:
Accepted
Year Published:
2021
Citation:
Ekramirad, N., Al Khaled, Y.A., Donohue, K., Villanueva, R., Parrish, C.A., Doyle, Lauren, and *Adedeji, A. A. Vis/NIR hyperspectral imaging with machine learning for detection of codling moth infestation in apples. A paper in the proceeding and presented (oral) during On-Demand Q&A oral session at 2021 Annual International Virtual Meeting of American Society of Agricultural and Biological (ASABE) held online from July 12 16, 2021. Paper #: 2100066.
- Type:
Conference Papers and Presentations
Status:
Accepted
Year Published:
2021
Citation:
Al Khaled, Y.A., Ekramirad, N., Donohue, K., Villanueva, R., Parrish, C.A., Doyle, Lauren, and *Adedeji, A. A. Detection of codling moth infestation in apple using acoustic impulse response signals. A paper in the proceeding and presented (oral) during On-Demand Q&A oral session at 2021 Annual International Virtual Meeting of American Society of Agricultural and Biological (ASABE) held online from July 12 16, 2021. Paper #: 2100070.
|