Source: STEVENS INSTITUTE OF TECHNOLOGY (INC) submitted to NRP
DSFAS-AI: FOOD QUALITY EVALUATION LEVERAGING ROBUST, DOMAIN ADAPTIVE DEEP LEARNING ON MILLIMETER WAVE (MMWAVE) IMAGES
Sponsoring Institution
National Institute of Food and Agriculture
Project Status
COMPLETE
Funding Source
Reporting Frequency
Annual
Accession No.
1028347
Grant No.
2022-67021-36866
Cumulative Award Amt.
$300,000.00
Proposal No.
2021-11540
Multistate No.
(N/A)
Project Start Date
Feb 15, 2022
Project End Date
Feb 14, 2025
Grant Year
2022
Program Code
[A1541]- Food and Agriculture Cyberinformatics and Tools
Recipient Organization
STEVENS INSTITUTE OF TECHNOLOGY (INC)
1 CASTLE POINT ON HUDSON
HOBOKEN,NJ 070305906
Performing Department
Department of Biomedical Engineering
Non Technical Summary
The modern food industry has been rapidly developing with a goal to strive supply markets and consumers with safe, nutritious, and high-quality products. However, quality assurance is challenging with raising issue of learning bias in processing large amount of data and generalizing to the evaluation of new food type. Traditional quality assurance methods based on chemical experiments and subjective evaluations also fail to meet the requirement of high volume food production and high food quality criteria. It is critical to search for more advanced, non-invasive, fast, reliable, and affordable sensing techniques and data science tools that will enable automated and real-time decision making to ensure food product safety.In this project, we propose to combine artificial intelligence (AI) and millimeter wave imaging to ensure the analysis of massive food data efficient and effective. We will use robust learning technology to reduce bias in machine learning and use few-shot learning to develop a generalizable platform for images from new food types. The system will be implemented by inexpensive imaging system using millimeter wave chipset. Ultimately, we expect to develop this innovative nanotechnology that will make a long-range contribution to data science and U.S. Agriculture and Food Systems.
Animal Health Component
40%
Research Effort Categories
Basic
30%
Applied
40%
Developmental
30%
Classification

Knowledge Area (KA)Subject of Investigation (SOI)Field of Science (FOS)Percent
5115310202050%
4027299303050%
Goals / Objectives
We target on addressing the bias issue and domain adaption issue in data science to facilitate the application of millimeter imaging on food quality evaluation. The objective of this research are (1) to build a mmWave platform for food quality evaluation; (2) to develop deep learning algorithm that is robust to bias and deep learning algorithms to make existing food quality evaluation easily adaptive to new application domain.
Project Methods
The used methods in this project are listed below:In Objective 1, we will develop an mmWave imaging system for food quality evaluation. Experiments on real egg samples will be conducted and we will generate images from contrast of permittivity of mmWave signals. In Objective 2, we will develop bias-robust learning and few-shot learning algorithms. We assess the performance of new AI methodology by measuring and comparing segmentation performance of image processing and statistical analysis.

Progress 02/15/24 to 02/14/25

Outputs
Target Audience:The target audiences are: academic researchers, industrial users, and students in class. We are disseminating our study in food science community, with one survey paper in Critical Reviews in Food Science and Nutrition, one open -access publication in the journal of Current Research in Food Science, and one paper in IEEE Access. We also have our first machine learning paper published in special issue of AI technology in food and agriculture in the Journal of Food and Agriculture research. We expect our research will broadly impact all the target audience in the world. In addition, the PI, Dr. Yu Gan, has continued teaching a machine learning, BME 571-Machine Learning in Biomedical Application in 2024-2025, to enhance student's understanding on machine learning, biological engineering, and food science. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided?Four PhD students (two from Stevens, two from UA) are trained in this project. As part of the training, students learn how to collaborate with researchers from different disciplines and institutions, and how to communicate their findings effectively to various audiences. All students participate in regular discussions with the project partners to receive feedback on their work bi-weekly. How have the results been disseminated to communities of interest?We have disseminated our research results through seminar talks at University Research Symposium, IEEE international conferences, and journal publications such as Journal of Food and Agriculture research, Critical Reviews in Food Science and Nutrition, Current Research in Food Science, and IEEE Access What do you plan to do during the next reporting period to accomplish the goals?This is the final report, as the project has ended on February 2025.

Impacts
What was accomplished under these goals? We have made significant efforts in both goals. We have made significant efforts in both goals. (1) We completed hardware of a millimeter-wave imaging system and currently working on software to automatically collect the food image data and reconstruct the images. (2) We continued optimizing our microwave platform for food quality evaluation by imaging avocado and eggs in this research period. (3) We have developed a few-shot learning machine learning algorithm to identify avocado ripeness. (4) We have developed a robust learning framework to classify the ripeness of avocado from both microwave images and natural images.

Publications

  • Type: Peer Reviewed Journal Articles Status: Published Year Published: 2024 Citation: Jeong N, Gan Y, Kong L. Emerging non-invasive microwave and millimeter-wave imaging technologies for food inspection. Critical Reviews in Food Science and Nutrition. 2024 Jun 6:1-2.
  • Type: Peer Reviewed Journal Articles Status: Published Year Published: 2025 Citation: Z. M. Choffin, L. Kong, Y. Gan and N. Jeong, "A CNN-Based Microwave Imaging System for Detecting Watermelon Ripeness," in IEEE Access, vol. 13, pp. 21413-21421, 2025


Progress 02/15/22 to 02/14/25

Outputs
Target Audience:The target audiences are academic researchers, industrial users, and students in class. We are disseminating our study in food science and electrical engineering community, in high-impact journals such as Critical Reviews in Food Science and Nutrition, Current Research in Food Science, Journal of Food and Agriculture research, and IEEE Access. In addition, all three PDs jointly held a in special issue of AI technology in food and agriculture in the Journal of Food and Agriculture research. We expect our research will broadly impact all the target audience in the world. In addition, the PI, Dr. Yu Gan, has started a new machine learning course at Stevens in 2022, BME 571-Machine Learning in Biomedical Application, to enhance student's understanding on machine learning, biological engineering, and food science. This course is taught accumulatively three times from 2022 to 2025. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided?Four PhD students (two from Stevens, two from UA) are trained in this project. As part of the training, students learn how to collaborate with researchers from different disciplines and institutions, and how to communicate their findings effectively to various audiences. All students participate in regular discussions with the project partners to receive feedback on their work bi-weekly. How have the results been disseminated to communities of interest?We have disseminated our research results through seminar talks at University Research Symposium, and four journal publications including Critical Reviews in Food Science and Nutrition, Current Research in Food Science, Journal of Food and Agriculture research, and IEEE Access. What do you plan to do during the next reporting period to accomplish the goals? Nothing Reported

Impacts
What was accomplished under these goals? We have made significant efforts in both goals. (1) We have built two imaging systems in this project. One is a microwave imaging system. The platform employs a circular array with 10 Coplanar Vivaldi antennas offering wide bandwidth, high gain, and high efficiency. The second one is a milimeter wave imaging system with high imaging resolution. We tested the platform with three types of food samples: avocado, egg, and watermelon. (2) We have been developing machine learning algorithm to identify how long the avocado decays. To address the issue of small data size, we devised a few-shot learning algorithm using prototypical network. In addition, we also developed machine learning algorithm that is based on support vector machine to classify the ripeness of watermelon (3) We have developed a robust learning algorithm to deal with the supervised learning framework with corrupted labels. We have testified the performance on both public dataset and our local dataset

Publications

  • Type: Peer Reviewed Journal Articles Status: Published Year Published: 2023 Citation: Garvin J, Abushakra F, Choffin Z, Shiver B, Gan Y, Kong L, Jeong N. Microwave imaging for watermelon maturity determination. Current Research in Food Science. 2023 Jan 1;6:100412.
  • Type: Peer Reviewed Journal Articles Status: Published Year Published: 2024 Citation: Ahmed, M., Mustafa, H., Wu, M., Babaei, M., Kong, L., Jeong, N. and Gan, Y., 2024. Few shot learning for avocado maturity determination from microwave images. Journal of Agriculture and Food Research, 15, p.100977.
  • Type: Peer Reviewed Journal Articles Status: Published Year Published: 2024 Citation: Jeong N, Gan Y, Kong L. Emerging non-invasive microwave and millimeter-wave imaging technologies for food inspection. Critical Reviews in Food Science and Nutrition. 2024 Jun 6:1-2.
  • Type: Peer Reviewed Journal Articles Status: Published Year Published: 2025 Citation: Z. M. Choffin, L. Kong, Y. Gan and N. Jeong, "A CNN-Based Microwave Imaging System for Detecting Watermelon Ripeness," in IEEE Access, vol. 13, pp. 21413-21421, 2025


Progress 02/15/23 to 02/14/24

Outputs
Target Audience:The target audiences are: academic researchers, industrial users, and students in class. We are disseminating our study in food science community, with an open -access publication in the journal of Current Research in Food Science. We also have our first machine learning paper published in special issue of AI technology in food and agriculture in the Journal of Food and Agriculture research. We expect our research will broadly impact all the target audience in the world. In addition, the PI, Dr. Yu Gan, has designed course-project in his course, BME 571-Machine Learning in Biomedical Application in both academic year 2022-2023 and 2023-2024, to enhance student's understanding on machine learning and food science. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided?Three PhD students (two from Stevens, one from UA) are trained in this project. In addition, one undergraduate students (one from Stevens) are involved in this project. One senior design team are engaged in this project as well. As part of the training, students learn how to collaborate with researchers from different disciplines and institutions, and how to communicate their findings effectively to various audiences. All students participate in regular discussions with the project partners to receive feedback on their work bi-weekly. How have the results been disseminated to communities of interest?We have disseminated our research results through seminar talk and journal publications in peer review journal, such as Journal of Agriculture and Food Research and Current Research in Food Science. The results are also presented in related courses such as BME 571 Machine Learning in Biomedical Engineering, ECE 493/593 Engineering Data Analytics, and senior design presentation. What do you plan to do during the next reporting period to accomplish the goals?There three directions we plan to accomplish: 1) Imaging samples from different species. We are planning to acquire more food types other than avocado and eggs and building a dataset of microwave images for a variety of fruits. 2) Hardware implementation. We plan to improve our antenna design and build the millimeter-wave imaging system. Theoretically, millimeter-wave imaging will lead to higher resolution to identify food quality. 3) Software implementation. We plan to implement a robust learning framework by correcting samples with imperfect labels. This algorithm development will be investigated with both simulated data and truly corrupted data.

Impacts
What was accomplished under these goals? We have made significant efforts in both goals. (1) We continued optimizing our microwave platform for food quality evaluation by imaging avocado and eggs in this research period. Eleven avocado samples were imaged and processed to determine whether it was ripe or not for a few-shot learning setup. In addition, we acquired images from 48 eggs. (2) We have been developing machine learning algorithm to identify how long the avocado decays. To address the issue of small data size, we devised a few-shot learning algorithm using prototypical network. In addition, we also developed machine learning algorithm that is based on support vector machine to classify the ripeness of watermelon. Label correction network is in development to address imperfect labeling issue in avocado and egg images.

Publications

  • Type: Journal Articles Status: Published Year Published: 2024 Citation: Ahmed, M., Mustafa, H., Wu, M., Babaei, M., Kong, L., Jeong, N. and Gan, Y., 2024. Few shot learning for avocado maturity determination from microwave images. Journal of Agriculture and Food Research, 15, p.100977.
  • Type: Journal Articles Status: Under Review Year Published: 2024 Citation: Jeong, N., Kong, L., and Gan, Y. Emerging Non-invasive Microwave and Millimeter-wave Imaging Technologies for Food Inspection. Critical Reviews in Food Science and Nutrition.


Progress 02/15/22 to 02/14/23

Outputs
Target Audience:There are multiple groups of audience in this research period: academic researchers, industrial users, and students in class. We are disseminating our study in food science community, with a open -access publication in the journal of Current Research in Food Science. We expect our research will broadly impact all the target audience in the world. In addition, we are in close collaboration with a food company, Peco Food, Inc, to explore the possibility of using our software and hardware in their manufacturing facility. Through in person discussion and demonstration of our research outcome, we expect this effort will initiate and enhance collaborations several food companies. In addition, the PI, Dr. Yu Gan, has designed side-project in his course, BME 571-Machine Learning in Biomedical Application, to enhance student's understanding on machine learning and food science. Two senior teams at UA are engaged in this project as well. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided?One PhD studentis trained in this project. In addition, three undergraduate students (one from Stevens, two from UA) are involved in this project. Two senior design teams are engaged in this project as well.Students are trained in system development, data acquisition, data analysis, and academic writing. In particular, they are involved in instrumentation of imaging system, algorithm development of machine learning. Graduate students are taking bi-weekly meeting to report research progress and collect feedback from the whole team. How have the results been disseminated to communities of interest?The results have been disseminated largely through seminar talk and journal publications in peer review journal. The results are also presented in related courses and senior design presentation. What do you plan to do during the next reporting period to accomplish the goals?There three directions we plan to accomplish: 1) Increase data size. We are planning to acquire more avocado images and egg images for both model training and validation purpose. 2) Hardware implementation. We plan to expand the hardware implementation from microwave antenna to millimeterwave antenna, the later of which will lead to higher resolution to identify food quality. 3) Software implementation. We plan to improve the few shot machine learning algorithm by exploring the similarity between egg image and avocado images. This algorithm development will be investigated when more egg and avocado images are acquired.

Impacts
What was accomplished under these goals? We have made significant efforts in both goals. (1) We have built a microwave platform for food quality evaluation. The platform employs a circular array with 10 Coplanar Vivaldi antennas offering wide bandwidth, high gain, and high efficiency. We tested the platform with three types of food samples: avocado, egg, and watermelon. As a prototype, we chose to start with watermelon, which has larger size than egg or avocado. Eight samples were imaged and processed to determine whether it was ripe or not. In addition, we acquired images from five avocados and five eggs. (2) We have been developing machine learning algorithm to identify how long the avocado decays. To address the issue of small data size, we devised a few shot learning algorithm that utilize a small number of samples of training while maintaining high accuracy. It was a machine learning model based on convolution neural network. In addition, we also developed machine learning algorithm that is based on support vector machine to classify the ripeness of watermelon.

Publications

  • Type: Journal Articles Status: Published Year Published: 2023 Citation: Garvin J, Abushakra F, Choffin Z, Shiver B, Gan Y, Kong L, Jeong N. Microwave imaging for watermelon maturity determination. Current Research in Food Science. 2023 Jan 1;6:100412.