Source: MISSISSIPPI STATE UNIV submitted to NRP
IMAGE-BASED ASSESSMENT OF WOODCHIP QUALITIES: TRANSFER LEARNING MODEL DEVELOPMENT TO FIELD VALIDATION
Sponsoring Institution
National Institute of Food and Agriculture
Project Status
ACTIVE
Funding Source
AFRI COMPETITIVE GRANT
Reporting Frequency
Annual
Accession No.
1028790
Grant No.
2022-67022-37861
Cumulative Award Amt.
$590,005.00
Proposal No.
2021-11106
Multistate No.
(N/A)
Project Start Date
Aug 1, 2022
Project End Date
Jul 31, 2025
Grant Year
2022
Program Code
[A1521]- Agricultural Engineering
Recipient Organization
MISSISSIPPI STATE UNIV
(N/A)
MISSISSIPPI STATE,MS 39762
Performing Department
Industrial and Systems Enginee
Non Technical Summary
This study aims to develop a computationally sound, industry-scale image-based transfer learning (TL) model for quickly assessing the varying woodchip qualities (e.g., moisture/ash contents, size distribution). Woodchips are extensively utilized as raw materials for many industries, including pelleting mills, biorefineries, pulp and paper industries, and biomass-based power plants, and their quality variability significantly impacts the overall processing/logistics/maintenance costs, throughput, and the final product quality. Due to the recent advent of computational resources and advanced technologies, the pre-evaluation of woodchip qualities via optical analysis (image capturing and analysis) becomes a reality. Such technology will enable automized production, fast and high-quality property predictions without any human intervention or interaction. Unfortunately, industry-scale deployable image-based woodchip quality assessment is extremely difficult due to either image-related (e.g., with/without camera flash, quality of the daylight, sample variability, wood species, and shape irregularity) or computational-related challenges (e.g., time-consuming real-time image analysis, high model training time). To alleviate such challenges, our multi-disciplinary project team will collect field samples and the associated images under varying conditions, train an offline deep learning (DL) algorithm, and test them (via developing a TL model) in real-time under laboratory and industry settings. The TL model will leverage the knowledge (e.g., features, weights) acquired from a highly trained offline DL algorithm, thereby alleviate the time and effort required to quickly evaluate a new image. The portability of the proposed development will not only allow them to be integrated with a single machine but could be taken to the fields for onsite woodchip quality evaluations.
Animal Health Component
60%
Research Effort Categories
Basic
25%
Applied
60%
Developmental
15%
Classification

Knowledge Area (KA)Subject of Investigation (SOI)Field of Science (FOS)Percent
12306502020100%
Knowledge Area
123 - Management and Sustainability of Forest Resources;

Subject Of Investigation
0650 - Wood and wood products;

Field Of Science
2020 - Engineering;
Goals / Objectives
This study aims to develop a computationally sound, industry-scale image-based transfer learning (TL) model for quickly assessing the varying woodchip qualities (e.g., moisture/ash contents, size distribution). Woodchips are extensively utilized as raw materials for many industries, including pelleting mills, biorefineries, pulp and paper industries, and biomass-based power plants, and their quality variability significantly impacts the overall processing/logistics/maintenance costs, throughput, and the final product quality. Due to the recent advent of computational resources and advanced technologies, the pre-evaluation of woodchip qualities via optical analysis (image capturing and analysis) has become a reality. Such technology will enable automized production, fast and high-quality property predictions without any human intervention or interaction. Unfortunately, industry-scale deployable image-based woodchip quality assessment is extremely difficult due to either image-related (e.g., with/without camera flash, quality of the daylight, sample variability, wood species, and shape irregularity) or computational-related challenges (e.g., time-consuming real-time image analysis, high model training time). To alleviate such challenges, our multi-disciplinary project team set three specific objectives: Specific Aim 1# Image-based transfer learning algorithm development for woodchip quality assessment: This specific objective will develop a computationally sound, industry-scale image-based transfer learning algorithm to assess the woodchip qualities (e.g., moisture/ash contents, size distributions) under varying conditions (e.g., with/without camera flash, quality of the daylight, sample variability, wood species, and shape irregularity). This is one of the first efforts to assess woodchip qualities entirely based on image analysis (contact-free analysis). Specific Aim 2# Portable and stationary digital woodchip quality assessment tool development and testing and validating the tool in a laboratory setting: This specific objective will first develop a machine-integrated (stationary) and then a portable counterpart tool to reliably and cost-effectively assess woodchip qualities. The utility of the developed tools will be tested and validated in the laboratory facilities available at Mississippi State University (MSU) under different experimental conditions. Specific Aim 3# Testing and validating the developed tools in industrial settings and technology transfer plans: This specific objective will assess the utility of the developed tools (from Aims#1 and 2) in different facilities of our industrial collaborator. Additionally, we will disseminate an open-source technology transfer plan to ensure that the interested stakeholders can readily utilize the tools upon development.
Project Methods
Our project team will first collect raw woodchip samples from three different pellet mills of our industrial partner Drax Biomass Inc.: (i) Morehouse Bioenergy (Bastrop, LA), (ii) LaSalle BioEnergy (Urania, LA), and (iii) Amite Bioenergy (Gloster, MS). To understand seasonal fluctuations, we will collect woodchip samples in 4 different time periods of the year: (i) peak summer (mid-June to mid-July), (ii) late summer/early winter (mid-October to mid-November), (iii) peak winter (January to February), and (iv) late winter/early summer (mid-March to mid-April). From each time period, a bulk bag containing approximately 225 kg of woodchips will be collected from each site. A randomly selected sample of 100 grams of woodchips will be collected using ASTM C702M-18 method. After the photographs are taken, the properties (e.g., moisture/ash contents, particle size) of the samples will be experimentally measured (moisture content via ASTM E871, ash content via ASTM D1102, and particle size distribution via ISO 17827-1 methods) at the Pace Seed Laboratory at MSU, and their statistical properties will be reported (e.g., average, standard deviation, maximum/minimum values). Given the prediction accuracy of the woodchip qualities will entirely be made based on image analysis, the quality of the images being captured and further processed would be a critical step. Three cameras with different resolutions will be utilized to take the images. The images will be taken automatically by means of a script written in the python programming language. We foresee that a minimum of 432,000 images will be taken and trained during the entire project period. We then propose to develop a deep transfer learning (TL) algorithm, which leverages knowledge from a pre-trained deep learning model to classify woodchip qualities for the new images. In general, the algorithm will be developed in two major steps: Step 1: Deep learning for source model training and validating in the laboratory settings, and Step 2: Transfer knowledge from the deep learning model developed in Step 1 to develop a targeted deep TL model for the field/industrial applications.

Progress 08/01/23 to 07/31/24

Outputs
Target Audience:Target audiences: Local forest product industries (for data collection purposes) andacademic researchers Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided?The project involves 3 PhD and 1 undergraduate students. The students are currently working on their dissertations based on different tasks of thisproject. The students have already submitted several journal articles to prestigious avenues, such asRenewable and Sustainable Energy Reviews,Expert Systems with Applications,IEEE Transactions on Pattern Analysis and Machine Intelligence, and Energy & Fuels, and presented their research outcomes at several conferences, such asASABE Annual International Meeting andIISE Annual Conference.Abdur Rahman, one of the graduate students working on this project, received severalawards inpresentations for his research at several conferences and symposiums, such as Finalist in the IISE DAIS Best Student Paper Competition,IISE Annual Conference and Expo, Montreal, Canada, May 18-22, 2024. Awarded 1st Position in Graduate Poster Winner,2024 IISE Southeastern Regional Conference, March 21-23, 2024, hosted by Mississippi State University. Awarded 3rd position in Engineering PhD category in theGraduate Research Symposium, Fall 2023 at Mississippi State University. Awarded 3rd position in Engineering PhD category in theGraduate Research Symposium, Spring 2023 at Mississippi State University. How have the results been disseminated to communities of interest?Yes, the results from this project have been disseminated to local pellet industries and scientific communities via onsite visits and academic journals and conferences. During this reporting period, one journal article has been accepted inRenewable and Sustainable Energy Reviews, and three journal articles are currently under review inExpert Systems with Applications,IEEE Transactions on Pattern Analysis and Machine Intelligence, and Energy & Fuels. Besides, 8 conference presentations are made, primarily atthe ASABE Annual International Meeting andIISE Annual Conference. The project data was also collected from Drax Biomass and Enviva facilities in Mississippi. The initial tool has been disseminated to the stakeholders. What do you plan to do during the next reporting period to accomplish the goals?1. Applying domain adaptation model for wood chip moisture content assessment - Collected images from 2 sources of woodchips, 3 batches in one source - Verifying the robustness of the proposed domain adaptation model 2. Developing transformer-based high-performing model for moisture content assessment - Developed a Bayesian Optimization driven transformer model - Evaluating the proposed transformer model for wood chip data 3. Publishing the developed image data - Completed cleaning and preprocessing the wood chip image data of 4700 images - Will publish on https://zenodo.org/ 4. Aim #3 Building Portable Quality Assessment Tool - Collected all required equipment and devices - Conducted preliminary tests for deploying trained model on NVIDIA Jetson Nano developer Kit with Industrial grade camera. - Will deploy the trained MoistNet and trained domain adaptation model on the portable tool.

Impacts
What was accomplished under these goals? Key outcomes accomplished under Aim#1: - Developed a woodchip imaging platform - Studied different imaging environment conditions - Scanned data from different sources - Collected 4,700 moisture-level labeled images - Studied the moisture level binning strategies Key outcomes accomplished under Aim#2: - Developed a neural architecture search model -Optimized a highly accurate model: MoistNetMax -Optimized a cost-efficient model: MoistNetLite - Tested on 14 baseline models

Publications

  • Type: Journal Articles Status: Under Review Year Published: 2024 Citation: Rahman, A., Street, J. T., Wooten, J., Marufuzzaman, M., Gude, V. G., & Wang, H., (2024). Boosting Discriminability of Transferable Features in Unsupervised Domain Adaptation. (under review)
  • Type: Journal Articles Status: Under Review Year Published: 2024 Citation: Rahman, A., Street, J. T., Wooten, J., Marufuzzaman, M., Gude, V. G., Buchanan, R., & Wang, H., (2024). Moisture Content Prediction of Wood Chip using Texture Features. (under review)
  • Type: Conference Papers and Presentations Status: Accepted Year Published: 2024 Citation: Rahman, A., Eskorouchi, A., Street, J. T., Wooten, J., Gude, Marufuzzaman, M., & Wang, H. Wood Chip Moisture Content Assessment Using Infrared Image-Based Machine Learning. 2024 ASABE Annual International Meeting, Anaheim, CA, USA, July 28-31, 2024
  • Type: Conference Papers and Presentations Status: Accepted Year Published: 2024 Citation: Rahman, A., Marufuzzaman, M., Street, J. T., Wooten, J., Gude, V. G., & Wang, H. Boosting Discriminability of Transferable Features in Unsupervised Domain Adaptation. IISE Annual Conference and Expo, Montreal, Canada, May 18-22, 2024.
  • Type: Conference Papers and Presentations Status: Accepted Year Published: 2023 Citation: Rahman, A., Marufuzzaman, M., Street, J. T., Wooten, J., Gude, V. G., & Wang, H. MoistNet: Neural Architecture Search and Bayesian Optimization-driven Model for Moisture Content Prediction in Wood Chips. Graduate Research Symposium, Mississippi State, USA. October 21, 2023.


Progress 08/01/22 to 07/31/23

Outputs
Target Audience: Nothing Reported Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided?The project support allowed multiple graduate students and undergraduate students to experiment and test deep learning models with realistic data. Overall, two graduate students from Industrial and Systems Engineering and one graduate and one undergraduate student from the Sustainable Bioproduct Department were trained under this project during this reporting period. How have the results been disseminated to communities of interest?Our first-year project effort was disseminated to several avenues in the form of journal article submission and conference/symposium presentations, Details are listed below. Journal Papers: Rahman, A., Marufuzzaman, M., Street, J. T., Wooten, J., Gude, V. G., Buchanan, R., & Wang, H., (2023). A Comprehensive Review on Wood Chip Moisture Content Assessment and Prediction.Renewable and Sustainable Energy Reviews (Submitted) Rahman, A., Street, J. T., Wooten, J., Marufuzzaman, M., Gude, V. G., Buchanan, R., & Wang, H., (2023). MoistNet: Machine Vision-based Deep Learning Models for Wood Chip Moisture Content Measurement. Computers and Electronics in Agriculture (Submitted) Conference proceedings: Rahman, A., Marufuzzaman, M., Street, J. T., Wooten, J., Gude, V. G., & Wang, H. An Interpretable Deep Learning Model for Wood Chip Moisture Content Prediction. IISE Annual Conference and Expo, New Orleans, Louisiana, USA, May 21-24, 2023 Symposium Presentations: Rahman, A., Marufuzzaman, M., Street, J. T., Wooten, J., Gude, V. G., & Wang, H. An Interpretable Deep Learning Model for Wood Chip Moisture Content Prediction. Research that Matters in ISE, Mississippi State, USA. April 12, 2023. Rahman, A., Marufuzzaman, M., Street, J. T., Wooten, J., Gude, V. G., & Wang, H. An Interpretable Deep Learning Model for Wood Chip Moisture Content Prediction. Graduate Research Symposium, Mississippi State, USA. February 25, 2023. Awards: Awarded 3rd position in Engineering PhD category in the Graduate Research Symposium, Spring 2023 at Mississippi State University. What do you plan to do during the next reporting period to accomplish the goals? Even though our initial deep learning models achieved nearly 95% accuracy in predicting woodchip moisture content, we are developing domain adaptation techniques (transfer learning techniques) to further enhance the algorithm results Creating a data inventory of our collected and labeled woodchip images with varying moisture contents to share with the professional community Collecting images to test and predict ash contents in moisture contents Create a GitHub account and share the data and codes with the research community Test the initial results with our collaborating industrial partners Prepare our first extension report

Impacts
What was accomplished under these goals? - We performed a comprehensive litarature review on the existing woodchip quality assessment techniques - We collected ~1600 RGB images with varying moisture content levels. The images were collected from two different industries in Mississippi; We further collected ~1600 IR images (without heating) and ~560IR images (with heating) with varying moisture content levels. - Leveraging Neural Architecture Search through AutoML and Bayesian Optimization, we develop two deep learning models, MoistNetLite and MoistNetMax, showcasing exceptional performance in predicting woodchip moisture content. MoistNetLite achieves remarkable accuracy with minimal computational overhead (500 times lighter than MobileNet), while MoistNetMax exhibits exceptional precision (4.76% higher accuracy than the best baseline model ResNet152V2), despite increased computational complexity. With improved accuracy and faster prediction speed, MoistNet holds great promise for biofuel industries to use with any portable device or even smartphones.

Publications

  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Rahman, A., Marufuzzaman, M., Street, J. T., Wooten, J., Gude, V. G., & Wang, H. An Interpretable Deep Learning Model for Wood Chip Moisture Content Prediction. IISE Annual Conference and Expo, New Orleans, Louisiana, USA, May 21-24, 2023
  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Rahman, A., Marufuzzaman, M., Street, J. T., Wooten, J., Gude, V. G., & Wang, H. An Interpretable Deep Learning Model for Wood Chip Moisture Content Prediction. Research that Matters in ISE, Mississippi State, USA. April 12, 2023.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Rahman, A., Marufuzzaman, M., Street, J. T., Wooten, J., Gude, V. G., & Wang, H. An Interpretable Deep Learning Model for Wood Chip Moisture Content Prediction. Graduate Research Symposium, Mississippi State, USA. February 25, 2023.
  • Type: Journal Articles Status: Under Review Year Published: 2023 Citation: Rahman, A., Marufuzzaman, M., Street, J. T., Wooten, J., Gude, V. G., Buchanan, R., & Wang, H., (2023). A Comprehensive Review on Wood Chip Moisture Content Assessment and Prediction. Renewable and Sustainable Energy Reviews (Submitted)
  • Type: Journal Articles Status: Under Review Year Published: 2023 Citation: Rahman, A., Street, J. T., Wooten, J., Marufuzzaman, M., Gude, V. G., Buchanan, R., & Wang, H., (2023). MoistNet: Machine Vision-based Deep Learning Models for Wood Chip Moisture Content Measurement. Computers and Electronics in Agriculture (Submitted)