Progress 10/01/22 to 03/31/25
Outputs
Target Audience:The research conducted for this project contributes to the training of two Ph.D. students who have an opportunity to be involved in interdisciplinary research, including process control, environmental agricultural systems, and wastewater treatment, and plant growth. The target audience includes graduate students or post-doctors, undergraduate students, and the broader academic communities through conference presentations and published journals. Changes/Problems:
Nothing Reported
What opportunities for training and professional development has the project provided?This project has provided the opportunity to train two graduate students and several undergraduate students in the interdisciplinary areas including environmental control system, process control, wastewater treatment, etc. How have the results been disseminated to communities of interest?We use our UGA collaborator's environmental agricultural system and co-PI's extension service to disseminate to communities of interest. GT pilot plant is also serviced as the resource for dissemination. At the University of Florida, the grant partly supported the development of the prototype IoT-based research lab - the Scanning Plant IoT facility (SPOT) facility. SPOT has been central to generating collaborative research with agencies such as the Florida Department of Agriculture and Consumer Services (FDACS) Division of Plant Industry and the USDA Invasive species laboratory Ft. Lauderdale on utilizing the latest IoT and AI technologies in precision agriculture and invasive species biocontrol. The SPOT facility has also generated collaborations with industry with emerging vertical farming corporations such as Kalera. The lab has been showcased to visiting State Representatives, industry leaders, and the Florida Farm Bureau. What do you plan to do during the next reporting period to accomplish the goals?
Nothing Reported
Impacts
What was accomplished under these goals?
Objetive 2: The implementation of non-invasive spectroscopy and imaging for assessing plant morphology andnutrition. The grant provided partial support for students working on configuring and programming the Scanning Plant IoT (SPOT) facility. The SPOT facility consists of a 512 sq. ft. cubic enclosure, equipped with a scanning imaging spectrometer, thermal camera, and laser scanner. These sensors enable the SPOT facility to monitor plants' biochemical, thermal, and structural responses to external stimuli on a near-real-time basis. Fully operational in 2020, multiple faculties utilize this facility across the University of Florida Institute of Food and Agricultural Sciences (UF/IFAS) to test crop responses to biotic and abiotic stresses and assess biocontrol agents' impact on invasive species. SPOT has been described in the publication Lantin et. al (2023). Hardware for the SPOT facility was generated via an internal UF/IFAS grant. Objective 4: The resulting CPS is implemented in simulation software and experimentally validated to determine a feasible deployment scale. Our AD model requires inputs for protein, lipid, and carbohydrate content, but obtaining these properties directly from AD feedstock is often challenging. Typically, AD characterization involves measuring parameters such as total solids (TS), volatile solids (VS), chemical oxygen demand (COD), total Kjeldahl nitrogen (TKN), volatile fatty acids (VFA), and pH. We devised a method to estimate the protein, lipid, and carbohydrate (P, L, C) content from these common measurements. The baseline P, L, and C distribution can either be estimated using values from literature or by solving a mass balance equation through optimization, using data for carbon (C), hydrogen (H), oxygen (O), and nitrogen (N), as proposed in a previous study. Protein content can be inferred from the nitrogen-to-protein conversion factor, based on TKN data. Additionally, typical COD/VS ratios from literature (lipids~2.9, proteins~1.5, carbohydrates~1.1) allow us to derive estimates. This enables us to solve a system of three equations with three unknowns (P, L, C) to determine the feedstock composition from standard AD measurements. We calibrated the AD model using experimental data. We performed parameter calibration by simultaneously fitting biogas production, methane concentration, and pH data obtained from our collaborators' experimental results. The resulting model showed a strong alignment with the experimental trends, achieving an acceptable error range. This demonstrates the model's ability to capture key process dynamics accurately. Our approach provides a practical way to determine feedstock composition using commonly available AD measurements. This enables more precise simulation and optimization of anaerobic digestion processes without requiring extensive chemical analysis.
Publications
- Type:
Peer Reviewed Journal Articles
Status:
Accepted
Year Published:
2024
Citation:
Gao, Ji, Abigael Wahlen, Caleb Ju, Yongsheng Chen, Guanghui Lan, and Zhaohui Tong. "Reinforcement learning-basedcontrol for waste biorefining processes under uncertainty." Communications Engineering 3, no. 1 (2024): 38.
- Type:
Other Journal Articles
Status:
Accepted
Year Published:
2023
Citation:
Stephen Lantin, Kelly McCourt, Emerick Larkin, Nicholas Butcher, Varun Puri, Eric McLamore, Melanie Corrrell, Aaditya Singh (2023) Scanning Plant IoT (SPOT) Facility for High-Throughput Plant Phenotyping, HardwareX.
- Type:
Conference Papers and Presentations
Status:
Accepted
Year Published:
2022
Citation:
Gao, J., Wahlen, A., Tong, Z., Reinforcement Learning Based Control for Biorefining Processes Under Uncertainty, 2022 AIChE annual meeting, Phoenix, AZ
- Type:
Conference Papers and Presentations
Status:
Accepted
Year Published:
2023
Citation:
Gao, J., Wahlen, A., Tong, Z., Reinforcement Learning Based Control for Biorefining Processes Under Uncertainty, 2023 AIChE annual meeting, Orlando, FL
|
Progress 10/01/23 to 09/30/24
Outputs
Target Audience:The research conducted for this project contributes to the training of two Ph.D. students who have an opportunity to be involved in interdisciplinary research including process control, environmental agricultural systems, and wastewater treatment, and plant growth. The target audience we will target includes graduate students or post-doctors, undergraduate students, and the broader academic communities through conference presentations, and published journals. Changes/Problems:Due to the difficulty of getting the real pilot plant data, we used the literature data to build up our process control model. What opportunities for training and professional development has the project provided?This project has provided the opportunity to train two graduate students and several undergraduate students in the interdisciplinary areas including environmental control system, process control, wastewater treatment, etc. How have the results been disseminated to communities of interest?Weuse our UGA collaborator's environmental agricultural system and co-PI's extension service to disseminate to communities of interest. GT pilot plant isalso serviced as the resource for dissemination. What do you plan to do during the next reporting period to accomplish the goals? Application of PDA to AD control problem We will formulate the AD control problem for continuous state and action spaces. This formulation will be implemented within the Gymnasium framework, allowing us to easily apply, test, and compare various RL algorithms to the problem. Finally, we'll conduct tests to evaluate the effectiveness of PDA for AD control.
Impacts
What was accomplished under these goals?
Objetive 2: The implementation of non-invasive spectroscopy and imaging for assessing plant morphology and nutrition. 1. Development of a prototype phenotyping facility We have designed and implemented a prototype plant phenotyping facility: the Scanning Plant IoT (SPOT) and have equipped it with a hyperspectral imaging system and laser scanner. The SPOT Facility is a 10 ft. x 10 ft. x 10 ft., open-air, cubical structure made of extruded aluminum. It contains three main sensors: a Headwall Nano-Hyperspec® hyperspectral sensor (VNIR, 400-1000 nm, 270 spectral bands), a FLIR Vue Pro R thermal camera, and an Intel RealSense L515 LiDAR camera. All sensors are pointed nadir (downward) toward the scanning region and are mounted on a plate near the top of the structure; the X-Y position of the plate is controlled via computer numerical control (CNC) of stepper motors. As such, the field of view of the sensors can be adjusted to completely capture subjects in the scanning region. Compared to the staring arrays present in many consumer cameras and other phenotyping systems, SPOT's hyperspectral sensor (Headwall Nano-Hyperspec®) uses a pushbroom scanner to capture image data. This imaging technique allows the sensor to acquire data in hundreds (270) of spectral bands over the visible-near infrared (VNIR, 400-1000 nm) region. With more wavebands, SPOT broadens the analytical space to partial least square regression and more sophisticated machine learning techniques not available to system with fewer wavebands such as multispectral imaging. Additionally, the camera has a high spectral resolution (~2 nm), ideal for producing clean spectral reflectance data and differentiating between spectrally close phytochemical responses. A live waterfall feed of the hyperspectral data is available for viewing on SPOT's computer from Headwall's Hyperspec III® desktop application. The sensor is shipped from the manufacturer with a radiometric calibration file that can be applied in the SpectralView® application to correct for image aberrations. A Spectralon® panel is also present in SPOT as a white reference to ensure that pixel values are standardized between images. Thermal image data acquisition We evaluated the performance of PDA against established RL algorithms such as Advantage Actor-Critic (A2C), Proximal Policy Optimization (PPO), Trust Region Policy Optimization (TRPO), and Natural Policy Gradient (NPG). Benchmarking was first conducted using Mujoco, a widely used robotics and mechanics simulation library. Subsequently, we tested the algorithms in Or-Gym environments, which focus on classical operations research problems solvable through both RL and traditional operations research techniques. While PDA offers strong theoretical convergence guarantees, its practical implementation faces a bottleneck: solving optimization subproblems during policy execution. To address this, we applied three algorithms aimed at more efficient training and execution of the PDA algorithm. These include the auto-conditioned fast gradient method (AC-FGM), alternating minimization with action discretization, and a random gradient descent method. We then compared the performance of these approaches to assess their impact on accelerating PDA. Benchmarking For RL-based control in AD, a more natural approach involves using continuous state and action spaces, alongside function approximation methods, rather than relying on discrete tabular solutions. Recent research by Lan introduced policy dual averaging (PDA) algorithms for continuous state and action spaces with function approximation, demonstrating favorable convergence properties. Our initial step was to implement the PDA algorithm in a general state and action space, following the methodology outlined in Lan's work. To streamline this process, we utilized the Tianshou RL framework. Accelerating PDA SPOT's employs tight motion control to ensure that sensors are positioned properly above the plants. Because the hyperspectral scanner captures image data with a pushbroom scanning mechanism, to image the entire scanning region, the hyperspectral sensor, thermal imager, and LiDAR imaging system are all mounted on a block above the scanning region, pointed nadir toward plants in the facility. This mounting block is connected to two sets of perpendicular lead screws, which form guide rails for translational motion in two axes. A stepper motor utilizes messages from an automated Python script for computer numerical control (CNC) to move the sensor pod over the plant. Objective 3: control algorithms with the purpose of achieving desired closed-loop performance despite the presence of disturbances and/or uncertain dynamics. RL in continuous state and action space Algorithm implementation SPOT utilizes an RPLIDAR A2 linear scanning system to build a structural map of the scanning region using motion control detailed in the next subsection. Standard Python packages provided by RPLIDAR facilitate data acquisition. Motion control SPOT uses a FLIR Vue Pro R thermal camera to collect thermal imaging data from plants being grown in the system. Automated data collection is performed alongside hyperspectral imaging using via serial commands from SPOT's desktop computer. LiDAR structural data acquisition
Publications
- Type:
Journal Articles
Status:
Published
Year Published:
2024
Citation:
Gao, Ji, Abigael Wahlen, Caleb Ju, Yongsheng Chen, Guanghui Lan, and Zhaohui Tong. "Reinforcement learning-based control for waste biorefining processes under uncertainty." Communications Engineering 3, no. 1 (2024): 38.
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2023
Citation:
Gao, J., Wahlen, A., Tong, Z., Reinforcement Learning Based Control for Biorefining Processes Under Uncertainty, 2023 AIChE annual meeting, Orlando, FL
|
Progress 10/01/22 to 09/30/23
Outputs
Target Audience:The research conducted for this project contributes to the training of two graduatestudent or two undergraduate studentswho has an exciting opportunity to be involved in interdisciplinary research including controlled environment agricutlure, sustainable process control, and plant phenotyping facilities design. Changes/Problems:There are no major changes and problems. The slight change of process control algorithm by focusing on the AD process due to the unavailable data from the hydroponic pilot plant. What opportunities for training and professional development has the project provided?This project has provided the opportunity to train two graduate students and several undergraduate students in interdisciplinary areas including process control, machine learning algorithm design, spectroscope, and imaging instruments for invasive plant/nutrient tracing. How have the results been disseminated to communities of interest?The results have been disseminated to communities of interest through workshops, conference presentation, farmer field trips, and journal publications. What do you plan to do during the next reporting period to accomplish the goals?1. Data-Driven Modeling Currently, a rigorous mechanistic model ADM1 is used for simulation purposes. However, the model is large and still slow to run despite the implementation of some acceleration techniques. In addition, ADM1 requires specific input, i.e., proteins, lipids and carbohydrates to the ODE model for simulation. This creates some challenges since many real-world applications do not provide these measurements. Hence, it will be worth investigating a computationally cheaper and easy-to-use surrogate model. We will explore various data-driven modeling methods that combines the fidelity of the mechanistic model, and the speed of the ML model for more efficient simulations. 2. RL in continuous state and action space A limitation of the current MDP model formulation is that it involves discretizing the state and action space, while the actual problem is entirely continuous. This leads to several difficulties, such as the need to identify appropriate upper and lower bounds for discretization. A more natural approach involves using continuous state and action and employing function approximation methods to avoid the use of the tubular solution. We will extend the discrete space PMD to continuous space with function approximation methods. To implement this approach, we will manually design state descriptors or features and map them to the value function or state-value function, using techniques such as linear, non-linear, kernel, or neural network methods. 3. Development of prototype phenotyping facilities We plan to further improve these facilities by collecting more data in practic systems including building relationships betweenplant phenotyping, and nutrient time-series providing curves.
Impacts
What was accomplished under these goals?
Objective 2: The implementation of non-invasive spectroscopy and imaging to for assessing plant morphology and nutrition. Development of a prototype phenotyping facility We designed and built a prototype plant phenotyping facility: the Scanning Plant IoT (SPOT) and have equipped it with a hyperspectral imaging system and laser scanner. The SPOT Facility is a 10 ft. x 10 ft. x 10 ft., open-air, cubical structure made of extruded aluminum. It contains three main sensors: a Headwall Nano-Hyperspec® hyperspectral sensor (VNIR, 400-1000 nm, 270 spectral bands), a FLIR Vue Pro R thermal camera, and an Intel RealSense L515 LiDAR camera. All sensors are pointed nadir (downward) toward the scanning region and are mounted on a plate near the top of the structure; the X-Y position of the plate is controlled via computer numerical control (CNC) of stepper motors. As such, the field of view of the sensors can be adjusted to completely capture subjects in the scanning region. Compared to the staring arrays present in many consumer cameras and other phenotyping systems, SPOT's hyperspectral sensor (Headwall Nano-Hyperspec®) uses a pushbroom scanner to capture image data. This imaging technique allows the sensor to acquire data in hundreds (270) of spectral bands over the visible-near infrared (VNIR, 400-1000 nm) region. With more wavebands, SPOT broadens the analytical space to partial least square regression and more sophisticated machine learning techniques not available to system with fewer wavebands such as multispectral imaging. Additionally, the camera has a high spectral resolution (~2 nm), ideal for producing clean spectral reflectance data and differentiating between spectrally close phytochemical responses. A live waterfall feed of the hyperspectral data is available for viewing on SPOT's computer from Headwall's Hyperspec III® desktop application. The sensor is shipped from the manufacturer with a radiometric calibration file that can be applied in the SpectralView® application to correct for image aberrations. A Spectralon® panel is also present in SPOT as a white reference to ensure that pixel values are standardized between images. Specifications of SPOT: Primary hardware • Hyperspectral imaging: SPOT's hyperspectral sensor (Headwall Nano-Hyperspec®) uses a pushbroom scanning configuration to capture image data. o Visible-near infrared (VNIR, 400-1000 nm, 270 wavebands, ~2 nm resolution, ideal for producing high spectral resolution data enabling the differentiation between spectrally similar phytochemical responses. A live waterfall feed of the hyperspectral data available for viewing on SPOT's computer from Headwall's Hyperspec III® desktop application enables observation of the spectral data collection in real-time. o Radiometric calibration software that allows the conversion of raw image data into at-sensor radiance and reflectance. o A Spectralon® panel can be used as a spectrally 'flat' white reference to convert at-sensor radiance to apparent at-surface reflectance to ensure image spectra are standardized across images. • Thermal imaging: SPOT uses a FLIR Vue Pro R thermal camera to collect thermal imagery. Automated data collection can be performed alongside hyperspectral imaging using either the free FLIR UAS phone application, or via serial commands issued from a desktop computer. • LiDAR imaging: SPOT utilizes an Intel RealSense L515 LiDAR imaging system to collect high-resolution point clouds of scanned objects. Standard Python scripts available for free on Intel's website facilitate data acquisition. • Motion control: Imaging system motion control is guided by two perpendicular lead screws mounted above the scanning region. Each lead screw is controlled by separate motor drives AutomationDirect SureStep® Advanced Microstepping Drive STP-DRV-80100), which guide the actuation of two AutomationDirect SureStep® Stepper Motor STP-MTRH-34127. Research performed using SPOT: • Stephen Lantin, University of Florida - lettuce, carotenoids, anthocyanin characterization o NASA Grant #80NSSC21K1257 • Kelli McCourt, University of Florida (senior thesis) - lettuce, salt stress • Naqeeb Cahacci, University of Florida (master's research) - lettuce, exploratory analysis • Cosimo Bettiol, University of Pisa, Italy - tomato, salt stress • Ayush Sharma, University of Florida - potato, macronutrient analysis • Dr. Esteban Rios, University of Florida - alfalfa; water stress, exploratory analysis • Sedonia Steininger, FDACS - Brazilian peppertree, thrip biocontrol efficacy • Alwin Hopf, University of Florida - hemp, CBD content • Kendall Schlitt, University of Florida (senior thesis) - food waste, microplastics • Usman Mohammed, University of Florida - water hyacinth, exploratory analysis Objective 3: we devise control algorithms with the purpose of achieving desired closed-loop performance despite the presence of disturbances and/or uncertain dynamics. 1. Model development The full ADM1 model with temperature dependence was implemented. It has the capability for dynamic simulation of the anaerobic digestion process. The model is implemented in Python and accelerated by just-in-time (JIT) compilation techniques for fast simulation. This allows us to use the model for a broad spectrum of applications including machine learning and reinforcement learning. 2. Data analysis Using the collected literature data, we developed a machine learning (ML)-based model (kernel density estimation (KDE) with Gaussian type kernel) for composition analysis of the waste feedstocks. KDE allows the estimation of feed composition distribution in a more natural manner without the need for additional assumptions. We carried out sensitivity analysis for biogas production rate with respect to the uncertainties in feedstock for three major types of wastes: food waste from restaurants, agriculture waste from crops and animals, and the organic fraction of the municipal solid wastes. The sensitivity analysis allows us to understand how the composition in waste can affect biogas production. 3. Algorithm development and implementation We applied 2 reinforcement learning (RL) algorithms for the control of the AD process - dynamic programming (DP) and policy mirror descent (PMD). We also applied a standard proportional integration and derivative (PID) controller for comparison purposes. The DP and PID were applied for a finite horizon control problem with varying target. The PMD algorithm was used to control the AD process with combined feedstock inventory control for steady production over an infinite horizon. For Markov decision process (MDP) modeling of the process, we expanded the observation space to two consecutive observations of biogas production rate. This allows the RL algorithm to inherently obtain the gradient information of the biogas production process, even though the MDP is partially observable with around many intermediate reactants. 4. Results For the finite horizon control problem, the DP algorithm achieved better performance in accuracy, precision and lag compared with the PID algorithm, indicating a successful deployment of RL methods in AD process. For the infinite horizon robust control problem, the PMD successfully managed the feedstock inventory and biogas production under seasonal and compositional uncertainties in waste feedstocks to satisfy a downstream demand.
Publications
- Type:
Conference Papers and Presentations
Status:
Accepted
Year Published:
2022
Citation:
Gao, J., Wahlen, A., Tong, Z., Reinforcement Learning Based Control for Biorefining Processes Under Uncertainty, 2022 AIChE annual meeting, Phoenix, AZ
- Type:
Conference Papers and Presentations
Status:
Awaiting Publication
Year Published:
2023
Citation:
Gao, J.; Wahlen, A.; Ju, C.; Chen, Y.; Lan, G.; tong, Z. Reinforcement Learning-Based Control for Waste Biorefining Processes Under Uncertainty. Preprints, 2023. https://doi.org/10.21203/rs.3.rs-2860936/v1.
|
|