Source: PRAIRIE VIEW A&M UNIVERSITY submitted to NRP
MULTI-SCALE MULTI-RESOLUTION AGRICULTURE DATA ANALYTICS FOR CROP/VEGETATION HEALTH PREDICTION AND OPTIMAZATION
Sponsoring Institution
National Institute of Food and Agriculture
Project Status
ACTIVE
Funding Source
Reporting Frequency
Annual
Accession No.
1028528
Grant No.
2022-38821-37338
Cumulative Award Amt.
$450,000.00
Proposal No.
2021-12889
Multistate No.
(N/A)
Project Start Date
May 1, 2022
Project End Date
Apr 30, 2026
Grant Year
2022
Program Code
[EQ]- Research Project
Recipient Organization
PRAIRIE VIEW A&M UNIVERSITY
P.O. Box 519, MS 2001
PRAIRIE VIEW,TX 77446
Performing Department
Electrical Engineering
Non Technical Summary
Recent advances in artificial intelligence (AI) especially deep learning and big data analytics have created exciting new opportunities for applications in precision agriculture. In this project, a framework of agriculture data analytics for crop/vegetation health prediction and optimization is proposed to create novel paradigm-shifting approaches for quantitatively evaluating crop/vegetation health using multi-scale multi-resolution data. Three research and extension objectives are proposed to achieve this goal: 1) collect multi-scale multi-resolution data of crop/vegetation health at PVAMU research farm, by ground sensors and drones, to complement satellite data; 2) apply and develop cutting edge deep learning algorithms for crop/vegetation health assessment and prediction; 3) develop a visualization/WebApp tool for the proposed crop/vegetation health prediction and optimization on PVAMU research farm to enhance research and outreach activities, and educate and train limited resource farmers and other stakeholders on outcomes of the project. If successful, this project will benefit farmers and growers to apply the best agricultural management practice and improveyield. Furthermore, this project is multidisciplinary in nature and it will foster collaborations between experts in agriculture and AI researchers and data scientists. The proposed project will help different stakeholders such as farmers to collaborate with scientists and engineers, to apply AI especially deep learning and big data analytics to extract valuable knowledge from big agriculture datasets and transform the knowledge into actionable strategies. This will make America's farmers more competitive in the global agricultural market and contribute to a strong and sustainable economy.
Animal Health Component
50%
Research Effort Categories
Basic
40%
Applied
50%
Developmental
10%
Classification

Knowledge Area (KA)Subject of Investigation (SOI)Field of Science (FOS)Percent
2051520208050%
1020199303050%
Goals / Objectives
In this project, a framework of agriculture data analytics for crop/vegetation health prediction and optimization is proposed to create novel paradigm shifting approaches for quantitatively evaluating crop/vegetation health using multi-scale multi-resolution data. Three research and extension objectives are proposed to achieve this goal:1) collect multi-scale multi-resolution data of crop/vegetation health at PVAMU research farm, by ground sensors and drones, to complement satellite data;2) apply and develop cutting edge deep learning algorithms for crop/vegetation health assessment and prediction;3) develop a visualization/WebApp tool for the proposed crop/vegetation health prediction and optimization on PVAMU research farm to enhance research and outreach activities, and educate and train limited resource farmers and other stakeholders on outcomes of the project.
Project Methods
1) Collect the in-situ measurements of ground sensors (highest resolution, most accurate), digital imaging using a drone or UAV (mid-resolution, mid-scale), and satellite data (lowest resolution, biggest scale).The normalized difference vegetation index (NDVI) has been used for past decades as a method of assessing plant health and recognized as a standard indicator for vegetation and crop health assessment. NDVI imagery has been widely used to monitor crop health. NDVI is a calculated index used to monitor crop health and photosynthetic activity. The higher the index value the better the crop health. NDVI has been correlated to many variables such as crop nutrient deficiency, crop yield, water stress, and others. On the other hand, NDVI can be used to monitor plant growth that reflects various plant growth factors.We will use the PVAMU Research farm (778-acre) as a focused study area to monitor vegetation health during the growing season using MODIS satellite NDVI product and UAV derived NDVIs. The UAV derived NDVI will also be used to downscale the MODIS NDVI from 250-m to 10-m spatial resolution. During the study period, we will grow two major crops (Corn and Sorghum). There are four major crops (Cotton, Corn, Wheat, and Sorghum) grown in the state of Texas. We will also select two farms which are used to grow cornand sorghum from the Brazos River Watersheds (BRW) to monitor crop health during the study period.To obtain ground-truth and perform calibration and validation, we will train and validate MODIS NDVI using a drone and in situ measurements at two different locations and two growing seasons. We will train NDVI during the growing season 1, and validate it during the growing season 2. Similarly, we will train Sentinel 1 & 2 high-resolution satellite products using a drone and in situ measurements at same locations and same growingseasons.To improve scalability of the project, we will extrapolate our approach/tool to otherparts of Texas at the state level using MODIS NDVI (250 m) and Sentinel 1 & 2 (10 m) since these two satellite products are available at the global scale. At the watershed and state scale, we will develop tool/model to upscale MODIS NDVI from 250 m to 10m spatial resolution using Sentinel 1 & 2 images. The final MODIS NDVI product would be MODIS NDVI at 10 m resolutions, which will be used to develop a visualization tool for farmers and other stakeholders.2) Based on the collected multi-scale multi-resolution data, we plan to develop novel deep learning algorithms to extract knowledge and predict crop/vegetation health accurately.We propose a novel framework of deep semi-supervised learning on the big agriculture data for predicting crop/vegetationhealth. Specifically, we evaluate the network for each training input(e.g., images taken by drone) with the supervised path and the unsupervised path to complete two tasks. One task is to learn how to mine patterns of images regarding the labels (e.g., crop health) while the other is to optimize the representations of images without the labels. Before these two paths, there is a shared neural networks to extract low-level features to feed the later two neural networks.In addition, these twopaths can have independent neural networks with the identical or different setups for supervised learning and unsupervised learning, respectively. They generate two prediction vectors that are new representations for the inputs with respect to their tasks. For the identical setups of these two path, i.e., using the same structure of neural networks for both paths, it is important to notice that, because of dropout regularization, training neural networks in these two paths is a stochastic process. This will result in the two neural networks having different link weights and filters during training.For precision farming, we plan to employ the proposed semi-supervised deep learning for predicting crop/vegetation health. The input will be images of crop and the ground truth is the crop health condition. For training the semi-supervised model, cross-entropy loss will be the difference between ground truthand prediction.Crop/vegetation health is the key to increase yield to optimize the profit for each field. In this task, we plan to monitor the health of crops through analyzing different sources of data. There are three main sources of data related to crop/vegetation health: Satellite data (lowest resolution, biggest scale), Drone images (mid-resolution, mid-scale), and Field sensor data (highest resolution, smallest scale). Unfortunately, resource constrained farmers cannot obtain the drone images and field sensor readings due to the cost. Only satellite readings together with temperature, humidity, and sunshine time are available to these farmers. In this project, the technique of transfer learning is applied. A combined autoencoder and multimodal machine learning model is proposed and trained using the full spectrum of data from the PVAMU farm. Then the trained model will be used by resource constrained farmers to forecast the crop/vegetation health in fine granularity solely based on the limited information. The farmers will provide valuable feedback for us to further improve the model, maybe with augmented data (we planto outreach to some of the farmers to get additional data).3) We will further develop visualization/WebApp tool for the eld test for farmers, to predict the crop/vegetation health only with satellite data, and build monitoring and management tools for farmers to better manage their crop/vegetation and maximize yields.Information visualization tools are critical in data analytics to deliver the vital information to end users effectively to support user communication and data exploration.Withthe explosion of big data, it has become increasingly challenging to analyze and visualize data. There are many commercial visualization tools such as Tableau, Oracle Visual Analyzer, Qlik, Power BI by Microsoftand SAS Visual Analytics. Becausethe commercial visualization tools usually extend the analytics with a specific product from a company,they are not interoperable with different data analytic tools ordatabase systems.The PVAMU team has explored cloud-based visualization software packages, and developed a preliminary big data visualization capability on the PVAMU cloud to support streaming data visualization. A cloud based visualization package is developed based on the D3 Javascript packageto dynamically create a variety of 2Dchartsand graphs to visualize large amounts of streaming data set using any web browsers on any devices. With the developed modules, users are able to remotely visualize big data with small network bandwidth required. These big data sets are rendered at a render server resides in the cloud, and the rendered images are transferred to users via web browsers with H.264 video decoding.In order to address the challenge of visualizing the crop/vegetation health in a farm, we propose to extend our preliminary visualization work to build a "Crop/Vegetation Health Dashboard". This dashboard will be integrated seamlessly with the machine learning algorithms developed in Task 2 to provide real-time and user-friendly interaction to support data exploration and on-demand visual data mining. It will provide detailed information of the crop/vegetation health with high resolution based on the satellite data and the trained machine learning models in Task 2. Moreover, we plan to develop the smartphone apps for iOS and Android to better support data visualization, user interactivity, and event notifications.

Progress 05/01/24 to 04/30/25

Outputs
Target Audience:ThetargetaudienceincludefacultyandstudentsinbothCollegeofEngineeringandCollegeofAgriculture,Food,and Natural Resources at PVAMU. Specifically, the following seminars had been offered to the faculty and students in both College of Engineering and College of Agriculture, Food and Natural Resources at PVAMU: Title: "Deep Knowledge Tracing for Personalized Adaptive Learning at Historically Black Colleges and Universities (HBCUs)," by Co-PI Dr. Xishuang Dong, Assistant Professor, Department of Electrical and Computer Engineering, Prairie View A&M University, on Oct 23, 2024. Title: "Building urban climate resilience using spatial data science," by Wei Zhai, Ph.D., Assistant Professor, Urban and Regional Planning, Architecture and Planning, University of Texas San Antonio, on Oct 9, 2024. Title: "Mechanical Stimulation Increases Sorghum Stem Strength and Flexibility and is Associated with Altered Transcriptome Expression and Hormone Homeostasis," by Qing Li, Ph.D., Post-doctoral researcher, CARC, College of Agriculture, Food, and Natural Resources, Prairie View A&M University, on April 24, 2024. Title: "Toward reliability evaluation of computational models of protein molecules and their interactions," by Md Hossein Shuvo, Ph.D., Assistant Professor, Department of Computer Science, Roy G. Perry College of Engineering, Prairie View A&M University, on Feb 19, 2025. Title: "Vascular tissue-mediated molecular responses to low phosphorus," by Cankui Zhang, Ph.D., Associate Professor, Dept. of Agronomy at Purdue University, on March 5, 2025. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided? Multiple students have worked closely with the PIs on the project. The PIs hold regular bi-weekly meetings with the students, discuss their progress and provide guidance on their research works. The PIs also provides feedback on their draft papers and thesis, project reports and presentations. The students are encouraged to present their research results in professional conferences and workshops and develop their professional networks. We have provided multiple tutorial workshops and trainings to students. For instance, a Python Programming Workshop was offered in Fall 2024 at PVAMU. How have the results been disseminated to communities of interest?The results are disseminated through a seminar series organized by the PIs, journal and conference publications, new courses, trainings, and a website dedicated to this project. In addition, the PIs also visited many researchers in the field to discuss research results and seek potential collaborations. Specifically, (1)A website has been created and maintained to help disseminate the research results and recruit students to participate in this project: https://www.pvamu.edu/engineering/usda-nifa/ (2)A seminar series had been offered: Title: "Deep Knowledge Tracing for Personalized Adaptive Learning at Historically Black College and Universities (HBCUs)," by Co-PI Dr. Xishuang Dong, Assistant Professor, Department of Electrical and Computer Engineering, Prairie View A&M University, on Oct 23, 2024. Title: "Building urban climate resilience using spatial data science," by Wei Zhai, Ph.D., Assistant Professor, Urban and Regional Planning, Architecture and Planning, University of Texas San Antonio, on Oct 9, 2024. Title: "Mechanical Stimulation Increases Sorghum Stem Strength and Flexibility and is Associated with Altered Transcriptome Expression and Hormone Homeostasis," by Qing Li, Ph.D., Post-doctoral researcher, CARC, College of Agriculture, Food, and Natural Resources, Prairie View A&M University, on April 24, 2024. Title: "Toward reliability evaluation of computational models of protein molecules and their interactions," by Md Hossein Shuvo, Ph.D., Assistant Professor, Department of Computer Science, Roy G. Perry College of Engineering, Prairie View A&M University, on Feb 19, 2025. Title: "Vascular tissue-mediated molecular responses to low phosphorus," by Cankui Zhang, Ph.D., Associate Professor, Dept. of Agronomy at Purdue University, on March 5, 2025. What do you plan to do during the next reporting period to accomplish the goals? We plan to collect images of plants using UAV at the experimental farm field at PVAMU. We plan to explore the application of the emerging generative AI tools for this project.

Impacts
What was accomplished under these goals? During this funding period, we started to explore images from drones for crop/vegetation health assessment and prediction. Specifically, themulti-spectral images of farm land taken by drones are used to calculate theNormalized Difference Vegetation Index (NDVI) values.Then various machine learning models such as CNN based models and Transformer-based models have been designed and implemented to forecast Normalized Difference Vegetation Index (NDVI) values.

Publications

  • Type: Peer Reviewed Journal Articles Status: Published Year Published: 2025 Citation: Wilson, D.D., Tefera, G.W., Ray, R.L. 2025. Application of Google Earth Engine to Monitor Greenhouse Gases. A Review. Data/MDPI, 10(1), 8. https://doi.org/10.3390/data10010008


Progress 05/01/23 to 04/30/24

Outputs
Target Audience:The target audience include faculty and students in both College of Engineering and College of Agriculture,Food, and Natural Resources at PVAMU. Specifically, we have reached to the following groups: (1) A new course, "Machine Learning for Engineering Applications" has been developed in Fall 2022 and taught in both Fall 2022 andFall 2023semester by co-PI (Dong). Many students enrolled in the class and very positive feedback had been obtained from the students. (2) The following seminars had been offered to the faculty and students in both College of Engineering and College of Agriculture, Food and Natural Resources at PVAMU: (i) Title: "Proformer-Based Ensemble Learning for Gene Expression Prediction," by Co-PI Dr. Xishuang Dong, Assistant Professor,Department of Electrical and Computer Engineering, Prairie View A&M University, on March 6, 2024. (ii) Title: "Branches, Boundaries and Bracts: in Search of the Meristematic Homunculus," by Dr. Clint Whipple, Associate Professor,Whipple Lab, Biology, College of Life Sciences, Brigham Young University, on October4, 2023. (iii)Title: "Functionally repolarizing myeloid stroma with synthetic STING agonists drives immune clearance of Glioblastoma," by Dr.Michael Curran, Associate Professor, Department of Immunology, Division of Basic Science Research, The University of Texas MD Anderson Cancer Center, Houston, TX on May 10, 2023. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided?(1) Multiple students have worked closely with the PIs on the project. The PIs hold regular bi-weekly meetings with the students, discuss their progress and provide guidance on their research works. The PIs also provides feedback on their draft papers and thesis, project reports and presentations. (2) The students are encouraged to present their research results in proefsssional conferences and wrokshops and develop their professsional networks. (3) We have provided multiple tutorial workshops and trainings to students: for instance, we organized and delivered (together with NVIDIA) the Workshop and Training Courses of "Fundamental of Deep Learning" on October 19, 2023 at PVAMU campus. Many students received NVIDIACertificate in Deep Learning after participating the training. How have the results been disseminated to communities of interest?The results are disseminated through a seminar series organized by the PIs, journal and conference publications, new courses, trainings, and a website dedicated to this project.In addition, the PIs also visited many researchers in the field to discuss research results and seek potential collaborations. Specifically, (i) Two conference papers were published and presented and one journal paper was submitted and under review. -- Journal paper: Olamofe, R. Ray, X. Dong, and L. Qian, "Normalized Difference Vegetation Index Prediction using Reservoir Computing and Pretrained Language Models," submitted to Artificial Intelligence in Agriculture, 2024. -- Conference papers: 1) J. Olamofe, R. Ray, X. Dong, and L. Qian, "Normalized Difference Vegetation Index Prediction using Reservoir Computing and Pretrained Language Models," The 2024 AI in Agriculture and Natural Resources Conference, April 15-17, 2024, College Station, TX. 2) L. Nwosu, X. Li, S. Kim, L. Qian, X. Dong (2023). "Proformer-based Ensemble Learning for Gene Expression Prediction," ICIBM. (ii) A website has been created and maintained to help disseminate the research results and recruit students to participate in this project: https://www.pvamu.edu/engineering/usda-nifa/ (iii) A seminar serieshad been offered: 1) Title: "Proformer-Based Ensemble Learning for Gene Expression Prediction," by Co-PI Dr. Xishuang Dong, Assistant Professor,Department of Electrical and Computer Engineering, Prairie View A&M University, on March 6, 2024. 2) Title: "Branches, Boundaries and Bracts: in Search of the Meristematic Homunculus," by Dr. Clint Whipple, Associate Professor,Whipple Lab, Biology, College of Life Sciences, Brigham Young University, on October4, 2023. 3)Title: "Functionally repolarizing myeloid stroma with synthetic STING agonists drives immune clearance of Glioblastoma," by Dr.Michael Curran, Associate Professor, Department of Immunology, Division of Basic Science Research, The University of Texas MD Anderson Cancer Center, Houston, TX on May 10, 2023. What do you plan to do during the next reporting period to accomplish the goals?(1) We plan to collect sensor data in the Summerof 2024 using the field sensors deployed in the PVAMU research farm. We also plan to collect images of plants using UAV at the same experimental farm field. (2) We plan to explore the application of the emerging generative AI tools for this project. (3) We plan to recruit more students especially female students to participate this project.

Impacts
What was accomplished under these goals? During this funding period, we evaluated the performance of Reservoir Computing (RC) and Transformer-based model to forecast Normalized Difference Vegetation Index (NDVI) values. Using "MODIS/Terra Vegetation Indices 16-Day L3 Global 250m SIN Grid V061" dataset, we compared the forecast performance of these models to traditional ML/DL methods such as Nonlinear Regression, Decision Tree, Reservoir Computing (RC), Pretrained LSTM Reservoir, Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM) network, and DLinear. Notably, the DLinear/LSTM model showed exceptional predictive accuracy, while the Pretrained RC model significantly enhanced traditional RC model forecasts. Additionally, Frozen Pretrained Transformer (FPT) showed superior performance in forecasting specific NDVI values (most often peak/lowest NDVI), suggesting its effectiveness in precise time series predictions. Transformer-based models, specifically PatchTST and FPT, demonstrated substantial mean squared error reductions, particularly in limited data scenarios (50% and 1% sample sizes), indicating their robustness in precise NDVI time series predictions under data constraints.

Publications

  • Type: Journal Articles Status: Under Review Year Published: 2024 Citation: J. Olamofe, R. Ray, X. Dong, and L. Qian, Normalized Difference Vegetation Index Prediction using Reservoir Computing and Pretrained Language Models, submitted to Artificial Intelligence in Agriculture, 2024
  • Type: Conference Papers and Presentations Status: Accepted Year Published: 2024 Citation: J. Olamofe, R. Ray, X. Dong, and L. Qian, Normalized Difference Vegetation Index Prediction using Reservoir Computing and Pretrained Language Models, The 2024 AI in Agriculture and Natural Resources Conference, April 15-17, 2024, College Station, TX.


Progress 05/01/22 to 04/30/23

Outputs
Target Audience:The target audience include faculty and students in both College of Engineering and College of Agriculture and Human Sciences at PVAMU. Specifically, we have reached to the following groups: (1) A new course, "Machine Learning for Engineering Applications"has been developed and taught in Fall 2022 semesterby co-PI (Dong). fifteen (15) students enrolled in the class and very positive feedback had beenobtained from the students. (2) The following seminars had been offered to thefaculty and students in both College of Engineering and College of Agriculture and Human Sciences at PVAMU: (i) Title: "Decoding Sorghum genome using a mutant population," by Dr. Yinping Jiao, Assistant Professor, Institute of Genomics for Crop Abiotic Stress Tolerance (IGCAST), Texas Tech University, on Jan.25, 2023. (ii) Title: "Several partially distinct molecular pathways control shoot branching in the grasses," by Dr. Tesfamichael Kebrom, Research Scientist, CARC, College of Agriculture and Human Sciences & CCSB, Electrical and Computer Engineering Department, Roy G. College of Engineering, Prairie View A&M University, on Feb. 08, 2023. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided?(1) Students have worked closely with the PIs on the project. The PIs hold regular bi-weekly meetings with the students, discuss their progress and provide guidance on their research works. The PIs also provides feedback on their draft papers and thesis, project reports and presentations. (2)A hands-on Python tutorial training was offered in Fall 2022semester. Many faculty, staff, and students participated the training and very positive feedback has been received. The PIs plan to offer more tutorials in the coming semesters. (3) A new course, "Machine Learning for Engineering Applications"has been developed and taught in Fall 2022 semesterby co-PI (Dong). fifteen (15) students enrolled in the class and trained on using machine learning to solve real-world problems. How have the results been disseminated to communities of interest?The results are disseminated through a seminar series organized by the PIs, journal and conference publications, new courses, trainings, and a website dedicated to this project.In addition, the PIs also visited many researchers in the field to discuss research results and seek potential collaborations. Specifically, (1) A new course, "Machine Learning for Engineering Applications"has been developed and taught in Fall 2022 semesterby co-PI (Dong). The research results have been incorporated in the class materials. Fifteen (15) students enrolled in the class and very positive feedback had beenobtained from the students. (2) The following seminars had been offered to thefaculty and students in both College of Engineering and College of Agriculture and Human Sciences at PVAMU: (i) Title: "Decoding Sorghum genome using a mutant population," by Dr. Yinping Jiao, Assistant Professor, Institute of Genomics for Crop Abiotic Stress Tolerance (IGCAST), Texas Tech University, on Jan.25, 2023. (ii) Title: "Several partially distinct molecular pathways control shoot branching in the grasses," by Dr. Tesfamichael Kebrom, Research Scientist, CARC, College of Agriculture and Human Sciences & CCSB, Electrical and Computer Engineering Department, Roy G. College of Engineering, Prairie View A&M University, on Feb. 08, 2023. (3) A conference paper was submitted for publication. (4) A webpage has been created for this project.Publications, research activities and software packages are available on the website to share with the research community at large. What do you plan to do during the next reporting period to accomplish the goals?We will collect drone image data and field sensor data in Spring and Summer of 2023 and complete the multi-resolution data processing as planned. Inaddition, novel semi-supervised deep learning models and reservoir computing models will be developed for plant health prediction.

Impacts
What was accomplished under these goals? (1) The satellite image dataset had been obtained and converted to NDVI values for the farms in Texas from 2000 to 2022. Then various machine learning and deep learning models including decision trees, linear regression, CNN, LSTM have been trained on the obtained data. It is observed that LSTM outperformed other models in terms of MAPE and MSE. (2) Field sensors have been deployed in the testbed of the PVAMU farm. We plan to collect sensor data in the Spring and Summer growing season of 2023.

Publications

  • Type: Conference Papers and Presentations Status: Submitted Year Published: 2023 Citation: L. Nwosu, X. Li, S. Kim, L. Qian, X. Dong (2023). Proformer-based Ensemble Learning for Gene Expression Prediction, submitted to ICIBM.
  • Type: Websites Status: Published Year Published: 2023 Citation: https://www.pvamu.edu/engineering/usda-nifa/