Progress 09/01/18 to 08/31/23
Outputs
Target Audience:FINAL REPORT. Throughout the duration of this project, our target audience continued to be scientists, engineers, and students at profession conferenes andseminars, as well as readers of relevant peer-reviewed publications generated from this effort. Specifically, our target audiences included the agriculture scientists and engineers;technologists; electrical and computer engineers; and K- 12, undergraduate, and graduate students. Over the duration of this project, we have contribtued to mulitpleIEEE SENSORS Conference proceedings and andpeer-reviewed journal articles to reach the widest possible audience. In additoin, we have presented this work at multiple workshops connected to the NC State Plant Sciences Initiative,whoseparticipants include both crop and plant scientists, extension specialists, and farmers. In addition to the graduate students and postdoctoral researchers trained during the project, high school and undergraduate students have participated in the design and development of sensor and autonomous systems hardware. Changes/Problems:Challenges in general research operations were realized due the COVID-19 global pandemic. These were accordinlgy responded to and alternative research methods were initiated. What opportunities for training and professional development has the project provided?Over the duration of this project, atotal of 10 graduate students have contributed to the development of the sensing hardware, assisted with the development of the automated imaging, and data analysis. All gradaute students were trained in interdsciplinary engineering principles and worked collaboratively on this effort. Gradaute students ranged in backgrounds, including statistics, biomedical engineering, electrical engineering, and machine learning/computer vision. We have also collaborated with and training 1 research technician and 3 undergradaute students, respectively. One graduate student defended his doctoral dissertation, entitled "Electrical Techniques and Systems for Precision Agriculture and Rapid Phenotyping", and this graduate student has recently accepted a role as Research Assistant Professor to continue efforts on developing and testing this project's technologies. Lastly, this project led to the development of course modules on plant electrohphysiology for BME 425/525 Bioelectricity, instruced by PI Daniele. This course will also feature a plant electrphysiology experiemntal lab in Spring 2024. This course enrolls approximately 20-30 undergradaute and graduates students every year. How have the results been disseminated to communities of interest?The results of this reserach has been published (5 peer-reviewed proceedings papers) and presented at international conferences (IEEE SENSORS 2019, 2020, 2021, 2022) and invited seminars. We have published 1 peer-reviewed journal article, 1 peer-reviewed journal article is in submission, and 2peer-reviewed journal articles are in preparation based on the final experimental results. Furthermore, thanks to the collaborative reserach efforts, we have been enabled to present this work at multiple local, national, and interanational workshopos focused on agri-food technologies and plant sciences. What do you plan to do during the next reporting period to accomplish the goals?
Nothing Reported
Impacts
What was accomplished under these goals?
Integration of Plant Bioimpedance Monitoring Hardware:This project aimed to create a cost-effective integrated system merging plant electrophysiological and environmental sensors for accurate data collection. Notably, the integration of plant bioimpedance sensing with various environmental parameters demonstrated impressive cost savings. The development of a low-cost light sensor and a soil moisture sensor, comparable to expensive counterparts, showcased the project's success. Calibration experiments played a vital role in achieving accurate measurements. Additional system features, such as a solar charging mechanism and a data transmission radio, were implemented. Extensive outdoor and indoor tests demonstrated promising performance, with initial on-plant sensors providing baseline measurements presented at a professional conference. Research delved into the impact of drought stress on bioimpedance, revealing a significant increase within 20 minutes. Verification of these measurements was conducted using electrochemical impedance spectroscopy and a plant tissue model, breaking down bioimpedance into capacitive and resistive elements representing cellular components. A novel method for automatically calculating model parameters highlighted increased extracellular resistance during drought stress, aligning with biological evidence. The findings were extended to maize grown in different mediums, correlating bioimpedance with soil parameters. Preliminary imaging results suggested bioimpedance could indicate drought stress before visual indicators, warranting further investigation. Future work involves integrating bioimpedance measurements into current methodologies, addressing biological variation in drought sensing, and exploring the impact of soil nutrients. Recent efforts focused on investigating the relationship between low-frequency impedance and leaf water potential in maize and soy. Additionally, the project included combining sensing modalities, simultaneously measuring leaf impedance, soil moisture, band circumference, and spectral imaging, with ongoing analysis planned for future publication. Fig 1: Plant bioimpedance response to draught stress, and performance of the integrated sensing system during a 12-day segment of operation. Physical Sensing System:The initial plan for circumferential strain gauges encountered fabrication challenges, leading to conductor delamination and limitations in achievable percent elongation. An alternative strategy emerged, utilizing a non-stretching substrate looped around the plant stalk to detect circumference changes. Three prototypes with different transduction circuits were developed to address issues like resistive dividers and multiplexer circuits. Exposed conductive pads, vulnerable to moisture and debris, prompted the creation of a second generation of insulated sensor designs using AC excitation signals for capacitive coupling. Fabrication and testing are ongoing alongside the original sensors. Recent efforts on plant stalk growth sensors focused on enhancing the fourth-generation original sensor and developing a standalone variant. Iterations to the original sensor improved data quality, reliability, and production yield. Testing against normally-watered and drought-stressed plants demonstrated a daily accuracy of approximately 3.65% over 90 days, with a 50% sensor uptime. The standalone variant, functioning as an independent I2C-based peripheral device, underwent successful benchtop tests. Field testing is planned, and ongoing trials assess performance in a multimodal data collection ensemble, combining stalk growth measurement, soil moisture measurement, electrochemical impedance spectroscopy, and photogrammetric reconstruction of plant geometry. Physical sensor performance, compared to manual stalk growth measurements, is illustrated in Fig 2. Fig 2: Comparison of electronic plant growth sensor to manual measurements. Automated Imaging System.The goal of this project was to develop a low-cost imaging setup capable of performing high throughput phenotyping for early detection of drought stress in maize plants, which could be reinforced by "at-plant" sensing systems. Accordingly, we developed the following methodology and hardware: YEL Detection and Segmentation/Choosing an appropriate region for extracting NIR pixels and ViT Experiments: Detecting the youngest expanding leaf (YEL) in maize plants under water stress posed challenges due to its small size, hidden position, and shifting views from pot movement. Overlapping leaves in later trial days complicated analysis. A novel approach was taken, selecting a region encompassing YEL and stem for each plant instead of training a YEL-specific model. When YEL wasn't visible, the next youngest leaf was annotated, streamlining with Labelbox's Model Assisted Labeling (MAL) and Facebook's Detectron2 library for FasterRCNN model training. A Vision Transformer (ViT) excelled in classifying drought-stressed and well-watered plants, fine-tuned from Google's pre-trained ViT Model. The dataset from three trials included 1278 training images, employing an 85%-10% train-validation split. Test datasets underwent inference using the trained model, excluding Trial 001 from sets 3 and 4 due to poor results. Five models with varied bounding box sizes were trained, aiming to find an optimal size for drought stress detection while assessing leaf occlusion impact. Larger widths were expected to perform worse. Extracted regions from bounding box detections are shown in Fig 3. Fig 3: a) Bounding box including the YEL and stem. b)-e) Bounding boxes dilated by 200, 400, 700 and 1000 pixels, respectively. Combining all trials reveals consistently high accuracy, with a slight increase in classification accuracy across trials (Fig 4 b) and c)). Accuracy improves for n=0,200,400 compared to n=700 and 1000, and a similar pattern is observed for Fig 4 e) and f). However, overall accuracy is lower for e) and f) compared to b) and c), attributed to different trials in train and test sets. A notable accuracy increase is observed from the first to the last day of the trial, possibly due to increased drought stress aiding ViT classification. For e) and f), higher accuracy is seen for n=0,200 and 400 than n=700 and 1000, aligning with the hypothesis that larger bounding boxes with more leaf occlusions result in poorer accuracy. No leaf occlusion is present in n=0,200,400, whereas n=700 and 1000 exhibit increasing occlusion, explaining the accuracy drop. In contrast, results on Trial 001 (Fig 4 a) and d)) deviate from expected patterns seen in Trials 002 and 003. Trial 001's longer duration and overgrown plants towards the end may contribute to increased occlusion and noisy data, warranting further investigation. Fig 4: Classification accuracy results for the experiment sets shown in Table 1. a)-f) Experiments 1 through 6, respectively. Building a 3D Imaging System:To enable more accurate phenotyping of the plants including extraction of geometric parameters, we have built a robotic system for imaging plants. The system consists of a set of cameras placed 3 feet away from the center target. The cameras are rotated around the target as shown in the 3D reconstruction in Fig. 5 [Bottom-Right]. This allows us to create a Neural Radiance Field (NeRF) model capable of generating new photorealistic views of the plant. This information can be used to extract accurate 3D meshes for geometric analysis or to generate new images for data augmentation to enhance training of our deep learning models. Fig 5: 3D Reconstruction using Robotic Imaging System. [Top Row] Images captured with the robotic system are used for training of a Neural Radiance Field (NeRF) 3D model [Bottom Right]. This model can be used for generated new views [Bottom Left] or to extract a 3D mesh to extract geometric phenotyping characteristics
Publications
- Type:
Journal Articles
Status:
Published
Year Published:
2023
Citation:
Reynolds, James, Matt Taggart, Devon Martin, Edgar Lobaton, Amanda Cardoso, Michael Daniele, and Alper Bozkurt. "Rapid Drought Stress Detection in Plants Using Bioimpedance Measurements and Analysis." IEEE Transactions on AgriFood Electronics (2023).
|
Progress 09/01/21 to 08/31/22
Outputs
Target Audience:Our target audience has remained consistent. We have reached audiences at virtual conferenes and seminars too. Our target audiences included the agriculture scientists and engineers; technologists; electrical and computer engineers; and K-12, undergraduate, and graduate students. High school, undergraduate, and graduate students have participated in the design and developmetn of sensor and autonomous systems hardware. We have presented this work at a workshop, whose participants include both crop and plant scientists, as well as engineers. To further reach this audience, we organized and contributed to a dedicated agricultural sensing track at the IEEE Sensors 2022conference. Changes/Problems:Our collaborator, Dr. Thomas Rufty, has retired. Our new team member, Dr. Amanda Cardoso, brings similar expertise and new capabilities and analytical technolgoies for plant monitoring which will further support the research objectives. What opportunities for training and professional development has the project provided?Training: To date total of 8 graduate students have contributed to the development of the sensing hardware, assisted with the development of the automated imaging, and data analysis. All gradaute students were trained in interdsciplinary engineering principles and worked collaboratively on this effort. We have also collaborating with and training 1 research technicianand 3 undergradaute students, respectively.And one student defended his doctoral dissertation during FY2021 and is currenlty participating in the efforts as a postdoctoral researcher. How have the results been disseminated to communities of interest?The results of this reserach has been publsihed and presented at international conferences and invited seminars. In the upcoming research period, we have 3 manuscripts in preparation for peer-reviewed journal publication (2 already in submission and under review or revision). What do you plan to do during the next reporting period to accomplish the goals?In order to allow for compatibility with arbitrary third-party systems, the validated and tested sensor design is being translated into a standalone sensing platform, which will be capable of interfacing with external electronics over a communications bus (I2C, SPI, or UART). This change requires the incorporation of a small microcontroller onto the flexible sensor board itself, eliminating the need for multiplexing functions to be implemented by an off-sensor microcontroller, reducing processing overhead for the base unit to which the sensor is interfaced. Additionally, this method requires fewer conductors to be connected between the sensor and the base station, dramatically reducing net weight (in spite of a small addition of weight on the sensor board itself). An additional development has been the development of a band-based impedance sensor which recycles the tested band sensor architecture, with a different electrode configuration. This additional functionality will be tested further in future trials, along with the rest of the multimodal sensing platform. Lastly, devleopment and integration of the imaging system is under revision to reduce the number of moving parts and improve reproducibility. In the final research period, we will integrate all compenents and evaluate a static imaging system for informing the machine learning-supported prediciotn of water stress.
Impacts
What was accomplished under these goals?
The primary efforts involving bioimpedance this year focused on improving and expanding upon the techniques we've previously developed. Instead of measuring the effect of drought on the maize leaves, we inserted electrodes into the main stalk. While this does provide a better overall correlation with the plant's health, it can cause developmental damage to the plant. We then tested a version of the maize circumference band that had electrodes on the inner side. This new method seems to be a promising improvement to the prior approaches by limiting the damage to the plant while providing long term measurements. Ongoing development of the plant stalk growth sensor has been focused on main areas: continued data collection, improving fabrication quality control, improving signal quality, and the development of the existing sensor into a standalone sensor peripheral that can be integrated into other embedded systems using a standard communications bus. Additional growth trials have been conducted to increase the replicate number of data series collected using the sensor. Thus far, these tests have been performed under normal growth conditions - an additional set of trials under drought conditions to serve as a control are planned for the near future. Data collection efforts have been aided by improvements to the sensor's output signal quality, primarily through the incorporation of a pulldown resistor on the main collection line - the inclusion of potting compound, initially expected to further improve noise immunity, was not found to be necessary following additional trials. Additionally, a prototype testing utility has been created to provide external hardware and real-time sensor output feedback to test newly fabricated sensors in a rapid manner, increasing the likelihood of successful deployment due to the difficulties previously encountered in assessing sensor functionality after placement on-plant until sufficient time has elapsed. We continued the analysis of changes in the reflectance of the plant by comparing the values from the Youngest Expanding Leaf (YEL), the stem, and the leaves. Differences in the histograms of the values were observed for most of the trials with some irregularities present in other trials. In order to further explore differences in the images themselves, we started using Vision Transformers (VIT), a type of deep neural network architecture, to train a classifier between well-watered and water-stressed plants. The self-attention mechanism of this model was used as an indicator that the model was paying attention to the correct structural features. Our results indicate that the accuracy for classification over time (i.e., from day 1 of the water-stress protocol until day 6) improved from 79% accuracy to 100% when we considered the raw images. However, closer inspection of the self-attention masks indicated that features such as the pots and items in the background were used as landmarks for the classification. This is not ideal, so we proceeded to remove the background of the images (including the pots) and noticed that accuracy was down to 50% (i.e., random guessing). We further considered focusing only on the YEL and stem by masking out the leaves and that resulted in a high accuracy of 92% on day 1 up to 100% on day 6. We are working on further validating these results with all the trials that we currently have in our dataset.
Publications
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2022
Citation:
Jack Twiddy, Matthew Taggart, James Reynolds, Chris Sharkey, Thomas Rufty, Edgar Lobaton, Alper Bozkurt, Michael Daniele. "Real-Time Monitoring of Plant Stalk Growth Using a Flexible Printed Circuit Board Sensor." IEEE SENSORS 2022 (Dallas, TX)
|
Progress 09/01/20 to 08/31/21
Outputs
Target Audience:In the thirdyear of this effort, our target audience has remained consistent. We have reached audiences at virtual conferenes and seminars too. Our target audiences included the agriculture scientists and engineers; technologists; electrical and computer engineers; and K-12, undergraduate, and graduate students. High school, undergraduate, and graduate students have participated in the design and developmetn of sensor and autonomous systems hardware. We have presented this work at a workshop, whose participants include both crop and plant scientists, as well as engineers. To further reach this audience, we organized and contributed to adedicated agricultural sensing track at the IEEE Sensors 2021 conference. Changes/Problems:In FY2021, we were able to eventually return to full capacity in the labs. This enabled a restart to on-plant testing and evaluation. While the pandmeic restrictions did delay the work, it is progressing towards our research goals. What opportunities for training and professional development has the project provided?Training:To date total of 7 graduate students have contributed to the development of the sensing hardware, assisted with the development of the automated imaging, and data analysis. All gradaute students were trained in interdsciplinary engineering principles and worked collaboratively on this effort. Most notably, one technicican at the initial outset of the project is not an enrolled doctoral student. And one student defended his doctoral dissertation during FY2021. How have the results been disseminated to communities of interest?Proceedings Publication: The results of the data analysis pipeline for bioimpedance monitoring was acceptedfor 1 publication and presented atIEEE SENSORS 2021.Two additional journal manuscripts are in prepartaion for submission, and expected publication in FY2022. Other: A doctoral dissertation stemming from this work has been defended and pusblished. What do you plan to do during the next reporting period to accomplish the goals?In the following reporting period, we aim to finalize the full intergration of the electical and physical sensing system with automated imaging and anlysis. The final integration will be evaluated in a series of exerpiments to evaluate the abiliyt of the system to acheive an early prediction of water and nutrient stress in maize. Specifically, continuing efforts are focused on improving sensor reliability and output data quality, through the incorporation of pull-down resistors to minimize extraneous electrical noise, the application of an insulating compound to all areas of the output circuitry except the array to improve environmental isolation, and fine-tuning fabrication parameters. In particular, achieving good electrical contact between the mobile electrode and the stationary electrode array is contingent upon the deposition of sufficient solder to form the raised mobile electrode, to successfully bridge the gap between this electrode and the array electrodes. This has been observed to vary based on the fabrication method used, and work is ongoing to standardize this process to ensure consistent sensing capability prior to the collection of additional performance data. Further, debugging tools have been created to allow for better interrogation of individual sensor performance prior to deployment, providing real-time observation of sensor output upon manual manipulation as an additional quality control metric. Concerning the application to drought sensing, there's the issue of biological variation and determining which location on the plant provides the most reliable results. Knowing that we can now accurately measure the changes to the plant cellular conductivities, we will also look at the effect soil nutrients have on the bioimpedance measurements.
Impacts
What was accomplished under these goals?
Integration of Plant Bioimpedance Monitoring Hardware: Our focus over the past year has been evaluation and understanding bioimpedance measurements and the physical properties they correspond to.Having shown that a bioimpedance change results from drought stress, we investigated the fundamentals of the process. This involved two efforts. The first was inducing immediate drought conditions on plants while reducing the effects of other variables on the measurement. This was accomplished by hydroponically growing maize and then placing it in a 10bar polyethylene glycol solution. The bioimpedance (as measured using platinum-iridium needle electrodes on a portion of a leaf midrib) increased within 20 minutes and was significantly different from the controls.The second effort involved verifying that the measurements corresponded with the biological processes occurring in the plants. During rapid drought conditions, the extracellular water potential drastically decreases relative to the intracellular water. Using electrochemical impedance spectroscopy and a plant tissue model, the bioimpedance measurements can be broken down into capacitive and resistive elements that represent the cellular components. The problem is that this process has rarely been done on this scale with hundreds of timepoints and dozens of measurements per timepoint. We developed a novel method to automatically and accurately calculate the model parameters over large datasets. This was presented at and published by the IEEE SENSORS 2021 conference. Using this method, we found that the extracellular resistance increased while the intracellular resistance was maintained during drought stress, which corresponds with the biological evidence.We further demonstrated that this occurs in maize grown in sand and soil. The bioimpedance was also correlated with the soil volumetric water content and the fraction of transpirable soil water. Physical Sensing System. After shifting to a new circumference sensor, we have evlauted performance in the past year. This was implemented in the form of a flexible printed circuit board ("PCB") incorporating an array of "sensor" pads and a single "probe" pad - in a manner similar to a conventional linear or rotary encoder, this arrangement creates a "circumferential encoder." Motion of the probe pad along the array of sensor pads is transduced using one of several proposed transduction schemes. Three initial prototype designs were produced, utilizing three different transduction circuits. Continued development of the flexible circuit board-based mechanical sensor for plant stalk growth included a comparison of several possible sensing circuit topologies using static and dynamic tests, on- and off-plant tests of sensor band application techniques, and initial on-plant data collection and the subsequent revision of these sensors to improve reliability and performance. All sensors tested relied on a similar fundamental scheme: a "mobile" electrode is moved across an array of closely-spaced "static" electrodes as the sensor band expands toaccommodate the growing plant stalk . A signal is transferred between the mobile electrode and the static electrodes to indicate the position of the mobile electrode relative to the array, and thus the perimeter length of both the sensor band and plant stalk. Three circuit topologies - "array-type," "multiplexer-type," and "encoder-type" - allowed for the utilization of analog, digital, and mixed (respectively) output encoding of the mobile electrode's position. To compare performance, electrodes of each type were initially tested under static conditions, in which a wire lead was used to simulate movement of the mobile electrode across the array, secured flat against a testing surface. Next, these sensors were assembled onto a conical mandrel. By sliding the sensor bands down the mandrel from the smaller-diameter end to the larger-diameter end and recording the sensor output, placement on a growing plant could be quickly simulated. From these tests, only the multiplexer-type design produced an output signal reasonably representative of the actual diameter change occurring. This improved performance is likely due to the better resistance of a fully-digital signal to distortion in an electrically-noisy environment (although intermittent noise was still observed). Sensor band presence was found to generate reversible discoloration immediately under the sensor application site, and occasional superficial damage to nearby leaves, however differences in growth were not observed compared to control plants grown without sensors. On-plant tests of the fabricated sensor bands demonstrated successful collection of maize growth data over a 12-day periodfrom a majority (83%) of deployed sensors. ?Automated Imaging System. Maize Data Collection and Water Stress. The team has contributed to the data collection efforts in the phytotron focusing on the capture of color and near infrared (NIR) imaging of maize using the robotic imaging system develop by his lab. The team aimed to get enough trials to validate some previous observations on an experiment in which NIR intensity values were used to differentiate between water-stress and well-watered plants. The detection of the regions of interest were based on manual and automated annotations obtained using deep learning detection algorithms. Preliminary analysis of the new trials shows the possible validation of the earlier results. Some discrepancies are being solved that are due to irregularities in the execution of the watering protocols, which will be resolved before a publication is prepared. 3D Model Reconstruction. The team aimed to develop a pipeline using structure from motion (SfM) techniques based on the images obtained from maize in the phytotron to characterize phenotype using shape and geometry descriptors. The team was able to successfully build 3D models from the images. However, a preliminary study of some hand-crafted shape parameter did not indicate that the selected descriptors (e.g., plant width) could be used to discriminate between the water-stressed and control plants. More analysis will be performed once more data is curated and prepared for analysis. Besides making use of standard SfM techniques, the group has also explored the use of state-of-the-art implicit reconstruction approaches based on deep-learning architectures such as the Neural Radial Field (NeRF) models. Water Stress on Soybeans. The group has previously trained deep learning models for the predictionof soybean leaf-wilting from images of crops in the field. The images were obtained through a collaboration with Dr. Chris Reberg-Holton's group at NC State. The teamfocused on new techniques based on attention mechanisms and multi-instance learning to get better performance and a more explainable model. The improvements on performance come from creating simpler models that incorporate more sophisticated architectures that aim to better model the nature of our weak annotations in which an image is labelled based on feature that may be present in a small region of the picture. The explainable portion comes from the attention mechanisms highlighting regions in the image that contain useful information that is used for prediction. This may result on the ground been ignored and removing other portions that may be ambiguous. The results show are very encouraging and there is a publication in preparation.
Publications
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2021
Citation:
Towards Continuous Plant Bioimpedance Fitting and Parameter Estimation
D Martin, J Reynolds, M Daniele, E Lobaton, A Bozkurt
2021 IEEE Sensors, 1-4
- Type:
Theses/Dissertations
Status:
Published
Year Published:
2021
Citation:
Electrical Techniques and Systems for Precision Agriculture and Rapid Phenotyping.
JL Reynolds - 2021
|
Progress 09/01/19 to 08/31/20
Outputs
Target Audience:In the second year of this effort, target audiences included the agriculture scientists and engineers; technologists; electrical and computer engineers; and K-12, undergraduate, and graduate students. High school, undergraduate, and graduate students have participated in the design and developmetn of sensor and autonomous systems hardware. We have presented this work at a workshop, whose participants include both crop and plant scientists, as well as engineers. To further reach this audience, we initiated and organized a dedicated agricultural sensing track at the IEEE Sensors 2020 conference. Changes/Problems:Due to the COVID-19 pandemic, and limitations placed on on-campus research, we have had to delay our on-plant sensing. We have made alternative plans in which we were able to conduct research remotely. This resulted in continued progress andresults on the sensing and hardware development. We intend to accelerate the on-plant sensing and testing protocols, as we have returned to limited campus research, starting in August 2020. What opportunities for training and professional development has the project provided?Training: A total of 6 graduate students have contributed to the development of the sensing hardware,assisted with the development of the automated imaging, anddata analysis. All gradaute students were trained in interdsciplinary engineering principles and worked collaboratively on this effort. An addition twonew high school students participated in a summer training program,and they aided with code assembly and programming assignments. How have the results been disseminated to communities of interest?Proceedings Publication: The results of the sensing system and imaging have been accepted for 2 publications in IEEE SENSORS 2020, and these results will be presented at the corresponding international conference in October 2020. We will also be charing the agricultural sensing track at the IEEE SENSORS 2020 conference. A manuscript is in submission for publication in the Journal of Micromechanics and Microengineering. What do you plan to do during the next reporting period to accomplish the goals?Integration of Plant Monitoring Hardware and On Plant Characterization: Future research involves doing a more thorough investigation between the fraction of transpirable soil water and the bioimpedance. Lighting seems to possibly have an effect as well. Another potential insight might come from comparing bioimpedance and biopotental data with the imaging data. Concerning biopotentials, the objective is to correlate the underlying biological response with environmental stressors as a result of the biopotential activity. Hardware development and optimization are still ongoing as the system and sensors are tested and deployed. We are at the stage to fully integrate the sensor hardware with the imaging platform to collect longitudinal data during the entire growth of the maize models. This will begin in the upcoming period. We expected to have begun this portion of the effort, but due to the limitaitons on research imposed by the COVID-19 pandemic, we have had to dealy the on-plant sensing in the NC State Phytotron.
Impacts
What was accomplished under these goals?
Integration of Plant Monitoring Hardware:Our focus over the past year has been on understanding bioimpedance measurements and the physical properties they correspond to. With needles inserted into the vascular system or adjacent on the midrib, a distinct diurnal pattern of impedance magnitude values emerges with low frequency measurements. This pattern is species specific (at least among the 5 species measured). One hypothesis was that transpiration was the cause of the systematic fluctuations in bioimpedance measurements. However, varying the relative humidity at a fixed temperature in order to change the transpiration rate had no effect on the bioimpedance. The bioimpedance does correlate with the amount of water available in the soil. With soybean plants in a covered pot to keep limit water loss to just transpiration, the water stressed plants bioimpedance increased on average by over 200% from the initial state, which was significantly higher than that of the control group. Alongside this success, a potentiostat was integrated with the current portable measurement system to allow for bioimpedance measurements in the field. This system along with the preliminary results was accepted for presentation at an international conference (IEEE SENSORS 2020). Alongside the bacterial nanocellulose (BNC) electrode physiological effect testing on maize, we compared its efficacy in measuring electrical potentials as a result of chilling, burning, and cutting. Compared to the signal from the standard needle electrodes, the response via the BNC electrodes was well correlated and had a higher average amplitude. This verifies their usage in making reliable measurements as part of our phenotyping system. Electrodes and Sensor Materials:?n response to difficulties encountered in fabricating a circumferential sensor based on a strain gauge - specifically, delamination of the conductor from the substrate due to simultaneous expansion and tension, and limits to achievable percent elongation - an alternative strategy was developed to sense changes in circumference. In this method, a non-stretching substrate is instead looped around the plant stalk, and circumference changes are detected through movement of the substrate against itself as one end of the substrate slides to accommodate the growing stalk. This was implemented in the form of a flexible printed circuit board ("PCB") incorporating an array of "sensor" pads and a single "probe" pad - in a manner similar to a conventional linear or rotary encoder, this arrangement creates a "circumferential encoder." Motion of the probe pad along the array of sensor pads is transduced using one of several proposed transduction schemes. Three initial prototype designs were produced, utilizing three different transduction circuits. The first design arranges the pad array as nodes in a simple resistive divider, with the probe pad being used to detect the analog voltage of the contacted pad - specific positions correspond to specific pad voltages. The second design holds the probe pad at a constant voltage, and each sensor pad is polled using a multiplexer circuit to detect which sensor pad the probe pad is currently in contact with. The final design combines these two schemes in an attempt to reduce component count and geometric constraints related to routing of the array pads - in this case, three multiplexed nets are used to monitor local motion, coupled with a "reference" pad every tenth pad displaying a unique voltage to provide absolute position-keeping. The primary limitation of this approach is the vulnerability of these circuits to ingress of moisture or conductive environmental debris, due to the presence of exposed conductive pads on the sensor surface. To mitigate this, a second generation of insulated sensor designs is currently being developed, using a similar strategy to the first generation of sensors but modifying these circuits slightly to capacitively couple the probe pad to the sensor pads using an AC excitation signal. These pads are insulated by the normal soldermask layer over the pads, allowing for an environmentally-resistant surface across the entirety of the sensor. Modeling has suggested inter-pad capacitance on the order of picofarads, and additional simulations have been used to determine the frequency range required to implement this sensing scheme - this is expected to be in the range of 1MHz-50MHz, which can be generated by a local oscillator circuit. These circuits are currently being fabricated and characterized, in parallel with deployment testing in a controlled environment using the original first-generation sensors. Since the geometry of the active regions of each sensor generation are essentially the same, lessons learned from testing the first-generation sensors "on-plant" can be used to inform the final sensor design while the AC circuit is refined. Automated Imaging and Data Collection Platform: Exploratory Analysis of Drought on Maize. Based on the designed data collection system around 60 thousand maize images have been collected. Using thresholding techniques, pseudocolor was attributed to the plants in order to indicate the amount of IR being reflected from them. A binary mask was created using the saturation channel from the Hue Saturation Values (HSV) image using OpenCV thresholding techniques to remove the background (see Fig. 1b-d). This binary mask was applied to the NIR image to obtain a masked NIR image with the plant only see Fig. 1e). The NIR intensity is visualized by attributing a pseudo-color map, see Fig. 1f and Fig. 1g, where the youngest expanding leaf (YEL) is shown. In Fig 2. top images are from water stressed maize, the ones in the bottom are well watered. The pseudocolor images showed that water stress can be identified before such stress is visually apparent. At day 4 the differences can be noticed but it only becomes apparent after day 6. We evaluated the differences in NIR reflection by computing histograms from the pixels after the aforementioned steps comprising the entire plant and an localized analysis for YEL. Additionally, we evaluated the possible distinctions between the plants and themselves (i.e.: by comparing each day with the first day of experiment) over time, by utilizing Earth Mover's Distance (EMD). Evaluating the EMD enables one to see a significant transient behavior between the days 4 and 6, which is close to the period where the drought stress becomes apparent (see the Fig. 3a-c). It is important to note that YEL was not visible in a couple of sessions as one can see the missing points in Fig. 3c and Fig. 3e. After such intervals, specifically at day 7, the EMD increases continuously as expected. The EMD was also calculated comparing the plant to its first day of experiment (Figure 3d-e). Such comparison was done in two modalities, removing the stem with dilation and erosion and using the plant as a whole. The leaves-only plot demonstrates a linear trend for the drought stressed plant, which better aligned with our expectations. Figs. 3c and 3e show a more drastic distinction in the YEL NIR reflection, which aligns with elongation dependency results for water stress as found in the literature. We believe that such results have the potential to allow automatic early detection of drought stress using predictive models to find the level of drought stress from which the plant can still recover. Exploratory Analysis of Drought on Soybean. We have also collaborated with Chris Reberg-Horton's group at NC State making use of the images that they have collected of soybean in the field in order to develop algorithms for detection of leaf wilting. We are working towards a journal article with our results. We plan to integrate these efforts with our work with maize so we can either adapt models or perform multi-task learning in the near future.
Publications
- Type:
Conference Papers and Presentations
Status:
Accepted
Year Published:
2020
Citation:
An Environmental Station with Bioimpedance Capabilities for Agricultural Deployment
- Type:
Conference Papers and Presentations
Status:
Accepted
Year Published:
2020
Citation:
Feasibility Study of Water Stress Detection in Plants using a High-Throughput Low-Cost System
- Type:
Journal Articles
Status:
Submitted
Year Published:
2020
Citation:
Bacterial Nanocellulose Electrodes for the Study of Plant Electrophysiology
|
Progress 09/01/18 to 08/31/19
Outputs
Target Audience:In the first year of this effort, target audiences included the agriculture scientists and engineers; technologists; electrical and computer engineers; and K-12, undergraduate, and graduate students. High school, undergraduate, and graduate students have participated in the design and developmetn of sensor and autonomous systems hardware. We have presented this work at a workshop, whose participants include both crop and plant scientists, as well as engineers. Changes/Problems:
Nothing Reported
What opportunities for training and professional development has the project provided?Training:Two high school studentsassisted with the development of the automated imaging and sensing hardware. An addition two high school students participated in a summer training program, and they aided with code assembly andprogramming assignments. How have the results been disseminated to communities of interest?Proceedings Publication:The results of the on-plant sensing electrodes have been accepted for publication in IEEE SENSORS 2019, and these results will be presented at the corresponding international conference in October 2019. Presentations:Our results and future outlook for this work was presented at the "Stewards of the Future Workshop on Communicating withPlants: Deciphering and Exploiting Chemical Signals through Electronics" which was held at NC State on December 2-4, 2018. What do you plan to do during the next reporting period to accomplish the goals?Integration of Plant Monitoring Hardware andOn Plant Characterization:A myriad of improvements and goals are currently in progress or upcoming for the sensor hardware. A stretch sensor will integrate into the system to add stalk diameter to the list of known plant parameters. The soil moisture sensor needs to take into account the soil temperature--this can affect its accuracy. The imaging, impedance, and biopotential sensors need to be developed and incorporated into the system utilizing the techniques we have established with benchtop devices. These are multi-disciplinary, incorporating materials development, plant biology, and electrical engineering. The current packaging needs a redesign to provide a robust connection method for the increased number of sensors and also to improve the weatherproofing. Dissemination of Results:Manuscripts are in preparation for publication during the next reporting period.
Impacts
What was accomplished under these goals?
Integration of Plant Monitoring Hardware?: Our focus this past year has been on integrating the environmental sensors into a single system. In line with the aims of the project, our goal has been to provide accurate data at a fraction of the price for professional instruments. Sensors for ambient light, ambient temperature, ambient humidity, soil temperature, and soil moisture are all integrated and functioning. The light sensor costs just a few dollars, but thanks to our graduate student's developments, we can now record the photosynthetic photon flux density that matches readings from an Apogee quantum sensor, which costs hundreds of dollars. The soil moisture sensor also provides volumetric water content readings comparable to those provided by the METER EC-5 at a much lower price. The key was performing careful calibration experiments. Additional system component developments include a solar charging mechanism and a data transmission radio for long distance communication. Long term outdoor and indoor tests have shown promising performance. Initial on-plant sensors with laboratory-grade backends have provided baseline measurements, some of which were presented to colleagues and professionals at the NC State University's College of Agriculture and Life Science Stewards of the Future conference. Electrodes and Sensor Materials:Screen printed dry electrodesshowed advantagesover wet electrodesfor plant biopotential measurements.Bacterialnanocelluloseprovides an ultra-thin, vapor-permeablesubstrateupon which to fabricate the electrodes.Electrodes were characterizedfor impedance and electrochemical surface area using electrochemical impedance spectroscopy, cyclic voltammetry, and chronoamperometry.We were able to tune the electrochemical parameters via processing methods to improve recording characteristics. Automated Imaging and Data Collection Platform:Our main accomplishments for this period include: 1. The development of a platform for automatic imaging of maize in the phytotron. This platform will be used for phenotyping of the plants over time. The design is based on a mechanical rack with mobile stages for imaging several plants. The platform will carry RGB imaging sensors as well as a low-cost NDVI sensor based on the Raspberry PI camera. 2. The testing of existing tools for 3D modeling and key-point identification. Existing Structure from Motion (SfM) pipelines were tested in order to determine the feasibility of building an accurate 3D model from RGB images in order to extract accurate phenotyping information. We identified some weaknesses at capturing fine structures at the end of the leaves, which is essential for some of the geometry-based measurements. In order to exploit the finer details in the raw images, we explored the user of image-based object detectors for more accurate localization of the key-points in the plant. A hardwar framehas been designed in order to accommodate the cameras needed for imaging the plants. The design has a rail system on top of the frame in order to take pictures from the top and will have a linear motion mechanism on the side to take side pictures as the plant grows. The control software is currently being developed using Python and Node-Red. This provides remote control and monitoring of the data collection. Computer Vision for Plant Phenotyping We have carried out some exploratory data analysis, preliminary data collection, and model analysis for computer vision based phenotyping of maize plants. We have explored using transfer learning techniques to fine tune pre-trained deep learning models to detect key points relevant to the phenotype of maize plants. The models are pretrained on the COCO object detection dataset and fine tuned using a publicly available annotated data set. Preliminary results are promising. We have also explored various techniques for multi-view stereo reconstruction (MVS) to build 3D models of maize plants for phenotyping purposes. We have explored various structure from motion (SFM) techniques for extracting a dense point cloud from sequences of overlapping images taken at various positions around the plant. Figure 2 shows an example of a 3D point cloud we have extracted from pictures taken in the NCSU Phytotron. We have also done preliminary 3D scans using a Time-of-Flight (ToF) depth camera to compare results to MVS. We are working on developing code to collect raw point clouds from the ToF sensor. Lastly, we have set up our low cost NDVI imaging system which utilizes a raspberry pi with a NoIR pi camera with a blue filter added. This setup essentially replaces the red color channel with near infrared (NIR) wavelengths. This allows us to extract NDVI directly from the color images. We are currently testing code for automatically calibrating the white balance and calculating NDVI from the images. On Plant Characterization:With the first prototypes of the monitoring system coming online, we have commenced recording on plants. Current work to date has focused on characterizing soil moisture, including soil sensor testing, and characterizing vegetative maize growth within the NC State University Phytotron. Two types of soil moisture sensors were evaluated and an economical sensor was selected. Soil specific calibrations were performed and sensor coatings were tested under varying soil temperatures and moisture contents. Zea mays(maize) was grown under varying levels of nutrient stress in order to identify phenotypic expressions which may prove useful in real-time phenotyping with visible light and infrared imagery. Stem color, leaf appearance rate, leaf color, leaf chlorophyll, and stem circumference have been identified for this purpose. Nanocellulose electrode adhesion has been tested on both maize andGlycine max(soybean).
Publications
- Type:
Conference Papers and Presentations
Status:
Accepted
Year Published:
2019
Citation:
Characterization of Screen-Printed Bioimpedance Electrodes on Nanocellulose Substrate; IEEE SENSORS 2019
|
|