Progress 09/01/23 to 08/31/24
Outputs
Target Audience:The target audience for this effort focuses on commcercial mushroom farmers as well as mushroom scientists and agricultural engineers Changes/Problems:One challenge faced for this project that arose in 2023/2024 is the closure of the cooperating mushroom farm located in Florida. With the closure of the farmit has been a challenge for the engineer at the UFl to work with her graduate student and test the vision sensing on-farm. We have come up with a solution in which the graduate student will visit Penn State University in 2025 to test the accuracy of her system on a shelf system at Penn State's Mushroom Research Center. What opportunities for training and professional development has the project provided?The project has provided training for 5 graduate students (1 in Information Technology, 1 in Plant Pathology and Environmental Microbiology and 3 in Agricultural and Biological Engineering during the past year. Additionally 2 post doctoral scientists have been trained on the project as well as several undergraduate students. The graduate students and postdocs have had the opportunity to present their work at both national and international scientific and grower meetings during 2024. How have the results been disseminated to communities of interest?Results from the accomplishments achieved during the past year have been presented to the scientific communitiy through an array of professional society meetings. The information has also been presented to the mushroom scientific community and US mushroom growers through both poster and oral presentationsat the joint International Society for Mushroom Science Congress/North American Mushroom Conferencethat was held in Las Vegas in Feb. 2024. The results were also presented to US mushroom growers at Penn State University's mushroom short course that was held in Kennett Square, PA October 2024 with approx. 125-130 registrants. What do you plan to do during the next reporting period to accomplish the goals?Results will be continued to be presented and published in scientific journals and at professional meetings as well as the 2025 Penn State Unviersity Mushroom Short Course. The engineers involved in the project will also put on a workshop demonstrating their prototypes with one workshop planning to take place at the University of Maryland to demonstrate use of the prototype automatic packaging robotic system(with a virtual component for growers outside of Pennsylvania). The engineers working on the robotic harvesting have agreed to put on a demonstration/workshop at University Park, PA (and probably virtually) to demonstrate the progress made with their system.
Impacts
What was accomplished under these goals?
Objective 1 - Carbon dioxide was monitored and increased by applying perforated lids to individual growing bins. Lids were equipped with a wireless IoT air temperature, relative humidity, and CO2 sensor that relayed information to a data collection software called CropSmarts. Because fungi respire CO2, the metabolically produced gas was elevated and contained in the growing systems, reaching average levels of >10,000 ppm for both CO2 treatments. The CO2 treatment lids were put on for 24 or 48 hours and initiated on day 13 of casing hold. Controls with no lid were also included. This experimental design was repeated with three replicates per treatment over three separate crops. The overall results indicated no direct effect on the yield (Kg/m2), but there was a significant reduction in the number of mushrooms for both CO2 treatments. The findings supported that the CO2 treatments did not significantly change the average mushroom mass. This implies that the alteration in CO2 in a mushroom growing environment may decrease the total amount of mushrooms but allow for an increase in the mass of the CO2-treated mushrooms. Over the last year we have continued to design, develop, and test the Cropsmarts applications to support monitoring the micro-climates in mushroom growing rooms. Specific accomplishments include the following: Ongoing refinement of the Cropsmarts sensor integration architecture to include easily adding new sensor types (e.g. compost temperature probe), and remote monitoring of sensor availability and health (signal strength and battery voltage). Design, development, and test of new spatial data visualizations to monitor environmental conditions in growing rooms by row, level, and section. Design, development, and test of new data mining capabilities including correlation and multiple regressions analyses of crop input, crop measures, and crop output variables. Integration of a new data visualization library to a) provide a broader range of data visualizations, b) enhance the aesthetics of our visualizations, and c) enable faster development of new visualizations. Design, development, and test of new user interface alternatives for data capture by growing room section in the Cropsmarts mobile application. This is a critical area to speed manual capture of growing room micro-climate data. We have also begun exploring commercialization opportunities for Cropsmarts including participation in the regional NSF I-Corps program. ?Objective 2 a.Developed a novel image processing method for mushroom detection and segmentation: Created an algorithm that integrates RGB and depth images for enhanced detection accuracy Achieved high precision (0.99) and F1-score (0.92) for mushroom detection Successfully implemented background homogenization and peak identification techniques Developed methods to handle cojoined mushrooms and irregular shapes b.Implemented and compared YOLOv8 deep learning model for mushroom segmentation: Tested different patch sizes for optimal performance Achieved best results with 128x128 patches (precision: 0.971, F1-score: 0.889) Conducted comparative analysis between traditional image processing and deep learning approaches c.Developed a maturity classification system: Created a Support Vector Machine (SVM) classifier for mushroom maturity assessment Used depth image features including mean slope, maximum diameter, and minimum diameter Achieved 95.31±4.2% accuracy in testing data for classifying mature and immature mushrooms A deep learning algorithm (YoLov5 model) was developed to identify mushrooms in growing beds, and mushroom cap size, centroid point, and pose estimation were calculated for each individual mushrooms. A decision-making algorithm was developed to assign the picking sequencing and the picking motion, such as bending direction for each individual mushroom, thus to avoid collision between mushrooms. An unmanned ground robot (Farm-ng Amiga) based harvesting system was developed for picking mushrooms at different height of growing layers. In this robotic system, a ZED camera was used for mushroom detection, a vacuum cup type of picking mechanism was used for detaching mushrooms, and a cartesian manipulator was used to position the camera and picking mechanism to the targeted mushrooms. Objective 3. In this reporting period, we present two mushroom related studies and an advanced version of our preliminary working prototype for the vision-guided in-line mushroom weight optimization machine. The first study examined mushroom till error at grocery stores for 8-, 16- and 24-ounce tills. The tills have 8.0 %, 4.4% and 2.6% error respectively. This gave us a better understanding of errors for the most commonly sold mushroom tills. The second study is a mushroom characterization study where we studied the visual characteristics of mushrooms and whether there is correlation with their weight. The study also examined whether mushrooms characterization changes after refrigerated storage for three days. On average mushrooms had 50.6±14.3 mm diameter, 22.6±7.1 cap height, 2348.7 mm2± 2472.9 area and 26.0±17.00 grams. After three days of refrigeration, mushrooms only lost an average of 1.36 grams. The current version of the developed mushroom weight optimization machine has several advanced aspects such as fully functional perception with post processing, robotic conveyor tracking and coordination, and an upgraded electron-pneumatic system. Data-driven experimentations elucidated the robot parameters necessary for mushroom manipulation. We perform a 100 till full-task experiment, where we optimize the weight of 100 tills consecutively. The experiments showed mushroom till weight optimization yielded an improvement of 87.3% at a rate of 12 tills per minute, or 0.36 tons per hour (for 16 Oz. mushroom tills). The perception task input is an array of concatenated intensity images and depth maps, followed by the application of the Mask2Former algorithm, for instance segmentation to identify all the location of the mushrooms in the till. The algorithm provides a list of mushroom instances, their locations, and classifies their orientation as either "cap-up," "sideways," or "stalk-up." The algorithm is trained on 1000 images (4657 instances) and tested on 100 images. The algorithm has 72.1 average precision and 83.97average recall. The 2D boolean masks are superimposed (matrix multiplied) on to the depth map and ranked in descending order to identify the top-most mushroom in the till. Further post processing of prediction masks is performed to ignore double predictions and determine a target for the robot. For cap-up and stalk-up mushrooms, the middle-most pixel is the target. For sideways mushrooms, the highest pixel in the height map is the determined to be the target. The perception task is executed on an Nvidia RTX 4090 GPU, and it takes 75 millisecond per till. To enhance the machine's throughput, the manipulation conveyor belt's encoder signal triggers a Conveyor Tracking Module (CTM) in the robot controller. This synchronizes conveyor movement with the robot, eliminating the need to stop the conveyor belt to add or remove mushrooms and significantly increase throughput. The electro-pneumatic system has been upgraded with larger tubing and a buffer air tank. This allows for a higher flow of on-demand, which has improved our performance in handling mushrooms. Objective 4 We deployed the Mushroom Industry Survey and are still in the process of collecting responses. A 2022 Census Overview and preliminary survey results were presented during the mushroom short course in October in PA:
Publications
- Type:
Conference Papers and Presentations
Status:
Other
Year Published:
2024
Citation:
Dutt,N and Choi, D. 2024. A computer vision system for mushroom detection and maturity estimation using depth images. ASABE International Meeting. Anaheim, CA, USA.
- Type:
Conference Papers and Presentations
Status:
Other
Year Published:
2024
Citation:
Gabel, N., Pecchia, J and Beyer, D. 2024 The influence of recycled mushroom compost casing mixtures on white button mushroom yield. Poster - International Society for Mushroom Science Congress. Las Vegas, NV. USA
- Type:
Conference Papers and Presentations
Status:
Other
Year Published:
2024
Citation:
Gabel, N., Haynes, S., Pecchia, J., Beyer, D. 2024. A description of the effects caused by temporary atmospheric alteration on Agaricus bisporus yield and development. Poster - The International Society for Mushroom Science Congress. Las Vegas, NV USA
- Type:
Other
Status:
Other
Year Published:
2024
Citation:
Delane, R and Haynes, S. 2024. Enabling "smart" mushroom agriculture. Poster- North American Mushroom Conference. Las Vegas, NV. USA
- Type:
Conference Papers and Presentations
Status:
Other
Year Published:
2024
Citation:
Mahnan, S., and He, L. 2024. Development of a decision-making algorithm to identify picking strategies for robotic mushroom harvesting. ASABE International Meeting. Anaheim, CA, USA.
- Type:
Conference Papers and Presentations
Status:
Other
Year Published:
2024
Citation:
Mahnan, S., and He, L. 2024. Development of robotic mushroom harvester identifying mushrooms and picking strategies autonomously. Northeast Agricultural and Biological Engineering Conference. State College, PA., USA
- Type:
Conference Papers and Presentations
Status:
Other
Year Published:
2024
Citation:
Pawikhum, K., and He, L. 2024. Design end-effector for automatic mushroom harvesting. ASABE International Annual Meeting. Anaheim, CA USA
- Type:
Conference Papers and Presentations
Status:
Other
Year Published:
2024
Citation:
Pawikhum, K., and He, L. 2024. Design of automatic mushroom harvesting systems. Northeast Agricultural and Biological Engineering Conference. State College, PA USA
- Type:
Other
Status:
Other
Year Published:
2024
Citation:
He., L, Mahnan, S., and Pecchia, J. 2024. Robotic solutions for button mushroom harvesting. August 13, 2024 Penn State University Ag Progress Days Legislative Tour. State College, PA USA
- Type:
Conference Papers and Presentations
Status:
Other
Year Published:
2024
Citation:
He, L., Choi, D. and Pecchia, J. 2024. Investigation of robotic solution for button mushroom harvesting. International Society for Mushroom Science Conference. Las Vegas, NV USA
|
Progress 09/01/22 to 08/31/23
Outputs
Target Audience:The target audience for this effort focuses on commercial mushroom farmers as well as mushroom scientists and agricultural engineers. Changes/Problems:
Nothing Reported
What opportunities for training and professional development has the project provided?This project has provided training to 4 graduate students (3 agricultural engineers and a plant pathologist), a research technician and several undergraduate students that have or are currently working on the project. How have the results been disseminated to communities of interest?Updates were given to US mushroom growers at the Sept. 2022 Penn State University Mushroom Short Course in Kennett Square, Pa. Engineering results were presented to the engineering community at the national ASABE conference as well as the Northeast Agricultural/Biological Engineering Conference in Guelph Canada. Results will also be presented to both US growers as well as international mushroom scientists at the upcoming International Society for Mushroom Science Congress beinging held in Las Vegas, NV in Feb. 2024. What do you plan to do during the next reporting period to accomplish the goals?Completion and analysis of the grower survey will provide needed information to the investigators regarding the potential economic impacts as well as casing use impacts on cropping systems and the ability to stagger the crop to improve harvesting speeds. The usability and sensor data integration work described above are both focused on lowering the cost of capturing crop performance data. Another aspect of our ongoing work is providing growers with tools to analyze and use this data to support farm and crop decision making. Because many farms rely on paper-based and other ad hoc approaches to time-consuming data collection and analysis, this work has significant potential to reduce information management costs. Among the features we have developed and continue to test and revise are approaches to aggregating and summarizing crop data both as tabular reports and graphical visualizations. We have designed and developed a crop compassion feature to allow growers to better understand the effects of different crop interventions (e.g. supplements) as well as the impact of pests and disease on crop performance. We continue to refine a spatial visualization that supports tracking environmental data cross the topology of a growing room. Most recently we have begun design and development of a general-purpose data mining feature to explore correlations between different crop variables, for example, between a particular compost type and crop yields, or between growing room air temperature and fly counts. We expect to invest a significant portion of our time going forward on these kind of data analytics and visualizations. As with any significant software development effort quality assurance is a primary concern. At any time at least one member of the team is responsible for testing new features and performing regression tests of existing feature that may have been impacted by new development work. Our goal is to achieve zero software defects but this requires a program of constant, time-consuming testing and debugging. During the upcoming year progress will be made with the vision and end-effector system to speed up the automatic harvesting timing as well as implementing additional data collection on the impacts of the end-effector on mushroom quality and shelf life to confirm that automatic harvesting can be accomplished without adversely affecting mushroom marketability. Automatic harvesting experiments have been conducted on small growing tubs for the first stage of development since the onset of the project. An additional goal for the remainder of the project (2023-2025)is to utlize the harvesting end-effector on a shelf system at Penn State to develop a protocol that can work in a more realisting commercial cropping system. In the future, to achieve higher throughput with the packaging system, we have planned several improvements. Firstly, we intend to design and fabricate custom checkweigher conveyor belts tailored specifically for our application. These new conveyor belts will be securely bolted to the robot frame to maintain calibration between the conveyor sections and the robot. Additionally, we will implement two weighing sections: one before the robot workspace, similar to our current prototype, and another downstream from the robot to confirm the final till weight and determine if it meets specifications. To maximize efficiency, we will elongate the manipulation section of the conveyor belt to span the robot's workspace. The camera will be mounted on this section to ensure long-term calibration with the robot. To further enhance throughput, the manipulation conveyor belt will have an encoder that triggers the camera, monitors the till's location, and activates a Conveyor Tracking Module (CTM) in the robot controller. This will synchronize conveyor movement with the robot, eliminating the need to stop the conveyor belt to add or remove mushrooms and significantly increase throughput. Once the new hardware is fabricated and implemented, we plan to quantitatively assess the system's performance metrics, including accuracy and maximum throughput. Additionally, we will evaluate the longevity of the mushroom-indexed queue without requiring conveyor belt stops for queue refills. We hypothesize that the distribution of 150 queued mushroom weights will continually span a wide range, obviating the need for external refills.
Impacts
What was accomplished under these goals?
Objective 1: Six mushroom cropping trials were conducting at Penn State Mushroom Research Center (MRC) to study how the carbon dioxide concentration and casing material affect develpoment/stagger. The technique was referred to as a "Temporary Atmospheric Alteration" (TAA). This technique consisted of a temporarily semi-closed system to generate higher concentrations of carbon dioxide produced by aerobic respiration from the developing A. bisporus mycelium and fruitbodies. Data on how the TAA impacted the staggered flush, harvest development, and yield results were collected through visual observations, Spectral imaging, and harvest results. This study was designed to determine if the inclusion of 20% or 30% recycled mushroom compost (RMC)into the casing layer affects mushroom development and yield. The data included in this research was collected from individual crops each composed of four RMC treatments and a peat moss control. Yield data was calculated from the number and weight of mushrooms from each treatment. The stagger was estimated by the number of days of 1st break harvest. Over the last year work on the Cropsmarts application suite has been focused in a number of key areas: overall usability, integration of sensor data, data analytics and data visualization, and general software quality assurance. Usability work has primarily involved analytic evaluations of different features within the application suite, and responding to feedback from commercial growers and other stakeholders external to the project team. A central finding from this work has been challenges associated with the speed and ease of use of some of the data capture features of the Cropsmarts smartphone application. Part of the challenge relates to the constrained screen size of a smartphone compared to a clipboard. In response to this we have designed and developed three different interaction modes for this type of data capture and are in the process of testing the relative efficacy of each. After some early explorations with development of low-cost, custom environmental sensor suites using hobbyist components, we concluded that this approach would not result in a sensor architecture that could perform or even survive in the relatively harsh conditions of a commercial mushroom growing room. We have subsequently begun experimenting with purpose-built, commercial-off-the-shelf (COTS) sensor technologies. We have been working with technologies sourced from National Control Devices (ncd.io) to create a more robust sensor architecture for the mushroom growing environment which has been very positive.We have provided the MRCat Penn State with a sensor architecture for automating the capture of air temperature, humidity, and carbon dioxide. This architecture has proven to be very reliable and as the unit costs for these technologies come down it has the potential to provide commercial grower with a significant labor-saving solution to capturing, storing, and analyzing crop environmental measures. In addition to the cropping and software development, the project team has developed a comprehensive survey for mushroom producers, with the aim of gaining insights into various aspects of the mushroom industry. This includes cultivation practices, the utilization of casing materials, labor requirements, and identifying potential areas for future research. Currently, the survey is undergoing pre-testing and is scheduled to be distributed at the beginning of next year (2024). Objective 2 We developed a deep learning-based algorithm for accurate mushroom detection, overcoming obstacles like occlusion, size variation, and illumination issues. The algorithm classifies mushrooms into different maturity stages, considering factors such as cap color and shape. Utilizing DenseNet with a Convolutional Neural Network, the model processes RGB images to extract color and texture features, aiding in precise mushroom segmentation. To facilitate the development of a machine vision system tailored for more modern mushroom farming methods, we have established a simulated Dutch-style mushroom growing bed within our laboratory. This initiative was necessitated by the absence of Dutch-style mushroom farms in Florida. This setup allows us to experiment and refine our machine vision technology under conditions that closely mimic those of modern farming operations. Continued working on the mushroom picking dynamic analysis, especially the mushroom cap surface friction coefficient was measured. Also, the picking force and bending angles were tested for successfully detaching mushrooms. These can be used for designing the picking end-effector. A newly designed bending mechanism with a vacuum cup was developed and tested for mushroom picking. Bending followed by twisting and lifting motion was used in the test, and over 95% success rate was achieved under the vacuum pressure of 40 kPa. A series of images were taken for mushrooms before harvesting, and machine learning based algorithms are in the development to identify the 3D pose and the surrounding spatial conditions for each individual mushrooms. Objective 3 We present a functional prototype of the complete mushroom packaging system that efficiently optimizes till weights while maintaining the correct orientation of the mushrooms. We developed a Robot Operating System (ROS)-based centralized PC controller to establish communication with six essential peripheral devices: check-weigher conveyor belt, depth camera, ABB delta robot, industrial scale for weighing mushroom queue, Beckhoff PLC for conveyor belt encoder reading and Allen Bradley PLC for controlling the electro-pneumatic system of the Robot's end-effector. The operational process begins with commanding the conveyor belt to a precise velocity and initiating motion. The conveyor belt triggers a rotary encoder to generate pulses that activate a laser triangulation camera for capturing grayscale intensity images and depth map reconstruction. Next, the till is transferred to the checkweigher section. During the weighing process, the perception task is executed on an Nvidia RTX 4090 GPU. This involves the concatenation of intensity images and depth maps, followed by the application of a Mask2Former algorithm. The computer vision system provides a list of mushroom instances, their locations, and classifies their orientation as either "button up," "sideways," or "stem up." The Beckhoff PLC continuously monitors the pulses generated by the encoder, and the conveyor is instructed to halt once the till is positioned under the ABB IRB 360 robot. We employ a weight correction strategy, whereby an underweight till is either supplemented with additional mushrooms from the queue or has one of its mushrooms replaced with a more suitable one. Similarly, an overweight till has one or more mushrooms removed and replaced with more optimal replacements. The replacement procedure involves a vision-guided robotic motion routine targeting the highest mushroom in the till. The vacuum end-effector is used to remove it, and the mushroom is placed in the continuously weighed and indexed side queue. By weighing the removed mushroom at the side queue, an optimization and search algorithm identifies the mushroom with the nearest weight to replace the removed one, bringing the till to a near-perfect weight. The location of the side queue grid cells remains constant, and the system continuously updates information regarding any mushroom removal or addition to the virtual grid index. Objective 4 The development of the survey and future analyses of the results and technologies will be used in addressing the economic impacts of the proposed technologies.
Publications
- Type:
Conference Papers and Presentations
Status:
Accepted
Year Published:
2023
Citation:
Pawikhum, K and He, L. 2023. Design of automatic mushroom harvesting systems. ASABE International Annual Meeting. July 10, 2023. Omaha, NE. (oral presentation)
- Type:
Conference Papers and Presentations
Status:
Accepted
Year Published:
2023
Citation:
Pawikhum, k and He., L. 2023. Design of end-effectors for automatic mushroom harvesting. Northeast Agricultural and Biological Engineering Conference. July 31, 2023. Guelph, Canada.
- Type:
Conference Papers and Presentations
Status:
Accepted
Year Published:
2023
Citation:
He, L, Choi, D and Pecchia, J. 2022. Development of machine vision system and picking mechanism for robotic mushroom harvesting. Sept. 2022. Penn State University Mushroom Short Course (poster). Kennett Square, PA.
- Type:
Conference Papers and Presentations
Status:
Accepted
Year Published:
2023
Citation:
Ali, M.A. and Tao, Y. 2023. Automated Mushroom in-line packaging for weight correction and cap orientation - a vision-guided robotics approach. (oral presentation). ASABE Annual International Meeting. Omaha, NE. July 10, 2023
- Type:
Conference Papers and Presentations
Status:
Accepted
Year Published:
2023
Citation:
Ali, M., and Yao, Y. 2023. Vision-guided robotic mushroom packaging and weight correction. (oral presentation - first place award). Northeast Agriculture/Biological Engineering Conference. Guelph, Ontario, Canada. July 31, 2023.
|
Progress 09/01/21 to 08/31/22
Outputs
Target Audience:US mushroom growers and suppliers as well as training undergraduate, graduate students and postdoctoral scientists. Changes/Problems:Personnel challenges have been the biggest hurdle so far. Dr. Jayson Harper (Ag Economist) announced his retirement so we recruited Dr. Claudia Schmidt to replace him on the project. Dr. Dana Choi (PSU Ag Engineer) left Penn State University and took a job at the University of Florida. Dr. Choi has agreed to continue on this project and a subcontract was setup with the University of Florida. This had delayed some of the work on objective 2. The timing of the receiving grant funds also made it difficult to recruit graduate students at the start (fall 2021) of the project. We now have most of the planned graduate students selected and hired with 2 additional students starting soon on objective 2 (one at PSU and one at the Univ of Florida). What opportunities for training and professional development has the project provided?We have a graduate student (Plant Pathology department) that was hired Jan 2022 to work on the alternative materials and climate modifications in objective 1. In the first year we have also hired agraduate and three undergraduate students to assist with the information and technology component of objective 1. All of these students are in the College of Information Sciences & Technology. Over the summer of 2022 the project also employed an undergraduate from the department of Biological and Biosystems Engineering. The graduate student on the project is an experienced software engineer and is acting as the technical project lead with the co-PI (Haynes) acting as project manager and overall system designer. Part of the graduate student's role involves mentoring the four undergraduate students. Responsibilities of the undergraduate students included web and mobile software development, and software/hardware quality assurance (testing). Originally, the project has one graduate student for Task 2a. However, due to co-PI Choi's relocation, student hire has been delayed. During recruiting a student, an engineer has been working on the project and has been given professional development experiences for developing image processing and machine learning applications We have a graduate student for Task 2b, while he could not come due to the visa issue. Now we have another student working on the project partially. In Spring 2023, we will have a new graduate student working on the project. For objective 3, this project supported students' training in several technical avenues and educational levels for the packaging component of the project. K12, undergraduate and graduate students worked on hardware-related projects that utilize Computer-Aided Design and metal machining skills to fabricate parts for mounting line-scan cameras to the check-weight conveyor belt.Students are also trained on software control schemes to develop communication protocols between computers and other peripheral devices. Students had the opportunity to learn deep learning tools such as Pytorch and TensorFlow. One high school student took science credits for this project for one semester in HS senior year and learned to CAD using SolidWorks. The PhD students on the project are making excellent progresses in the research and development toward the innovative technologies. How have the results been disseminated to communities of interest?Faculty, students and postdocts presented updates of their work on this project at the Penn State University Mushroom Short course that was held Sept. 25-27, 2022 at the Mendenhall Inn in Chester County. Information was presented as scientific posters or as an oral presentation. The audience consisted primarily of mushroom industry personnel (growers and suppliers) from throughout the US as well as a few from Canada and Europe. Additionally, updates on objecti 3 were presented through a conference in Edgewood, Maryland. PhD student Mohamed Ali presented the research results in the 2022 Northeast Agricultural and Biological Engineering Conference (BABEC) and won the third prize for NABEC Graduate Students Oral Competition. The audience included extension faculty, graduate students, communities of multi-state regions. What do you plan to do during the next reporting period to accomplish the goals?Objective 1. The camera setup is currently in construction and will be functional in the near future. The master's student on this project has been working on the camera software and with other professors to optimize the methodology for data collection and processing. The mushrooms' maturity will be determined by shape and size the measurements of these traits will be extracted from the images using a previously designed computer vision algorithm. The data from these images will provide important values to quantify and classify a staggered pin set. The economic survey is important to this project to develop an overall examination of current practices in the mushroom growing industry and their impact. This includes practices such as "staggered pinning" and alternative casing materials. The survey will be developed by working with Claudia Schmidt (PSU Ag Economics) who is now a collaborator on this project (in place of Jayson Harper who is retiring).These preliminary results with the sesnor data and analytics suggest that these existing features provide a good basis for users to interrogate the sensor data but further user-centered research in year 2 and onward will likely generate a range of new requirements for this data to support decision-making for farm and crop operations. Work planned for this second year of the project will include more structured and rigorous testing of the architecture. The major disadvantage of this second sensor suite is cost. Each of the three-way sensors is about $350 versus $140 or so for the raspberry pi based solution. In addition, this architecture requires the internet gateway device which is about $200, though one gateway can serve at least 10 different sensors spread throughout a space (e.g. a mushroom growing room or farm). We are working on improving the sensing accuracy and picking mechanism with higher speed actuator and softer vacuum cup material. More tests will be done in the spring 2023. A mushroom growing shelf was established in the lab, and the picking mechanism will be integrated to the shelf in the near future. For the automated packaging process, we intend to focus on the perception task to automatically detect individual mushrooms and respective position/pose. A dataset for manually labeled mushrooms will be prepared and deep learning algorithms will be developed by utilizing Detectron2 package to train segmentation tasks such as Mask RCNN, DeTr and MaskFormer.
Impacts
What was accomplished under these goals?
Task 1-1 - Quantifying Stagger using computer vision technology - A computer/camera imaging system was aquired and the framework was designed to capture images. Task 1-2 - Test different casing materials to determine effects on mushroom stagger.A total of seven crop trials were conducted so far to screen various alternative casing materials. The results of these crop trials indicated that Recycled Mushroom Compost (RMC) showed promise as an alternative to peat moss and warrant further research. Task 1-3 - Determine different bulk densities of tested materials. Bulk densities are still being determine for different casing materials being tested. Task 1-4 -Measure and quantify exclusion zones by capturing images of the colonized casing layer and implementing machine learning techniques. To be done Task 1-5 - Measure the microtopography of the casing layer to give insight on the best physical features contribute to an optimum staggered pin set To be done Task 1-6 - Use micro sensors to monitor and control the micro-climate around developing mushrooms to better understand the role of microblimate on pin set and mushroom development. Use of CO2/temperature microarry system tested and currently increasing capacity of system. Task 1-7-Yield and harvest rates will be compared to determine what impact changes have on production and harvesting efficiencies. In cooperation with several commercial mushroom farms we will incorporated the results of our proposed research and compare harvesting rates and yields with crops not using these procedures. Yields analyzed for 7 different casing materials. Harvesting rates and effects of changing microclimate to be done. Objective 2 - Development of a robotic harvesting system of mushrooms for the fresh market. Task 2a:Development of machine vision system for mushroom detection PI moved to Univ of Florida.A system was designed to adopt a motorized camera slider to capture images of the multiple mushroom beds automatically to increase the field of view of the imaging system Task 2a-1: Developing an image platform for three-dimensional scanning with extended field of view for capturing smaller objects.Hardware development and image acquisition was developed to capture and register multiple point cloud images from various point of views of mushrooms. Task 2a-2: Developing mushroom detection algorithms.Mushroom detection algorithms using RGB-D (generated from the high resolution point cloud images) and color images were compared by using three different mushroom detection methods to test if the RGB-D images from the high resolution system can outperform other types of images Task 2a-3: Creating a mushroom growing stage map based on point cloud images. To be done. Task 2b.Development of robotic mushroom harvesting mechanism. To be completed. Task 2b-1 -Investigate the mushroom removal dynamics with harvesting motions, include required force and spatial.Working on the mushroom picking dynamic analysis. We have tested three different mushroom removal methods, including direct pull, pull and twist, and bend and twist, and the preliminary results showed that the bend with twist motion is the best for mushroom harvesting Task 2b-2: Mushroom removal mechanism developmentA bending mechanism with a vacuum cup was developed in the lab, and has tested with mushroom picking. It picked about 90% of mushrooms, while the quality of the harvested mushroom could be a concern at this moment, and the speed of harvesting is another issue. We are working on improving the picking mechanism with higher speed actuator and softer vacuum cup material. More tests will be done in the spring 2023. Task 2c:Decision making and path planning for selectivemushroom harvesting. To be completed. Task 2d:Integration of the robotic system and evaluation. To be completed. Objective 3 -To develop vision-guided automated on-linesmart packingtechnology embedded with AI to eliminate try-and-error manual handling of mushrooms for weight correction. To attain face-up packing quality, gentle handling via digital precision, and yet 10-fold greater labor efficiency and productivity. Task 3-1 Development of automated weighing and 3D vision and systems We integrated a central computer with 1) a check-weight conveyor, 2) line scanning depth camera, 3) a high speed delta robot equipped with a suction end-effector. Our systems consist of three sections: an infeed, weighing, and an outfeed section. Task 3-2 Mushroom Object Recognition by Deep Learning.The goal of this step is to identify the object to be picked by the. To be completed. Task 3-3 Development of vision-guided mushroom picking (selecting) once the weight and position of mushrooms are known from the preceding steps, the robotic pick action can take place. To be completed. Task 3-4On-line picking strategy selected and developed to finalize product weight. To be completed. Task 3-5 Online Tests and Evaluation.vThe vision-guided packing system will be tested both in-lab and at out-reach workshops for live demonstrations. To be completed. Objective 4 -To assess the economic impacts of the proposed technologies. To be completed.
Publications
|
|