Source: UNIV OF WISCONSIN submitted to NRP
APPLICATIONS OF REINFORCEMENT LEARNING ALGORITHMS TO IMPROVE CROP INPUT USE
Sponsoring Institution
National Institute of Food and Agriculture
Project Status
COMPLETE
Funding Source
AFRI COMPETITIVE GRANT
Reporting Frequency
Annual
Accession No.
1018646
Grant No.
2019-67023-29418
Cumulative Award Amt.
$498,662.00
Proposal No.
2018-08550
Multistate No.
(N/A)
Project Start Date
May 1, 2019
Project End Date
Apr 30, 2023
Grant Year
2019
Program Code
[A1643]- AERC: Economic Implications and Applications of Big Data in Food and Agriculture
Recipient Organization
UNIV OF WISCONSIN
21 N PARK ST STE 6401
MADISON,WI 53715-1218
Performing Department
AG & APPLIED ECONOMICS
Non Technical Summary
The long-term project outcome is to improve the environmental and economic performance of U.S. crop production by increasing input use efficiency by integrating machine learning methods into existing precision ag and variable rate technologies. This increased input efficiency will enhance the profitability and global competitiveness of U.S. farmers and improve agricultural sustainability with less inputs such as fertilizers and pesticides lost to the environment. The project will use reinforcement learning and Bayesian optimization to optimally integrate on-farm experimentation into standard crop production practices (soil sampling, variable rate application, yield monitoring). Surveys show that >50% of farmers use field sampling and have variable rate technologies and yield monitor capabilities, but feel that they are underutilized. As farm consolidation continues and the number of publicly-funded applied researchers dwindles, it becomes critical that farmers become more active citizen scientists on the lands they manage. Based on these on-farm experiments, farm-, field- and sub-field-specific input use recommendations will be developed and annually updated to improve input use efficiency of crop farmers. The project team will develop basic software and possibly apps as decision aids, as well a variety of outreach materials to educate famers and crop consultants in the underlying logic and the practical use of these decision aids. An additional goal of the project is to develop curriculum to incorporate into new machine learning courses for agricultural economics graduate students and advanced undergraduates to meet the growing demand for these skills among students and employers.
Animal Health Component
50%
Research Effort Categories
Basic
10%
Applied
50%
Developmental
40%
Classification

Knowledge Area (KA)Subject of Investigation (SOI)Field of Science (FOS)Percent
20515993010100%
Knowledge Area
205 - Plant Management Systems;

Subject Of Investigation
1599 - Grain crops, general/other;

Field Of Science
3010 - Economics;
Goals / Objectives
Our long-term project outcome is to improve the environmental and economic performance of crop production by increasing input use efficiency, which will increase profitability and improve sustainability with less inputs such as fertilizers and pesticides lost to the environment. We propose to integrate machine learning algorithms from reinforcement learning with existing variable rate equipment and yield monitors to reduce the cost of on-farm experimentation, thus facilitating farmer adoption of on-farm experiments. Surveys show that >50% of farmers use field sampling and have VRT and yield monitor capabilities, but feel that they are underutilized. As farm consolidation continues and the number of publicly-funded applied researchers dwindles, it becomes critical that farmers become more active citizen scientists on the lands they manage. Hence, we have developed the following short-term project objectives: 1) Adapt and refine reinforcement learning algorithms to crop input management, 2) Field test algorithms and data flow processes on cooperating farms, 3) Develop and empirically apply economic models to optimize on-farm experimentation, 4) Create training materials for farmers and other professionals and agricultural data science curricula for students. We have assembled an interdisciplinary team with expertise in economics, engineering, agronomy, and horticulture. We have recruited 3 cooperating farmers to test the system on their farms and leaders of 3 associations to serve on our Advisory Board and to help disseminate findings to farmers and other agricultural professionals. This project will establish a research foundation for our outreach programs and help train the next generation in agricultural applications of machine learning.
Project Methods
Methods used will include analysis of existing data sets on field and crop conditions and weather and soil data and crop input choices and yield outcomes. Based on this analysis, recommendations for on-farm experiments will be developed and field prescription files given to cooperating growers and for university research farms. A key part of this analysis and on-farm implementation will be the development and refinement of new reinforcement learningy algorithms, with Bayesian optimization the likely method used. These efforts will be carried out by the principle investigators and graduate research assistants funded by the project. In addition, curriculum development efforts will include creation of lecture and problem sets to be integrated into a new course for graduate students on machine learning. Finally, outreach efforts will develop extension materials for farmers, crop consultants and other agricultural professionals. Again, these efforts will be carried out by the principle investigators and graduate research assistants funded by the project.

Progress 05/01/19 to 04/30/23

Outputs
Target Audience:The primary audience is crop farmers, crop consultants, and other agricultural professionals interested in improving input use efficiency using site specific and variable rate technologies. The other major audience is graduate and undergraduate students interested in learning and applying machine learning algorithms in agricultural situations. The final audience is academics, scientists and other practitioners who attend conferences and read journal papers and other academic publications to learn about and use reinforcement learning algorithms in agriculture. Changes/Problems:Due to difficulties in engaging with farmers for on-farm experiments as a result of covid and the need to get graduate students research completed for normal academic progress, we shifted to online data collection from growers more broadly and then developing machine-learning and AI approaches to using these observational data to recommend management practices and then having farmers field-validate these recommendations. This research generated journal publications and extension materials in alignment with the original project intent and allowed the students to graduate. What opportunities for training and professional development has the project provided?Graduate research assistants working on the project learned how to conduct academic research in their respective fields, including doing the analysis and writing up results for presentation at academic conferences and eventually peer-reviewed journal articles. Also, the faculty (and in some cases graduate students) made presentations to extension (non-academic) audiences, which helped train the graduate students and also was professional development for the extension audience. In addition, written work was created for outreach venues including research focused newsletters and industry magazine articles. Lastly, the PhD graduate research assistants delivered a few guest lectures on reinforcement learning in a master's level class on machine learning. This was training for both the students doing the lectures as well as the students attending the lectures. How have the results been disseminated to communities of interest?Communication with the research community was via presentations and posters at academic conferences and regional workshops, and by publication of peer-reviewed journal articles. In addition, results were disseminated to stakeholder and practitioner communities through extension presentation and written materials (fact sheets, industry magazine and newspaper articles), as well as informal discussions with practitioners after these presentations at extension meetings. For example, these included fact sheets on the CoolBean Extension web page (https://coolbean.info/), presentations at the Wisconsin Cranberry School, discussions at the Wisconsin Cranberry Research Roundtable. What do you plan to do during the next reporting period to accomplish the goals? Nothing Reported

Impacts
What was accomplished under these goals? Overall, the project contributed to advancing scholarship for applying reinforcement learning and other algorithmic to improve crop management. Among the best papers form the project were Saikai et al. (2020) and Mourtzinis et al. (2021). The first summarized a reinforcement learning algorithm that enables farmers to efficiently learn their own site-specific management through on-farm experiments using their precision agriculture equipment. Simialry, the second introduced algorithms that used existing databases of experiments across years to demonstrate how to advance agricultural research to pursue currently unrealized yield potential. The project had several other papers that made contributions along these same lines. Objective 1 focused on adapting and refining reinforcement learning algorithms to crop input management. Due to difficulties in getting data from on-farm cooperators for on-farm experiments due to covid, we have shifted to developing algorithms that utilize data farmers have shared to identify optimal input use recommendations from observational data (not experimental data). For Objective 2 (field testing algorithms) and Objective 3 (economic modeling), we shifted to evaluating these recommendations when used and the factors creating barriers for farmer use of these and related tools. These revised objectives were primarily research focused and so we published multiple peer-reviewed journal papers. This activity also involved training graduate students (Saikai, Drewry, Matcham; Gallagher, Robran) who were co-authors on several of these papers and made presentations at academic conferences as part of their dissertation or thesis research. For Objective 4 (outreach), we created extension publications and presented results at extension meetings.

Publications

  • Type: Other Status: Published Year Published: 2023 Citation: Colquhoun, J., and P.D. Mitchell. 2023. Can we use big data to add precision and make management decisions in cranberry? Cranberry Crop Management Journal 36(1):1-3: https://fruit.wisc.edu/cranberries/cranberry-crop-management-newsletters/


Progress 05/01/21 to 04/30/22

Outputs
Target Audience:The primary audience is crop farmers, crop consultants, and other agricultural professionals interested in improving input use efficiency using site specific and variable rate technologies. The other major audience is graduate and undergraduate students interested in learning and applying machine learning algorithms in agricultural situations. The final audience will be academics, scientists and other practitioners who will attend conferences and read journal papers and other academic publications to learn about and use reinforcement learning algorithms in agriculture. Changes/Problems:Due to difficulties in engaging with farmers for on-farm experiments and the need to get graduate students research completed for normal academic progress, we shifted to online data collection from growers more broadly and then developing machine-learning and AI approaches to using these observational data to recommend management practices and then having farmers field-validate these recommendations. We expect this research to continue to generate journal publications and extension materials. What opportunities for training and professional development has the project provided?Nick Gallagher (Graduate RA) presented a poster at the American Agricultural Economics Association Annual Meeting in Aug 2021. Evan Robran (Graduate RA) presented a paper at the ASA, CSSA, SSSA International Annual Meeting, Nov 7-11, 2021. How have the results been disseminated to communities of interest?Colquhoun (Co-PI) presented at the 2022 Wisconsin Cranberry School and at the 2021 Wisconsin Cranberry Research Roundtable. Conley (Co-PI) wrote an extension publication on the impact of an A.I. decision-making tool on farmer profits and made it available on the CoolBean Extension web page. Nick Gallagher (Graduate RA) presented a poster at the American Agricultural Economics Association Annual Meeting. J. Colquhoun. The future of precision weed management in cranberry. 2022 Wisconsin Cranberry School (virtual), Jan 19, 2022. Evan Robran (Graduate RA) presented a paper at the ASA, CSSA, SSSA International Annual Meeting Extension publication for farmers and crop consultants: Mourtzinis, S., J. Gaska, and S.P. Conley. 2022. Evaluating the potential of an algorithm based A.I. decision-making tool to increase farmers' profitability in Wisconsin. CoolBean Extension Webpage, Agronomy Department, University of Wisconsin, Madison, WI. https://coolbean.info/wp-content/uploads/sites/3/2022/01/2022_Soybean_AI_final.pdf What do you plan to do during the next reporting period to accomplish the goals?We plan to continue this shift to developing machine-learning and AI algorithms that use observational data shared by farmers in order to develop management recommendations and publish this work in peer-reviewed journals. This will also include continued efforts to field-validate these recommendations and to develop outreach materials to share this information with farmers, crop consultants and other agricultural professionals.

Impacts
What was accomplished under these goals? For objective 1 (adapt and refine reinforcement learning algorithms to crop input management) we continue to publish journal papers (and conference presentations) to pursue our research mission and help graduate students progress on research. Due to difficulties in getting data from on-farm cooperators for on-farm experiments due to covid, we have shifted to developing algorithms that utilize data farmers have shared to identify optimal input use recommendations from observational data (not experimental data). For objective 2 (field testing algorithms), we have shifted to evaluating these recommendations when used and the factors creating barriers for farmer use of these and related tools. For the outreach objective 4, we have created an extension publication and presented project results at extension meetings, as well as academic conferences.

Publications

  • Type: Journal Articles Status: Published Year Published: 2022 Citation: Andrade, J. F., S. Mourtzinis, J. I. Rattalino Edreira, S. P. Conley, J. M. Gaska, H. J., Kandel, L. E., Lindsay, S. Naeve, S. Nelson, M. Sigh, L. Thompson, J. E. Specht and P. Grassini. 2022. Field validation of a farmer-data approach to close soybean yield gaps in the US North Central region. Agricultural Systems Volume 200, 103434 https://doi.org/10.1016/j.agsy.2022.103434.
  • Type: Journal Articles Status: Published Year Published: 2021 Citation: Mourtzinis, S., P. D. Esker, J. E. Specht, and S. P. Conley. 2021. Advancing agricultural research using machine learning algorithms. Scientific Reports 11, 17879. doi: https://doi.org/10.1038/s41598-021-97380-7
  • Type: Journal Articles Status: Published Year Published: 2022 Citation: Matcham, E. G., F. Matias, B. D. Luck, S. P. Conley. 2022. Filtering, editing, and cropping yield maps in a R environment with the package cleanRfield. Agronomy Journal 114(3): 1672-1679. https://doi.org/10.1002/agj2.21055
  • Type: Journal Articles Status: Published Year Published: 2022 Citation: Feng, L., Z. Zhang, Y. Ma, Y. Sun, Q. Du, P. Williams, J. Drewry, and B. D. Luck. 2022. Multitask learning of alfalfa nutritive value from UAV-based hyperspectral images. IEEE Geoscience and Remote Sensing Letters 19: 1-5. https://doi.org/10.1109/LGRS.2021.3079317
  • Type: Journal Articles Status: Accepted Year Published: 2022 Citation: Drewry, J. L., J. M. Shutske, D. Trechter, and B. D. Luck. 2022. Assessment of digital capacity needs and access barriers among agricultural service providers and agricultural extension professionals. Journal of the ASABE. Accepted and In Press.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2021 Citation: Zhou Zhang, Luwei Feng, Yuchi Ma, Qingyun Du, Parker Williams, Jessica Drewry, Brian Luck. Alfalfa Nutritive Value Prediction Using UAV-Based Hyperspectral Imagery and Multi-task Learning. Journal: AGU Fall Meeting Abstracts. Volume 2021 Pages B31C-04
  • Type: Conference Papers and Presentations Status: Accepted Year Published: 2021 Citation: Fatima Amor Tenorio, Juan I Rattalino Edreira, Juan Pablo Monzon, Achim Dobermann, Armelle Gruere, Juan Martin Brihet, Sofia Gayo, Shawn P Conley, Spyridon Mourtzinis, Patricio Grassini. Protocol for Strategic Minimum Agricultural Data Collection. ASA, CSSA, SSSA International Annual Meeting, Nov 7-11, 2021, Salt Lake City, UT
  • Type: Conference Papers and Presentations Status: Accepted Year Published: 2021 Citation: Evan Robran, Spyros Mourtzinis, John M Gaska, and Shawn P. Conley. Evaluating Recommendations Derived from Artificial Intelligence Tools for Soybean and Corn. ASA, CSSA, SSSA International Annual Meeting, Nov 7-11, 2021, Salt Lake City, UT
  • Type: Conference Papers and Presentations Status: Published Year Published: 2021 Citation: Gallagher, N., P. Mitchell, M. Ruark, and K. Shelley. 2021. Composite Indicators for Incorporating Environmental Externalities into On-farm Economic Decision-Making using Farm Management Information Systems. Poster at American Agricultural Economics Association Annual Meeting, Aug 1-3, 2021, Austin, TX.


Progress 05/01/20 to 04/30/21

Outputs
Target Audience:The primary audience is crop farmers, crop consultants, and other agricultural professionals interested in improving input use efficiency using site specific and variable rate technologies. The other major audience is graduate and undergraduate students interested in learning and applying machine learning algorithms in agricultural situations. The final audience will be academics, scientists and other practitioners who will attend conferences and read journal papers and other academic publications to learn about and use reinforcement learning algorithms in agriculture. Changes/Problems:Meeting with cooperators and project co-PIs in late winter/early spring was stalled by COVID-19 and application of preliminary algorithms with farmer cooperators did not occur in 2020 or 2021. We hope to reengage with grower collaborators when college rules allow in-person meetings and engagement. We have maintained a virtual outreach presence and have continued on research in a limited fashion. Some online big data collection from soybean growers occurred with analysis in progress. What opportunities for training and professional development has the project provided?Students were unable to participate in training or professional development activities. How have the results been disseminated to communities of interest?Luck presented Kernel processing score determination with SilageSnap on the Badger Crop Connect (Virtual) in Aug 12, 2020. What do you plan to do during the next reporting period to accomplish the goals?We plan to use precision agriculture equipment to field test algorithms on university research farms and work to re-engage with farmer collaborators. Also, we will develop outreach materials for farmers and crop consultants to communicate some of the practical implications of project findings for farm use of precision agriculture in crop production.

Impacts
What was accomplished under these goals? For short-term project objective 1 (Adapt and refine reinforcement learning algorithms to crop input management) we published two additional journal papers. Field work associated with Objectives 2 and 3 stopped due to the covid pandemic and the difficulty in meeting with growers and conducting field research. Two papers were published: Saikai et al. (2020) linked a spatially explicit population genetics model for European corn borer to an agent-based model of farmer adoption of Bt corn to evaluate spatially explicit resistance mitigation policies. The model evaluated multiple resistance mitigation policies, including combinations of increased refuges for all farms, localized bans on Bt maize where resistance develops, area-wide sprays of insecticides on fields with resistance and taxes on Bt maize seed for all farms. Results support using refuges for resistance mitigation for high-dose Bt maize, just as for resistance management. Luck et al. (2020) empirically evaluated an image-processing algorithm available as a smartphone app to determine its utility for estimating corn silage processing score in-field during harvest. Importantly, the app can be used in the field during harvest to give actionable information regarding corn silage feed quality so that machine operators can make harvest equipment adjustments to improve feed quality. The alternative method takes two or more days, as it requires collecting harvest samples, sending them to a lab for analysis.

Publications

  • Type: Journal Articles Status: Published Year Published: 2020 Citation: Saikai, Y, V. Patel, P.D. Mitchell. 2020. Machine learning for optimizing complex site-specific management. Computers and Electronics in Agriculture 174: 105381 https://doi.org/10.1016/j.compag.2020.105381
  • Type: Journal Articles Status: Published Year Published: 2020 Citation: Saikai, Y, T.M. Hurley, and P.D. Mitchell. 2020. An agent?based model of insect resistance management and mitigation for Bt maize: A social science perspective. Pest Management Science 77: 273-284. https://doi.org/10.1002/ps.6016
  • Type: Journal Articles Status: Published Year Published: 2020 Citation: Brian D. Luck, Jessica L. Drewry, Randy D. Shaver, Rebeca M. Willett, Luiz F. Ferraretto. 2020. Predicting in situ dry matter disappearance of chopped and processed corn kernels using image-analysis techniques. Applied Animal Science 36(4): 480-488. https://doi.org/10.15232/aas.2020-01993
  • Type: Conference Papers and Presentations Status: Published Year Published: 2019 Citation: Saikai, Y, V. Patel, L. Gutierrez, P. Mitchell, S. Conley, J. Colquhoun, B. Luck. Adaptive Experimental Design Using Bayesian Optimization to Improve the Cost Efficiency of Small Plot Field Trials. 2019 American Society of Agronomy, Crop Science Society of America, and Soil Science Society of America International Annual Meeting, Nov 10-13, 2019, San Antonio, TX.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2020 Citation: Saikai, Y, V. Patel, P. Mitchell. Machine learning for optimizing complex site-specific management. Annual Conference of the Australasian Agricultural and Resource Economics Society (AARES), Feb 12-14, 2020, Perth, WA, Australia.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2020 Citation: Luck, B.D. 2020. Precision agriculture considerations for forage production. Global Solutions 2020 International Virtual Conference: Consistent Management from the Soil to Cattle, Dec 2-3, 2020


Progress 05/01/19 to 04/30/20

Outputs
Target Audience:The primary audience is crop farmers, crop consultants, and other agricultural professionals interested in improving input use efficiency using site specific and variable rate technologies. The other major audience is graduate and undergraduate students interested in learning and applying machine learning algorithms in agricultural situations. The final audience will be academics, scientists and other practitioners. This audience will attend conferences and read journal papers and other academic publications to learn about and use reinforcement learning algorithms in agriculture. Changes/Problems:Yuji Saikai (Graduate RA) was interviewing for positions globally, which included extensive preparation and travel, which slowed his project work. He was successful in obtaining a faculty position at the University of Melbourne in the Maths Stats Department. Meeting with cooperators and project co-PIs in late winter/early spring was stalled by COVID-19 and application of preliminary algorithms with farmer cooperators will not occur in 2020. Some non-machine learning algorithms were developed in fall/winter 2019/2020 and will be implemented on one of the research farms for corn and soybeans. What opportunities for training and professional development has the project provided?-Yuji Saikai (Graduate RA) taught 2 lectures in AAE 722: Machine Learning in Applied Economic Analysis on deep learning during the summer of 2019 with over 20 students in the course. -Brian Luck (Co-PI) presented a lecture in AAE 320 Farming Systems Management on farmers' use of precision ag in fall 2019 with 66 students in the class. -Yuji Saikai (Graduate RA) made an oral presentation at NCERA 180 meeting in August 2019 -Yuji Saikai (Graduate RA) presented a poster at the AAEA annual meeting in July 2019 How have the results been disseminated to communities of interest?-Yuji Saikai (Graduate RA) taught 2 lectures in AAE 722: Machine Learning in Applied Economic Analysis on deep learning during the summer of 2019 with over 20 students in the course. -Brian Luck (Co-PI) presented a lecture in AAE 320 Farming Systems Management on farmers' use of precision ag in fall 2019 with 66 students in the class. -Yuji Saikai (Graduate RA) made an oral presentation at NCERA 180 meeting in August 2019 -Yuji Saikai (Graduate RA) presented a poster at the AAEA annual meeting in July 2019 What do you plan to do during the next reporting period to accomplish the goals?In the next reporting period, we will develop prescription files for farmers and research stations to field test algorithms and data flow processes and empirically apply economic models to optimize on-farm experimentation.

Impacts
What was accomplished under these goals? For short-term project objective 1 (Adapt and refine reinforcement learning algorithms to crop input management), we have written two journal papers, one accepted and one under review. In the accepter paper, we developed a machine learning algorithm that enables farmers to efficiently learn their own site-specific management through on-farm experiments, testing the algorithm using a simulation model (APSIM). Results show that, relative to uniform management, site-specific management learned from 5-year experiments generates $43/ha higher profits with 25 kg/ha less nitrogen fertilizer in the first scenario and $40/ha higher profits with 55 kg/ha less nitrogen fertilizer in the second scenario. Thus, complex site-specific management can be learned efficiently and be more profitable and environmentally sustainable than uniform management. In the paper under review, we propose a novel approach, called Bayesian optimization, to adaptively designing agricultural experiments for optimization and demonstrate significant gain in the cost efficiency of small-plot field trials. Based on simulations using existing small plot data, results show that the annual difference between two estimated profits by the factorial design and our approach can be as high as $272/ha despite the fact that our approach uses less than half of the plot numbers used in the factorial design. Work proceeds under short-term project objectives 2 and 3, however, field application have stalled due to the COVID-29 pandemic and the difficulty in meeting with growers (with slow internet connectivity) and conducting field research.

Publications

  • Type: Journal Articles Status: Accepted Year Published: 2020 Citation: Saikai, Y, V. Patel, P.D. Mitchell. 2020. Machine learning for optimizing complex site-specific management. Computers and Electronics in Agriculture Forthcoming
  • Type: Journal Articles Status: Under Review Year Published: 2020 Citation: Saikai, Y, V. Patel, S.P. Conley, P.D. Mitchell. 2020. Adaptive experimental design using Bayesian optimization to improve the cost efficiency of small-plot field trials. Revise and resubmit request at PLoS One.
  • Type: Journal Articles Status: Under Review Year Published: 2020 Citation: Saikai, Y, T.M. Hurley, P.D. Mitchell. 2020. An agent-based model of insect resistance management and mitigation for Bt maize: A social science perspective. Under 1st review at Pest Management Science.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2019 Citation: Saikai, Y., and P.D. Mitchell. Bayesian optimization for precision agriculture. Poster at American Agricultural Economics Association Annual Meeting, Atlanta, GA, July 21-23, 2019.
  • Type: Conference Papers and Presentations Status: Accepted Year Published: 2019 Citation: Saikai, Y., and P.D. Mitchell. Bayesian Optimization for Precision Agriculture. NCERA 180: Precision Agriculture Technologies for Food, Fiber, and Energy Production, Madison, WI. August 13, 2019.