Progress 09/01/23 to 08/31/24

Outputs
Target Audience:The following market segments (i.e., customers and end users) were explored and actively considered through the first year of the period of performance: Primary Customer Extension Agencies: Bringing evidence-based science and modern technologies to farmers, consumers, and families to improve lives. Cooperatives: Ensuring safety and economic viability and serve as advocates for their members and have expressed interest in the tool's potential use as an educational resource. Insurers: Helping producers reduce accidents and lower insurance rates through proactive safety auditing. Professional Associations: Their reputation in the agriculture industry, as they support producers through the provision of guidance on best practices, training and operational resources, and connect producers with others in their sector. Educational Organizations: Supporting the future of agriculture through public school curricula focused on farming (not solely for future producers, but future agricultural insurers, businesspeople, veterinarians, etc.). Secondary Customer Small-to-Mid Sized Farms: Care about safety, sustainability and profitability Primary End User Agriculture Extension Agents: Need tool that assist in best-practice safety training and audits with producers. Insurance Underwriters: Need tools that assist them with site visits and voluntary safety audits to appropriately assess risks for small and mid-size producers. Underwriters have already sought such a tool, but nothing was available on the market. Agriculture Educators: Need tools that give young students firsthand experience with safety, risk, and the benefits of mitigation. Secondary End User Producers: Care about Sustainability; Profitability; Keeping family safe. Changes/Problems:Technical work for the first year of the period of performance went mostly as planned with a few minor adjustments to the planned workflow. For example, minor modifications were made to the alpha testing procedure. Alpha testing was broken into three stages, with time between each stage for iterative refinement and development to occur. The first stage was VISIMO internal testing. During this period, VISIMO employees not working on this project were tasked with following the user guide and testing a series of components to ensure functionality and identify bugs. In the next stage, VISIMO's educational and industry partners were provided with the tool. Using their subject matter expertise, they provided suggestions for changes to make Demeter easier to use. Finally, VISIMO provided key stakeholders with the application to solicit additional feedback. Between each of these stages and after the final stage, the application was updated to integrate the changes recommended by alpha testers and to ensure bugs were fixed. The primary modification to the planned alpha testing phase was delaying robust data collection. VISIMO decided to delay data collection until the beta testing stage to ensure data collectors had the most functional and usable application possible. The purpose of this is to guarantee VISIMO collects as much usable data as possible, and providing an unpolished application could be detrimental to users' trust and willingness to use the app. Because of this, training the GNN on real data must be delayed until the beta testing phase. The primary challenge that has been encountered has been the data requirements for training the deep learning models. VISIMO will conduct data collection during the beta testing stage because a more refined front end of the application will the improve quantity and quality of data that the team will be able to collect. This presents a challenge in the interim, because the models which are being developed must be trained and tested using synthetic data. Despite this challenge, VISIMO has developed training pipelines which will automate and speed up hyperparameter tuning once the data is available. This will mitigate the risk of beginning the training at a later stage of the project life cycle and will help to ensure the milestones of the project can be completed on time. What opportunities for training and professional development has the project provided?During the first year of working on this award, the VISIMO staff augmented their skills across a variety of topics. First, the data science team conducted robust investigation into Large Language Models (LLMs) and assessed the feasibility of integrating them into Demeter. Capitalizing on the newest trends and state-of-the-art advancements in LLMs has been a priority during the development of this project. Additionally, the software team has cross-trained on a variety of software best practices, to include robust unit testing and greatly enhancing the User Interface (UI) (i.e., intuitiveness) of the application. A fundamental goal on this project is establishing a user-friendly application that facilitates significant amounts of data collection in the field. Lastly, new project managers have assisted with this award by investigating customer and end-user perspectives and understand the USDA contracting process, which has allowed success on subsequent USDA-funded Phase I awards. How have the results been disseminated to communities of interest?Throughout the first year of the period of performance, the VISIMO team conducted customer and end-user discovery with a variety of stakeholders, to include those mentioned in the "Target Audience Section" (e.g., Extension Agents, Cooperatives, Insurers, and Educational Organizations). Specifically, members of participating organizations were provided Demeter credentials and access to help with external alpha testing. Their experience and candid feedback accelerated the development and allowed a robust improvement to the application prior to fielding to a broader audience. What do you plan to do during the next reporting period to accomplish the goals?VISIMO intends to begin the process of beta testing and robust data collection which will last for the remainder of the period of performance. Additionally, VISIMO intends to begin beta testing in September of 2024. This process will include data collectors from key stakeholders, and other supporting cooperatives and partners. The goals of beta testing will be to (1) collect data through observations of real activities on farms, and (2) solicit additional feedback and recommendations from a broader audience on the utility of Demeter. VISIMO will continue to investigate the integration of LLMs as an alternative method for users to enter data into Demeter. This will streamline the data entry process and will provide additional insights using historical data and general insights into safety and efficiency on a farm. Integrating an LLM as an alternative form of data entry could provide some users with a more intuitive user experience by allowing them to write out their daily activities using natural language, or even to speak it into their device using a voice-to-text feature. VISIMO plans to fully integrate state-of-the-art technology for voice-to-text capabilities in Demeter during the next year in the period of performance. This feature will enhance the productivity and overall experience of end users, particularly given diverse conditions.

Impacts
What was accomplished under these goals? VISIMO's technical work during the first half of the Phase II period of performance can be broken into three categories: (1) iteration on the PWA and alpha testing; (2) improvements to the Graph Neural Network (GNN); and (3) improvements to the formula-based risk assessment model. There were several major modifications made to the PWA, informed by feedback from external alpha testers, designed to increase flexibility and ease of use. One of these new features is the introduction of a new organizational structure to simplify the process of data collection and creation. One such structure consists of named collections of categories and hazards. Another feature designed to ease data entry is the "Template" feature, which allows input of tasks that are repeated frequently (e.g., a series of actions completed every day). Both allow for work to be planned and decrease the amount of time it takes in the field to fill out an observation. Secondly, there were two main focuses for the modification of the GNN. The first focus was on improving the GNN architecture used as the primary predictive model in the application. This additional structural information will help the network exploit the relative relationships between variables in the data collected. Furthermore, VISIMO modified the inputs to the model, so they are not limited to a specific set of categories. To do this, the team integrated a model to embed text descriptions of the variables rather than a pre-determined set of inputs. This allows the model to operate on natural language descriptions uploaded by users which will increase both the amount and quality of data that can be collected. Lastly, the mathematical risk estimation formula provides the ability to produce risk scores without the need for massive amounts of collected data. VISIMO investigated a method by which the AgInjuryNews dataset can be used to correlate observations in Demeter with the incident rate corresponding to a risk score. The key principle behind this method is that while the total amount of incidents in both AgInjuryNews and Demeter are unable to be matched, the fraction of incidents corresponding to an activity in Demeter should be related to the fraction of incidents seen in the AgInjuryNews dataset that involve that activity by a factor corresponding to the risk score of an observation and the inherent risk of an activity. The result of this research was that the expected calibration error will be less than 10% even if fewer than five actual incidents are seen in the Demeter data.

Publications