Progress 09/01/20 to 08/31/24
Outputs
Target Audience:The four modules (online and immersive) developed in this project targeted small and mid-sized farms, beginning farmers, socially disadvantaged farmers, small processors, and small produce merchant wholesalers. The soil amendment and pre and post water sampling and testing online modules were disseminated through emails to the project stakeholders (e.g., extension agents at Vtech and NCSU to share with their local farmers), through IAFP Connect, and at scientific meetings by distributing printed cards with QR codes providing access to the modules. Members of our Steering Committee, which represented the target audience for these modules, also helped disseminate the two courses. For a period of time, they were also included in the online Enhancing the Safety of Locally Grown Produce course from Virginia Tech. The modules were also posted to the NECAFS Food Safety Clearinghouse (Food Safety Clearinghouse, 2024). Because we did not collect specific information on who took the online courses, we cannot provide specific details on the individuals that completed the courses, but we estimate that, between May 2022 and May 2023, the soil amendment course was accessed 50 different times (43 English version and 7 Spanish) and the water was accessed 35 times (29 English version and 6 Spanish). In total we conducted, ten in-person training using the immersive virtual tours. They were incorporated into current in-person trainings offered by VT and NCSU extension. In Virginia, they were incorporated into two trainings for Extension Specialists and Agents, primarily to get those individuals comfortable with the technology and to demonstrate how it could be used in future trainings if they wanted to offer it in their respective counties. We focused on promoting this with Agents as a way to encourage them to invite us to deliver the immersive trainings as part of any local programs they offered. Two other trainings conducted in Virginia were with farmers market managers. This was ahead of the market season as 1) a refresher for best market practices and 2) to expose the managers to a farm environment so they could better engage with their farmer vendors and what is expected of them. In North Carolina, two sessions were offered to Family and Consumer Sciences agents on the Safe Plates for Farmers' Markets as a train the trainer session so that they could implement the virtual tours into their offerings in their counties. A similar session was also offered at the NC Extension Conference. Three trainings were delivered directly to farmers/producers and farmers/market managers. The virtual tours were also featured at a produce safety day as part of an exhibitor booth. Changes/Problems:The COVID-19 pandemic significantly impacted this project. It caused delays in getting the materials for the modules completed in time, but most importantly it changed the existing training dynamic moving forward. With COVID-19 the teleconference technology significantly improved, and quickly in person training programs adapted to be delivered via zoom or other platforms allowing farmers to attend them online, saving their time and money. With other more convenient ways to get training the need for in-person training drastically reduced post pandemic, even after the university reopened and group activities resumed. Several in person trainings that our partners scheduled and used to be well attended ended up being cancelled for lack of participants and since the immersive modules were part of the in-person training, it ended up being significantly affected. The uptake on online courses was also below what we expected. We believe that given the limited time producers and farmers have and the fact that those courses were not part of a mandatory curriculum for FSMA, there was less interest. Further, the water testing and sampling as part of FSMA was evolving and therefore there was significant uncertainty around the content, and we believe individuals might have felt that would be best to wait for more definition from FDA. What opportunities for training and professional development has the project provided?In addition to farmers and farmer's market managers, during this project our university partners trained a total of 21 Family and Consumer Sciences agents and ten extension specialists in using the VR headsets and implementing it into the courses and activities they lead. How have the results been disseminated to communities of interest?The two online courses, soil amendment and pre and post water sampling and testing, were disseminated through emails to the project stakeholders (e.g., extension agents at Vtech and NCSU to share with their local farmers), through IAFP Connect, the Southern Center, and at scientific meetings by distributing printed cards with QR codes providing access to the modules. For a period of time, they were also included in the online Enhancing the Safety of Locally Grown Produce course from Virginia Tech. The modules were also posted to the NECAFS Food Safety Clearinghouse (Food Safety Clearinghouse, 2024). Finally, members of our Steering Committee, which represented the target audience for these modules helped disseminate the two courses among their members and network. The in-person trainings offered by VT and NCSU extension that incorporated the immersive virtual tours were scheduled and disseminated by the Universities via their Extension agents. Trainings were free and participants could choose to try or not the VR headset with the tours. Extension agents were also encouraged to use the new technology as part of any local programs they offered. Each year we presented updates of our project at the Southern Center Annual Meetings and in addition we presented our work at the following meetings: 2024 IAFP Meeting: Accepted Presentation - Immersive Education: Results and Lessons Learned from Incorporating Innovative Technologies into Food Safety Education Efforts 2023 CFSEC Meeting: Accepted Presentation - Immersive Food Safety Training: Applying Lessons Learned to Develop Engaging Consumer Food Safety Education Efforts 2023 VCE Winter Conference: Accepted Professional Development Session - Virtual Reality Farmer & Farmers Market Food Safety Training Tool Demonstration And were also part of a local VT news story: https://news.vt.edu/videos/k/2022/09/1_77anwsru.html?utm_source=cmpgn_news&utm_medium=email&utm_campaign=vtUnirelNewsDailyCMP_090822-fs Lastly, we hope our paper, Innovative Technologies to Enhance Training and Stimulate Learning and Engagement in Food Safety Education for Small and Mid-Sized Farmers and Producers, will be accepted by the Food Protection Trends. All courses will remain available at the Food Safety Clearinghouse and will still be used by VT and NCSU. The headsets are also still being used and there are plans to leverage these equipment and develop new virtual tours to enrich the food safety training conduct by VT and NCSU. What do you plan to do during the next reporting period to accomplish the goals?
Nothing Reported
Impacts
What was accomplished under these goals?
We developed a series of training tools available without cost viathe NC Fresh Produce Safety page and the Food Safety Clearinghouse. - Two online training modules: A) Production and Post Harvest Water Testing and Sampling(English and Spanish) and B) Soil Amendments (English and Spanish) - the English version was peer reviewed by the Food Safety Clearinghouse ? - Twoimmersive virtual tours: A) Farmer's marketm and B) Small-diversified farm. Overall, we saw more demand for the English version of both online courses and the soil amendment course was visited more frequently than the pre- and post-harvest water course. response to the learning modules was positive. At the end of each online course there was an optional survey to collect feedback on the courses. We received 35 questionnaires for the water course (28 in English and 7 in Spanish) and 21 for the soil amendment course (14 in English and 7 in Spanish). However, only 20 (11 for water and 9 for soil amendment) users completed the questionnaire (the remaining surveys were partially completed).Overall, responses were very similar for the two courses. Most respondents were either adopting or planning to adopt some of the recommended food safety practices presented in the courses within the next 6 months. Respondents had favorable opinions about the courses and agreed that the information learned would help them improve the food safety practices on their farm. Despite the limited knowledge gain and the fact that most respondents were already familiar with the information presented in the courses and already used some of the practices, most had favorable opinions and would recommend the online courses to others. Ten in-person training sessions were conducted using the headsets. The evaluation questionnaire was distributed at the conclusion of six of the training sessions and 17 participants completed it. Based on the questionnaire responses, respondents felt more prepared to implement the food safety practices portrayed in the farmers market tour (e.g.: good handling practices and best practices when offering samples) compared with the small farm tour (e.g.: separate livestock from produce/processing areas and identify high-risk areas and areas for improvement (data not shown). Notably, for nine of the respondents, knowledge of high-risk areas and areas for improvement increased by 2 or more levels (e.g., response was "Know Some About This" before training but changed to "Expert Knowledge," after training for the measure. This measure increased by 1 level for 4 of the respondents. Seven of the respondents reported their knowledge of specific food safety practices increased after engaging with the farm tour, but knowledge gain was limited for the farmer's market tour. Respondents were satisfied with the information presented in the training sessions and the use of the virtual tours in the course. Respondents agreed that the virtual tours helped them to understand the information presented in the course and kept them engaged. The fidelity assessment was conducted in five of the 10 training sessions that included the immersive virtual tours. Of the 45 participants in these five training sessions, two participants chose not to wear the headsets and three removed their headsets because of nausea or motion sickness. In addition, a few took breaks from wearing the headsets. Figure 5 shows photographs of participants interacting with the headsets during the training. For most of the training sessions, the facilitator wore a headset to provide context. The facilitators adhered to the training protocol by helping trainees adjust the headsets to ensure proper fit and ensuring trainees had enough space to move around safely. The training protocol specified that the facilitator provide specific instructions on navigating the virtual tours (e.g., how to use the hand controller and how to take the quizzes). The training protocols for the two tours provided instructions for the facilitators to use to guide trainees through the different tours within each tour (e.g., produce handling and animal enclosures for the farm tour) and instructions to provide the answers to the tour's quizzes, if applicable, and questions to ask the trainees after completing the tour (e.g., what indicators of good practices did you see?). The facilitators generally followed these protocols for both tours suggesting high fidelity. Overall, there was a small learning curve for participants when first wearing the headsets, but most participants easily adjusted and were able to navigate the tours. Some participants initially needed assistance from the facilitator but were then able to follow along. Problems encountered included issues with the boundary (part of the headset set up), error messages, and needing to reset or switch out headsets (e.g., the headset required charging). All of the participants were engaged in the immersive virtual tours, excited about the new technology, and asked relevant questions. Following the training, the observer rated the facilitator on their knowledge, confidence, level of engagement, and their comfort with the headset technology on a 5-point Likert type scale with 1 meaning not at all and 5 meaning very much. The mean rating for each of these items was 4.7 to 5.0. Overall, response to the courses was positive. Participants reported enjoying the immersive aspects of the training and the novelty of the virtual tours. However, some experienced technical difficulties and/or issues with using the headsets themselves. Because the online courses were not part of a mandatory training, the uptake of those courses was low during the study period. This project serves as a case study into the potential use of immersive content in food safety education programs, and the challenges to be addressed when using such technologies. Our project was highlighted in the local VT news: https://news.vt.edu/videos/k/2022/09/1_77anwsru.html?utm_source=cmpgn_news&utm_medium=email&utm_campaign=vtUnirelNewsDailyCMP_090822-fs We had three requests to share our VR modules by Annie Fitzgerald from the Northeast Center to Advance Food Safety at University of Vermont, Yaohua Feng Food Safety Extension Specialist, from the Department of Food Science at Purdue University, and Andrea O'Dell, Produce Safety Programs Manager, from the West Virginia Department of Agriculture.
Publications
- Type:
Other Journal Articles
Status:
Under Review
Year Published:
2024
Citation:
Ruzante JM, Schonberger L, Chapman B, Boyer R, Viator C, Cates S, Shumaker E. (2024). Innovative Technologies to Enhance Training and Stimulate Learning and Engagement in Food Safety Education for Small and Mid-Sized Farmers and Producers. Food Protection Trends. Under consideration.
|
Progress 09/01/22 to 08/31/23
Outputs
Target Audience:? ONLINE COURSES.The two online courses on soil amendments and water testing and sampling(English and Spanish versions) have been available via the North Carolina State University (NCSU)'s fresh produce safety webpage for over a year now. The courses are also available via the Food Safety Clearinghouse and were disseminated thought the Southern Center. The English version of the soil amendments course was submitted to Peer Review via the Food Safety Clearing house and is in the process of being accepted (responses to reviewers were submitted in October 2023 and updates were made in the EN and SP courses). These courses are designed to be taken independently and werelyon ournetwork of collaborators and extension agents at NCSU and Virginia Tech to help disseminate the courses among small and mid-size producers. Since the courses were made available (May 2022), we estimated that about 60 individuals took the courses. Due to privacy concerns, we do not collect participants identify or affiliation and therefore we cannot determine exactly who took the courses and if they are part of our targeted audience. We printed and distributed 500 small business cards with QR codes for the courses ramp up our dissemination efforts. Universities and partners such as the SC are key in this effort. VIRTUAL TOURS. NCSU and Virginia Tech are experiencing a low demand for in person training in general after COVID-19. To date, NCSU has conducted three training sessions to demonstrate the virtual tours on the head-mounted display (HMD) (i.e.: Oculus Quest) as a supplement to the existing "Safe Plates for Farmers' Markets" training for Family Consumer Science (FCS) extension agents to expand their options for teaching in their counties. 17 FCS agents were trained. NCSU also presented the VR headsets with corresponding curriculum to 9 individuals who work in extension throughout North Carolina at the NCSU Annual Extension Conference. There have been three opportunities to offer the virtual tours directly to farmers. NCSU offered the virtual tours at the Carolina Farm Stewardship Conference, where 15 farmers/producers participated in the session. NCSU also partnered with an FCS agent to deliver the Safe Plates for Farmers' Markets curriculum with the virtual tours in their county, which 12 farmers/farmers' market managers attended and 10 participated in the headset activity. Fourfarmers also experienced the virtual tours during a Produce Safety Day organized by NCSU. Changes/Problems:The major problem that happened with our project was COVID-19. With the pandemic, Universities suspended all in-person training. That restarted in late spring 2021/2022 depending on the university, but the reality is that is has not gone back to normal. Universities are having a hard time recruiting participants and since those trainings are not mandatory, several times they ended up being scheduled and later canceled due to low enrollment. Another factor to consider is that the training schedules are determined by the extension agents, and they have many priorities to address in addition to food safety programming. As an example, in Virginia the VR headsets were going to be incorporated into a training program that, prior to COVID, was delivered only in-person in partnership between state specialists and local agents. During the early months of the pandemic, the course was moved online for asynchronous engagement by participants. As COVID-related restrictions eased and local agents were able to conduct in-person programs, we received feedback that participants preferred the online modality for the course. One reason for this is that agents may not have the needed number of participants in this program to justify offering it in-person when participants can instead go online to gain the knowledge. Whenever this program has been demonstrated with VCE Agents, there is interest in using it but not the needed opportunities to put it into practice. What opportunities for training and professional development has the project provided?We have engaged with extension agents from NCSU to demonstrate how the virtual modules could be used to augment their current trainings. The demand for in-person training has been slow and has not returned to what was pre-COVID and so our partners continue the work with agents to develop these opportunities. How have the results been disseminated to communities of interest??Yes. We presentedit during the annual FSOP meeting help in May 2023 and have submitted a round table proposal to IAFP entitled "Incorporating Innovative Technologies into Food Safety Education: Where Have We Been and Where Are We Headed?" where we hope to discuss the current work-to-date in incorporating VR and other novel technologies into food safety education, collective lessons learned for those who want to begin doing this, and where the panel foresees the future of VR and online food safety education. The PI and Co-PIs are organizers and panelists in the round table. What do you plan to do during the next reporting period to accomplish the goals?Inthe next reporting period, we will continue to focus on dissemination and outreach. We must ensure that the target audience is aware and can access the online trainings and that in-person trainings are being scheduled and that the VR are being incorporated. Those tasks are the responsibility of our university partners who are working diligently to expand the reach. With more frequent in-person trainings we will be able to conduct our fidelity assessments.
Impacts
What was accomplished under these goals?
? During this reporting period we revised the soil amendment course to address the comments from peer reviewers, we disseminated QR codes to target audience as part of our dissemination efforts. Our university partners have reached out to extension specialists and conducted in person training to apply the 360-degree virtual tours modules and encourage the use of the online courses. To date, NCSU has conducted three training sessions to demonstrate the virtual tours on the head-mounted display (HMD) (i.e.: Oculus Quest) as a supplement to the existing "Safe Plates for Farmers' Markets" training for Family Consumer Science (FCS) extension agents to expand their options for teaching in their counties. 17 FCS agents were trained. NCSU also presented the VR headsets with corresponding curriculum to 9 individuals who work in extension throughout North Carolina at the NCSU Annual Extension Conference. There have been three opportunities to offer the virtual tours directly to farmers. NCSU offered the virtual tours at the Carolina Farm Stewardship Conference, where 15 farmers/producers participated in the session. NCSU also partnered with an FCS agent to deliver the Safe Plates for Farmers' Markets curriculum with the virtual tours in their county, which 12 farmers/farmers' market managers attended and 10 participated in the headset activity. Fourfarmers also experienced the virtual tours during a Produce Safety Day organized by NCSU. Two fidelity assessments were conducted. Lastly, we presented results at the Annual FSOP meeting organized by the Southern Center, as well as the one help at IAFP. We presented our work at the Consumer Food Safety EducationConference in March 2023.
Publications
|
Progress 09/01/21 to 08/31/22
Outputs
Target Audience:ONLINE COURSES.In May2022, the two online courses on soil amendments and water testing and sampling(English and Spanish versions) were posted on North Carolina State University (NCSU)'s fresh produce safety webpage. The courses are also available via the Food Safety Clearinghouse and were disseminated thought the Southern Center during the annual meeting in May. These courses are designed to be taken independently and werelyon ournetwork of collaborators and extension agents at NCSU and Virginia Tech to help disseminate the courses among small and mid-size producers. Between May and September, we estimated that 20 to 30 individuals took the courses. Due to privacy concerns, we do not collect participants identify or affiliation and therefore we cannot determine exactly who took the courses and if they are part of our targeted audience. In this next period of the grant, we will ramp up our dissemination efforts. As part of that, we recently printed business cards with QR codes to help the dissemination efforts. VIRTUAL TOURS. Due to the slow return toin person training, NCSU and Virginia Tech had only a few opportunities to reach to the target audience using the virtual devices.NCSU conducted three training sessions to demonstrate and train extension agents during the reporting period. 17 extension agents working with small and mid-size producers were trained. Fourfarmers also tried the head-mounted display (HMD) (i.e.: Oculus Quest) and the tours during a Produce Safety Day organized by NCSU. Changes/Problems:The major problem that happened with our project was COVID-19. With the pandemic, Universities suspended all in-person training and restarted training late spring 2022. Since part of our project relied on the use of the VR headsets during the in-person training, we were not able to deploy those until very recently. Universities are getting back into their regular schedule for training and so we expect to be back on track. Another factor to consider is that the training schedules are determined by the extension agents, and they have several priorities to address in addition to delivering the training we have developed. What opportunities for training and professional development has the project provided?We have engaged with extension agents from NCSU to demonstrate how the virtual modules could be used to augment their current trainings. In-person training have been slowly returning to what was pre-COVID and so our partners are working with agents to develop these opportunities. How have the results been disseminated to communities of interest?Yes. We have presented our final product to our steering committee who is composed of representatives of the target audience. We also presentedit during the annual FSOP meeting heldin May in Orlando, FL. What do you plan to do during the next reporting period to accomplish the goals?Inthe next reporting period, we must focus on dissemination and outreach. We must ensure that the target audience is aware and can access the online trainings and that in-person trainings are being scheduled and that the VR are being incorporated. Those tasks will be the focus on our university partners during the next period. With more frequent in-person trainings we will be able to conduct our fidelity assessments.
Impacts
What was accomplished under these goals?
During this reporting period we finalized and made available all of the leaning modules.There are twointeractive online coursesonpre- and post-harvest water testing and sampling principles and practices and soil amendments management procedures. Those are available via the NCSU food safety extension website (https://ncfreshproducesafety.ces.ncsu.edu/online-learning-modules-modulos-de-ensenanza/)and also via the Food Safety Clearinghouse. The360-degree virtual tours of a farmers' market and a small, diversified produce operation were also completed,and the University partners have now 36 Oculus Quest headsets loaded with the two virtual tours to be used inin-person food safety trainings to enhance learning. An internet connection is not required to conduct the virtual tours. In-person trainings are slowly returning as part of the regular activities for extension agents. The online courses have been disseminated via our network of collaborators and extension agents. We printed business cards with the courses QR codes to help with the dissemination effort and are distributing it to our target audience. NSCU also held threeoutreach efforts to demonstrate and train food safety extension agents during the year. 17 extension agents working with small and mid-size producers were trained. Fourfarmers also tried the oculus containing the virtual tours during a Produce Safety Day. We alsodeveloped and tested questionnaires to evaluatethe effectiveness of both computer-based and immersive learning modules. In the end of each online course, the participant is given an option to answer a survey and for the virtual tours, since those are related to in-person trainings, participants are asked to also complete a survey and learning assessment tool that was developed by our team this year. Afidelity assessment instrument was also developed and tested to evaluate the in-person trainings and how the virtual tours are being deployed. Lastly, we attended the Annual FSOP meeting organized by the Southern Center, as well as the one help at IAFP. We are scheduled to present our work at the Consumer Food Safety EducationConference in March 2023.
Publications
|