Source: PENNSYLVANIA STATE UNIVERSITY submitted to
DEVELOPMENT OF AN ALTERNATIVE FSMA COMPLIANT PRODUCE SAFETY CURRICULUM FOR PLAIN SECT AND OTHER SMALLER FRESH PRODUCE GROWERS
Sponsoring Institution
National Institute of Food and Agriculture
Project Status
COMPLETE
Funding Source
OTHER GRANTS
Reporting Frequency
Annual
Accession No.
1010626
Grant No.
2016-70020-25798
Cumulative Award Amt.
$163,085.00
Proposal No.
2016-07340
Multistate No.
(N/A)
Project Start Date
Sep 1, 2016
Project End Date
Aug 31, 2019
Grant Year
2016
Program Code
[A4182]- Regional FSMA Center
Project Director
LaBorde, L. F.
Recipient Organization
PENNSYLVANIA STATE UNIVERSITY
408 Old Main
UNIVERSITY PARK,PA 16802-1505
Performing Department
Food Science
Non Technical Summary
This project will develop a farm food safety learning curriculum specifically adapted for the unique learning styles of plain sect (Amish, Mennonite) growers of fresh produce. Because many will be affected by food safety new regulations under the Food Safety Modernization Act, our efforts will help maintain the safety of the foods they produce and allow them to remain competitive within the fresh produce industry.
Animal Health Component
(N/A)
Research Effort Categories
Basic
(N/A)
Applied
(N/A)
Developmental
(N/A)
Classification

Knowledge Area (KA)Subject of Investigation (SOI)Field of Science (FOS)Percent
71250103030100%
Knowledge Area
712 - Protect Food from Contamination by Pathogenic Microorganisms, Parasites, and Naturally Occurring Toxins;

Subject Of Investigation
5010 - Food;

Field Of Science
3030 - Information and communication;
Goals / Objectives
The overall goal of this project is to maintain the safety of fresh fruits and vegetable grown by plain sect growers and thus allow them to remain competitive within the fresh produce industry. We will accomplish this through the following objectives:Objective 1: Develop a modified FSMA produce safety curriculum developed for plain sectgrowers.Objective 2: Pilot the alternative curricular materials at multiple locations in PennsylvaniaObjective 3: Disseminate curricular materials to other states with plain sect produce growers?
Project Methods
We expect to conduct the following activities and deliver the following products:i. Recruitment. We have already established trusted relationships with plain sect growers and a key aspect of our project is to involve them in identifying, planning, implementing, and evaluating the materials we develop. Key to the success of our recruitment is our close working relationships already established the AFSET, produce auction and cooperative managers and board members, as well as commercial distributors who will be important resources for directly or indirectly contacting plain sect growers and helping to arrange training sessions. Over the 2 year course of this project, we expect to train growers from at least 450 farms.ii. Delivery methods. We plan on using multiple delivery methods over the course of this project. A summary of our approach is shown in the Table 1 below. In Year 1, much effort and time will be devoted to developing the alternative curriculum materials. Therefore, during this period, we will present the standard PSA curriculum to 200 plain sect growers, who are not averse to training using computer slides, in 5 separate workshops. We will use this opportunity to gather evaluation data on knowledge gained and the appropriateness of the presentation format which will inform of us as we develop the alternative curriculum materials. In Year 2, we expect that our alternative curriculum will be completed and approved by FDA and we will present it in two formats. In one group, we will present our alternative slide presentation and our printed training materials that present the information in each slide in text. We expect that 200 growers will be trained in this manner over 5 separate workshops. In the second group, no computer slides will be use; only the printed grower training manual be used to train 75 growers over 5 separate workshops. This last group will represent training conditions in which electricity is not available and where training is conducted for smaller audiences where we believe such an approach is more manageable. Each grower trained in Year 1 using the standard PSA curriculum will be provided a copy of the alternative growers manual at no cost to them.Table 1.Year12CurriculumPSA StandardAlternativeFormatComputer slides + Grower trainer manualComputer slides + Grower trainer manualGrower trainer manualWorkshops555Regs/Wkshp404010Total Registrants20020050iii. Curriculum. It is also important to note that the activities conducted in this project are a subset of all of our farm food safety outreach efforts. We will continue to conduct our regular USDA GAP Harmonized Plan Writing workshops, mock audits, and standard PSA training as needed during the course of this project The alternative curriculum we develop will be aligned with the learning objectives of the FDA approved farm food safety curriculum developed by the PSA. Materials will cover worker hygiene, sanitary facilities, water use and testing, use of manure and compost on fields, post-harvest sanitation, traceability, and recall training. Based on our experience with Amish audiences, we will explore the incorporating short case study discussion sessions after each section to increase participant involvement through sharing of problems and solutions. The take-home farm food safety booklet will reinforce the main points in the certificate training and be constructed such that it will meet the need for regular training of all food harvesters and handlers on the farm. We are including a children's activity book because it is common for the entire family to participate in the growing and harvesting of crops. The AFSET advisory group will review presentations, the curriculum book, posters, the food handler training booklet, and the course evaluation for compatibility with plain sect culture and learning styles. It is important to state that although are alternative curriculum will be developed with the cultural characteristics and learning styles of plain sect growers, we intend that the contents will fully satisfy the learning objects that FDA and PSA require for general grower audiences. Therefore we expect this curriculum to be suitable for any situation where growers can assemble in smaller groups where more personal learning can take place under the facilitation of an approved Lead Trainer. Although FDA has not mandated yearly training of supervisors, the material we will develop will serve as reinforcement materials that can be referred in other farm food safety courses and presentations well beyond the length of this project.iv. Verification. Multiple methods will be used to verify milestones and performance targets. Initial guidance from our advisory committee and regular feedback during curriculum development will assure that our program meets the needs of plain sect farmers. Important to our success will be the workshop evaluation tools we create to gather feedback from growers. We know from prior experience that pre- and post-tests measuring acquired knowledge and skills needed for GAP compliance must be written at a comprehension level readily understood by the audience (Tobin et al. 2011). Small group structured interviews immediately after workshops will complement data from written evaluations. We will also interview produce auction/cooperative managers to determine if our program has met their need for grower members to achieve FDA certification and the extent to which this has made them knowledgeable about GAPs. The PSA and or the northeast regional FSMA outreach center will verify that FDA mandated learning objectives have been met.v. Plans to communicate results to stakeholders, NIFA staff and the public. While our intent is to develop and test our FSMA aligned curricular materials in Pennsylvania, there are several other states with significant numbers of plain sect produce growers that can benefit from out program. Yearly reports will be sent to the Northeast Center for the Advancement of Food Safety (NECAF) and upon completion of the project, a presentation summarizing our findings will be made at one of the regional group's annual meetings. Dr. LaBorde is a Co-PI and executive committee member within the NECAFS is well positioned to assure direct communication with the group. After the project is completed, long term sustainability of our efforts will be assured by making hard copies of the curriculum available for a cost-recovery based purchase price, or as PDF documents downloadable from the Penn State Extension Farm Food Safety Web Site (http://extension.psu.edu/food/safety/farm).vi. Follow-up technical support. Prior experiences inform us that workshops are not by themselves sufficient to provide participants confidence and skills needed to make changes on their farms or to write a food safety plan. We have an established record of supporting smaller farmers gain knowledge about GAPs and we continue to offer GAP food safety plan writing courses to provide skills needed to comply with commercially mandated third party audits. These activities, which are outside the scope of this grant, will continue to be offered. Penn State Extension has indicated that farm food safety is a priority issue, as evidenced by the recent hiring of a Lancaster County Penn State Extension educator (the Co-PI on this grant) with specific responsibilities for pre- and postharvest food safety. His experience and skills as a horticulture and food safety educator and his position within the plain sect community as a trusted source for information ensures the long term sustainability of the efforts described in this proposal. In addition, five Penn State Horticulture Team Extension educators we have enlisted to co-teach the PSA and the alternative curricula, have successfully taken the PSA train-the-trainer course and therefore are willing and able to participate in training.

Progress 09/01/16 to 08/31/19

Outputs
Target Audience:The primary target audience for this project wereplain sect (Amish and Mennonite)growers of fresh produce. Secondarily, we are also reaching out to food safety educators who work with plain sect growers to provide them withtraining tools to help growers meet the prdoduce safety standards estbablished withn the Food Safety Modernization Act Produce Safety Rule. Changes/Problems:Early on in the project, we struggled with the lack of clear guidance or rules from PSA or FDA on what content or image changes could be made to the standardized PSA Grower Training Manual and slide set while still meeting the regulatory requirement in 112.22 that "At least one supervisor or responsible party for your farm must have successfully completed food safety training at least equivalent to that received under standardized curriculum recognized as adequate by the Food and Drug Administration." We addressed these problems by working with PSA to get informal approval of the equivalency of our slide books to the computer based ones provided by PSA. Because we assumed that demonstration of equivalency was required for alternate curricula, we had proposed in our grant request and conducted a controlled study to compare knowledge gain for each format. It was only in 2019 that guidance came out about terms related to equivalency and we learned that our approach is classified as "alternate mode of delivery" and no formal approval from PSA of FDA was needed. Nevertheless, our comparison of computer based vs slide book format effectiveness is useful because it will inform future efforts to create low technology training formats that meet regulatory requirements. We were also made aware that Version 1.0 of the grower training manual would be periodically updated. We also decided to use a 3-ring format to make the training slide book adaptable to the extent possible to changes to the grower training manual. We developed our slide books to be aligned with V1.1. Recently V1.2 has been issued. However, PSA policy is to allow continued use of V1.1 materials which will allow us to continue to use our slide book at future workshops and we expect we will reach most Amish produce growers who do not prefer computer based training before V1.1 books are out of stock. We will examine how to make changes to the current slide book during this period. What opportunities for training and professional development has the project provided?We delivered an online presentation on best practives for delivering educational materials to plain sect growers to farm food safety educators on June 9, 2017. The presentation was presented through the North East Center for the Advancement of Food Safety (NECAFS) webinar series which is archived at on best practives for delivering educational materials to plain sect growers which is archived at https://www.uvm.edu/extension/necafs/past_webinars. Wedeloped a listserv of educators with interests in Amish farm food safety training that we use to communicateprogress on the project and provide access to materials How have the results been disseminated to communities of interest?The materials are intended for educators who, in their individual states, use themfor their FSMA farm food safety workshops This information is also presented in the descriptoin of Objective 3 of the accomplishements and in the professional development section. What do you plan to do during the next reporting period to accomplish the goals? Nothing Reported

Impacts
What was accomplished under these goals? Objective 1: Develop a modified FSMA produce safety curriculum developed for plain sectgrowers. We developed three products to contribute to Amish farm food safety training. 1) A276 pageFSMA Produce Grower Training Slide Set was completed. This product is intended as an alternative delivery mode for presenting the PowerPoint slides developed by the Produce Safety Alliance. We regularly consulted with PSA and the Amish Food Safety Team (a multi-state group that develops materials and advocates for culturally appropriate training for Amish produce growers on farm food safety) to make sure we were aligned with their (and FDA's) requirements for curriculum delivery and that the materials as presented wereculturally appropriate.Although it was designed to beuseful at produce safety workshops aimed at plain sect (Amish) growers who do not prefer computer training,the books can be used in any situation where access to projectors is inconvenient. The slide book presents the same material found in the PSA PowerPoint slides, with some adaptations of images and word emphases made so they are viewable in book form. It is not intended to replace the PSA Grower Training Manual, a copy of which is still required to receive the course certificate. The slide numbers in the book correspond to those in the PSA Grower Manual V1.1, so it is easy to simultaneously follow along in the manual. The slide books are 3-ring bound on durable paper so that they may be collected at the end of the workshop and re-used at future events.The slide books are available at no cost (except postage) at extension.psu.edu/amish-farm-food-safety. PSA recently introduced a V1.2 grower manual with slight modifications to the material. PSA allowscontinued use of V1.1 which are recommended for use byslide book users. 2) We also developed "Farm Food Safety Training for Harvests and Handlers of Fresh Produce";a training toolin flip chart form aimed atsmall scale plain sect growers. The material is consistent with training and knowledge requirements for harvesters and handlers issued under the FDA Produce Safety Rule (21 CFR Part 112) as well as3rd party audits. Thespiral bound flip chart is printed on durable heavy paper stock (11X17")and includes over 35 professionally drawn images that reflect Amish growing activities in a culturally appropriate and respectfulmanner. The Amish Food Safety Team reviewed the illustrations for cultural accuracy and appropriateness. A preview of the material,a training guide, and an online order formisaccessibleat extension.psu.edu/amish-farm-food-safety. 3) An article titled "Working with Plain Sect Growers" was written to provide background information, based on our experiences during this project,on the Amish and Mennonite communitieswe worked with. It was written with the goal ofhelpingeducators with limited knowledge ofthis group to deliver training. The article is accessibleat extension.psu.edu/amish-farm-food-safety. Objective 2: Pilot the alternative curricular materials at multiple locations in Pennsylvania A study was conducted with four groups of Amish produce growers (N=94), each of whichparticipated in a 1-day farm food safety FSMA certificationworkshop. Two of the groups used the computer based Power-Point slide set created by the PSA while the other groups used the slide book developed at Penn State. Each group received a copy of the PSA Grower Training manual (V1.1) as required by PSA for certification. Training site locations were selected so they were conveniently located within a few hours travel distance to Pennsylvania Amish settlements. A PSA lead trainer and 3-4 PSA trainers presented the material. The same instructors were used at each of the two paired computer-based and slide book workshops. Pre-test and post-test questionnaires were used at each workshop location to compare knowledge of the FSMA Produce Safety Rule regulation, use of agricultural water, preventing contamination from animals, and health and hygiene practices. Demographic and sales data were also collected at each training. Overall, the respondents were predominantly male (99%), with over one-half aged 26-40 years (57%). Nearly one-half (44%) had been farming for 6-10 years, while 37% had been farming for 11-20 years, and the remainder 18.7% had been farming for 21+ years. The majority (97.7%) had received only up to an 8th grade education. There were significant (p<0.05) knowledge changes among those how viewed the slides by computer-based Power-Point slides and those who used the slide books. However, there were no significant (p>0.05) differences between pre- and post-test scores for the two training delivery modes. This indicates that training using the low technology slide book is equivalent to the power-point training in terms of knowledge gain. The results from this study demonstrate that educators can reach Amish and other audiences that, due to cultural norms, prefer low technology means of training. A more complete analysis of the data is forthcoming for publication in the Journal of Extension. Objective 3:Disseminate curricular materials to other states with plain sect produce growers An initial set of the FSMA Produce Grower Training Slide Set was mailedinthe spring of 2019 to selected educators who were in immediate need of the sets for the Amish audience scheduled workshops for FSMA Produce Rule certification. Educators were located in Ohio, Maine, Iowa, Wisconsin, Michigan, and New York. Each reported that delivery of the slides in book form in tandem with the PSA grower training manual did not introduce anytraining challenges in terms of comprehension orfollowing along with the PSA grower manual. Once final printingslide books was completed,a web site was created at extension.psu.edu/amish-farm-food-safety with instructions for online ordering. We will continue to seek feedback on the usefulness and effectiveness of our developed materials.

Publications

  • Type: Websites Status: Published Year Published: 2017 Citation: Stoltzfus, J. 2017. Working with Plain Sect Growers. Penn State Extension Article. Accessible at https://extension.psu.edu/working-with-plain-sect-growers
  • Type: Journal Articles Status: Other Year Published: 2020 Citation: Thorn K. and LaBorde L. Comparison of the Produce Safety Alliance Farm Food Safety Curriculum with an Alternative Equivalent Curriculum for Plain Sect Produce Growers. In preparation for submission to the Journal of Extension.
  • Type: Websites Status: Awaiting Publication Year Published: 2019 Citation: Farm Food Safety Curriculum Materials for Plain Produce Growers from Penn State Extension. Penn State Cooperative Extension Web Site. 2019. Available at extension.psu.edu/amish-farm-food-safety


Progress 09/01/17 to 08/31/18

Outputs
Target Audience: Nothing Reported Changes/Problems:As previously stated, to avoid potential approval hurdles for curriculum equivalance, we chose limit PSA curriculum changes in the training modality and not the text content. In consultation with the PSA, we therefore decided to maintain curriculum equivalency by including all the text content in the standardized PSA slide set. What opportunities for training and professional development has the project provided?Because the modified modalitycurriculum had been provisionally approved by PSA, the 65 Amish produce growers in attendance at the 2 pilot workshops received a certificate of completion from the Association of Food and Drug Officials (AFDO). How have the results been disseminated to communities of interest? Nothing Reported What do you plan to do during the next reporting period to accomplish the goals?We will finalize the slide text book and have sets printed and bound. Sets will be sent to educators who are conducting Amish FSMA certification trainings. We will also design and complete supplementary training materials in the form of a durable desktop flip chart that growers can use to train their food handlers as required under the Produce Safety Rule.

Impacts
What was accomplished under these goals? We have made substantial progress on the first two objectives of this project. We made a key decision during this period on how to move forward with this project in a manner that meets all regulatory requirements for training. According to the FSMA Produce Safety Rule (21 CFR Part 112.22 (c), "at least one supervisor or responsible party for your farm must have successfully completed food safety training at least equivalent to that received under standardized curriculum recognized as adequate by the Food and Drug Administration". Despite this clear directive in the rule, there has been no final guidance from FDA, PSA, or the FSMA National Coordination Center (NCC) on criteria for determining equivalency. To avoid potential approval hurdles in the future, we chose to adopt the terms posted on the NCC web site (no longer active). Our interpretation of that document was that content modifications to the PSA standardized curriculum would require approval by an as yet unknown entity, but that changes in the training modality would not. In consultation with the PSA, we therefore decided to maintain curriculum equivalency by including all the text content in the standardized PSA slide set. According to PSA, original images within thePSA slides could be deleted or altered and new images could be added as needed. We therefore obtained original higher resolution images from PSA and evaluated them for relevance to the learning objectives and picture clarity when presented in print form. Those not judged as useful to support the slide text were discarded and new images added as needed. Verbatim transfer of the PSA slide text to Publisher software and images were added. 30 draft versions of the slide text set (147-pages) were printed in color and bound in 3-ring binders. We also created a 20-question pre-/post-test to be used for a year 3 comparison of learning comprehension of the text and the computer projected slide set.We conducted 2 pilot trainings to Amish growers; 25 attended in Smicksburg, PA in January 19th, 2018 and 30 attended in Springs, PA on June 20, 2018. The pilot trainings allowed us to evaluate learners' and instructors' ease of use when presenting simultaneously the material in the slide text book and PSA grower manual and to adjust our pre/post test questions as necessary.

Publications


    Progress 09/01/16 to 08/31/17

    Outputs
    Target Audience:During Year 1 we have primarily been reaching out to the Produce Safety Alliance, the Amish Food Safety Education Team, and extension educators with interests in training Amish audiences on farm food safety. Changes/Problems:We struggled with the lack of clear guidance or rules from PSA or FDA what content or image changes could be made to the standardized PSA Grower Training Manual and slide set while still meeting the regulatory requirement in 112.22 that "At least one supervisor or responsible party for your farm must have successfully completed food safety training at least equivalent to that received under standardized curriculum recognized as adequate by the Food and Drug Administration." However we were able to resolve this issue through discussions with PSA. What opportunities for training and professional development has the project provided?Training and professional development will follow final development of the print verstion of the curriculum in Year 2 How have the results been disseminated to communities of interest?On June 9, 2017 we presented a NECAFS hosted webinar (archived athttps://www.uvm.edu/extension/necafs/past_webinars) titled "Tailoring FSMA Education to the Plain Community". We provided a summary of the literature on training Amish audiences and our approach to our project to 101 individuals either live during the webcast or by accessing the archived version. What do you plan to do during the next reporting period to accomplish the goals?In Year 2, we will pilot test the print version of the curriculum to 15-20 Amish produce growers in order to test the validity of the evaluation instrument questions, and to better understand any challenges faced by educators in presenting the written curriculum compared to the PowerPoint version that they are more likely to be comfortable with. We will then proceed with our planned experimental study to compare the print and the PowerPoint version using the evaluation questions developed by PSA in addition to our own pre- and post test questions. Once the final version is completed we will write an educators user guide based on what we have learned and will seek cooperating educators from our contact list to present the curriculum and report evaluation results back to us.

    Impacts
    What was accomplished under these goals? Progress has been made on this project. As promised for Year 1, much effort and time was devoted to understanding the finalized version of the authorized curriculum developed by the Produce Safety Alliance (PSA). We struggled with the lack of clear guidance or rules from PSA or FDA what content or image changes could be made to the standardized PSA Grower Training Manual and slide set while still meeting the regulatory requirement in 112.22 that "At least one supervisor or responsible party for your farm must have successfully completed food safety training at least equivalent to that received under standardized curriculum recognized as adequate by the Food and Drug Administration." During a conference call with the Penn State PI's, PSA, and 2 Amish members of the Food Safety Education Team, we discussed the project and recommendations going forward. PSA strongly suggested to us that we do not alter the text since the approval process remained undefined. We did learn that a modified delivery method, such as conversion of slide text into print form, would not require PSA or FDA approval as long as each participant received a copy of the PSA Growing Training Manual. There were no restrictions on adding or removing images in the slides. We therefore decided to move in this direction rather than rewrite the entire curriculum. In Year 1 we transferred the PowerPoint text to Publisher software format and systematically evaluated the usability of PSA image, paring them down to only those that were essential and with high enough resolution to be viewable in the smaller sized printed format. We have finished this transition and assembled the slide set content into booklet format for Year 2 piloting. Also in Year 1, we established a network of Extension educators with interests in delivering farm food safety programs to Amish and Mennonite produce growers. We began our have reached out to educators through presentations and discussions at the of North East Center for the Advancement of Food Safety (NECAFS)/North East State Departments of Agriculture (NASDA Annual Meeting in Boston, MA on January 13-14, 2017 and the Northeast Region Food Safety Outreach Program Project Directors Meeting in Syracuse, New York on March 16-17, 2017.To date we have collected contact information for 33 educators in 15 states in the North East, Central States, and Western States. These individuals will be kept informed of our progress and we will ask for feed back as we move into the pilot stage in Year 2.

    Publications