Progress 09/01/18 to 08/31/22

Outputs
Target Audience:The GREW Project aimed to serve the staff members of beginning farmer and rancher (BFR) training and support organizations by helping them to better evaluate their programs. Our project announcements were sent to a variety of beginning farmer service organizations through partner and collaborating organizations' mailing lists and listservs (including listservs focusing on apprenticeship and incubator programs), and the BFRDP project mailing list. In order to ensure that all those who serve beginning farmers were contacted, we also sent announcements to groups with a broader focus who might also benefit from our project. These included the Sustainable Agriculture Education Association and the Comfood listservs. Responses to our post-session surveys helped identify who the project served. Overall, respondents from the different sessions were from non-profit organizations (61%), with some from cooperative extension (15%), 1862 Land Grant Universities (10%), four-year colleges or universities (7%), and the rest were from other organization types (11%). These organizations serve a broad diversity of audiences. On average, these organizations reported serving aspiring farmers (54%), start up farmers (60%), more established beginning farmers (43%), small farmers (57%), female farmers (55%), socially disadvantaged farmers (45%), limited resource farmers (37%), organic/sustainable growers (43%), veteran farmer or ranchers (20%), refugee farmers (18%), youth or school groups (11%), farm workers (7%), and a wide variety of farmers and ranchers (28%). With 13% indicating "other" as their audience served. Over the entire project term, from September 1, 2018 to August 31, 2022, the project served over 222 unduplicated individuals (not all sessions could be assessed) who attended 7 webinars and 18 learning community sessions (there were 577 including duplicates). Additionally, many more have accessed our online materials. From September 1, 2018 to July 15, 2022 we have had 3,010 website users and 10,414 pageviews on the GREW Project webpage. We have surpassed the project's goal of serving 400 individuals. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided? Webinar: " Evaluation 101." This webinar helped orient or remind participants of the basic steps of evaluation, and help them identify needed resources to support their evaluation efforts. This webinar was asynchronous and It was posted on August 1, 2019. There were 422 views on YouTube. https://www.youtube.com/watch?v=FnwkHEjxuYI Learning Community - "Building Capacity for Program Evaluation: A Critical Conversation about the Meaning of Farmer 'Success'." Conducted at the National Farm Viability Conference on October 23, 2019. 60 people in attendance. Learning Community: "Evaluation-Stories from the Field-Learning from Fellow Beginning Farmer Training Organizations." This learning community helped inspire participants with ideas for enhancing the usefulness of evaluation by way of stories from other beginning farmer and rancher programs. It was held on September 18, 2019 and we do not have the record of the participant number. Webinar: "The Development and Use of Outcomes to Evaluate Beginning Farmer and Rancher Training Programs." This webinar provided assistance with developing, tracking, and using outcomes. It was held on February 3, 2020 and had 80 people in attendance. 106 views on YouTube. https://agroecology.ucsc.edu/education/bfrdp/webinars.html Webinar: "Alternative Outcome Development & Data Collection Strategies: Most Significant Change (MSC) and Photovoice (PV)." This webinar explored two participatory and qualitative evaluation methods. It was held February 25, 2020 and 53 people were in attendance. 264 views on YouTube. https://agroecology.ucsc.edu/education/bfrdp/webinars.html Learning Community:"Reflection in a Time of Rapid Response: How to Better Pivot Beginning Farmer and Rancher Programming in Times of COVID-19." The goal of this learning community session was for participants to experience evaluative thinking so they feel inspired and able to implement this process at their organizations. It was held on May 7, 2020 and 55 people were in attendance. https://agroecology.ucsc.edu/education/bfrdp/learning-community.html Webinar: "Documenting Past Participant Outcomes: Beginning Farmer and Rancher Training Organizations Share Their Experience." The purpose of this webinar was to showcase how two different beginning farmer training organizations have conducted follow-up evaluations, to provide examples, tips and inspiration to others. It was held on June 25, 2020 and 32 people attended. 70 views on YouTube. https://agroecology.ucsc.edu/education/bfrdp/webinars.html Learning Community: "Identifying Beginning Farmer and Rancher Needs in Times of Change: Exploring an Appreciative Perspective for Responsive Programming." This session provided participants with examples and reflective questions for doing future needs assessments at their organizations. It was held on July 16, 2020 and there were 14 people in attendance. https://agroecology.ucsc.edu/education/bfrdp/learning-community.html Learning Community: "Farmer Training Program Planning and Evaluation: GREW Resources and Applied Lesson." This session, conducted as part of the 2020 National Field School, provided participants with an overview of the tools developed by the GREW project, and examples of how two different organizations used and adapted them to assist with the planning and evaluation efforts. It was held on November 6, 2020. There were 53 people in attendance. 123 views on YouTube. https://agroecology.ucsc.edu/education/bfrdp/learning-community.html Webinar: "Self-Assessment for Evaluation and Enhanced Learning: using and Adapting the Business Health Assessment and the Refugee & Immigrant Participatory Benchmarking Tool." This webinar offered participants a glimpse into two different assessment instruments that can be used and modified for their organizations, presented by two beginning farmer program leads. It was held on December 17, 2020. There were 28 people in attendance. 115 views on YouTube. https://agroecology.ucsc.edu/education/bfrdp/webinars.html Webinar: "Developing Your Beginning Farmer and Rancher Development Project Evaluation." The GREW Project co-hosted a webinar for the BFRDP Program that offered insight for how participants can develop evaluation plans for their BFRDP proposal. It was held February 4, 2021. There were 65 people in attendance. https://agroecology.ucsc.edu/education/bfrdp/webinars.html Learning Community: "Evaluation Co-Working Series: Session 1." This session, part of a 4 part series, provided participants with accountability, a designated time, and evaluator support for them to make progress on their evaluation tasks and goals. It was held on April 20, 2021. There were 7 people in attendance. https://casfs.ucsc.edu/education/bfrdp/learning-community.html Learning Community: "Evaluation Co-Working Series: Session 2." This session, part of a 4 part series, provided participants with accountability, a designated time, and evaluator support for them to make progress on their evaluation tasks and goals. It was held on May 4, 2021. There were 9 people in attendance. https://casfs.ucsc.edu/education/bfrdp/learning-community.html Learning Community: "Evaluation Co-Working Series: Session 3." This session, part of a 4 part series, provided participants with accountability, a designated time, and evaluator support for them to make progress on their evaluation tasks and goals. It was held on May 18, 2021. There were 6 people in attendance. https://casfs.ucsc.edu/education/bfrdp/learning-community.html Learning Community: "Evaluation Co-Working Series: Session 4." This session, part of a 4 part series, provided participants with accountability, a designated time, and evaluator support for them to make progress on their evaluation tasks and goals. It was held on June 1, 2021. There were 3 people in attendance. https://casfs.ucsc.edu/education/bfrdp/learning-community.html Mentoring: Learning Community participants were offered an opportunity to schedule time with an evaluator for feedback and support with their organization's evaluation. Five people took advantage of these sessions. Webinar: "Are they still farming? Insights into Conducting Long-Term Evaluation of a Beginning Farmer Training Program." This webinar shared the process by which New Entry Sustainable Farming Project conducted an evaluation of its incubator program going back 15 years. It was held on September 2, 2021. There were 40 people in attendance. 93 views on YouTube https://agroecology.ucsc.edu/education/bfrdp/webinars.html Learning Community: "Culturally Responsive, Equitable, and Transformative Evaluation (CREaTE) Series." This series helped beginning farmer and rancher training and support organizations build their capacity to do and use evaluation to measure progress toward their goals and leverage efforts to shift the needle towards equity and systems change. It included 8 sessions held from October 2021 to June 2022. There were 31 participants of the series. How have the results been disseminated to communities of interest?Announcements for webinars, learning community sessions and completed tools have been sent to the following locations: GREW Project listserv, BFRDP Projects listserv (through UMN), Virginia Beginning Farmer and Rancher Coalition general listserv, Virginia Beginning Farmer and Rancher Coalition Member listserv, Sustainable Agriculture Education Association listserv, Southern Region Extension and Research System (SERA-47) listserv, NIFTI Incubator Farms listserv, AgALN listserv, New Entry BFRDP TA listserv, New Entry General Newsletter, NIFTI/AgALN istserv, CFP Technical Assistance listserv and CASFS social media. What do you plan to do during the next reporting period to accomplish the goals? Nothing Reported

Impacts
What was accomplished under these goals? The long-term goal of this project was to create strong, effective, and long-lasting farmer and rancher training programs to ensure that beginning farmers and ranchers (BFRs) can enter the field and establish successful farming businesses. This educational enhancement team project aimed to meet this goal by equipping BFR training organizations nationally with tools and skills to effectively conduct evaluations that clearly document program impact, improve programming, and enable program staff to identify and respond to farmer needs. Such documentation and program improvements aimed to increase participants' success and provide the accountability needed for the long-term viability of BFR programs. The intended outcomes of this project were that BFR organizations will find evaluation more useful and will more effectively design and conduct evaluations. We accomplished the following for each project objective: Objective 1) Identify and collect existing tools and best practices for program evaluation. A needs assessment survey was developed and implemented in January 2019. The goal of the survey was to identify the learning needs of the people this project plans to serve. There were 135 people who responded to the survey. The survey was used to identify gaps and refine planned programming. Objective 2) Identify gaps in evaluation outcome measures. Project partners and members of the advisory team confirmed that important gaps in outcome measures revolve around tracking progress of aspiring farmers, measuring farmer success and identifying long-term outcomes or different system level outcomes. It was also confirmed that identifying example outcomes for all types of BFR programming would be useful. Objective 3) Create tools for efficient evaluation development. An outcomes list document was created that highlights example outcomes, indicators, and data collection questions that BFR organizations can use to inform development of their evaluations. We also created 5 outcome diagrams that illustrate ways to help BFR staff think about outcomes that would be most useful to their organizations. There were 720 total views to our Tools webpage (which houses the outcomes list and other diagrams) from September 1, 2018 to July 15, 2022. We also created 6 resource tip sheets housed on the GREW website that provided guidance on evaluation and compiled and described key GREW resources and more. Combined the resource tip sheets had 651 views from January 1, 2021 to July 15, 2022 (these began to be posted in Jan 2021). Objective 4) Build capacity of BFR staff through training. One goal of the project was to develop webinars to help build staff capacity to conduct program evaluation. Seven webinars were conducted over the course of the project. There were over 298 duplicated participants for all sessions and 66 of them responded to post-session surveys (no data for one webinar). Overall, the survey respondents at least somewhat agreed they gained useful knowledge (95%), felt inspired to begin or continue evaluation work for their organization (89%), and feel evaluation work appears more doable (79%). Most also indicated they know where to go for more resources and information regarding doing evaluation work (88%) and are at least somewhat likely to make additions, changes, or improvements to their program's evaluation efforts (76%). The webinar recordings are housed on the GREW website and have a combined total of 1551 web hits. Objective 5) Run an online evaluation learning community. The learning community sessions provide an avenue for BFR staff to discuss best practices and explore evaluation issues, and their solutions, with peers and evaluators. Seventeen sessions were conducted over the course of the project; 5 were one-time events, 4 were part of co-working series, and finally the Culturally Responsive, Equitable and Transformative Evaluation (CREaTE) series included 8 sessions. Over 222 duplicated participants attended one of the five one-time sessions and 38 responded to the post event survey. Respondents at least somewhat agreed they felt inspired to begin or continue evaluation work for their organization (82%). Most also noted they are likely or very likely to make additions, changes, or improvements to their program's evaluation efforts (74%). One learning community was recorded and posted to the website with 123 webviews. The co-working series provided space for participants to work on their evaluation tasks and included four 1.5 hour sessions with 26 duplicated participants with 5 participants responding to the post series survey. Individual mentoring was also offered to 5 interested participants. The respondents reported they found the co-working series to be helpful or very helpful to them (100%) and that they were all at least somewhat likely to make additions, changes, and improvements to their evaluation efforts due to the series (100%). The CREaTE series aimed to help beginning farmer and rancher training and support organizations to build their capacity to do and use evaluation to measure progress toward their goals and leverage efforts to shift the needle towards equity and systems change. Thirty-one people participated for the 8 session series and 16 responded to the post series survey. Participants indicated they at least somewhat agreed they gained useful knowledge (100%), felt inspired to begin or continue evaluation work (100%), felt evaluation work appears more doable (75%), found a sense of support (81%), and know where to go for more resources and information on evaluation work (88%). Objective 6) Develop web-based repository for, and conduct broad dissemination of, project products. A website for the project was developed and maintained on the lead organization's website. From September 1, 2018 to July 15, 2022 we have had 3,010 website users and 10,414 pageviews on the GREW Project webpage. An online Evaluation Resource Library was also updated and maintained during the project. This library is hosted on a project partner site. Overall, there were 977 users and 1,551 site visits between September 1, 2018 and July 15, 2022. The project also maintains a listserv to keep people informed about the project's educational activities, events and materials. As of 8/1/22, there were 228 people on the GREW listserv. Evaluation. To track the medium-term outcomes of the project, a follow-up survey was conducted at the end of year four. All participants who attended at least one session, or were on the GREW listserv, were contacted (364 people). The survey had a 15% response rate (53 individuals). Of these respondents, 49% reported that they used something they learned from engaging with the GREW project sessions or resources. Of those who utilized something or answered the question (22 individuals), 81% reported some kind of impact. The main impacts reported include that their evaluation plan is more realistic, clear or feasible (45%); their grant proposal evaluation plan is stronger (41%); their outcomes are more realistic and relevant (32%) and their data collection is more useful for staff/participants (27%). There were 45% who said it is too soon to tell impact. Regarding the project's proposed metrics, we met three: 1) more than 400 people accessed our materials 2) more than 200 people participated in training and learning community sessions (unduplicated), and 3) 88% reported they at least somewhat agree they were inspired make changes or improvements to their program's evaluation efforts. While we did not meet two other targets (that 75% would report doing something, and 50% would report making programming changes), we still made a significant impact. As would be expected, our intensive course, the CREaTE series, reported much higher outcomes, with 88% having implemented something they learned. Additionally, as participants noted, it is still too soon to identify the full extent of impacts.

Publications

• Type: Conference Papers and Presentations Status: Other Year Published: 2018 Citation: Perez, Jan. 2018. Evaluation Support and Technical Assistance EET for Beginning Farmer and Rancher Programs. Presented at the Beginning Farmer and Rancher Development Program Directors Meeting, Albuquerque, NM. September 27, 2018.
• Type: Other Status: Published Year Published: 2020 Citation: Perez, Jan, Jennifer Hashley, Kim Niewolny, Damian Parr, and Allyssa Mark. 2020. Outcomes for Aspiring Farmers and Ranchers. University of California Santa Cruz, Center for Agroecology & Sustainable Food Systems. Santa Cruz, CA. https://casfs.ucsc.edu/education/bfrdp/outcomes-aspiring-farmers-ranchers.pdf
• Type: Other Status: Published Year Published: 2020 Citation: Perez, Jan, Kim Niewolny, Jennifer Hashley, Damian Parr, and Allyssa Mark. 2020. Identifying Farmer and Rancher Success. University of CaliforniaSanta Cruz, Center for Agroecology & Sustainable Food Systems. Santa Cruz, CA. https://casfs.ucsc.edu/education/bfrdp/identifying-farmer-rancher-success.pdf
• Type: Other Status: Published Year Published: 2020 Citation: Perez, Jan, Damian Parr, Kim Niewolny, Jennifer Hashley, and Allyssa Mark. 2020. Long Term and System Level Outcome Examples. University of CaliforniaSanta Cruz, Center for Agroecology & Sustainable Food Systems. Santa Cruz, CA. https://casfs.ucsc.edu/education/bfrdp/long-term-system-level-outcomes.pdf
• Type: Other Status: Published Year Published: 2020 Citation: Perez, Jan, Jennifer Hashley, Kim Niewolny, Damian Parr, and Allyssa Mark. 2020. Example Pathway of a 9-Month Training Program. University of CaliforniaSanta Cruz, Center for Agroecology & Sustainable Food Systems. Santa Cruz, CA. https://casfs.ucsc.edu/education/bfrdp/example-pathway.pdf
• Type: Conference Papers and Presentations Status: Other Year Published: 2018 Citation: Search Citation Publication Type Year Perez, Jan. 2018. Evaluation Support and Technical Assistance EET for Beginning Farmer and Rancher Programs. Presented at the Beginning Farmer and Rancher Development Program Directors Meeting, Albuquerque, NM. September 27, 2018. Conference Papers and Presentations 2018 Hashley, Jennifer & Lindsay Davies. 2018. Beginning Farmer Pathway Analysis. Presented at the New Entry Sustainable Farming Project, 2018 NIFTI (National Incubator Farm Training Initiative) Field School, Boulder, CO. November 7, 2018.
• Type: Conference Papers and Presentations Status: Other Year Published: 2018 Citation: Perez, Jan. 2018. Evaluation Support and Technical Assistance EET for Beginning Farmer and Rancher Programs. Poster displayed at the Beginning Farmer and Rancher Development Program Directors Meeting, Albuquerque, NM. September 24-27, 2018.
• Type: Other Status: Published Year Published: 2020 Citation: Perez, Jan, Allyssa Mark, Pam Mavrolas, Kim Niewolny; Jennifer Hashley and Damian Parr. 2020. Identifying Outcomes for Program Evaluation: Example Outcomes for Beginning Farmer and Rancher Training and Support Efforts. University of CaliforniaSanta Cruz, Center for Agroecology & Sustainable Food Systems. Santa Cruz, CA. https://casfs.ucsc.edu/education/bfrdp/evaluation-outcomes-list.pdf

Progress 09/01/20 to 08/31/21

Outputs
Target Audience:The GREW Project aims to serve the staff members of beginning farmer and rancher (BFR) training and support organizations by helping them to better evaluate their programs. Our project announcements were sent to a variety of beginning farmer service organizations through partner and collaborating organizations' mailing lists and listservs (including listservs focusing on apprenticeship and incubator programs), and the BFRDP project mailing list. In order to ensure that all those who serve beginning farmers were contacted, we also sent announcements to groups with a broader focus who might also benefit from our project. These included the Sustainable Agriculture Education Association and the Comfood listservs. Responses to our post-session surveys helped identify who the project served. Overall, respondents from the different sessions were from non-profit organizations (75%), cooperative extension (17%), and other organization types (1%). These organizations serve a broad diversity of audiences. On average, these organizations reported serving aspiring farmers (58%), start up farmers (17%), organic/sustainable growers (8%), limited resource farmers and ranchers (8%), and a wide variety of farmers and ranchers (8%). During the third year, as of August 31, 2021, the project served at least 187 unduplicated individuals who attended three webinars and five learning community sessions (there were 211 including duplicates). Additionally, many more have accessed our online materials. There were 871 users of the GREW webpage from September 1, 2020 to August 30, 2021, and 1,210 total pageviews in that same time period. We have surpassed the project's goal of serving 400 individuals. Changes/Problems:While the project had encountered a number of challenges in the second year due to COVID-19, we were still able to meet our overall targets in the third year (a total of eight sessions between the webinars and learning community sessions). However, as with most other organizations, we changed our activities slightly to better meet the needs of the moment. Instead of holding learning community sessions focused on specific discussion topics, we changed the format to a co-working series. Due to the extent that many individuals spent working online, and the proliferation of professional development offerings, webinar attendance was declining. We chose to focus more specifically on helping BFR program staff make progress in their evaluation work with this series, by offering structured working time, and then the opportunity to discuss evaluation issues and questions they had with their peers and project evaluators. Additionally, we aimed to make our information more easily accessible by creating the "Ideas, Tips, and Resource Highlights" section of our website. It provides a space to help people access the abundant resources developed and cataloged by this project, in a more easily accessible manner. What opportunities for training and professional development has the project provided? Learning Community: "Farmer Training Program Planning and Evaluation: GREW Resources and Applied Lesson." This session, conducted as part of the 2020 National Field School, provided participants with an overview of the tools developed by the GREW project, and examples of how two different organizations used and adapted them to assist with the planning and evaluation efforts. It was held on November 6, 2020. There were 53 people in attendance. There were 67 youtube views (55 unique viewers). https://agroecology.ucsc.edu/education/bfrdp/learning-community.html Webinar: "Self-Assessment for Evaluation and Enhanced Learning: using and Adapting the Business Health Assessment and the Refugee & Immigrant Participatory Benchmarking Tool." This webinar offered participants a glimpse into two different assessment instruments that can be used and modified for their organizations, presented by two beginning farmer program leads. It was held on December 17, 2020. There were 28 people in attendance. There were 65 youtube views (54 unique viewers). https://agroecology.ucsc.edu/education/bfrdp/webinars.html Webinar: "Developing Your Beginning Farmer and Rancher Development Project Evaluation." The GREW Project co-hosted a webinar for the BFRDP Program that offered insight for how participants can develop evaluation plans for their BFRDP proposal. It was held February 4, 2021. There were 65 people in attendance. https://agroecology.ucsc.edu/education/bfrdp/webinars.html Learning Community: "Evaluation Co-Working Series: Session 1." This session, part of a 4 part series, provided participants with accountability, a designated time, and evaluator support for them to make progress on their evaluation tasks and goals. It was held on April 20, 2021. There were 7 people in attendance. https://casfs.ucsc.edu/education/bfrdp/learning-community.html Learning Community: "Evaluation Co-Working Series: Session 2." This session, part of a 4 part series, provided participants with accountability, a designated time, and evaluator support for them to make progress on their evaluation tasks and goals. It was held on May 4, 2021. There were 9 people in attendance. https://casfs.ucsc.edu/education/bfrdp/learning-community.html Learning Community: "Evaluation Co-Working Series: Session 3." This session, part of a 4 part series, provided participants with accountability, a designated time, and evaluator support for them to make progress on their evaluation tasks and goals. It was held on May 18, 2021. There were 6 people in attendance. https://casfs.ucsc.edu/education/bfrdp/learning-community.html Learning Community: "Evaluation Co-Working Series: Session 4." This session, part of a 4 part series, provided participants with accountability, a designated time, and evaluator support for them to make progress on their evaluation tasks and goals. It was held on June 1, 2021. There were 3 people in attendance. https://casfs.ucsc.edu/education/bfrdp/learning-community.html Mentoring: Learning Community participants were offered an opportunity to schedule time with an evaluator for feedback and support with their organization's evaluation. Five people took advantage of these sessions. Webinar: "Are they still farming? Insights into Conducting Long-Term Evaluation of a Beginning Farmer Training Program." This webinar shared the process by which New Entry Sustainable Farming Project conducted an evaluation of its incubator program going back 15 years. It was held on September 2, 2021. There were 40 people in attendance. https://agroecology.ucsc.edu/education/bfrdp/webinars.html How have the results been disseminated to communities of interest?Announcements for webinars, learning community sessions and completed tools have been sent to the following locations: GREW Project listserv, BFRDP Projects listserv (through UMN), Virginia Beginning Farmer and Rancher Coalition general listserv, Virginia Beginning Farmer and Rancher Coalition Member listserv, Sustainable Agriculture Education Association listserv, Southern Region Extension and Research System (SERA-47) listserv, NIFTI Incubator Farms listserv, AgALN listserv, New Entry BFRDP TA listserv, New Entry General Newsletter, NIFTI/AgALN istserv, CFP Technical Assistance listserv and CASFS social media. What do you plan to do during the next reporting period to accomplish the goals?Our desired short-term outcomes from the project are that staff from beginning farmer and rancher (BFR) organizations gain more knowledge and skills about evaluation, have the tools they need to conduct evaluation, find evaluation to be more doable, feel more motivated and inspired to do it, and find value in learning with their peers. It is hoped that these short-term outcomes lead to medium-term outcomes, which include BFR staff implementing things they learned and finding evaluation to be useful. It is also hoped that their evaluations are more effective and that BFR programming is enhanced. To meet these outcomes, we plan to shift from offering stand alone webinars, to running a more in-depth 8-month learning community series to better support BFR organizations to make effective change in their evaluation activities. The learning community is titled the Culturally Responsive, Equitable, and Transformative Evaluation (CREaTE) Series. This series aims to help beginning farmer and rancher training and support organizations to build their capacity to do and use evaluation to measure progress toward their goals and leverage efforts to shift the needle towards systems change. We'll also continue to develop our "Ideas, Tips, and Resource Highlights" to provide easier access to a number of evaluation topics since webinars and online sessions have become burdensome in the COVID-19 climate. We will utilize social media to reach a broader audience. Additionally, as time and resources are available, we will develop other tools to support evaluation.

Impacts
What was accomplished under these goals? The long-term goal of this project is to create strong, effective, and long-lasting farmer and rancher training programs to ensure that beginning farmers and ranchers (BFRs) can enter the field and establish successful farming businesses. This educational enhancement team project will meet this goal by equipping BFR training organizations nationally with tools and skills to effectively conduct evaluations that clearly document program impact, improve programming, and enable program staff to identify and respond to farmer needs. Such documentation and program improvements will increase participants' success and provide the accountability needed for the long-term viability of BFR programs. The intended outcomes of this project are that BFR organizations will find evaluation more useful and will more effectively design and conduct evaluations. We accomplished the following for each project objective: Objective 1) Identify and collect existing tools and best practices for program evaluation. The majority of the work to meet this objective was conducted in year one of the project. However, staff continued to identify existing tools and upload them on the website (see objective 6) and explore best practices in the learning community sessions (see objective 5). These resources will continue to be collected and will be put in the online evaluation resource library as they are received. Objective 2) Identify gaps in evaluation outcome measures. This objective was completed in year one of the project. Objective 3) Create tools for efficient evaluation development. In year three of the project, other tools (and attendant resources) were created. In order to provide information in a more easily accessible manner, the project started producing one to two page "Ideas, Tips, and Resource Highlights" documents. Two documents were published this year, one focusing on how to write a BFRDP proposal's evaluation section, and another spotlighting already produced documents to help people ensure their evaluation is conducted in a more culturally competent manner. Several other Tips and Resource Highlight documents were developed as drafts during this time period, and will be posted in year four. Additionally, since conducting follow-up evaluations on medium and long term outcomes was one of the primary areas where BFR organizations wanted support, an example long-term outcome evaluation process was created for incubators. A follow-up survey tool was created, using other GREW Project example surveys and tools, then pre-tested. It was implemented with 60 past participants on the New Entry incubator program. The lessons learned from implementing and analyzing the survey and a focus group, along with the results from the data collection and analysis, were shared in a webinar (see Objective 4) and will be shared in year four in a Resource & Tip Sheet. Regarding all tools created and posted on the website, there were 413 unique views, and 717 total views to our Tools webpage (which houses the outcomes list and other diagrams) from September 1, 2020 to August 30, 2021. Objective 4) Build capacity of BFR staff through training. One goal of the project is to develop webinars to help build staff capacity to conduct program evaluation. A list of potential webinars was identified based on the results of the needs assessment survey (conducted in year one). Three webinars were conducted in year three. The first was presented by two BFR organizations, focusing on self-assessment. The next one was offered by the GREW project lead for the BFRDP program and focused on tips for developing evaluation plans for a BFRDP proposal. The third was offered by Tufts New Entry and focused on long term evaluation of BFR program participants. There were 133 participants for all three sessions, though surveys were not collected for the webinar hosted by the BFRDP program. Overall, post session survey respondents (13) reported that they at least somewhat agreed that they gained useful knowledge about evaluation (100%), feel inspired to begin or continue evaluation work (100%), feel that evaluation appears more doable (92%) and know where to go for more resources and information (77%). Additionally, 85% of respondents reported that they were at least somewhat likely to make additions, changes or improvements to their evaluation efforts as a result of the session, and 70% reported they were likely or definitely likely. Objective 5) Run an online evaluation learning community. The learning community sessions provide an avenue for BFR staff to discuss best practices and explore evaluation issues, and their solutions, with peers and evaluators. Five sessions were conducted this year. One focused on farmer training program planning and evaluation and it was conducted as part of the 2020 New Entry National Field School. The remaining four sessions were a part of a supported evaluation co-working series (11 unduplicated attendees), with an individual mentoring session offered to interested participants (5). Overall, there were 57 total attendees, unduplicated. A post-series evaluation was conducted for those who attended the co-working series. Overall, the respondents (5) reported they found the co-working series to be helpful or very helpful to them (100%) and that they were all either are somewhat likely, likely, or definitely likely to make additions, changes, and improvements to their evaluation efforts due to the series. The main things people got out of the sessions were they found a sense of support being with others doing the same type of work (100% agreed or strongly agreed), they used GREW resources, coaching and materials to further their program's evaluation work (80% agreed or strongly agreed), and made progress on their evaluation work (60% agreed; 40% somewhat agreed). During the third year, development began on an extended learning community series to support organizations to develop evaluations that are culturally responsive and address systems issues. Objective 6) Develop web-based repository for, and conduct broad dissemination of, project products. A website for the project was developed and maintained on the lead organization's website (https://agroecology.ucsc.edu/education/bfrdp/index.html). From September 1, 2021 to August 30, 2021, there were 871 users, 974 unique views, and 1,210 total views. An online Evaluation Resource Library was also updated and maintained during the project's third year. This library is hosted on a project partner site (https://nesfp.org/program-evaluation/library). Overall, there were 409 new users and 616 site visits between September 1, 2020 and August 31, 2021. The project also maintains a listserv to keep people informed about the project's educational activities, events and materials. As of 8/31/21, there were 237 people on the GREW listserv. Evaluation. To track the impact of the project, a follow-up survey was planned for the end of year three. However, due to receiving a no cost extension, we are postponing the final evaluation to the fourth year to assess project impact. Given extremely low response survey rates during the covid time period, we will explore other methods for assessing impact with the project's staff, evaluators and advisory team.Regarding the project's proposed metrics, we met the first three: 1) more than 400 people accessed our materials 2) more than 200 people participated in training and learning community sessions (duplicated), and 3) 89% reported they were at least somewhat likely to make changes or improvements to their program's evaluation efforts. Having not conducted a follow-up survey in year three, we cannot fully assess if we've met our final two metrics: 4) that 75% of participants would implement something they learned and 5) that 50% would report making changes to programming.

Publications

Progress 09/01/19 to 08/31/20

Outputs
Target Audience:The GREW Project aims to serve the staff members of beginning farmer and rancher (BFR) training and support organizations by helping them to better evaluate their programs. Our project announcements were sent to a variety of beginning farmer service organizations through partner and collaborating organizations' mailing lists and listservs (including listservs focusing on apprenticeship and incubator programs), and the BFRDP project mailing list. In order to ensure that all those who serve beginning farmers were contacted, we also sent announcements to groups with a broader focus who might also benefit from our project. These included the Sustainable Agriculture Education Association and the Comfood listservs. Responses to our post-session surveys helped identify who the project served. Overall, respondents from the different sessions were from non-profit organizations (61.3%), cooperative extension (17.2%), two-year colleges (1.1%), four-year colleges (5.4%), 1862 land grant universities (10.8%), and 1890 land grant universities (1.1%). These organizations serve a broad diversity of audiences. On average, these organizations reported serving aspiring farmers (60%), those in their first five years of farming (66.3%), organic/sustainable growers (41.1%), people from socially disadvantaged populations (43.2%), veterans (23.2%) and small scale farms (54.7%). As of August 31, 2020 the project served at least 176 unduplicated individuals who attended five webinars or learning community sessions. However, more have been served through two additional learning community sessions that did not have registrations lists, for a potentially duplicated total of 329. Additionally, many more have accessed our online materials. There were 711 unique hits to our Tools webpage (which houses the outcomes list and other diagrams) and 179 views of the most popular online video, entitled, "Alternative Outcome Development & Data Collection Strategies: Most Significant Change and Photovoice." We have surpassed the project's goal of serving 400 individuals. Changes/Problems:The project encountered a few challenges in the second year. While the project team had dealt with a fair amount of staff turn over in the first year, we were still able to stay on target. However, the beginning of the second year saw another distinct loss of staffing from one project partner. Because of this, along with COVID-19, the racial equity protests in the U.S., and California wildfires, our work was impacted. Several staff have been very impacted. As with many others around the U.S., we pivoted our work to attend to the needs of the moment. We were able to complete all objectives, except one: we were short one learning community session. While we were able to meet almost all our desired objectives, the ability to plan ahead was impacted, and may account for slightly delayed programming in year 3. What opportunities for training and professional development has the project provided?Learning Community - "Evaluation - Stories from the Field: Learning from Fellow Beginning Farmer Training Organizations." The goal of this learning community was to explore the qualities and characteristics of conducting a successful evaluation. There were 35 people in attendance. The session was conducted on September 18, 2019. Learning Community - "Building Capacity for Program Evaluation: A Critical Conversation about the Meaning of Farmer 'Success." Conducted at the National Farm Viability Conference on October 23, 2019. 60 people in attendance. Webinar - "The Development and Use of Outcomes to Evaluate Beginning Farmer and Rancher Training Programs." This webinar provided assistance with developing, tracking, and using outcomes. There were 80 people in attendance. It was conducted on February 2, 2020 and a recording was posted on the web on Feb 2020. https://www.youtube.com/watch?v=8q4nVBb3NiM&feature=youtu.be Webinar - "Alternative Outcome Development & Data Collection Strategies: Most Significant Change (MSC) and Photovoice (PV)." This webinar explored two participatory and qualitative evaluation methods. There were 53 people in attendance. It was conducted on February 25, 2020 and a recording was posted on the web in Feb 2020. https://www.youtube.com/watch?v=0gIeLDwV3Sc&feature=youtu.be Learning Community - "Reflection in a Time of Rapid Response: How to Better Pivot Beginning Farmer and Rancher Programming in Times of COVID-19." The goal of this learning community session was for participants to experience evaluative thinking so they feel inspired and able to implement this process at their organizations. There were 55 people in attendance. It was conducted on May 7, 2020. The session pre-worksheet was posted on the web in May 2020. https://casfs.ucsc.edu/education/bfrdp/learning-community.html Webinar - "Documenting Past Participant Outcomes: Beginning Farmer and Rancher Training Organizations Share Their Experience." The purpose of this webinar was to showcase how two different beginning farmer training organizations have conducted follow-up evaluations, to provide examples, tips and inspiration to others. There were 32 people in attendance. It was conducted on June 25, 2020 and a recording was posted on the web in June 2020. https://www.youtube.com/watch?v=XSFeleD1pFo Learning Community: "Identifying Beginning Farmer and Rancher Needs in Times of Change: Exploring an Appreciative Perspective for Responsive Programming." This session provided participants with examples and reflective questions for doing future needs assessments at their organizations. There were 14 people in attendance. It was conducted on July 16, 2020. The session pre-worksheet was posted on the web in July 2020. https://casfs.ucsc.edu/education/bfrdp/learning-community.html How have the results been disseminated to communities of interest?Announcements for webinars, learning community sessions and completed tools have been sent to the following locations: GREW Project listserv, BFRDP Projects listserv (through UMN), Virginia Beginning Farmer and Rancher Coalition general listserv, Virginia Beginning Farmer and Rancher Coalition Member listserv, Sustainable Agriculture Education Association listserv, Southern Region Extension and Research System (SERA-47) listserv, NIFTI Incubator Farms listserv, AgALN listserv, New Entry BFRDP TA listserv, New Entry General Newsletter, NIFTI/AgALN istserv, CFP Technical Assistance listserv and CASFS social media. What do you plan to do during the next reporting period to accomplish the goals?Our desired short-term outcomes from the project are that staff from beginning farmer and rancher (BFR) organizations gain more knowledge and skills about evaluation, have the tools they need to conduct evaluation, find evaluation to be more doable, feel more motivated and inspired to do it, and find value in learning with their peers. It is hoped that these short-term outcomes lead to medium-term outcomes, which include BFR staff implementing things they learned and finding evaluation to be useful. It is also hoped that their evaluations are more effective and that BFR programming is enhanced. To meet these outcomes, we plan to continue offering webinars, learning community sessions and tools. After reviewing evaluation data from post-session surveys and the follow-up survey, and discussion with the GREW Project Advisory Team, we will stay within the main framework, but adjust it to better meet audience needs. We will continue to develop and post webinars (two to three) highlighting the experiences of BFR organizations conducting evaluation. We will adjust the learning community session format slightly. Based on participant feedback, we will experiment with a four session commitment, where participants spend half of a 90 minute session in a co-working environment doing evaluation related work, then spend the other 45 minutes in discussion with their peers and project evaluator to talk about issues and get feedback. We'll also spend more time developing resources than previously planned. We'll develop a blog/newsletter to provide easier access to a number of evaluation topics since webinars and online sessions have become burdensome in the COVID-19 climate. We will utilize social media to reach a broader audience. Additionally, we will develop instruments that will provide useful templates for other programs, such as a follow-up survey to assess medium-term and long-term outcomes from incubator programs. We are also currently exploring developing an instrument/question series for assessing how programs are doing at becoming more equitable.

Impacts
What was accomplished under these goals? The long-term goal of this project is to create strong, effective, and long-lasting farmer and rancher training programs to ensure that beginning farmers and ranchers (BFRs) can enter the field and establish successful farming businesses. This educational enhancement team project will meet this goal by equipping BFR training organizations nationally with tools and skills to effectively conduct evaluations that clearly document program impact, improve programming, and enable program staff to identify and respond to farmer needs. Such documentation and program improvements will increase participants' success and provide the accountability needed for the long-term viability of BFR programs. The intended outcomes of this project are that BFR organizations will find evaluation more useful and will more effectively design and conduct evaluations. We accomplished the following for each project objective: Objective 1) Identify and collect existing tools and best practices for program evaluation. The majority of the work to meet this objective was conducted in year 1 of the project. However, staff continued to identify existing tools and upload them on the website (see objective 6) and explore best practices in the learning community sessions (see objective 5). These resources will continue to be collected and will be put in the online evaluation resource library as they are received. Objective 2) Identify gaps in evaluation outcome measures. This objective was completed in year 1 of the project. Objective 3) Create tools for efficient evaluation development. Several tools were further refined and completed in year 2 of the project. The purpose of these tools is to make it easier for BFR organization staff to conceptualize, identify and track program outcomes. They were promoted and made available for use for those writing their BFRDP 2020 proposals. These tools include the following: "Outcomes for Aspiring Farmers & Ranchers," "Identifying Farmer & Rancher Success," "Long-Term and System Level Outcome Examples," and "Example Pathway of a 9-Month Training Program." Additionally, the document titled "Identifying Evaluation Outcomes: Example Outcomes for Beginning Farmer and Rancher Training Efforts" was completed. In our follow-up survey at the end of year 2 (see more details in the evaluation section below), people who had registered for one or more of our webinar or learning community sessions were asked if they had used any of these tools. There were 25.7% of 35 question respondents that had least glanced at the diagrams, and 37.1% had read, searched or used them. Similarly, of the 37 question respondents, 40.5% had at least glanced at or downloaded the outcomes list, and 24.3% had read, searched or used it. Of those who downloaded, reviewed or used the outcome list (24), 62.5% (15) said that it is/will be useful for doing evaluation related work, and 91.7% said it is/will be at least somewhat useful. Objective 4) Build capacity of BFR staff through training. One goal of the project is to develop webinars to help build staff capacity to conduct program evaluation. A list of potential webinars was identified based on the results of the needs assessment survey. Three webinars were conducted. The first was presented by a team evaluator, focusing on outcome development. The other two showcased the evaluation work of other organizations who serve beginning farmers and ranchers. These two sessions focused on qualitative methods (Most Significant Change and Photovoice techniques) and how to do follow-up evaluations with past participants. There were 165 participants for all three sessions. Overall, post session survey respondents (54) reported that they at least somewhat agreed that they gained useful knowledge about evaluation (94.4%), feel inspired to begin or continue evaluation work (87.0%), feel that evaluation appears more doable (75.9%) and know where to go for more resources and information (90.7%). Additionally, 83.3% of respondents reported that they were at least somewhat likely to make additions, changes or improvements to their evaluation efforts as a result of the session. Objective 5) Run an online evaluation learning community. The learning community sessions provide an avenue for BFR staff to discuss best practices and explore evaluation issues, and their solutions, with peers and evaluators. Four sessions were conducted this year, focused on the following topics: 1) exploring successful evaluation strategies, 2) thinking about definitions of success in evaluation, 3) using evaluative thinking in times of change and 4) utilizing needs assessment in time of uncertainty (164 total attendees). Overall, post session survey respondents (41) reported they at least somewhat agreed that they feel inspired to begin or continue evaluation work (78.0%) and feel that evaluation appears more doable (63.4%). Respondents from two of the three sessions (23) were asked additional questions, and reported that they at least somewhat agreed that they gained useful knowledge about evaluation (78.3%) and know where to go for more resources and information (78.3%). There were 70.7% of the 41 respondents who said they were at least somewhat likely to make additions, changes or improvements to their evaluations efforts as a result of the session. Objective 6) Develop web-based repository for, and conduct broad dissemination of, project products. A website for the project was developed and maintained on the lead organization's website (https://casfs.ucsc.edu/education/bfrdp/index.html). During the project's second year, there were 1,201 total and 930 unique views of the website. An online Evaluation Resource Library was also updated and maintained during the second year. This library is hosted on a project partner site (https://nesfp.org/program-evaluation/library). As of August 2020, there were 397 visits to the online learning library. The project also maintains a listserv to keep people informed about the project's educational activities, events and materials. As of 10/12/20, there were 208 people on the GREW listserv. Evaluation. To track the impact of the project, a follow-up survey was conducted at the end of the second year (September 2020). All individuals who signed up to participate in a webinar or learning community session (310) were contacted to provide feedback about those sessions, and the tools the project created. There were 37 people who completed at least one question, for a 12% response rate. There were 59% (17 of 29) who agreed that they are thinking differently about evaluation now. There were 42.9% of 28 respondents who said they utilized something you learned through a GREW event or tool (46.4% said they were unsure if they utilized something). Of those who knew they used something, 66.7% (8) said their evaluation plan was now more realistic, 58.3% (7) said they wrote a stronger evaluation plan in a grant proposal, and 50.0% (6) said their outcomes were more relevant or realistic. Regarding the projects proposed metrics, we met the first three: 1) more than 400 people accessed our materials 2) more than 200 people participated in training and learning community sessions, and 3) 90% reported a knowledge increase (69 of 77 webinar or learning community participants at least somewhat agreed that they gained useful knowledge). However, the other two metrics have not yet been met. We originally hoped that 75% of participants would implement something they learned. While we did not meet that goal officially, over 40% of respondents did not recall if they used something or not, so it is possible that more did in fact implement something. Additionally, evaluation is often something that gets dropped in the face of more urgent matters, and this year presented multiple crises for many. Our final goal of having 50% report making changes to programming will be better assessed at the end of the program's third year.

Publications

• Type: Other Status: Published Year Published: 2020 Citation: Perez, Jan, Allyssa Mark, Pam Mavrolas, Kim Niewolny; Jennifer Hashley and Damian Parr. 2020. Identifying Outcomes for Program Evaluation: Example Outcomes for Beginning Farmer and Rancher Training and Support Efforts. University of California⿿Santa Cruz, Center for Agroecology & Sustainable Food Systems. Santa Cruz, CA. https://casfs.ucsc.edu/education/bfrdp/evaluation-outcomes-list.pdf
• Type: Other Status: Published Year Published: 2020 Citation: Perez, Jan, Jennifer Hashley, Kim Niewolny, Damian Parr, and Allyssa Mark. 2020. Outcomes for Aspiring Farmers and Ranchers. University of California⿿ Santa Cruz, Center for Agroecology & Sustainable Food Systems. Santa Cruz, CA. https://casfs.ucsc.edu/education/bfrdp/outcomes-aspiring-farmers-ranchers.pdf
• Type: Other Status: Published Year Published: 2020 Citation: Perez, Jan, Kim Niewolny, Jennifer Hashley, Damian Parr, and Allyssa Mark. 2020. Identifying Farmer and Rancher Success. University of California⿿Santa Cruz, Center for Agroecology & Sustainable Food Systems. Santa Cruz, CA. https://casfs.ucsc.edu/education/bfrdp/identifying-farmer-rancher-success.pdf
• Type: Other Status: Published Year Published: 2020 Citation: Perez, Jan, Damian Parr, Kim Niewolny, Jennifer Hashley, and Allyssa Mark. 2020. Long Term and System Level Outcome Examples. University of California⿿Santa Cruz, Center for Agroecology & Sustainable Food Systems. Santa Cruz, CA. https://casfs.ucsc.edu/education/bfrdp/long-term-system-level-outcomes.pdf
• Type: Other Status: Published Year Published: 2020 Citation: Perez, Jan, Jennifer Hashley, Kim Niewolny, Damian Parr, and Allyssa Mark. 2020. Example Pathway of a 9-Month Training Program. University of California⿿Santa Cruz, Center for Agroecology & Sustainable Food Systems. Santa Cruz, CA. https://casfs.ucsc.edu/education/bfrdp/example-pathway.pdf

Progress 09/01/18 to 08/31/19

Outputs
Target Audience:The GREW Project aims to serve the staff members of beginning farmer and rancher (BFR) training and service organizations by helping them better evaluate their programs. Our project announcements were sent to a variety of beginning farmer service organizations through partner and collaborating organizations' mailing lists and listservs (including listservs focusing on apprenticeship and incubator programs), and the BFRDP project mailing list. In order to ensure that all those who serve beginning farmers were contacted, we also sent announcements to groups with a broader focus who might also benefit from our project. These included the Sustainable Agriculture Education Association and the Comfood listservs. Responses to our needs assessment survey more clearly identified who the project will serve. We received 135 usable responses to the survey. Respondents were from non-profit organizations (55%), cooperative extension (19%), two-year colleges (3%), four-year colleges (7%), 1862 land grant universities (6%), and 1890 land grant universities (1.5%). There were also a few who identified as "other" organizations, including people from working farms, private for-profits, consultants, and cooperatives. Of all the respondents, 56% either have a BFRDP grant or plan to apply for one. The respondents serve a broad diversity of audiences. A majority of these organizations serve aspiring farmers, those in their first five years of farming, organic/sustainable growers, and small scale farms. As of September 30 (a month after the announcement of our online learning materials), there were 222 people who had connected with our learning resources (though some individuals may have been counted twice). There were 78 unique views of the online webinar and 144 views of the online learning library. Changes/Problems:No major problems to report for year 1 of the project. What opportunities for training and professional development has the project provided?Webinar - "Evaluation 101: The basics of program evaluation for beginning farmer and rancher programs." The webinar outlines the basics of doing evaluation, along with useful resources, for getting started doing an evaluation. It was posted on the web in Aug 2019: https://www.youtube.com/watch?v=FnwkHEjxuYI&t=43s. How have the results been disseminated to communities of interest?Project Launch Announcement and Needs Assessment Survey Request. The purpose of this document was to announce the launch of the project, get people to sign up for our listserv, and encourage people to take the needs assessment survey. The announcement includes the project title: GREW Project - Gaining Results through Evaluation Work. The announcement was sent to the networks of the project partners and advisors, which includes the NIFTI incubator listserv, the AgAln national apprenticeship listserv, all BFRDP grantees, all CASFS apprenticeship alumni, the COMFOOD listserv, the California Farmer Educator Network, the Sustainable Agriculture Education Association (SAEA) listserv, as well as to several other regional network lists. Evaluation 101 Webinar and Online Evaluation Library Announcement. The purpose of this announcement was to alert people to the new resources that were available for them to use. This includes alerting them to the online evaluation resource library, the Evaluation 101 webinar that was posted online, and the upcoming learning community session. This announcement was sent to the GREW Project listserv, Virginia Tech's listserv, the AgAln listserv, the NIFTI incubator listserv, and all BFR organizations who signed up for technical assistance information from New Entry. Learning Community Announcement. The purpose of this announcement was to get people to register for our first online learning community event scheduled for September 18, 2019. This announcement was sent to the GREW listserv. What do you plan to do during the next reporting period to accomplish the goals?Our desired short-term outcomes from the project are that staff from beginning farmer and rancher (BFR) organizations gain more knowledge and skills about evaluation, have the tools they need to conduct evaluation, find evaluation to be more doable, feel more motivated and inspired to do it, and find value in learning with their peers. It is hoped that these short-term outcomes lead to medium-term outcomes, which include: BFR staff implementing things they learned and finding evaluation to be useful. It is also hoped that their evaluations are more effective and that BFR programming is enhanced. To meet these outcomes, we will finish the outcome list and pathway diagrams. They will be posted on the project website and widely advertised. BFR organization staff can use these documents to plan or revise their evaluations or performance monitoring plans. We will also continue to create a full plan for the webinar and learning community sessions. We will contact more of the needs assessment survey participants who said they had information to share about best practices and schedule times for them to share this information in the sessions. More specifically, the webinars will focus briefly on evaluators sharing instruction, but will spend most of the time showcasing BFR programs who have done extensive work on the topic. There will be one to two webinars this year. The learning community sessions will continue to be a place for peer discussion, but, as relevant, we will also find BFR organization staff with valuable experience to share briefly on their work as a way of organizing the peer discussions. Due to losing several staff, we will train new staff to manage the web-based learning technology. We will utilize evaluation findings from the first online session to inform the following ones. We will continue to utilize the reflection exercises to orient people to the sessions. Our goal will be to hold at total of five to six sessions this coming year. At the end of the first year, we will conduct a follow-up evaluation to assess how people utilized what they learned (from webinars, learning community sessions, and online resources) and how their actions impacted them, as well as their organizations. At the beginning of the third year, we will review these findings, along with the results of post-session surveys, to refine and revise how we conduct the learning activities in the third year of the project.

Impacts
What was accomplished under these goals? The long-term goal of this project is to create strong, effective, and long-lasting farmer and rancher training programs to ensure that beginning farmers and ranchers (BFRs) can enter the field and establish successful farming businesses. This educational enhancement team project will meet this goal by equipping BFR training organizations nationally with tools and skills to effectively conduct evaluations that clearly document program impact, improve programming, and enable program staff to identify and respond to farmer needs. Such documentation and program improvements will increase participants' success and provide the accountability needed for the long-term viability of BFR programs. The intended outcomes of this project are that BFR organizations will find evaluation more useful and will more effectively design and conduct evaluations. We accomplished the following for each project objective: Objective 1) Identify and collect existing tools and best practices for program evaluation. A needs assessment survey was developed and implemented in January 2019. The goal of the survey was to identify the learning needs of the people this project plans to serve. The survey also gathered information about people willing to share data collection tools and reports, as well as people who have best practices to share with others through the webinars and the learning community. There were 135 people who responded to the survey. There were 44 people who had resources to share, and 24 willing to share their best practices. Project staff have started to collect some of these resource documents (surveys, reports, etc.). These resources will continue to be collected and will be put in the online evaluation resource library as they are received. Objective 2) Identify gaps in evaluation outcome measures. Project partners and members of the advisory team confirmed that important gaps in outcome measures revolve around tracking progress of aspiring farmers, measuring farmer success and identifying long-term outcomes or different system level outcomes. It was also confirmed that identifying example outcomes for all types of BFR programming would be useful. Activities intended to address these gaps included reviewing relevant research and literature, RVS reports, REEport reports, and a recent evaluation of BFRDP programming. Expert opinion on gaps was sought from project partners and advisors. Additionally, staff from a variety of BFR organizations were asked to review draft tools (see objective 3) to ensure that outcomes important to their organizations were included. It was concluded that the primary outcomes important to programs were identified. Thus, we determined there was no need to conduct farmer interviews at this time. Objective 3) Create tools for efficient evaluation development. The information collected on the gaps in evaluation measures was utilized to create the developmental outcome pathway diagrams and the outcome list. The purpose of these tools is to make it easier for BFR organization staff to conceptualize, identify and track program outcomes. By the end of the first year, four pathway diagrams were close to complete, which included: "Outcomes for Aspiring Farmers & Ranchers," "Identifying Farmer & Rancher Success," "Long-Term and System Level Outcome Examples," and "Example Pathway of a 9-Month Training Program." Additionally, the document titled "Identifying Evaluation Outcomes: Example Outcomes for Beginning Farmer and Rancher Training Efforts" had a solid draft. These documents had been conceptualized, developed and revised by core project partners and advisors, and reviewed by a sample of other BFR serving organizations. These documents will be completed in the middle of the second year. Objective 4) Build capacity of BFR staff through training. One goal of the project is to develop webinars to help build staff capacity to conduct program evaluation. A list of potential webinars was identified based on the results of the needs assessment survey. No webinars were planned for the first year. However, the advisory team members and project partners determined that a webinar covering the basics of doing an evaluation was important to have as a first step in the project. An "Evaluation 101" webinar was produced, utilizing feedback from the project evaluators and a few advisory team members to ensure it would be useful and appropriate for the primary audience. The webinar covered the basic steps of doing an evaluation and identified some specific resources for each of the steps along of the way. The webinar was recorded and posted online (https://www.youtube.com/watch?v=FnwkHEjxuYI&t=43s) in August 2019, and announced at the end of that month. As of September 30, the webinar had 78 unique views. Objective 5) Run an online evaluation learning community. The goal of the learning community is to provide an avenue for peer-to-peer learning. It is a place for BFR staff who conduct evaluation to discuss best practices and explore evaluation issues, and their solutions, with peers and evaluators. Only one session was targeted for the first year, but a webinar (described above) was conducted in its place. The needs assessment was used to create an overall strategy and education framework, as well as to develop a plan, for the first few learning community sessions. Preparation work took place to implement the first session in September, including developing an agenda, a reflection assignment, and outreach materials. Much time was also invested learning the web-based technology to manage break-out groups in an online meeting environment, which were a necessary component for fostering peer-to-peer learning. The first session was announced and participants recruited in August, 2019. Objective 6) Develop web-based repository for, and conduct broad dissemination of, project products. A website for the project was developed and is hosted on the lead organization's website (https://casfs.ucsc.edu/education/bfrdp/index.html). By the end of the project's first year, there were 789 unique views of the website. An online Evaluation Resource Library was also developed during the first year. This library is hosted on a project partner site (https://nesfp.org/program-evaluation/library). The purpose of this library is to help BFR training organization staff more easily access the abundant materials available to help people do evaluation, as well as to provide useful examples of evaluation materials created by and for BFR organizations. Project staff conducted extensive research to identify available evaluation instruction materials. These materials were reviewed for relevance, categorized, summarized and formatted for uploading to a web-based database. Additionally, example surveys, logic models, and reports were collected from project partners and other organizations (who noted their willingness to share materials in the needs assessment survey). They were then summarized, categorized and uploaded. The database went live and was announced to BFR organizations in August 2019. By the end of September, there were 144 views of the online learning library. As part of the communications plan for the project, a project name was developed, along with a logo, and used with all communications. The project is entitled: Gaining Results through Evaluation Work - also known as the GREW Project. A listserv was also created to keep people informed about the project's upcoming educational activities, events and materials. As of November 2019, there were 177 people on the GREW listserv. Evaluation. The evaluation plan for this project was developed with feedback from all project partners and advisors. The first drafts of the post webinar and learning community post session surveys were developed in the first year. These surveys were implemented at the beginning of the second year of the project, and summaries will be reported in second year reporting.

Publications

• Type: Conference Papers and Presentations Status: Other Year Published: 2018 Citation: Perez, Jan. 2018. Evaluation Support and Technical Assistance EET for Beginning Farmer and Rancher Programs. Presented at the Beginning Farmer and Rancher Development Program Directors Meeting, Albuquerque, NM. September 27, 2018.
• Type: Conference Papers and Presentations Status: Other Year Published: 2018 Citation: Hashley, Jennifer & Lindsay Davies. 2018. Beginning Farmer Pathway Analysis. Presented at the New Entry Sustainable Farming Project, 2018 NIFTI (National Incubator Farm Training Initiative) Field School, Boulder, CO. November 7, 2018.
• Type: Other Status: Other Year Published: 2018 Citation: Perez, Jan. 2018. Evaluation Support and Technical Assistance EET for Beginning Farmer and Rancher Programs. Poster displayed at the Beginning Farmer and Rancher Development Program Directors Meeting, Albuquerque, NM. September 24-27, 2018.