Performing Department
(N/A)
Non Technical Summary
The goal of this project is to increase undergraduate students' experiences with program evaluation, research methods, and data analysis, while simultaneously providing evaluative support to Extension programming efforts in the state of West Virginia. To fulfill this goal, the project will specifically focus on the following objectives: 1) Increase undergraduate students' knowledge and skills related to program evaluation methods and data analysis techniques; and 2) Develop a system of continued collaboration between undergraduate students and Extension faculty and staff to enhance the evaluation of Extension programming. Students selected for participation in the program will serve in a two-year cohort, consisting of approximately 10-15 fellows per year over the course of the five-year project. Fellows will be afforded the unique opportunity to serve both in the role of a student during the first year of their fellowship, as well as that of a mentor to the upcoming fellowship cohort during the second year of their fellowship. Students will engage in a formal learning environment through participation in the online undergraduate-level AGEE Introduction to Extension Program Evaluation courses (one lecture-based course, one seminar-based course), as well as in a non-formal learning environment through placement at a designated Extension evaluation site, where they will apply the knowledge gained in the classroom to conduct an evaluation of Extension programming efforts in either individual counties or broader programmatically-based teams. Students will prepare final evaluation reports/impact statements that will be disseminated back to the evaluation host sites and across the state.
Animal Health Component
0%
Research Effort Categories
Basic
(N/A)
Applied
100%
Developmental
(N/A)
Goals / Objectives
The overarching goal of the Extension Mentoring IN EvaluatION (MINION) Project is to increase undergraduate students' experienceswith program evaluation, research methods, and data analysis by assessing current programming efforts being undertaken byWest Virginia University Extension. To fulfill this goal, the project will specifically focus on the following objectives:Increase undergraduate students' knowledge and skills related to program evaluation methods and data analysis techniques through the development and implementation of two undergraduate evaluation courses and the subsequent implementation of at least one or more evaluations conducted over county- and/or programmatically-based West Virginia University Extension programs;Increase undergraduate students' knowledge of the Cooperative Extension Service, and more specifically, WVU Extension;Develop a system of continued collaboration between undergraduate students and Extension faculty and staff to enhance the evaluation of Extension programming efforts in West Virginia.
Project Methods
MethodsThis program seeks to prepare undergraduate students in program evaluation, research methods, and data analysis by participation in two undergraduate-level academic courses and through the implementation of at least one or more evaluations conducted over county- and/or programmatically-based West Virginia University Extension programs. Students selected for the program will serve in a two-year cohort, consisting of approximately 10-15 fellows per year, for a total of 65-70 fellows over the course of the five-year project. Fellows will be afforded the unique opportunity to serve both in the role of a student during the first year of their fellowship, as well as that of a mentor to the upcoming fellowship cohort during the second year of their fellowship.Applications to serve as an evaluation "host site" will first be submitted by WVU county Extension offices and/or programmatically-based working groups (e.g., the 4-H and Youth Development - Youth Agriculture content group). Applications will be vetted by the project team. After evaluation host sites/projects have been selected, student applications will then be accepted, with a list of participating sites/projects provided. Applications will first be reviewed by the project team and will then participate in a virtual interview with the project team, as well as the supervising agent in each host site of preference. Selected applicants will be matched with an evaluation host site. Both host-site and student applications will be solicited and reviewed during the spring semester of each project year.For the first year of each fellowship cohort, beginning in the fall semester, students will complete the AGEE Introduction to Extension Program Evaluation [LECTURE] course.The course will be offered online, synchronously, for one-credit hour and will meet once a week for 16 weeks. The course is modeled after a professional development series focused on program evaluation previously offered to WVU Extension faculty.Following in the spring semester, each fellowship cohort will complete the AGEE Introduction to Extension Program Evaluation [SEMINAR] course.The course will be offered online, synchronously, for one-credit hour and will meet bi-weekly for 16 weeks. This course will provide a touchpoint for students as they begin their evaluation field work - meeting with agents/stakeholders, forming their evaluation plan, and beginning to collect data. The cohort will be afforded the opportunity to discuss successes and setbacks and can brainstorm the best methods for approaching each evaluation project.During the summer semester, students will be fully engaged in their evaluation efforts, primarily collecting and analyzing data and writing final evaluation reports. While initial meetings and preliminary planning efforts during the spring semester may be conducted through virtual methods, data collection during the summer months should occur at the host site location. Students may remain for a designated number of days to collect data or may travel to the host site location as needed. Final evaluation reports and/or impact statements will be reviewed by the project team before dissemination back to the evaluation host sites.For the second year of each fellowship cohort, the fellows will now serve as mentors to the newly recruited upcoming cohort. During the fall semester, the mentors will participate in a mentor orientation training, consisting of content in mentorship, team building, leadership development, and communication skills. While the new cohort of fellows is participating in coursework and beginning to conduct their evaluations (fall and spring semesters), the mentors (with supervision from the project team), will conduct regular check-ins and provide feedback as evaluation plans develop. During the summer months, the mentors will assist the fellows with data collection/analysis and the writing of final evaluation reports/impact statements. Students serving in the mentorship role will conclude their two-year fellowship at the end of the summer semester.Program Evaluation and ReportingProgram evaluation will start at the beginning of the program and continue throughout. We will seek appropriate IRB approval through the WVU Human Research Protections Program (HRPP) in the Office of Research Integrity and Compliance (ORIC). Since our target audience is students, we will adhere to additional IRB requirements as needed. The key evaluation questions for the program will be:Does the program increase self-efficacy and skills around research and evaluation and other program outcomes among youth?To what extent does the program increase Extension professional's capacity to conduct program evaluation?Are there demographic differences in the program's intended outcomes?What unintended outcomes, if any, did the program have on students and communities?The evaluation lead will guide formative data collection at the early and mid-stages of each year of the program to assess whether the program is being implemented with fidelity. Extension educators, coaches, and the project team will provide formative data that could be used to identify opportunities for program improvements. Furthermore, fellows will be completing weekly assignments while enrolled in the course. These assignments will be used as a formative assessment to determine if the course is successful in teaching desired knowledge and skills. Summative data will be collected at each the end of each program year to determine the impacts of each programming on participants. The five years will be evaluated separately and then holistically using data collected from students who participate in both the fellowship and mentorship areas of the program. The distinct evaluation methods follow:Each year, students will complete a pre-program assessment that measures their knowledge and attitudes of evaluation and research methods in Extension. A post-program assessment will be conducted at the end of each year. All students enrolled in the program will be assessed using a cohort-based survey design, this will allow us to follow up with them after graduation to determine lasting effects of the program. Lastly, focus groups will be conducted regionally with Extension agents that hosted fellows. These focus groups will be used to conduct ripple effect mapping. This will allow us to see the effects and outcomes of this program on the extension agents, fellow, and community.Limitations to this evaluation design include the willingness of fellows to complete pre- and post- surveys accurately. We hope to address this by using a multi-modal approach. We will offer the survey in Qualtrics, on paper, and through a QR (Quick Response) code. Automatic reminders will be set up to remind students to complete surveys before and after the program. Another limitation would be the willingness of the extension agents to participate in the focus groups. This will be addressed by hosting regional focus groups in person to reduce travel time and offer a zoom version for those that cannot travel.?