Progress 04/01/09 to 08/31/13
Outputs
Target Audience: TARGET AUDIENCES: The national eXtension community, the viticulture industry, wheat producers, Cooperative Extension Educators, and Specialists. Changes/Problems:
Nothing Reported
What opportunities for training and professional development has the project provided?
Nothing Reported
How have the results been disseminated to communities of interest? This project practiced an emerging evaluation design, Developmental Evaluation (Patton, 2011), as well as formative and summative methods to support educative, values engaged evaluation resulting in timely and meaningful feedback to program decision makers to reduce uncertainty for planning and implementing programs. The results of the evaluation projects during the past year have provided feedback to program planners regarding project implementation and accomplishments and have resulted in several scholarly publications at regional and national conferences and peer-reviewed journals. In addition, a new evaluation method was developed called Prospect Evaluation and will be disseminated nationally at regional, national conferences in addition to publication in journals. What do you plan to do during the next reporting period to accomplish the goals?
Nothing Reported
Impacts
What was accomplished under these goals?
The objective the project was to introduce program planners, implementers, and stakeholders to evaluative practices. Results include clear documentation of program inputs, activities, participation, outputs, and outcomes; and program planner's ability to plan and execute formal evaluation for program improvement and accountability. Significant accomplishments were made toward project goals during 2013. Seven federally-funded evaluation projects were undertaken in the past year to build evaluation capacity and transfer the technology of evaluation to stakeholders. A USDA-HEC project titled Teaching Locally, Engaging Globally: Increasing Undergraduates Knowledge of International Dimensions (Amy Harder, University of Florida, PI) completed year two and was evaluated for formative outcomes in 2013. Results highlighted team accomplishments in developing reusable learning objects to support international education. Work for iWheat: A web-based IPM information and delivery system to unify reduced risk management strategies for winter wheat in the central and southern plains (USDA, Risk Assessment Mitigation Program Kris Giles, OSU, PI) continues with assessing wheat producers intention to use Web 2.0 tools. Data were collected in the Great Plains region to measure wheat producers use of the Internet, specifically Web 2.0 tools that allow for interactivity. A new model for evaluation, Prospect Evaluation (Brown & Kelsey, 2013) was developed and results disseminated nationally. Evaluation services were provided on the NSF-funded project, Collaborative Research: URM: Preparing Biologists through Stewardship, Professionalism and Practice (Kristen Baum and Don French, OSU) to determine program process with undergraduate science majors for retention in the biological sciences. Evaluation activity was undertaken on the USDA-funded project, Oklahoma State University Coordination Program for IPM Oklahoma! (Tom Royer, OSU, PI) to develop original evaluation instruments to document workshop outcomes statewide. Instruments were presented to the IPM team for use in collecting data to measure impacts of Extension efforts. A USDA AFRI food security grant funded project was initiated in 2012 titled A predictive Model to Increase Adoption of IPM of a Mint-Vitus Disease Complex in Wheat (Gary Hein, UNL). Evaluation activity will assist the project leaders to document impacts and outcomes while providing feedback on educational activities as they are developed and delivered. First year evaluation work is underway in collecting project implementation data to inform project leaders about progress made. External evaluation advisory services were provided on a USDA-FIFA Capacity Building Grant titled Building Laboratory and Intellectual Capacity in order to Effectively Detect and Reduce Salmonella in the Food Supply (Todd Brashears, TTU) as an external evaluation adviser.
Publications
- Type:
Journal Articles
Status:
Accepted
Year Published:
2013
Citation:
Brown, N. R., Terry, Jr. R., Kelsey, K. D. (2013). The impact of learning styles on learning outcomes at FFA camp: What campers retain over time. Journal of Agricultural Education, 54(3).
- Type:
Journal Articles
Status:
Accepted
Year Published:
2013
Citation:
Blackburn, J., & Kelsey, K. D. (2013). Understanding authentic assessment in a secondary agricultural mechanics laboratory: An instrumental case study. Journal of Human Sciences and Extension, 1(1), 1-16.
- Type:
Other
Status:
Accepted
Year Published:
2013
Citation:
Brown, N. R. & Kelsey, K. D. (2013). Exploring Great Plains wheat producers intention to use Web 2.0 technology: A prospect evaluation. Funded by the United State Department of Agriculture. Oklahoma State University, Stillwater, June 12, 2013.
- Type:
Journal Articles
Status:
Accepted
Year Published:
2013
Citation:
Brown, N. R., & Kelsey, K. D. (2013). Sidewalks and city streets: A model for vibrant agricultural education in urban American communities. Journal of Agricultural Education, 54(2), 57-69. http://www.jae-online.org/attachments/article/1737/54.2.5%20%20Brown.pdf
- Type:
Journal Articles
Status:
Accepted
Year Published:
2013
Citation:
Schrock, J., & Kelsey, K. D. (2013). A phenomenological look at 4-H volunteer motives for service. Journal of Extension, 51(2), article 2RIB1. Available: http://www.joe.org/joe/2013april/pdf/JOE_v51_2rb1.pdf
- Type:
Conference Papers and Presentations
Status:
Accepted
Year Published:
2013
Citation:
Brown, N. R. & Kelsey, K. D. (2013). Introducing prospect evaluation through exploring Great Plains wheat producers' behaviors and attitudes regarding Web 2.0 Technology. (2013-1394). Western Region of the American Association of Agricultural Educators Conference, Lubbock, TX, September 23-25, 2013
|
Progress 10/01/11 to 09/30/12
Outputs
OUTPUTS: The objective this project was to introduce program planners, implementers, and stakeholders to evaluative practices. Results include clear documentation of program inputs, activities, participation, outputs, and outcomes; and program planner's ability to plan and execute formal evaluation for program improvement and accountability. Significant accomplishments were made toward project goals during 2012. Seven federally-funded evaluation projects were undertaken in the past year to build evaluation capacity and transfer the technology of evaluation to stakeholders. A USDA-SCRI project titled Development of a Grape Community of Practice for the eXtension System (Eric Stafne, OSU, PI) concluded with summative evaluation work in 2012. Results of the evaluation found that the aspirations of the Grape CoP had been realized and the project accomplished 100% of the objectives. A national repository for viticulture information was created (http://www.extension.org/Grapes) and a community of practice emerged from the project. A USDA-HEC project titled Teaching Locally, Engaging Globally: Increasing Undergraduates Knowledge of International Dimensions (Amy Harder, University of Florida, PI) commenced and was developmentally evaluated in 2012. Results highlighted team accomplishments as setting the stage to implement the project goals for years two and three. Work for iWheat: A web-based IPM information and delivery system to unify reduced risk management strategies for winter wheat in the central and southern plains (USDA, Risk Assessment Mitigation Program Kris Giles, OSU, PI) continues with assessing wheat producers intention to use Web 2.0 tools. Data were collected in the Great Plains region to measure wheat producers use of the Internet, specifically Web 2.0 tools that allow for interactivity. Evaluation services were also provided on the NSF-funded project, Collaborative Research: URM: Preparing Biologists through Stewardship, Professionalism and Practice (Kristen Baum and Don French, OSU) to determine program process with undergraduate science majors for retention in the biological sciences. Evaluation activity was undertaken on the USDA-funded project, Oklahoma State University Coordination Program for IPM Oklahoma! (Tom Royer, OSU, PI) to develop original evaluation instruments to document workshop outcomes statewide. A USDA AFRI food security grant funded project was initiated in 2012 titled A predictive Model to Increase Adoption of IPM of a Mint-Vitus Disease Complex in Wheat (Gary Hein, UNL). Evaluation activity will assist the project leaders to document impacts and outcomes while providing feedback on educational activities as they are developed and delivered. External evaluation advisory services will also be provided on a USDA-FIFA Capacity Building Grant titled Building Laboratory and Intellectual Capacity in order to Effectively Detect and Reduce Salmonella in the Food Supply (Todd Brashears, TTU). PARTICIPANTS: Not relevant to this project. TARGET AUDIENCES: The national eXtension community, the viticulture industry, wheat producers, Cooperative Extension Educators, and Specialists. PROJECT MODIFICATIONS: Not relevant to this project.
Impacts
This project practiced an emerging evaluation design, Developmental Evaluation (Patton, 2011), as well as formative and summative methods to support educative, values engaged evaluation resulting in timely and meaningful feedback to program decision makers to reduce uncertainty for planning and implementing programs. The results of the evaluation projects during the past year have provided feedback to program planners regarding project implementation and accomplishments and have resulted in several scholarly publications at regional and national conferences and peer-reviewed journals.
Publications
- Fabregas Janeiro, M. G., Kelsey, K. D., Robinson, J. S. (2012). Predicting intercultural sensitivity using demographic variables among college of agriculture undergraduate students. US-China Education Review, 2(8), 710-719.
- Kelsey, K. D., & Stafne, E. T. (2012). A model for evaluating eXtension communities of practice. Journal of Extension, 50(5), 5FEA1. Available http://www.joe.org/joe/2012october/a1.php
- Xu, X. & Kelsey, K. D. (2012). Will eXtension survive? Oklahoma Cooperative Extension Service employees perceptions of adopter attributes of eXtension. Journal of Extension, 50(6). Article Number 6RIB2.
- Baughman, S., Boyd, H. & Kelsey, K. D. (2012). The impact of the Government Performance and Results Act (GRPA) on two state Cooperative Extension systems. Journal of Extension, 50(1). Article Number IFEA3.
- Kelsey, K. D., Stafne, E. T., & Greer, L. (2011). Land-grant university employee perceptions of eXtension: A baseline descriptive study. Journal of Extension, 49(6). Article Number 6FEA2.
- Kelsey, K. D., Baughman, S. (2012). Using a developmental evaluation approach for innovative Extension programs and processes. American Evaluation Association National Conference. Minneapolis, MN, October 24-27, 2012.
- Kelsey, K. D. & Stafne, E. T. (2012). A model for evaluating eXtension Communities of Practice. American Evaluation Association National Conference. Minneapolis, MN, October 24-27, 2012.
- Brown, N. R., Baker, M. & Kelsey, K. D. (2012). Q-sorts, observations, and conversations: Evaluating school integrated pest management programs using novel methods. American Evaluation Association National Conference. Minneapolis, MN, October 24-27, 2012.
- Gill, G. & Kelsey, K. D. (2012). Extending Cooperative Extensions reach through the senses: An expertise-oriented evaluation of the design and implementation of a sensory garden. American Evaluation Association National Conference. Minneapolis, MN, October 24-27, 2012.
- Blackburn, J., McCool, M., & Kelsey, K. D. (2012). Using developmental evaluation for curriculum rejuvenation. American Evaluation Association National Conference. Minneapolis, MN, October 24-27, 2012.
|
Progress 10/01/10 to 09/30/11
Outputs
OUTPUTS: OUTPUTS: The objective this project was to introduce program planners, implementers, and stakeholders to evaluative practices. Results include 1) clear documentation of program inputs, activities, participation, outputs, and outcomes; and 2) program planner's ability to plan and execute formal evaluation for program improvement and accountability. Significant accomplishments were made toward project goals during 2011. Two major evaluation projects were undertaken using a new evaluation approach, M. Q. Patton's Developmental Evaluation (2011). A USDA funded project was secured, Development of a Grape Community of Practice for the eXtension System (Eric Stafne, OSU, Lead PI). Kathleen Kelsey, Co-PI and lead evaluator undertook a developmental evaluation approach to determine the following: 1) how were interactions among the community of practice (CoP) unfolding 2) What was the collaboration accomplishing 3) How did the collaborators work together 4) How did collaborators see themselves and their shared effort in regard to degrees of engagement 5) Did the aspirations of the collaboration emerge In addition to these focus questions, the evaluator worked systematically to infuse the values of evaluation among team members. Results of the evaluation found that the aspirations of the Grape CoP had been realized and the project was accomplishing 100% of their goals to a high degree of efficiency. The project was well organized, well managed, and all goals for years one and two had been achieved. The founding team envisioned a national repository for viticulture information. The Grape CoP has become a tangible product (http://www.extension.org/Grapes), the infrastructure is in place to grow the repository, and a true community of practice has emerged for members to benefit from the CoP professionally. The second evaluation project included collecting baseline data for a USDA funded project called iWheat: A web-based IPM information and delivery system to unify reduced risk management strategies for winter wheat in the central and southern plains (USDA, Risk Assessment Mitigation Program. Contract agreement no: 2010-51101-21642. Kelsey is the lead evaluator with Kris Giles, et al., Oklahoma State University). The leadership team desired to learn how wheat producers used Web 2.0 tools. An original survey was created and administered in a five-state region (Texas, Oklahoma, Nebraska, Kansas and Colorado) to gather data regarding perceptions and use of the Internet, specifically Web 2.0 tools that allow for interactivity. The evaluator worked closely with clients and stakeholders to conduct formative evaluation for the purpose of program improvement. The clients were included in the evaluation planning and data collection phases to disseminate evaluation technologies. Clients were empowered to use evaluation data for program improvement and accountability purposes. Evaluation expertise was also shared with the larger community via workshops and evaluation reports. Evaluation reports were submitted to program planners for program improvement. PARTICIPANTS: Not relevant to this project. TARGET AUDIENCES: The national eXtension community, the viticulture industry, wheat producers, Cooperative Extension Educators, and Specialists. PROJECT MODIFICATIONS: Nothing significant to report during this reporting period.
Impacts
The outcomes of program evaluation are multifaceted. This project practiced an emerging evaluation design, Developmental Evaluation (Patton, 2011); to support educative, values engaged evaluation methods resulting in timely and meaningful feedback to program decision makers to reduce uncertainty regarding planning and implementing programs. The results of the evaluation projects during the past year have provided feedback to program planners regarding Community of Practice formation and cohesion, and readiness to engage in the task of enhancing our understanding of the role and use of evaluation in Cooperative Extension System collaborative efforts, specifically eXtension.org. The impact of the iWheat project includes gaining a deeper understanding of wheat producers use of Web 2.0 tools so that Extension Specialists can better meet their needs for information and forecasting of pests.
Publications
- Stafne, E. T. & Kelsey, K. D. (2011). Perceptions of land-grant university specialists regarding competition and collaboration with community college programs in viticulture and enology. HortTechnology, 21(6), 789-796. Available: http://horttech.ashspublications.org/content/21/6/789.fullijkey=bVlH 1iTdUxhG7h0&keytype=ref
- Roberts, G. T., Barrick, R. K., Dooley, K. E., Kelsey, K. D., Raven, M. R., & Wingenbach, G. J. (2011). Enhancing the quality of manuscripts submitted to the Journal of Agricultural Education: Perceptions of experienced reviewers. Journal of Agricultural Education, 52(3), 1-5. Doi: 10.5032/jae.2011.03001
- Kelsey, K. D., Lin, H. & Franke-Dvorak, T. C. (2011). A longitudinal study to determine if wiki work builds community among agricultural adult education students. Journal of Agricultural Education, 52(2), 1-10. Doi: 10.5032/jae.2011.02001.
- Fabregas-Janeiro, M. G., Kelsey, K. D., Robinson, J. S. (2011). Assessing changes in intercultural sensitivity among agricultural students exposed to international experiences. Journal of International Agricultural and Extension Education 18(1), 34-44. DOI: 10.5191.jiaee.2011.18103 Available: http://www.aiaee.org/vol-181-spring-2011/499-assessing-changes-in-int ercultural-sensitivity-among-agricultural-students-exposed-to-interna tional-experiences.html
- Roman, T., Kelsey, K. D., & Lin, H. (2010). Enhancing online education through instructor skill development in higher education. Online Journal of Distance Learning Administration, 13(4). Available: http://www.westga.edu/~distance/ojdla/winter134/roman_kelsey134.html
- Stafne, E. T. & Kelsey, K. D. (2011). Perceptions of land-grant university Extension viticulture specialists regarding competition and collaboration among land-grant university cooperative extension and community college viticulture programs. Oklahoma State University, Stillwater, June 17, 2011.
- Kelsey, K. D. (2011). Development of a grape community of practice (CoP) for the eXtension system: Evaluation report year two. Funded by the United State Department of Agriculture. Oklahoma State University, Stillwater, June 17, 2011.
|
Progress 10/01/09 to 09/30/10
Outputs
OUTPUTS: The objective this project was to introduce program planners, implementers, and stakeholders to two evaluative practices 1) articulation and documentation of program theory (logic modeling), and 2) mainstreaming evaluation through the transfer of evaluation technologies using participatory action research (PAR) techniques. Results include 1) clear documentation of program inputs, activities, participation, outputs, and outcomes (short, medium, and long term); and 2) program planners ability to plan and execute formal evaluation for program improvement (formative) and accountability (summative). Important strides were made toward project goals during 2010. One major evaluation project was undertaken using a new evaluation approach, M. Q. Patton's Developmental Evaluation (2011). A USDA funded project was secured, Development of a Grape Community of Practice for the eXtension System (Eric Stafne, OSU, Lead PI). Kathleen Kelsey, Co-PI and lead evaluator undertook a developmental evaluation approach to determine the following: 1) how are interactions among the community of practice (CoP) unfolding 2) What is the collaboration accomplishing 3) How are the collaborators working together 4) How do collaborators see themselves and their shared effort in regard to degrees of engagement 5) Did the aspirations of the collaboration emerge Results of the evaluation found that the aspirations of the Grape CoP have been realized. The project was well organized, well managed, and all goals for year one had been achieved. The founding team envisioned a national repository for viticulture information. The Grape CoP has become a tangible product (http://cop.extension.org/wiki/Grapes), the infrastructure is in place to grow the repository, and a true community of practice has emerged for members to benefit from the CoP professionally. The evaluator worked closely with clients and stakeholders to conduct formative evaluation for the purpose of program improvement. The clients were included in the evaluation planning and data collection phases to disseminate evaluation technologies. Clients were empowered to use evaluation data for program improvement and accountability purposes. Evaluation expertise was also shared with the larger community via workshops and evaluation reports. Evaluation reports were submitted to program planners for program improvement. PARTICIPANTS: Not relevant to this project. TARGET AUDIENCES: The national eXtension community, the viticulture industry PROJECT MODIFICATIONS: Not relevant to this project.
Impacts
The outcomes of program evaluation are multifaceted. This project practiced an emerging evaluation design, Developmental Evaluation (Patton, 2011), to support educative, values engaged evaluation methods resulting in timely and meaningful feedback to program decision makers to reduce uncertainty regarding planning and implementing programs. The results of the evaluation project during the past year have provided feedback to program planners regarding Community of Practice formation and cohesion and readiness to engage in the task and enhanced understanding of the role and use of evaluation in Cooperative Extension System collaborative efforts, specifically eXtension.org.
Publications
- Kelsey, K. D. (2010). Development of a grape community of practice (CoP) for the eXtension system: Evaluation report year one. Funded by the United State Department of Agriculture. Oklahoma State University, Stillwater, August 1, 2010.
- Kelsey, K. D. & Bunch, J. C. (2010). Summative evaluation of the US-Brazil Higher Education Consortia Program: Consortium for global BIO energy knowledge exchange. Funded by the US Department of Education. Oklahoma State University, Stillwater, July 1, 2010.
- Franke, T. C., Kelsey, K. D., & Royer, T. (2010). Is extension still reaching stakeholders An assessment of minor crop producers educational needs. Journal of Agricultural Education, 51(1), 56-64. Available: http://jae.fp.expressacademic.org/article.phpautoID=198&issueID=68
- Kelsey, K. D. Stafne, E., Greer, L., & Attaway, D. (2010). Creating community cohesiveness and examining barriers to working differently: A discussion of the all about blueberries and grape CoP experiences in eXtesnion. Critical Conversations: A Symposium in Working Differently. 2010 National eXtension Virtual Conference, October 18-20, 2010.
- Stafne, E.T., Hellman, E., Striegler, R.K. and Kelsey, K. D. (2010). Formation of an eXtension grape community of practice. HortScience 45:490 (abstr.)
|
Progress 10/01/08 to 09/30/09
Outputs
OUTPUTS: The objective this project was to introduce program planners, implementers, and stakeholders to two evaluative practices 1) articulation and documentation of program theory (logic modeling), and 2) mainstreaming evaluation through the transfer of evaluation technologies using PAR techniques. Results include 1) clear documentation of program inputs, activities, participation, outputs, and outcomes (short, medium, and long term); and 2) program planner's ability to plan and execute formal evaluation for program improvement (formative) and accountability (summative). Significant progress was made toward project goals during 2009. Four evaluation studies were conducted using the educative evaluation paradigm. They included 1) a study of using wikis to foster collaborative learning to determine the pedagogical potential of Web 2.0 tools; 2) evaluating the sustainability of externally funded projects for persons with developmental disabilities for the Oklahoma Developmental Disabilities Council; 3) evaluating producers' stake in the bioeconomy by determining Oklahoma crop producers' knowledge and willingness to grow dedicated biofuel crops; and 4) evaluating the role of Cooperative Extension in serving stakeholders by assessing minor crop producers' educational needs. The PI worked closely with clients and stakeholders to conduct formative evaluation for the purpose of program improvement. The clients were included in the evaluation planning and data collection phases to disseminate evaluation technologies. Clients were empowered to use evaluation data for program improvement and accountability purposes. Evaluation expertise was also shared with the larger community via workshops, journal articles, and evaluation reports. Evaluation reports were submitted to program planners for program improvement. Funding was secured to support evaluation efforts from the Centennial Scholars Program, Oklahoma Agricultural Experiment Station. Collaborative effort with Dr. Tom Royer, Entomology and Plant Pathology to fund a Ph.D. student for 3 years to develop a needs assessment for minor crops industries ($60,000), also by a grant from the Oklahoma Developmental Disabilities Council Evaluation of Program Processes and Outcomes ($49,577.00). PARTICIPANTS: PI/PD: Kathleen Kelsey, Professor, Oklahoma State University Research Associate: Tanya Franke, Graduate student, Oklahoma State University Collaborators: Tom Royer, Professor, Entomology; Hong Lin, Oklahoma State University TARGET AUDIENCES: Oklahoma State University staff and faculty responsible for developing research and extension education materials. Oklahoma minor crop producers. National Cooperative Extension leaders. PROJECT MODIFICATIONS: Not relevant to this project.
Impacts
The outcomes of program evaluation are multifaceted. These projects practice educative, values engaged evaluation methods resulting in timely and meaningful feedback to program decision makers to reduce uncertainty regarding planning and implementing programs. The results of the evaluation projects undertaken during the past year have been used to 1) develop Extension Education trainings for producers regarding integrated pest management best practices and for reporting purposes to government agencies, 2) enhance understanding of the impact of government mandates on Cooperative Extension reporting requirements, 3) plan appropriate pedagogy for using wiki technology for collaborative writing assignments in graduate education, 4) refine and improve programs targeted toward persons with developmental disabilities, 5) enhance international exchange programs, and 6) improve extension workshops delivered to commercial greenhouse growers in Oklahoma.
Publications
- Lin., H. & Kelsey, K. D. (2009). A case of using wikis to foster collaborative learning: Pedagogical potential and recommendations. In Inoue, Y. (Ed.), Cases on Online and Blended Learning Technologies in Higher Education: Concepts and Practices. Hershey, PA: IGI Publishing.
- Lin, H., & Kelsey, K. D. (2009). Building a networked environment in wikis: The evolving phases of collaborative learning in a wikibook project. Journal of Educational Computing Research, 40(2), 141-165.
- Kelsey, K. D. & Franke, T. C. (2009). The producers stake in the bioeconomy: A survey of Oklahoma producers knowledge and willingness to grow dedicated biofuel crops. Journal of Extension, 47(1). Article Number 1RIB5.
- Kelsey, K. D., Lin, H., & Franke, T. C. (2009). An evaluation of wiki as a tool for building communal constructivism in a graduate-level course. American Evaluation Association, Orlando, FL, November 14, 2009
- Kelsey, K. D., Lin, H., & Franke, T. C. (2009). Assessing wiki as a tool for building communal constructivism in a graduate-level course. North American Colleges and Teachers of Agriculture Conference, Stillwater, OK, June 17-20, 2009
- Kelsey, K. D., Lin, H., & Franke, T. C. (2009). Is wiki the right tool for building communal constructivism in a graduate-level course Oklahoma Distance Learning Association, Oklahoma City, April 16, 2009
- Franke, T. C., Rucker, K. J., Krysher, S., & Kelsey, K. D. (2009). Are projects for persons with developmental disabilities sustainable beyond initial external funding Paper presentation. Oklahoma State University Research Symposium, Stillwater, Oklahoma. February 20, 2009.
- Franke, T. C., Kelsey, K. D., & Royer, T. (2009). Is extension still reaching stakeholders An assessment of minor crop producers education needs. AAAE-Southern Region Research Conference, Atlanta, GA, January 31-February 3, 2009.
|
|