Progress 09/01/09 to 08/31/11
Outputs
OUTPUTS: CYFERnet SEARCH finalized CYFERnetSEARCH.org website. In addition to the innovative logic model and survey builders, the site houses interactive learning modules, CYFAR Common Measures, additional evaluation instruments vetted by CYFERnet Evaluation, video resources, and an evaluation glossary. Website content was finalized in including the development of an interface for SMART clicker software and a data upload feature. The CYFERnet Evaluation team developed a new theme for the webiste and revised the look of the evaluation videos housed there. CYFERnet SEARCH collaborators at North Carolina State University completed a comprehensive literature review on life skills, including a review of research and practice to develop a framework for life skills programming, and a review of existing measures of life skills. A final report was submitted in January 2011. NCSU also identified core elements of youth development programming, with an emphasis on the 4-H club model. A final report and executive summary was submitted in January 2011. Collaborators at Texas A&M also identified survey items related to life skills components. These items were integrated into a draft survey administered with 4-H centers. A final report was submitted in January 2011. CYFERnet SEARCH completed to two program quality assessment pilot studies. The first, by collaborators at the University of Minnesota, was to test whether or not youth and adult volunteers could conduct observations and planning with data sessions at local 4-H clubs. A final report was submitted in January 2011. UMN also developed a database of program quality assessment tools and conducted a literature review on current trends in assessing program quality. The final database and 3 research briefs were submitted in January 2011. UMN also hosted a two-day seminar, Pathways to Impact: The Next Generation of Ideas, December 2-3, 2010. This event was designed to advance new ideas about youth programs through quality and youth outcomes data. A report of the event proceedings was also submitted in January 2011.The Evaluation Team at the UA also completed a pilot study implementing a program quality tool. The study was conducted in four Arizona counties. The pilot study also integrated the use of clickers to test the use of this data collection method with youth populations. County-level reports were sent in February 2011. CYFERnet SEARCH.org resources have been presented via webinar to CYFAR grantees, including the 2012 CYFAR NEW SCP RFA Informational Webinar and the CYFAR 2011 Sustainable Community Projects Orientation and CYFAR Liaison Meeting. CYFERnet Evaluation attended conferences (2011 Forging the Partnership Conference, NCFR, AEA, Governors Community Service Conference) to promote visibility of CYFERnet SEARCH. Most notably, the website is now the platform for the implementation of the CYFAR Common Measures Data Collection process and all CYFAR grantees funded in 2011 and forward will be using the site to conduct program evaluation at sites nationally. The project is now concluded and future development and maintenance of the website will be managed by CYFERnet Evaluation PARTICIPANTS: Individuals who worked on this project include Lynne Borden, PhD, the principle investigator, Christine Bracamonte Wiggs, MPH, MA, the co-PI, Amy Schaller, MA, Evaluation Specialist, Bryna Koch, MPH, Evaluation Specialist, Ken Molnar IT Specialist, and Robert Lanza, IT Director. Dr. Borden and Ms. Bracamonte Wiggs provided oversight and direction of project deliverables. Ms. Schaller and Ms. Koch coordinated program activities and provision of technical assistance. Mr. Molnar and Mr. Lanza coordinated and completed development of website design and technology interface(s) with the website. Additionally, collaborators included Lydia Marek at Virginia Tech (identification and vetting of common evaluation measures), Dale Safrit, North Carolina State University(identification of Core Elements for 4H Youth Development and Children, Youth and Families At Risk (CYFAR) Programs), Ben Silliman, Ph.D., North Carolina State University (Life Skills Research Review), Mat Duerden and Peter A. Witt, Texas A&M University (CYFAR Life Skills Measurement Study), Dale Blythe & Deborah Moore, University of Minnesota (program quality improvement study and literature review). TARGET AUDIENCES: The intended target populations for the work of CYFERnet SEARCH are CYFAR grantees as well as the general public. PROJECT MODIFICATIONS: Not relevant to this project.
Impacts
Efforts of the CYFERnet SEARCH project enabled a better understanding of the work being conducted within its programs and provided standardized procedures and protocols for assessing program quality and implementing cross-site program evaluation. These efforts increased the data available regarding the impact of CYFAR programs system-wide. To date, CYFAR grantees (and prospective grantees), as well as the general public, have participated in conferences and webinars that have increased their understanding of the resources offered by CYFERnetSEARCH.org. As such, CYFAR Common Measures and other evaluation capacity building resources found on the website are currently available for use within the CYFAR initiative, as well by the general public. The development of a content neutral program quality tool for widespread use is now available to CYFAR grantees and is part of the CYFAR Required Common Measures. Additionally, the project has made contributions to the field of youth development, including exploration of the integration of technology to data collection with youth, which is now being implemented within the CYFAR initiative, the development of a life skills framework, identification of core elements in 4-H programming, and program quality instrumentation.
Publications
- Borden, L., Bracamonte Wiggs, C., Schaller, A. (2011, November). Evaluating Impact at the Systems-Level: Implementing the First Cross-Site Evaluation Within the Children, Youth, and Families At Risk (CYFAR) Initiative. Session accepted for the 25th Annual Conference of the American Evaluation Association. Anaheim, California.
- Borden, L., Bracamonte Wiggs, C., Schaller, A. (2011, November). Evaluating Systems-Level Impact: Implementing Common Evaluation Measures. Poster accepted for the 73rd Annual National Council on Family Relations Conference. Orlando, Florida.
- Borden, L., Bracamonte Wiggs, C., Koch, B., & Schaller, A. (2011, October). Important Evaluation Tools: Logic Models, Surveys & Technology. Webinar presented as part of the 2012 CYFAR NEW SCP RFA Informational Webinar. Tucson, Arizona.
- Schaller, A., Borden, L., Bracamonte Wiggs, C., & Koch, B. (2011, July). Going Places with Evaluation: Exploring the Interactive Resources at CYFERnetSEARCH.org. Workshop presented at the Governors 17th Annual National and Community Service Conference. Flagstaff, Arizona.
- Schaller, A., Koch, B., Borden, L., & Bracamonte Wiggs, C. (2011, July.) CYFERnet Clicker Application Training: CYFAR Common Measures Webinar. Tucson, Arizona.
- Schaller, A., Koch, B., Borden, L., & Bracamonte Wiggs, C. (2011, July.) CYFERnetSEARCH.org: Interactive Tools to Support CYFAR SCP Program Evaluation. CYFAR Common Measures Orientation for Opt-In Grantees Webinar. Tucson, Arizona.
- Borden, L., Bracamonte Wiggs, C., Koch, B., & Schaller, A., (2011, June). CYFERnetSEARCH.org: Interactive Tools for Enhancing Program Evaluation. Webinar presented for University of Nebraska Extension. Tucson, Arizona.
- Borden, L., Bracamonte Wiggs, C., Schaller, A. & Koch, B. (2011, June). CYFERnetSEARCH.org: Interactive Tools to Support CYFAR SCP Program Evaluation. Session presented at the CYFAR 2011 Sustainable Community Projects Orientation and CYFAR Liaison Meeting. Alexandria, Virginia.
- Schaller, A., Koch, B., Borden, L., & Bracamonte Wiggs, C. (2011, April). Identifying and Using Common Measures to Support the Children, Youth, and Families At-Risk Initiative. Session presented at the 2011 Forging the Partnership Conference. Chicago, Illinois.
- Schaller, A., Koch, B., Borden, L., & Bracamonte Wiggs, C. (2011, April). CYFERnetSEARCH.org: Interactive Tools for Enhancing Program Evaluation. Session presented at the 2011 Forging the Partnership Conference. Chicago, Illinois.
- Schaller, A., Koch, B., Borden, L., & Bracamonte Wiggs, C. (2011, April). Engaging Youth in Evaluation: Using Clickers for Data Collection. Session presented at the 2011 Forging the Partnership Conference. Chicago, Illinois.
- Borden, L., Bracamonte Wiggs, C., & Schaller, A. (2011, February). CYFERnetSEARCH.org: Interactive Tools to Support Program Evaluation. Presentation made at the 7th Annual FCHS In-service Innovation, Collaboration, and Integration: A Time for Excitement. Tucson, Arizona.
- Schaller, A., Borden, L., Bracamonte Wiggs, C., & Schlomer, G. (accepted to the Journal of Youth Development, 2011). Engaging Youth in Evaluation: Using Clickers for Data Collection.
|
Progress 09/01/09 to 08/31/10
Outputs
OUTPUTS: CYFERnet SEARCH finalized the platform for the CYFERnetSEARCH.org evaluation capacity building website. In partnership with Virginia Tech, an online Logic Model Builder was designed using the CYFAR SCP logic model template as a framework. After CYFERnet Evaluation had vetted and adopted 7 CYFAR Common Measures, those measures were integrated into the online Survey Builder, which also includes measures to assess Participant Demographics, Core Competencies, Participation Level, and Program Quality. In addition to these evaluation resources, CYFERnetSEARCH.org houses interactive learning modules, additional evaluation instruments vetted by CYFERnet Evaluation, video resources, and an evaluation glossary. Website content will be finalized in Spring 2011. CYFERnet SEARCH collaborators at North Carolina State University began work on a comprehensive literature review on life skills, including a review of research and practice to develop a framework for life skills programming, as well as a review of existing measures of life skills. Additionally, NCSU began work on identifying the core elements of youth development programming, with emphasis on the 4-H club model. Collaborators at Texas A&M also began work on identifying survey items related to life skills components. These items were integrated into a draft survey for administration with 4-H centers. IRB approval for the study is pending and data collection is projected to begin in Fall 2010. Final reports of findings will be completed early 2011. CYFERnet SEARCH conducted to two program quality assessment pilot studies. The first, spearheaded by collaborators at University of Minnesota, enlisted 40 youth and adult in 4-H programming to become trained Quality Coaches from 50 4-H sites on the implementation of the YPQA tool. In addition, UMN developed a database of program quality assessment tools and conducted a literature review on current trends in assessing program quality. Final reports on the study will be completed early 2011. The Evaluation Team at the UA also coordinated a pilot study implementing a program quality tool developed by Borden and Perkins. The study was conducted in 4 counties (Apache, Coconino, Yavapai, and Yuma) and County Extension Agents were trained on data collection protocol. The pilot study also integrated the use of clicker technology to test the use of this data collection method with youth populations. A total of 143 youth and 38 adult program volunteers completed the pre-survey. Post-surveys will be conducted in the Summer/Fall of 2010 and a final report of findings will be completed Spring 2011. PARTICIPANTS: Lynne M. Borden PI -- University of Arizona Christine Bracamonte Wiggs Co-PI Amy Schaller Evaluation Specialist Dale Blyth U of Minnesota (subcontract) Deborah Moore U of Minnesota Ben Sillman North Carolina State University (subcontract) Dale Safrit Peter Witt Texas A & M (subcontract) Chris Bolemna Lydia Marek Virgina Tech (subcontract) TARGET AUDIENCES: Children, Youth and Families at Risk Project Directors and Evaluators PROJECT MODIFICATIONS: Nothing significant to report during this reporting period.
Impacts
Efforts of the CYFERnet SEARCH project will enable a better understanding of the work being conducted within its programs and provide standardized procedures and protocols for assessing program quality and implementing cross-site program evaluation. These efforts will increase the data available regarding the impact of CYFAR programs system-wide. To date, CYFAR grantees (and prospective grantees) have participated in conferences and webinars that have increased their understanding of the resources offered by CYFERnetSEARCH.org. As such, CYFAR Common Measures and other evaluation capacity building resources found on the website are currently available for use within the CYFAR initiative, as well by the general public. The development of a content neutral program quality tool for widespread is also available to CYFAR grantees. Additionally, the project has made contributions to the field of youth development, including exploration of the integration of technology to data collection with youth, which will be implemented within the CYFAR initiative in Summer 2011, the development of a life skills framework, identification of core elements in 4-H programming, and program quality instrumentation.
Publications
- No publications reported this period
|