Source: PENNSYLVANIA STATE UNIVERSITY submitted to
EVALUATION PLANNING FOR CHILD AND YOUTH DEVELOPMENT SYSTEM
Sponsoring Institution
National Institute of Food and Agriculture
Project Status
TERMINATED
Funding Source
Reporting Frequency
Annual
Accession No.
1004985
Grant No.
2014-39585-22638
Project No.
PENW-2014-09763
Proposal No.
2014-09763
Multistate No.
(N/A)
Program Code
CYSC
Project Start Date
Sep 1, 2014
Project End Date
Aug 31, 2017
Grant Year
2014
Project Director
Perkins, D. F.
Recipient Organization
PENNSYLVANIA STATE UNIVERSITY
408 Old Main
UNIVERSITY PARK,PA 16802-1505
Performing Department
Agricultural Economics, Sociol
Non Technical Summary
This project is designed to develop a rigorous, flexible, replicable, and executable plan to systematically and rigorously evaluate the Department of Defense (DoD) Child and Youth Development System: Evaluation Planning (CYEP). At the macro level, through a thorough review of scientific literature, we will identify relevant theory and research to develop an overarching logic model for CYEP (i.e., How does the System work to enhance the health and well-being of military children and youth?). Evaluation plans will then be created for the overall System and for its individual programmatic areas (e.g., child development centers, school-aged child care, youth recreation and sports, teen centers). At the micro or local level, each of the individual programmatic areas will be provided a choice of clearly delineated evaluation approaches for their specific programs. The level of evaluation rigor will fall along a continuum from acceptable to robust, with increasing resources required to move from an acceptable evaluation to a robust evaluation. Evaluation approaches will include design, data collection, measurement, and data analytic options. The Clearinghouse will also develop pre-plans for a System wide monitoring and evaluation program (i.e., the ability to collect system wide and local data that can be used for program management at the national and local level). This evaluation plan will promote continuous quality improvement at the System, programmatic area, and specific program levels to improve both family and military readiness.
Animal Health Component
0%
Research Effort Categories
Basic
(N/A)
Applied
100%
Developmental
(N/A)
Classification

Knowledge Area (KA)Subject of Investigation (SOI)Field of Science (FOS)Percent
80260203070100%
Goals / Objectives
Goal 1: Establish a DoD Child and Youth Development System, system-wide theory of change that takes into account individual system components (e.g., child development centers, school-aged child care, and teen centers), as well as their potential synergistic impacts.Goal 2: Develop programmatic area and individual system components (e.g., child development centers, school-aged child care, and teen centers) and their sub-programs (e.g., youth sports and fitness) logic models.Goal 3: Establish specific, measurable, attainable, relevant, and time-bound (SMART) outcomes.Goal 4: Develop a measurement model and evaluation plan options.
Project Methods
Clearinghouse research scientists will complete literature reviews. Literature will be reviewed, summarized, and synthesized to inform theory of change and logic model development. In-depth discussions with DoD Child and Youth Development System stakeholders (e.g., programmatic area leaders, program managers, and front line program staff) will also be conducted. Information from literature reviews and qualitative data analysis will be synthesized. The development of plans will follow a systematic review of design options, measurement possibilities, and possible waves of data collection.

Progress 09/01/14 to 08/31/17

Outputs
Target Audience:The target audience includes DoD and Service Project Manager for the Child and Youth Development Program. The development of the plans are to provide them with accurate information to decide how to implment a quality accountability assessment. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided? Nothing Reported How have the results been disseminated to communities of interest?We conducted a face-to-face report session with Child and Youth DoD Headquarter staff and the Services Child and Youth program managers. What do you plan to do during the next reporting period to accomplish the goals?This grant is ending. We are initiating the evaluation of the CDC under the new Clearinghouse grant.

Impacts
What was accomplished under these goals? All goals have been accomplised. Within the new Clearinghouse we are conducting the evaluation of CDC. The expectation is the other evaluations will be initiate in the next fisical year.

Publications

  • Type: Journal Articles Status: Accepted Year Published: 2017 Citation: Saathoff-Wells, T., Karre, J., Davenport, K., & Perkins, D. F. (In Press). Flexicution in program evaluation: Developing a two-tier plan to address diverse priorities across the four services intensive home visitation programming. Journal of Family Social Work.
  • Type: Other Status: Published Year Published: 2016 Citation: Davenport, K. E., Mathies, B. R., Karre, J. K., Kaye, M. P., & Perkins, D. F. (2016, September). Child and Youth Development Evaluation Plan Project: Child Development Centers (CDC) and Family Child Care (FCC) evaluation plan. University Park, PA: Clearinghouse for Military Family Readiness at Penn State.
  • Type: Other Status: Published Year Published: 2016 Citation: Karre, J. K., Davenport, K. E., Mathies, B. R., & Perkins, D. F. (2016, September). Report of work completed for the Child & Youth Development Evaluation Plan Project: Final report. University Park, PA: Clearinghouse for Military Family Readiness at Penn State.
  • Type: Other Status: Other Year Published: 2016 Citation: Mathies, B. R., Davenport, K., Karre, J. K., Kaye, M. P., & Perkins, D. F. (2016, September). Child and Youth Programs (Age 5 to 15) system-wide longitudinal evaluation: Evaluation plan. University Park, PA: Clearinghouse for Military Family Readiness at Penn State.
  • Type: Other Status: Other Year Published: 2016 Citation: Davenport, K. E., Mathies, B. R., Karre, J. K., & Perkins, D. F. (2016, May). Child and Youth Development Evaluation Plan Project evaluability assessment: Child Development Centers. University Park, PA: Clearinghouse for Military Family Readiness at Penn State.
  • Type: Other Status: Published Year Published: 2016 Citation: Davenport, K. E., Mathies, B. R., Karre, J. K., Kaye, M. P., & Perkins, D. F. (2016, September). Child and Youth Development Evaluation Plan Project: School Age Care (SAC)-after school evaluation plan. University Park, PA: Clearinghouse for Military Family Readiness at Penn State.


Progress 09/01/16 to 08/31/17

Outputs
Target Audience:We provide headquarter staff the final evaluation plans in November 2016. We are awaiting the implementation work. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided?We meet quarterly with Headquarter CY staff and Service-level CY Program Managers. How have the results been disseminated to communities of interest?The reports have been shared with all stakeholders and discussion about implementation are underway. What do you plan to do during the next reporting period to accomplish the goals?We hope to have a status of implementation available.

Impacts
What was accomplished under these goals? All the original goals have been addressed. We are currently in dicussion related to the implementation of evaluation plans.

Publications


    Progress 09/01/15 to 08/31/16

    Outputs
    Target Audience:The target audience for the development of the logic models and evaluation plans is the Children and Youth Program Managers within DoD and the Services. Once plans are agreed upon by the PMs, then we will begin collecting data with the goal of evaluating the program. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided?We have hosted two webinars to present the logic models and the SMART Measures. How have the results been disseminated to communities of interest?Yes, we have instituted a iterative process that enables partners to provide feedback and insights for the development of all products. What do you plan to do during the next reporting period to accomplish the goals?We expect to have evaluation plans finalized. We then expect to begin to implementin g the plans.

    Impacts
    What was accomplished under these goals? For goal 1, the system-wide theory of change has been developed, refined with feedback from partners, and finalized. For goal 2, the categorization of the various sub-programs within the larger program areas has been completed across all four Services including comparison across all for common programming efforts. Goal 3 of SMART goals was finalized. Goal 4, the development of evaluation plans, is currently underway.

    Publications


      Progress 09/01/14 to 08/31/15

      Outputs
      Target Audience:The target audience includes DoD and Service program managers for the Child and Youth development Program. The development of plans are to providede them with accurate information to decide how to implement a quality accountability assessment. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided?We conducted a face-to-face feedback session with all project managers about logic models and about the specific program components. How have the results been disseminated to communities of interest?The results have been shared with the DoD and Service project mangers and they have just provided their second round of feedback. Once financial the research materials will be shared by the Service project managers to their staff. What do you plan to do during the next reporting period to accomplish the goals?We will working on an evaluability plan of the CDP. this includes all day programs for children 0-5 as well as all-day programsfor children 6-12.

      Impacts
      What was accomplished under these goals? Goal 1 has been accomplished. Goal 2 is currently underway. Progression has been slowed down due to the complex nature of the material. and the number of stakeholders whom review the information being created. Goals 3 and 4 will be the major task of this up coming year.

      Publications