Source: CORNELL UNIVERSITY submitted to NRP
EVALUATING THE NEW YORK STATE SCHOOL AGE CARE CREDENTIAL
Sponsoring Institution
National Institute of Food and Agriculture
Project Status
COMPLETE
Funding Source
Reporting Frequency
Annual
Accession No.
0198518
Grant No.
(N/A)
Cumulative Award Amt.
(N/A)
Proposal No.
(N/A)
Multistate No.
(N/A)
Project Start Date
Oct 1, 2003
Project End Date
Sep 30, 2006
Grant Year
(N/A)
Program Code
[(N/A)]- (N/A)
Recipient Organization
CORNELL UNIVERSITY
(N/A)
ITHACA,NY 14853
Performing Department
HUMAN DEVELOPMENT
Non Technical Summary
The New York State School Age Care Credential is a new in-service credential for school age staff. Now in its fourth year of implementation, New York is further ahead than any other state attempting to create such a system for training school-age staff. This project will evaluate the effect of the SACC on the quality of school age programs and school age staff professional development.
Animal Health Component
100%
Research Effort Categories
Basic
(N/A)
Applied
100%
Developmental
(N/A)
Classification

Knowledge Area (KA)Subject of Investigation (SOI)Field of Science (FOS)Percent
8026010302025%
8026010307025%
8026050302050%
Goals / Objectives
This project will evaluate the recently developed New York State School Age Care Credential (SACC) program for school-age care providers. It expands upon current work conducted by the Cornell Early Childhood Program (CECP) to document the history, development, and program delivery of the SACC throughout New York State. The specific objectives of this project are threefold. First, the project will examine the impacts of the SACC on the quality of school age care programs. We will begin by examining how the overall quality of school-age care programs is affected when individual staff participate in the New York State School Age Care Credential. We will also examine whether particular aspects of programs (e.g. structural vs process quality indicators) improve more than others as a result of staff participation in the SACC. In addition, we will explore whether parents and children participating in after-school programs are more satisfied with their school-age care experiences when staff earn their SACC. Our second objective is to document how participation in the SACC affects staff professional development. Specifically, do SACC recipients show changes in knowledge about school-age child development and education practices? How does SACC participation affect professional expectations, attitudes, plans for future training, and retention in the field of school-age care? Efforts related to the first two objectives will take place in project years one and two. Finally, this project aims to disseminate evaluation findings to multiple audiences. As the credential is a new program, we expect that developers of the program will be very interested in feedback about how the SACC impacts program quality. We expect that feedback to the SACC development team will be ongoing and involve periodic summary reports and presentations, as well as informal communications. The results of the SACC evaluation will also be of interest to those in the NYS school-age community, including individual school-age staff, SACC host agencies, and state-level policy makers. To reach these audiences, we will engage in a variety of dissemination activities, including presentations at state-wide conferences. We also plan to feature the SACC evaluation in research and policy briefs developed by staff at CECP. The presence of Cornell Cooperative Extension staff on the SACC evaluation team will enable us to efficiently distribute these briefs through the Cooperative Extension system state wide. In addition, we plan to make them available to a wider audience on the CECP website. Finally, results from this evaluation will be of interest to a national audience. A few states around the country have begun the process of developing their own versions of a school-age care credential, yet none is as far along or has trained as many SAC staff as has the New York State program. Thus we plan to share our findings nationwide through scholarly publications, presentations at national conferences, and summary reports that will be available on line (through the CECP website) and distributed through the nation wide Cooperative Extension system.
Project Methods
This SACC program evaluation will involve a quasi-experimental design to compare school-age professionals who participate in the SACC with a second group of individuals, matched as closely as possible for program (e.g. size) and individual (e.g. race/ethnicity, training, education) characteristics, but who do not participate in the SACC. This design maximizes sample size with minimal burden to this newly developed program by not artificially creating waitlists (as would be necessary in a true randomized study). Data collection will take place in two phases. In Year 1, data collection efforts will focus on two upstate NY cities. Year 2 will replicate data collection for Year 1 (with revisions as necessary based on Year 1 findings) in two additional NY cities (funding from other sources may allow us to expand to additional cities Year 2). In both years we will use a pre- post-test design to gather information about SACC (N=60; 30 in year 1 and 30 in year 2) and comparison group (N=60; 30 in year 1 and 30 in year 2) participants and their programs. Questionnaires administered to site supervisors will focus on structural features of program quality (e.g. child to staff ratios, program flexibility). We will also survey parents and children regarding their satisfaction and experiences with their school-age program at the beginning and end of the evaluation period. Finally, we will conduct observational assessments of program quality. Trained observers (likely drawn from a pool of SAC Quality Advisors in NY state) will use well established measures such as the School Age Child Environment Rating Scale (Harms, Jacobs, & White, 1996) and time samples of staff-child interactions (Vandell & Posner, 1995) to gather this program quality information. Observer neutrality will be maintained by ensuring that observers do not know the school age staff under observation (and thus do not know status of SACC participation). To examine how the SACC credentialing process affects those individuals earning their credential, school-age staff in program and comparison groups will be asked to complete short questionnaires designed to assess how school-age professionals learn and change over time (e.g. do they demonstrate increased knowledge about school-age child development and developmentally appropriate practices?). Questionnaires will also capture school-age care professionals' demographic information, training and education background, and attitudes and plans regarding their work with school-age children. Those in the credential group will also answer questions about their experiences with the SACC process (a questionnaire capturing some of this information is already administered to SACC candidates as part of the credential process). In order to evaluate the effect of the SACC on program quality and teacher development we will use a variety of methods including descriptive statistics, regression analyses, and hierarchical linear modeling. These analyses will begin in Year 1 and continue in Years 2 and 3 of the project. Year 3 will also involve a concerted effort to disseminate the results of the study to practitioners, policy makers, and academic audiences.

Progress 10/01/03 to 09/30/06

Outputs
This study aimed to evaluate the impact of the New York State School Age Care Credential on school age care program quality and providers professional development. To accomplish these goals, 2 primary methods of data collection were used. First, observations of program quality were conducted using the School Age Care Environment Rating Scale, the Arnett Caregiver Interaction Scale and a qualitative scale of program quality. Seven data collectors were trained until they were at least 80 pct reliable on observation instruments with the project manager or training assistant. Pre- and post observations were then conducted in New York City and Albany with 37 SAC providers in 2004-2005. Twenty-two of the observed providers were enrolled in the SAC Credential program and 15 served as a comparison group who were not currently enrolled. Second, questionnaires were designed and distributed to SAC Credential program participants between 2002 & 2006. A total of 432 questionnaires (pre n=291; post n=141) were gathered from SAC Credential participants throughout NY State. Questionnaires covered topics such as provider demographics, beliefs and attitudes about caring for children, knowledge about child development, and knowledge of and participation in the SACC program. Results from this work show that the typical participant in the SAC Credential program is female (86 pct), White (56 pct) and a Program Director or Site Coordinator (61 pct). About 45 pct of the school age care providers in the SAC Credential program are over age 40. Educational background varies with about 1/3 (31 pct) having a high school diploma, 46 pct with at least some college, and 21 pct with a college or graduate degree. Preliminary findings indicate that the NYS SACC program has a significant impact on school age care providers and program quality. Specifically, data from the first few years of the evaluation show that about 40 pct of providers received a raise as a result of earning their credential. Of those who received a raise, 46 pct received a raise of at least $1. A little less than 1/3 of credential participants also changed job titles or took on new responsibilities (31 & 28 pct, respectively) as a result of becoming a Credential recipient. Nearly all participants (98 pct) would recommend the credential program to a SAC colleague. Preliminary findings also suggest that the NYS SACC training program has a positive impact on program quality. Specifically, overall SACERS scores jumped from 3.6 to 4.2 (with 3 being minimal and 5 being good quality) for the program participant group. The most significant changes were made on the space (p less than .01), activities (p less than .05), interaction (p less than .05) and program structure (p less than .05) subscales of the SACERS. In the comparison group, overall SACERS scores remained essentially unchanged (3.2 & 3.3 at Times 1 & 2, respectively). SACERS Subscale scores for the comparison group also stayed about the same from time 1 to time 2. These findings should be interpreted with caution, however, as they are based on a small sample of SAC providers to date. A research brief to disseminate these findings is under development.

Impacts
Findings from this evaluation of the New York State School Age Care Credential Program suggest that this intensive new training program can have a positive impact on professional development and program quality. Specifically, upon receiving their credential, almost half of providers received a raise and nearly 1/3 changed their job title or took on new responsibilities. In addition, observational data suggest that program quality increases when providers participate in the credential training program. These increases are evident in the physical space, structure of a program, and teacher-child interactions. Significant differences were seen in each of these areas in a group of SAC Credential participants, but not in a comparison group of similar school age care staff not enrolled in the SAC Credential program. These findings are relevant to school age care front line workers who may participate in the Credential program in the future. They are also helpful for school-age care trainers who continue to develop and implement the program. In addition, states or communities who may be developing similar training programs will find these results of use. Finally, these results are important for policy makers who develop and oversee regulations and funding for this kind of training opportunity. Ultimately, these results help to shape training opportunities for school age care providers, which in turn, lead to high quality programming that supports the optimal development of school age children.

Publications

  • No publications reported this period


Progress 01/01/05 to 12/31/05

Outputs
This project terminated on 12/31/05. The credential was developed, field-tested, and evaluated. Responsibility for maintaining the credential has been transferred to the New York State School-age Care Coalition. An article summarizing evaluation findings is currently in preparation (2006).

Impacts
The School-age Care Credential has been completed by over 150 professionals in New York over the past several years. New York is the only state to have developed such and educational credential, which has the potential to improve the quality of care in New York significantly. Whether this happens depends on the effectiveness of the NYS School-age Care Coalition and cooperation by the NYS Office of Children and Family Services.

Publications

  • No publications reported this period


Progress 01/01/04 to 12/31/04

Outputs
In the fall of 2003 and early spring 2004 measures were selected for conducting observations of school age care program quality. Selected measures include the School Age Care Environment Rating Scale, the Arnett Scale of Caregiver Interactions, and a qualitative scale of program quality. Questionnaires were also designed for school age program staff, including those participating in the New York State School Age Care Credential (SACC) program and a comparison group of providers not currently enrolled in the SACC program. These questionnaires included questions about provider demographics, beliefs and attitudes about caring for children, knowledge about child development, and knowledge of and participation in the SACC program. Director questionnaires were also designed in order to gather additional information about structural features of program quality (e.g. staff turnover and salaries). Questionnaires were distributed to school age staff and directors in spring 2004. In May 2004, 2 data collectors were trained to conduct observations of school age staff and programs using the selected measures. Data collectors were trained until they were at least 80% reliable with the project manager. Observations were conducted of 13 school age staff while at work in their programs in Rochester New York. Four of these staff were participants in the SACC program and 9 served as a comparison group of providers not currently enrolled in the SACC program. Observations were conducted at the end of the school year in late May and early June, 2004. Based on this spring pilot work, revisions were made to data collection instruments and procedures in the summer of 2004. Preparations were made to train data collectors and gather observational and questionnaire data in Albany, New York City, Poughkeepsie, and Long Island in fall 2004. These preparations included securing training facilities and practice sites, hiring data collectors, and recruiting SACC program and comparison group evaluation participants.

Impacts
Results from the evaluation will be useful to developers of the SACC program as they make revisions and additions to the SACC program. In addition, those who train school age care providers (extension educators, child care resource and referral agencies, and state government offices providing funding for such training) will also find the results relevant to their efforts. Policy makers in New York and other states may use the results to inform state regulations and funding priorities in the area of school age care and education. For example, evaluation results documenting the prevalence of poor or mediocre school age programs may lead to stricter state regulations regarding training requirements for school age care providers. Similarly, evidence that participation in the SACC is associated with increased quality may encourage additional funding for expansion of the program. The goal of the SACC is to train school age care providers to nurture school age child development through the provision of safe, developmentally appropriate care experiences. Thus we expect school age professionals to change their education and care practices on the job in order to better meet these goals. Increased training will also likely lead to decreased turnover rates in the field of school age care. In turn, we anticipate that overall school age program quality will increase in local communities. Higher quality school age programs will promote better social and cognitive development in 6 to 12 year old children.

Publications

  • No publications reported this period