Progress 10/01/15 to 09/30/18
Outputs
Target Audience:The study uses Community Based Participatory Research methods which emphasize equitable and respectful engagementof participants and partners in conducting research which has practical benefits at the same time as contributing to theknowledge base of the field in which it is conceptualized. The youth involved in this study reflectthe diversity of youth involved in the NYC 4-H/Youth Development program. During FY2016, youth involved in the NYC 4-H Programparticipated infocus groups andworked to co-design survey items to be used as the project proceeded. The youth participantsrepresented the ethnic diversity of young people involvedin the NYC program and includedBlack/African-American, Asian, Hispanic, Native American, Pacific Islander and White young people, some of whom wereimmigrants. Most of the youth involved in the first round of focus groups were high school age.Focus groups were then conductedwith middle school youth during Spring 2017 to ensure that survey instruments were accessible to younger youth. Theinitial test-retest reliability activities during FY2017 engaged youth from sites that were primarily linked with Asian communities, so additionaltest-retest reliability work wereconducted during FY2018 to ensure that fulldemographic diversity informedthereliability assessments prior to administration of the survey to a larger number of youth later inFY2018.The research team workedwith the staff and leadership of NYC's 4-H/Youth Development program toplan focus group and survey administration in program sites. Discussions and planning meetings focusedonrecruitment and retention of youth participants and on identifying ways theemerging data can be used in program planning and improvement. Changes/Problems:A key staff member, the Research Coordinator, experienced a life threatening health emergency in January 2018. She returned to work part-time in August 2018. This had a substantial impact on the progress of the project. Although recruitment and data gathering continued, the pace slowed. Data cleaning and initial analysis were conducted during August and September, but more intensive data analysis, including Confirmatory Factor Analysis, was not completed prior to September 30, 2018. Planned abstracts for potential presentation at conferences were also deferred. Data analysis is continuing as of January 2019. When data analysis is complete, the planned conference abstracts and publications will be submitted. What opportunities for training and professional development has the project provided?This project continued to offer a unique opportunity for in-depth professional development and capacity development within the 4-H program team. An experienced research coordinator continued to work one-on-one with an entry level program assistant and with other 4-H staff on recruitment, retention and partnership development skills essential to successful applied research. How have the results been disseminated to communities of interest?4-H youth, including youth involved in the NYC 4-H Youth Leadership Academy, have been highly engaged in reviewing andco-designing the measures and have demonstrated strong commitment to the success of the project and to the disseminationand utilization of its results. Thisis evidenced by the ready recruitment of participants and the high retention rates. Many youth involved in the focus group and test-retestreliability activities actively maintain contact with project staff in order to participate in future activities and phases of the study. The project has also continued to work with 4-H staff and leadership to promote the benefits of applied research for programdevelopment and improvement. What do you plan to do during the next reporting period to accomplish the goals?
Nothing Reported
Impacts
What was accomplished under these goals?
The "Measuring Positive Youth Development Outcomes for Diverse Urban Adolescents" study built on the Complementary Strengths Research Partnership (CSRP), aCommunity Based Participatory Research project conducted in New York Citywith NIH and NIFA support initiated in 2005.Measurement instruments generated by the CSRP included the Tiffany-Eckenrode Program Participation Scale (TEPPS), complemented for this project by 4-H specific measures developedby RichardLerner and his team. The project emphasized measurement development - in particular,the participatory adaptation and validation of the individual items and scales in the TEPPS and Lerner measures, to ensure that they were valid, reliable, understandable and meaningful to the largely minority youth engaged in the NYC 4-H/Youth Development activities. Efforts during Year One of the project established the infrastructure needed to develop valid and reliable measures of positive youth development for use with diverse urban adolescents.The foundationalstages of the research processincluded engaging research support staffwith strong experience and credibility within the NYC 4-H program,obtaining IRB approval for eachphaseof the project (these included specific plans/protocols for focus groups with youth involved in 4-H,preliminary survey instruments to be reviewed and fine tuned through those focus groups, revised versions of the focus groups to be used in test-retest reliability and one time survey administration), and developinga realistictimeline andrecruitment plan for the focus groups, the test-retest reliability study,and subsequent project activities. The project's measurement development objectives were completed successfully. Work during YearOne concentrated on the participatory review and revision of the survey instruments.Work during Year Two of theproject continued tofocuson measurement development and ensuring that the measures were meaningful, understandable, valid and reliable when used with the very diverse youth involved in NYC 4-H/Youth Developmentprogramming. The research team workedin close partnership with the youth involved in the NYC 4-H/Youth Developmentprogramand in collaboration with program staff andcommunity partners. In total, 28youth participated in the focus groups and contributed to the review, adaptation, and development of the survey instrument fielded by the project. Approximately one third of the focus group participants were Asian, Black/African American and Hispanic/Latinx respectively. Slightly more than one third were middle school aged, with high school aged youth comprising the rest of the focus group participants. The majority of participants were girls/young women (29% male, 71% female). Adaptations to ensure quality and inclusivenesswere made when necessary. For example, thefirst series offocus groupsengaged primarily high school aged youth. Additional focus groups were organized in order toengageyounger (middle school aged)youth, though this meant that the total number of youth involved in focus groups exceeded the number initially proposed.This adaptation ensuredthat the candidate items and the layout/design of the survey were accessible and madesense to younger program participants as well as to older youth. After revisions resulting from the focus groups were incorporated into the measures, the survey instrument was again reviewed byyouth, many of whom had been involved in earlier focus groups and felt they had a stake in the project. This sense ofownership by the youth involved in the project may havecontributedto the ease of recruitment and retention noted over the course of the project. The number of youth involved in test-retest activities also exceeded the number projected in the initial proposal.Because the first wave of test-retest activities were implemented in program sites that served predominantly Asian youth, additional test-retest surveys were implemented to engage a more demographically diverse cohort of youth.Thus, thetest-retest reliability process continued into Year Three to ensure that adequate numbers of youth from alldemographics wereengaged in this critical phase of assessing the measures. A total of 56youth completed the initial "test" of thesurvey and returned two weeks later for the "retest". Initial assessments suggest that responses are stableover the two week interval and that most of the survey items arereliable. The adjustments necessary to ensure engagement of a strong age-diverse and ethnically representative sample of youth in focus groups and in test-retest reliability assessments resulted in slower, though steady, progress in meeting study objectives. The medical issues and extended leave experienced by theResearch Coordinator during Year Three slowed data analysis, implementation of pre/post surveys in program contexts,and recruitment of the larger number of youth anticipated to be engaged to address the project's second objective. Descriptive data analyses have been completed and factor analysis work has begun. Data analysis and preparations for "data dialogues" with 4-H/Youth Development program participants, staff and partners outlinedfor the project are continuing althoughthe project has termed. After the test-retest data gathering had been completed, a larger number of youth in 4-H programs throughout the city completed the survey. Of the 154 youth who participated in this phase of the study, 64% were female and 36% were male. Using a "check all that apply" approach to racial/ethnic identity,approximately one fourth identified as Hispanic/Latinx, one fourth as Asian, and half as Black/African American. 6% identified as Native American,6% as White and 3% as Pacific Islander. 12% offered a different, self-described ethnicity that did not fit the above categories. The average age of participants was 12.3 years (range 9-24 years). Confirmatory Factor Analysis (CFA) will likely be challenging because of the floor effect generated by the very positive skewof survey responses. For example, 93% of participantsendorsed statements regarding Connection,Care, and Character, 85% endorsed statements regarding Confidence, and 83% endorsed statements regarding Competence derived from Lerner's instruments. Results for statements derived from the TEPPS were only slightly more differentiated, with 76% of participants endorsing Community Engagement statements, 89% endorsing statements regarding a sense of Safety/Support within the program, 91% endorsing statements regarding Voice/Influence within the program, and 91% endorsing statements regarding opportunities for Personal Development provided by the program. Data analysis in addition to CFA will explore whether there are different patterns of response by participant subgroup. The study succeeded in meeting its fundamental measurement development objective, though it fell short of meeting aims to test the measures with a very large group of program participants. It demonstrated success in engaging a diverse set of participants and sustaining high retention rates. It laid the foundation for subsequent studies, and further analysis of the data generated during Year Three promises to offer insights for future real world program development,delivery and evaluation.Over the course of the study, youth participants were drawn from the full diversity of NYC 4-H/Youth Development programs, including in-school clubs, after school clubs for high school and middle school aged youth, summer day camps, clubs facilitatedby both youth and adult leaders, the Juntos intergenerational middle school program promoting educational attainment by Hispanic/Latinx youth, and internship programs. Further dataanalysis will examine potential differences among subgroups (for example, do afterschool clubs led by youth have different strengths compared to those led by adults?).
Publications
|
Progress 10/01/16 to 09/30/17
Outputs
Target Audience:The study uses Community Based Participatory Research methods which emphasize equitable and respectful engagement of participants and partners in conducting research which has practical benefits at the same time as contributing to the knowledge base of the field in which it is conceptualized.The youth involved in this study during this reportingperiodreflect the diversity of youth involved in the NYC 4-H/Youth Development program. The youth who completed focus groups and worked to co-design survey items were Black/African-American, Asian, Hispanic, and white. Focus groups were conducted withmiddle school youth during Spring 2017 to ensure that survey instruments were accessible to younger youth. The initialtest-retest reliability activities engaged youth from sites that were primarily linked with Asian communities, so additional test-retest reliability work will be conducted during FY2018 to ensure that additional demographic diversity informsthe reliability assessments. The research team works with the staff and leadership of NYC's 4-H/Youth Development program to plan focus group and survey administration in program sites. Discussions and planning meetings involving the research team and 4-H staff are geared tohighlighting attention to recruitment and retention of youth participants and identifying areas where the emerging data can be used in program planning and improvement. Changes/Problems:
Nothing Reported
What opportunities for training and professional development has the project provided?This project continued to offer a unique opportunity for in-depth professional development and capacity development within the 4-H program team. An experienced research coordinator continued to work one-on-one with an entry level program assistant and with other4-H staff on recruitment, retention and partnership development skills essential to successful applied research. How have the results been disseminated to communities of interest?4-H youth, including youth involved in the NYC 4-H Youth Leadership Academy, have been highly engagedin reviewing and co-designing the measures and have demonstrated strong commitment to the success of the project and to the dissemination and utilization of its results. This is evidenced by the ready recruitment of participants and the high retention rates so far. The project has also continued to work with 4-H staff and leadership to promote the benefits of applied research for program development and improvement. What do you plan to do during the next reporting period to accomplish the goals?Although the need to conduct additional focus groups with youngermiddle school aged youth andto conduct a final series of focus groups to review and confirm the revised surveys created delays in relation to the projected timeline, they contributed to the overall quality and sustainability of the research.Test-retest reliability data gathering and analyseswill be completed during the first months ofYear Three, enabling the use of the measures with the larger sampleof youth projected in the research plan. We anticipate surveying 300-600youth participants in NYC 4-H programs during spring and summer 2018.
Impacts
What was accomplished under these goals?
Work during Year Two of this project continued to focus on measurement development and ensuring that the measures are accessible, valid and reliable when used with the very diverse youth involved in NYC 4-H programming. This work continued to be carried out in close partnership with the youth involved in those programs and in close collaboration with staff and community partners. Earlier focus groups had engaged primarily high school aged youth. Additional focus groups engaged younger middle school aged youth to ensure that the candidate items and design of the survey were accessible and made sense to them. After suggested revisions were incorporated into the measures, the survey instrument was again reviewed by youth, many of whom had been involved in earlier focus groups and felt they had a stake in the project. This sense of ownership by the youth involved in the project may contribute to the ease of recruitment and retention noted so far. The test-retest reliability process will continue into Year Three to ensure that adequate numbers of youth from all demographics are engaged in this critical phase of assessing the measures. So far, 39 youth completed the initial "test" of the survey and 37 youth (95%) returned two weeks later for the "retest". Initial assessments suggest that responses are stable over the two week interval and thatthe items are quite reliable. There are a number of reasons the research team determined that it would be important to expandthetest-retest sample. Currently, two thirds of the participants in this phase of the study are young women, and there could be significant gender differences in the reliability of the measures though they are not apparent at this point. We also would like to engage more youth who identify as African American, Hispanic, and white.Highlighting changing demographics in some NYC neighborhoods and programs, approximately 72% of the youth involved in the test-retest process so far have been Asian. Thus, the research team decided to continue this phase of the project to engage youth from additional demographics. Full reliability analyses will be conducted after the additional data has been collected.
Publications
|