Progress 09/01/17 to 07/21/21
Outputs
Target Audience:This project support National Guard and Reserve units pre, during, and post deployment. Changes/Problems:
Nothing Reported
What opportunities for training and professional development has the project provided?The Clearinghouse conducted trainings with YRRP staff. Event Support / Training Virtual Event Planner Training (28 & 30 March 2020) Follow up meetings extended into April 2020 Preparation for and attendance at 13-14 Jan 2021 Event Planner Training Presentation on YRRP curriculum materials and components Preparation for and presentation on 26 August 2020 call with Reserve Component Coordinators Presentation on Next Generation Post-Event Survey capabilities How have the results been disseminated to communities of interest?All reports and materials were delivered to and uploaded on the YRRP website. What do you plan to do during the next reporting period to accomplish the goals?
Nothing Reported
Impacts
What was accomplished under these goals?
All the goals were addressed as outlined below. Curriculum (YRRP & TAP) 7 YRRP classes developed 5 online courses developed from existing face-to-face content 1 online course developed with new content (no previous content) 1 face-to-face course developed from existing content 1 TAP class developed online (Financial Planning for Transition) Review of content PowerPoint provided by TAP minus instructions for visuals, images, and icons Development and/or application of components to enhance existing TAP content Narration Edited, added, and solicited approval for changes Review of all audio modules, editing of audio, requests for editing of audio, hybridizing audio content to fill gaps Visual design Updated links 35 knowledge items Financial planning workbook in Excel Novel interactive elements based on ongoing communication with TAP Vetting calculators - Review of 3rd party calculators Screen captures of different inputs for purposes of providing examples of how to use the calculators 63 YRRP fillable and accessible handouts developed for use during virtual and/or online courses 15 YRRP course knowledge items developed for face-to-face and/or online courses 20 YRRP courses reviewed 35 YRRP course knowledge items reviewed for content alignment and relevance 9 Curriculum Working Group course reviews 11 SMEs identified for YRRP course reviews Employer Support of the Guard and Reserve (ESGR) 8 volunteer courses developed from existing content (New Member Intro; Privacy Act and Cyber Awareness; Ethics Training; Volunteer Recruitment; Volunteer Mentoring; Volunteer Management; Military Outreach; Employer Outreach) Review provided content (if applicable) - 8 courses Assess redundancy of content across and within sections Assess coherence of content across and within sections Literature/ evidence reviews - 8 courses Identify content in need of empirical support Identify articles relevant to the content in need of empirical support Review identified articles for methodological rigor Review article findings Verify alignment of content to evidence (adjust if necessary) Development of storyboard - 8 courses Apply Educational Psychology principles (e.g., skill practice, self-reflection, definitions, examples, theoretical model, task completion, optimal content structure) Organize content into coherent flow Update outdated content and resources Incorporate partner feedback Partner review meetings (review recommendations & requirements) - 7 courses Development of 5 knowledge items per course - 7 courses Development of course satisfaction item - 8 courses Ensure mutual exclusivity and exhaustiveness of response options, coherence of stem and response options Op test item with unfamiliar reviewer User acceptance testing (UAT) - 4 courses Ensure fidelity to the storyboard Ensure exercises function as intended Review grammar, flow, coherence Discuss with development team where clarification on recommendations is needed Review of live course - 4 courses Ensure anticipated functionality and fidelity to the UAT version Evaluation Data Integration Review of stakeholder feedback on the evaluation efforts Development of an automated reporting file 3 reports developed (Next Generation Post Event Survey Preliminary Report, YRRP Annual Report, Event Type Preference Report): Data cleaning Data analysis Data visualization Report development Online class knowledge item data Initial review of data and communication of inconsistencies and questions to partners United States Navy Reserve Individual Augmentee Longitudinal Study Tracking of participants and distribution of surveys Reports (Cohort 3 Update Report, Final Report - in-progress) Data cleaning Data analysis Data visualization Report development Summarization of findings - 1 report Formatting and editorial review - 1 report After Action Reports (AAR) Management of data (Qualtrics) Reviewed and revised the survey (Qualtrics) based on partner and Cadre feedback Review of feedback/comments Review of change submissions Investigated AAR submissions for frequency of class facilitations in Spanish speaking areas and by Cadre with Spanish as a second language Reports (FY20 Q2 report, FY20 Annual report) Status of Forces Survey-Reserve Component Reviewed current YRRP items in the survey Revised the YRRP items Met with Office of People Analytics to discuss revisions Post-Event Survey (Paper) Conducted quarterly quality checks of the automated reporting file - 3 checks Air Force Reserve Supplemental Analysis Data cleaning Confirming that the data cleaning from automated report file was conducted Data analysis Running requested analyses Descriptive statistics Data visualization Generation and formatting of charts Class Utilization Report Data analysis Analysis of class utilization from AAR data Analysis of class utilization from event attendance data Analysis of class utilization from online class data Data visualization Identifying optimal visualization type and components (e.g., chart/table type, layout, proper x- and y-axis anchor points and intervals to minimize biased interpretation, label positioning and color scheme for maximal readability, measures of central tendency and dispersion for interpretability) Generation and formatting of charts/tables USAR Regionality Survey Reviewed current survey and provided feedback Met with USAR to discuss feedback
Publications
|
Progress 09/01/19 to 08/31/20
Outputs
Target Audience:The target audience includes YRRP Center of Excellence staff as well as Service-level YRRP implementers. Changes/Problems:
Nothing Reported
What opportunities for training and professional development has the project provided?The following presentations have been made: -Curriculum enhancements. Presentation at the Event Planner training, New Orleans, LA. -Evaluating thinking for YRRP event planners. Presentation at the Event Planner training, New Orleans, LA. -Curriculum enhancements. Presentation at the Joint Working Group meeting, Alexandria, VA. -Data integration pilot. Presentation at the Joint Working Group meeting, Alexandria, VA. How have the results been disseminated to communities of interest?All reports and products (e.g., curriculum refinements, online resources, literature reviews) have been shared with target audiences. What do you plan to do during the next reporting period to accomplish the goals?substantial step in the direction in the measurement of program impact. Moroever, this year will continue to involve the multiple-year rotation for full content and reference reviews for YRRP curriculum. In addition, each class will undergo a formatting and update review to ensure hyperlinks are active and any revisions, recommended through feedback collected throughout FY20, are applied. Finally, the PES and AAR will continue to be analyzed and reported on a quarterly basis.
Impacts
What was accomplished under these goals?
72 subject matter experts vetted16 previously vetted resources across 2 domains reviewed For instance, we completed a literature review and 11 articles written for Army Guard & Reserve Homefront magazine.
Publications
- Type:
Other
Status:
Published
Year Published:
2019
Citation:
Balotti, B., Richardson, C., & Perkins, D. F. (2019). FY19 annual report on post-event survey data: October 1, 2018 to September 30, 2019. Report submitted to the Yellow Ribbon Reintegration Program Office. University Park, PA: Clearinghouse for Military Family Readiness at Penn State.
- Type:
Other
Status:
Published
Year Published:
2019
Citation:
Balotti, B., Richardson, C., Irvin, J., & Perkins, D. F. (2019). FY19 Q2 report on post-event survey data: January 1, 2019 to March 31, 2019. Report submitted to the Yellow Ribbon Reintegration Program Office. University Park, PA: Clearinghouse for Military Family Readiness at Penn State.
- Type:
Other
Status:
Published
Year Published:
2019
Citation:
Balotti, B., Weldon, B., Richardson, C., & Perkins, D. F. (2019). FY19 annual Yellow Ribbon Reintegration Program After Action report: October 1, 2018 to September 30, 2019. Report submitted to the Yellow Ribbon Reintegration Program Office. University Park, PA: Clearinghouse for Military Family Readiness at Penn State.
|
Progress 09/01/18 to 08/31/19
Outputs
Target Audience:The main audience for the work of this project is the National Guard and Reserve Members and their families who are support through the Yellow Ribbon Reintegration Program (YRRP). Changes/Problems:
Nothing Reported
What opportunities for training and professional development has the project provided?We have had quarterly professional training for YRRO HQ staff related to data interpretation How have the results been disseminated to communities of interest?Yes, all reports have been shared to YRRP HQ staff and Service-Level YRRP stafff What do you plan to do during the next reporting period to accomplish the goals? This proposed continuation of the multifaceted project addresses the YRRP's overarching goals of providing evidence-informed support for curriculum review/development, offering program evaluation, and recommending improvements. Currently, the Clearinghouse has staff placed in critical roles (e.g., research scientists, research assistants, learning design specialists) and has existing lines of communication and workflows with YRRP personnel, Service-level personnel, and contract personnel. As such, the Clearinghouse is equipped to continue activities and will work diligently to be successful throughout the process. Along with the ongoing efforts developed and continued in FY19, FY20 may also introduce an opportunity to evaluate the online resources effort. In addition, the development of automated data cleaning and analysis files, for the data integration effort, will be a substantial step in the direction of program sustainability and measurement of program impact. The coming year will be the beginning of a mutliple-year rotation for full content and reference reveiws for YRRP curriculum. In addition, each class will undergo a formatting and update review to ensure hyperlinks are active and any revisions, recommended through feedback collected throughout FY19, are applied. Throughout the upcoming year data collection for the USNR IA Longitudinal Study will continue. In addition, the PES and AAR will continue to be analyzed and reported on a quarterly basis.
Impacts
What was accomplished under these goals?
Over the past year, the Clearinghouse has continued to provide support to the YRRP in curriculum and program evaluation activities. One major curriculum initiative that occurred was the identification, noted by bolded text in the facilitator guide, and refinement of core content for all face-to-face classes. This two-stage process was a result of the prior year's multifaceted review of existing curriculum that noted concerns with the implementation of the face-to-face classes. Specifically, the implementation concerns noted a tendency to minimize skill practice, self-reflection, and task completion and to focus on didactic instruction, which was likely to the detriment of attendees (see, for instance, Prince, 2004). Content is considered core if it falls within one or more of the following evidence-informed categories: Theoretically important information Example: "By using 'I' statements instead of 'you' statements, choosing words wisely, and paying attention to nonverbal cues, we can avoid putting those with whom we are communicating on the defense... (Kubany, Bauer, Muraoka, Richard, & Read, 1995; Kubany, Richard, Bauer, & Muraoka, 1992)." Statements that attempt to normalize a stigmatized challenge Example: "Moral injury can result from a morally permitted and even morally praiseworthy act..." Statements that show how important constructs and/or relationships between important constructs exist in the real world Example: "[One way to expand your personal support network is to] Genuinely work to help others (without trying to get something in return) ... (Crocker & Canevello, 2008)" Activities that request audience participation to gauge their understanding of the concept and allow for greater depth of processing of the information (for empirical support of the importance of active engagement in learning see Prince, 2004) Example: "How might you change the following criticisms into complaints?" Statements that ask the audience to reflect on their life circumstances as a means of increasing self-awareness Example: "...think about how comfortable you are communicating in different situations." A core material checklist has been developed for each class to assist facilitators in identifying and presenting content and activities that have been determined to be most impactful for audience engagement and comprehension and in improving implementation fidelity. Categorizing core content is one step in a multistep approach to increasing implementation fidelity. In addition, discussions regarding monitoring curriculum implementation via audio/video aids and providing booster training when deviation is noted were held (Dunlap et al., 2000; Durlak, & Dupree, 2008; Humphrey, Barlow, & Lendrum, 2018; Miller-Day et al., 2013; Pettigrew et al., 2015). Another major task involved reviewing the curriculum adaptation of the face-to-face class materials to an online format. Once the classes were approved for use, the online development team at Affinity translated them for web-based consumption. The Clearinghouse reviewed class content in storyboard and online format for alignment with educational psychology and instructional design best practice (Mayer & Moreno, 2003) in an attempt to maximize the impact, the courses would have on virtual attendees' knowledge, skills, and motivation to sustain behavior change. The Clearinghouse has focused on improving and developing comprehensive evaluation tools for integration into a central data management hub (i.e., the EventPLUS event registration portal). These evaluation tools will be employed before, during, and after YRRP in-person and online events and for during deployment mailer recipients. Data integration will link disparate data collection efforts, which will allow more in-depth analyses into the data being collected. For instance, analysts will be able to compare topic interest collected prior to an event with attendee perceived class usefulness following class completion. This could help determine if a class is meeting the needs of the audience or if the class is a necessary component of the curriculum. This type of triangulation analysis is one of many across the multiple evaluation efforts that are planned (e.g., PES, pre/post live polling, breakout session SMS, Well-being Survey). Overall, the goal of this effort is to: (1) reduce the attendee burden as demographic information will only be collected during registration for all event evaluations: (2) provide the YRRP with the opportunity to assess program impact with enhanced rigor; and (3) allow for evidence-informed adjustments to the program as necessary. In addition to data integration, the Clearinghouse has begun to report on data from the AAR, which was developed in Fiscal Year (FY) 18 to collect feedback from multiple informants (i.e., Cadre, event planners, and observers) about class feedback and communication processes during the planning and execution of YRRP events. Results from the AAR survey are reported on a quarterly basis. The class-related feedback collected is used to assist in the curriculum review process. This feedback has helped update class materials, such as creating alternate scenarios and correcting outdated information. The Clearinghouse also collaborated with the United States Army Reserve (USAR) YRRP team and Susan Davis International to develop a During Deployment magazine and evaluation tool. Throughout the process, the Clearinghouse was responsible for ensuring the content of the magazine aligned with evidence and the YRRP curriculum. In addition, the Clearinghouse and USAR YRRP team developed an evaluation tool for the mailer.
Publications
- Type:
Journal Articles
Status:
Accepted
Year Published:
2019
Citation:
Richardson, C. B., Chesnut, R. P., Morgan, N. R., Bleser, J. A., Perkins, D. F., Vogt, D., &
Finley, E. (In Press). Examining the factor structure of the moral injury events scale in a veteran sample. Military Medicine.
- Type:
Other
Status:
Published
Year Published:
2018
Citation:
Balotti, B. Richardson, C. B., Irvin, J. B., & Perkins, D. F. (2018, October). Year-End Report on Post-Event Survey Data. University Park, PA: Clearinghouse for Military Family Readiness.
- Type:
Other
Status:
Published
Year Published:
2018
Citation:
Balotti, B. Richardson, C., Irvin, J., & Perkins, D. F. (2018, December). FY18 Q4 Yellow Ribbon Reintegration Program After Action Report. University Park, PA: Clearinghouse for Military Family Readiness.
- Type:
Other
Status:
Published
Year Published:
2019
Citation:
Balotti, B. Richardson, C. B., Irvin, J. B., & Perkins, D. F. (2019, January). FY19 Q1 Report on Post-Event Survey Data. University Park, PA: Clearinghouse for Military Family Readiness.
- Type:
Other
Status:
Published
Year Published:
2019
Citation:
Balotti, B. Richardson, C., Irvin, J., & Perkins, D. F. (2019, May). FY19 Q2 Yellow Ribbon Reintegration Program After Action Report. Pennsylvania State University. University Park, PA: Clearinghouse for Military Family Readiness
- Type:
Other
Status:
Published
Year Published:
2018
Citation:
Mathies, B. R., Balotti, B. A., Irvin, J. B., Richardson, C. B., & Perkins, D. F. (2018, August). Yellow Ribbon Reintegration Program Preliminary Data Report: United States Navy Reserve Individual Augmentee Longitudinal Study. University Park, PA: Clearinghouse for Military Family Readiness.
|
Progress 09/01/17 to 08/31/18
Outputs
Target Audience:The target audience is DoD and regional YRRP staff as well as Service-level staff. The ultimate audience is the National Guard and Reserve service members and their families who attend YRRP events. Changes/Problems:
Nothing Reported
What opportunities for training and professional development has the project provided?This project is focused on evaluation and curriculum refinement. The Clearinghouse develops these products but does not deliver the training. How have the results been disseminated to communities of interest?Yes, all Service-level evaluation reports were shared with all stakeholders. What do you plan to do during the next reporting period to accomplish the goals?This proposed continuation of the multi-faceted project addresses the Yellow Ribbon Reintegration Program's (YRRP) overarching goals of providing evidence-informed support to both program evaluation and improvement efforts. The work represents opportunities for continued refinement and expansion of the evaluation and curriculum development approach. Whether an existing effort or a new endeavor, the Clearinghouse has staff already placed in critical roles (e.g., research scientists, research assistants, and learning design specialists) and has existing lines of communication and workflows with YRRP personnel, Service-level personnel, and contract personnel. As such, the Clearinghouse is equipped to continue activities and will work diligently to be successful throughout the process. The work on data integration will continue, which is anticipated to reduce burden on attendees and increase efficiency of continued curriculum review, refinement, and development.
Impacts
What was accomplished under these goals?
Over the past year, the Clearinghouse has continued to provide support to the YRRP in both curriculum and program evaluation activities. One major curriculum initiative that occurred this past year was a multi-informant review of existing classes. Data from Cadre, attendees, and program managers, in addition to online class utilization statistics were collected and analyzed to guide future curriculum efforts. Another major undertaking involving curriculum was to address curriculum implemetation concerns. The Clearinghouse, in collaboration with OSD, presented on the curriculum development and review processes at this past year's Cadre training. The presentation also focused on defining boundaries for adaptations to curriculum as well as the importance of implementation fidelity more generally. In addition to this presentation, the Clearinghouse helped to redevelop the Cadre's self-reporting class feedback tool to reduce the respondent's burden and collect more actionable data. With OSD's approval and encouragement, the Clearinghouse and Cadre members have established an open communication process that increases the opportunities for collaboration and mutual understanding. Throughout the year, the Clearinghouse also completed an administrative review of all YRRP standardized classes to ensure all referenced links were active and formatting consistency. The Logic Model was revised to be more comprehensive and incorporate a recently developed Veterans well-being framework. Additional evaluation efforts continue, with the Post-Event Survey remaining the most consistently administered and analyzed survey. In addition, the Clearinghouse supported a special request by the Air National Guard to observe and evaluate a set of three YRRP events for the remotely piloted aircraft (RPA) community; and a request by the Army Reserve to assist in devleoping a During deployment mailer. The Navy Individual Augmentee Longitudinal Study completed recruitment and Time 1 data collection for all five cohorts and submitted a report detailing this baseline data. The Clearinghouse has also been involved in preliminary conversations regarding data integration within the EventPLUS portal. Another steady evaluation effort is the vetting of online resoures. This past year the finance domain has been revetted, with the TBI/PTSD domains scheduled to be revetted by quarter 4.
Publications
- Type:
Other
Status:
Published
Year Published:
2018
Citation:
Mathies, B. R., Balotti, B. A., Irvin, J. B., Richardson, C. B., & Perkins, D. F. (2018). Yellow Ribbon Reintegration Program preliminary data report: United States Navy Reserve individual augmentee longitudinal study. University Park, PA: Clearinghouse for Military Family Readiness.
- Type:
Other
Status:
Published
Year Published:
2018
Citation:
Richardson, C., Balotti, B., Irvin, J. Rodgers, J., & Perkins, D. F. (2018). YRRP event for Air National Guard remotely piloted aircraft operators: Report on data 5 June 2018 & 21 June 2018. University Park, PA: Clearinghouse for Military Family Readiness.
- Type:
Other
Status:
Accepted
Year Published:
2018
Citation:
Balotti, B., Richardson, C., Saroka, S., Irvin, J., & Perkins, D. F. (2018). Curriculum interest survey for stakeholders. University Park, PA: Clearinghouse for Military Family Readiness.
- Type:
Other
Status:
Published
Year Published:
2018
Citation:
Kaufman, M., Balotti, B., Richardson, C., Irvin, J., & Perkins, D. (2018). Yellow Ribbon Reintegration Program final data report: United States Army Reserve 451st Expeditionary Sustainment Command. University Park, PA: Clearinghouse for Military Family Readiness.
- Type:
Other
Status:
Published
Year Published:
2018
Citation:
Richardson, C., Balotti, B., Weldon, B., Rodgers, J., Saroka, S., Mathies, B., Irvin, J., & Perkins, D. F. (2018). Curriculum data triangulation report. University Park, PA: Clearinghouse for Military Family Readiness.
- Type:
Other
Status:
Published
Year Published:
2017
Citation:
Richardson, C., Schiavone, W., Perkins, K., Irvin, J., & Perkins, D. (2017). Employer Support of the Guard and Reserve volunteer questionnaire: Revised final report. University Park, PA: Clearinghouse for Military Family Readiness.
|
|