Progress 09/01/16 to 08/31/18
Outputs
Target Audience:Broadly, the target audience of this project is the Office of the Assistant Chief of Staff for Installation Management (OACSIM) in conjunction with the larger Office of the Secretary of Defense and, ultimately, the Department of Defense (DoD). This target audience includes individuals who oversee the Army's Mobilization, Deployment, & Stability Support (MD&SSO) program and individuals whoare affiliated with the MD&SSO program (including program staff) aswell as the Soldiers and Families served by the MD&SSO program. Specific efforts used to reach the target audience include disseminating electronic and hard copies of the evaluation plan and communicating with various audiences to share findings from our work and discuss specific evaluation efforts that could be implemented in the future, including discussions of cost estimates and the strengths and weaknesses of various approaches. These communications included (1) briefing senior Army leaders and (2) briefing program managers across services to ensure that similar programs within other service branches are aware and up-to-date regarding best practices for evaluation efforts.? Changes/Problems:No changes occurred since the last project report. As reported in last year's report, the timeline of the project was extended from 12 months to 24 months. This was necessary for several reasons. First, although the project officially began September 1, 2016, the funding was not available to us through the university until December. This three-month delay limited our ability to begin work on the project. Second, it took additional time to schedule and carry out the installation visits due to the number of people involved, the number of approvals required for our visits, and the need to schedule visits around MD&SSO activities we needed to observe. Consequently, we did not finish the required travel until May 2017. The extension of the project period did not create any problems. We ultimately fulfilled the project deliverables to the satisfaction of stakeholders at OACSIM and on a timeline that met their needs. What opportunities for training and professional development has the project provided?
Nothing Reported
How have the results been disseminated to communities of interest? We shared information we learned from our observations and interviews with interested individuals at IMCOM and OACSIM. We briefed senior Army leaders on the final evaluation plan submitted. Program managers across services were briefed on the evaluation planning efforts to ensure that similar programs within other service branches are knowledgeable on evaluation efforts and best practices for program evaluation. The final evaluation plan (containing deliverables 1-3, including an asset map, logic model, evaluability assessment, and three evaluation options) was submitted electronically and via hard-bound copies to stakeholders at OACSIM and IMCOM. The final evaluation plan (containing deliverables 1-3, including an asset map, logic model, evaluability assessment, and three evaluation options) was shared electronically with program managers who expressed a desire to see the final product. What do you plan to do during the next reporting period to accomplish the goals?
Nothing Reported
Impacts
What was accomplished under these goals?
The Army's Mobilization, Deployment and Stability Support Operations (MD&SSO) program assists leaders, Soldiers, and Families in successfully navigating stressors at each stage of the mobilization and deployment cycle. However, the program has not been formally evaluated to determine if it meets the needsof Service members and their Families. Having data that speak to the merits and demerits of programs designed to enhance family life and family capabilities during stressful times is pivotal for supporting families and achieving military mission readiness. Before implementing an evaluation of the program Army-wide, it is imperative to develop a comprehensive, scientific, and rigorous evaluation plan that is also practical and ultimately evaluates the program in a manner that provides relevant information for the stakeholders. We addressed the need for a formal evaluation plan of the MD&SSO program and equipped the Army with a comprehensive evaluation plan that, when enacted, will enable them to (1) monitor how the program is implemented and (2) establish program merit. To reach this end goal, we gained a thorough knowledge of the program from various perspectives (stakeholders, policymakers, program managers, program participants). We developed an asset map and revised logic model. We completed an evaluability assessment to determine which program components are ready for evaluation, and used this information to develop three distinct options for program evaluation, providing explicit next steps for enacting each plan. We compiled these products into a single evaluation plan, which together provide the Army with the content needed to conduct a feasible and realistic evaluation of the MD&SSO program marrying a nuanced understanding of the program with the need for an accurate and research-based assessment of the program. This plan provides enhanced knowledge of (1) the MD&SSO program, (2) how distinct program components can be evaluated, and (3) the types of information that can come from such an evaluation. Enacting this plan will enable Army leaders and helping professionals to remain at the forefront of providing high-quality family programming. Specific accomplishments for each deliverable are discussed below. To complete the objectives, it was necessary to gain a macro- and micro-level understanding of the Army's Mobilization, Deployment, and Stability Support Operations (MD&SSO) Program, data were collected via document review, observations during site visits, interviews with program staff and stakeholders, and quantitative data abstraction (Losby et al., 2015). Armed with this information, we delivered a final document in May 2018 to our contacts at theOffice of the Assistant Chief of Staff for Installation Management (OACSIM). The final document satisfied each of the projectobjectives. The document contained: an asset map and updated logic model for the program linking activities and outputs of the MD&SSO program to intended program outcomes (Objective 1), an evaluability assessment detailing the individual components of the program, identifying the proposed level of evaluation for each program component, and providing a rationale for the proposed level of evaluation. That is, we identified which program componentslendthemselves to an outcome-oriented evaluationassessing short-, mid-, and/or long-term outcomes, and which activities, instead, are best-suited for a monitoring approach to evaluation counting outputs and other indicators of program component usage (Objective 2), and an evaluation plan detailing three possible evaluation options varying in intensity, length of time/effort required, and estimated cost. For each option, we also identified strengths, weaknesses, next steps, and the knowledge to be gained by enacting the evaluation. We also proposed specific evaluation instruments and methods of data collection and analysis (Objective 3).
Publications
|
Progress 09/01/16 to 08/31/17
Outputs
Target Audience:Broadly, the target audience of this project is the Office of the Assistant Chief of Staff for Installation Management (OACSIM) in conjunction with the larger Office of the Secretary of Defense and, ultimately, the Department of Defense (DoD). More specifically, the target audience includes individuals who oversee or are affiliated with the Mobilization, Deployment, & Stability Support (MD&SSO) program as well as the Soldiers and Families served by the MD&SSO program. Changes/Problems:The timeline of the project was extended from 12 months to 24 months (although we anticipate final products will be delivered in advance of this date). This was necessary for several reasons. First, although the project officially began September 1, 2016, the funding was not available to us through the university until December. This three-month delay limited our ability to begin work on the project immediately. Second, it took additional time to schedule and carry out the installation visits due to the number of people involved, the number of approvals required for our visits, and the need to schedule visits around MD&SSO activities we needed to observe. Consequently, we did not finish the required travel until May 2017. This change has not created any problems, and we are confident in our ability to complete the remaining work in the next reporting period. What opportunities for training and professional development has the project provided?
Nothing Reported
How have the results been disseminated to communities of interest?Although dissemination was not the primary purpose of Year 1 activities, we have shared information we learned from our observations and interviews with interested individuals at IMCOM and OACSIM. What do you plan to do during the next reporting period to accomplish the goals?We are in the process of drafting written products as deliverables for each of the three objectives. In the next reporting period, we will meet with invested stakeholders (including individuals at IMCOM and OACSIM) to receive their feedback on these drafts. Working in consult with these individuals, we will conduct revisions and ultimately submit final products for each of the three objectives/deliverables. The future implementation of this evaluation will strengthen military families by identifying effective components of the MD&SSO program that support Soldier and Family needs and amplify their inherent abilities, therefore promoting resilience and healthy outcomes. Having data that speak to the merits and demerits of programs designed to enhance family life and family capabilities during stressful times is pivotal for supporting families and achieving military mission goals.
Impacts
What was accomplished under these goals?
To gain the necessary macro- and micro-level perspective of program functioning necessary to accomplish the three objectives, data were collected via document review, observations during site visits, interviews with program staff and stakeholders, and quantitative data abstraction (Losby et al., 2015). Reviewed documents include ACS program documents and staff guides, such as accreditation standards, program manager handbook, the MD&SSO strategic plan, curricula training materials, process maps, logic models, and the existing evaluation plan, as well as relevant army regulations and evaluation materials from other pertinent military programs. Installation visits occurred in the Spring of 2017 at: • Fort Hood, Texas • USAG Bavaria, • Fort Stewart, Georgia, and • Fort Benning, Georgia. Visits included interviews with various stakeholders, including but not limited to: • MD&SSO project manager • MD&SSO project specialists • ACS Director, • Company Commanders • Garrison Commander • Installation Emergency Manager • Soldiers, and • Family members with varying levels of ACS/MD&SSO involvement. Visits also included observations of MD&SSO briefings and trainings including but not limited to: • Care Team training • FRG training (utilizing the REAL piloted training) • RD commander training • Newcomer briefings, and • pre-deployment briefings. The Troop and Family Readiness Training was an event attended by MD&SSO program managers and/or MD&SSO program specialists from across the world. During this training, we interacted with MD&SSO personnel from a wide variety of installations to learn more about the specific program activities and responsibilities at each installation. Grant activities also included teleconferences as needed (generally, monthly) with various stakeholders. These phone calls were an opportunity for us to ask questions and receive feedback on our understanding of the MD&SSO program. These meetings were also a time to learn about additional materials to include in our data review hear insights on MD&SSO program from various perspectives. In addition to data collection focused on the MD&SSO program itself. We also scoured existing scholarly research to enhance or understanding on program evaluation. We have amassed a large collection of existing research, sample evaluation plans, and theoretical writings on program evaluation to inform our efforts. Based on these activities we are confident that we are well-positioned to incorporate cutting edge, best practices in the final products of this project. In Year 1, these activities have resulted in our team establishing a comprehensive understanding of the MD&SSO program. We are in the process of drafting written products as deliverables for each of the three objectives.
Publications
|