Progress 09/01/23 to 08/31/24
Outputs
Target Audience:The main audience for our project activities includes Army FAP personnel with direct responsibilities for implementing the IDC-CCSM model as well as those who supervise and train them. IDC-CCSM Quality Assurance Reviews were conducted for all 45 garrisons implementing the IDC-CCSM model. The process provided feedback and guidance to US Army FAP professionals responsible for incident assessments and CCSMs (FAP clinical supervisors, assessing social workers, FAP Managers) and to all IDC members including Garrison Commanders/Deputy Garrison Commanders who chair the IDC, Command Sergeant Majors, law enforcement representatives, JAGs, medical representatives, FAP clinical supervisors, and FAP Managers. Reports are also provided to FAP HQ-level leaders responsible for ensuring the quality and fidelity of the model. We facilitated six online New FAP Personnel Trainings (123 clinical supervisors, assessing social workers, and FAP Managers), two online New Site Coordinator Trainings, and seven in-person Getting It Right: Advanced IDC-CCSM Trainings (one in Europe, one in Hawaii for the Pacific region, and five CONUS, 108 clinical supervisors, assessing social workers, and FAP Managers). We created the FAP-M Trainer for IDC Chair slide deck, which FAP-Ms can use with chairs (i.e., Garrison Commander, Deputy Garrison Commander) as an annual refresher. Our dissemination efforts were directed towards family maltreatment researchers and practitions. These included one manuscript (Zaninovic et al., In Press) and three presentations at the Violence Prevention Research Conference. Changes/Problems:
Nothing Reported
What opportunities for training and professional development has the project provided?Wefacilitated six online New FAP Personnel Trainings (123 FAP-Ms, FAP clinical supervisors, and assessing social workers) and two online New Site Coordinator Trainings for Site Coordinators and their supervisors.We also led seven in-person Getting It Right: Advanced IDC-CCSM Trainings (108 FAP-Ms, FAP clinical supervisors, and assessing social workers). We also distributedthe FAP-M Trainer for IDC Chair slide deck, which FAP-Ms can use with chairs (i.e., Garrison Commander, Deputy Garrison Commander) as an annual refresher. How have the results been disseminated to communities of interest?We prepared a report for Army FAP summarizing FY23 quality assurance and training progress. The report was completed and distributed to Army FAP in September, 2023. We also brief Army FAP leadership during a biweekly meeting (30-60 minutes) and hold longer (3-6 hours)quarterly planning meetings throughout the year. FAP headquarters staff brief their chains of command. During year 2 of the project, we conducted analyses ofdata from our MOST trial testing the impact of three training approaches on the IDC-CCSM model fidelity andquality during the transition year (FY22). During year 3, our manuscript was submitted and accepted for publication. Zaninovic, V., Heyman, R. E., Drew, A. L., Slep, A. M. S., Lapshina, N., Neglio, B., Rhoades, K. A., & Daly, K. A. (In Press). Testing evidenced-based training methods in the worldwide dissemination of a reliable and valid family maltreatment determination model. Journal of Family Psychology. In addition, findings from this project were presented during a symposium ("On Three Decades of Research on the Validation, Dissemination, and Implementation of a Standardized Family Maltreatment Classification System") at the Violence Prevention Research Conference in July, 2024. We presented three papers:A Research Program Testing Field-Usable, Reliable, and Valid Criteria for Partner and Child Maltreatment, Adoption of a Maltreatment Classification System: Insight from Implementation Science, andChallenges Reaching Consensus: Case Qualities that Sway Maltreatment Substantiations. What do you plan to do during the next reporting period to accomplish the goals?We will continue to implement the IDC-CCSM QAR process. This will include continuing QAR for garrisons that have not yetpassed fidelity for FY24and beginning FY25annual QAR for all garrisons JAN-AUG 2025. We will continue to facilitate the in-personGetting It Right:Advanced IDC-CCSM Trainings(2x/year CONUS)and will offer new, in-personFAP-M and FAP-C IDC-CCSM workshops (2x/year/workshop CONUS). OCONUS in-person training needs will be determined during FY25 with consultation from FAP leadership.We will continue to offeronline New FAP Personnel Trainings (at least once per quarter); and 2 online Site Coordinator Trainings. We will complete and disseminate the IAT, Reasonable Suspicion, and Credibility training tools that were drafted in year 3 of the project.Additional training priorities for the next reporting period will be determined with Army FAP during our SEPT 2024 quarterly planningmeeting. We are preparing a report for Army FAP summarizing our activities during FY24including: (a) QA progress; (b)implementation quality and fidelity at each garrison during the QA observations (including comparisons between FY23 and FY24); and (c) FY24trainingefforts. This report will be completed in NOV 2024. We will prepare a manuscript for submission addressing the role of QA in improving the fidelity and quality of the IDC-CCSM model.
Impacts
What was accomplished under these goals?
The transition to the IDC-CCSM model was completed in AUG 2022 during the first year of this project. To ensure the fidelity of the model, in the second and third years, we have conducted annual quality assurance (QA) reviews of each garrison implementing the model and supported training with the goals of (a) training new personnel to execute their IDC-CCSM responsibilities and (b) increasing FAP-Ms' and FAP-Cs' competence and confidence in serving as a subject matter expert regarding relevant IDC-CCSM policies and procedures and the DTA. Quality Assurance QA reviews follow the same procedures determined during year 2 in consultation with Army Family Advocacy Program leadership. Each garrison implementing the IDC-CCSM model is required to complete annual quality assurance reviews (QA) of their IDC, CCSM, and IAR to ensure fidelity. NYU Subject Matter Experts ("QA assessors") are conducting all QA reviews. For IDCs and CCSMs, a QA assessor observed the meeting telephonically or via MS Teams video conferencing. During each IDC or CCSM, the QA assessor completed a fidelity checklist. In addition, the QA assessor recorded the IDC's determination for each incident. The QA assessor observed a minimum of two IDCs for a maximum of 2 hours per meeting. Across IDCs, the QA assessor observed decisions on a minimum of 10 incidents, including at least 5 partner and 5 child incidents. If the garrison regularly has multiple FAP presenters, at least 3 incidents were observed for each presenter. The QA assessor observed a minimum of one CCSM for a maximum of 2 hours per meeting. The QA assessor observed discussions of at least 10 families (one family could have multiple IDC incidents), at least 5 of which had partner maltreatment incidents and 5 of which had child maltreatment incidents. The QA assessor also observed a mix of new and review cases. If a garrison regularly has multiple CCSM supervisors, presentations of at least 3 families were observed for each supervisor. The QA assessor reviewed the final IATs that were presented by FAP during the observed IDCs as a proxy for IAR observation. The QA observer produced a report for each meeting summarizing its strengths, identified areas of improvement, a recommended improvement plan, and a timeline for reassessment. Any garrison that did not meet fidelity for a given meeting on their initial QA assessment was provided an improvement plan tailored to meet their identified needs. The improvement plan can include steps such as meeting with the QA assessor to review challenging incidents, reviewing policy materials, just-in-time feedback during their next meeting, or attending in-person training. Upon completion of the improvement plan, QA is reassessed. If the garrison did not meet fidelity at the reassessment, a corrective action plan was put in place, after which the garrison was reassessed again. The garrison continued to be assessed and provided intervention until fidelity was achieved or a new annual QA cycle began. Training Each year Army FAP leadership -- in consultation with QA assessors -- identifies training priorities to help garrisons/clinics progress toward QA fidelity. Training products include in-person and online group trainings and the creation of self-guided training tools. FY24 training included continued implementation of FY23 online and in-person trainings and developing several new training tools as described below. The NYU team, with logistical support from IMCOM FAP HQ, facilitated six online New FAP Personnel Trainings (123 participants) and two online New Site Coordinator Trainings. Trainings were scheduled with times to accommodate Army's worldwide reach, including one New FAP Personnel Training specifically targeting personnel in Europe. Both trainings used materials created in year 2 of this project. We also led seven in-person Getting It Right: Advanced IDC-CCSM Trainings (one in Europe, one in Hawaii for the Pacific region, and five CONUS, 108 participants).The overall agenda was similar to year 2 (previously titled Advanced IDC-CCSM Boot Camp). The FAP-M-specific practice materials for the Getting It Right training were substantially updated this year to use deliberate practice training techniques (e.g., Boswell & Constantino, 2021). In addition, due to the relatively high fidelity of CCSMs during FY23, CCSM content moved from a focus on how to achieve CCSM fidelity to how to enhance treatment planning and address challenges with engagement. These trainings were offered to FAP-Ms and FAP-Cs who had not attended in FY23, and assessing social workers (clinicians were not eligible to attend in FY23 due to space constraints). We created the FAP-M Trainer for IDC Chair slide deck, which FAP-Ms can use with chairs (i.e., Garrison Commander, Deputy Garrison Commander) as an annual refresher. The FAP-M Trainer for IDC Chair was disseminated by IMCOM HQ FAP to all FAP-Ms via Microsoft Teams. In addition, the NYU team has been developing three training tools aimed at improving IAT quality and IAR fidelity. Drafts of all three tools were completed in year 3. Revisions with feedback from MEDCOM FAP leaders will be completed in during the next year prior to dissemination to the field. The first tool is an interactive, online IAT Trainer, which can be used by new assessing clinicians to improve their skills in writing and self-assessing the quality of their IATs. FAP-Cs can also assign the IAT Trainer to assessing clinicians needing improvement, and it can be recommended as part of the QA improvement/ corrective action plan. The IAT Trainer includes instructive guidance, an example IAT with explanations of needed areas of improvement, and ten practice IATs with accompanying questions that are used to assess IAT quality and make needed improvements. The second tool is a Reasonable Suspicion Trainer, which reviews each component of reasonable suspicion as defined by the Army, provides multiple examples demonstrating how each component is applied including an explanation of whether there is/is not reasonable suspicion or if further assessment is needed prior to making a determination, and an in-cludes skills assessment. The Reasonable Suspicion Trainer will be used to train FAP-Cs and assessing social workers on more accurately determining reasonable suspicion and can be assigned to FAP-Cs as part of a QA improvement/corrective action plan. It can also be used as a reference guide after initial training. The third tool is a Credibility Trainer, which includes slides and accompanying voiceover explanations. The Credibility Trainer reviews state-of-the-science research on credibility and memory, presents a framework and rubric for assessing credibility, and reviews examples of credibility statements while using the rubric. The Credibility Trainer can be used by new assessing clinicians to improve their skills preparing credibility statements. FAP-Cs can also assign the Credibility Trainer to assessing clinicians needing improvement, and it can be recommended as part of the QA improvement/corrective action plan.
Publications
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2024
Citation:
Daly, K. A., Zaninovic, V., Heyman, R. E., & Slep, A. M. S. (2024, July). Adoption of a Maltreatment Classification System: Insight from Implementation Science. Paper presented at Violence Prevention Research Conference, Portsmouth, NH.
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2024
Citation:
Zaninovic, V., Heyman, R. E., & Slep, A. M. S. (2024, July). Challenges Reaching Consensus: Case Qualities that Sway Maltreatment Substantiations. Paper presented at Violence Prevention Research Conference, Portsmouth, NH.
- Type:
Other
Status:
Published
Year Published:
2023
Citation:
Drew, A. L., Slep, A. M. S., Heyman, R. E., Neglio, B., & Zaninovic, V. (2023). Army Family Advocacy Program Incident Determination Committee (IDC)Clinical Case Staffing Meeting (CCSM) implementation: 2022-2023 quality assurance and training report.
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2024
Citation:
Heyman, R. E., & Slep, A. M. S. (2024, July). A Research Program Testing Field-Usable, Reliable, and Valid Criteria for Partner and Child Maltreatment. Paper presented at Violence Prevention Research Conference, Portsmouth, NH.
|
Progress 09/01/22 to 08/31/23
Outputs
Target Audience:IDC-CCSM Quality Assurance Reviews were conducted for each garrison implementing theIDC-CCSM model. The processprovided feedback and guidance to US Army FAP professionals responsible forincident assessments and CCSMs(FAP clinical supervisors, assessing social workers,FAP Managers) and to all IDC members including Garrison Commanders/Deputy Garrison Commanders who chair the IDC, Command Sargeant Majors, law enforcement representatives, JAGs, medical representatives, FAP clinical supervisors, and FAP Managers. We developed a 3-day, in-personAdvanced IDC-CCSM Boot Camp (to be referred to as "Getting It Right: Advanced IDC-CCSM Training" in FY24). A total of 7 Advanced Boot Camps were conducted during the study period (5 CONUS, 2 OCONUS) for approximately 160 FAP clinical supervisors, assessing social workers, and FAP Managers. We developed aNew FAP Personnel Training that was delivered to newFAP clinical supervisors, assessing social workers, and FAP Managers prior to the Europe Advanced IDC-CCSM boot camp. The content was adapted for synchronous, online training in FY24. We developed and conducted two FAP Site Coordinator trainings. These one-hour, online trainings provided information for new site coordinators and their supervisors (FAP Managers) on the IDC-CCSM process. In addition, we developed and distributed training materials to support all garrisons implementing the IDC-CCSM process including: an Advanced IDC Process Trainer and Guide for FAP Managers and FAP clinical supervisors to use to train IDC members and new FAP staff; an online, annual refresher training required for all IDC members with supplemental content for the chair and FAP Manager; practice IDC incidents that can be used by IDCs for training purposes, in particular smaller sites that have few incidents annually; and a cheat sheet to help Unit Commanders prepare for and participate in the IDC. Changes/Problems:
Nothing Reported
What opportunities for training and professional development has the project provided?We conductedsynchronous (i.e., live) trainings and developed training tools aimed to increase FAP personnel and IDC committee members' implementation of the IDC-CCSM model across the US Army.We developed a 3-day, in-personAdvanced IDC-CCSM Boot Camp (to be referred to as "Getting It Right: Advanced IDC-CCSM Training" in FY24). A total of 7 Advanced Boot Camps were conducted during the study period (5 CONUS, 2 OCONUS) for approximately 160 FAP clinical supervisors, assessing social workers, and FAP Managers. We developed aNew FAP Personnel Training that was delivered to newFAP clinical supervisors, assessing social workers, and FAP Managers prior to the Europe Advanced IDC-CCSM boot camp. The content was adapted for synchronous, online training in FY24. We developed and conducted two FAP Site Coordinator trainings. These one-hour, online trainings provided information for new site coordinators and their supervisors (FAP Managers) on the IDC-CCSM process. In addition, we developed and distributed training materials to support all garrisons implementing the IDC-CCSM process including: an Advanced IDC Process Trainer and Guide for FAP Managers and FAP clinical supervisors to use to train IDC members and new FAP staff; an online, annual refresher training required for all IDC members with supplemental content for the chair and FAP Manager; practice IDC incidents that can be used by IDCs for training purposes, in particular smaller sites that have few incidents annually; and a cheat sheet to help Unit Commanders prepare for and participate in the IDC. How have the results been disseminated to communities of interest?Weprepareda report for Army FAP summarizing the Phase 1 IDC-CCSM transition including: (a) implementation quality and fidelity during each garrison's first three IDC meetings and their accompanying CCSMs and IARs (launch phase); (b) identifying potential challenges garrisons may face after MTT support ended (post-launch phase); (c) the results of the MOST trial. The report was completed in Sept 2022. Results werediscussed at an in-person meeting in Washington, D.C. and used to determine priorities for the sustainment phase. We brief Army FAP headquarters biweekly and hold a quarterly planning meeting. FAP headquarters staff brief their chains of command. What do you plan to do during the next reporting period to accomplish the goals?We will continue to implement the IDC-CCSM QAR process. This will include continuing QAR for garrisons that have not yet passed fidelity for FY23 and beginning FY24 annual QAR for all garrisons JAN-AUG 2024. Training priorities for the next reporting period were determined with Army FAP during our AUG 2023 quarterly planning meeting. Live trainings will include: 8additionalAdvanced IDC-CCSM Boot Camps (to be referred to as "Getting It Right: Advanced IDC-CCSM Training" in FY24),6CONUS and 2 OCONUS; at least 6online New FAP Personnel Trainings; and at least 2online Site Coordinator Trainings. In addition, we will create training materials that can be used by FAP staff including: an interactive IAT trainer; a deskside brief for FAP Managers to conduct annually with IDC chairs; and trainings on reasonable suspicion and credibility. We are preparing a report for Army FAP summarizing our activities during FY23 including: (a) QA progress; (b) implementation quality and fidelity at each garrison during the QA observations; (c) the development of FY23 training materials. This report will be completed in SEPT 2023. We are preparingour first manuscript for peer review as part of this study, which will focus on the dissemination of the IDC-CCSM model in the Army context.
Impacts
What was accomplished under these goals?
The transition to the IDC-CCSM model was completed in AUG 2022. After the conclusion of the transition, we analyzed data for ourMOST trialtesting the impact of three training approaches on the IDC-CCSM model implementation fidelity and quality. Each garrison was randomly assigned to each condition, and IDC members were blind to the experimental conditions. The first condition was related to how the MTT gave feedback about voting and fidelity during the IDC. IDCs either received all feedback at the end of the meeting or received feedback after each case (just-in-time [JIT] feedback condition). We hypothesized that the IDC feedback condition would be associated with higher (a) IDC and IAR fidelity and quality and (b) IAT quality. The second condition was related to how IATs were reviewed. The assessing social worker either received feedback on the IAT from their supervisor at the IAR (i.e., as usual) or, in the enhanced condition, the social worker completed a self-assessment of the IAT before the IAR. We hypothesized that the self-assessment condition would be associated with higher (a) IDC and IAR fidelity and quality and (b) IAT quality. The third condition addressed how training briefs were presented to the incoming chairs, Garrison Commanders. The standard-practice briefs contained informational slides only. The motivation-enhanced briefs included the same informational slides with added discussion prompts FAPMs could use to engage the GCs in motivating topics (e.g., having them articulate reasons to change to the IDCCCSM model, reasons why the GC is the right role to chair the meeting, how this will help Soldiers). We hypothesized that the briefs condition would be associated with higher IDC fidelity and quality. None of the hypotheses were supported. To ensure the fidelity of the model, FY23 began the sustainment phase, which includes quality assurance and ongoing training. Each garrison implementing the IDC-CCSM model is required to complete annual quality assurance reviews (QA) of their IDC, CCSM, and IAR to ensure fidelity.NYU Subject Matter Experts ("QA assessors") are conducting all QA reviews. For IDCs and CCSMs, an NYU QA assessor observed the meeting telephonically or via MS Teams video conferencing. During each IDC or CCSM, the QA assessor completed a fidelity checklist. In addition, the QA assessor recorded the IDC's determination for each incident. The QA assessor observed a minimum of two IDCs for a maximum of 2 hours per meeting. Across IDCs, QA assessor observed decisions on a minimum of 10 incidents, including at least 5 partner and 5 child incidents. If the garrison regularly has multiple FAP presenters, at least 3 incidents were observed for each presenter. The QA assessor observed a minimum of one CCSM for a maximum of 2 hours per meeting. The QA assessor observed discussions of at least 10 families (one family could have multiple IDC incidents), at least 5 of which had partner maltreatment incidents and 5 of which had child maltreatment incidents. The QA assessor also observed a mix of new and review cases. If a garrison regularly has multiple CCSM supervisors, presentations of at least 3 families were observed for each supervisor.The QA assessor reviewed the final IATs that were presented by FAP during the observed IDCs as a proxy for IAR observation. The QA observer produced a report for each meeting summarizing its strengths, identified areas of improvement, a recommended improvement plan, and a timeline for reassessment. Any garrison that does not meet fidelity for a given meeting on their initial QA assessment is provided an improvement plan tailored to meet their identified needs. The improvement plan can include steps such as meeting with the QA assessor to review challenging incidents, reviewing policy materials, just-in-time feedback during their next meeting, or in-person training at an Advanced IDC-CCSM Boot Camp. Upon completion of the improvement plan, QA is reassessed. If the garrison does not meet fidelity at the reassessment, a corrective action plan is put in place, after which the garrison is reassessed a second time. The garrison will continue to be assessed and provided intervention until fidelity is achieved. In addition, at the end of the IDC-CCSM transition in FY22, we collaborated with Army FAP to identifya number of training priorities needed to support the sustainment phase. During the study period, NYU created materials for and conducted live trainings, including 7in-person IDC-CCSM Advanced Boot Camps, 2 New FAP Personnel Training, and 2 online Site Coordinator Trainings. In addition, we developed and distributed training materials to support all garrisons implementing the IDC-CCSM process including: an Advanced IDC Process Trainer and Guide for FAP Managers and FAP clinical supervisors to use to train IDC members and new FAP staff; an online, annual refresher training required for all IDC members with supplemental content for the chair and FAP Manager; 31 practice IDC incidents that can be used by IDCs for training purposes, in particular smaller sites that have few incidents annually; and a cheat sheet to help Unit Commanders prepare for and participate in the IDC.
Publications
- Type:
Other
Status:
Published
Year Published:
2022
Citation:
Drew, A. L., Slep, A. M. S., Heyman, R. E., Lapshina, N., Neglio, B., Rhoades, K. A., & Zaninovic, V. (2022). Army Family Advocacy Program Incident Determination Committee (IDC) Clinical Case Staffing Meeting (CCSM) implementation: 2021-2022 Dissemination Findings. Prepared for the Department of Army; National Institute of Food and Agriculture/USDA.
|